Learning to serve the white man while being "invisible in the room", Gaines soon becomes very good at what he does.
Some years later, now married with two sons, Gaines is spotted serving in an upmarket Washington D.C. hotel by a senior White House official.
And so he goes on to serve and listen to events as they unfold in the White House while attending to the latest president.
But tearing Gaines in two is his eldest son Louis (David Oyelowo) - a strong, intelligent young man with a thirst for equality and justice.
As a university student he becomes involved in the fight for Civil Rights.
His peaceful protesting leads him to numerous stints in prison.
Gaines can't understand why his son won't accept the good life he has brought them and why he must put his own life in danger in a futile attempt to change the way things are.
Gaines also sees things from the other side as he watches the various presidents from Eisenhower onwards ignore or react to the Civil Rights Movement.
The Butler hardly moves along at a rapid pace but it is fascinating story about the slow emergence of black equality and basic human rights for all.
The Butler
(M) 132 minutes
Rating: 3.5 / 5