A source told the Mirror it was a "crass comment that went down like a lead balloon", branding Boycott a "dinosaur".
The source said: "Is Boycott suggesting these great West Indians got knighthoods merely because of the colour of their skin? Birmingham is celebrated for its ethnic diversity and racial tolerance.
"We can do without dinosaurs like him spouting rubbish."
Responding on Twitter today, Boycott accepted his comments were 'unacceptable' and "clearly wrong".
Boycott wrote: "Speaking at an informal gathering I was asked a question and I realise my answer was unacceptable.
"I meant no offence but what I said was clearly wrong and I apologise unreservedly.
"I have loved West Indian cricket my whole life and have the utmost respect for its players."