"It doesn't go down with the public. He's tarnishing his image by saying he's not going to sign and 'I'm not sure what I'm going to do.'
I think he's a hell of a nice guy ... but now the general public is switching off him."
Sir Colin also believed the fewer cup games Williams got, the more likely he was to leave.
The Sydney Morning Herald has reported Williams is considering a return to the NRL and that speculation has linked him to the Sydney Roosters.
Many fans hit out against Williams on the Herald's rugby Facebook page yesterday, and in a nzherald.co.nz poll answered by more than 1400 readers, 65 per cent did not want Williams to re-sign with the NZRU.
Julie Beazer, among those who commented, believed Williams' high-profile manager Khoder Nasser and fellow Nasser client Anthony Mundine were damaging his career.
"They are ruining (if it isn't already ruined) his career. Ego, sponsorships, boxing ... just play the bloody game," she wrote.
Rugby commentator Phil Gifford told Newstalk ZB yesterday the relationship between Williams and Nasser was similar to that between Elvis Presley and Colonel Tom Parker: "It seems the management has called all the shots," he said.
All Black legend Andy Haden was reluctant to comment on the issue, but said: "I hope he makes some decisions that are good for rugby and then after that, perhaps good for Sonny Bill."
All Blacks Supporters Club managing director Kevin Dittmer backed Williams, saying "at the end of the day, it's Sonny Bill Williams' choice.
"I don't see how the New Zealand public can be commenting on Sonny Bill Williams' situation when they don't know everything that's going down."
Meanwhile, the coaches of the Blues and Chiefs still hope they will secure Williams for next year.
Chiefs coach Dave Rennie told the Herald: "Nothing's changed from our point of view - we've put our best foot forward in trying to encourage him to come to us but the key thing for me is that he's got to want to come to the Chiefs."
Blues coach Pat Lam could not be reached for comment yesterday, but told a news website that signing Williams would be a great bonus.