"We suspect there was something underhand about him," the institute's chairman, Johan Blaauw, told the domestic SAPA news agency. "How else was he working on such an important occasion when he did not seem to know what he was doing?
"I'm afraid this thing has left the whole of South Africa with egg on its face."
He said there had been complaints over his signing at past ANC events, yet the ruling party took no action.
Delphin Hlungwane, a spokeswoman for the Deaf Federation of South Africa, said the man picked for the job was "just gesturing in the air".
"He didn't interpret at all, he had zero per cent accuracy," she said.
"International people were watching it as well and they all said he was not using any kind of sign language. They all said they don't understand him," said Hlungwane.
"He's a complete fraud," Cara Loening, director of Sign Language Education and Development in Cape Town told AFP on Wednesday.
"He wasn't even doing anything, There was not one sign there. Nothing. He was literally flapping his arms around."
"The deaf community in South Africa are completely outraged and nobody knows who he is," said Loening, who added he looked as if he was "trying to swat a few flies away from his face and his head".
"We can't find a name or anything. The organisations who have accredited interpreters do not know him at all."
The government said it had launched an investigation following the allegations, the results of which would be made public.
Minister in the Presidency Collins Chabane said at a news conference the government was "looking into this matter", but would be unable to conclude its investigations on Wednesday because of other pressing demands ahead of Mandela's funeral on Sunday.
The White House meanwhile referred all questions on the matter to the South African government, but said it would be regrettable if the incident overshadowed Obama's "very powerful remarks" at the memorial service.
"It would be a shame if a distraction about an individual who was on stage in any way detracted from the importance of that event and the importance of president Mandela's legacy," deputy spokesman Josh Earnest told reporters in Washington.
Loening said her organisation was getting emails from around the world "wondering what on earth this man was doing there".
"It's a real embarrassment. It's complete disrespect for the deaf community and for what Nelson Mandela stood for and the support which he gave toward the deaf community."
While sign language semantics may differ from country to country, the basic grammar remained similar, said Blaauw.
"The common grammar can be recognised across the various sign languages," said Blaauw. "It's been the response across the world that he wasn't using the language but just making funny gestures."
-AFP and AAP
