He referred to the Taleban's destruction of the giant Bamiyan Buddhas in Afghanistan, and said New Zealanders should learn from what happened at that sacred site.
He said the colonising governments of New Zealand, "egged on by land-hungry settlers", vandalised Maori culture and the environment.
Speaking on behalf of the Prime Minister, who was absent from Parliament, Dr Cullen said Mr de Bres would not be summoned to a "please explain" meeting.
"I'm sure it is not going to be necessary for me to talk to Mr de Bres for him to be aware his comments have caused some degree of concern in certain quarters.
"I also have no doubt he will continue to advocate issues which are consistent with his role.
"I'm sure he will have noted the reaction to his comments, and will consider whether provoking such a reaction is helpful to the performing of his tasks."
Last night, Mr de Bres said his comments had not amounted to a direct comparison and were instead a "starting point internationally" for thinking about cultural heritage.
"I've had positive comments and negative comments. I'll chew it all over." He was not asking people to "wring their hands" and feel guilty about the past.
"It wasn't meant to be a metaphor for race relations in New Zealand today. But maybe the metaphor was too strong for some."
National leader Bill English said New Zealanders were tired of "the cringing guilt of self-appointed consciences of the nation telling us what to think".
NZ First leader Winston Peters described Mr de Bres as a "loony tunes socialist activist" and a "self-appointed zealot".
He said Mr de Bres, who has Dutch heritage, was educated at a communist university in East Germany and came to New Zealand from a country with an appalling record of colonisation in Africa and Asia.
"He has the breathtaking audacity to describe part of my heritage as akin to the actions of the iconoclastic thugs who staged public executions, mutilations and harboured terrorists," said Mr Peters.
"We are not going to have the new version of the white man's burden foisted on the Maori people of this country."
Act leader Richard Prebble said it was ridiculous to have a Race Relations Commissioner, but if the country was going to have one, it might as well be someone who could keep everyone amused.
Full text of the Joris de Bres speech