The comments prompted a complaint from one survivor, who wrote to both men threatening further action if they did not apologise.
Murray Heasley, a member of a support network for survivors of faith-based abuse, also objected, saying the comments had caused "shock, astonishment and deep dismay".
Yesterday, Hamilton Bishop Steve Lowe, in an email on behalf of the New Zealand Bishops, apologised "unreservedly" for the comments.
"We sincerely regret the hurt these comments have caused to victims, survivors and their families."
Lowe did not elaborate, and nor did he, Dunn or Harrison respond to requests for comment yesterday.
Instead, in the email, Lowe offered to meet Heasley "to hear anything you wish to say to me about the matters raised in the ODT, and also to share with you my response and course of action".
Heasley said the comments had prompted "understandable outrage and disbelief", offending victims, survivors and their families, as both Harrison and Dunn carried "considerable clout".
He was pleased the rest of New Zealand's bishops appeared "equally uncomfortable" with the pair's comments, and would accept Dunn's offer to meet and discuss his group's concerns.