"As many locals already know, Kohuora is also the name of a volcanic crater close to the prison."
The Sensible Sentencing Trust dubbed the move "political correctness gone mad".
"We would like inmates to be reminded they are serving their sentences in a prison, not some kind of spiritual retreat," spokeswoman Ruth Money said.
Law and order hardliner Kenneth Wang, deputy leader of the Act Party, said the idea was "bizarre".
"The authorities have already tried to reinvent prisons as 'correctional facilities' and now they are trying to give them another makeover. It is senseless."
Brian Brown, whose 24-year-old daughter Natasha Hayden was strangled by double killer Michael Curran in 2005, was appalled to hear of the new names for the prison.
"So, when someone asks a criminal 'where have you been bro?' they can say "I have been spending a few months at Kohuora, bro," which sounds a lot better than 'I have been in prison the last few months, bro'. It is disgusting."
Tolley insisted the name was not important. "Regardless of what it is called, I expect the new prison to be high-performing, secure and safe, and to provide excellent rehabilitation opportunities, so we can have fewer victims of crime."
Meanwhile, a report into how to improve New Zealand's prison system will be unveiled on June 24 by criminal justice organisation JustSpeak.