The country's top cop has conceded police could have gone public about the Roast Busters scandal earlier.
News of the group - whose members bragged online about having sex with drunk and underage girls - was made public in media reports last Sunday.
But it emerged during the week that police had received a complaint from a 13-year-old girl as far back as 2011.
Speaking on TVNZ's Q+A programme today, Police Commissioner Peter Marshall said police could have done more in hindsight.
"We could have gone into the public arena perhaps earlier - that's one of the areas that we're going to explore. But we had the best intentions at heart, to have victims come to us so that we could progress matters.
"We're actually in the business of helping victims like this, we're in the business of putting people before the court where the evidence is there."
Police had investigated and interviewed members of the group, but no charges have been laid because police said there was not enough evidence.
Mr Marshall said police had called for more evidence this week.
"The request this week, throughout all this week, was if there's anyone out there who can give very specific evidence, we would follow it through."
Police have been criticised about the questions asked of the complainant, now aged 15, including about what clothes she had been wearing.
Mr Marshall told Q+A the girl had been interviewed by a specially trained female interviewer.
He said it was important to have a description of clothing because that was "fundamental" when it came to interviewing a suspect later on.
"I can understand, from a 13-year-old girl, being asked about clothing is unusual. She should have had it explained to her, I don't know whether she did. And I'm sure that she would find the whole question of an interview difficult."
In a 2007 report, Dame Margaret Bazley identified a culture of scepticism within police when dealing with sexual assault complaints. The report urged changes to police attitudes and behaviour.
Mr Marshall said Dame Margaret had wanted police to increase the number of people who had been formally trained in sexual assault complaints, and police had done that last year.
"We undertake dozens of interviews every month and I would say 99.9 per cent of the interviews conducted by our specialist interviewers are conducted in an entirely professional, appropriate way, and I'm sure this one was a well."
The Roast Busters scandal comes after Central District commander Superintendent Russell Gibson apologised in September for describing a 10-year-old rape victim as a "willing" participant.
Mr Marshall said that was a "disappointing comment" that was under intense review at the moment.
He disagreed police needed a cultural shift.
"No I don't, I think the New Zealand police has gone from strength to strength in recent years. Of course I would say that, but I make that point very clearly - I have absolute faith in the senior executive of New Zealand police."
Mr Marshall said he had a "very good team" working on the Roast Busters case.
"If there is the evidence, they will be charged, but the evidence requires people to come forward."