A Wanaka mother asked to leave the Queenstown District Court by Judge Kevin Phillips for breast-feeding her 18-week-old daughter said while she understood the reasoning, "The way he went about it was not the best".
A spokesman for judges believed it may be the first time the issue has arisen in a New Zealand courtroom.
Catherine Owen was seated near the front of the public gallery on Tuesday with her daughter, supporting her partner who was due to appear in court.
No other members of the public were close to her. She was, however, in the judge's direct line of sight.
The baby began to cry, so Ms Owen started breast-feeding her.
Shortly after, Judge Phillips asked "Why is there a baby being breast-fed in my courtroom?", at which point a court bailiff approached Ms Owen, but she had already started to leave the courtroom.
"I'm glad I heard him say what he said before the bailiff came over to me," Ms Owen said. "It would have been a lot more embarrassing to have been escorted by the bailiff".
Ms Owens said she remained in the public waiting area until her partner's name was called, but decided not to speak to court staff about the incident.
"There's not much point making a massive deal about it - it's his courtroom and it doesn't [adhere] to the same rules.
"A few people have said I should lay a complaint."
She had decided not to take the matter further, but said the embarrassment was caused by "the way he went about it".
Senior judicial communications adviser Neil Billington, of Wellington, told the Otago Daily Times yesterday that judges had a "legal authority" to run their courtrooms "in a manner that they consider appropriate to the administration of justice".
"It's based on a legal power that each and every judge has ... each judge is independent in terms of how they exercise that authority."
Mr Billington said while Judge Phillips had the authority to request Ms Owens leave the courtroom, "I'm not aware an issue of this nature has arisen before".
"Although courtrooms are open to the public, they're not 'public places' in the usual sense ... where members of the public are free to do whatever they like."
A Ministry of Health website reports that "under the Human Rights Act, it's illegal for someone to stop you breast-feeding in public".
Mr Billington said Judge Phillips would not comment on his decision "and that's the usual situation".