Police and court security staff had to intervene on Tuesday when a defendant and a member of the public gallery began shouting at each other.
Joshua Carnegie appeared in the Whanganui District Court on charges of raping a female aged 12-16, unlawful sexual connection with a female 12-16, indecent assault on a female 12-16, and two counts of indecent assault on a girl under 12.
He also faces charges of possessing a weapon in a public place, and possessing drugs.
Carnegie applied for bail at his appearance, but it was declined by Judge Patrick Treston.
As the appearance concluded, Carnegie addressed a man in the back of the public gallery, who began to yell and swear at him.
The pair began shouting at each other and each had to be removed from the courtroom.
Another man in the gallery also piped up as Carnegie was being taken down to the cells, but remained seated.
Judge Treston said one of Carnegie's alleged victims was five years old when the offending is said to have happened. The allegations are historical.
Carnegie will next appear in court on July 19.