"The Indian family attempted to ignore the defendant but was soon involved in a heated argument," a police summary said.
Rowe told the victims to "f... off back to their own country" and called them "black n.....s", the court heard.
The family left the area, worried they would be subject to a physical attack.
"Disgusted" members of the public witnessed what happened and reported the incident to police.
When police found Rowe shortly afterwards they questioned him about what he had said.
"He admitted to making racist comments but believed it was made in a light-hearted manner and he did not intend to upset anybody," court documents said.
Rayner told the court the incident began with a joke.
"He accepts at the very least it was in extremely bad taste," he said.
Community magistrate Simon Heale said the victims had not been identified and therefore no restorative-justice conference could take place.
"Your behaviour does you no credit," he told Rowe.
The court heard the defendant would leave Dunedin later this month to work in the North Island.
Rowe was fined $700 and court costs $130.