He talked about how he had visited Auckland twice, once being last year on his way back from Apec.
He then lapsed into a story about his companion not being allowed to take live fish somewhere - it wasn't clear where.
He said his main interest was in the 50,000 Filipinos in New Zealand.
"I may be able to say they decided right because Auckland treats it citizens superbly," he said. "No doubt about it."
"New Zealand is a country small but governed well and protected well ahead of the time and that is good and you can see how it is now.
"The only thing that is lacking is you do not have atom bombs."
Ardern said New Zealand was a peaceful nation and its police were not armed either. It advocated for those principles and values and lived by them.
Ardern and Duterte sat down to sign an MOU on education co-operation before realising they were meant to be witnessing it, not signing it.
Once the signing was over, media left the room and the meeting continued behind closed doors.
Ardern said earlier in the day she planned to raise human rights issues with Duterte, as many other leaders have in light of the state sanctioned execution of thousands of drug traffickers without trial.