His comments came after Ardern criticised Dutton's government's policies on deportation last week.
She said Australia was well within its rights to deport individuals who break its laws but it is too often deporting people with no connection to New Zealand.
"We have a simple request. Send back Kiwis, genuine Kiwis - do not deport your people and your problems," Ardern said.
While she was in Sydney last week, Ardern lobbied Australian Prime Minister Scott Morrison to change the deportation rules, but he said Australia would be making no changes.
Speaking to the ABC, Peters said the policy was not fair, and it was a bit rich for Australia to be shipping its unwanted people to New Zealand when an Australian has been charged over the deaths of 51 people in the Christchurch mosque attacks.
"Of all countries, Australia should understand the unfairness of deportation for criminal reasons," he said.
"You've sent thousands back to New Zealand and we have sent a handful of people back to you.
"Did we make a song and dance about Australia about that? It was the worst tragedy we've ever had - 51 people lost their lives - scores and scores were damaged forever," he said.
"Far worse than Port Arthur, and nobody in my country sought to abuse Australia about that, that's my point ... and we want a thing called respect."
Peters said it was time the "politicisation" of the issue is put aside but then suggested it was Dutton who politicised the issue in the first place.
"Let's not guild the lily, let's be honest with each other because Australians and New Zealanders have enormous respect for a lot of reasons and we are both better than that."