The exchange came as Parliament wrapped up its last day before the campaign proper and politicians got back on the campaign trail.
Electioneering was put on hold to deal with the latest Covid-19 outbreak but has now been taken off pause in the lead-up to the October 17 election date.
Peters skipped the last day of Parliament to get a jump on his campaign and kicked off the second half of his bus tour of New Zealand at a brewery in Upper Hutt.
"Don't keep on gilding the lily and saying everyone's fine when it wasn't going fine.
"The testing wasn't going on, the surveillance wasn't going on, the oversight and scrutiny that should have been done by the military was not happening. And masks were not used."
And he put the responsibility for those mistakes at Labour ministers' feet.
"We let our guard down," said Peters.
"Too many things fell through the traps, or the holes so to speak, that were deliberately left there by the bureaucracy ... The fact of the matter is that the Labour ministers are the only ones in charge of all that."