Labour MP Trevor Mallard said Mr Dunne's party entitlements should be immediately removed as he was effectively an independent MP. After saying Parliament was "a farce" and accusing Mr Carter of "looking after your mates", he walked out of Parliament rather than apologise.
When Mr Carter refused to provide the advice he had received on the matter, NZ First leader Winston Peters led his party out in protest.
Outside Parliament, Mr Mallard said the Speaker was simply trying to protect Mr Dunne because he supported the National Government.
He also questioned whether re-registering United Future would mean those entitlements were automatically restored. Rules stated independents could only be recognised as MPs for a newly registered party if that party had at least six MPs.
Mr Mallard later wrote to the Auditor-General asking whether continued funding for United Future was within the rules for public funding.
Mr Carter said there were clear rules when an MP formed a new party, but there was no precedent or specific rule if a party's registration ended while it was in Parliament.
"The consequences of loss of recognition are significant and I consider I should follow fair and proper process in determining this matter."
Mr Carter said he would ask the standing orders committee to see if a rule change was needed.
Mr Dunne said Mr Mallard and Mr Peters were "just game-playing".