The 11 weapons have now been recovered by police, Clement said.
No one else has been charged over the incident.
Clement said alleged theft took place "about the same time" as the local Anzac Day dawn services and that when staff left the police station, one of the rear doors was "inadvertently left open".
"As a consequence of that, we've undertaken an immediate review of that circumstance and across the rest of our police stations to make sure that this isn't repeated."
The weapons were not police firearms - they were being held as exhibits or had been handed in for destruction.
The embarrassing loss occurred while gun owners are surrendering military-style semi-automatics (MSSAs) and assault rifles nationally as part of a Government buy-back scheme following the Christchurch mosque shootings.
MSSAs and assault rifles were banned in rapid law changes after the March 15 shootings. An amnesty for handing in the weapons is operating until the end of September.
When the buy-back was announced, Police Minister Stuart Nash sought police assurance they had strong processes for handling the firearms. Police said at the time they were confident they could manage the collection of the guns.
Following the alleged burglary, Nash has sought a new assurance.
However, Clement has urged people to keep hold of their guns until police make a formal announcement on how they will collect and destroy the firearms.
"We've said right from the outset, since the announcement of the legislation changes, just hold your firearms at your own secure locations while we work through a process to identify a safe way for them to be forfeited," he said.
"Handing them into police stations is one of the least preferred options with regard to how we want to take them back."
