Police Minister Stuart Nash is demanding an explanation from police after 11 guns were stolen from a police station.
Police said Alan James Harris, 38, allegedly stole the firearms from the Palmerston North Police Station yesterday morning.
The guns were not police firearms but were being held as exhibits or had been handed in for destruction.
The embarrassing blunder occurred as gun owners are surrendering military-style semi-automatics (MSSAs) and assault rifles around the country as part of a Government buyback scheme which followed the Christchurch mosque shootings.
Nash said this afternoon that he had "grave concerns" about the alleged burglary.
When the buyback scheme was announced, he had sought assurances from police they had strong processes for handling the firearms, he said.
Police said at the time they were confident they could manage the collection of the high-powered weapons.
"I have now sought fresh assurances from the Commissioner that all stations and firearms-handling processes are secure and fit for purpose, and that firearms are stored safely, whether they are crime scene exhibits or otherwise," Nash said.
Harris was found by a member of police staff in the yard of the police station at 7.40am.
He fled in a car, which police have since recovered, along with another car owned by him.
Acting Central District Commander Inspector Sarah Stewart said the burglary was being taken very seriously.
"Locating Harris is a high priority for Central District Police," she said.
"I urge anyone with information about his whereabouts to come forward to police."
"A warrant for his arrest has been issued. He should not be approached."
She added: "I am very concerned about what has occurred — it is absolutely unacceptable."
Police Commissioner Mike Bush has asked for an investigation into how Harris was able to get into the police station.
The investigation would also require an immediate audit of security around firearms at all police stations nationwide.
Military-style semi-automatics (MSSAs) and assault rifles were banned in an urgent law change which following the shootings at two Christchurch mosques on March 15.
While MSSAs and assault rifles guns are now illegal, an amnesty is in place until the end of September.