The guns were being held as exhibits or had been handed in for destruction.
The embarrassing blunder has police minister Stuart Nash demanding answers about the alleged burglary.
When the buyback scheme was announced, he had sought assurances from police they had strong processes for handling the firearms, he said.
"I have now sought fresh assurances from the Commissioner that all stations and firearms-handling processes are secure and fit for purpose, and that firearms are stored safely, whether they are crime scene exhibits or otherwise," Nash said.
Harris was allegedly found by a member of police staff in the yard of the police station at 7.40am.
He allegedly fled in a car, which police have since recovered, along with another car owned by him.
The investigation would also require an immediate audit of security around firearms at all police stations nationwide.
Military-style semi-automatics (MSSAs) and assault rifles were banned in the urgent law change which following the shootings at two Christchurch mosques on March 15.
MSSAs and assault rifles guns are now illegal, but the amnesty is in place until the end of September.