Although there had been a number of armed incidents in recent weeks, it was important to remember these events were extremely rare, Mr Bush said.
Read more:
Details of charges against Upper Hutt gunman revealed
Upper Hutt shooting: Cousin says Pera Smiler wanted police to kill
Man shot dead by police outside Upper Hutt McDonald's 'was an amazing guy'
It was also rare for police to have to use lethal force, he said.
These incidents should never be used as a reason to arm police, he said.
The police and the public were safer if they were not armed with guns, Mr Bush said.
"They recently have been equipped with Tasers at all times, which are not lethal and function at close quarters. Guns reduce the options in tense situations."
Referring to an editorial in the New Zealand Herald, Mr Bush said it was "heartening" to see the professionalism of police staff acknowledged in these events.
"Among other things, it says: 'It may be that just about everybody now recognises the risks we ask police officers to take in situations where offenders are, or may be, armed. The more these dangerous situations occur, the more grateful we should be to those prepared to put themselves in harm's way for the public's protection."'