Members from the armed offenders squad were also called to the scene.
Mr Hurst said the man had his face covered, possibly with a balaclava, and was wearing gumboots and a black trench-coat.
He was of a medium build and was "quite tall", Mr Hurst said.
Police had not yet caught the man and a unit remained at the scene, Mr Hurst said.
Canterbury police said a police dog and handler were now at the scene.
The incident had left the young boy "shaken".
A homeowner reported that a man armed with a firearm fled from their property after being disturbed by one of the occupants about 6.20am.
The armed man had started a small fire in the house before fleeing but the occupants managed to extinguish it without needing assistance from the Fire Service.
The man is described as tall, medium build, wearing dark clothing, gumboots and something covering his face.
Anyone who saw the man or has any information that may be of use to the investigation should contact Canterbury police by calling 03 687 9809.
Information can also be provided anonymously by calling 0800 555 111.