Liberal senator Ian Macdonald suggested there was a security breakdown.
Senator Brandis said a fair bit had been made of Hicks' interjection, particularly by the ABC, on what appeared to be a slow news day.
"There was no point whatsoever when I felt even remotely threatened," he said.
"There was a random individual who turned out to be a terrorist yelling at the side of a room for about three seconds."
Australian Federal Police (AFP) commissioner Andrew Colvin said there was no direct threat. AFP officers later spoke to Hicks who was fully co-operative.
Adelaide-born Hicks was captured while fighting with the Taliban in Afghanistan in 2001.
He was subsequently transferred to the US prison at Guantanamo Bay, Cuba, pleading guilty to supporting terrorism.
He was released in 2007. He claims he was mistreated while in Guantanamo Bay and convicted on a statement of facts for which he never received any evidence.
In his interjection, Hicks shouted out that he had been tortured in Guantanamo Bay with the full knowledge of the coalition government, asking Senator Brandis: "What do you have to say?"
Read next: 'CIA detainees were tortured', US senate report reveals
- AAP