Sir Geoffrey said Mr Key should have been told from the start.
"I would have thought if the GCSB was using its sophisticated surveillance methods in a situation like this, it would be prudent to tell the minister. I don't understand that at all.
"In my experience with them, they were meticulous in consulting ministers when they needed, and should have."
Mr Key has apologised to Mr Dotcom - an apology that has been reported worldwide.
Yesterday, the Prime Minister said that while it was unacceptable, it was a human error and someone was being held accountable. It was an employment matter for agency director Ian Fletcher.
Mr Key said he had made it clear he expected better from both the GCSB and the Security Intelligence Service in future.
However, former GCSB acting director Simon Murdoch warned his political masters that the bureau was under strain just four months before its work on the Dotcom case.
In its annual report last October, Mr Murdoch wrote that it had been through four CEOs in a year, there had been several efficiency reviews, a fiscal tightening, and an office move.
He said these "stresses" were the primary factor behind a workforce survey that showed "staff concern about levels of drive, clarity and alignment".
He also wrote that in some of its intelligence-gathering work "GCSB struggled to make headway because of critical staff absences".
Labour has claimed Mr Key neglected his duty to oversee the bureau, and yesterday, the party's deputy leader, Grant Robertson, said the concerns raised in the annual report were a further sign of this.
Mr Key also defended the police over another legal misstep in the case - the search warrants that Chief High Court Judge Helen Winkelmann found were illegal.
The Prime Minister said describing that as a bungle by police was "a bit harsh" because it was a matter of differing legal interpretation.
Mr Fletcher, the GCSB's current director, has apologised for the lapse and will conduct an internal review with support from the State Services Commission.
Meanwhile, Mr Key dismissed calls for an independent inquiry, saying that the Inspector-General of Intelligence and Security, Paul Neazor, who reported on the issue, was respected by both sides of the political divide.
Justice Neazor told 3 News last night that the GCSB had "made a mistake".
Asked about a further three instances relating to the GCSB's use of its powers that he referred to in his last annual report, Justice Neazor, a retired High Court judge, said he could not recall them.
However, Mr Key said he had asked Justice Neazor about those instances and was assured they were minor and had been addressed.
Any payout likely to be low
The unlawful eavesdropping on Kim Dotcom by the Government's foreign intelligence agency is unlikely to net him a substantial amount in damages, even if he does sue.
Chen Palmer lawyer Nick Russell said there were two possible avenues for Mr Dotcom to seek damages for the Government Communications Security Bureau intercepting communications from him and an acquaintance unlawfully.
One was to take action for unreasonable search and seizure under the Bill of Rights Act and the other was to take an action in common law for abuse of process.
However, neither course was likely to result in a major payout.
Mr Russell said the courts had previously awarded modest sums under the Bill of Rights Act and made it clear that was not the road to riches for claimants.
To secure compensation under the common law would require proof the actions of the GCSB were malicious or at least in flagrant disregard of the law, rather than a mistake or accident.
Even if that could be proven, the amount in compensation would largely be decided by actual losses Mr Dotcom had suffered as a result.
