The newspaper reported that the recording had been inadvertent and the cameraman had tried to retrieve the recording device but was stopped by Mr Key's security staff.
It had sought legal advice suggesting it could publish the conversation but decided not to on ethical grounds, given the conversation was private.
Mr Ambrose said he was filming the meeting when Mr Banks started answering questions.
That was when he put his microphone on the table.
"I had enough time and enough space just to reach my arm and drop it on the table to try and get his questions,'' he told Newstalk ZB.
"I wasn't able to get a shot so I backed off and while I was backed off trying to get other shots we were basically hustled out of the room, told to get out.''
Mr Ambrose said he completely forgot about his microphone as he struggled to get a shot of the pair.
Speaking to reporters this afternoon, Mr Key said he had decided to lay a complaint under the Crimes Act.
"The law's quite clear, it's illegal to secretly tape someone if it's deliberate, it's my view that it was deliberate,'' Mr Key said.
The complaint would refer in the first instance to the cameraman, but could extend to the Herald on Sunday.
The Prime Minister refused to discuss what he had said in the conversation with Mr Banks, but remained adamant he had not said anything he was concerned about the public hearing.
"I'm not bothered in the slightest about what is on the tape, secondly, I am very bothered by the tactics that I believe have been deliberately deployed by the Herald on Sunday,'' he said.
Asked whether he had undermined Act leader Don Brash in the conversation, Mr Key again refused to go into the contents of the conversation.
He likened the recording to "News of the World tactics'', and said he was taking a stand against it.
Mr Key said he had not noticed the recording device on the table, and that his attention had been on Mr Banks.
Now there are calls for the conversation, which Mr Key has referred to as "bland'' to be made public.
Greens co-leader Russel Norman said Mr Banks and Mr Key have a right to privacy, but he said the case had raised a lot of interest and there was an easy way out for Mr Key.
He said that would be to release it and prove that he has got nothing to hide.
Speaking to Newstalk ZB this morning, Labour leader Phil Goff also said that while he was not in favour of secret recordings, he doubted the political pair would have been saying anything controversial.
"If I was sitting in a very public place, with twelve television cameras peering through the window at me, I would be very careful about my conversation, and I imagine two of them were."
He said it was up to the prime minister whether the conversation transcript should be released.
Herald On Sunday editor Bryce Johns said the freelance cameraman did not deliberately leave his microphone behind and the Prime Minister were deflecting the attention from what was said to how it was gathered.
Labour leader Phil Goff called the situation "a farce''.
"If I'd organised a media stunt and invited all the media along there and then had a conversation with somebody, I'd expect that conversation to be public. I think there were 12 different cameras pressed at the window there. It's a farce to say it wasn't a public conversation - very clearly it was. He'd invited the media to be there.''
He said he would not justify any unethical action by the media "but Mr Key set up this media opportunity''.
