He said he would not release them until he had all the records he required "to complete this investigation, which clearly the Prime Minister is not prepared to do himself."
He said rather than deliver it up, he wanted to "squeeze, like pulling teeth, the truth out of the Prime Minister." He said it was the Prime Minister's responsibility to investigate the leak.
Meanwhile, Speaker David Carter has not yet made a decision on a request for Mr Dunne to face the Privileges Committee over the leak of the GCSB report.
Labour leader David Shearer asked the Speaker to refer Mr Dunne to the Privileges Committee for his denial at a select committee that he was responsible for the leak. Mr Dunne has continued to deny it, but resigned as a Minister last week for withholding emails from an inquiry by David Henry.
Act leader John Banks also revealed he laid a breach of privilege complaint against Labour's Trevor Mallard last week after Mr Mallard strongly criticised the Speaker's decision to allow United Future to retain its Parliamentary entitlements while it sorted out its registration.
Mr Mallard described it as "farcical" and left the Debating Chamber rather than be forced to withdraw and apologise for it. Mr Mallard stood to withdraw and apologise before Question Time today, which Mr Carter said related to last week.
He said he was yet to make a decision on Mr Banks' complaint, lodged last Friday, because he needed to first seek a response from Mr Mallard.