Key Points:

The Prime Minister says the public should have no confidence in the Department of Corrections' handling of parole procedures, and is backing Judith Collins' efforts to remove its chief executive, Barry Matthews.

Ms Collins cannot sack Mr Matthews herself, but made further pointed remarks yesterday about wanting accountability from those responsible for the department's failings - essentially an ongoing message to his employer, the State Services Commissioner, that she cannot work with him.

Prime Minister John Key said the Auditor-General's report that found major deficiencies in the way Corrections managed parole made "damning reading".

Mr Key said he did not believe Ms Collins was putting undue pressure on the apolitical State Services Commissioner, Iain Rennie, to sack Mr Matthews.

"The New Zealand public is entitled to expect accountability, and quite frankly, that report made such damning reading they can have no confidence at this point that the department is following an approved set of procedures that they promised they would follow."

Mr Key said it was "a real issue" if a minister could not affirm confidence in his or her chief executive - another message to Mr Rennie.

Mr Rennie has 10 days to report back to Ms Collins on who is accountable for the department's failings and how the public's confidence in the parole system can be restored.

Ms Collins refused to express confidence in Mr Matthews when the Auditor-General's report was released on Tuesday. It showed the department failed to follow its own procedures in nearly every one of 100 cases of paroled offenders.

Yesterday, Ms Collins used set-up questions in Parliament to continue to criticise Corrections.

She said she was "staggered" by the report's findings, especially as it came after the Graeme Burton case, "when we might expect those responsible had learnt from the failings of the past".

Ms Collins initially directed Mr Matthews not to talk to the media, but lifted this yesterday.

He plans media interviews tomorrow but whether he will resign or defend his position is unclear.

Mr Rennie said that over many years, ministers had made comments about having no confidence in a chief executive, but the State Services Commissioner was the employer and had the final call. "It really is very important that ministers and chief executives have a highly productive relationship and I see part of my job as assisting that outcome."

Ms Collins received support from National's governing partners, Act and the Maori Party.

Act MP David Garrett asked her if she had received a letter of resignation from Mr Matthews, and if she had not, did she still believe her previous statements that "she is confident Mr Matthews is fully aware how seriously I view this issue".

Mr Garrett subsequently called for Mr Matthews' "head to roll".

Tariana Turia said the Maori Party stood by Ms Collins' call to Mr Rennie to "please explain".

Mr Matthews could find little support from Labour leader Phil Goff, the previous Minister of Corrections.

Mr Goff would not give his personal opinion on whether Mr Matthews should be sacked, but said as chief executive" he was "operationally responsible" for the deficiencies.

Mr Goff said dismissal was likely, because "when a minister loses confidence in a top official the result is almost automatic".