MSD has also hired an independent security expert to carry out its own investigation.
Ministry chief executive Brendan Boyle admitted embarrassment over the latest privacy breach.
He declined to name the expert but said the terms of reference for the investigation would be drafted overnight, and an interim report would be completed in two weeks.
An internal taskforce would be set up to support the review and work with the Privacy Commissioner.
Mr Boyle said KPMG and other IT experts were hired regularly by the ministry to attack their sites and expose any vulnerabilities.
He would not say why KPMG had not picked up on the security flaw with the WINZ kiosks.
KPMG said they had not been involved in auditing the WINZ kiosks, but had worked on MSD servers separately.
Staff at MSD installed the system more than two years ago and it had been changed after a year.
Mr Boyle and Social Development Minister Paula Bennett both apologised for the breach.
Ms Bennett confirmed MSD had been contacted by a man last week asking for a "reward" in order to share details of a privacy loophole in their system.
Labour's spokeswoman for social development Jacinda Ardern called for an urgent debate on the privacy breach.
She said the information revealed by blogger Keith Ng had "exposed a massive weakness" in the system.
Questions would be asked in Parliament tomorrow.
Assistant Privacy Commissioner Katrine Evans said the breach was of major public interest and had dented confidence in the Government.
"Any time we have an episode like this, it has the capacity for very serious damage to trust in government and in business.
"I think people are waking up to the fact that protection of personal information is one of the foundation stones.
Beneficiary Advocate Kay Brereton said she had alerted the Ministry to the problem a year ago. She said an IT person at the Wellington People's Centre was able to view IP addresses across the server.
Mr Boyle said the earlier breach was unrelated, but Ms Brereton said she believed the security breach last year was related to the breach uncovered this week.
She said a decision had been made to not delve into the MSD server due to breaching privacy, but said she believed it was possible.
Mr Ng has raised $4000 for writing the story though fundraising website Givealittle.
"Basically, it's busking journalism. I do the story, then ask for money. It beats the hell out of freelancing," said Mr Ng.
What the blogger found:
Keith Ng, who blogs on publicaddress.net, wrote in a post at 10pm on Sunday (14/10/12) that he had followed up on a tip-off about a security lapse last week.
He had gone to two Wellington offices and found anyone could open private files through public computer kiosks.
Mr Ng said data exposed to public view included:
* Names of candidates for adoptions and foster parents
* Debt collectors' invoices, which listed the names of clients who owed money
* Names of children living in Child, Youth and Family care homes
* Addresses of the care homes
* Names of children and their medical prescriptions on pharmacy invoices
* Names of investigators and clients in fraud investigations
"This stuff was all a few clicks away at any Winz kiosk, anywhere in the country," Mr Ng said on his blog post.
"The privacy breach is massive, and the safety of vulnerable children was put at risk."
