There have been 32 privacy breaches at Inland Revenue involving 6,300 people in the past year.
Revenue Minister Peter Dunne says it came as a very big shock to him.
He says he's not happy with any breaches.
"But we deal with 25 million transactions a year so 32 breaches out of 25 million is a pretty small number, but frankly to my way of thinking - one breach is one too many."
Mr Dunne says he's very sorry to anyone who's had their information compromised, and he says he got guidance from the office of the Privacy Commissioner about 5,700 of those people.
"The advice was that their security was not compromised therefore they didn't need to have the level of protection that we've courted to the 630-odd."
He says people who were at particular risk of things like identity fraud were contacted and steps have been taken to make sure their security is protected.
Revenue spokesman David Clark says the Government admits the 20-year-old computer system at IRD needs updating.
"This is the kind of thing that happens when the systems are under pressure and we need the Government to step up because it's our private details that are getting sent out and it's not on."
However Peter Dunne says it's got nothing to do with that.
"The Labour Party keeps running this line that it's all about the computer system - this is human error, this has got nothing to do with the computer system, and their constant repetition of that line doesn't make one lie suddenly come right."
Mr Dunne says new steps have been put in place to make sure these types of breaches won't occur in the future.
He says this breach involved information being sent to the wrong people by email or in other ways.
"But it's not acceptable and the department is taking steps to make sure that there won't be repetitions in the future."