Tens of millions of dollars have been incorrectly paid out to beneficiaries since 1993 because of an automatic payment mistake at the Ministry of Social Development.
Almost 90,000 beneficiaries have been affected by the error, which resulted in thousands of people being given incorrect accommodation supplement payments across 21 years, according to RNZ.
It is believed more than 41,000 people have been underpaid a total of $23 million, and just under 46,000 people have overpayments of about $32 million.
The mistake was discovered in September 2014 and was fixed a few months later, but Social Development Minister Anne Tolley was not informed until March this year.
Speaking to reporters at Parliament this morning, Ms Tolley said she did not know why she was not told about the mistake until 18 months after it was detected.
"I want to know how we're going to fix it. That's my first focus."
Ministry of Social Development chief executive Brendan Boyle had formally apologised to the minister for the error.
According to RNZ, Ministry of Social Development deputy chief executive Ruth Bound said most of the underpayments were less than $1000.
"Work is under way right now to work through options to address the error and do the right thing by people who have been underpaid," she told RNZ.
About 800 of the 286,000 people receiving the accommodation supplement were affected by the error, Ms Bound said.
Ms Tolley said she hoped to have all the wrongful payments addressed by October. But she warned that the cost of tracking down the underpaid beneficiaries could be larger than the cost of repaying them.
"I've had initial advice on this, but I'm awaiting further advice around the total number of people affected, the total amounts involved, and the best way to deal with any reimbursements.
"We will not be seeking any repayments."
The review which detected the incorrect payments also picked up five other areas "which could have some implications for repayments".
Ms Tolley said these were mostly minor mistakes which did not involve large amounts of money.
Labour Party social development spokeswoman Carmel Sepuloni said the Ministry of Social Development needed to take people's entitlements seriously.
She said there was a culture at Work and Income of "where success is based on how much you can deny people".
The Ministry of Social Development said it would report back to ministers in June on proposals to address the error.