Prime Minister Helen Clark is "astounded" officials did not pick up earlier the fact Kelly Chal was ineligible to run for Parliament.
United Future MP Mrs Chal lost her seat before being formally sworn in, as she was not a New Zealand citizen and therefore not eligible to be a candidate.
She was replaced by the next candidate on the United Future party list, Paul Adams.
United Future leader Peter Dunne and the Chief Electoral Officer, David Henry, today argued over who was to blame.
Mr Henry has written a "please explain" letter to Mrs Chal and the party, his office said today.
United Future has taken the matter to a lawyer, who told it to make no further comment, a spokesman for Mr Dunne said today.
Asked about the affair at a press conference today, Miss Clark said no party could afford to be sanctimonious about it.
"I can not say hand on heart that no other well-established party will not in future make the same error," she said.
"All of us have in our caucuses people who are not born in our country.
"What does astound me is ... Mrs Chal having clearly signalled permanent residency on the form didn't then have it picked up at the party level or at the returning officer level.
"There needs to be some discussion by the minister, and justice and electoral officials now as to whether or not it is their role to point out what appears to be a bit of a discrepancy."
Miss Clark did not believe charges should be laid over making a false declaration, saying that would be heavy handed.
"It was not a calculated attempt to deceive, in my opinion," she said.
"It's a most unfortunate oversight, which puts everyone on notice for the future. All these details must be checked."
United Future last week signed a deal with the minority coalition Government to support it on confidence and supply votes for this three-year term.
United Future secretary Murray Smith was responsible for providing paperwork to the Electoral Office, which ensures parties, party lists and MPs meet the law.
Yesterday, Mr Henry said Mrs Chal was ineligible. Mr Dunne has asked why it took so long to be picked up.
MPs have to be New Zealand citizens and must assert that when they complete nomination forms, as do party secretaries when they complete list forms.
"We rely on that assertion by those two people unless we have some reason to think otherwise," Mr Henry said.
Neither Mrs Chal nor Mr Smith have returned phone calls.
Mrs Chal applied for citizenship only after being elected on July 27.
Internal Affairs advised the Electoral Office she was not a citizen, which then acted on the information.
Her citizenship application was a "perfectly innocent coincidence", Mr Dunne has said.
It was made when she "became aware of her situation".
Mrs Chal indicated on her candidate declaration form that she had residency, not citizenship.