Labour will review its performance in the provincial seats after being caned by National.
The party lost 10 general seats - two held by Cabinet ministers - to National, mostly in the provinces.
"We're certainly going to have to have a look at what was going on in provincial cities," said party president Mike Williams.
"That was a surprise to us and we are obviously going to rebuild."
Napier, lost by MP Russell Fairbrother to National's Chris Tremain, was a particular surprise.
"The campaign was a good one. It was active, hardworking. We really need to find out what happened there. That's a seat we haven't lost since 1951."
Mr Williams said a lot of the seats had been targeted by the Exclusive Brethren pamphlets and "push-polling" but he could not attribute the losses to those reasons at present.
Adding to Labour's woes from National's resurgence was the Maori Party's success in taking three of the Maori seats from Labour to add to co-leader Tariana Turia's seat.
Labour's party vote was almost the same as it was at the last election - 40.74 per cent versus 41.26 per cent, giving it 50 MPs, two fewer than in 2002.
Among the four new faces in caucus is Shane Jones, former chairman of Sealord and chairman of Te Ohu Kaimoana. He is widely seen as strong ministerial material.
The others are former party president Maryan Street, Service and Food Workers Union national secretary Darien Fenton and Piako candidate Sue Moroney.
But eight sitting MPs in general electorates were defeated in the 10 seats National took - although all eight are back in Parliament through the party list.
Those who lost their seats but return to Parliament from Labour's list are Cabinet ministers Jim Sutton and Rick Barker, and David Parker, Jill Pettis, Ann Hartley, Moana Mackey, Dianne Yates and Mr Fairbrother.
Labour also lost Invercargill, where MP Mark Peck is retiring, and Wairarapa, where Georgina Beyer opted for the list only.
Auckland University political scientist Raymond Miller said losing so many electorates would have hurt the Labour Party.
It had suffered serious provincial losses, particularly in the South Island. In centres such as Invercargill, Timaru and Napier, heavy 2002 majorities had been turned around.
He said the country was now split between provincial towns and the rural hinterland held by National, and the low and middle- income urban seats held by Labour.
Dr Miller said Labour would be pessimistic about its chances of regaining many of these seats. For example the situation in Mr Sutton's Aoraki seat - where he went from a 6400-vote majority to a 6600-vote defeat - was a "pretty substantial changeover".
Labour Maori MPs returning thanks to their party list ranking are Dover Samuels and Mita Ririnui. John Tamihere will not be back, as he was not on the list.