Paul Swain sent a text message to Helen Clark the day after the election to tell her he was quitting the Cabinet for the love of his family.
Yesterday, he made his intentions public, catching everyone by surprise. The 53-year-old Rimutaka MP said he was stepping down so he did not miss seeing his 10-month-old daughter, Madeline, grow up.
Mr Swain married his second wife, Toni Reeves-Swain, 28, in March 2003.
He would not say if he would stand again in the 2008 election.
Indicating to his daughter and wife, who were with him during an interview with the Herald in his Beehive office, Mr Swain said: "Having done six years, was I going to go on and do another three, do another term in Cabinet?
"She's just changing so much, you know, and I thought if I do that, then I get to the end of three years and she's 4 and I've missed so much.
"It's not just the physical time, it's the quality time. For me it's really been about weighing all that up."
Mr Swain also has two sons from his previous marriage, Ben, 23, and Sam, 20, who both live in Wellington.
"In the end the decision is that I want to spend more time with my family, spend more time with Toni and Madeline, and spend more time with my two boys."
Mr Swain said he also wanted more time for local electorate issues, and his standing aside gave a chance to "refresh" the Cabinet by allowing new talent to come through.
He followed up his text message to Helen Clark with a long conversation on the Tuesday after the election.
The Prime Minister was surprised, he said, but also grateful given the issues she faced in refreshing the Cabinet.
Mr Swain has had a heavy workload, being Minister of Immigration, Labour, SOEs and Corrections, as well as holding associate portfolios in Information Technology and Communications.
He was admitted to hospital in December 1999 with a serious illness - a ruptured bowel - and had three operations. He was on extended sick leave until mid-2000.
An MP since 1990, Mr Swain said the thing he would miss the most was the "gladiatorial sport" of Parliament's question time.
"It's the closest to batting in cricket that you can get. You are out on your own in the middle of the pitch. You are trying to protect your wickets from very fierce bowling and if you make a mistake you lose your wicket.
"There's no one to blame except yourself. I've just loved every minute of it," he said. "This job you need to be at it 110 per cent. When your focus starts to shift ... you need to be in a position where you need to make some decisions because you cannot cruise this job."
Helen Clark said Mr Swain had been a very able and highly regarded member of the Cabinet.
* Caretaker Minister of Immigration, Labour, SOEs, and Corrections; Associate Minister for Information Technology and Communications.
* Aged 53.
* Married with three children: two sons aged 20 and 23 from his former marriage and a 10-month-old daughter from his current marriage.
* In Parliament since 1990, when elected MP for Eastern Hutt.By Kevin Taylor