"I will have John Key as a bench mate, which is pretty funny."
Mr Foss was elected into parliament in 2005 and has had some "amazing" experiences during his 11 years.
"All the roles I have been in have made a difference to New Zealand, people may not see them in the headlines but there have been some significant changes."
He recalled the legislative side of things as being something he enjoyed, with the financial market reform, intellectual property and chasing down loan sharks, to name a few.
"I am proud of my ministerial achievements, including ushering through world's-best financial markets and consumer legislation, the Patents Act and enabling crowd-funding."
On the other hand he said representing New Zealand in places around the world was an "incredible experience."
"A personal favourite and highlight was representing New Zealand at the Battle of Chunuk Bair Centenary commemorations in Gallipoli during Anzac celebrations, it was incredibly powerful."
Mr Foss said it was a privilege to be an MP, as he believed there were fewer MPs and ministers in New Zealand than All Blacks.
"I have had a lot of support and at the end of the day you have to look after yourself and people respect that. I am looking forward to catching up with some friends in Hawke's Bay and not talking politics."
With Mr Key resigning and Mr English stepping up as the new Prime Minister there had been speculation that Mr Foss could get demoted from his positions in cabinet.
Mr Foss said the new prime minister "plays his cards pretty close to his chest", but he had not been given a message that he would lose his place.
"He expressed confidence in what I have done and my abilities but at the end of the day there is only room for 20 in Cabinet, and so many outside Cabinet.
"There's no axe falling on me ... who knows what could have played out next week. I don't know."
Labour Tukituki candidate Anna Lorck said Mr Foss has represented Hawke's Bay with "pride and passion".
"While politically he and I approach things from different angles, we have always agreed on one thing - Hawke's Bay is going places and this is an extremely exciting time for our region," Ms Lorck said.
She said Tukituki is totally a marginal seat and the race to elect a new MP would be "very close".
Napier Mayor Bill Dalton said he doesn't have any doubts that the decision to step down was a political one.
"At the end of the day we live in a political world," Mr Dalton said. "Craig decided he didn't want to be a back bench MP," he said.
He was quick to commend the Tukituki MP.
"Of course he did good things. You don't get into politics by not doing good things," he said.
Mr English said Mr Foss had been very competent and successful.
Asked if Mr Foss would have been demoted had he stayed on, the Prime Minister said he was not far enough through the process to say.
"He was a perfectly competent minister and I enjoyed working with him."
Mr English said he had a "few more discussions to have" but the reshuffle was coming together well.
Mr Foss said he was supportive of the new team, Mr English and Paula Bennett.
"I am leaving a great team behind and they have strong track records. I wish them well."
Mr Foss's decision came a day after local government minister and MP for Maungakiekie Sam Lotu-Iiga announced his move away from politics.