John Key was given a standing ovation as he was farewelled by his caucus colleagues today who presented him with a silver tray.

The former Prime Minister had just left the caucus room and on hearing much laughter he poked his head back inside and pleaded with them to feel a little sad for a bit longer.

His successor Bill English said in truth they're sorry to see him go.

English said his departure was a poignant moment and describes Key as the most outstanding Prime Minister of the last 30 years.