Steve Kingsbury of Coastguard Whitianga said he and three other volunteers was called in to assist in the wee hours of the morning - and with good sea conditions, they were able to get to Port Charles in about 50 minutes.
"We were tasked with a shoreline search - so we just followed the shoreline up from south of Port Charles and into the harbour area."
They found the man on rocks in the harbour area after about two hours of searching.
"The surge of the sea meant we couldn't go directly on to the rocks where he was, so we got him to move along the rocks to an area where we could get the boat in and one of our guys got ashore and assisted him back to our vessel," Mr Kingsbury said.
They then took him to the Lion Foundation Rescue boat, which was "a few hundred metres away".
"He was cold - hypothermic - he'd been in the water for some hours and also sitting on the rocks. He wasn't dressed for being outdoors, he was just wearing his normal shorts and shirt, no jumper, no lifejacket.
"Once we got him into some warmer clothes we were able to speak to him, and get some basic details."
"He was a happy chap - understandably so."
Coastguard Operations Centre's Johnny Clough said the rescue effort was outstanding, "with all agencies working together for the best outcome".
"The search took 7 1/2 hours with 20 coastguard volunteers responding to the call for help in the middle of the night."
Metservice reported a low temperature of 15C on Sunday.