They then decided to swim for shore, about 1.5km away.
Luckily, one of the men had a water-resistant mobile phone, which he used to call police, after half an hour in the water.
A mayday relay was broadcast and a search was conducted by police with the help of the Centreport harbour pilot boat, two tug boats (Tapuhi and Tiaki) and Coastguard.
Two men were rescued by the Police Maritime Unit on board the Lady Elizabeth IV.
One was treated for hypothermia symptoms.
Senior Constable Kyle Smith of the Wellington Maritime Unit said without lifejackets and a waterproof cellphone, it could have been a very different outcome for the men.
"It's situations like this that show just how important it is to wear a life jacket when you're out on the water," he said.
"Ideally you should have two forms of waterproof communication devices such as a handheld VHF radio, a mobile phone in a waterproof case, a personal locator beacon or EPIRB, or a rocket or smoke flares."
Police would like to thank all those who assisted with the search.