A man who was onboard a 360 Discovery Cruises passenger ferry said someone on the vessel spotted the body about two miles off the coast of Mairangi Bay around 4pm yesterday.
The witness told the Weekend Herald he thought the man may have been the victim of a shark attack because his right leg was missing up to his waist.
"It was a very distressing mental image."
James Bailey, manager of 360 Discovery Cruises, confirmed one of the company's vessels was involved in the gruesome discovery.
"Our thoughts and condolences are with the family at this time," he said.
The witness said the dead man was wearing a wetsuit and it didn't appear he had been in the water very long.
The skipper stopped the boat and waited with the body until maritime police and the Harbour Master arrived.
The company that ran the ferry service had offered the passengers who had been onboard counselling, the man said.
The witness said he wanted to offer his condolences to the man's family.
Police retrieved the man's body from Mairangi Bay on Friday afternoon, a police spokeswoman said.
They were making enquiries to formally identify him and believed the man was the swimmer who failed to return from Narrow Neck Beach.
DoC marine scientist Clinton Duffy said it was quite possible a shark could have bitten the man.
"There are a number of different species of shark in the harbour. There's generally more over the spring and summer. The numbers are trailing off at the moment but there will still be some around."