Police continued the search until 9pm, with the assistance of Surf Life Saving New Zealand.
An Otago Regional Rescue Helicopter assisted.
Otago Surf Life Saving search and rescue co-ordinator Max Corboy said six inflatable rescue boats combed the water starting at the Second Beach cliff, where the swimmer had entered the water.
Corboy said they had covered an area stretching from St Clair to Tunnel Beach over a period of more than four hours.
"We're pretty gutted that we couldn't find a result for the family."
More than 15 surf life-savers would continue the search this morning and were not giving up hope.
"All the surf life-savers are pretty determined."
The conditions were "reasonably flat".
An Otago Daily Times reporter at the scene yesterday said a helicopter could be seen hovering above Second Beach.
Two ambulances, a fire appliance and three police cars were also at the scene.
Water conditions at St Clair Beach yesterday were calm, with a light breeze.