The withdrawal continued Wild Thing's troubled history in the race, having capsized in 2004 and withdrawn shortly after starting in 2009. Wild Thing struck a media boat in Sydney Harbour last year, but still finished fifth across the line.
Apart from taking line honours in 2003, Wild Thing finished third in 2005 and 06 and second in 2008.
Celestial also withdrew Tuesday with a broken gooseneck, which links the boom to the mast, and Accenture Yeah Baby was forced out with an unspecified equipment failure.
The yachts were sailing in good conditions after running into a southerly front on their first night out from Sydney, bringing winds of up to 30 knots and rough seas.
"The front passed last night with quite a punch, with pelting rain that lasted for about 40 minutes," said Mike Broughton, navigator on eighth-placed Jazz.
Broughton said swells were 3 to 4 meters high, short and confused, and the wind veered 180 degrees as the front passed over.
By early afternoon Tuesday, most yachts were still hugging the coast of New South Wales state and were hard on the wind. In Bass Strait the wind was blowing 25-30 knots and had veered to the southwest, allowing the boats to head south on starboard tack.
Wild Oats XI was making about 15 knots.
The wind was expected to lighten through the afternoon, then pick up again close to the Tasmania coast.
- AP