The community say will come through formal public consultation on the council's draft 10-year budget, beginning in the first quarter of next year.
The council will review its decision mid-2018 after hearing community feedback.
Bill Grainger and Gail McIntosh voted against the proposal, with both preferring an integrated library and museum on Willow St.
Grainger said he wanted a museum but believed there were too many barriers to make the Cliff Rd site work.
"Building in a residential area will have constraints and I personally don't think it will come to fruition."
He advocated keeping the Cliff Rd area as open space and said the council should heed the results of a community survey of 400 residents, the majority of whom favoured an integrated facility on Willow St.
Baldock said he did not believe the site would be as problematic as had been predicted.
He said the council had gone through a year-long analysis to make a viable plan, looking through "oodles of material" in the process.
"If we do it right we will all be proud of it. There are challenges ahead but we solve those one at a time."
The council heard from Graeme Horsley, a member of the Heart of the City technical advisory group, who said this year he had seen consultants and advisers move from being strongly in favour of the Willow St site to preferring the Cliff Rd site for a museum.
A former property developer, Horsley said he had spoken to residents of Cliff Rd, including the developers of the Latitude apartments.
Horsley believed that only the residents on the ground floor apartment would potentially be negatively impacted by a museum.
A woman in the public gallery interjected: "Excuse me!"
Goal of a museum
"Through the residents come cognisance; through cognisance comes understanding; through understanding comes knowledge; through knowledge comes life and wellbeing."
- Tauranga Moana Museum Trust chairman Neil Te Kani
New central library squeaks over the line
The council also voted seven to four in favour of building a $30m new central library on Willow St.
Some felt the community could not afford the $25m ratepayer chunk of the cost, and the current building would do for now with a little maintenance and remediation.
The rest were for listening to the experts, who had told them remediating what deputy mayor Kelvin Clout described as a "dark, dank, gloomy place" would be a waste of money.
The ayes had it and the project will go out for public consultation alongside the museum decision next year.
Library: How they voted
For: Max Mason, Leanne Brown, Terry Molloy, Kelvin Clout, Greg Brownless, Gail McIntosh
Against: Bill Grainger, Rick Curach, Steve Molloy, Catherine Stewart.
