The main building at Waiora House in Taupō is to be demolished.
The former maternity home housed a number of social service and community groups and was closed after the discovery of asbestos containing materials last August.
Taupō mayor David Trewavas said while the site was stable, industry experts had expressed grave concerns about the building being left in its current state for any length of time.
They met with the council as the building owner, and its tenant the Waiora House Trust, and all had agreed demolition was the best option going forward.
"There is still a number of materials containing asbestos in the building and trying to remove it all and reinstating the building will be costly and time consuming," Trewavas said.
"We'd be facing a bill upward of $3 million."
By demolishing the building to a clean site, the council and the trust could be sure that all asbestos had been removed and any associated health and safety issues would be completely eliminated.
Demolition cost and remedial works to the site was estimated to be about $580,000.
Trewavas said the council would be working with the trust over options for the future and helping the tenants to extend the leases on their temporary premises.
"It is sad that a little bit of Taupō history will be lost, but we've got to do what is right for everyone involved in the long term," he said.
Demolition will start this week.