The building was owned by Masterton Trust Lands Trust.
Construction manager David Hargood said that in recent years the building had been used as a community centre.
"It was originally a factory that employed quite a few people to produce plastic raincoats, I think. Then I think the market wasn't there, so they shut it down, and then the Wairarapa Community Centre went in there for a few years," Mr Hargood said.
The community centre moved to Perry St about 18 months ago.
Mr Hargood said the decision was made to demolish the building because it would be too hard to find another use for it and the land was probably better for developing than anything else.
It took workers one day to pull the building down and will take about a week to get rid of leftover rubbish.