"I think it is really about trashing the work of volunteers as much as it is about destroying an icon for our area."
She said the loss of the Hay Family would impact on tourism in the town.
"People were stopping to photograph themselves there, then heading to the shops to have an ice-cream."
The Slee family, from Kurow, had been involved in the annual decoration of the hay bales for more than 20 years, and Janina Slee said the incident was "really disappointing".
Mrs Slee said she and about nine other volunteers had spent about five hours painting and stacking the hay bales, which were this year decorated to look like the Minions from the Despicable Me movie.
The bales were to have come down at the end of the month, so the Hay Family would not return until next year, she said.
Kurow Volunteer Fire Brigade deputy chief fire officer Richard Nehoff said the fire brigade was first called to the fire at 3am, and spent two hours extinguishing flames, but was called back at 7.45am, after the flames re-ignited.
He said volunteers spent a total of six hours fighting the fire and used about 16,000 litres of water.
The cause of the fire was not yet known, he said.
Hannah McKenzie, of Christchurch, said she was in Kurow on holiday with friends and they had all been disappointed at the loss of the Hay Family.
"We saw them yesterday, but now they're gone."
She said her friend's two-year-old daughter was particularly upset.
"She thought they were pretty cool."