"In among the junk, was a rotting canvas bag, containing three broken shotguns."
He said there was always an amnesty for handing in firearms, although these ones did not pose a danger.
"What annoys me the most, is people who would dump in a cemetery."
He said rocks and rubble were among the junk. "Someone's cleared out their back yard."
He said he discussed the idea of putting a chain across the cemetery entrance.
"It's supposed to be a sanctuary - obviously not a sanctuary for everybody."
Masterton District Council spokesman Sam Rossiter-Stead said the police had collected the shotguns by the time a clean-up crew arrived.
Mr Rossiter-Stead reckoned the dumpers had used a truck or a ute to unload the rubbish.
However, dumping at Riverside Cemetery was a rarity. He said Riverside Cemetery was a "very unusual" dumping spot for illegal tippers.