Ferrier, who bought the property with his four siblings from an uncle five years ago, said the property was a perfect escape from modern society.
With no electricity, his family have had to get creative when it came to entertaining guests.
"It's going back to basics, that's what I'm going to miss most," he said.
In 1862, Governor Sir George Grey bought the entire island and began building the mansion on the coast of Home Bay. Ten years later he sold the property for less than $2000 and the new owners leased it out to Auckland fisherman Albert Sanford.
Sanford built the fish smoking shed, which remains today, and started selling snapper from the waters of his front yard.
Baileys agent Pat Regan said it's still a pleasant spot for fishing as rolling hills surround the property protecting it from wind.