It's a rough passage today. We've left the Sandspit wharf and are bouncing up and down in the waves on a boat owned by Kawau Lodge's Dave and Helen Jeffery. I cling to my lightweight children as they lift off the ground with each bump.
Into the calm Kawau Bay, we cruise past two yachts belonging to residents who have sailed around the world in them - one as many as 12 times and the other in a home-made boat with no radio.
Looking at well-maintained homes as close to the water as possible, I can easily appreciate the self-sufficiency of the people who live here - around 30 permanent residents, out of a possible 400 homes on the island.
"One man built his house himself carrying most of the materials up the hill on his back," says Dave as we notice most homes have makeshift railway lines to transport bags, or people, up the steep slopes.
We're dropping one of Dave's neighbours, Sue, at her home across the bay. She says one of her favourite things to do here is to take a boat to Mansion House for dinner on moonlit nights when there is phosphorescence in the water.
She hops off at one of the jetties poking far out into the water and we head for ours, so long because of the tidal waters here. Some nights, when Dave and Helen have been for dinner across the bay, they've been able to walk home as the water has drained completely. It takes a bit of planning not to get stranded.
After dropping off our bags, we set out on an afternoon boat tour of the island, where Dave tells us about its fascinating and colourful history - piracy, cannibalism, copper mining, Sir George Grey and his monkeys, zebra and exotic plants, as well as some famously wild parties after fishing contests. The island definitely has a lively spirit.
An element of nonconformist behaviour must be a prerequisite to life here. We're told about one resident who is building an interesting-looking home without building permits. He's known to say: "I'll be dead before they get through the red tape to check up on me," but nevertheless builds his walls with Velcro attached so he can show the inspectors the work if needed.
We pass the ritzy homes in Pembles Bay and a tiny island covered in seagulls. We notice flags on jetties, which is how residents announce they are at home.
"Everyone loves their own part of the island," explains Dave. "Each bay is quite different and if it's windy on one side, it's calm on the other so there's always a safe haven."
Deciding it's too rough to get to Bostaquet Bay, where local legend has it that cannibal feasts took place, we find instead "the nicest beach on the island" according to Dave. It's called Viviene Bay and has 14 baches. Around 100 boats anchor here in summer and it takes about an hour to walk to it from the lodge -it's rough walking, with sudden drops - not recommended after a glass of wine in the evening to see the stars.
Next is Mansion House Bay, which is apparently always pretty calm and where a colony of shags welcomes us. Hundred-year-old pine trees all around the island are dying and float past in the water as well as dotting the edges, sinking into the sand.
Mansion House is postcard-pretty, but looks in need of some TLC. Gone is the golden age of steam excursions from Auckland or even the Phantom Fleet, a group of coastal traders who once sheltered from storms here.
Instead, families picnic on the lawn and children with floaties practise jumping off the jetty.
Back at the lodge we take a walk in Dave and Helen's beautiful garden filled with vegetables and the sweetest strawberries.
We leave their property and follow a pathway we hope leads us to a lookout point, past the rubble of old houses that have proved too difficult to remove by helicopter.
I'm glad our children are creating a path out of broken shells, just like Hansel and Gretel, in case we get lost - the markings are not too clear as we make our way past a row of olive trees and some purple thistles.
We eventually find our way back and watch from the deck as the sun goes down and a couple of kayakers paddle past. A few Lasers with colourful sails tack in the distance at the campground for school groups.
Helen is in the kitchen preparing dinner. She looks similar to my mother-in-law and the feel of the lodge is like staying with relatives you've never met before - there's a shared eating area and lounge and spare clothes in the wardrobe.
Over dinner, we meet a couple from Meadowbank on a night off from their two children, as well as a couple from Herne Bay with a baby daughter, on a wedding anniversary.
We all admire Helen's home-cooking - a fish bouillabaisse soup followed by fillet steak marinated in a ginger and mirin sauce served with crunchy potatoes, asparagus and a beetroot and fresh date salad.
As we leave the next morning, Dave shows us a part of the island where he likes working: "There's nobody there, just me," he says fondly.
While to some extent we shared our holiday with the other guests at the lodge, there was always a sense we could disappear into the bush and see no one at all if we wanted.
Which must be what people move to an island for, yet it still feels a very friendly place. The isolation means when they do bump into each other out on the water, or in whatever port around the world they have sailed to, they are always very pleased to see each other.
Kawau Lodge provides a full island experience including picking you up from the Sandspit wharf. A three-hour private boat tour of the island is $345. Accommodation, including breakfast, starts at $160 per night and dinner is just $30-$60 per person.
Stop off at the 60-year-old Kawau Island Yacht Club, it's one of the few bars right on the water in the whole of the Hauraki Coast. According to our host Dave, it has "very nice burgers" as well. Ph (09) 422 8845 for opening hours.
Danielle Wright was a guest of Kawau Lodge.