"There weren't many Maori in Palmerston North and it attracted a lot of interest from the neighbours when a group of Maori Battalion soldiers visited my grandparents' home and shook the house with a haka."
Young Potonga attended school for a brief time in Palmerston North but his family would return to Wanganui before the end of the war to live in Castlecliff, where he attended Aranui School.
"Castlecliff was a wild place in the 1940s. We roamed the beach and river, and we would fish off the wharf, where we caught a lot of herring and kahawai.
"It would be a feeding frenzy under the wharf because there were bits of offal in the water. No one cared about pollution then and there were around 20 outlets dispatching their waste material straight into the river."
He left school at 15 and worked as a shearer and scrubcutter, enjoying the seasonal work. "It did occur to me that I was working on land that had once belonged to my ancestors, and it rubbed me up the wrong way at times.
"When I got married, I decided it was time to get regular work and a trade, so I trained as a welder at Tongariro and got my trade certificate. I worked for Wanganui Boats and I loved the work because I have always loved being on the water, and we got to work on trawlers and we built the support boat for the America's Cup race here."
Mr Neilson spent a couple of years as a social worker in the 1980s, supporting young people who were before the Youth Court. "I had to give up, the way social welfare worked was very frustrating. It seemed they spent a lot of money on 'Bandaid' solutions instead of looking at causes."
One thing Mr Neilson has never given up on is the preservation of the Waipapa marae. As trust chairman, he worked tirelessly to have it moved from the floor of the Waitotara Valley, where it was flood damaged in 2004, to the hilltop.
"I want to preserve it so mokopuna of today can experience what I had as a child. We need to preserve marae as places of sanctuary for whanau to come back to.
"I have always dreamed of my early childhood and those warm, brown faces of the old people there. It was how things were before the urban drift of the 1950s."
Mr Neilson says the Swiss and British heritage of his Pakeha lineage, as well as his Nga Rauru whakapapa, gives him a strong sense of family.
"My pakeha grandparents were very warm, loving people too. I have been very fortunate to have those examples while growing up. It gave me my inspiration for what I do now."