Last week, Wilson cooked Mr Magill a lunch of steak, eggs and onions and showed him his garden. They spent four hours together, before Mr Magill hitchhiked back to Wanganui, which remains a pastime of his, despite his age.
The 88-year-old is bound to attract criticism for spending time with a man known publicly for serious sexual offending, including offences against children and bestiality, but then, he is used to going against the grain.
He was president of the YMCA in the 1970s when it ran the Downtown Y, a youth club in Napier which quickly became a place for disenfranchised Maori youth.
At first those who went to the teenage drop-in centre were mostly pakeha.
But after three years, 98 per cent of the kids there were Maori - it was the only place they could go for free, where it did not matter who they were or how they dressed.
Mr Magill remembers a concerned father asking him: "What are you doing, Pat? Are you turning the YMCA into a ghetto?"
It got him thinking about our nationhood, and how the past is often misunderstood.
"The effects of colonisation and urbanisation, and the sufferings of some people who are affected by it, are not understood, and some of it is inter-generational," he says.
Experiencing firsthand the positive community effects of the youth club, he was moved to join the International Community of Penal Abolitionists, which took him as far as Lagos, Dublin, and Oslo, studying methods of restorative justice.
Having met some of the world's foremost criminologists in more "enlightened" countries like Norway, he was struck in 2012 by controversial broadcaster Michael Laws' castigation of Wilson.
The convict had been released on parole with the most stringent release conditions ever imposed on a person in New Zealand. The conditions included that he would be required to live in a house on the grounds of the Whanganui Prison. His release nonetheless sparked public outrage from local citizens and the Wanganui City Council, which was fuelled by Mr Laws.
"I saw that former MP Michael Laws was castigating Murray as the Beast of Blenheim, and no one wanted him in Wanganui.
"I'd never seen a human beast," he jokes. "I went over there when he first moved into the house, to see how we handled it compared with Oslo in Norway. If someone like Murray was hooked in Oslo, the question would be, how did he slip through the cracks?"
On the prison grounds, Mr Magill met a man who was well aware of his label, all the time accompanied by his two minders as they walked in his garden.
"He's an avid reader - he knows everything. He's just a survivor."
Wilson has spent his life in institutions, including psychiatric hospitals. The Crown Health Planning Agency recently sent him a letter of apology for the way he was treated.
"Everywhere he's been, he's been abused. He's damaged. The longer you stay in prison, the worse you are really."
Mr Magill believes the answer to society's ills does not lie in the current penal system. His Scandinavian counterparts agree.
"The general opinion is [New Zealanders] are lovely people, but we're also a very young, retributive nation, that doesn't yet know its own history.
"We're only a young country and I think that given time we could be the social laboratory for good, for restorative justice, in the world."
I begin to understand why Mr Magill chose the John Robson Collection in the Napier Library as the setting for our interview.
It was justice reformer Dr Robson who abolished the death penalty in New Zealand in 1962.
Mr Magill wrote to him congratulating him on the reform, to which he received the sobering response:
"Beware the powers of darkness that hang over us at all times. Hard-won legislation is never set in concrete."
It was Dr Robson who nominated him for the OBE he received in 1978, which represented mainly his commitment to the YMCA.
"I said, 'I don't want that'. He said, it might make a difference to you."
Really, it has not made much difference, Mr Magill says. Even if he were a Sir, he would still cop flak for his work.
The library's collection includes items relating to criminal, social and restorative justice.
Having been developed as a community initiative by Napier Pilot City Trust, it relies on donations from the community and around the world.
In 1983 the Department of Internal Affairs funded a study which, three years later, resulted in Napier being designated - by then Labour Government Police and Social Welfare Minister Ann Hercus - a pilot city for the study and implementation of positive alternatives to violence.
Mr Magill himself has been the driving force behind the trust, busy door-knocking government agencies and community groups, for decades.
He is heartened by the community initiatives he's seen flourish in both Hastings and Napier over the years.
"Solutions are in your own town, not anywhere else. We've got to listen to the good that's there."
But there's a way to go yet.
"I've been sitting in the court every Wednesday since 1979, and it's still full of these kids.
"You'll see judges, lawyers, everybody, living off these kids. And what we know now, it needn't be that way."
A retired businessman and former Ranfurly Shield-era Hawke's Bay Rugby Union president, Mr Magill was also a Senior New Zealander of the Year finalist in 2012.
