Parole Board reports show Gray has paranoid delusions.
An initial written response was that Mr Key did not have time to meet residents but he was aware of the issues.
But as Mr Key left Kensington Stadium after watching a performance at the Secondary Schools Nationals, he told the Advocate the law is very clear that once Mr Gray had completed his time he had to be released.
"He's served his time and the law says we can't keep him there any longer."
The Government was looking at changing the law so those who were still deemed a risk after serving their full sentence could be kept inside, under certain circumstances.
He understood neighbours' anxiety over Mr Gray and expected the police and Department of Justice to monitor the situation and ensure the community's safety was paramount.
The man who extended the invitation was disappointed with the response.
"Obviously he was here socialising and pow-wow talking. Everyone in the neighbourhood is waiting for the ticking timebomb to go off.
"I've gone down every avenue I know and I've had no response."
Mr Key was in Northland yesterday as part of his regular regional visits. His tour took him to Kamo High School first, followed by a National Party lunch in central Whangarei then a visit to Kensington stadium to see those competing in the national secondary schools' kapa Haka competition.
Mr Key rounded out the day with a tour of local business Culham Engineering and a visit to Northland Club.
