"First things first, you can never right the wrongs. These are people from the 50s and 60s, for the most part who have suffered terribly at the hands of the state.
"Secondly, if you look at the best thing we can do, I think it is that process of settling directly with them, for the most part."
The Ministry of Social Development has settled 900 claims with victims so far.
Key said there was no guarantee more victims would come forward if an inquiry was held, and the victims may not be any better off.
"Public inquiries of themselves don't necessarily change the level of compensation."
An inquiry in the United Kingdom had been "a bit of a disaster" and "had not delivered for the victims", he said.
But despite his resistance to an inquiry, he repeatedly refused to rule one out.
Judge Carolyn Henwood chaired the Confidential Listening and Assistance Service panel that heard more than 1100 cases of people abused in state care between the 1950s and 1980s.
The panel wrapped up in June 2015.
Judge Henwood said last week the government ignored a number of recommendations made by herself and others.
These included that an independent inquiry be established to ascertain the extent of the abuse, investigate complaints and monitor the Ministry's care of children.
Social Development Minister Anne Tolley has ruled out an apology for victims.