Mr Key told Q+A the fact that the State Services Commissioner had publicly supported his role in Mr Fletcher's appointment had not been properly reported by some media.
"My reputation matters to me because I am honest and I am upfront," Mr Key said.
"And I am also way more accessible than virtually any other leader in the world."
Mr Key said speculation that he would be replaced as the National Party leader before the next election was untrue.
He said he was "actually enjoying" the job.
"As Prime Minister, what are they going to remember when they look back? The answer is going to be is the economy stronger, does the education system work better, does the health system work better, is New Zealand a stronger, more confident country," Mr Key said.
"I've been Prime Minister for four-and-a-half years, my own personal view is that we are building that sort of New Zealand.
"Is there perfection? There'll never be perfection in politics but you can do your very best."
He said he would lead National to the next election "because I don't think we've actually finished the job yet".