He was asked for his response to a poll finding 57 per cent of people thought Parliament got it wrong by keeping the status quo on the drinking age.
He responded by saying he wasn't surprised by that result, which is why he had voted to raise the age to 20.
Mr Key first voted for the age to be split, so 18-year-olds could drink in bars but only those aged 20 or over could purchase alcohol at off-licences.
When that option failed, Mr Key voted for the age to stay at 18, instead of being raised to 20.
Mr Key cast his vote by proxy while he was at the Pacific Islands Forum in the Cook Islands.
Labour's deputy leader, Grant Robertson, said Mr Key either had "brain fades'' or he had not told the truth.
"I think New Zealanders deserve a bit better than that from the Prime Minister.''
Green Party co-leader Russel Norman said is appeared Mr Key "can't remember anything''.
He said it was almost like he had a "stunt double''.