"There must be a lot of people voting for change the other way because the polls tell you that. "
Mr Key also addressed criticism of the referendum's cost. He highlighted reports in Australia that a referendum on same-sex marriage there would cost A$520 million ($585 million).
"When people say $ 26 million is a lot - it is an awful lot of money and I appreciate that - but I think that it's a debate that's been worth having."
Mr Little said he had voted to retain the existing flag.
"I don't think the new flag that has been put up at all represents New Zealand," he said.