Mr Key said he didn't expect the video to cause a stir.
He said he didn't think it showed the family were out of touch with everyday New Zealanders for taking a holiday in Hawaii.
"At the end of the day I am who I am. If I went out there and sold the house in Parnell and got rid of the place in Hawaii and said we are never going on holiday... people would say 'oh the guy's a fraud'."
He said he had always been up front about the fact of his wealth, and gave up a multi-million dollar salary to be Prime Minister.
"I don't think I'm the sort of the guy who flaunts things... I'm not a ridiculously lavish guy, but I've never made any sort of secret of the fact that we take the kids on holiday once a year... we're also a family and I think a lot of New Zealanders would say they want a Prime Minister that goes through the normal things," he said.
"Until the day I die, I will vigorously defend the things I've done as Prime Minister to support the less well off of New Zealand," he said.
He said he thought if he told Max and daughter Stephie to stop using social media they would listen to him, but he wouldn't want to do that.
Families of politicians were in the media "whether we like it or not", Mr Key said, but there was nothing in Max's video that was out of the ordinary.
"He was walking along the beach holding hands with his girlfriend, that's just the way it goes, isn't it?"