Mr Abbott and Mr Key attended a gala dinner in Auckland last night as part of the Cricket World Cup which both countries are hosting.
Mr Key said he wanted New Zealand to be the home of the World Cups, plural, not just the Rugby World Cup.
Mr Abbott said there had been some iconic games between the two countries, one in particular the Boxing Day test of 1987. Sir Richard Hadlee had taken five wickets.
Mike Whitney, "the most unlikely batsman" held on and the match ended in a draw.
"In one of those classic gestures of sportsmanship between our two countries, the first man to congratulate Mike Whitney was Sir Richard Hadlee," Mr Abbott said.
"And then the New Zealand wicket keeper, Ian Smith, gave the ball, which rightly belonged to Hadlee ...to Mike Whitney.
"But grace begets grace and my understanding is that Mike Whitney has recently given that ball to Sir Richard Hadlee."
Mr Abbott had a formal meeting with Opposition Leader Andrew Little, as is customary for visiting dignitaries. They met in one of the spare VIP rooms at the cricket.
Mr Little took deputy leader Annette King with whom Mr Abbott remembered from their days as Health Ministers.
Mr Abbott was Health Minister in the John Howard Government from 2003 to 2007 while Ms King was Health Minister in Helen Clark's Government from 1999 to 2000.