"I'm never going near him again," she said.
Ms Fenton later apologised after Sir Peter said he was "gutted" by her comments and pointed out he had also supported Helen Clark.
Mr Goff swiftly distanced himself from the MP's comments, saying he greatly admired Sir Peter who was entitled to express any opinion he wished.
Sir Peter had originally praised Mr Key when explaining why he had invited him to Sydney to watch the Warriors play in the NRL grand final against Manly.
It was at that game that the photo opportunity sprang up.
After a booking mix-up, six league fans found themselves in Sydney without tickets to the big match.
By chance one of the men had a card with Sir Peter's number on it.
"We phoned him on Saturday afternoon and thought it was a long shot," said former Halswell Hornets player Kris Muirhead.
"But it was the only shot we'd got and fortunately he was able to come through for us.
"We were complete strangers to him and he moved heaven and hell to get tickets. It shows what an amazing guy he is. We were really lucky."