"One of my boys said 'One of the Kiwis is giving you stick' to which I replied 'So what?'
Kearns said it was bizarre the focus was on him instead of New Zealand having two teams in the final but stood by his comments about Joubert.
"If you're having a good game as a referee you don't have 30,000 people chanting 'Joubert's a wanker'. And at the end of the game he got booed off the field. That would tend to indicate that he didn't have a good game," Kearns said.
"I think Joubert's performance overall was average. I thought there were a lot of inconsistencies. That decision on Jacques Potgieter, he only got half of it wrong. The bit he got right was the yellow card, that Potgieter made contact with the head and should have been yellow-carded. I don't think it was going to be a definite try. I think that hit to the head had very little impact on where the ball ended up. So I think he got that half right.
"He also put George Ayoub, the TMO, in a very difficult position the way he went ahead. He made his decision and then he asked George 'do you agree?'. George can't argue back. I know there were other people in the TMO's box who were bemused by how it happened.
"In the commentary position you just call it as you see it. That's how I saw it."
Kearns pointed out New Zealanders bagged Joubert after last year's final when the Waratahs edged the Crusaders after to a late penalty.
"All of New Zealand was up in arms when Richie McCaw was penalised and Foley kicked the goal. Joubert also did the World Cup final when the rest of the world were up in arms, and New Zealand thought he had a blinder."
Kearns also defended his commentary style and said he wasn't a cheerleader but was within his right to get excited if the Waratahs were doing well.
"When the Waratahs have done poorly I say they've done poorly and when they've done well. I say they've done well. It would be a bit ridiculous if I didn't get excited when the Waratahs played given I played for them for 12-13 years. It's the same when you hear Justin Marshall commentate and the Crusaders are doing well, you hear his voice go up an octave or two.
"Part of our job is to make it exciting and to have a little bit of controversy in there as well."
Kearns hoped viewers noticed the positive things he said about the Highlanders, who face the Hurricanes in Saturday's final.
"I commented various times how good [Lima] Sopoaga's kicking was, how good the Highlanders defensive lineout was, in fact how poor the Waratahs lineout was. What a great workman-like pack they are despite not having any superstars. Certainly [Waisake] Naholo, the magnificence of him and how brilliant Aaron Smith it. Certainly those guys got a rap.
"It just so happens that Australians think New Zealand and South Africa commentators are the most biased in the world, the South Africans think the Australian and Kiwi commentators are the most biased in the world and the Kiwis think the South African and Australian commentators are the most biased in the world. You can't win," Kearns added.
The good news for Highlanders fans is Kearns will be cheering the men from the deep south in Saturday's final.
"One of the things I love about the Highlanders is not only do they have a game plan, they actually play it. The greatest frustrations as a coach is to have a game plan, know how you want to play the opposition and then the team goes out there and don't execute it.
"Quietly I'm going to be cheering for the Highlanders because they're real underdogs but I think the Hurricanes are going to be hard to beat."
What social media said:
Even Australians weren't quick to come to Kearns' defence.
