TV correspondent Michelle Tapper and her colleagues tried and failed to get on the privately-owned island on Wednesday morning ahead of the big event.
"I was met at the airport yesterday by half a dozen security guards," Tapper said in a video for Sunrise.
"He's told us we need to leave the waters but I'm not sure how legal that is. We were told yesterday that legally they could ask us to leave Hamilton island because it is privately owned," she said.
The island was purchased in 2003 by millionaire winemaker and sailing enthusiast Bob Oatley.
It is understood that Swift told organisers of the concert that no media would be allowed to attend, even as private guests of the sponsors.
But some have clearly snuck in to the island before the ban was imposed.
"I do know there are a number of journalists on the island," Tapper said during a live cross from a boat anchored near Qualia Resort.
According to some unimpressed holidaymakers the island has been inundated with paparazzi defying the ban.
"Taylor Swift has arrived on Hamilton Island = pause on our family holiday due to the paparazzi taking all the golf buggies!" tweeted Snezana.
Swift will play a stadium show in Brisbane on Saturday December 5.
-AAP