"There is speculation - some of it is media speculating about media and journalists reporting the words of journalists - and yes, there is rumour-mongering and speculation. It's wasted breath."
But pressure is being applied to Gillard powerbroker Bill Shorten to swing his support to a challenge in the face of a tsunami of polls predicting the devastation of Labor at the election.
Polls indicate the carnage would spread beyond marginal electorates into safe Labor seats, dislodging a raft of ministers as well as backbenchers.
Polls have also consistently found that the Government's chances would improve significantly if Rudd was returned to power, keeping losses to a minimum if unable to retain office.
The crucial period will come when Parliament resumes next week. Advocates for dumping Gillard would need to turn Shorten or other powerful figures and engineer a mass exodus from the Prime Minister.
While nothing can be assumed in Labor's present turmoil, Gillard appears to have held the backing of Shorten, senior ministers and key union officials.
Shorten, widely seen as a future Labor leader, is crucial to Gillard's survival because of his high profile, his central role in Rudd's ousting, the influence he gained as former national secretary of the Australian Workers Union, and his continuing close ties with the union. If he turned against Gillard the symbolism of the defection would unleash a momentum that would almost certainly swamp her.
Gillard is confident of Shorten's continued backing. "Mr Shorten has indicated publicly that he is supporting me as I get about this important work for the nation," she said.
The influential national secretary of the Australian Workers Union, Paul Howes, remains a staunch Gillard supporter and warns that backroom whispering is undermining Labor's election prospects.
Tertiary Education and Trade Minister Craig Emerson said there was plenty of talk about a challenge, but there was no action, and Treasurer Wayne Swan said it was only "fevered speculation".