Ms Gillard last night congratulated Mr Rudd on his election as leader of the minority Labor government.
She said she was humbled by having had the privilege to be prime minister, and Australia's first female prime minister.
"When I first put myself forward for consideration as Labor leader in 2010 I had the overwhelming support of my colleagues to do so. I thank them for that,'' she said.
"And I thank them for giving the opportunity to me not only to serve the nation but to serve as the first female prime minister of this country.''
Ms Gillard said she had faced the "twin problems'' of a minority parliament and "internal division'' within Labor.
"It has not been an easy environment to work in.''
Ms Gillard urged Labor MPs and candidates to get out and win the election.
"Don't lack the guts, don't lack the fortitude, don't lack the resilience to go out there with our Labor agenda and to win this election,'' she said.
"I know that it can be done.''
Ms Gillard said there had been a lot of talk about her playing the "gender card''.
"The reaction to being the first female prime minister does not explain everything about my prime ministership, nor does it explain nothing about my prime ministership,'' she said.
"It explains some things and it is for the nation to think in a sophisticated way about those shades of grey."
"What I am absolutely confident of is it will be easier for the next woman and the woman after that ... and I am proud of that.''
The ballot took place three years and two days after Gillard ousted Rudd in a similar internal government showdown. It makes him leader of the party, but not prime minister.
'Sombre' mood during vote
Party official Chris Hayes says Gillard lost 57 votes to 45.
Labor whip Chris Hayes said both contestants spoke before the caucus vote.
"The mood was quite sombre,'' he said.
"Most people are happy that the issue of the leadership is now put behind us so the party can now concentrate on preparing for the September election.''
The meeting took almost an hour to get a result on the leadership because the votes were counted one by one.
"I think the prime minister wanted certainly to advise of her vision and the strength over her leadership in pursuing that vision for the nation,'' he said.
"Any of these challenges, I think, are particularly emotional.
"They're all human beings out there so please take that into account.''
No surprises after Bill Shorten backed Rudd
Media reports picked the result after Labor cabinet minister Bill Shorten backed Rudd over Gillard.
The vote began at 9pm (NZT) and various media reports said sources inside the caucus room had indicated Rudd had the numbers to win the day.
Mr Shorten said his decision may come at a personal cost to himself.
"It has weighed heavily on my mind in recent weeks,'' he said.
"I have now come to the view that Labor stands the best chance to defend the legacies of this term of government, and to continue improving the lives of millions of Australians, if Kevin Rudd is our leader.''
However, he was a great admirer of Ms Gillard and had supported her loyally for three years since she toppled Mr Rudd from the prime ministership in June 2010.
"I believe she has accomplished remarkable things,'' Mr Shorten said.
But the future of the nation and the Labor Party was at stake because of the Tony Abbott-led coalition.
Mr Abbott represented a "once-in-a-generation risk'' to Australia.
Mr Shorten said he knew some of his friends wouldn't support the choice he had made.
"It is my personal conviction that the best interests of the Australian nation and the Labor Party must come first, not debates about factions of personalities,'' he said.
"I believe that Kevin Rudd being elected leader tonight provides the best platform for Labor to be competitive at the next election.''
He said the achievements of the first and second terms of the current Labor government were many.
"It is these achievements that I want to fight for,'' he said.
"I also believe fundamentally Australians, regardless of their politics, want to see the Labor Party perform as strongly as it can at the next election.''
- AAP / AP
