"This win tonight should signal a return of decency, integrity and humanity to the Australian parliament," she said.
The result will leave Morrison's Liberal-National Coalition with 75 MPs in the 150-seat House of Representatives, forcing it to rely on cross-bench MPs to avoid a no-confidence vote.
Morrison became leader in August after the Liberal Party ousted Malcolm Turnbull, the former prime minister, following a bitter internal battle.
The switch meant Australia - which has become known as the coup capital of the world - has had four prime ministers deposed by their party in the past eight years.
Voters in Wentworth, where Turnbull was the popular former member, vented their fury at the ballot box.
The Liberal party suffered a swing against it of 22 per cent, believed to be the worst ever in an Australian byelection.
The result places intense pressure on Morrison, whose party will grow nervous ahead of a general election, due early next year.
Admitting that "Liberals are angry", Morrison said it was a "tough day" but pledged to work with the cross-bench.
"Tonight is a night when we listen, learn and accept the blows," he said.
"The Liberal Party has paid a big price tonight for the events of several months ago."
But the result will also be seen as a judgment of Morrison's leadership. Campaign staff for Phelps said the biggest issue cited by voters was the coalition's reluctance to take action on climate change.
David Crowe, a political commentator, said the Liberal party would now face internal recriminations that would leave it further divided.
"This was a swift and savage message to Scott Morrison that puts him in a diabolical position in Parliament and sets him on course for catastrophe at the next election," he wrote in the Sydney Morning Herald.