The Sweden Democrats, a party with roots in the white supremacist fringe, won 17.6 per cent and 63 seats, up from 12.9 per cent and 49 seats in the last election four years ago, the biggest gain by any party in Sweden's Parliament, the Riksdag.
Prime Minister Stefan Lofven, who brought the Social Democrats to power in 2014, said he intended to remain in the job.
However, Alliance leader Ulf Kristersson called on him to resign and claimed the right to form Sweden's next government.
Sounding sombre and firm, Lofven told his supporters the election presented "a situation that all responsible parties must deal with", adding that "a party with roots in Nazism" would "never ever offer anything responsible, but hatred".
"We have a moral responsibility. We must gather all good forces. We won't mourn, we will organise ourselves," he said.
Sweden democrats leader Jimmie Akesson's had been hoping for at least 20 per cent of the vote but told a party rally yesterday that the results was still a win.
"We will gain huge influence over what happens in Sweden during the coming weeks, months and years," Akesson told party colleagues.
He hopes his party, which wants Sweden to leave the European Union and freeze immigration, can play a decisive role in negotiations over forming a government.
Like other far-right parties in Europe, the Sweden Democrats worked to soften its neo-Nazi image in the lead-up to the election. The party symbol was switched from a flame thrower to a flower. Members known for making pro-Third Reich statements were pushed out.
It made its first mark in politics with municipal council seats in 2006, and since then slowly helped revise long-accepted social norms for what Swedes could say openly about foreigners and integration without being considered racist.
- AP