Turnbull appeared to chuckle only slightly at Alexander's gaffe, the Mail said.
Sky News presenters covering the outcome of the election suggested Alexander would surely apologise before too long - and his comments were quick to draw condemnation.
Alexander's victory against Labor's star candidate, former NSW premier Kristina Keneally, will come as a relief to Prime Minister Malcolm Turnbull. The Liberal Party will retain the seat with a margin of 9.7 per cent, despite a swing against Alexander of around 5 per cent.
"There is no sign of a swing larger than that or significantly larger than that which could endanger the Liberal Party holding the seat and John Alexander will be re-elected," ABC election analyst Antony Green said.
The by-election in the northwestern suburban electorate in Sydney was called after Alexander resigned from parliament after he suspected he was a dual citizen.
Saturday's by-election became a tight contest after Labor took up the fight for the seat by selecting former NSW premier Keneally, and throwing big names and resources into its campaign.
But the by-election has been less about the candidates, and more about the future of the Turnbull government and Malcolm Turnbull's leadership.
A Liberal loss would have left the government with 75 seats, meaning the coalition would no longer have had a majority in the House of Representatives.
Both Prime Minister Malcolm Turnbull and Opposition Leader Bill Shorten were at polling booths in the Sydney electorate.