But Szabo proved his first set performance wasn't a flash in the pan by battling back to 2-2 before Taylor stumbled over the line.
Taylor, the reigning champion, is traditionally a slow starter in this competition but was visibly rattled at the end of the first set.
He was 250-1 on to win the match with plenty of punters betting that he wouldn't drop a leg, let alone a set.
Szabo earned his place in a London by winning the New Zealand qualifiers in July.
After the match, Szabo spoke with reporters while clutching a dart board signed by Taylor.
Even though he was a set up against the darting champ, Szabo said he didn't anticipate a win.
"I thought he would do one of his nice comebacks - I really enjoyed the match.''
He said he was looking forward to bringing the signed board back to New Zealand as a symbol of what could have been, Sky Sports reported.
A visibly stunned Taylor said the New Zealander was playing darts he had never before seen in his life.
"He was very difficult to beat...he's not the normal player that we're used to.
"He was very steady and very methodical and he was dead slow.''
Szabo's mother, Maria told APNZ she hadn't expected her son to do so well.
"By playing with [Taylor], well that's winning,'' she said.
Szabo's partner Carolyn Johnson said she was "just so proud'' of him.
"He's amazing, he's just done brilliantly.''
She and her step-daughter Natasha were "pacing the house'' during the match, but was confident if he could "keep his head'' he would do well.
Szavo started paying darts when he was 22-years-old when he was asked to play in a 4th division team for the Workingman's Club at Wellington Darts Association.
He started playing 1st division for the last four years.