The wicket gave Hazlewood figures of 3-5 and Cummins had 2-3 from their first three overs.
"There was pace and bounce, I was happy," said Cummins. "We wanted to hit our areas and fortunately they nicked some. It was one of those days where it all came together."
England's early collapse was even more unexpected given they were leading the series 3-0.
At 5-8, England were on track for one of the worst scores ever recorded in one-day cricket.
The incredible collapse had commentators reaching for the record books.
The score was the third lowest ever recorded in a one-dayer after the loss of the fifth wicket, beaten only by Sri Lanka in 2008 (5-6) and Canada in 2003 (5-7).
England's worst ODI score is 86 but at 5-8, there was concern they might not make it past the lowest ever total of 35, set by Zimbabwe against Sri Lanka in Harare in 2004.
But Moeen Ali and Eoin Morgan steadied the ship behind a 53-run partnership before Ali combined with Chris Woakes to partner up for 51 runs. Ali top-scored with 78 as England recovered to be all out for 196.
New Zealand's lowest ODI score is 64 against Pakistan in Sharjah in 1986.