Losing 5-58 in search of 245 did not seem possible with Ross Taylor compiling 82 from 91 in combination with Kane Williamson.
One by one they departed, though, leaving Mitchell Santner to ice the game in remarkable circumstances, with Lockie Ferguson at the other end.
Tension, there was plenty. Santner survived an lbw shout on two that looked plumb but umpire Paul Reiffel correctly judged to marginally pitch outside leg stump.
In truth, it should never have reached that point.
The somewhat daunting reality now is New Zealand's semifinal scrap has barely begun.
This World Cup, largely thanks to the compelling format, promises to be the most competitive, unpredictable of them all.
With much tougher tests and many twists and turns to come, these jitters do little to inspire confidence.
South Africa seem shot after three successive defeats but will surely front at some stage. Pakistan and the West Indies are capable of anything on their day. England, India, Australia will be there when it matters most – and New Zealand have them all to face.
Those contenders will cannibalise prospects.
In terms of opponents, no doubt the Black Caps have savoured the dream start. Sri Lanka and Bangladesh they know best.
On the basis of their two meek matches, Sri Lanka appear the second-worst team behind Afghanistan. And while Bangladesh stunned the Proteas and will pose others problems, the Tigers remain long, long odds to reach the final four.
Of course the Black Caps would rather be where they are than on the receiving end of a South African style vilification inquest, or even in England's shoes after they crumbled under home pressure against Pakistan.
Seven wins from nine games will be enough to make the semis but, equally, someone could yet sneak through with five. It could be that tight.
The Black Caps were always expected to be 3-0 by the end of their first week. After silencing the "Bangladesh, Bangladesh" roar in London and with Afghanistan next on Saturday, even on Taunton's turning track, barring a horror show that will now be the case.
Such a luxurious position cannot be taken for granted, nor wasted.
But New Zealand's time under the blowtorch nears, and only then will we know their true character and capabilities. Ultimately, where they sit in the grand scheme won't be revealed until they face India at Trent Bridge next week.
Two victories aside, concerns remain around their opening partnership, specifically Colin Munro after another squandered start, and whether Jimmy Neesham can be trusted with the ball.
The all-rounder conceded 24 runs from two overs, 17 from his second, to put the Black Caps under unnecessary pressure in the middle stages, though he hit a vital 25 from 33 balls.
Kane Williamson and Taylor, before settling to combine for their 106 run partnership, survived a major run out reprieve at 61/2 after wicketkeeper Mushfiqur Rahim broke the stumps with the New Zealand skipper well short after being caught ball watching.
This hesitancy is an ongoing worry, too.
Batting fragility was evident throughout the middle order after Williamson and Tom Latham went in the space of two runs, both falling to rash shots.
On a positive note the hat must be tipped to another bowling performance that set the platform for victory. The fielding, led by Martin Guptill and Santner, was also supreme.
On a slow surface that offered nothing in the way of movement, Williamson's decision to bowl first initially seemed amiss as Bangladesh cruised through the first nine overs.
Matt Henry struck first and finished with 4-47 after claiming two wickets in the final over. Boult chipped in with 2-44 but Ferguson was the clear standout, with Santner also offering control while varying his pace.
With black boots and his alpha moustache, Ferguson provides the menacing presence. Backing up his 3-22 from the Cardiff opener, Ferguson changed the game at the Oval. Figures of 1-40 from 10 overs don't reveal the extent of his influence.
He and Colin de Grandhomme strangled Bangladesh, with Ferguson's fast, short fireballs proving near unplayable. After four overs he had 1-7 to hint at the threat he could be in this tournament.
When the ball does not swing or seam, Ferguson may be main strike weapon.
Tim Southee and Henry Nicholls are now expected to recover from respective niggles to provide selection headaches.
For now at least, the Black Caps journey remains delicately poised.
