In the end they were all out for 128 in the 18th over, with Kerr picking up the final three wickets in the space of five balls while missing out on a hat-trick.
She finished with 3-30, while Jensen took 3-32, Jess Kerr took 2-11, and Frankie Mackay took 2-22.
New Zealand had four players unavailable for the fourth game of the series, with hand injuries for spinner Fran Jonas and batter Maddy Green, Lea Tahuhu was missing as a precaution after she left the field injured in the last ODI and batter Brooke Halliday is out for the remainder of the series due to being a close contact of a positive Covid-19 case.
Halliday dined outside with her partner, who subsequently tested positive for Covid-19, and was therefore required to isolate in her hotel room for seven days.
Halliday returned a negative Covid-19 test yesterday. She will be tested again on day five of her isolation and if she returns a second negative test, can finish her isolation period and rejoin the team on Friday.
Both Halliday and her partner have followed all Government and team health protocols.
Coach Bob Carter said the team was rallying around Halliday.
"This is obviously a frustrating time for Brooke," said Carter.
"But she's remaining upbeat, and as a team we're supporting her and checking in on her by phone etc.
"Of course, we're looking forward to her rejoining the team once she's met the appropriate protocols.
The fifth and final game of the series is in Queenstown on Thursday.