After toiling away for 40 overs and getting Sri Lanka past the follow-on mark on day three, Thilan Samaraweera and Suraj Randiv failed to add to Sri Lanka's overnight score when Boult broke up their 97-run seventh-wicket partnership in the fourth over of the day.
Boult got a touch of away swing which squared Samaraweera up and forced a thick edge through to Martin Guptill at second slip. Samaraweera departed for 76 and became Boult's 100th first-class wicket in the process.
He made it 101 in his next over, trapping Suraj Randiv lbw for 39.
Southee completed his well deserved five-wicket bag when the out swing continued, getting the edge from Nuwan Kulasekara to Taylor at first slip for six. Boult made it four thanks to a brilliant one-handed diving catch by Kane Williamson at gully to dismiss Rangana Herath for five. Southee finished with 5-62 and Boult 4-42, taking 3-8 from his four overs on day four.
The bad batch of balls continued in New Zealand's second innings, with two replacements needed inside seven overs. Guptill and Brendon McCullum got through to 32 before Shaminda Eranga had Guptill edging to Dilshan at gully for 11.
McCullum got to 35 from 43 balls and looked in ominous form with seven boundaries before he out-stretched for a Herath delivery and was stumped.