Doubling their efforts, Whanganui worked into position for lock Jay Tora, another player in his first Heartland start, to score in the grandstand corner, and then first-five Sheldon Pakinga made a rapid blindside switch for a try.
Buller closed the gap to just a point when Iliesa Tora got clear on the onside and went under the posts for 20-19.
However, a mistake in their half gave Whanganui a last-minute chance and winger Alekesio Vakarorogo powered down the touchline for his 41st first-class try.
While fullback Adam Boult was not landing goal kicks, he made an excellent run and pass for winger Harry Symes to score in the 49th minute.
Whanganui then struck the decisive blow after a turnover at their tryline was kicked ahead by second-five Ethan Robinson, and a chasing Hemi regathered the ball to score beside the posts, with reserve first-five Dane Whale converting for 37-19.
Boult then scored a fine converted try when he cut through the line and beat several dive tackles.
Whanganui raised their half century as reserve lock Peter-Travis Hay-Horton took a quick tap and reached out in the tackle to plant the ball.
While they would have liked to keep a clean sheet in the second half for two weeks in a row, Buller got their bonus-point try right on full-time as Tora claimed a hat-trick.
In the absence of Jason Hamlin due to knee surgery, assistant coach Kim McNaught took the reins down south.
“We got away to a bit of a start, and it was just our skill set and missed tackles, especially against Buller, you let them back in the game,” McNaught said.
“Give them a sniff and they’ll score tries, and that’s what happened.
“Halftime we just said to the boys [it is] just mainly around our skill set and, if we can fix that and get into the wider channels, we’ll be right.
“It’s tough to come here and win – they haven’t got the flashest team but they got a lot of heart down here.”
Whanganui made a number of changes in key positions in both the back and forwards, so there was still a bit to do in terms of cohesion.
“Having Sheldon running the cutter, he’s a totally different player to Dane, and when we tried to win the ruck we got a bit loose and tried to do some individual stuff,” said McNaught.
“When Dane come on he settles the team down and drives us around the paddock pretty good.
“Mitai Hemi, he was unreal for us today – got his first start and scored a couple of tries.
“It was good to have Peter-Travis [Hay-Horton] back.
“It was good to get away early and get a bit of team bonding going for us, so roll on next week.”