Fullback Vaughen Isaacs scored in the seventh minute for South Africa, who made ground up the middle with their forwards, then found space for Isaacs to score.
Twelve minutes later JJ van der Mescht stole a ball from a New Zealand lineout and displayed tremendous speed for a lock to run 50 metres to score.
South Africa led 19-3 at halftime.
Five minutes into the second half New Zealand were awarded a penalty try when James Molentze tackled flanker Kaylum Boshier while he was attempting to ground a grubber kicked in goal.
Molentze was shown a yellow card for his actions and the Baby Blacks were back in the game.
However, the Kiwis weren't able to score again with a numerical advantage and in the 57th minute found themselves camped on their goal line with South Africa throwing everything at them.
The pressure from South Africa was relentless, but New Zealand were given a reprieve when an error eventually came as a Junior Boks player went for the line.
But even though South Africa weren't able to score, the pressure was building on New Zealand to at least pick up that vital bonus point.
They got close to the line in the 64th minute before openside flanker Kohan Herbert threw a poor inside pass which was knocked on.
Mistakes came from both sides as the minutes ticked by, but in the 70th minute the Baby Blacks got monstered in a scrum on their 22m line and Sanele Nohamba kicked a penalty from it to go up 22-10.
It was desperate times for Philpott's troops who looked to get the breakthrough with their forwards.
That didn't work, but when the ball was eventually thrown by centre Billy Proctor to winger Lalomilo Lamolilo there was enough room for him to score. While Fergus Burke kicked the conversion, New Zealand conceded another penalty soon after, which Nohamba slotted with two minutes remaining.
The final act came when Etene Nanai-Seturo knocked on when New Zealand were in panic mode.
Kereru-Symes admitted his lads struggled to compete with the South Africans in the first half and found themselves playing catch up.
"The South African boys brought the pressure that we knew they were going to bring. We didn't adapt properly to it until the last few minutes of the first half, which was a bit too late to get into the game. A well deserved win for them and all credit."
Kereru-Symes said the word at halftime was for the team to go back to keeping things simple and while it worked to some extent, they still couldn't handle the South African forward pack.
"Our main message was to stick to our processes and take our time."
"In the first half we were rushing our plays a bit and that's why we were put under pressure quite a lot.
"But we did that, we took our time and we got some good results out of it, but couldn't quite execute."
