Things got worse in Kleinveldt's next over when, with the score stuck on 27, Daniel Flynn went lbw to a marginal call - New Zealand referred it to the third umpire and it still looked borderline.
But that wasn't the end of the carnage.
At 39, captain Brendon McCullum, having grafted 97 minutes, drove loosely at left arm spinner Robin Peterson's second ball and edged to slip; then debutant Colin Munro prodded his first ball to short leg.
Bracewell survived the hat-trick but by stumps New Zealand's chances of surviving the test were all but gone.
Earlier, Guptill was dropped off Steyn, by Jacques Kallis at second slip in the first over of the innings but couldn't prosper from it, smartly caught by Alviro Petersen at third slip in Steyn's second over.
Williamson was softened up by a painful blow to the stomach by tall Morne Morkel before edging Steyn to Graeme Smith at first slip in the next over from Steyn.
Steyn's first spell of 6-2-8-2 was near irresistible and when big Kleinvelt got his turn he got rid of Brownlie and Flynn New Zealand were on the ropes.
Earlier, Faf du Plessis and Dean Elgar added centuries to Hashim Amla's first day ton.
It was du Plessis' second century in his fourth test and a composed, tidy performance in making 137.
Elgar's progress to his century was more tortured and dragged out beyond tea. He reached his century belting offspinner Jeetan Patel to the long on boundary.
His 103 not out was his maiden century in his third test.
Munro's two for 40 was a steady effort, while Bracewell ended with the best figures, three for 84.
