The slide started with the loss of McCullum for 61 when he was caught in the deep when pulling Morne Morkel and he was quickly followed by Taylor (44), Kane Williamson (0), Daniel Vettori (0) and Doug Bracewell (0).
A big cheer went up around the ground at Seddon Park when Mark Gillespie edged through slips for four to move New Zealand off 133.
Gillespie approached his batting in much the same way he bowls. He was aggressive and unorthodox, regularly stepping outside his stumps and flailing the bat somewhere near the ball, and one shot even flew over fine leg that somehow came off the middle of the bat as he was looking to avoid a Vernon Philander bouncer.
He put on 43 for the eighth wicket with Kruger van Wyk - the second-highest partnership of the innings - and hit 27 in an entertaining 20-ball innings that included three fours and two sixes.
In truth, it doesn't look nearly enough against a powerful Proteas lineup.
The pitch has a little in it for the bowlers but there certainly weren't any of the sort of demons the scoreboard would suggest. It played true and should flatten out to be a good batting deck.
McCullum and Taylor made life look relatively easy despite the best efforts of a South African pace attack who bowled with venom as they dug the ball in short.
It was a testing time for the pair as Steyn and Morkel regularly topped 140kmh - Steyn bowled one delivery to Gillespie that was 152kmh - but they handled it well and took the initiative soon after a two-hour rain delay.
McCullum was particularly severe on anything remotely loose and smacked a six and four off consecutive Morkel deliveries as he brought up his second half-century of the series. He's in a good run of form (83, 48, 58* and 61) with the only criticism his inability to go on. It was brought into sharp focus as his teammates shuffled in and out of the pavilion.
It doesn't help when the openers don't provide any sort of platform - Rob Nicol was dismissed early for two and Martin Guptill (22) played on in similar fashion to the way he got out in Dunedin _ heaping plenty of pressure on McCullum and Taylor to score the bulk of the runs.
Philander continued his remarkable entry to test cricket, picking up another four wickets - the sixth time in 11 innings he has claimed for or more - and now has 39 wickets at 14.56.
New Zealand were given a sniff when they dismissed Smith, who was brilliantly caught by van Wyk off an inside edge, and nightwatchman Dale Steyn but will need to pick up early wickets tomorrow to give themselves a chance.
