At the start of the lunch interval light rain began to fall and it persisted for most of the afternoon as a disappointed crowd eventually filtered out of the ground.
The covers came off at one point and the umpires said play would resume at 3.50pm, after tea, but the rain returned shortly after their announcement and a resumption was delayed further.
The covers were again partly removed around 4.30pm and the umpires confirmed that play would resume at 5.10pm.
But only six overs were possible before the rain returned and Williamson finished the day not out on 55, while Taylor has 41 next to his name.
New Zealand will start day five at the earlier time of 10am tomorrow in a good position to draw the test providing they can bat for at least one-and-a-half sessions.
More rain is forecast for tomorrow so neither side may have the final say but if the weather does clear, England will be in need of early wickets to force the game along.
The series is level 0-0 after the first test in Dunedin last week resulted in a draw.
The third and final test will be played at Eden Park in Auckland this week.