Fulton's dismissal, slashing at a wide half-volley from Jerome Taylor, came after a failure in the first innings and the opener's days in a black cap could be numbered.
First innings centurion Kane Williamson then shouldered arms to a straightening delivery from Kemar Roach, leaving Latham (8no) and Sodhi (4no) to see their side through to stumps.
Earlier, after beginning the day on 19-0 in reply to New Zealand's 508-7 declared, Craig ended the West Indies' opening partnership by trapping Kieran Powell in front for 28.
The Otago off-spinner struck again just two balls later to remove Kirk Edwards for a duck and, the following over, things got worse for the hosts when Darren Bravo was also dismissed without scoring, the victim of a caught-and-bowled by Ish Sodhi.
The Kiwi spin duo left the Windies teetering at 97-3 at the lunch break, although dangerman Chris Gale was still at the crease, unbeaten on 59. The opener's stint didn't last much longer, however, denied a century in his 100th test when Southee had him caught behind for 64.
Southee followed up that key wicket by trapping Marlon Samuels for a duck, leaving the Windies trailing by more than 400 runs with only five first-innings wickets in hand.
And, after a brief period on consolidation between Shivnarine Chanderpaul (43no) and Denesh Ramdin (39), Southee struck again to remove the Windies' captain and reduce the home side to 176-6 at tea.
Craig made early inroads in the final session of the day, claiming the scalps of Kemar Roach and Sulieman Benn to finish with figures of 4-91 in his first run in the national side, while Southee took the final wicket to fall to record 4-19.