Both sides played the last 10 minutes with 14 men. Paul Williams, outstanding again, was sent off after Heinrich Brussow made a meal of an open-hand slap. Referee Nigel Owens, who made few Samoan friends, was left with no choice after the ultimate sanction was recommended by assistant Stuart Terheege.
Afterwards Mahonri Schwalger described the card as "a little harsh" and accused Brussow of play-acting.
But the drama was not over. Replacement hooker Smit, on the field for a matter of minutes, was harshly yellow-carded for wasting time.
On last night's evidence, South Africa are not the team people feared after they thumped the hapless Fijians and over-matched Namibians. Samoa rattled them to the core.
The Boks lost all accuracy as the sheer physicality of the Samoans disturbed their equilibrium.
If Schwalger had not knocked-on with a try begging with 13 minutes to go and momentum in their favour, we might have been writing a Samoan fairytale rather than a lament.
It was all quite weird, really, seeing as the Springboks walked off at halftime in utter control. Their forwards were more cohesive, their set-piece destructive.
Bryan Habana got a slick try, Frans Steyn struck a 60-plus-metre penalty and Morne Steyn hit one from in front. At 13-0 with 30 minutes gone, a bath was on the cards.
Athletically the Samoans were fine in one-on-one encounters, but with a bevy of penalties and knock-ons, it kept coming back to the set-piece and they had no answer.
Nor did they have an answer to Owens. The punctilious Welshman can ref, but his need to demonstrate that fact can grate. His penalty against Samoa when they were pressing the South African line just 0-7 down wounded the Samoans; marching them 10m for querying the decision just rubbed the salt in.
"The ref was hard on us," claimed Schwalger. "South Africa were slowing our ball down. They've got to give us a chance. We felt it was one way, but we can't change the result."
They turned that anger into energy in the second half.
The loose forwards rampaged, while clever five-eighths Tusi Pisi and Eliota Fuimaono-Sapolu created. George Stowers scored and if Pisi had kicked the easy conversion, things would have got really spiky.
So barring a mathematical anomaly not even worth explaining, Samoa leave this World Cup feeling they were not given a fair shake.
South Africa came to Albany with three more days of rest up their sleeves than the underdogs, just as Wales had done in Hamilton.
In a five-team pool, three leave disappointed, two fight on. For Samoa, the sense of injustice remains acute.
South Africa: 13 (Bryan Habana try; Morne Steyn pen con, Frans Steyn pen).
Samoa: 5 (George Stowers try).
Halftime: 13-0.
