In the coming weeks, Parliament is due to hold a no-confidence vote that theoretically could cut short Zuma's term, up in 2019. Experts say that outcome is unlikely, but the ANC's longer-term prospects appear to be dimming as Zuma's popularity declines.
"This is a major mobilisation that is inflicting a massive amount of damage on the ANC," said Susan Booysen, a professor at the University of the Witwatersrand who has written extensively about the party. "Even if the rallying call is around Zuma, it's an indictment of the ANC."
The ANC suffered historic losses in last year's municipal elections, but until recently, analysts suggested that the party had enough nationwide support to win the presidential election in 2019. Now, with Zuma stumbling and nostalgia-driven attachment to the ANC fading, some question that assumption.
"He has pissed on the graves of our forefathers and those who fought for our freedom," said John Moodey, the leader of the Democratic Alliance (DA) in Gauteng province. The DA is the most popular opposition party in South Africa.
More than two decades after apartheid ended, South Africa still struggles with poverty and inequality. For many, frustration with the Zuma Government is heightened by resentment that the country's rebirth as a democracy has brought them little economic benefit.
"They told us they were going to build us homes. They made so many promises, but we've seen nothing," said Mthobehi Sigagayi, 35, who had travelled to Pretoria from the Alexandra township, a development just across from Sandton, Johannesburg's modern business capital, that has battled poverty and crime.
Many South Africans saw the Cabinet reshuffle as a naked attempt to consolidate power, with Zuma surrounding himself with inexperienced, manipulable bureaucrats. But that scandal was not the President's first. Last year, he was ordered to reimburse the state after the top court ruled that he had spent public money on his private mansion.
Also last year, then-public protector Thulisile Madonsela alleged that South Africa's wealthy Gupta family had bribed public officials and driven policy under Zuma. Zuma disputed her report, but the opposition embraced its findings, branding the president "Zupta" because of his alleged ties to the family.
In Pretoria today, protesters carried signs reading "Zupta must go" and "Our country is not for sale". And opposition leaders made an effort to apply the lessons of the anti-apartheid struggle to their new political crusade.
"Apartheid was brought down by a united front," Floyd Shivambu, deputy president of the Economic Freedom Fighters (EFF) party, told the roaring crowd. "Now we must unite against Jacob Zuma."
The EFF and its leader, Julius Malema, have advocated a major redistribution of white-owned land. In February, Malema told his supporters: "Where you see a beautiful land, take it. It belongs to you."
In a statement last year, DA leaders said Malema's "radical and hateful statements" have "sought to divide South Africa". Yet today, Malema and his supporters marched alongside DA members, many of them white.
"We don't agree on many things," Moodey said, "but we agree that Zuma must go. The rest we can sort out later."