Groups at the rally included members of the Triangle People's Assembly, Workers World Party, Industrial Workers of the World, Democratic Socialists of America and the anti-fascist movement, the Herald Sun reported.
"Charlottesville and racist monuments across the country are the result of centuries of white supremacy," Alissa Ellis, a member of Workers World Party Durham branch that was a participant in the Charlottesville protests, told the Herald Sun. Her group mobilised members on Facebook to attend the Durham event.
Protesters have targeted the Durham monument before. The statue was spray-painted with a message reading "Black lives matter" in 2015.
At the weekend, white nationalists, neo-Nazis and Ku Klux Klan members clashed with protesters at the University of Virginia in Charlottesville. The fringe groups gathered to protest the removal of a statue of Confederate Gen. Robert E. Lee.
Virginia Governor Terry McAuliffe said at a news conference on Sunday that he had a message for "all the white supremacists and the Nazis who came into Charlottesville today: Go home. You are not wanted in this great commonwealth".
During the rally, a car ploughed into the crowd, killing Heather Heyer, 32, and injuring 19 others. James Alex Fields, 20, the alleged driver of the vehicle, has been charged with murder, hit and run, and three counts of malicious wounding. A former teacher described Fields as a Nazi sympathiser.