But that quickly turned to horror as one of the statues came free suddenly and fell into the crowd, hitting the man.
The Virginian Pilot reported the man was in his 30s and lost consciousness. He was taken to hospital.
Police chief Angela Greene said they quickly stepped in after the incident and told protesters to leave.
The Portsmouth monument sits on a site where there used to be a whipping post to punish slaves, the newspaper reports.
Protesters had spray painted slogans such as 'Black Lives Matter' over the monument and covered the statues in a bucket of yellow paint.
The Norfolk city mayor Kenny Alexander confirmed a person was seriously injured in an attempt to remove the statue.
He said anyone who attempted to scale another monument at Commercial Place, across the river from the Portsmouth monument, would be removed in the interests of their own safety.
The monument, which is 24m high, needs to be removed with a crane.
The council has voted to schedule a public hearing on its removal next month, intending to remove it by August 7.
Elsewhere in Virginia, protesters have toppled the statue of Confederate President Jefferson Davis along Richmond's famed Monument Avenue.
-with news.com.au