The Metropolitan Police said he was arrested "following a stop and search as part of an ongoing operation", and was held by armed officers from the specialist firearms command.
Scotland Yard said the man was arrested on suspicion of possession of an offensive weapon and preparing an act of terrorism.
"Knives have been recovered from him.
"He is being detained under the Terrorism Act and is in custody in a south London police station.
Counter terrorism detectives are continuing their investigation, but there was no immediate known threat.
Dressed in dark clothing, the suspect was wrestled to the ground on a traffic island, close entrance to Downing Street.
Handcuffed, with his arms behind his back, one of his hands appeared to be wrapped up in a bandage as four officers stood next to him.
Images from the scene showed three knives on the ground close to a brown rucksack, and an officer in forensic overalls could be seen taking photographs of the knives.
The suspect was detained at the scene for several minutes within view of journalists and passers by before he was eventually taken away in a police vehicle.
Prime Minister Theresa May was away from Number 10 on a campaign visit at the time of the incident, which did not disrupt work in Downing Street, said a spokeswoman.
- PA