Since then, he's been in two Iron Man sequels and The Avengers, which have helped him become the highest paid actor in Hollywood - twice.
In 2014, Downey Jr topped the Forbes magazine's annual list of the highest paid actors for the second year in a row, with his estimated $75 million paycheck largely believed to have come from his work with Marvel.
However, while Downey Jr has signed onto two more Avengers sequels - one due out next year and the other in 2018 - he has not committed to any more Iron Man films at this stage.
The 49-year-old found the idea of 'letting' someone take over his role funny.
"I like that the idea is that it would be up to me, like I'm casting director for Marvel," he said.
It certainly would be a tough gig trying to find anyone to replace him.
What the teaser trailer for The Avengers: Age Of Ultron below:
* The Avengers: Age of Ultron releases in cinemas next year.
- AAP