His reputation took even more of a dive when he was filmed allegedly snorting cocaine in 2012 - allegations his management has steadfastly refused to comment on over the years.
Matthews has managed to stay in the spotlight over the years with stints on The Bachelor, The Jump and I'm A Celebrity: Get Me Out Of Here!, which filmed in Australia in 2015.
He only lasted two days filming in the jungle before producers became aware of his secret steroid addiction and booted him out of both I'm A Celebrity and Made In Chelsea.
But he relaunched his character this year on British challenge show The Jump, a wildly popular reality show in which celebrities take on winter sports challenges over 25 episodes.
Matthews was crowned overall winner of the fourth series of the show earlier this year, and he also seemed to have won in the love department, meeting his current girlfriend, Vogue Williams, during filming.
However Williams, 31, a model and reality star, has been black-listed from the society wedding of the year with British media reporting Pippa believed it was not necessary to invite her because the pair hadn't been dating long enough.
Matthews has made his £2.7 million (AU$4.7 million) fortune through his family's successful hotel business - his parents David and Jane Matthews own popular celebrity resort Eden Rock at St Barths - as well as his work as a PR manager and TV star.
He also penned a confronting autobiography in 2013 titled Confessions of a Chelsea Boy, in which he goes into details about a wild 'sixsome' he had with some 'nifty Canadian girls', as well as making the extreme claim he had slept with more than 1000 women and lost his virginity at 13.
Despite his wild ways and lothario reputation, Matthews has had his fair share of heartache.
His older brother Michael, who is also James' sibling, tragically died in 1999 climbing Mount Everest, hours after becoming the youngest Briton to conquer the peak at age 22.
Matthews and his family created the Michael Matthews Foundation, which provides education to children by constructing school buildings in remote areas.