"I doubt many people can say losing their dream job at the age of 22 was the best thing that ever happened to them," Mr Kelly said.
"For me, probation as an investment banker – and my subsequent firing – was a huge learning curve.
"I was a young guy, unsure what my options were. But I'd quickly worked out from this experience, and from the people I worked with, that I wanted to build a company."
After he was let go, Mr Kelly's father handed him two books to help him plan his next step.
They were Think and Grow Rich by Napoleon Hill, and Dale Carnegie's How to Win Friends And Influence People – which focused on building good relationships and achieving goals.
They gave him a "crazy idea" – to write his own book based on interviews with some of Australia's most successful and well known people, from former prime ministers to business leaders, musicians and artists.
"I thought I'd find people who had been successful, ask them what they did and model myself on that," he said.
"I wasn't the son of Kerry Packer or anything – I had never met people I'd call 'extreme achievers' before and I didn't grow up next to billionaires or genius scientists … but I had the crazy idea to contact the most interesting people in Australia and ask them to spend an hour with me.
"I made 5500 calls over three months and gently harassed people and 34 of them ended up speaking with me face-to-face."
One interviewee said yes but then stood him up two times before he secured the interview.
"The third time, he said he deliberately turned me away the first two times because he thought if I didn't keep coming back, I'd never publish a book," he said.
"It was a great lesson for a young guy. I'm genuinely relentless, and I'll keep turning up."
His book, Collective Wisdom: Prominent Australians On Success And The Future, ended up being a bestseller, but Mr Kelly always knew he wanted to start his own business.
So he went back to university and became a qualified tax agent before finding a job at an accounting firm.
But by 2006, he decided to take the lessons he'd learnt from Australia's highest achievers and start his own company – Kelly & Partners Chartered Accountants – from scratch.
Today, the ASX-listed firm turns over more than $47 million a year.
Over the years, Mr Kelly has also published several other books based on interviews with high-flyers, and his latest effort, Investment Wisdom, has just been released.
The father-of-three told news.com.au over the decades he had discovered many high achievers had some key traits in common, including the drive to be generous with their knowledge and "persistence, determination and relentlessness".
He said successful people had also often found something "much bigger than themselves" that they truly believed in, they worked "much harder than what people think is normal" and had incredibly high standards.
They were all traits he put to use while building his own empire.
"There's very little that can't be overcome with an amazing amount of work – when I started the firm I'd be heading to work at 3.30am, I'd work til 6.30pm, have dinner at 7pm and do more work from 8-11pm," he said.
"Being prepared to work 16-hour days is a very common trait among people who achieve their goals.
"Some people say you have to have a life, but I would say, why do you think your work is not your life? If you're doing work you really love and care about you're never really at work."
