Eventually - with the help of antidepressants and swimming - he clambered out of long-term depression.
In his memoir The Desert Swimmer, he writes about training for and swimming the English Channel - something only 20% of people who attempt it achieve - despite living on a farm 600km from the coast.
Working on a 60,000-head sheep station often afflicted by drought, Cullen tells RNZthat for a long time he drank alcohol to cope.
“It became the thing that I thought was helping me out at the time.
“It made me feel better. But I ended up drinking a lot of beer.”
“Self-imposed pressure” can be a heavy load for farmers, he says.
“You envisage what you want to achieve in the day or the week or the month, and if you’re not reaching that target...
“You can’t foresee what your future is going to look like, you don’t know if you’re going to be earning any money.
“So it’s quite difficult to navigate around.”
Pushing himself too hard, Cullen says he wasn’t present with his family and spent too much time dwelling on the past.
“I had an inability to be able to get rid of a lot of the rubbish that was in my head and had been there for a long time.”
In 2015, after he was diagnosed with depression, antidepressants were “life-changing” for Cullen and from that point, his life pivoted for the better.
But it was seeing his brother Lachlan - who lost a hand at age 2 in a farming accident - complete the Bondi to Bronte Ocean Swim that really got him on the new, healthier track.
“I was in awe, probably both of his ability to be able to do that. But secondly, the camaraderie, all his mates around him... And I thought, I’ve got to have a crack at that.”
Active from an early age, Cullen says he’s always been a runner and played a lot of AFL, but had never considered himself a swimmer.
Yet back home at the sheep station after watching Lachlan, he “pulled on the goggles and the budgies” and joined the local Broken Hill swim club - a 60km drive away.
In 2018, after seeing an ad about swimming the English Channel, Cullen decided he would train to take on the challenge in 2022.
While he loved the “physical exertion”, finding time to train in the water was difficult with the pool so far away.
Brendan Cullen describes his memoir as "a story of resilience, love, and the power of rediscovering yourself".
Sometimes he swam in the Menindee Lake system - “zero visibility, murky brown water, dead trees” - and on a tethered band in his pool at home.
If it was late, he’d drive to the local dam and shine his car headlights at the water.
“It was an interesting way to train, I must admit.”
To get accustomed to the freezing cold wearing just Speedos, he “pushed himself to the absolute limits” in a freezer full of cold water, and ate a lot to put on weight.
“I just gutsed myself with so much tucker, you know, like ice cream and carbohydrates and whatever I could get my mouth into.
“I started at 87 kilos, and by the time I was heading over to England, I was 97.”
Apart from the physical prep for a big ocean swim, having an incredible team is the most important thing, Cullen says, and this inculded his wife Jacinta, oldest daughter, and sister-in-law.
“Just swim until you hit something, meaning France,” his coach, Mike Gregory, said as Cullen entered the water.
After 17 hours, in which strong currents dragged him off-course and extended his journey to almost double the most direct route, the 52-year-old farmer felt his left hand scrape the sands of Cap Gris-Nez.
“I couldn’t believe it, it was an emotional point of my life.
“Mike just said, ‘Stand up, mate, and walk out’.
“So I stood up - I didn’t think I was going to be able to - and I walked out and got on the beach and put my arms up.
“They blew the horn, which meant that the swim had finished. I embraced my coach and just sobbed.
“Then I just had this thought, ‘Geez, I’ve got to go and get some momentos for the kids’.
“I started racing around on the beach looking for bloody pebbles of all things at three o’clock in the morning.”
Although Cullen says “there were warning signs everywhere” that he was severely depressed, for a long time, he just ignored them and kept ploughing on.
On average, it takes someone struggling with depression about eight years to put their hand up to ask for help, he says, which can be especially difficult for men because it requires vulnerability.
“But once you’ve done that, you’ll find that you attract people who want to help you, and vice versa, and that’s what I found.”
Cullen now speaks to farming communities around Australia about his story and how important swimming has been for his mental health.
“[Exercise] gives you natural endorphins, which you don’t get from alcohol, and that sort of coincides with nutrition, you know, all the fundamentals of having a clear mind...you know, fit body, fit mind.
“You don’t have to do a channel swim. You can just go and buy yourself a pair of sneakers and go for a walk and get out of your own head.
“I knew there was a lot of hurt out there.
“Once I exposed myself and I was diagnosed with depression, and I just wanted to change that, and I wanted to be a sounding board for those people that were struggling.”