"Theoretically I get this Christmas off."
The chairman of Wellington International Airport since 2013, Brown said he would temporarily remain on the board of the airport - two-thirds owned by Infratil - until the fund found a replacement or he found another role which "squeezed out" the directorship.
He told the Herald it was "time for some sort of change" but he did not have anything in particular lined up.
The change had been in planning before Morrison & Co chief executive Marko Bogoievski announced he was stepping down.
"To be honest I would have gone a year or so earlier at least, but Covid came along last year and that was extremely disruptive, and there were just quite a few things that needed to be done," Brown said.
Brown has, he said, basically only had two jobs in his life, previously working in capital markets for the National Bank.
Here, he encountered Lloyd Morrison, Morrison & Co's late founder, when he was attempting to put together a consortium to buy New Zealand Railways.
That deal ultimately failed and has been credited as the basis for Morrison looking to launch its own fund.
"I'd also been doing a lot of stuff with, in particular electricity companies, for the bank, so I could see this was going to be a very interesting space, and it was going to be way more interesting to be on the equity side than the debt side, so I was very attracted to what Infratil was, at that point, seeking to do."
After a brief break between jobs, he joined Morrison & Co in 1994, not long after it floated Infratil, which then had a market capitalisation of around $50m.
"It was also a good time to join, because they'd gone through all the pain in the arse stuff of setting it up."
The business had changed dramatically in the time since.
"It's really interesting to see a business which starts out with all of the sort of entrepreneurial flair and enthusiasm of Lloyd, plus all the stresses and strains, which probably killed him, and under Marko, kind of changes into a much more organised, reliable business which it has become."
Formed during a period of privatisations, Brown said the business model was, in simple terms, to buy as many shares as could be acquired in businesses such as electricity lines companies and wait to see what happened.
"It didn't take long for the penny to drop that the people owning those businesses really didn't have a clue. That was when the model changed to one where we realised we would be way better off, or the world would be better off, if we could actually get more involved with running the companies."
While many of the assets have long since been sold, one of the original investments, a cornerstone shareholding in Trustpower, remains. That initial investment - just part of the original $50m fund - has ballooned in value. Tilt Renewables, which was earlier this year sold for around $3b, was spun out of Trustpower.
One of Infratil's investments came close to killing Brown, who was struck by an NZ Bus vehicle in Wellington in July 2012, several months after Lloyd Morrison's death.
Brown was trapped under the vehicle for a time and suffered serious injuries, meaning he was off work for months. He later blamed himself for looking the wrong way when crossing the road. The accident followed a reconfiguration of road layout in central Wellington, causing a number of pedestrians to be hit.
He would later make a full recovery and the following year, was made chairman of Wellington Airport.
Brown said the experience made him wish he had spent more time running companies rather than advising.
"When you end up in a role like being a chair in a place like that, you suddenly see a whole new landscape of activities which you don't really see if you're distant at the next level down.
"I would probably say that I should have spent a little more time running businesses rather than being a technician so to speak, but there's also an element of personality so I was probably more effective at a technical side than the bossing people around side. But I'm glad I had that opportunity."
