In the podcast, Bridges said he did not want Orr reappointed when his five-year term expires in March 2023.
"I don't see anyone on the centre-right wanting to do that," he said. "This is a guy who ... ultimately ... has probably made things worse.
"His policy implementation has led to the housing problems; we've got the big inflation, and he's been focused on a whole bunch of other things that may or may not be important.
"But these are not the job of the Reserve Bank, whether it's Māori issues, climate change and the like."
The position of Reserve Bank governor is independent of the Government and monetary policy decisions are made by the Monetary Policy Committee.
RNZ contacted BusinessNZ, the New Zealand Banker's Association and a fund manager for comment on whether they support Adrian Orr.
Housing criticism 'a bit on the nose'
The CTU's director of policy and Finance Minister Grant Robertson's former economic advisor, Craig Renney, was complimentary of Orr's performance.
"The bank has done a really good job of the things that it's responsible for, financial stability is really strong in New Zealand, the bank has helped, along with fiscal policy, to manage the economy really well through Covid.
"I think Adrian has clearly been a really important part of that."
Renney said it was difficult to blame the RBNZ for the acceleration in house prices when the country had not been building enough houses for 30 years.
"I think it's a little bit on the nose to suggest he should be blamed for the housing crisis."
The RBNZ's basket of policies in response to the pandemic was similar to many other central banks in developed economies, including slashing the official cash rate, buying government bonds to keep downward pressure on interest rates, a business loans scheme, suspension of loan to value ratios, and support for retail banks to allow mortgage holidays.
Renney also said it was a good thing that the Reserve Bank wanted to work with the Māori and Pasifika communities and was thinking about the implications of climate change on the financial sector.
"The Reserve Bank is not doing anything unusual by international standards in that space."
Renney said the position of Reserve Bank governor is one of the most important politically independent roles in New Zealand.
He said it did not help financial markets or financial stability to have individuals suggesting that governors should stay or go.
Not-for-profit KiwiSaver provider Simplicity managing director Sam Stubbs said he thought Orr had done a "very, very good job" in trying circumstances.
"If you had said before Covid that the worst outcome will be [that] the economy will be trucking along, we'll have very strong employment and house prices would have gone up 25 per cent to 40 per cent - any Reserve Bank governor in history would have taken that."
He said he did not think there was any justifiable criticism of the RBNZ's work to become a bicultural institution because it was a reflection of what was already taking place at other public organisations.
"In terms of talking about climate change, aren't we all doing this?"
- RNZ
