Artficial intelligence: AI could worsen market crashes, increase fraud, Reserve Bank warns

One expert says the RBNZ's report doesn't go far enough, another says it overplays the risks. Image / Getty Creative

Artificial intelligence has the potential to “significantly improve the productivity and efficiency of financial institutions” and contribute to better modelling but it could also make market crashes worse, increase insurance company profit-seeking and increase cyber security, disinformation and fraud risks, according to a Reserve Bank report.

Rise of the machines: How could artificial intelligence impact financial stability?, released this morning, is a taster extract from the Reserve Bank’s (RBNZ) biannual Financial Stability Report, due on Wednesday.

“The increasing use of AI [artificial intelligence] has the potential to amplify herding behaviour in financial markets,” says the report, which is authored by RBNZ senior analyst and former Bank of England data scientist Matt Hankin.

“In this context, AI systems or their users may follow other AI systems instead of making independent decisions. This is a potential risk due to the concentration of AI providers, and could happen within trading, lending and insurance pricing.”

The report sees a danger of “tacit collusion ... where AI models independently learn to match or anticipate competitors’ pricing without direct co-ordination”.

“Herding behaviour through these channels could lead to market distortions and inefficiency. For example, through reducing liquidity and creating excess volatility. The amplification of procyclicality [exaggerating cycles or fluctuations] could make market crashes worse.”

Insurance - ‘short-term profit over ethics’

The RBNZ also paints a possible dark scenario in insurance.

“An insurance company could deploy an AI agent designed to maximise profits through reducing payouts when making underwriting decisions,” its report says.

“Over time, that AI may exploit loopholes in policy terms, reject claims unfairly and engage in anti-competitive practices.

“Without human oversight, AI could prioritise short-term profit over long-term reputational and ethical concerns.”

‘Poorly informed’

A spokesman for the Insurance Council of New Zealand (ICNZ) told the Herald: “[The] ICNZ has already reached out to the RBNZ as we think their commentary on the hypothetical example in the article is poorly informed.”

Insurers were committed to acting responsibly to deliver better outcomes for customers, the spokesman said. He noted the line in the report that said: “Anecdotally, uptake of AI tools has been quicker in the banking sector than in the insurance sector as insurers prefer tried-and-tested approaches.”

‘Black boxes’

Lack of transparency is identified as another pain point.

“GenAI models can be considered ‘black boxes’,” the report says.

“While their inputs and outputs are observable, the processes the models use to arrive at conclusions remain opaque.

“For example, most providers of LLMs [large language models like ChatGPT] do not publish their training data or full training methodology.”

Potential standards from 2028

Unlike the EU, New Zealand has no laws or regulations specific to AI.

But the RBNZ report sees the technology implicitly falling under two of its potential future standards.

“In New Zealand, starting in 2028, the proposed Risk Management Standard (RMS) and Operational Resilience Standard (ORS) will introduce principles-based requirements for deposit takers. These standards will contribute to mitigating the risks that can arise from the use of AI tools,” the report says.

“The RMS requires deposit takers to manage all material risks, including cybersecurity.

“The ORS sets out specific requirements to manage operational risks to ICT systems from both internal failures and external threats, including cyber risks.”

AI putting people out of jobs?

“Labour market disruption may occur due to AI. Shocks to the labour market could result in increased structural unemployment, with the potential for more credit and mortgage defaults,” the report warns.

“The creation of AI-complementary jobs may not keep pace with AI-driven unemployment.”

While it is likely that AI will cause labour market disruption, the severity of this is uncertain, the report says.

It notes a survey published in September 2024 by the AI Forum, a New Zealand community of AI users, investors and regulators that showed “only” 8% of surveyed organisations had experienced any AI-related job displacement at that time.

‘Left flat-footed’

The report has received a mixed reaction from AI experts.

“The Reserve Bank’s report is a wake-up call, but it doesn’t go nearly far enough,” AI New Zealand cofounder Justin Flitter said.

“Research from METR shows that the majority of roles in finance and beyond are already exposed to automation. With the capability of AI doubling every seven months, the pace of AI development is leaving our regulatory and business leaders flat-footed.

“If we keep treating AI as a slow-moving trend or a back-office experiment, we risk a collapse in competitiveness and ceding control of our financial future to offshore tech giants.

“The time for cautious pilots and incremental change is over. We need urgent, co-ordinated action with investment in AI skills, sovereign infrastructure and bold policy reform.”

Auckland University Associate Professor of Finance Gertjan Verdickt says there has already been algorithmic trading for decades – and that AI could help spot human errors and reduce human bias.

Report ‘overplays the negatives’

But Auckland University Associate Professor of Finance Gertjan Verdickt said: “In my opinion, they [the RBNZ] overplay the negatives.

“The report’s warnings of herd behaviour and increasing market volatility with the rise of AI may be overblown,” he said.

“We’ve already had quantitative computer models – that is, algorithmic and high-frequency trading – for the last 20 years. This would be another type of computerised trading model. This could not increase market volatility any more than we already have.

“On the contrary, it could help human traders if they can spot weaknesses in those computer models.”

There are way more positives than the report points out, Verdickt said.

“AI is better at digesting large documents, industry trends, and macroeconomic indicators than humans. So you could get more efficient markets.”

He added: “The report points toward biases and discrimination as a negative. I would say this is a positive rather than negative: there is a large literature showing that humans discriminate against women. An average AI could be more standardised and improve these services rather than worsen them.”

The RBNZ was a bit behind its peers such as the Federal Reserve and the European Central Bank, Verdickt said.

“But the only thing they need to do right now is ensure clear communication on AI usage. Consumers should also know who is using what. And make sure you don’t have AI-washing. That is, institutions saying that they do something just for marketing purposes.”

‘Considerable uncertainty’

“There is still considerable uncertainty around how AI will shape the financial system,” RBNZ financial stability assessment director Kerry Watt said.

“While its impact could be positive, especially in enhancing resilience, it could also introduce or amplify vulnerabilities.”

Chris Keall is an Auckland-based member of the Herald‘s business team. He joined the Herald in 2018 and is the technology editor and a senior business writer.