"Everyone is telling us we need to be lower, everyone is telling us global growth is slowing" so the monetary policy committee decided to get ahead of the curve, Orr told Bloomberg from Jackson Hole in Wyoming where central bankers are gathered for their annual conference.
Mike Shirley, a dealer at Kiwibank, says Orr's comments moved the New Zealand dollar by a fifth to a quarter of a US cent.
"In an environment where very little else is going on, that move really stands out," Shirley says.
Orr said the New Zealand economy is in a very good position, as are the government's accounts, and the central bank would like to see businesses investing more but the global outlook is holding people back.
He also said that if other central bank keep cutting interest rates and the New Zealand currency moves higher as a result, that would be a problem for a small, open economy like New Zealand's.
Orr also said RBNZ stands ready to act and "we will do whatever it takes."
"It's nothing we didn't already know but just hearing it in that Jackson Hole environment from the big man himself" was enough to spark the small rally, Shirley says.
Whether the kiwi can rally further will depend very much on what Federal Reserve chair Jerome Powell says when he speaks overnight, New Zealand time, he says.
The markets will be looking for clues to the Fed's current thinking and to whether it's likely to cut rates as much as the markets have been anticipating.
Australia's Reserve Bank governor Philip Lowe is also scheduled to speak on Sunday morning. The RBA held its cash rate steady at its last meeting and, at 1 per cent, it's at the same level as the OCR.
The New Zealand dollar was trading at 94.55 Australian cents from 94.19, at 52.22 British pence from 51.94, at 57.69 euro cents from 57.42, at 68.06 yen from 67.72 and at 4.5294 Chinese yuan from 4.5064.
The New Zealand two-year swap rate edged up to a bid price of 0.9587 per cent from yesterday's close at 0.9285. The 10-year swap rate rose to 1.2775 per cent from 1.2300.
