"Leaning against the wind is especially required when investor confidence may be transforming into overexuberance - when recklessness starts to overtake sober assessments of risk," he said.
The NZX 50 gross index has gained over 10 per cent this year to close at an all-time high of 5253.9 on Friday, while 10 companies have listed on the local sharemarket so far in 2014 and more floats are expected later in the year.
"What we're not saying is 'the market's going to turn' because frankly, who knows?" Everett told the Business Herald after his speech. "When you've got markets at all-time highs you might find people chasing the top and following people in."
NZX chief executive Tim Bennett agreed that investors needed to stay cautious in the current environment, especially as the market became more diverse through new listings and companies with different risk profiles joined the exchange.
But he said the pulling of the proposed float of equipment rental firm Hirepool in June - after institutional investors baulked at its pricing - was an example of the market "correcting itself".
Everett, who joined the FMA in February, also told the conference that a "laser-like focus" needed to be applied to listed firms' continuous disclosure.
There had been some recent "hiccups" in that area, he said, notably with Lyttelton Port and software developer Gentrack.
The FMA would be paying particular attention to information listed firms were providing to financial analysts and not making publicly available.
Everett said there had been suggestions that Gentrack might have briefed analysts in greater depth than the public following the release of a profit downgrade last month, just five weeks after the company listed on the NZX.
Gentrack chairman John Clifford has said no material information was discussed during the briefing.