Two brands of sunscreen are being recalled following tests that revealed their SPF rating was lower than claimed.
Consumer NZ is now calling for mandatory standards for the industry.
A Consumer NZ test found the Pure Blend Sunscreen SPF15++ in reality had an SPF rating of just 4.
The sunscreen, which was marketed as a natural product, would be pulled from shelves.
Consumer NZ chief executive Sue Chetwin said the same was applicable to Snowberry Family+ Sunscreen SPF30.
The product had been recalled after it was found it had an SPF of only 20 and failed to meet requirements for broad-spectrum protection.
Chetwin says neither of the products had actually been tested on people before going to market, despite the Australian-New Zealand standard requiring an assessment involving at least 10 volunteers.
The core issue was that New Zealand's sunscreen standards were voluntary, she said.
"While companies are encouraged to comply with the Australian-New Zealand standard, it's clear some don't," she said.
"Making the standard mandatory would help ensure consumers aren't misled about the protection a sunscreen provides."
The group says it will now be testing more sunscreens and publishing the results later in the year.