It involved two men being seen masturbating in view of others in November.
"That beach for 100 years has been known for [being nudist friendly]. But it's a giant leap to then have sex in public on the beach.
"You don't want to take your kids down there and find there's a bunch of blokes getting it off."
Such incidents hadn't been a regular occurrence in the past, Milne said.
"However, two complaints in four months is alarming and concerning to the board."
It'd be a shame if people were put off enjoying Ladies Bay because of the behaviour of those involved in Friday's incident, or what had occurred in November, New Zealand Naturist Federation marketing coordinator Alice Dewet said.
"You should be able to go to any beach and feel safe, whether you're clothed or not. If I was there I would've left as well.
"If that happened at a naturist club it wouldn't be tolerated."
Naturism - where people are nude in public or private spaces - was "not a sexual thing", Dewet said.
Instead, it was family friendly and about "exposing people to natural bodies, but not in a sexual way", she said.
"Nudity doesn't have to equal sex."
Police confirmed an officer went to Ladies Bay after report of a sexual incident yesterday.
The officer spoke to those involved and no charges had been laid, a police spokeswoman said.