The line-up included Benee, Gin Wigmore, Ladi6, and The Beths.
Peachy Keen posted on the festival's Facebook page this afternoon they were extremely saddened to hear of the alleged assaults.
They said they never thought something like this would happen at Peachy Keen.
"We built this festival because of a lack of comfort at other festivals we have attended around the world.
"This sort of behaviour is completely unacceptable and we will continue to work as hard as we can to build events where everyone feels safe, secure and comfortable at; and enjoy their time while in our care."
Organisers said police responded within minutes of their call reporting the man.
They said their first priority was removing him from the festival to ensure the safety of eventgoers, followed by locating the people involved.
"With 4,500 people in a dark and loud environment we are not 100 per cent sure that we have identified everyone that was approached by this man."
Organisers said they have a photo of the man, but rather than posting the image online, they invited anyone who may have been in contact with him to get in touch with them directly.
"Supporting anyone who felt in any way uncomfortable at our event is very important to us and we want to help where we can."
They commended the actions of their security team who they said remained calm and detained the man despite being spat on, sworn at, and kicked.
Police said this afternoon they didn't have any further updates on the case.
Anyone who believes they have information that could assist police with ongoing inquiries is encouraged to contact them via 105 and quote event number P046047162.