Mr Taylor-Hill said it happened so fast it was frightening.
He dived in, rescued the three and tied them to a life buoy while the other four lifeguards rushed in to help as well.
"I'll never, ever forget the look on the little girl's face, she was so terrified and shocked."
St John Ambulance was called and the girl was taken to hospital to be checked out.
"But they're all okay ... it was a terrible experience for them, just a few more seconds and they would have drowned," Mr Taylor-Hill said.
Castlecliff Beach was a popular destination with local children, Mr Gilmore said.
"Which is great ... but little kids, primary aged kids, that is, really need to be supervised by their parents. "Yes, I know we have the lifeguards, but again it's an open west coast beach which is very changeable every single day."
He said many parents drove into the carpark and let several young kids hop out and head down the beach on their own.
Mr Taylor-Hill said it scares him every day. "You see very young children trotting along the beach to where the flags are so they can go in and swim.
"It's great that they know not to swim outside the flags but not so great that they are so young and on their own - and I mean little kids, like 6, 7 and 8-year olds."
Mr Gilmore said it could be time to install a system to identify people dropping off young children unsupervised.
Setting up a security camera would probably be costly but would worth it, he said. "I'll be looking into it this year because, as I said, this is not a little calm safe beach here. It is an amazing beach but it is a surf beach and there are always holes and rips caused by the tides and the wind conditions."