Another woman yelled "oh my god … what the f***?" in disbelief.
Emergency services were called to the area near Buchan Point, Malabar, after receiving a report that a shark attacked a swimmer in the water just after 4.30pm on Wednesday.
Authorities then located human remains in the water and closed off Little Bay Beach, according to NSW Police.
"Footage clearly shows a body, half a body being taken by a shark," a police officer had told colleagues over a scanner. "They have found some remains."
A NSW Ambulance spokeswoman said: "Unfortunately this person had suffered catastrophic injuries and there wasn't a lot paramedics could do when we arrived."
A witness, who was fishing off rocks on the point, said the swimmer was right in front of him when they were dragged under the water.
"It was terrible. I am shaking," he told ABC. "I keep vomiting. It's very, very upsetting."
Other witnesses said they heard the swimmer yelling at the time of the attack.
"Some guy was swimming and a shark came and attacked him vertically," Kris Linto, a witness, told 9 News.
"We heard a yell and turned around it looked like a car had landed in the water, a big splash then the shark was chomping at the body and there was blood everywhere.
"It was really bad."
Linto claimed he saw a 4.5-metre great white shark attack the swimmer.
Randwick Council announced on Wednesday night that Malabar, Maroubra, Coogee, Clovelly and La Perouse beaches would also be closed for 24 hours while council lifeguards looked for any further sightings of sharks.
Dozens of people were swimming and paddle boarding at Little Bay Beach at the time of the attack.
It is the first fatality from a shark attack in Sydney since 1963.
Randwick Mayor Dylan Parker said the community was in shock.
"To lose someone to a shark attack like this is chilling. We are all in shock," he said.
"Our entire community's hearts go out to the family of the victim."
Police will liaise with the Department of Primary Industry to investigate the circumstances surrounding the death of the swimmer.
A report will be prepared for the coroner.
The attack on Wednesday comes after a 20-year-old woman was bitten by a great white shark on the other side of the country just over a week ago.
Jacquelin Morley was paddling in an inflatable pool ring offshore at Kelp Beds Beach in Esperance, Western Australia when the 3 metre shark sunk its teeth into her.
The young swimmer is lucky to be alive after the horror attack
She managed to swim to shore where locals tended to her before she was flown to hospital in Perth.
