A Twitter user posted video of what appeared to be Chicago police officers searching the interior of the hospital - and a blood trail on the hospital floor.
Witness James Gray told reporters that he saw a man and a woman who looked like they were walking towards the parking lot.
"The gentleman turned around and shot three times in the chest," Gray said.
"Once she fell to the ground, he turned around and shot her three more times."
Gray described the shooting as surreal, "like a movie scene". He said the shooter and the woman had been speaking before the shooting, but he didn't know if they knew each other.
"They were walking and talking and he just turned around and started shooting. It wasn't a heated exchange. It was just like we're talking now," he said.
The gunman then began firing randomly.
"Once he entered he just started shooting at random," Gray said.
"It looked like he was turning and pointing at people at random.
""It was chaos, mass chaos."
Nigary Thompson, who works at the hospital's family clinic, said she heard eight or nine shots. The clinic's employees and patients locked the doors and hid in the back of the clinic.
"I am scared as hell. I have never been so scared, I hear of shootings going on every day at people's workplaces, but not where I work at," she said.
"This was very too close to me. That could have been us back there and if any bullets had pierced the wall we all would have been hurt," Thompson said.
Police say "at least one possible offender is shot" and at least 10 ambulances were dispatched.
Officers were doing a "methodical search" of the hospital, a spokesman said.
A hospital employee said a notice came over Mercy's public address system telling everyone in the hospital to lock their doors,
The shots were fired about 3.28pm local time near 26th Street and Michigan streets.
A source in the Chicago Fire Department told NBC 5 the hospital had been partially evacuated.