One of the dead was named as Dave Shapiro, a music agent who founded San Diego-based Sound Talent Group (STG).
The company said two other members of staff who were aboard the plane had also died.
“We are devastated by the loss of our co-founder, colleagues and friends,” an STG spokesperson told US media.
“Our hearts go out to their families and to everyone impacted by today’s tragedy.”
There was no official confirmation of the death toll, but first responders at the scene said the plane – versions of which can carry up to 10 people, including the pilots – had been totally destroyed and they expected the toll to rise.
‘Engulfed in flames’
Yasmine Sierra told AFP how she had helped her neighbours escape their burning house in the middle of the night after being woken by what she initially thought was an earthquake.
“It looked like all the homes were on fire because I could see the smoke and the flames, it looked like the trees were on fire,” she said.
Moments later, she heard screams from her neighbours who were trapped in their back garden.
“Me and my son grabbed the ladder, we jumped on our trampoline, and we tried to bounce as much as we possibly could, to throw that ladder over so that they can climb onto the ladder into our backyard,” said Sierra, 35.
A woman, two children and two small dogs climbed to safety over the ladder.
“She was very distraught when she came over. I brought her to the front of the house, and I told her that, you know, we needed to leave.”
Jeremy Serna, 31, who is in the Navy, said he and his wife had been awoken by a loud bang.
“We looked outside, and the sky was orange. And then I came running outside to see what it was, and everything was on fire over here,” he told AFP.
“I saw the corner house was just engulfed in flames. And then came back over here and told my wife, hey, we have to get out of here.
“It was pretty scary.”
Footage of the immediate aftermath of the crash showed a line of burning fuel linking cars that were entirely in flames.
Thick fog
Investigators were combing the scene on Thursday, picking through the scattered debris of the plane, which appeared to have broken into hundreds of pieces.
Bits of fibreglass were scattered among the twisted and charred remains of cars, and the smell of fuel hung in the air.
The accident happened in thick fog when the plane, which had come from Kansas, was nearing the Montgomery-Gibbs Executive Airport.
It was not immediately clear what had happened, but Eddy said a nearby power line appeared to have been clipped.
The plane, which can carry up to eight people, went down around 3.45am, according to the Federal Aviation Administration, striking the Murphy Canyon neighbourhood.
The residential area it hit is largely military housing. San Diego is home to US Navy facilities, Marine Corps bases and Coast Guard stations.
The accident came at a time of heightened tension in the skies above America, with a number of incidents rocking public confidence in flying.
Air traffic control outages have struck the busy Newark airport on the East Coast at least twice in recent weeks, and in January there was a mid-air collision over Washington between a passenger plane and a military helicopter.
This month two people died when their small plane crashed into a residential neighbourhood northwest of Los Angeles.
-Agence France-Presse