The military officials said the attack may have been timed to target a planned meeting of top commanders but missed because the meeting had been delayed until later in the morning.
They also said that two army vehicles went missing from the complex last month, but they did not know whether those were used in Thursday's attack.
The Defence Ministry got a tip last week that a major attack was imminent in the capital, prompting authorities to reinforce army and security forces normally deployed in Sanaa, the officials said.
Thursday's blast badly damaged a military hospital inside the complex, wounding two Germans who were part of a visiting medical team. It also blew out windows and the doors of nearby homes and offices, as well as destroying an armoured vehicle and reducing three cars outside to charred hulks.
Military helicopters hovered over the aftermath as soldiers and ambulances arrived and gunfire echoed in the streets.
Hospital and military officials gave the death toll and said at least 40 people, most soldiers, also were wounded.
The Defence Ministry said "most" of the gunmen - who were armed with assault rifles and grenades - also were killed but did not give a number or more details.
President Abed Rabbo Mansour Hadi, who succeeded Saleh, later met with military commanders inside the devastated complex. He also ordered an investigation into the incident, military officials said.
Al-Qaida militants are concentrated in the southern and eastern parts of Yemen, but they occasionally strike in the capital.
Yemen is strategically located at the southwest corner of the Arabian Peninsula, bordering Oman and Saudi Arabia, two of Washington's closest Arab allies. Yemen has a shoreline on the Red Sea and the Arabian Sea close to the vital shipping lines carrying oil from the energy-rich Gulf region to the West.
- AP