The asylum seekers waved at the patrol plane but didn't appear to be in distress.
When HMAS Warramunga arrived in the area early on Thursday, it couldn't find the boat.
Its submerged hull wasn't spotted until Friday morning.
A Customs aircraft helping with the search spotted nine bodies in the water on Saturday afternoon.
Another four bodies were found later in the evening.
The bodies are yet to be recovered as the search for survivors continues.
Australian authorities didn't receive a distress call from the boat, Mr Clare said.
Border protection commander Rear Admiral David Johnston has acknowledged the personal toll the operations are taking on search and rescue teams.
"We are humans and the human dimensions of the circumstances are very difficult to deal with," he's told reporters.
Admiral Johnston says the search will continue and it's still possible that survivors can be found.
He says when the boat was first detected weather conditions were "quite favourable".
Opposition border protection spokesman Michael Keenan said the incident was another terrible human tragedy.
Australian Greens senator Sarah Hanson-Young said the loss of life highlighted the importance of offering asylum seekers safer pathways to Australia.
- AAP