Musk announced the project at the opening of new offices in Seattle for SpaceX, his rocket company that has a contract to supply the International Space Station.
The billionaire said he would have 60 people working on the space internet project initially and that could rise to 1000 in a few years. The project would also form the basis of a communications system with Mars if Musk is able to achieve his stated ambition of establishing a human colony there.
"It will be important for Mars to have a global communications network as well," he said.
"I think this needs to be done, and I don't see anyone else doing it."
Sir Richard Branson is already investing in a project called OneWeb which hopes to put up 648 micro satellites to provide high-speed internet and telephone services from space.
Branson said last week: "I don't think Elon can do a competing thing. There isn't space for another network, like there physically is not enough space. If Elon wants to get into this area the logical thing for him would be to tie up with us."
Speaking on US television, Branson said small satellites could now be built at a fraction of the cost of a decade ago. He said: "The quantity of satellites will enable us to drive the cost down to an incredibly competitive level.
"We can literally take off every three or four hours. Imagine the possibilities for the three billion people in hard-to-reach areas who are currently not connected."
Other big players in the race to provide worldwide internet from the sky include Google. Facebook founder Mark Zuckerberg is exploring giant drones, satellites and lasers as delivery methods.