London, as we have all heard, has been taken over by foreign oligarchs who hardly ever visit their dwellings, pricing out regular hard-working intellectuals and dulling the city's vibrancy.

But if that's the case, Janan Ganesh writes in the Financial Times, why is it that London's population -- after decades of decline back when the city was so "captivatingly earthy" -- has just passed its all-time peak of 8.6 million?

Why indeed? The same goes for New York, also known for glassy new luxury condominium towers that stand empty most of the time while the owners earn their money in Moscow or Shenzhen and spend it in the Swiss Alps or on the French Riviera. After declining through the 1950s, 1960s and 1970s, the city's population is at an all-time high that's almost identical to London's: 8.5 million. On the island at the city's core, Manhattan, the population of 1.6 million may still be a long ways from the peak of 2.3 million set in 1910 -- families of 10 in one-bedroom tenement apartments are a bit less common these days -- but it too has been growing during the past decade.

Then there's San Francisco, where tech billionaires get more blame than foreigners for sending real estate prices skyrocketing. Despite those crazy prices and a generally development-unfriendly political environment, the city's population hit an all-time high of 852,469 in 2014.

The story is similar in the other U.S. cities that number among the top five investment targets for members of the Association of Foreign Investors in Real Estate. AFIRE members are for the most part commercial investors -- foreigners buying residential real estate also really like Florida and Arizona, according to the National Association of Realtors -- but their list seems like a good place to start. And sure enough, the population of all these cities (the cities proper, not the metropolitan areas) has been growing faster than the national average since 2010.

Now, there really are ghost buildings and even ghost neighborhoods in New York and London where high-end apartments owned by foreign zillionaires stand dark and vacant most of the time. As the New York Times documented last year, 57 percent of the apartments from East 56th Street to East 59th Street between Fifth Avenue and Park Avenue in Manhattan are vacant at least 10 months a year.

But that's six very expensive city blocks. New York has a lot more blocks than that. Another scary statistic, trotted out last year by New York Magazine, is:

"According to data compiled by the firm PropertyShark, since 2008, roughly 30 per cent of condo sales in large-scale Manhattan developments have been to purchasers who either listed an overseas address or bought through an entity like a limited- liability corporation, a tactic rarely employed by local homebuyers but favored by foreign investors."

Large-scale Manhattan condo developments, though, are just a small, if growing, part of New York City's housing stock. And yes, there are a lot more "ultra-high-net-worth households" around the world than there used to be, but they're still a tiny minority. It seems more likely that the main factor behind the sharp gains in real estate prices in New York and other hot cities is simply demand from people who actually want to live and work there.

Sure, it's a bit perverse to have rich people -- foreign or otherwise -- buying apartments and leaving them vacant in cities where there's already huge demand for housing. In New York, which now effectively subsidizes absentee owners by taxing income and commercial property heavily while giving residential property a break, changing the tax code to remove that subsidy would probably be a good idea. Other cities and countries have tried various policies to discourage absentees -- Australia just renewed a crackdown on foreign property owners this week.

On the whole, though, that so many foreigners want to buy real estate in your city is a good sign. If this real estate were to stay empty until the end of time, that wouldn't be so great. But that's not how things usually work. The Chinese buyers who have become the leading foreign acquirers of US real estate are for the most part not ultra-high-net-worthers but affluent people (the average purchase price paid by foreign buyers in the U.S. in the 12 months ended in March was US$499,600, according to the NAR) who are hedging their bets about their country's future. If things go badly in China, they or their kids may move in to that house in Orange County or condo in Vancouver. If things go swimmingly, they'll sell and bring their money back home. Real estate may last forever, but real estate investments usually don't. And when today's wave of foreign investment ends, we may actually miss it.

• Justin Fox is a Bloomberg View columnist writing about business.

- Bloomberg