Analysis by Geoffrey Fowler, WP technology columnist
Just when you thought we had hit rock bottom on all the ways the internet could snoop on us - no. We've sunk even lower.
There's a tactic spreading across the web named after treatment usually reserved for criminals: fingerprinting. At least a third of the 500 sites Americans visit most often use hidden code to run an identity check on your computer or phone.
Websites from CNN and Best Buy to porn site Xvideos and WebMD are dusting your digital fingerprints by collecting details about your device you can't easily hide.
It doesn't matter whether you turn on "private browsing" mode, clear tracker cookies or use a virtual private network. Some even use the fact you've flagged "do not track" in your browser as a way to fingerprint you.
They're doing it, I suspect, because more of us are taking steps to protect our data. Privacy is an arms race - and we are falling behind.
Fingerprinting happens when sites force your browser to hand over innocent-looking but largely unchanging technical information about your computer, such as the resolution of your screen, your operating system or the fonts you have installed. Combined, those details create a picture of your device as unique as the skin on your thumb.
Sites can use your digital fingerprint to know if you've visited before, create profiles of your behaviour or make ads follow you around. They can also use it to stop you from sharing a password, identify fraudsters and block harmful bots.
• Google vows greater user privacy, after decades of data collection
• Goodbye, Chrome: Why Google's web browser has become spy software
• Gehan Gunasekara: Reform of pre-internet Privacy Act must be a top priority
• Google finds cheap way out of multi-billion-dollar 'Wi-Spy' privacy lawsuit
Fingerprinting has been around for more than a decade but considered mostly a theoretical threat for you and me. Not any more.
I asked Patrick Jackson, chief technology officer of privacy software company Disconnect, to test for signs of fingerprinting on the 500 most popular websites used by Americans.
He revealed what these sites hide in their code and do on our computers that we don't get to see on our screens.
I'm naming names. Of the 183 likely fingerprinters Jackson identified between September 30 and October 8, I asked 30 of the most well-known to explain their behaviour. (See below for a list.)
Some claimed it was industry-standard to fingerprint.
Many said they didn't realise it was happening or never collected our data themselves, because they had let ad and data partners operate parts of their websites. After hearing from me, six sites said they would remove fingerprinting code, including four run by the US government.
It's happening on sites you wouldn't think would be so intrusive, including Thesaurus.com and AllRecipes.com - even security and privacy software maker Norton.com. Two porn sites didn't answer my questions, but Jackson suspects they're using it to track and tailor content to the people who view them in private-browsing modes that turn out to be not so private.
The Washington Post website fingerprints visitors when they've blocked cookies, which ought to be a signal visitors don't want to be tracked. In different ways, the Fox News and New York Times websites do it, too.
Fingerprinting isn't yet as widespread as cookies, those tiny files websites drop in your browser to track you. But it's concerning because it's much, much more aggressive.
"Fingerprinting is designed to be user-hostile," said Jackson.
"It even takes the fact that you don't want to be tracked as a parameter to make your fingerprint more unique."
Google, Apple and Mozilla, which make the world's most-used browsers, rarely agree on much, but they've all identified fingerprinting as a growing threat.
"Because fingerprinting is neither transparent nor under the user's control, it results in tracking that doesn't respect user choice," wrote Google's Chrome browser engineers in May.
What's at stake is a pretty fundamental attribute of the web: anonymity.
One of the original benefits of the internet is that anyone can express themselves and access information without fear. But to update an adage: Now on the internet, they definitely know you're a dog.
Why are some of the most well-known websites doing this? And what can we do to stop it? It's another tale of the tech industry putting its own concerns ahead of your privacy.
How they fingerprint you
Fingerprinting sites don't necessarily know you by name. But they're connecting the dots on information that could be just as valuable.
When you load a site, fingerprinting code starts asking your computer for things that aren't part of the usual process of drawing a page. Knowing what operating system you're running, what fonts you have installed or what your address is on your internal network makes you look different from other people visiting the site.
Some sites use as a signal whether people have turned on the "Do not track" flag in their browser. (That's not ironic; it's malicious.)
Many times, fingerprinting code will run the digital equivalent of a sonar test, sending out a signal just to see what comes back. Website code instructs your browser how to draw out text. The coding in it for fingerprinting can include words or icons that never show up on your screen, letting websites track minute differences in how each device responds. The Best Buy website used this invisible ink to write "F1n63r,Pr1n71n6!" - stand back and you might see it spells out "fingerprinting!"
Every site draws on different data points to build your fingerprint, which is part of what makes it so hard to stop. In his tests, which weren't definitive, Jackson just flagged the most suspicious behaviour.
Apps can fingerprint, too, using even more attributes available on phones and tablets.
Engineer Valentin Vasilyev helped take fingerprinting beyond academic research with free software called Fingerprint2.js. We found traces of it used on many websites. A demonstration on his site, which claims it has "99.5 percent identification accuracy," correctly spotted my browser a half-dozen times over a week.
Vasilyev told me fingerprinting just connects the dots on information browsers already make public - and he can't be held responsible for how people use it.
"By creating this product, I just showed everyone including browser vendors and researchers how it can be done," he said.
A "pro" version he sells "is mostly companies trying to protect themselves" from issues such as fraud, he said.
A digital strip search
It's true that not all fingerprinting is used for devious purposes. But it is the digital equivalent of airport security conducting strip searches of everyone. More effective? Perhaps. Good? No.
Chase, Wells Fargo, Airbnb, Best Buy, eBay and Marriott told me fingerprinting lets them bolster security, such as fighting attempts to use stolen credit cards or passwords. (A device looks suspicious if it attempts to try many different card numbers or logins.) Textbook firm Cengage said it was stopping piracy and tailoring content. The New York Times and Fox News said fingerprinting was helping identify automated bots that might interfere with site operation.
"We don't use fingerprinting to track our readers and have internal rules forbidding it," said Times spokeswoman Danielle Rhoades Ha.
"The simple act of producing a fingerprint is not aggressive; using it to target a user would be."
We discovered four federal agencies - the Internal Revenue Service, State Department, Citizenship and Immigration Services, and National Weather Service - had fingerprinting code as part of a customer satisfaction survey, to keep from repeatedly asking people to fill out the questionnaire.
After I reached out, they all said they would update their software to remove it. (The vendor that provides that software, Verint, said the code was added as part of a test and was "unutilised".)
Marketing appeared to be the largest use for fingerprinting among the sites Jackson identified.
Sites including Reddit and Thesaurus.com said it helps protect advertisers against fraud and sensitive content - all the while allowing a firm called DoubleVerify to probe details about the computers of millions of people. (The company didn't answer my questions about how it uses data, how long it holds on to it or how it protects it.)
Payroll firm ADP uses fingerprinting scripts from at least two ad-tech firms to support its marketing. "The data collected during this standard practice is anonymous, non-identifiable and aggregated," said spokeswoman Allyce Hackmann. Some claim the tech is anonymous because it identifies computers and phones rather than people's names.
Washington Post spokeswoman Molly Gannon said, "The Post is using industry-standard advertising systems to support our ad business and serve our users relevant ads."
Just because fingerprinting is becoming common doesn't make it right. Most sites don't expressly state they're fingerprinting in privacy policies, much less make it clear how they and their partners might use and share the data.
What's the big worry? It's hard to know how this snooping might be used to harm or exploit us. "Data collected today can be used against us today, tomorrow or even 10 years from now," says Jackson, who used to work for the National Security Agency. "Your browsing history, the apps you use and the data you give companies can lead to voter manipulation, targeted behavior modification, and further aids the mass surveillance of our activities on and offline."
At least a few sites understood fingerprinting was an ethical issue. After I contacted it, AccuWeather told its ad firms to cut it out.
So did Comcast, one of the country's largest media companies. When I reported we found its Xfinity.com site fingerprinting users, Comcast removed the code - and made the ad firm that had been collecting the data confirm it didn't store or share any of it.
"We don't use fingerprinting trackers on our website, and we don't permit our business partners and service providers to do so," said Comcast spokeswoman Jennifer Khoury.
"We'll be performing regular site scans to prevent this from happening and are putting in place additional review systems for our partners."
How to fight back
Fingerprinting isn't like other online snooping. We can't entirely stop it by blocking cookies or making other simple changes to our browsers. The tactics keep evolving.
The good news is that there are gradations of certainty in fingerprinting - not all devices and browsers are equally easy to detect.
Vasilyev, who created fingerprinting software, said it is still possible to make yourself hard to fingerprint by using software such as Tor. It's a privacy-first browser that goes to great lengths to make each user's device look the same - but only useful for highly technical people because it breaks common websites.
You can also get some protection from more consumer-friendly software.
Apple iPhones, iPads and Macs running the company's Safari browser are among the hardest to fingerprint. That is, in part, because Apple has a relatively limited product line and those devices tend to be standardised - so they look more similar to fingerprinting software (compared to the zillions of variations in Android phones and Windows laptops out there). It's a kind of online herd immunity.
Apple's Safari also has been tackling fingerprinting directly by reducing the amount of information it shares, such as a list of built-in fonts (instead of custom ones). Safari also asks you for permission before handing over information about your device orientation and motion, two more potential data points for fingerprinters. You don't have to adjust any settings to turn these protections on - they're the default.
However, most people in the world do not own Apple devices. Everyone else should consider the Firefox browser, which I've recommended before because of its aggressive default protection from tracker cookies.
It's in the final stages of adding some default fingerprinting protections, too, based around blocking traffic from known fingerprinting addresses - which, it acknowledges, fixes only part of the problem. You can turn on an early version of these protections now by going to the "Custom" tab under privacy and security settings.
Google's Chrome browser currently doesn't do much to stop fingerprinting by default. You can add browser privacy extensions such as uBlock Origin, the Electronic Frontier Foundation's Privacy Badger or Jackson's Disconnect to help stop some fingerprinting. But beware this software might break some of the sites you want to visit.
In May, Google promised it was going to join the fingerprinting fight - an important move because Chrome is by far the most-used browser. It says its plans include reducing the way browsers can be "passively" fingerprinted, so that it can detect and intervene against "active" fingerprinting efforts as they happen.
When these changes arrive on Chrome in the first half of 2020, they should make a difference. That is, until it's time for the next round of battle against the snoops.
(* = said it would stop)