There are lots of ways to make a little money in this world. Amazon Mechanical Turk, in an obscure corner of the behemoth's empire, offers a mystifying and maddening way to make very, very little money, writes Andy Newman.
The computer showed a photo of what looked like a school board meeting.
My job was to rate it on a scale of 1 to 5 for 23 different qualities: "patriotic," "elitist," "reassuring" and so on.
I did the same for a photo of a woman wearing headphones — I gave her a 4 for "competent" and a 1 for "threatening" — and another of five smiling women flanking a smiling man in a blue windbreaker.
I submitted my answers. I checked the clock. Three minutes had passed.
I had just earned another US5 cents (8c) on a digital work marketplace run by Amazon called Mechanical Turk. At least I thought I had. Weeks later, I'm still not sure.
There are lots of ways to make a little money in this world. Amazon Mechanical Turk, which since 2005 has flourished, to varying degrees, in an obscure corner of the globe-striding behemoth's empire, offers an uncertain, mystifying and often maddening way to make very, very little money.
On Mechanical Turk — named for a chess-playing "machine" from the 18th century that concealed a living chess master — scores of thousands of humans earn pennies or dollars doing tasks that computers cannot yet easily do.
It works like this: Employers, known as requesters, post batches of what are called Human Intelligence Tasks, or HITs, on Mechanical Turk's website. A task could be transcribing an invoice, or taking part in a study, or labelling photographs to train an artificial intelligence program. (Occasionally a photo shows something disturbing, like a beheading.)
Freelance workers, known informally as turkers, race to grab and do the tasks, providing what Amazon's founder, Jeff Bezos, once called "artificial artificial intelligence."
Most tasks pay a dime or less, and there is a daily churn of tasks that pay only a penny.
And the weird thing is that workers — mostly American workers — will do them, for many different reasons.
People turk to save for a motorcycle. They turk to buy insulin. They turk to pay off debt or pass the time profitably while on the clock at a boring job.
Some do it because there are few decent-paying jobs that can be done at will. People who are confined to their homes by disability or social anxiety or who live where there are few jobs do it because, despite lousy wages, it seems like the best option.
Plenty turk full time. In a 2016 Pew Research Center survey of nearly 3000 American turkers, a quarter said they made most or all of their earned income on the platform.
More than half the turkers surveyed said they earned under US$5 ($7.80) an hour.
Fighting for pennies
As little as turking appears to pay on paper, in practice it often pays less because MTurk, as it is known, is a sloppy, shoddy free-for-all.
Turkers spend their time fighting requesters over an unfair 10-cent rejection or a missing 60-cent payment. They waste minutes filling out bubbles on defective questionnaires that cannot be submitted. They abandon "10-minute" surveys after half an hour.
They swap horror stories and warnings on turker message boards ("rejection on a US$0.50 hit," read one recent bulletin, "reason is 'funds were not allocated'"). They leave scathing reviews on the turker-run site Turkopticon ("unfair and wild use of the rejection button").
Just how much turkers make is the subject of considerable scholarly debate, but one paper published last year analysed millions of tasks done by thousands of turkers.
Though they probably overrepresented novice turkers like me who do the lowest-paying tasks, the paper's authors concluded that if you count time spent looking for tasks and working on tasks that came to nothing, the median turker's hourly wage was US$1.77 ($2.76).
Only 4 per cent of turkers, the researchers found, made more than the federal minimum wage, US$7.25 an hour ($11.32).
Presiding over this production is the world's biggest tech company, feet firmly planted on the sidelines. Amazon usually declines to get involved when turkers say requesters rip them off, even as it lets requesters hide behind aliases that can make them impossible to track down.
It has ignored turkers' pleas to mandate higher wages, even as it takes a cut of each transaction ranging from 17 to 50 per cent; a requester posting a 1-cent HIT pays one penny to the turker and another to Amazon.
Amazon even finds ways to recoup some of the pennies turkers earn, a reminder of the days when miners were paid in scrip redeemable only at the company store.
While American turkers can get their wages direct-deposited, thousands of turkers overseas have only one way to get paid without incurring third-party fees: on an Amazon gift card.
While Amazon pays all of its American employees at least US$15 an hour ($23) and favours raising the minimum wage, it declined numerous requests to comment about the pay policy for turkers or anything else regarding Mechanical Turk. Minimum-wage laws generally do not apply to piecework jobs like turking.
Mechanical Turk is now one of a handful of big players in the field known as crowdwork or microwork. (One crowdwork company, Prolific, used by academic researchers, enforces a minimum wage: US$6.50 ($10.15) an hour.)
Crowdwork's proponents see a gleaming future — a borderless, no-overhead labour market where task-creator and task-doer meet at the intersection of supply and demand. Its critics see a throwback to something more Dickensian, where the lack of regulation and accountability keeps workers in the dark and on the defensive.
Mechanical Turk, in particular, combines the inconsistency and precariousness of gig work with Big Tech's tendency to dodge liability for the icky things that happen on its platforms.
"This is a great little microcosm of what happens when you take away any rules and the wages fall to the bottom," said Kristy Milland, a former turker turned labour activist who was one of the authors of the paper that analysed turkers' earnings.
Grind or Godsend
The bottom was where I dwelled during my brief tenure as a turker.
I tagged blurry, surveillance-looking photos of construction workers on job sites according to whether they were wearing hard hats or harnesses (US1c per photo).
I helped train a virtual paralegal by describing a hypothetical injury claim ("I suffered a serious injury because a defect in my lawn mower caused it to start") but earned nothing because the "submit" box never popped up.
I imagined myself as an "active investor" and rated the prospect of my supporting a Kickstarter for a device that makes nut milk "extremely likely."
Over the course of several weeks in September, I completed 221 HITs in a little over eight hours of dedicated turking, and earned a grand total of US$7.83 ($12.23). That works out to US97c ($1.50) an hour.
But hourly wages are not everything. Jane Lamont, a 30-year-old call centre worker in Louisville, quit her US$7.25-an-hour second job at McDonald's to turk.
She works from 7 to 11 on weeknights, five or six hours a day on weekends, and typically makes US$5 a day.
While the pay is "really low," she said, she prefers turking to fast food for its freedom — freedom from having to wear a uniform, freedom to spend time at home with her mother, freedom to watch videos between tasks.
Besides, she said, she would probably be online anyway — "I like being paid while just being leisurely." On weekends, she brings her laptop to her boyfriend's house. While he's wasting time gaming, she's making money.
"I love it," Lamont said. "I feel like that money I can spend on extra things that I want," like two trips to New York City last year.
Katie Boehm of Pittsburgh turned to turking in 2017 after her husband, who has diabetes, lost his job and insurance coverage. Her own health issues keep her from working outside the house, and turking seemed like a lifeline.
She turks at least 50 hours a week, sets herself a minimum goal of US$20 ($31) a day and usually makes US$30-$50.
Her husband's insulin costs US$1500 a month.
"MTurk covers about half of what he needs to survive," Boehm, 40, said. "Silly insulin."
'Pretty good pay'?
Amazon claims a turking workforce of half a million, but independent researchers say the number of active turkers is smaller.
An NYU data scientist who studies Mechanical Turk, Panos Ipeirotis, estimates that there are from 100,000 to 200,000 turkers, and that at any moment several thousand are doing tasks.
The vast majority of turkers are believed to be in the United States — at least three-quarters, researchers say — with India a distant second.
Mechanical Turk was created to solve an in-house problem. In 2001, looking for help weeding out duplicate product listings, Amazon applied for a patent for "a hybrid machine/human computing arrangement which advantageously involves humans to assist a computer to solve particular tasks."
The name was an homage to a contraption built by a Hungarian nobleman, featuring a bearded mannequin in a turban, that dazzled Europe with its chess-playing expertise. The moves were actually executed by a magnet-wielding human hidden beneath the board.
Mechanical Turk opened to the public in 2005 with considerable fanfare.
"Market forces will define how effective it is for requesters and how lucrative it is for workers," an Amazon executive, Peter Cohen, said at the time.
One requester I talked to has found MTurk quite effective. For a year, Ryan Schefke, the founder of a Texas company called Lead Liaison, has used turkers to transcribe business cards that salespeople collect at events. The contacts go into a database of sales prospects, a process known as "lead capture."
Multiple turkers transcribe the cards to yield 99 per cent accuracy, and the turnaround from the moment a customer scans a card is 10 to 12 minutes, Schefke said. A turker is paid US3 cents per card.
"Basically it's set it and forget it," he said. "It's awesome."
I mentioned that the one Lead Liaison task I did, transcribing a card covered in handwriting, took me 84 seconds, yielding an effective hourly wage of US$1.29 ($2), and asked whether that was an issue for him.
"I don't know why it would be relevant to us as a business," he said. "The pay is per task, not hourly, and if a turker did one card and then went off to pick cotton in the field, that's up to them. I actually think that's pretty good wages. If you look at people in the Philippines or some other countries, it's pretty good pay."
The minimum wage in Metro Manila is US$1.17 ($1.83).
Many major corporations have availed themselves of Mechanical Turk. The New York Times Co. has used turkers for at least three data projects.
MTurk also has big fans in the social sciences. Over 50,000 academic studies are conducted using MTurk each year, according to Leib Litman, a founder of CloudResearch, a company that helps researchers use the platform.
Turking is not always so low-paying. The most proficient turkers make upward of US$12 ($18.70) an hour, using computer scripts and free tools like HIT Catcher and HIT Forker to beat their fellow turkers to the sweetest HITs and racking up special qualifications that open the door to higher pay.
Litman said that a small cadre of super-turkers actually do most of the tasks on MTurk and estimated that the average wage for turkers who take part in studies run with CloudResearch at $6.50 an hour.
Top turkers' tricks
Amber Smoot is good at turking. She sits on the porch of her future in-laws' house in Middleburg, Florida, and runs HIT Forker in the background while she watches her dogs play.
When the script catches a task that pays a dollar or more, her computer blasts a snippet of operatic rock and she grabs the HIT and sets it aside until she has a bunch. "I'll queue 'em up and kick 'em all out," she said.
Smoot, 32, doesn't depend on MTurk to survive, but she knows that many people do. She spends a lot of time on message boards that offer tips to rookies.
"I love MTurk and I love the opportunities it gives people — it's great for what it is, a side gig," she said. "But the state of America is that people are turning to something that shouldn't be a job and trying to make a living off it."
Most turkers are relatively young — the majority who answered the Pew survey were under 32 years old — but some are older.
Jeff Archacki of Corvallis, Oregon, is 60 and a former IT worker and computer store owner. Since 2014, he has gotten by on disability payments for a hand injury, and turking.
He sits at the computer from 2am to noon, doing crossword puzzles while waiting for good HITs, and typically makes around US$40 ($62) a day. "I don't live a lavish lifestyle," he said, "but I'm not going to the food bank or anything."
A lack of accountability
Milland, the turker advocate, said the biggest problem on MTurk was that requesters could decline to pay turkers by "rejecting" their submissions, but still keep the work. "I've known a requester to be open about the fact that they automatically reject 10 per cent of jobs to pay for Amazon fees," she said.
Turkers have little recourse in dealing with shady requesters.
Often, they have no idea for whom they're working. Many requesters operate under generic, untraceable names like "Events" or "Panel" or "David" or "Josh."
The task where I rated the patriotic quotients of photographs was posted by an outfit called Vision. I graded eight sets for Vision. At US5 cents per set, I should have made US40 cents. But Vision shortchanged me US15 cents.
I clicked "Contact Requester" and asked Vision to email me, twice. I never heard back.
What was I to do? According to a Mechanical Turk help page, it wasn't Amazon's problem.
"If you have questions about the instructions in a particular HIT, completing a HIT, or why your HIT was rejected, contact the requester," it says. "Please note that requesters determine when your HITs are approved."
For another requester, I transcribed handwritten disability findings. One contained the patient's name, describing him as "fully disabled" and "unable to sit/stand for more than 90 minutes w/o pain."
Why was this presumably confidential patient information posted on Mechanical Turk, a platform open to anyone with access to an email address?
The requester, p9r, did not respond to my "Contact Requester" query.
(Amazon has made one recent effort toward transparency. Since July, it has listed requesters' approval rates — valuable information for a turker who is deciding whether to do a task.)
An unsettling parade
Another occupational hazard of turking is HITs that contain graphic imagery. A few years ago, Milland did some work describing photos circulated by the Islamic State.
"You have to digest the content of the image in order to come up with the keywords," she wrote in an email. "Things like 'orange jumpsuit,' 'caged prisoner,' 'prisoner on fire,' 'kneeling on explosives,' 'basket full of heads.'"
The job paid US10 cents a photo.
A few months ago, a turker who uses the screen name sprinkles123 did a HIT for a legal-services company that uses turkers as mock jurors. It involved a car-crash victim with multiple amputations and severe burns.
"The case had a lot of pictures," sprinkles123 said in a direct message. "I still think about it and wish I hadn't opened it."
Such HITs are rare and usually come with warnings, and a turker always has the option not to complete a HIT, but turkers tend to finish what they start.
Ipeirotis of NYU said his research indicated that Amazon itself was one of the biggest requesters and that it posted tasks under many different aliases. Amazon refused to say whether it posts tasks on Mechanical Turk.
On the US$200 billion Amazon scale of things, Mechanical Turk is a tiny operation. Still, Ipeirotis said he believed that more than US$100 million worth of tasks were done on Mechanical Turk annually, of which Amazon's take would be tens of millions.
Turking can be deadly tedious one minute, deliriously random the next. Recently on a Reddit forum, a turker named Mohammed Talukder posted something like a prose poem about the turking life:
"I don't even know why I'm doing it I think MTurk is making me crazy following dots with my eyes ... looking at kids starving ... asking me about hotel stays ... telling me to bring up past experiences of trauma … judging faces so so so many faces … watching video clips about people stealing … judging the calories for foods ... is this a cat or a dog?"
'Garbage island'
My last tasks for Mechanical Turk consisted of looking at paintings and coming up with 10 descriptors for each one: "portrait," "woman," "long hair," "archway." The job paid 1 cent per painting.
The requester was an Austrian print dealer, Meisterdrucke, that posts tens of thousands of artworks. I told Meisterdrucke's chief operating officer, Georg Petritsch, that it had taken me nine minutes and 15 seconds to do 10 of his HITs, for an hourly rate of US65 cents, and asked how he felt about paying people so little.
"It would be a lie to say that we find it fair," he wrote. "But just yesterday, I saw a documentary about the 'garbage island' on the Maldives, where people work in burning garbage, risking their health and lives, and get US$200 a month. And I guess that's exactly what this platform is for."
Written by: Andy Newman
© 2019 THE NEW YORK TIMES