The social network has constructed a vast infrastructure to keep toxic material off its platform. At the centre of it is Accenture, the blue-chip consulting firm.
In 2019, Julie Sweet, the newly appointed chief executive of global consulting firm Accenture, held a meeting with top managers. She had a question: Should Accenture get out of some of the work it was doing for a leading client, Facebook?
For years, tensions had mounted within Accenture over a certain task that it performed for the social network. In eight-hour shifts, thousands of its full-time employees and contractors were sorting through Facebook's most noxious posts, including images, videos and messages about suicides, beheadings and sexual acts, trying to prevent them from spreading online.
Some of those Accenture workers, who reviewed hundreds of Facebook posts in a shift, said they had started experiencing depression, anxiety and paranoia. In the United States, one worker had joined a class-action lawsuit to protest the working conditions. News coverage linked Accenture to the grisly work. So Sweet had ordered a review to discuss the growing ethical, legal and reputational risks.
At the meeting in Accenture's Washington office, she and Ellyn Shook, head of human resources, voiced concerns about the psychological toll of the work for Facebook and the damage to the firm's reputation, attendees said. Some executives who oversaw the Facebook account argued that the problems were manageable. They said the social network was too lucrative a client to lose.
The meeting ended with no resolution.
Facebook and Accenture have rarely talked about their arrangement or even acknowledged that they work with each other. But their secretive relationship lies at the heart of an effort by the world's largest social media company to distance itself from the most toxic part of its business.
For years, Facebook has been under scrutiny for the violent and hateful content that flows through its site. CEO Mark Zuckerberg has repeatedly pledged to clean up the platform. He has promoted the use of artificial intelligence to weed out toxic posts and touted efforts to hire thousands of workers to remove the messages that AI doesn't.
But behind the scenes, Facebook has quietly paid others to take on much of the responsibility. Since 2012, the company has hired at least 10 consulting and staffing firms globally to sift through its posts, along with a wider web of subcontractors, according to interviews and public records.
No company has been more crucial to that endeavour than Accenture. The Fortune 500 firm, better known for providing high-end tech, accounting and consulting services to multinational companies and governments, has become Facebook's single biggest partner in moderating content, according to an examination by The New York Times.
Accenture has taken on the work — and given it a veneer of respectability — because Facebook has signed contracts with it for content moderation and other services worth at least US$500 million ($710 million) a year, according to The Times' examination. Accenture employs more than a third of the 15,000 people whom Facebook has said it has hired to inspect its posts. And while the agreements provide only a small fraction of Accenture's annual revenue, they give it an important lifeline into Silicon Valley. Within Accenture, Facebook is known as a "diamond client."
Their contracts, which have not previously been reported, have redefined the traditional boundaries of an outsourcing relationship. Accenture has absorbed the worst facets of moderating content and made Facebook's content issues its own. As a cost of doing business, it has dealt with workers' mental health issues from reviewing the posts. It has grappled with labor activism when those workers pushed for more pay and benefits. And it has silently borne public scrutiny when they have spoken out against the work.
Those issues have been compounded by Facebook's demanding hiring targets and performance goals and so many shifts in its content policies that Accenture struggled to keep up, 15 current and former employees said. And when faced with legal action from moderators about the work, Accenture stayed quiet as Facebook argued that it was not liable because the workers belonged to Accenture and others.
"You couldn't have Facebook as we know it today without Accenture," said Cori Crider, a co-founder of Foxglove, a law firm that represents content moderators. "Enablers like Accenture, for eye-watering fees, have let Facebook hold the core human problem of its business at arm's length."
The Times interviewed more than 40 current and former Accenture and Facebook employees, labour lawyers and others about the companies' relationship, which also includes accounting and advertising work. Most spoke anonymously because of nondisclosure agreements and fear of reprisal. The Times also reviewed Facebook and Accenture documents, legal records and regulatory filings.
Facebook and Accenture declined to make executives available for comment. Drew Pusateri, a Facebook spokesperson, said the company was aware that content moderation "jobs can be difficult, which is why we work closely with our partners to constantly evaluate how to best support these teams."
Stacey Jones, an Accenture spokesperson, said the work was a public service that was "essential to protecting our society by keeping the internet safe."
Neither company mentioned the other by name.
Pornographic posts
Much of Facebook's work with Accenture traces back to a nudity problem.
In 2007, millions of users joined the social network every month — and many posted naked photos. A settlement that Facebook reached that year with Andrew Cuomo, who was New York's attorney general, required the company to take down pornographic posts flagged by users within 24 hours.
Facebook employees who policed content were soon overwhelmed by the volume of work, members of the team said. Sheryl Sandberg, the company's chief operating officer, and other executives pushed the team to find automated solutions for combing through the content, three of them said.
Facebook also began looking at outsourcing, they said. Outsourcing was cheaper than hiring people and provided tax and regulatory benefits, along with the flexibility to grow or shrink quickly in regions where the company did not have offices or language expertise. Sandberg helped champion the outsourcing idea, they said, and midlevel managers worked out the details.
By 2011, Facebook was working with oDesk, a service that recruited freelancers to review content. But in 2012, after news site Gawker reported that oDesk workers in Morocco and elsewhere were paid as little as US$1 per hour for the work, Facebook began seeking another partner.
Facebook landed on Accenture. Formerly known as Andersen Consulting, the firm had rebranded as Accenture in 2001 after a break with accounting firm Arthur Andersen. And it wanted to gain traction in Silicon Valley.
In 2010, Accenture scored an accounting contract with Facebook. By 2012, that had expanded to include a deal for moderating content, particularly outside the United States.
That year, Facebook sent employees to Manila, Philippines, and Warsaw, Poland, to train Accenture workers to sort through posts, two former Facebook employees involved with the trip said. Accenture's workers were taught to use a Facebook software system and the platform's guidelines for leaving content up, taking it down or escalating it for review.
'Honey Badger'
What started as a few dozen Accenture moderators grew rapidly.
By 2015, Accenture's office in the San Francisco Bay Area had set up a team, code-named Honey Badger, just for Facebook's needs, former employees said. Accenture went from providing about 300 workers in 2015 to about 3,000 in 2016. They are a mix of full-time employees and contractors, depending on the location and task.
The firm soon parlayed its work with Facebook into moderation contracts with YouTube, Twitter, Pinterest and others, executives said. (The digital content moderation industry is projected to reach US$8.8 billion ($12.4 billion) next year, according to Everest Group, roughly double the 2020 total.) Facebook also gave Accenture contracts in areas like checking for fake or duplicate user accounts and monitoring celebrity and brand accounts to ensure they were not flooded with abuse.
After federal authorities discovered in 2016 that Russian operatives had used Facebook to spread divisive posts to US voters for the presidential election, the company ramped up the number of moderators. It said it would hire more than 3,000 people — on top of the 4,500 it already had — to police the platform.
"If we're going to build a safe community, we need to respond quickly," Zuckerberg said in a 2017 post.
The next year, Facebook hired Arun Chandra, a former Hewlett Packard Enterprise executive, as vice president of scaled operations to help oversee the relationship with Accenture and others. His division is overseen by Sandberg.
Facebook also spread the content work to other firms, such as Cognizant and TaskUs. Facebook now provides a third of TaskUs' business, or US$150 million ($212 million) a year, according to regulatory filings.
The work was challenging. While more than 90 per cent of objectionable material that comes across Facebook and Instagram is removed by AI, outsourced workers must decide whether to leave up the posts that the AI doesn't catch.
They receive a performance score that is based on correctly reviewing posts against Facebook's policies. If they make mistakes more than 5 per cent of the time, they can be fired, Accenture employees said.
But Facebook's rules about what was acceptable changed constantly, causing confusion. When people used a gas station emoji as slang for selling marijuana, workers deleted the posts for violating the company's content policy on drugs. Facebook then told moderators not to remove the posts, before later reversing course.
Facebook also tweaked its moderation technology, adding new keyboard shortcuts to speed up the review process. But the updates were sometimes released with little warning, increasing errors.
As of May, Accenture billed Facebook for roughly 1,900 full-time moderators in Manila; 1,300 in Mumbai, India; 850 in Lisbon, Portugal; 780 in Kuala Lumpur, Malaysia; 300 in Warsaw; 300 in Mountain View, California; and 225 in Dublin, according to staffing records reviewed by The Times.
At the end of each month, Accenture sent invoices to Facebook detailing the hours worked by its moderators and the volume of content reviewed. Each US moderator generated US$50 ($70) or more per hour for Accenture, two people with knowledge of the billing said. In contrast, moderators in some US cities received starting pay of US$18 ($25) an hour.
Psychological costs
Within Accenture, workers began questioning the effects of viewing so many hateful posts.
Accenture hired mental health counsellors to handle the fallout. Izabela Dziugiel, a counsellor who worked in Accenture's Warsaw office, said she told managers in 2018 that they were hiring people ill-prepared to sort through the content. Her office handled posts from the Middle East, including gruesome images and videos of the Syrian war.
"They would just hire anybody," said Dziugiel, who previously treated soldiers with post-traumatic stress disorder. She left the firm in 2019.
In Dublin, one Accenture moderator who sifted through Facebook content left a suicide note on his desk in 2018, said a mental health counsellor who was involved in the episode. The worker was found safe.
Joshua Sklar, a moderator in Austin, Texas, who quit in April, said he had reviewed 500 to 700 posts a shift, including images of dead bodies after car crashes and videos of animals being tortured.
"One video that I watched was a guy who was filming himself raping a little girl," said Sklar, who described his experience in an internal post that later became public. "It was just awful."
If workers went around Accenture's chain of command and directly communicated with Facebook about content issues, they risked being reprimanded, he added. That made Facebook slower to learn about and react to problems, he said.
Facebook said anyone filtering content could escalate concerns.
Another former moderator in Austin, Spencer Darr, said in a legal hearing in June that the job had required him to make unimaginable decisions, such as whether to delete a video of a dog being skinned alive or simply mark it as disturbing. "Content moderators' job is an impossible one," he said.
In 2018, Accenture introduced WeCare — policies that mental health counsellors said limited their ability to treat workers. Their titles were changed to "wellness coaches" and they were instructed not to give psychological assessments or diagnoses, but to provide "short-term support" like taking walks or listening to calming music. The goal, according to a 2018 Accenture guidebook, was to teach moderators "how to respond to difficult situations and content."
Accenture's Jones said the company was "committed to helping our people who do this important work succeed both professionally and personally." Workers can see outside psychologists.
By 2019, scrutiny of the industry was growing. That year, Cognizant said it was exiting content moderation after tech site The Verge described the low pay and mental health effects of workers at an Arizona office. Cognizant said the decision would cost it at least US$240 million ($340 million) in revenue and lead to 6,000 job cuts.
Internal debate
More than one Accenture chief executive debated doing business with Facebook.
In 2017, Pierre Nanterme, Accenture's chief at the time, questioned the ethics of the work and whether it fit the firm's long-term strategy of providing services with high profit margins and technical expertise, three executives involved in the discussions said.
No actions were taken. Nanterme died of cancer in January 2019.
Five months later, Sweet, a longtime Accenture lawyer and executive, was named chief executive. She soon ordered the review of the moderation business, three former colleagues said.
Executives prepared reports and debated how the work compared with jobs like an ambulance driver. Consultants were sent to observe moderators and their managers.
The office in Austin, which had opened in 2017, was selected for an audit as part of Sweet's review. The city was also home to a Facebook office and had large populations of Spanish and Arabic speakers to read non-English posts. At its peak, Accenture's Austin office had about 300 moderators parsing through Facebook posts.
But some workers there became unhappy about the pay and viewing so much toxic content. Organising through text messages and internal message boards, they called for better wages and benefits. Some shared their stories with the media.
Last year, a worker in Austin was one of two from Accenture who joined a class-action suit against Facebook filed by US moderators. Facebook argued that it was not liable because the workers were employed by firms like Accenture, according to court records. After the judge in the case ruled against Facebook, the company reached a US$52 million ($73 million) settlement with the workers in May 2020.
For Sweet, the debate over the Facebook contracts stretched out over several meetings, former executives said. She subsequently made several changes.
In December 2019, Accenture created a two-page legal disclosure to inform moderators about the risks of the job. The work had "the potential to negatively impact your emotional or mental health," the document said.
Last October, Accenture went further. It listed content moderation for the first time as a risk factor in its annual report, saying it could leave the firm vulnerable to media scrutiny and legal trouble. Accenture also restricted new moderation clients, two people with knowledge of the policy shift said. Any new contracts required approval from senior management.
But Sweet also left some things untouched, they said.
Among them: the contracts with Facebook. Ultimately, the people said, the client was too valuable to walk away from.
This article originally appeared in The New York Times.
Written by: Adam Satariano and Mike Isaac
Photographs by: Greg Kahn, Jessica Chou, Lauren Withrow and Zuza Krajewska
© 2021 THE NEW YORK TIMES