NZ Herald
  • Home
  • Latest news
  • Herald NOW
  • Video
  • New Zealand
  • Sport
  • World
  • Business
  • Entertainment
  • Podcasts
  • Quizzes
  • Opinion
  • Lifestyle
  • Travel
  • Viva
  • Weather

Subscriptions

  • Herald Premium
  • Viva Premium
  • The Listener
  • BusinessDesk

Sections

  • Latest news
  • New Zealand
    • All New Zealand
    • Crime
    • Politics
    • Education
    • Open Justice
    • Scam Update
  • Herald NOW
  • On The Up
  • World
    • All World
    • Australia
    • Asia
    • UK
    • United States
    • Middle East
    • Europe
    • Pacific
  • Business
    • All Business
    • MarketsSharesCurrencyCommoditiesStock TakesCrypto
    • Markets with Madison
    • Media Insider
    • Business analysis
    • Personal financeKiwiSaverInterest ratesTaxInvestment
    • EconomyInflationGDPOfficial cash rateEmployment
    • Small business
    • Business reportsMood of the BoardroomProject AucklandSustainable business and financeCapital markets reportAgribusiness reportInfrastructure reportDynamic business
    • Deloitte Top 200 Awards
    • CompaniesAged CareAgribusinessAirlinesBanking and financeConstructionEnergyFreight and logisticsHealthcareManufacturingMedia and MarketingRetailTelecommunicationsTourism
  • Opinion
    • All Opinion
    • Analysis
    • Editorials
    • Business analysis
    • Premium opinion
    • Letters to the editor
  • Politics
  • Sport
    • All Sport
    • OlympicsParalympics
    • RugbySuper RugbyNPCAll BlacksBlack FernsRugby sevensSchool rugby
    • CricketBlack CapsWhite Ferns
    • Racing
    • NetballSilver Ferns
    • LeagueWarriorsNRL
    • FootballWellington PhoenixAuckland FCAll WhitesFootball FernsEnglish Premier League
    • GolfNZ Open
    • MotorsportFormula 1
    • Boxing
    • UFC
    • BasketballNBABreakersTall BlacksTall Ferns
    • Tennis
    • Cycling
    • Athletics
    • SailingAmerica's CupSailGP
    • Rowing
  • Lifestyle
    • All Lifestyle
    • Viva - Food, fashion & beauty
    • Society Insider
    • Royals
    • Sex & relationships
    • Food & drinkRecipesRecipe collectionsRestaurant reviewsRestaurant bookings
    • Health & wellbeing
    • Fashion & beauty
    • Pets & animals
    • The Selection - Shop the trendsShop fashionShop beautyShop entertainmentShop giftsShop home & living
    • Milford's Investing Place
  • Entertainment
    • All Entertainment
    • TV
    • MoviesMovie reviews
    • MusicMusic reviews
    • BooksBook reviews
    • Culture
    • ReviewsBook reviewsMovie reviewsMusic reviewsRestaurant reviews
  • Travel
    • All Travel
    • News
    • New ZealandNorthlandAucklandWellingtonCanterburyOtago / QueenstownNelson-TasmanBest NZ beaches
    • International travelAustraliaPacific IslandsEuropeUKUSAAfricaAsia
    • Rail holidays
    • Cruise holidays
    • Ski holidays
    • Luxury travel
    • Adventure travel
  • Kāhu Māori news
  • Environment
    • All Environment
    • Our Green Future
  • Talanoa Pacific news
  • Property
    • All Property
    • Property Insider
    • Interest rates tracker
    • Residential property listings
    • Commercial property listings
  • Health
  • Technology
    • All Technology
    • AI
    • Social media
  • Rural
    • All Rural
    • Dairy farming
    • Sheep & beef farming
    • Horticulture
    • Animal health
    • Rural business
    • Rural life
    • Rural technology
    • Opinion
    • Audio & podcasts
  • Weather forecasts
    • All Weather forecasts
    • Kaitaia
    • Whangārei
    • Dargaville
    • Auckland
    • Thames
    • Tauranga
    • Hamilton
    • Whakatāne
    • Rotorua
    • Tokoroa
    • Te Kuiti
    • Taumaranui
    • Taupō
    • Gisborne
    • New Plymouth
    • Napier
    • Hastings
    • Dannevirke
    • Whanganui
    • Palmerston North
    • Levin
    • Paraparaumu
    • Masterton
    • Wellington
    • Motueka
    • Nelson
    • Blenheim
    • Westport
    • Reefton
    • Kaikōura
    • Greymouth
    • Hokitika
    • Christchurch
    • Ashburton
    • Timaru
    • Wānaka
    • Oamaru
    • Queenstown
    • Dunedin
    • Gore
    • Invercargill
  • Meet the journalists
  • Promotions & competitions
  • OneRoof property listings
  • Driven car news

Puzzles & Quizzes

  • Puzzles
    • All Puzzles
    • Sudoku
    • Code Cracker
    • Crosswords
    • Cryptic crossword
    • Wordsearch
  • Quizzes
    • All Quizzes
    • Morning quiz
    • Afternoon quiz
    • Sports quiz

Regions

  • Northland
    • All Northland
    • Far North
    • Kaitaia
    • Kerikeri
    • Kaikohe
    • Bay of Islands
    • Whangarei
    • Dargaville
    • Kaipara
    • Mangawhai
  • Auckland
  • Waikato
    • All Waikato
    • Hamilton
    • Coromandel & Hauraki
    • Matamata & Piako
    • Cambridge
    • Te Awamutu
    • Tokoroa & South Waikato
    • Taupō & Tūrangi
  • Bay of Plenty
    • All Bay of Plenty
    • Katikati
    • Tauranga
    • Mount Maunganui
    • Pāpāmoa
    • Te Puke
    • Whakatāne
  • Rotorua
  • Hawke's Bay
    • All Hawke's Bay
    • Napier
    • Hastings
    • Havelock North
    • Central Hawke's Bay
    • Wairoa
  • Taranaki
    • All Taranaki
    • Stratford
    • New Plymouth
    • Hāwera
  • Manawatū - Whanganui
    • All Manawatū - Whanganui
    • Whanganui
    • Palmerston North
    • Manawatū
    • Tararua
    • Horowhenua
  • Wellington
    • All Wellington
    • Kapiti
    • Wairarapa
    • Upper Hutt
    • Lower Hutt
  • Nelson & Tasman
    • All Nelson & Tasman
    • Motueka
    • Nelson
    • Tasman
  • Marlborough
  • West Coast
  • Canterbury
    • All Canterbury
    • Kaikōura
    • Christchurch
    • Ashburton
    • Timaru
  • Otago
    • All Otago
    • Oamaru
    • Dunedin
    • Balclutha
    • Alexandra
    • Queenstown
    • Wanaka
  • Southland
    • All Southland
    • Invercargill
    • Gore
    • Stewart Island
  • Gisborne

Media

  • Video
    • All Video
    • NZ news video
    • Herald NOW
    • Business news video
    • Politics news video
    • Sport video
    • World news video
    • Lifestyle video
    • Entertainment video
    • Travel video
    • Markets with Madison
    • Kea Kids news
  • Podcasts
    • All Podcasts
    • The Front Page
    • On the Tiles
    • Ask me Anything
    • The Little Things
  • Cartoons
  • Photo galleries
  • Today's Paper - E-editions
  • Photo sales
  • Classifieds

NZME Network

  • Advertise with NZME
  • OneRoof
  • Driven Car Guide
  • BusinessDesk
  • Newstalk ZB
  • Sunlive
  • ZM
  • The Hits
  • Coast
  • Radio Hauraki
  • The Alternative Commentary Collective
  • Gold
  • Flava
  • iHeart Radio
  • Hokonui
  • Radio Wanaka
  • iHeartCountry New Zealand
  • Restaurant Hub
  • NZME Events

SubscribeSign In
Advertisement
Advertise with NZME.
Home / Business

The worst job in technology is moderating Facebook pornography, racism and violence online

news.com.au
6 Jan, 2018 05:41 PM6 mins to read

Subscribe to listen

Access to Herald Premium articles require a Premium subscription. Subscribe now to listen.
Already a subscriber?  Sign in here

Listening to articles is free for open-access content—explore other articles or learn more about text-to-speech.
‌
Save

    Share this article

Nearly a third of the world uses the social media site Facebook, but how are New Zealanders using it?

Facebook founder Mark Zuckerberg has delivered a post promising to free the platform from the large amounts of awful, extreme and distressing content that gets published on a daily basis.

Zuckerberg said every year he sets a personal challenge for himself and given his belief that the world feels more anxious and divided than ever before, he wants to do his part by cleaning up the social media site.

"My personal challenge for 2018 is to focus on fixing these important issues. We won't prevent all mistakes or abuse, but we currently make too many errors enforcing our policies and preventing misuse of our tools. If we're successful this year then we'll end 2018 on a much better trajectory," he wrote on a post.

"This will be a serious year of self-improvement and I'm looking forward to learning from working to fix our issues together."

Every year I take on a personal challenge to learn something new. I've visited every US state, run 365 miles, built an...

Posted by Mark Zuckerberg on Thursday, 4 January 2018
Advertisement
Advertise with NZME.
Advertisement
Advertise with NZME.

'The worst of the web'

But Zuckerberg's task is not as easy as it sounds and requires the services of a small army of people to help - among other things - moderating the worst on the net.

Sarah Katz, 27, was tasked with searching for the worst on the web to keep it off Facebook, meaning on a daily basis she was forced to hunt online for pornography, racism and violence. It is largely considered to be the worst job in technology.

Despite little training on how to handle the distress, Katz had to look at as many as 8000 posts a day, which often include anti-Semitic speech, bestiality photos and child exploitation.

Hired by Facebook as a contractor, Katz was paid US$30 per hour for her services, which also required her to sign a waiver warning her about what she would encounter.

Advertisement
Advertise with NZME.

She has since quit her position — something common for this type of role with a high turnover rate that sees most content moderators lasting only a few months to a year.

It's also not uncommon for workers to quit on their first day, with some people even leaving for lunch and never returning.

Shaka Tafari was another worker tasked with managing the content of social media sites, having worked as a contractor at messaging app Whisper in 2016.

The 30-year-old said he was often subject to graphic images of bestiality or people killing dogs, plus a plethora of rape references.

Discover more

Business

$180 billion Bitcoin rival's meteoric rise

05 Jan 02:53 AM
Media and marketing

Billionaire Peter Thiel explores starting news outlet

05 Jan 08:11 PM
Business

Amazon to bid for Premier League rights

05 Jan 08:33 PM

"I was watching the content of deranged psychos in the woods somewhere who don't have a conscience for the texture or feel of human connection," he told the Wall Street Journal.

The task of weeding out the worst of the web is largely considered to be the worst job in technology. Photo / 123RF
The task of weeding out the worst of the web is largely considered to be the worst job in technology. Photo / 123RF

Tafari said the nature of the job was cruel as managers remotely monitored the productivity of moderators, sending them messages to ask why they weren't working if they dwelled too long after reviewing a post.

The policing of content on social media sites is one of the fastest-growing jobs in the technology world, and while Facebook, YouTube and Google are working to develop algorithms and artificial-intelligence tools, humans remain the first and best line of defence.

To highlight the gruelling task these people face, YouTube has the equivalent of 50 years of video uploaded each day, while Facebook receives more than a million user reports of dubious content per day — a lot of content to monitor.

In an attempt to stem the issue, Facebook has 7500 content reviewers and has plans to more than double this number by the end of 2018.

"I am dead serious about this," chief executive Mark Zuckerberg said last November when talking on the issue.

Advertisement
Advertise with NZME.

To help the contractors deal with the graphic content they are subject to, Facebook requires content moderators to be offered as many as three face-to-face counselling sessions a year.

A YouTube spokeswoman said the platform had similar methods in play.

"We strive to work with vendors that have a strong track record of good working conditions, and we offer wellness resources to people who may come across upsetting content during the course of their work," she said.

Lance Ulanoff, chief correspondent and editor-at-large for tech site Mashable, likened the positions to working a 24-hour crisis hotline.

"It's very intense work," Ulanoff told The Post. "These are people who are looking specifically for language or images that might indicate self-harm, violence or anything that would indicate someone might harm others.

"These monitors are seeing potentially intense information on a constant basis. At the same time, that's what they signed up to do."

Advertisement
Advertise with NZME.

Ulanoff feels Facebook's response to the spate of violence and suicides committed on the site has been good thus far, saying it recognises the role it plays in the larger conversation.

"They're coming around to the idea that they have to become stewards of this content platform and maybe make people's lives better and improve their product at the same time," he said.

"Keeping track of these monitors and maybe refreshing the group every now and then is a good idea, but they had to do something and I think this is a very good step … They're doing what they need to do for these monitors, at least for now."

Former Microsoft online safety program employees Henry Soto and Greg Blauert also know the damage working such a job can cause, with the men seeking damages from the company.

Both men are alleging negligence, disability discrimination and violations of the Consumer Protection Act.

The men claim Microsoft failed to warn them of the dangers of the job and did not provide adequate psychological support.

Advertisement
Advertise with NZME.

Microsoft disagreed with the plaintiffs' claims, saying it applies "industry-leading, cutting-edge technology" to identify questionable content and bans the users who shared that material. The company also said it has "robust wellness programs" to ensure employees who view the content are properly supported.

Mr Soto claims his job exposed him to "many thousands of photographs and video of the most horrible, inhumane and disgusting content you can imagine", according to the lawsuit.

The former-content moderator said he started having auditory hallucinations after seeing footage of a girl abused and murdered, according to the suit, ultimately going on medical leave in February 2015.

"Soto was embarrassed by his symptoms, which included panic attacks in public, disassociation, depression, visual hallucinations, and an inability to be around computers or young children, including, at times, his own son, because it would trigger memories of horribly violent acts against children that he had witnessed," the lawsuit reads.

Even though technology is working to ensure humans don't need to be the first line of defence, it will still be a while before people can remove themselves from the danger of these jobs.

Save

    Share this article

Latest from Business

Premium
Retail

'Give it a second chance': Ruby's recycled clothing venture takes off

06 Jul 03:00 AM
Premium
Opinion

Opinion: What to do if you have been left out of a parent's will

06 Jul 12:00 AM
New Zealand

26-year-old beats seven finalists to win Young Farmer of the Year

05 Jul 11:41 PM

Audi offers a sporty spin on city driving with the A3 Sportback and S3 Sportback

sponsored
Advertisement
Advertise with NZME.

Latest from Business

Premium
'Give it a second chance': Ruby's recycled clothing venture takes off

'Give it a second chance': Ruby's recycled clothing venture takes off

06 Jul 03:00 AM

Miller-Sharma aiming for 25% of revenue from non-new clothing by 2030.

Premium
Opinion: What to do if you have been left out of a parent's will

Opinion: What to do if you have been left out of a parent's will

06 Jul 12:00 AM
26-year-old beats seven finalists to win Young Farmer of the Year

26-year-old beats seven finalists to win Young Farmer of the Year

05 Jul 11:41 PM
Premium
Trump’s finances were shaky. Then he began to capitalise on his comeback

Trump’s finances were shaky. Then he began to capitalise on his comeback

05 Jul 08:00 PM
Gold demand soars amid global turmoil
sponsored

Gold demand soars amid global turmoil

NZ Herald
  • About NZ Herald
  • Meet the journalists
  • Newsletters
  • Classifieds
  • Help & support
  • Contact us
  • House rules
  • Privacy Policy
  • Terms of use
  • Competition terms & conditions
  • Our use of AI
Subscriber Services
  • NZ Herald e-editions
  • Daily puzzles & quizzes
  • Manage your digital subscription
  • Manage your print subscription
  • Subscribe to the NZ Herald newspaper
  • Subscribe to Herald Premium
  • Gift a subscription
  • Subscriber FAQs
  • Subscription terms & conditions
  • Promotions and subscriber benefits
NZME Network
  • The New Zealand Herald
  • The Northland Age
  • The Northern Advocate
  • Waikato Herald
  • Bay of Plenty Times
  • Rotorua Daily Post
  • Hawke's Bay Today
  • Whanganui Chronicle
  • Viva
  • NZ Listener
  • Newstalk ZB
  • BusinessDesk
  • OneRoof
  • Driven Car Guide
  • iHeart Radio
  • Restaurant Hub
NZME
  • About NZME
  • NZME careers
  • Advertise with NZME
  • Digital self-service advertising
  • Book your classified ad
  • Photo sales
  • NZME Events
  • © Copyright 2025 NZME Publishing Limited
TOP