NZ Herald
  • Home
  • Latest news
  • Video
  • New Zealand
  • Sport
  • World
  • Business
  • Entertainment
  • Podcasts
  • Quizzes
  • Opinion
  • Lifestyle
  • Travel
  • Viva
  • Weather forecasts

Subscriptions

  • Herald Premium
  • Viva Premium
  • The Listener
  • BusinessDesk

Sections

  • Latest news
  • New Zealand
    • All New Zealand
    • Crime
    • Politics
    • Education
    • Open Justice
    • Scam Update
  • Budget 2025
  • On The Up
  • World
    • All World
    • Australia
    • Asia
    • UK
    • United States
    • Middle East
    • Europe
    • Pacific
  • Business
    • All Business
    • MarketsSharesCurrencyCommoditiesStock TakesCrypto
    • Markets with Madison
    • Media Insider
    • Business analysis
    • Personal financeKiwiSaverInterest ratesTaxInvestment
    • EconomyInflationGDPOfficial cash rateEmployment
    • Small business
    • Business reportsMood of the BoardroomProject AucklandSustainable business and financeCapital markets reportAgribusiness reportInfrastructure reportDynamic business
    • Deloitte Top 200 Awards
    • CompaniesAged CareAgribusinessAirlinesBanking and financeConstructionEnergyFreight and logisticsHealthcareManufacturingMedia and MarketingRetailTelecommunicationsTourism
  • Opinion
    • All Opinion
    • Analysis
    • Editorials
    • Business analysis
    • Premium opinion
    • Letters to the editor
  • Sport
    • All Sport
    • OlympicsParalympics
    • RugbySuper RugbyNPCAll BlacksBlack FernsRugby sevensSchool rugby
    • CricketBlack CapsWhite Ferns
    • Racing
    • NetballSilver Ferns
    • LeagueWarriorsNRL
    • FootballWellington PhoenixAuckland FCAll WhitesFootball FernsEnglish Premier League
    • GolfNZ Open
    • MotorsportFormula 1
    • Boxing
    • UFC
    • BasketballNBABreakersTall BlacksTall Ferns
    • Tennis
    • Cycling
    • Athletics
    • SailingAmerica's CupSailGP
    • Rowing
  • Lifestyle
    • All Lifestyle
    • Viva - Food, fashion & beauty
    • Society Insider
    • Royals
    • Sex & relationships
    • Food & drinkRecipesRecipe collectionsRestaurant reviewsRestaurant bookings
    • Health & wellbeing
    • Fashion & beauty
    • Pets & animals
    • The Selection - Shop the trendsShop fashionShop beautyShop entertainmentShop giftsShop home & living
    • Milford's Investing Place
  • Entertainment
    • All Entertainment
    • TV
    • MoviesMovie reviews
    • MusicMusic reviews
    • BooksBook reviews
    • Culture
    • ReviewsBook reviewsMovie reviewsMusic reviewsRestaurant reviews
  • Travel
    • All Travel
    • News
    • New ZealandNorthlandAucklandWellingtonCanterburyOtago / QueenstownNelson-TasmanBest NZ beaches
    • International travelAustraliaPacific IslandsEuropeUKUSAAfricaAsia
    • Rail holidays
    • Cruise holidays
    • Ski holidays
    • Luxury travel
    • Adventure travel
  • Kāhu Māori news
  • Environment
    • All Environment
    • Our Green Future
  • Talanoa Pacific news
  • Property
    • All Property
    • Property Insider
    • Interest rates tracker
    • Residential property listings
    • Commercial property listings
  • Health
  • Technology
    • All Technology
    • AI
    • Social media
  • Rural
    • All Rural
    • Dairy farming
    • Sheep & beef farming
    • Horticulture
    • Animal health
    • Rural business
    • Rural life
    • Rural technology
    • Opinion
    • Audio & podcasts
  • Weather forecasts
    • All Weather forecasts
    • Kaitaia
    • Whangārei
    • Dargaville
    • Auckland
    • Thames
    • Tauranga
    • Hamilton
    • Whakatāne
    • Rotorua
    • Tokoroa
    • Te Kuiti
    • Taumaranui
    • Taupō
    • Gisborne
    • New Plymouth
    • Napier
    • Hastings
    • Dannevirke
    • Whanganui
    • Palmerston North
    • Levin
    • Paraparaumu
    • Masterton
    • Wellington
    • Motueka
    • Nelson
    • Blenheim
    • Westport
    • Reefton
    • Kaikōura
    • Greymouth
    • Hokitika
    • Christchurch
    • Ashburton
    • Timaru
    • Wānaka
    • Oamaru
    • Queenstown
    • Dunedin
    • Gore
    • Invercargill
  • Meet the journalists
  • Promotions & competitions
  • OneRoof property listings
  • Driven car news

Puzzles & Quizzes

  • Puzzles
    • All Puzzles
    • Sudoku
    • Code Cracker
    • Crosswords
    • Cryptic crossword
    • Wordsearch
  • Quizzes
    • All Quizzes
    • Morning quiz
    • Afternoon quiz
    • Sports quiz

Regions

  • Northland
    • All Northland
    • Far North
    • Kaitaia
    • Kerikeri
    • Kaikohe
    • Bay of Islands
    • Whangarei
    • Dargaville
    • Kaipara
    • Mangawhai
  • Auckland
  • Waikato
    • All Waikato
    • Hamilton
    • Coromandel & Hauraki
    • Matamata & Piako
    • Cambridge
    • Te Awamutu
    • Tokoroa & South Waikato
    • Taupō & Tūrangi
  • Bay of Plenty
    • All Bay of Plenty
    • Katikati
    • Tauranga
    • Mount Maunganui
    • Pāpāmoa
    • Te Puke
    • Whakatāne
  • Rotorua
  • Hawke's Bay
    • All Hawke's Bay
    • Napier
    • Hastings
    • Havelock North
    • Central Hawke's Bay
    • Wairoa
  • Taranaki
    • All Taranaki
    • Stratford
    • New Plymouth
    • Hāwera
  • Manawatū - Whanganui
    • All Manawatū - Whanganui
    • Whanganui
    • Palmerston North
    • Manawatū
    • Tararua
    • Horowhenua
  • Wellington
    • All Wellington
    • Kapiti
    • Wairarapa
    • Upper Hutt
    • Lower Hutt
  • Nelson & Tasman
    • All Nelson & Tasman
    • Motueka
    • Nelson
    • Tasman
  • Marlborough
  • West Coast
  • Canterbury
    • All Canterbury
    • Kaikōura
    • Christchurch
    • Ashburton
    • Timaru
  • Otago
    • All Otago
    • Oamaru
    • Dunedin
    • Balclutha
    • Alexandra
    • Queenstown
    • Wanaka
  • Southland
    • All Southland
    • Invercargill
    • Gore
    • Stewart Island
  • Gisborne

Media

  • Video
    • All Video
    • NZ news video
    • Business news video
    • Politics news video
    • Sport video
    • World news video
    • Lifestyle video
    • Entertainment video
    • Travel video
    • Markets with Madison
    • Kea Kids news
  • Podcasts
    • All Podcasts
    • The Front Page
    • On the Tiles
    • Ask me Anything
    • The Little Things
    • Cooking the Books
  • Cartoons
  • Photo galleries
  • Today's Paper - E-editions
  • Photo sales
  • Classifieds

NZME Network

  • Advertise with NZME
  • OneRoof
  • Driven Car Guide
  • BusinessDesk
  • Newstalk ZB
  • What the Actual
  • Sunlive
  • ZM
  • The Hits
  • Coast
  • Radio Hauraki
  • The Alternative Commentary Collective
  • Gold
  • Flava
  • iHeart Radio
  • Hokonui
  • Radio Wanaka
  • iHeartCountry New Zealand
  • Restaurant Hub
  • NZME Events

SubscribeSign In
Advertisement
Advertise with NZME.
Home / Business / Companies / Media and marketing

Christchurch mosque shootings: How social media's business model helped the massacre go viral

Washington Post
19 Mar, 2019 05:35 PM9 mins to read

Subscribe to listen

Access to Herald Premium articles require a Premium subscription. Subscribe now to listen.
Already a subscriber?  Sign in here

Listening to articles is free for open-access content—explore other articles or learn more about text-to-speech.
‌
Save

    Share this article

PM's speech in Parliament on Christchurch Mosque attack. / Parliament TV

The ability of Internet users to spread a video of Friday's slaughter in New Zealand marked a triumph — however appalling — of human ingenuity over computerised systems designed to block troubling images of violence and hate.

MORE:
• Christchurch killer's livestream: Facebook reveals viewer numbers, responds to critics
• $14K fines in offing as Commissioner asks Facebook to give police names of those who shared Chch clip

People celebrating the mosque attacks that left 50 people dead were able to keep posting and reposting videos on Facebook, YouTube and Twitter despite the websites' use of largely automated systems powered by artificial intelligence to block them. Clips of the attack stayed up for many hours and, in some cases, days.

This failure has highlighted Silicon Valley's struggles to police platforms that are massively lucrative yet also persistently vulnerable to outside manipulation despite years of promises to do better.

Friday's uncontrolled spread of horrific videos — a propaganda coup for those espousing hateful ideologies — also raised questions about whether social media can be made safer without undermining business models that rely on the speed and volume of content uploaded by users worldwide. In Washington and Silicon Valley, the incident crystallized growing concerns about the extent to which government and market forces have failed to check the power of social media.

Advertisement
Advertise with NZME.
Advertisement
Advertise with NZME.

"It's an uncontrollable digital Frankenstein," said Tristan Harris, a former Google design ethicist and co-founder of the Center for Humane Technology.

Those pushing videos of Friday's attack made small alterations — such as changing the color tones or length — of the shooting video originally live-streamed by the alleged killer himself through his Facebook page. Such tricks often were enough to evade detection by artificial-intelligence systems designed by some of the world's most technologically advanced companies to block such content.

But for all of the investment in such technology, even more has gone into making social media platforms powerful springboards for delivering images, sounds and words to as many people as possible — for the purpose of generating advertising revenue that fuels profits measured in the tens of billions of dollars each year.

Advertisement
Advertise with NZME.

"The only good thing that's come out of this is that it laid bare the lie that the big tech companies can solve this problem with AI because they really cannot," said Cathy O'Neil, author of "Weapons of Math Destruction," about the societal impact of algorithms. "Nobody knows how to counteract this stuff. They should stop pretending they do."

The video was spread across most of Facebook's platforms. Photo/Getty Images.
The video was spread across most of Facebook's platforms. Photo/Getty Images.

Mia Garlick, the head of communications and policy for Facebook in Australia and New Zealand, said the company would "work around the clock to remove violating content using a combination of technology and people." Garlick said the company is also now even removing edited versions of the video that do not feature graphic violence.

Twitter did not respond to a request for comment Monday, and Reddit declined to comment, but both have described working hard over several days to remove objectionable content from the shooting. For Reddit, that includes forums named "gore" and "watchpeopledie."

A YouTube executive, in an exclusive interview with The Washington Post, acknowledged that the platform's systems were overwhelmed and promised to make improvements. "We've made progress, but that doesn't mean we don't have a lot of work ahead of us, and this incident has shown that," said Neal Mohan, YouTube's chief product officer.

Discover more

New Zealand

Christchurch victims' families frustrated with wait for bodies

19 Mar 06:14 AM
New Zealand

Christchurch mosque attacks: The neighbours who offered refuge to dozens

19 Mar 06:45 AM
New Zealand

Heavy hearts held in love at flower memorials

19 Mar 07:36 AM
New Zealand

Cut off from terror reports: Accused gunman denied all media

19 Mar 08:20 AM

Facebook, YouTube and Twitter have scrambled before and largely failed to contain odious content on their platforms. Nor is this the first time killers have used social media to deliver images of their crimes.

But nobody before had staged a mass-casualty attack in a way so geared to spreading it virally across social media. Many critics worry that because it has happened once, it almost certainly will happen again.

Micah Schaffer, a technology policy consultant and a former director at YouTube, said that over the past decade social media companies have designed software to more effectively promote content to broad audiences. Those design choices, he said, have made it easier for content to spread online.

"Back in 2007, when I was at YouTube, if a video on the homepage was a success, that meant getting hundreds of thousands of views," Schaffer said. "These videos were handpicked, and we scrutinized each one. Compare that today to the recommendation algorithm sending millions of views to videos with no human intervention. To me, that's just irresponsible."

Those who study social media say that slowing the spread of appalling videos might require the companies to change or limit some features that help spread stimulating content. Those include powerful search and recommendation algorithms, nearly instantaneous uploads and autoplay.

The companies' losing battle to keep content in check already is having financial consequences. Facebook's stock recorded its steepest drop of the year, falling more than 3 per cent on Monday. In a note to clients, Needham & Co. analyst Laura Martin blamed the negative effects of "horrific images ... that are technologically difficult to block at the 100 per cent level and which hurt [Facebook's] brand."

Advertisement
Advertise with NZME.

Shares in Google's parent company, Alphabet, slid less than 1 per cent.

The day of the New Zealand shooting, authorities say, the admitted gunman, Brenton Tarrant, posted on the anonymous message board 8chan that he would "live stream the attack via facebook" and included a link to Facebook and a number of other sites where an anti-Muslim manifesto and other documents were stored.

Facebook's stock has dropped sharply due to fallout from the massacre. Photo/Getty Images.
Facebook's stock has dropped sharply due to fallout from the massacre. Photo/Getty Images.

Within minutes of this announcement, a flood of anonymous posters were cheering him on and rallying to save and re-upload the video widely online to give it maximum distribution: "I am downloading it"; "Stream saved"; "SAVE THIS S*** NOW"; "GRAB THEM WHILE YOU CAN."

One poster wrote that the gunman had "delivered" on his pledge: "I just saw him kill so many" people, using an obscene term for Muslims. "Nice shootin," another wrote. 8chan did not respond to requests for comment.

"Human nature has always had evil and horrific elements to it. But we're not asking for the platforms to solve humanity — that's not the issue," said Mike Ananny, an associate professor of communication at the University of Southern California. "These companies have the advertising monopolies, the eyeball-attention monopolies. … We have to expect them to be a much more responsible player than they are."

When the tech giants want to block a troubling video, they add the original to a vast internal blacklist so that their systems can quickly recognize whenever a copy of it resurfaces. This type of "hashing" technology is a key reason companies can automatically flag or block child pornography, terrorist propaganda and copyrighted material before it spreads widely online.

Advertisement
Advertise with NZME.

But their algorithms depend on pattern recognition, and even the most sophisticated systems are easily deceived. Anyone hoping to spread an otherwise blocked video — such as movie pirates and creators of extremist videos — can cut out shorter snippets or change the video's playing speed, colouring, sound or video size, then upload it again.

Facebook said that it blocked 80 per cent of the New Zealand first-person shooting videos from being uploaded within the first 24 hours but that roughly 300,000 videos slipped through — a showing that put them within reach of millions of people online. The company also joined its peers, including Twitter and YouTube, in sharing information about terrorist propaganda as part of the industry-led Global Internet Forum to Counter Terrorism. On Monday, the group said it had identified "more than 800 visually-distinct videos related to the attack," adding: "This incident highlights the importance of industry cooperation regarding the range of terrorists and violent extremists operating online."

U.S. law largely shields major tech platforms from being held liable for the content users post to their sites. Silicon Valley sees this decades-old legal shield — known as Section 230 for the provision in law it represents — as the reason that major social media sites have flourished without having to defend themselves against lawsuits.

But frustrated lawmakers from both parties increasingly are wondering whether the time has come to craft new legislation to curb digital ills such as hate speech and terrorist propaganda.

"I think that we absolutely need to have a hearing and understand exactly what happened in the situation to determine the best solutions to prevent it from happening in the future," said Rep. Suzan DelBene (D-Wash.). "What happened in New Zealand is horrifying and heartbreaking, and that video should never be available online."

But a chief architect of Section 230, Sen. Ron Wyden (D-Ore.), urged caution. While saying that tech giants must be "far more vigorous about identifying, fingerprinting and blocking content and individuals who incite hate and violence," Wyden said that eliminating the section could have unintended consequences for free speech.

Advertisement
Advertise with NZME.

"So often in the wake of horrible events politicians grasp for knee-jerk responses that won't solve real problems, and may even make them worse," he added in a statement.

European policymakers have been more aggressive in holding technology giants responsible for what appears on their platforms. Germany, for example, began enforcing a law last year that requires social media sites to remove instances of hate speech within 24 hours. The European Union broadly has proposed rules that would give Facebook, YouTube and others an hour to remove terrorist content — or face hefty fines.

Facebook chief Mark Zuckerberg told congressional lawmakers last year that human moderators and AI systems would help the company solve some of its most intractable problems, including automatically flagging harmful videos.

But AI researchers have said that view doesn't factor in the vast number of ways in which automated systems fail to come even close to human ability, including in understanding context and common sense. Today's algorithms excel at certain tasks but are what AI engineers call frustratingly "brittle," with even small changes to their tasks or training data quickly leading them to collapse.

Stephen Merity, a machine learning researcher in San Francisco, said tech companies do not want to use more drastic measures, such as tougher restrictions on who can upload or bigger investments in content-moderation teams, because of how they could alter their sites' usability or business model. "They don't want to do that, so they make these wild promises," he said.

But "you can't bank on future magical innovations," Merity said. "We're past the point of ceding the benefit of the doubt to these tech companies."

Advertisement
Advertise with NZME.

- Washington Post

Save

    Share this article

Latest from Media and marketing

Premium
Opinion

Fran O'Sullivan: Willis’ film industry backing shows Budget's focus on economic growth

16 May 09:00 PM
Premium
Business|companies

The big lessons for NZ in Australia's under-16 social media ban

14 May 05:32 AM
Entertainment

'Very sorry': Crushing news for Grand Theft Auto fans

04 May 10:28 PM

The Hire A Hubby hero turning handyman stereotypes on their head

sponsored
Advertisement
Advertise with NZME.

Latest from Media and marketing

Premium
Fran O'Sullivan: Willis’ film industry backing shows Budget's focus on economic growth

Fran O'Sullivan: Willis’ film industry backing shows Budget's focus on economic growth

16 May 09:00 PM

OPINION: Balancing growth and fiscal prudence will be a tough task for the Govt, though.

Premium
The big lessons for NZ in Australia's under-16 social media ban

The big lessons for NZ in Australia's under-16 social media ban

14 May 05:32 AM
'Very sorry': Crushing news for Grand Theft Auto fans

'Very sorry': Crushing news for Grand Theft Auto fans

04 May 10:28 PM
Premium
Roger Partridge: How asset recycling could solve NZ's infrastructure woes

Roger Partridge: How asset recycling could solve NZ's infrastructure woes

19 Apr 03:00 AM
Gold demand soars amid global turmoil
sponsored

Gold demand soars amid global turmoil

NZ Herald
  • About NZ Herald
  • Meet the journalists
  • Newsletters
  • Classifieds
  • Help & support
  • Contact us
  • House rules
  • Privacy Policy
  • Terms of use
  • Competition terms & conditions
  • Our use of AI
Subscriber Services
  • NZ Herald e-editions
  • Daily puzzles & quizzes
  • Manage your digital subscription
  • Manage your print subscription
  • Subscribe to the NZ Herald newspaper
  • Subscribe to Herald Premium
  • Gift a subscription
  • Subscriber FAQs
  • Subscription terms & conditions
  • Promotions and subscriber benefits
NZME Network
  • The New Zealand Herald
  • The Northland Age
  • The Northern Advocate
  • Waikato Herald
  • Bay of Plenty Times
  • Rotorua Daily Post
  • Hawke's Bay Today
  • Whanganui Chronicle
  • Viva
  • NZ Listener
  • What the Actual
  • Newstalk ZB
  • BusinessDesk
  • OneRoof
  • Driven CarGuide
  • iHeart Radio
  • Restaurant Hub
NZME
  • About NZME
  • NZME careers
  • Advertise with NZME
  • Digital self-service advertising
  • Book your classified ad
  • Photo sales
  • NZME Events
  • © Copyright 2025 NZME Publishing Limited
TOP