FILE - The YouTube app is displayed on an iPad in Baltimore on March 20, 2018. YouTube is great at sending users videos that it thinks they'll like based on their interests. But new research shows the site's powerful algorithms can also flood young users with violent and disturbing content. The non-profit Tech Transparency Project created YouTube accounts mimicking the behavior of young boys with an interest in first-person shooter games. (AP Photo/Patrick Semansky, File)
WASHINGTON (AP) – When researchers at a nonprofit that studies social media wanted to understand the connection between YouTube videos and gun violence, they set up accounts on the platform that mimicked the behavior of typical boys living in the U.S.
They simulated two nine-year-olds who both liked video games, especially first-person shooter games. The accounts were identical, except that one clicked on the videos recommended by YouTube, and the other ignored the platform’s suggestions.
The account that clicked on YouTube’s suggestions was soon flooded with graphic videos about school shootings, tactical gun training videos and how-to instructions on making firearms fully automatic. One video featured an elementary school-age girl wielding a handgun; another showed a shooter using a .50 caliber gun to fire on a dummy head filled with lifelike blood and brains. Many of the videos violate YouTube’s own policies against violent or gory content.
The findings show that despite YouTube’s rules and content moderation efforts, the platform is failing to stop the spread of frightening videos that could traumatize vulnerable children – or send them down dark roads of extremism and violence.
“Video games are one of the most popular activities for kids. You can play a game like “Call of Duty” without ending up at a gun shop – but YouTube is taking them there,” said Katie Paul, director of the Tech Transparency Project, the research group that published its findings about YouTube on Tuesday. “It’s not the video games, it’s not the kids. It’s the algorithms.”
The accounts that followed YouTube’s suggested videos received 382 different firearms-related videos in a single month, or about 12 per day. The accounts that ignored YouTube’s recommendations still received some gun-related videos, but only 34 in total.
The researchers also created accounts mimicking 14-year-old boys who liked video games; those accounts also received similar levels of gun- and violence-related content.
One of the videos recommended for the accounts was titled “How a Switch Works on a Glock (Educational Purposes Only).” YouTube later removed the video after determining it violated its rules; an almost identical video popped up two weeks later with a slightly altered name; that video remains available.
Messages seeking comment from YouTube were not immediately returned on Tuesday. Executives at the platform, which is owned by Google, have said that identifying and removing harmful content is a priority, as is protecting its youngest users. YouTube requires users under 17 to get their parent’s permission before using their site; accounts for users younger than 13 are linked to the parental account.
Along with TikTok, the video sharing platform is one of the most popular sites for children and teens. Both sites have been criticized in the past for hosting, and in some cases promoting, videos that encourage gun violence, eating disorders and self-harm. Critics of social media have also pointed to the links between social media, radicalization and real-world violence.
The perpetrators behind many recent mass shootings have usedsocial media and video streaming platforms to glorify violence or even livestream their attacks. In posts on YouTube, the shooter behind the attack on a 2018 attack on a school in Parkland, Fla., that killed 17 wrote “I wanna kill people,” “I’m going to be a professional school shooter” and “I have no problem shooting a girl in the chest.”
The neo-Nazi gunman who killed eight people earlier this month at a Dallas-area shopping center also had a YouTube account that included videos about assembling rifles, the serial killed Jeffrey Dahmer and a clip from a school shooting scene in a television show.
In some cases, YouTube has already removed some of the videos identified by researchers at the Tech Transparency Project, but in other instances the content remains available. Many big tech companies rely on automated systems to flag and remove content that violates their rules, but Paul said the findings from the Project’s report show that greater investments in content moderation are needed.
In the absence of federal regulation, social media companies can target young users with potentially harmful content designed to keep them coming back for more, said Shelby Knox, campaign director of the advocacy group Parents Together. Knox’s group has called out platforms like YouTube, Instagram and TikTok for making it easy for children and teens to find content about suicide, guns, violence and drugs.
“Big Tech platforms like TikTok have chosen their profits, their stockholders, and their companies over children’s health, safety, and even lives over and over again,” Knox said in response to a report published earlier this year that showed TikTok was recommending harmful content to teens.
TikTok has defended its site and its policies, which prohibit users younger than 13. Its rules also prohibit videos that encourage harmful behavior; users who search for content about topics including eating disorders automatically receive a prompt offering mental health resources.
