These apps — many of which were adopted by Gen Z as teens and young people ventured into more private corners of the internet — are ill-equipped to monitor such content. They are fundamentally designed to keep communications private and present different challenges than Facebook, YouTube and Twitter, where violent screeds and videos have been algorithmically amplified for millions of viewers.
The way this generation is using social media more broadly could eliminate years of work to spot and identify public signs of impending violence, social media experts warn.
“There’s this shift towards more private spaces, more ephemeral content,” said Evelyn Douek, senior research fellow at the Knight First Amendment Institute at Columbia University. “The content moderation tools that platforms have developed and that we have argued about are somehow outdated or speak of the last war.”
Texas Gov. Greg Abbott (R) said Wednesday that the Texas gunman, who authorities have identified as Salvador Rolando Ramos, wrote on social media, “I’m going to shoot my grandmother” and “I’m going to shoot an elementary school” shortly before the attack. Facebook confirmed the messages were sent privately, but declined to say which of its social networks were being used.
Stephen Garcia, who considered himself Ramos’ best friend in eighth grade, previously told the Washington Post that Ramos used the Yubo app, a platform where users Tinder-style swipe on each other’s profiles or in live Streaming rooms and hanging out virtually can “meet” other users by playing games and chatting.
Yubo spokeswoman Amy Williams said in an email that the company cannot release information outside of direct requests from law enforcement, but that the company is investigating an account that was suspended from its platform.
“We are deeply saddened by this unspeakable loss and are cooperating fully with law enforcement in their investigation,” she said.
In the Buffalo shooting case, suspected gunman Payton Gendron sent an online chat room invitation on instant messaging platform Discord, which was accepted by 15 users, allowing them to page back through months of Gendron’s voluminous writings and racist smears . The Post reported. Users who clicked through to the room were also able to view an online video stream that broadcast footage of the Buffalo attack. This attack was also broadcast on Twitch, a live streaming service popular with video game users.
Discord and Twitch did not immediately respond to requests for comment.
Twitch was able to remove the stream within two minutes after the shooter started shooting, Angela Hession, the company’s head of trust and safety, said previously. The website has a 24/7 escalation system to handle urgent reports such as B. live streaming of violence.
Discord has since said the messages were only visible to the suspect until he shared them with others on the day of the attack.
In the wake of the high profile mass shootings of recent years, communities, school districts and tech companies have made major investments in security systems aimed at rooting out violent screeds in hopes of stopping an attack before it happens. The Uvalde Consolidated Independent School District previously used an artificial intelligence-assisted program to scan social media posts for potential threats years before the attack, although it’s unclear if it was in use at the time of the shooting.
But these tools are ill-equipped to deal with the rising popularity of live video streaming and private or disappearing messaging increasingly used by young adults and teenagers. These messages are then sealed to outsiders who may be able to spot the warning signs that a troubled person is about to harm themselves and others.
These newer social networks also have much less experience handling violent content, and they likely have fewer policies and staff to respond to incitement to violence on their services, experts said.
“With smaller sites or newer sites, they’re experiencing the moments that bigger services like Facebook and YouTube had in 2015 and 2016,” said Emma Llansó, the director of the Free Speech Project at the Center for Democracy and Technology, a nonprofit organization big technology companies supported.
Sagittarius adoption of these upstart apps reflects a larger generational shift in social media usage. Gen Z, teens and young adults born after 1996 are flocking to apps that focus on private messaging or live streaming, or that allow their users to post content that will disappear from public profiles after a certain amount of time .
They have largely shunned older social media apps like Facebook, Twitter, and YouTube, which gained popularity by providing public and open spaces for communicating with the world.
The new apps’ role in the shootings has drawn the attention of New York and New Jersey Attorneys General, who are investigating after the Buffalo shooting Probes launched in Discord and Twitch.
“Time and time again, we have seen the real devastation emanating from these dangerous and hateful platforms,” New York Attorney General Letitia James (D) said in a statement announcing the investigation following the Buffalo shooting. “We are doing everything in our power to shine a spotlight on this alarming behavior and take action to ensure it never happens again.”
Social media has played a prominent role in many mass shootings, and there have been high-profile cases of gunmen posting their plans online in the public eye and not being caught.
Republican lawmakers, who have long resisted measures to expand background checks or restrict access to guns, wanted to shine a spotlight on social media’s role in Thursday’s Texas shooting. “The common theme of almost all of these mass shootings is the social alienation of sick young men, often fueled by social media,” Sen. Dan Sullivan (R-Alaska) tweeted. He didn’t mention access to guns in the post.
Tech industry officials pushed back, warning that such tweets could distract from broader gun control policy issues.
“Some people will try to do it through Facebook so it’s not about guns,” tweeted Brian Fishman, Facebook’s former director of counterterrorism, dangerous organizations and content policies. “Don’t let her.”
Tech giants are also locked in a years-long power struggle as they attempt to balance privacy with monitoring of content on their websites and demands from law enforcement.
Facebook and other companies have moved toward strong encryption, a technology that scrambles the contents of a message so only the sender and recipient can see it. WhatsApp and Apple iMessage use it, as do messaging apps like Signal. And Facebook has announced it will make encrypted messaging the default setting for Instagram and Facebook Messenger, prompting backlash from politicians and law enforcement officials, who have warned that the widespread adoption of this technology may leave them in the dark and make it harder for them to investigate violence .
Some large tech companies scan messages for malicious content, such as child sexual abuse or spam. However, experts warn that monitoring more private communication spaces is a delicate balance.
“There are so many incredibly legitimate reasons people might want to use private communications,” Llansó said. “It’s not something that should be sacrificed for all people because some people want to use private communications for cruel reasons.”
Social media users tend to be younger, but the generational gap in the user base of personal messaging apps like Snapchat is larger than that of more traditional public-facing sites like Facebook.
When Snapchat users send each other private messages, they disappear after the recipient reads them. The app also pioneered the concept of “stories” – public posts lasting just one day – which were later copied by Facebook.
Snap said Wednesday that it has suspended an account that may have been linked to Ramos and that it is also cooperating with law enforcement.
Meanwhile, Facebook is struggling to keep up with the rapidly evolving social habits of teenage users.
Facebook’s own internal research reports that young adults are “less engaged” than older adults, posing significant risk to the company’s business, according to a wealth of internal company documents known as the Facebook files. The company’s research found that young adults prefer sharing updates about their lives via text messages rather than sending them to a wide range of Facebook friends. The researchers suggested that the company should respond by focusing on groups and more private forms of sharing.
“It will always be a game of cat and mouse,” said Douek. “These are simply unsolvable problems. But that doesn’t mean we can’t improve or let platforms off the hook.”
Rachel Lerman contributed to this report.
