A new study has revealed troubling findings about Meta’s efforts to moderate self-harm content on Instagram, with researchers claiming the company’s policies are "extremely inadequate." The Danish research team discovered that rather than removing harmful content, Instagram’s algorithm appeared to actively foster the spread of self-harm networks.
The study involved creating a private network of fake profiles, some as young as 13 years old, which shared 85 pieces of increasingly explicit self-harm content. Despite Meta’s claims that it uses artificial intelligence (AI) to proactively remove harmful posts, not a single piece of content was removed during the month-long experiment.
Digitalt Ansvar, a Danish organization dedicated to promoting responsible digital practices, conducted the research. It found that, while Meta’s AI technology could identify 38% of self-harm images and 88% of the most severe ones, Instagram failed to use this technology effectively. The study suggests that Meta has access to the necessary tools to address the issue but has chosen not to implement them properly. As a result, the platform's moderation process may not be in compliance with European Union regulations, such as the Digital Services Act, which mandates that large digital services identify risks to physical and mental well-being.
In addition to failing to remove harmful content, the study found that Instagram’s algorithm was actually helping to expand self-harm networks. When the researchers connected one of their fake profiles to the self-harm group, the algorithm suggested that the 13-year-old befriend all other members, facilitating the spread of self-harm-related content.
Ask Hesby Holm, CEO of Digitalt Ansvar, expressed surprise that none of the images were flagged or removed as they increased in severity. “We thought there would be some kind of mechanism in place to recognize these images, but big surprise—nothing happened,” he said. Holm raised concerns that the lack of moderation could lead to devastating consequences, as these self-harm networks may go undetected by parents or authorities, depriving vulnerable individuals of the help they need.
Lotte Rubæk, a psychologist who left Meta’s global suicide prevention expert group in March, also criticized the company’s failure to address self-harm content. She noted that the results of the study were particularly shocking because Meta has repeatedly claimed to be improving its technology and moderation processes. “It’s a matter of life and death for young children and teenagers,” Rubæk said, adding that the lack of action from Meta contributes to rising suicide rates and worsens the harm inflicted on vulnerable individuals.
Meta has responded to the findings by reiterating its commitment to removing content that encourages self-harm, stating that it removed over 12 million pieces of self-harm-related content from Instagram in the first half of 2024. The company also pointed to the launch of Instagram Teen Accounts, which are designed to limit exposure to sensitive content for young users. However, critics argue that these measures are insufficient, and that Instagram's failure to remove harmful content is indicative of a larger issue with the platform’s moderation practices.
As concerns about the impact of social media on young people's mental health continue to grow, experts warn that without more effective moderation, platforms like Instagram may be complicit in worsening the mental health crisis, particularly among vulnerable users.
No comments:
Post a Comment