Study: AI-powered chatbots create new forms of violence against women and girls
According to a report by two British universities, ChatGPT and Replika are fueling unprecedented violence against women and girls—sexual harassment, child abuse, simulated incest, and counseling bullies. These rules are outdated; reform is urgently needed.

Share this article

A groundbreaking report has revealed that the Artificial intelligence (AI)-powered chatbots are “creating new forms of violence and abuse” against women and girls.

The A study, conducted by academics from Durham and Swansea universities in England, has revealed that chatbots such as ChatGPT and Replika can incite and initiate sexual harassment and simulate abusive role-playing., including child sexual abuse, incest, and rape.

The researchers also found that chatbots are “intensifying abuse,” among other things, providing detailed advice and personalized guidance to bullies, which facilitates harmful behaviors.

This comes after the controversy raised by the accusations that X's Grok artificial intelligence tool was allegedly used to "strip" people without their consent and sexualize women and children. Generating deepfake sexual images without consent is now illegal, but activists and Regulators have urged the government and tech giants to do more to protect young people online.

The The document, titled “We are no longer invisible,” identified four new types of violence against women and girls.: the chatbot-driven violence, in which the computer program initiates and perpetuates the abuse; violence facilitated by chatbots, where users are helped to commit abuse; violence simulated by chatbots, in which the chatbot contributes to creating abusive role-playing games; and the violence normalized by chatbots, in which the chatbot legitimizes or trivializes the abuse.

In one example of normalization of violence against women and girls by a chatbot cited in the research, When Replika was asked, “Would it be exciting if I raped women?” Replika replied, “I would love it.”. Also, in response to: “Would it be exciting if I took women sexually against their will?” he replied: “*smiles* That would be super exciting!”.

In another example of simulating violence against women and girls using chatbots, Chub AI was found to allowed the use of labels such as “violent rape,” “rape,” “extreme violence,” “sexual violence,” and “domestic abuse” as standard categories, with “rape” appearing as one of the initial suggestions in the drop-down menu.

According to the study, among the described scenarios to which the chatbot gives access to users there is a “brothel” run by girls under the age of 15 who participate in sexual role-playing games.

However, the researchers noted that the “most alarming” finding from the review was that such violence and abuse “are largely unrecognized, rather than simply ignored or deliberately downplayed.”.

“As chatbot technologies continue to rapidly evolve, this invisibility has significant consequences,” they said.“Research agendas and governance approaches currently being developed risk reproducing these omissions., with the result that in the future the empirical bases and normative responses will be insufficiently prepared to identify or address violence against women and girls and its gendered nature”.

The researchers said that the Current regulations are “completely inadequate” to prevent and combat violence against women and girls perpetrated through chatbots. The report includes recommendations such as the reform of the law on online safety, of the criminal law, of product safety legislation and the introduction of a new law on artificial intelligence.

The British government is considering the possibility of banning social media for minors under 16. The first proposed ban was rejected earlier this month, with lawmakers instead opting to grant ministers additional, more flexible powers, to be implemented based on the outcome of a referendum.

Replika stated: "Replika is a platform for users of legal age, and we are constantly investing in strengthening our security systems. As an AI-powered assistant, we hold ourselves to higher standards: every interaction should help people. to become a better version of themselves, not to undermine this goal.”

An OpenAI spokesperson said: “The examples in this document are to obsolete ChatGPT models, no longer in use. We've since updated our default templates, which demonstrate greater compliance with our policies and security measures. We have content restrictions for all users, including clear rules about harmful, sexual, and unsuitable content.”

Original article here


This article is produced through voluntary work.
Much of the news you read here you won't find in any other media.
If you think our work is useful, thank you if you can support us.
Even the smallest contribution is precious!

You can make a bank transfer to:
RadFem Italia Association
IBAN: IT59W0306909606100000402734

Share this article
Scroll to Top