🔗 Share this article UK Tech Companies and Child Protection Officials to Test AI's Ability to Generate Exploitation Images Technology companies and child safety organizations will receive authority to assess whether AI systems can produce child abuse material under recently introduced British laws. Substantial Increase in AI-Generated Harmful Material The declaration coincided with findings from a safety watchdog showing that cases of AI-generated child sexual abuse material have increased dramatically in the past year, rising from 199 in 2024 to 426 in 2025. New Regulatory Framework Under the changes, the government will permit designated AI developers and child safety groups to inspect AI systems – the foundational systems for chatbots and image generators – and verify they have sufficient protective measures to stop them from producing images of child exploitation. "Ultimately about stopping abuse before it occurs," declared the minister for AI and online safety, noting: "Specialists, under rigorous protocols, can now detect the risk in AI systems early." Tackling Regulatory Challenges The amendments have been introduced because it is against the law to produce and possess CSAM, meaning that AI developers and others cannot create such content as part of a testing regime. Until now, authorities had to wait until AI-generated CSAM was uploaded online before addressing it. This law is designed to averting that issue by enabling to halt the production of those images at source. Legal Structure The amendments are being added by the government as modifications to the crime and policing bill, which is also implementing a prohibition on owning, creating or sharing AI systems developed to generate exploitative content. Real-World Consequences This week, the minister visited the London base of a children's helpline and heard a simulated conversation to advisors involving a report of AI-based exploitation. The call depicted a teenager requesting help after facing extortion using a sexualised deepfake of himself, constructed using AI. "When I learn about children facing blackmail online, it is a cause of extreme anger in me and rightful anger amongst families," he said. Concerning Statistics A prominent internet monitoring foundation stated that instances of AI-generated abuse material – such as webpages that may include multiple files – had more than doubled so far this year. Instances of category A content – the gravest form of abuse – increased from 2,621 images or videos to 3,086. Female children were overwhelmingly victimized, accounting for 94% of prohibited AI images in 2025 Depictions of infants to toddlers increased from five in 2024 to 92 in 2025 Industry Reaction The legislative amendment could "represent a crucial step to ensure AI tools are secure before they are launched," commented the head of the internet monitoring foundation. "Artificial intelligence systems have made it so survivors can be victimised repeatedly with just a few clicks, giving criminals the capability to make potentially limitless amounts of advanced, lifelike exploitative content," she continued. "Content which further exploits survivors' trauma, and makes young people, especially girls, less safe on and off line." Support Session Data Childline also published information of counselling interactions where AI has been mentioned. AI-related risks discussed in the sessions include: Employing AI to rate body size, body and appearance Chatbots discouraging young people from talking to safe adults about abuse Being bullied online with AI-generated material Online extortion using AI-faked images During April and September this year, Childline conducted 367 support interactions where AI, conversational AI and associated terms were discussed, four times as many as in the same period last year. Half of the mentions of AI in the 2025 sessions were related to psychological wellbeing and wellness, encompassing utilizing chatbots for assistance and AI therapeutic apps.