Deepfake Pornography

The fight against AI-generated deepfake pornography is escalating in the United States. San Francisco City Attorney David Chiu has formally ordered Apple and Google to remove dozens of AI-powered “nudify” apps from their respective app stores, accusing the tech giants of enabling software that creates non-consensual intimate images (NCII). The move marks one of the strongest legal actions yet against platforms that distribute artificial intelligence tools capable of generating explicit deepfake content. It also raises new questions about AI regulation, app store accountability, digital privacy, and online safety. Why San Francisco Is Targeting Apple and Google According to letters sent by the San Francisco City Attorney’s Office, both Apple and Google have allegedly continued hosting and processing payments for apps that digitally alter photos to create fake nude images without a person’s consent. Officials argue that the companies have been aware of the issue for nearly a year but failed to take sufficient action. David Chiu said the companies have a responsibility to prevent technology that enables sexual exploitation. He stated that AI-powered nudify apps exploit women and girls by creating fake intimate images without consent while generating revenue through app downloads and in-app purchases. California’s Deepfake Laws Strengthen Enforcement California has introduced some of the nation’s toughest laws against AI-generated explicit content. State legislation now makes it illegal to knowingly facilitate or assist in creating non-consensual deepfake pornography. A separate law also allows victims to file civil lawsuits against third parties that help distribute or profit from such material. San Francisco officials argue that these legal protections apply not only to app developers but also to digital marketplaces that distribute and monetize these applications. The city has given Apple and Google 28 days to respond or potentially face civil penalties. Tech Transparency Project Raised Earlier Warnings The legal action follows multiple reports from the Tech Transparency Project (TTP), which warned both companies about the presence of AI nudify apps in their marketplaces. Earlier this year, TTP identified dozens of applications that allowed users to create deepfake intimate images in exchange for paid subscriptions or in-app purchases. The organization also claimed that Apple’s App Store and Google Play Store search systems helped users discover these apps, making the platforms active participants in distributing the technology. These reports intensified public scrutiny over how app marketplaces review AI-powered software before publication. Apple Responds by Removing Apps Apple says its App Store policies already prohibit nudify applications. In response to the city’s concerns, the company confirmed that it has: Apple maintains that applications violating its App Store guidelines are not permitted and that enforcement actions are ongoing. Google Suspends Violating Applications Google also announced enforcement measures following the allegations. The company stated that all five Google Play apps specifically referenced by San Francisco officials have been suspended. Google added that it has removed hundreds of apps violating its AI content policies and has restricted search terms such as “nudify” to reduce user access to similar applications. The company says it continues investigating reports and takes swift action when violations are confirmed. AI Deepfake Technology Raises Growing Privacy Concerns The rapid advancement of generative AI has made creating realistic fake images easier than ever. Modern AI image generation tools can manipulate publicly available photos to produce fabricated explicit content that appears highly convincing. While celebrities have frequently been targeted, cybersecurity experts warn that anyone with photos shared on social media could become a victim. This growing accessibility has fueled concerns surrounding: Lawmakers across the United States are increasingly focusing on regulating these emerging AI risks. App Store Responsibility Under Increased Scrutiny The latest dispute highlights a broader debate over whether app marketplace operators should bear greater responsibility for software distributed through their platforms. Critics argue that companies processing payments and promoting AI applications should conduct stronger reviews before allowing potentially harmful tools to reach consumers. Supporters of stricter oversight believe stronger moderation can reduce the spread of harmful AI-generated content while protecting victims from digital exploitation. The issue also reflects the growing challenge of balancing AI innovation with user safety, content moderation, and ethical artificial intelligence. What This Means for the Future of AI Regulation The action against Apple and Google could become a landmark case in how governments regulate AI-powered applications. As artificial intelligence continues transforming content creation, lawmakers are expected to introduce additional rules governing deepfake technology, AI-generated media, and digital platform accountability. For major technology companies, the message is becoming increasingly clear: app store operators may be expected to play a more active role in preventing harmful AI tools from reaching users. With deepfake abuse becoming a national concern, regulators, technology companies, and policymakers will likely face increasing pressure to strengthen safeguards that protect digital privacy while allowing responsible AI innovation to continue. The source of this news is From Tech Crunch.