Artists Can Fight Back Against AI by Killing Art Generators From the Inside

Kyle Barr
·4 min read
Artist woman teaching painting to a humanoid AI robot, she is pointing at canvas and giving advice to a robot surrounded by paint brushes and a canvas.
Artist woman teaching painting to a humanoid AI robot, she is pointing at canvas and giving advice to a robot surrounded by paint brushes and a canvas.


Some AI models are trained on hundreds of gigabytes worth of images, but some researchers show that poisoning just a handful of those could cause the AI model to hallucinate.

How can artists hope to fight back against the whims of tech companies wanting to use their work to train AI? One group of researchers has a novel idea: slip a subtle poison into the art itself to kill the AI art generator from the inside out.

Ben Zhao, a professor of computer science at the University of Chicago and an outspoken critic of AI’s data scraping practices, told MIT Technology Review he and his team’s new tool, dubbed “Nightshade,” does what it says on the tin— poisoning any model that uses images to train AI. So far, artists’ only option to combat AI companies was to sue them, or hope developers abide by an artists’ own opt-out requests.

Read more

A Stability AI spokesperson told Gizmodo that “Stability AI commits to equitable representation and bias reduction,” adding “when we set out to train SDXL 1.0, we worked hard to give the model a much more diverse and wide-ranging dataset. This included using sophisticated filters that create a more globally representative result of common items. We are always on a mission to learn and improve, and expect that subsequent models will be even more effective at avoiding bias.”

What Tools Do Artists Have to Fight Against AI Training?

Zhao was also the leader of the team that helped make Glaze, a tool that can create a kind of “style cloak” to mask artists’ images. It similarly disturbs the pixels on an image so it misleads AI art generators that try to mimic an artist and their work. Zhao told MIT Technology Review that Nightshade is going to be integrated as another tool in Glaze, but it’s also being released on the open-source market for other developers to create similar tools.

Other researchers have found some ways of immunizing images from direct manipulation by AI, but those techniques didn’t stop the data scraping techniques used for training the art generators in the first place. Nightshade is one of the few, and potentially most combative attempts so far to offer artists a chance at protecting their work.

There’s also a burgeoning effort to try and differentiate real images from those created by AI. Google-owned DeepMind claims it has developed a watermarking ID that can identify if an image was created by AI, no matter how it might be manipulated. These kinds of watermarks are effectively doing the same thing Nightshade is, manipulating pixels in such a way that’s imperceptible to the naked eye. Some of the biggest AI companies have promised to watermark generated content going forward, but current efforts like Adobe’s metadata AI labels don’t really offer any level of real transparency.

Nightshade is potentially devastating to companies that actively use artists’ work to train their AI, such as DeviantArt. The DeviantArt community has already had a pretty negative reaction to the site’s in-built AI art generator, and if enough users poison their images it could force developers to find every single instance of poisoned images by hand or else reset training on the entire model.

Still, the program won’t be able to change any existing models like SDXL or the recently released DALL-3. Those models are all already trained on artists’ past work. Companies like Stability AI, Midjourney, and DeviantArt have already been sued by artists for using their copyrighted work to train AI. There are many other lawsuits attacking AI developers like Google, Meta, and OpenAI for using copyrighted work without permission. Companies and AI proponents have argued that since generative AI creates new content based on that training data, all those books, papers, pictures, and art in the training data fall under fair use.

OpenAI developers noted in their research paper that their latest art generator can create far more realistic images because it is trained on detailed captions generated by the company’s own bespoke tools. The company did not reveal how much data actually went into training its new AI model (most AI companies have become reluctant to say anything about their AI training data), but the efforts to combat AI may escalate as time goes on. As these AI tools grow more advanced, they require even more data to power them, and artists might be willing to go to even greater measures to combat them.

Update: 10/24/23 at 8:22 a.m. ET: This post was updated to include a comment from a Stability AI spokesperson.

More from Gizmodo

Sign up for Gizmodo's Newsletter. For the latest news, Facebook, Twitter and Instagram.

Click here to read the full article.