Matthew Butterick is a writer, designer, programmer, and lawyer. Last November, Matthew and a group of lawyers filed a lawsuit against GitHub Copilot for its “unprecedented piracy of open source software.” The lawsuit is still ongoing.
This time he was interested in neural networks for image generation. Artists around the world are concerned that such AI systems are learning from vast amounts of their copyrighted work, without consent, without attribution, and without compensation.
On January 13, Matthew Butterick, along with artists Sarah Andersen, Kelly McKernan, and Karla Ortiz, and with the support of lawyers, filed a class action lawsuit against Stability AI, the developer of the image generator Stable Diffusion, and artist platform DeviantArt for using Midjourney, another AI-based image creation tool, who used the works of millions of artists for training.
“The image-generator companies have made their views clear. Now they can hear from artists,” Matthew Buterick reports in the announcement of the lawsuit.
The plaintiffs believe that artificial intelligence should be fair and ethical for all.
Stable Diffusion is a software product based on artificial of intelligence released in August 2022 by Stability AI. Butterick claims it contains unauthorized copies of millions – possibly billions – of copyrighted images. These copies were allegedly made without the knowledge or consent of the authors.
“Even assuming nominal damages of $1 per image, the value of this misappropriation would be roughly $5 billion. (For comparison, the largest art heist ever was the 1990 theft of 13 artworks from the Isabella Stewart Gardner Museum, with a current estimated value of $500 million)”, thinks Butterick.
The diffusion method using a neural network was invented in 2015 by AI researchers from Stanford University. The diagram below, taken from the Stanford team’s research, shows the two phases of the diffusion process using spiral-shaped training data.
Simply put, diffusion is how an AI program figures out how to reconstruct a copy of the training data through denoising. Because that’s the case, from a copyright perspective, it’s no different than MP3 or JPEG — a way of storing a compressed copy of certain digital data, the plaintiffs argue.
In 2020, researchers from the University of California at Berkeley improved the diffusion method in two ways:
- They showed how a diffusion model can store its training images in a more compressed format without affecting its ability to reproduce high-fidelity copies. These compressed copies of the training images are known as latent images.
- Researchers have discovered that these latent images can be interpolated — that is, blended mathematically — to produce new derivative images.
The diagram below, taken from the Berkeley team’s research, shows how this process works:
The image in the red box was interpolated from the two “source” images pixel by pixel. It looks like two translucent images of a face superimposed on each other, rather than a single convincing face.
The image in the green frame was generated differently. In this case, the two original images were compressed into latent images. After these latent images were interpolated, this new interpolated latent image was reconstructed pixel by pixel using a smoothing process.
Despite the difference in results, from a copyright perspective, these two methods of interpolation are equivalent: they both create derivative works by interpolating two original images.
In 2022, researchers from Munich perfected the diffusion technique. They figured out how to control the discoloration process with additional information. This process is called conditioning. (One of these researchers, Robin Rombach, currently works at Stability AI as a Stable Diffusion developer).
Founded by Emad Mostaque, Stability AI is based in London. The company finances LAION, a German organization, which creates ever-larger datasets of images for use by AI companies.
Butterick speculates that thousands or possibly millions of images in LAION were copied from DeviantArt and used to train Stable Diffusion.
The plaintiffs also accuse DeviantArt that instead of standing up for its community of artists, the platform decided to release a paid app called DreamUp, built on top of Stable Diffusion. In turn, a flood of AI-created art flooded DeviantArt, displacing human artists.
Loading comments …