Stable Attribution: Experimental Approach to Attributing Sources of AI Generated Images

OEGlobal Plaza
1

Maybe for a change from all the conversation about ChatGPT take a look at an interesting development in the space of AI Generated imagery (picking up on a topic of interest here for a while).

Might it be possible to attribute source images used to create new ones with the Stable Diffusion generator? That is the premise of Stable Attribution

When an A.I. model is trained to create images from text, it uses a huge dataset of images and their corresponding captions. The model is trained by showing it the captions, and having it try to recreate the images associated with each one, as closely as possible.

The model learns both general concepts present in millions of images, like what humans look like, as well as more specific details like textures, environments, poses and compositions which are more uniquely identifiable.

Version 1 of Stable Attribution’s algorithm decodes an image generated by an A.I. model into the most similar examples from the data that the model was trained with. Usually, the image the model creates doesn’t exist in its training data - it’s new - but because of the training process, the most influential images are the most visually similar ones, especially in the details.

You can explore a collection of examples to see what Stable Attribution suggests is the images training source – see one example I randomly explored (note the download is just a screen shot showing the first 2 source images out of 15 identified)

Stable Attribution says These human made images used by AI to generate this (a graphic style image of a 19th century man in a Victorian parlor
Stable Attribution says These human made images used by AI to generate this (a graphic style image of a 19th century man in a Victorian parlor1080×1080 101 KB

Downloaded “attribution” for full results of Stable Diffusion sourcing for an AI generated image

It’s interesting and one can see elements of some of the source images, but I cannot say for sure I understand how Stable Diffusion used them to create the final image. It’s not just mixing pixels.

And there is some speculation that its not identifying the source images but ones that are similar in the training dataset

https://twitter.com/matdryhurst/status/1622416416996577280

What’s interesting is that Stable Diffusion is built upon the LAION Open dataset which is clearly defined as public, but the license gets fuzzy to me:

We distribute the metadata dataset (the parquet files) under the most open Creative Common CC-BY 4.0 license, which poses no particular restriction. The images are under their copyright.

At least Stable Attribution offers a means to identify sources of images in the training data set (I am not sure where it goes)

Image of a victorian era male possible similar to the character in the final image, at bottom is field to share a source
Image of a victorian era male possible similar to the character in the final image, at bottom is field to share a source2524×1646 228 KB

Is this attribution or is it not? Do we really know how these images are created from sources in the data set? It remains a black box we have to guess at.

And in the spirit of attribution thanks to Alec Tarkowski for sharing this in Mastodon

2

Of relevance, Creative Commons is leading some discussion and exploration on Better Sharing for Generative Art

Community Meetings are planned by CC to collect input, and three different globally friendly times on February 22, 2023 (see post above for times and details)

Supporting community-driven solutions has also always been at the heart of Creative Commons’ approach to creativity. If you’re interested in this subject, we are going to be holding meetings with the Creative Commons community, and we also plan to continue meeting with diverse stakeholders to explore what sorts of solutions may be helpful in this area. As we go along we’ll continue to report on what we’ve learned and seek out more community feedback.

Join the CC team at a community discussion about generative AI: How can we make it work better for everyone and support better sharing in the commons?

I’m signing up now… anyone else?