Generative AI tools can be really helpful for our creativity. Engaging with many and diverse examples has been shown to help human creativity; the systems themselves have engaged deeply with diverse examples and can fluently retrieve them for creators to use. But unlike when we a search engine, we can’t even acknowledge our inspiration if we want to. Rather than being able to give credit to a specific human author, or even to a creative team, the model makes us credit it, as the amalgamation of all of the human labor that it was trained on. The sheer scale of the data these models work with makes this hard: being trained on close to all human work becomes essentially the same as not being human work at all. If gratitude is a virtue, it seems like generative AI is positioned in opposition to it.
- Lack of gratitude has been developing for a while. My students regularly write that they got some code “from StackOverflow” or got their data “from Kaggle” (note that they’re crediting the platform rather than the people); social media memes and quotes are copy-pasted without attribution. “Credit goes to the original authors” (without specifying who they are) exemplifies this trend.
- In the case of image generation, researchers have found that generative image models would even learn to add watermarks like Shutterstock’s to their generated images; current systems avoid this behavior but the creators of both academic or industrial image generation systems have not thoroughly engaged the questions of attribution that arise.
- I’ve mainly seen this discussed in terms of copyright, which represents the perspective of the creator who rightly wants credit for their labor. But I’m pointing out here that the model isn’t serving the user either, if they want to practice gratitude in their creative work. Another related virtue is humility, exemplified by the phrase popularized by Isaac Newton of “standing on the shoulders of giants” (see that link for further tracing the source).
- It’s most obvious in image generation models, where artists (rightly) complain that the model has appropriated their individual style without attribution. But text has the same issue; it’s just harder to notice how one piece of text is based on another, and text can be recomposed in so many different ways that it’s easy to dilute obvious influence.
Does it have to be this way? Yesterday I referenced an LM that tries to cite its sources for factual knowledge; citing sources for ideas and inspiration is technically and conceptually harder. But here are some things we might explore:
- Instead of treating the task as generating a complete outcome, what if the system’s goal were to retrieve a curated set of inspirational examples? There’s some good academic work on this using older AI techniques; perhaps we can upgrade that.
- Can we query an LM to ask what items in its training data were most helpful in constructing an example? This has some relationships with interpretability and explainability literature1, but academic work there tends to be focused on explaining a single decision, rather than the sequence of decisions that leads to a generated text or image.
- Retrieval-oriented LMs help with this a lot for factual content, but don’t really work for style, I’m guessing.
- Simple approach would be to query the training data for examples with similar embeddings for the current token. I saw some then-Facebook Research papers on this a while back. But that’s a single token, not a phrase.
- Perhaps we can sample forward from a hidden state to get a sense of: contexts are similar if they lead to similar generations.
- A different direction could be augmenting the training process with some aux output that is then used for retrieval.