Exploring AI Opportunities for Writing and Museum Publications
MoMA and Google Arts & Culture experiment with AI tools for improving the Museum’s workflow
In the summer of 2024, The Museum of Modern Art (MoMA) and Google Arts & Culture began an exploration of AI in the Museum’s workflow, starting with the question of how to leverage the Museum’s vast archives of exhibition catalogs and exhibition history.
Home Is a Foreign Place (1999) by ZarinaMoMA The Museum of Modern Art
Background
In 2016, MoMA made available its full exhibition history since its founding in 1929. This comprehensive digital account of the Museum’s 3,500-plus exhibitions featured installation photographs, press releases, checklists, and catalogs.
The catalogs associated with each exhibition contain detailed information on artists and artworks, commentary by Museum curators and other scholars, essays, and images that give additional context to the exhibition.
Focus
Given all of this publicly available information and the advancements with large language models that benefit from such rich context, we wondered if there was a way to use AI to help the Museum’s internal processes.
AI can be applied in many different ways, so we wanted to pilot a specific focus, and landed on a direction: looking at catalogs with AI to facilitate the research and writing of artist biographies for the public moma.org website.
Select Documents, Museum Archives (2004)MoMA The Museum of Modern Art
This use case came from a very specific Museum need.
MoMA’s collection management database lists nearly 20,000 artists as makers of the many works in its collection, as well as those may be catalogued for in temporary exhibitions.
MoMA endeavors to provide thoughtful artists’ biographies, in the institution’s voice, and taking its collection into account, for the public on moma.org.
But the staff bandwidth and time it takes to research, draft, edit, publish, and update the biographies for these artists means that only a modest, but ever-growing, percent of artists have biographies available on the website.
We wondered, is it possible to use AI to analyze a pre-selected set of MoMA’s exhibition catalogs and auto-generate a first draft artist biography based on the content found in those catalogs? Doing so would significantly reduce the time needed for a MoMA writer to research the relevant portions of these catalogs in order to write these biographies.
The Process
We started with a publicly available tool - NotebookLM. We loaded in a total of 1,689 pages from 5 exhibition catalogs. In our initial testing, we found the user interface easy to navigate, appreciated the promise of citations and the conversational aspect of the tool.
We found, however, we could not control the output as specifically as we needed for our use case. Some answers were too general, and some were not reliable, especially with respect to counting things i.e. “how many artists mentioned in x catalog?”.
Google Arts & Culture x MoMA prototype 1MoMA The Museum of Modern Art
Next, artist in residence at Google Arts & Culture Gael Hugo helped us experiment with a custom Gemini Pro and UI for our goals. This allowed us to easily upload multiple PDF catalogs, designate a specific artist, and generate a comprehensive biography.
Modern Art Despite Modernism, Robert Storr (2000)MoMA The Museum of Modern Art
The system extracted text from the PDFs and leveraged Gemini Pro's capabilities to identify relevant information from each source. Those snippets were then seamlessly integrated into a cohesive artist biography.
High and Low: Modern Art and Popular Culture, Kirk Varnedoe and Adam Gopnik (1990) by Adam Gopnik and Kirk VarnedoeMoMA The Museum of Modern Art
To encourage consistency with MoMA's existing writing style, we incorporated custom system instructions based on previously written bios. While this approach yielded promising results, including well-written biographies, we encountered challenges with accuracy.
In some instances, the model incorrectly attributed paintings to the wrong artists. This error rendered the tool ineffective as it completely undermined its credibility.
We refocused our efforts on a single artist and catalog at a time with the help of artist in residence at Google Arts & Culture Lynn Cherny. A new workflow tool was prototyped that enriches the factual background given to the LLM, providing a list of Museum assets associated with an artist from MoMA’s databases. A catalog’s pages can be searched and filtered for relevant artworks and passages using smaller models, before being used as prompt context for the final biography generation. Catalogs can range from under 100 pages to exceeding 400 pages in rich information. The method focused the AI models on only the most relevant pages, reducing hallucinations.
Google Arts & Culture x MoMA prototype 2MoMA The Museum of Modern Art
This approach is showing signs of promise, though still being researched and developed. Our hope is that this will improve both accuracy and citation quality.
Additionally, this new tool will allow fact-checking by tying the generated output back to the source pages in the catalog and the Museum collection management database.
The Olive Trees (1889) by Vincent van GoghMoMA The Museum of Modern Art
Lessons Learned
1. It’s ok if the latency is longer than your average chatbot.
2. Accuracy really matters.
If a work is attributed to the wrong artist, this is considered a critical error and the entire write up will be in question. Museum writers will prefer to research manually than trust the output of an AI once they observe even a single critical error.
3. Details matter too.
If the writeup is too general and lacking in detail, it is not enough to use even as a first draft.
4. Citations are very important.
Accurate citations enable the writer to research those specific pages and come up with a better writeup when needed. As AI tools continue to improve, we expect this facet to improve as well.
While we don’t ever want AI to replace the important work of curators and art historians, we see tremendous potential in the technology’s ability to leverage existing scholarship to help save time in the Museum workflow.
You are all set!
Your first Culture Weekly will arrive this week.