Who decides whether AI accurately represents a culture?

What makes an AI-generated image recognizably Mongolian? Which colors and ornaments make a piece of clothing distinctly Mongolian? And who gets to decide whether an AI-generated image accurately represents a culture?

These questions are explored in a new paper by University of Michigan School of Information doctoral candidate Nasanbayar Ulzii-Orshikh, assistant professor Justine Zhang and professor Mark Ackerman.

Their paper, “The curious case of culturally-aware AI: The need for CSCW methods in AI evaluations,” examines how Mongolians respond to AI-generated images depicting their history, national identity and cultural traditions.

The researchers analyzed 2,264 Facebook posts and comments about generative AI images in Mongolia and identified three prominent areas of discussion: the appearance of Mongolian people and historical figures, traditional ornaments and colors, and designs for Mongolia’s Olympic uniforms.

Across these conversations, people were not simply evaluating whether the images included the correct visual details. They were also negotiating larger questions about how Mongolia should represent itself and whether AI-generated images made the country recognizable to a global audience.

“Does this image actually make us legible as a nation, as people, for the global public?” Ulzii-Orshikh asks. “Where that question is concerned, it’s not about referring to some existing precedent or some existing referent of images. It’s about intentionally producing their own visible social difference.”

Beyond computational accuracy

Many efforts to make AI culturally aware, or better at depicting different cultures, focus on computational accuracy: whether a system produces an image with the features researchers or developers associate with a particular culture.

But the UMSI researchers argue that this approach can flatten culture into a fixed collection of measurable traits.

“Is this somehow Mongolian based on a set of features that we can easily measure?” Ackerman says. “That’s not enough. You miss a lot if you do that. You’re missing what people feel is true.”

Those debates appeared throughout the Facebook discussions that the researchers analyzed. Users argued how AI depicted Genghis Khan and other Mongolian figures, including what facial features should be read as Mongolian. They scrutinized the placement of traditional ornaments on AI-generated boots and clothing, as well as whether the colors used in the images carried the appropriate cultural meaning.

The 2024 Olympics offered another case study. After Mongolia unveiled its official uniforms, social media users created their own designs using generative AI. The resulting images became part of a broader conversation about how the country should present itself on an international stage.

The researchers found that comparing an AI-generated image with a single authoritative reference could not settle these questions.. The conversations reflected Mongolia’s complex history, from the era of the khans through socialism and into the country’s contemporary relationship with globalization.

“I think this work, and our ongoing work, is trying to trouble the way much of the literature thinks about accuracy,” Zhang says. 

Who gets to define cultural accuracy?

Researchers working on culturally aware AI have developed methods for reducing representational harms, preventing the erasure of underrepresented communities and incorporating community perspectives into AI evaluations.

Some approaches attempt to identify the criteria a community uses to evaluate an image, turn those criteria into a rubric and use that rubric to automatically assess AI outputs.

While these methods can improve representation, the UMSI researchers caution that they may still treat culture as a stable object that can be measured.

“We share an understanding that culture is not a thing ‘out there’ to be measured against,” Zhang says. “It is something people do, and it pulls in all kinds of complicated factors.”

The study emphasizes the importance of qualitative methods from computer-supported cooperative work, or CSCW, that allow researchers to examine how people actively interpret, debate and construct culture through their interactions with technology.

This approach also moves beyond a familiar narrative in which technologies are developed in the West and later introduced to communities elsewhere. Instead of treating Mongolian users as passive recipients of generative AI, the study examines how they use the technology to negotiate social differences and shape how their nation is represented.

The researchers describe the paper as both exploratory and “sensitizing.” It does not offer a single formula for building culturally aware AI. Rather, it asks AI researchers and developers to recognize what computational measures of accuracy leave out.

“We’re not troubling the idea of computational accuracy just to be obnoxious,” Ackerman says. “We’re trying to figure out what else people need to be looking at.”

RELATED 

“The curious case of culturally-aware AI: The need for CSCW methods in AI evaluations” was published in the Proceedings of the European Society for Socially Embedded Technologies (EUSSET) Conference on Computer-Supported Cooperative Work.

Learn more about Nasanbayar Ulzii-Orshikh, Justine Zhang and Mark Ackerman by visiting their UMSI faculty profiles.