University of Michigan School of Information faculty and PhD students are creating and sharing knowledge that helps build a better world. Here are some of their recent publications. 

Publications 

Taking a Closer Look: Refining Trust and its Impact in HRI

Companion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction, March 2024

Jiayuan Dong, Connor Esterwood, Xin Ye, Jennifer J. Mitchell, Wonse Jo, Lionel P. Robert, Chung Hyuk Park, Myounghoon Jeon

As robots are rapidly integrated into our daily lives, enhancing trust between humans and robots is crucial to accepting robots and the effectiveness of human-robot interaction (HRI). This workshop aims to provide a platform for HRI researchers, practitioners, and students from diverse disciplines to engage in a discussion to define/refine the construct, understand different factors that influence trust in HRI and their impacts, and measure different aspects of trust. The workshop will contribute to building a solid research community on this crucial construct and guiding future research and development of better human-robot interaction. 

Handling Name Errors of a BERT-Based Text De-Identification System: Insights from Stratified Sampling and Markov-based Pseudonymization

Proceedings of the Workshop on Computational Approaches to Language Data Pseudonymization, March 2024

Dalton Simancek, V.G.Vinod Vydiswaran

Missed recognition of named entities while deidentifying clinical narratives poses a critical challenge in protecting patient-sensitive health information. Mitigating name recognition errors is essential to minimize risk of patient reidentification. In this paper, we emphasize the need for stratified sampling and enhanced contextual considerations concerning Name tokens using a fine-tuned Longformer BERT model for clinical text de-identification. We introduce a Hidden in Plain Sight (HIPS) Markov-based replacement technique for names to mask name recognition misses, leading to a significant reduction in name leakage rates. Our experimental results underscore the impact on addressing name recognition challenges in BERT-based deidentification systems for heightened privacy protection in electronic health records.

“What are you doing, TikTok?” : How Marginalized Social Media Users Perceive, Theorize, and “Prove” Shadowbanning

Proceedings of the ACM on Human-Computer Interaction, March 2024

Daniel Delmonaco, Samuel Mayworm, Hibby Thach, Josh Guberman, Aurelia Augusta, Oliver L. Haimson

Shadowbanning is a unique content moderation strategy receiving recent media attention for the ways it impacts marginalized social media users and communities. Social media companies often deny this content moderation practice despite user experiences online. In this paper, we use qualitative surveys and interviews to understand how marginalized social media users make sense of shadowbanning, develop folk theories about shadowbanning, and attempt to prove its occurrence. We find that marginalized social media users collaboratively develop and test algorithmic folk theories to make sense of their unclear experiences with shadowbanning. Participants reported direct consequences of shadowbanning, including frustration, decreased engagement, the inability to post specific content, and potential financial implications. They reported holding negative perceptions of platforms where they experienced shadowbanning, sometimes attributing their shadowbans to platforms’ deliberate suppression of marginalized users’ content. Some marginalized social media users acted on their theories by adapting their social media behavior to avoid potential shadowbans. We contribute collaborative algorithm investigation: a new concept describing social media users’ strategies of collaboratively developing and testing algorithmic folk theories. Finally, we present design and policy recommendations for addressing shadowbanning and its potential harms.

The Digital Therapeutics Real-World Evidence Framework: An Approach for Guiding Evidence-Based Digital Therapeutics Design, Development, Testing, and Monitoring

Journal of Medical Internet Research, March 2024 

Meelim Kim, Kevin Patrick, Camille Nebeker, Job Godino, Spencer Stein, Predrag Klasnja, Olga Perski, Clare Viglione, Aaron Coleman, Eric Hekler

Digital therapeutics (DTx) are a promising way to provide safe, effective, accessible, sustainable, scalable, and equitable approaches to advance individual and population health. However, developing and deploying DTx is inherently complex in that DTx includes multiple interacting components, such as tools to support activities like medication adherence, health behavior goal-setting or self-monitoring, and algorithms that adapt the provision of these according to individual needs that may change over time. While myriad frameworks exist for different phases of DTx development, no single framework exists to guide evidence production for DTx across its full life cycle, from initial DTx development to long-term use. To fill this gap, we propose the DTx real-world evidence (RWE) framework as a pragmatic, iterative, milestone-driven approach for developing DTx. The DTx RWE framework is derived from the 4-phase development model used for behavioral interventions, but it includes key adaptations that are specific to the unique characteristics of DTx. To ensure the highest level of fidelity to the needs of users, the framework also incorporates real-world data (RWD) across the entire life cycle of DTx development and use. The DTx RWE framework is intended for any group interested in developing and deploying DTx in real-world contexts, including those in industry, health care, public health, and academia. Moreover, entities that fund research that supports the development of DTx and agencies that regulate DTx might find the DTx RWE framework useful as they endeavor to improve how DTxcan advance individual and population health.

From information access to production: New perspectives on addressing information inequity in our digital information ecosystem

Journal of the Association for Information Science and Technology, February 2024

Chelsea Peterson-Salahuddin 

Changes in our informational environment have brought new challenges and opportunities to address systemic issues of information inequity. Thus, when addressing systemic issues of information inequity, it is important to address is not only from the perspective of information access, as it is often considered in information science, but also from the perspective of how information objects are constructed and produced. This essay brings concerns within information science into discussion with journalism studies and critical technology studies to consider: (1) how the production of information, through the case of main-stream journalism, can create information inequity within information repre-sentations, and (2) how the dissemination and retrieval of this journalistic information through algorithmically-mediated online information systems,specifically social media and search platforms, can replicate and reinforce information inequity within a larger information ecosystem. Thus, this essay uses an interdisciplinary lens to suggest new approaches to holistically address information inequity, putting forth a conceptual framework with actionable steps to create a more equitable information ecosystem. 

Technology for transgender healthcare: Access, precarity & community care

Social Science and Medicine, February 2024

Avery R. Everhart, Kristi E. Gamarel, Oliver L. Haimson 

While much of the transgender health literature has focused on poor health outcomes, less research has examined how trans people find reliable information on, and actually go about accessing, gender-affirming healthcare. Through qualitative interviews with creators of trans technologies, that is, technologies designed to address problems that trans people face, we found that digital technologies have become important tools for proliferating access to gender-affirming care and related health information. We found that technologists often employed different processes for creating their technologies, but they coalesced around the goal of enabling and increasing access to gender-affirming care. Creators of trans health technologies also encountered precarious conditions for creating and maintaining their technologies, including regional gaps left by national resources focused on the US east and west coasts. Findings demonstrated that trans tech creators were motivated to create and maintain these technologies as a means of caring for one another and forming trans communities in spite of the precarious conditions trans people face living under systemic oppression.

Spot Report: An Open-Source and Real-Time Secondary Task for Human-Robot Interaction User Experiments

HRI '24: Companion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction, February 2024

Arsha Ali, Rohit Banerjee, Wonse Jo, Samantha Dubrow, Kayla Riegner, Jonathon M. Smereka, Lionel P. Robert Jr., Dawn M. Tilbury

The human-robot interaction (HRI) community is interested in a range of research questions, many of which are investigated through user experiments. Robots that occasionally require human input allow for humans to engage in secondary tasks. However, few secondary tasks transmit data in real-time and are openly available, which hinders interaction with the primary task and limits the ability of the community to build upon others' research. Also, the need for a secondary task relevant to the military was identified by subject matter experts. To address these concerns, this paper presents the spot report task as an open-source secondary task with real-time communication for use in HRI experiments. The spot report task requires counting target objects in static images. This paper includes details of the spot report task and real-time communication with a primary task. We developed the spot report task considering the military domain, but the software architecture is domain-independent. We hope others can leverage the spot report task in their own user experiments. The software files are available at https://github.com/UMich-MAVRIC/SpotReport.

A Turing test of whether AI chatbots are behaviorally similar to humans

Proceedings of the National Academy of Sciences of the United States of America, February 2024

Qiaozhu Mei, Yutong Xie, Walter Yuan, Matthew O. Jackson

We administer a Turing test to AI chatbots. We examine how chatbots behave in a suite of classic behavioral games that are designed to elicit characteristics such as trust, fairness, risk-aversion, cooperation, etc., as well as how they respond to a traditional Big-5 psychological survey that measures personality traits. ChatGPT-4 exhibits behavioral and personality traits that are statistically indistinguishable from a random human from tens of thousands of human subjects from more than 50 countries. Chatbots also modify their behavior based on previous experience and contexts “as if” they were learning from the interactions and change their behavior in response to different framings of the same strategic situation. Their behaviors are often distinct from average and modal human behaviors, in which case they tend to behave on the more altruistic and cooperative end of the distribution. We estimate that they act as if they are maximizing an average of their own and partner’s payoffs.

Exploring the Current Landscape of Trans Technology Design

 #AoIR2023: The 24th Annual Conference of the Association of Internet Researchers, December 2023. 

Oliver Haimson

Transgender people face substantial challenges in the world, such as discrimination, harassment, and lack of access to basic resources. Some of these challenges could be addressed to some extent with technology. In this paper I examine the world of trans technology design through interviews with 115 creators of trans technologies: apps, games, health resources, and other types of technology. I demonstrate that trans technology design processes are often deeply personal, and focus on the technology creator’s needs and desires. Thus, trans technology design can be empowering because technology creators have agency to create tools they need to navigate the world. However, in some cases when trans communities are not involved in design processes, this can lead to overly individualistic design that speaks primarily to more privileged trans people’s needs.

Pre-prints, Working Papers, Reports, Workshops and Talks

Imagine a dragon made of seaweed: How images enhance learning in Wikipedia

arXiv, March 2024

Anita Silva, Maria Tracy, Katharina Reinecke, Eytan Adar, Miriam Redi

Though images are ubiquitous across Wikipedia, it is not obvious that the image choices optimally support learning. When well selected, images can enhance learning by dual coding, complementing, or supporting articles. When chosen poorly, images can mislead, distract, and confuse. We developed a large dataset containing 470 questions & answers to 94 Wikipedia articles with images on a wide range of topics. Through an online experiment (n=704), we determined whether the images displayed alongside the text of the article are effective in helping readers understand and learn. For certain tasks, such as learning to identify targets visually (e.g., which of these pictures is a “gujia”?)1 , article images significantly improve accuracy. Images did not significantly improve general knowledge questions (e.g., where are gujia from?). Most interestingly, only some images helped with visual knowledge questions (e.g., what shape is a gujia?). Using our findings, we reflect on the implications for editors and tools to support image selection

Born Accessible Data Science and Visualization Courses: Challenges of Developing Curriculum to be Taught by Blind Instructors to Blind Students

arXiv, March 2024 

JooYoung Seo, Sile O’Modhrain, Yilin Xia, Sanchita Kamath, Bongshin Lee, James M. Coughlan

While recent years have seen a growing interest in accessible visualization tools and techniques for blind people, little attention is paid to the learning opportunities and teaching strategies of data science and visualization tailored for blind individuals. Whereas the former focuses on the accessibility issues of data visualization tools, the latter is concerned with the learnability of concepts and skills for data science and visualization. In this paper, we present novel approaches to teaching data science and visualization to blind students in an online setting. Taught by blind instructors, nine blind learners having a wide range of professional backgrounds participated in a two-week summer course. We describe the course design, teaching strategies, and learning outcomes. We also discuss the challenges and opportunities of teaching data science and visualization to blind students. Our work contributes to the growing body of knowledge on accessible data science and visualization education, and provides insights into the design of online courses for blind students. 

WSDM 2024 Workshop on Large Language Models for Individuals, Groups, and Society

Proceedings of the 17th ACM International Conference on Web Search and Data Mining, March 2024

Michael Bendersky, Cheng Li, Qiaozhu Mei, Vanessa Murdock, Jie Tang, Hongning Wang, Hamed Zamani, Mingyang Zhang

This workshop discusses the cutting-edge developments in research and applications of personalizing large language models (LLMs) and adapting them to the demands of diverse user populations and societal needs. The full-day workshop includes several keynotes and invited talks, a poster session and a panel discussion. 

Automated Detection and Analysis of Data Practices Using A Real-World Corpus

arXiv, February 2024

Mukund Srinath, Pranav Venkit, Maria Badillo, Florian Schaub, C. Lee Giles, Shomir Wilson

Privacy policies are crucial for informing users about data practices, yet their length and complexity often deter users from reading them. In this paper, we propose an automated approach to identify and visualize data practices within privacy policies at different levels of detail. Leveraging crowd-sourced annotations from the ToS;DR platform, we experiment with various methods to match policy excerpts with predefined data practice descriptions. We further conduct a case study to evaluate our approach on a real-world policy, demonstrating its effectiveness in simplifying complex policies. Experiments show that our approach accurately matches data practice descriptions with policy excerpts, facilitating the presentation of simplified privacy information to users.

Unlocking the ‘Why’ of Buying: Introducing a New Dataset and Benchmark for Purchase Reason and Post-Purchase Experience

arXiv, February 2024

Tao Chen, Siqi Zuo, Cheng Li, Mingyang Zhang, Qiaozhu Mei, Michael Bendersky

Explanations are crucial for enhancing user trust and understanding within modern recommendation systems. To build truly explainable systems, we need high-quality datasets that elucidate why users make choices. While previous efforts have focused on extracting users’ postpurchase sentiment in reviews, they ignore the reasons behind the decision to buy. In our work, we propose a novel purchase reason explanation task. To this end, we introduce an LLM-based approach to generate a dataset that consists of textual explanations of why real users make certain purchase decisions. We induce LLMs to explicitly distinguish between the reasons behind purchasing a product and the experience after the purchase in a user review. An automated, LLM-driven evaluation, as well as a small scale human evaluation, confirms the effectiveness of our approach to obtaining high-quality, personalized explanations. We benchmark this dataset on two personalized explanation generation tasks. We release the code and prompts to spur further research

Spot Check Equivalence: an Interpretable Metric for Information Elicitation Mechanisms

arXiv, February 2024

Shengwei Xu, Yichi Zhang, Paul Resnick, Grant Schoenebeck

Because high-quality data is like oxygen for AI systems, effectively eliciting information from crowdsourcing workers has become a first-order problem for developing high-performance machine learning algorithms. Two prevalent paradigms, spot-checking and peer prediction, enable the design of mechanisms to evaluate and incentivize highquality data from human labelers. So far, at least three metrics have been proposed to compare the performances of these techniques [33, 8, 3]. However, different metrics lead to divergent and even contradictory results in various contexts. In this paper, we harmonize these divergent stories, showing that two of these metrics are actually the same within certain contexts and explain the divergence of the third. Moreover, we unify these different contexts by introducing Spot Check Equivalence, which offers an interpretable metric for the effectiveness of a peer prediction mechanism. Finally, we present two approaches to compute spot check equivalence in various contexts, where simulation results verify the effectiveness of our proposed metric.

AI at Africa's Crossroads: Extractive or Generative Future?

Proceedings of the International Workshop on Social Impact of AI for Africa, February 2024

Ron Eglash

In African traditions, the crossroads is where the trickster makes his/her appearance. Eshu, Legba, Anansi and others create complexity when our decisions fold back on themselves. AI has created yet another crossroads, and again the trickster brings surprises. What might have seemed like Africa’s worst challenges-“underdeveloped” from the colonial perspective-could be the basis by which computational aids can facilitate more sustainable and egalitarian futures. Blending the heritage algorithms of Africa’s past with full stack decolonization can guide us through the crossroads, on the path towards generative justice.

Divide-or-Conquer? Which Part Should You Distill Your LLM?

arXiv, February 2024

Zhuofeng Wu, He Bai, Aonan Zhang, Jiatao Gu, VG Vinod Vydiswaran, Navdeep Jaitly, Yizhe Zhang

Recent methods have demonstrated that Large Language Models (LLMs) can solve reasoning tasks better when they are encouraged to solve subtasks of the main task first. In this paper we devise a similar strategy that breaks down reasoning tasks into a problem decomposition phase and a problem solving phase and show that the strategy is able to outperform a single stage solution. Further, we hypothesize that the decomposition should be easier to distill into a smaller model compared to the problem solving because the latter requires large amounts of domain knowledge while the former only requires learning general problem solving strategies. We propose methods to distill these two capabilities and evaluate their impact on reasoning outcomes and inference cost. We find that we can distill the problem decomposition phase and at the same time achieve good generalization across tasks, datasets, and models. However, it is harder to distill the problem solving capability without losing performance and the resulting distilled model struggles with generalization. These results indicate that by using smaller, distilled problem decomposition models in combination with problem solving LLMs we can achieve reasoning with cost-efficient inference and local adaptation.