University of Michigan School of Information
Afrofuturism, Bullhorns and Robotics: UMSI Research Roundup
Wednesday, 07/26/2023
University of Michigan School of Information faculty and PhD students are creating and sharing knowledge that helps build a better world. Here are some of their recent publications.
Participatory Noticing through Photovoice: Engaging Arts- and Community-Based Approaches in Design Research
Proceedings of the 2023 ACM Designing Interactive Systems Conference, July 2023
Alex Jiahong Lu, Shruti Sannon, Cameron Moy, Savana Brewer, Jaye Green, Kisha N. Jackson, Daivon Reeder, Camaria Wafer, Mark S. Ackerman, Tawanna R. Dillahunt
*Awarded Best Paper from SIGCHI*
Noticing differently commits to stepping out of familiar reference frameworks while attending to oft-neglected actors, relations, and ways of knowing for design. Photovoice is an arts- and community based participatory approach allowing individuals to communicate their lives and stories about pressing community concerns through photography. This paper bridges photovoice and the commitment to noticing in HCI and design through a photovoice project with Detroit residents on safety and surveillance. The photovoice process—alongside the production, reflection, and dissemination of photographs—makes residents’ everyday situations legible and sensible, allowing both community members and researchers to orient to and engage with multiple viewpoints, sensibilities, and temporal trajectories. This process confronts the invisibility of both the sociotechnical infrastructures (in our case, surveillance infrastructures) and minoritized communities’ relational ontologies. By advocating participatory noticing in design research, we show the opportunities for adopting arts- and community-based participatory approaches in decentering dominant ways of knowing and seeing, while at the same time fostering community capacity and relations for future potentialities.
Toward work’s new futures: Editors’ Introduction to Technology and the Future of Work special issue
Journal of Computer-Mediated Communication, July 2023
Nancy Baym, Nicole B Ellison
This special issue is based in the belief that theoretically informed, methodologically diverse, and sociotechnically inspired research is our best approach for understanding contemporary entanglements between the technological and social aspects of work, and for grappling with what that means for our futures. In this Editors’ Introduction to JCMC’s Technology and the Future of Work special issue, we synthesize emergent themes across the eleven papers included and reflect on productive analytic lenses for anticipating how technologies may shape social and work practices, and vice versa. We identify four themes woven across the papers—visibility, relationships, boundaries, and power—and explicate some of the ways that social, technical, temporal, and communicative dimensions of work emerge across a variety of work contexts. Together, these papers highlight the creative, sense-making, and collaborative dynamics of the technologically infused workplace while acknowledging the amorphous nature of work and place, past, present and future.
Navigating the empty shell: the role of articulation work in platform structures
Journal of Computer-Mediated Communication, June 2023
This article explores platform workers’ strategies for producing sustainable, quality services within platform structures that simultaneously over- and under-determine their work. We present findings from interviews with U.S.-based mental health professionals (n = 48) working on teletherapy platforms. These therapists describe navigating both the presence of platformic controls and the absence of features supporting professional best practices and regulatory requirements. We describe this absence as the “empty shell” characteristic of platforms and argue that it is a central technique through which platforms create scale. Our findings detail the communicative strategies therapists employ to navigate the empty shell and provide quality care to their clients. These strategies can be seen as a form of “articulation work,” a concept drawn from the sociology of work. Attending to articulation work in an emerging platform labor context, such as teletherapy, contributes to our understanding of the politics of platforms.
Human, Machine, or Hybrid? Using Anthropomorphism to Conceptualize Trust in Robots
Proceedings of the 26th Americas Conference on Information Systems, August 2023
Samia Bhatti, Lionel P. Robert
While robots appear to be more and more human-like in form and function, they are still machines. People can hence perceive them as humans or machines. With varying human-like designs and user perceptions, there is much confusion about how to measure trust in human-robot relationships. While some researchers use human-like trusting beliefs to conceptualize trust, others use machine-like trusting beliefs to do the same. In this paper, we present a conceptual model and related research propositions to help researchers determine the correct conceptualization of trust for human-robot interaction. We propose that anthropomorphism, or perceptions of humanness about the robot, can dictate the conceptualization of trust in human-robot relationships.
Scaffolding to Support Liberal Arts Students Learning to Program on Photographs
Digital photographs are part of liberal arts students' classes (e.g., art, history, and production classes in film and television) and their daily smartphone-based life, in apps like Instagram and Snapchat. Building image filters can be a relevant and engaging context into using computing for humanities students. We have designed a new course for introducing computing in terms of creative expression. We use a scaffolded sequence of programming languages and activities to explore computing on photographs: (a) a teaspoon language for generating image filters, (b) a set of custom Snap blocks for even more sophisticated image filters, and (c) an ebook activity for mapping from Snap to Python. Each stage takes less than 10 minutes to introduce, with a wide variety of possible student activities (for in-class active learning or for later homework). While the tools build on each other, the earliest stage (the teaspoon language) could be used within a single class session in other liberal arts courses.
Feasibility, Acceptability, and Potential Impact of a Novel mHealth App for Smokers Ambivalent About Quitting: Randomized Pilot Study
JMIR mHealth and uHealth, June 2023
Jennifer B McClure, Jaimee L Heffner, Chloe Krakauer, Sophia Mun, Predrag Klasnja, Sheryl L Catz
Background:Most smokers are ambivalent about quitting—they want to quit someday, but not now. Interventions are needed that can engage ambivalent smokers, build their motivation for quitting, and support future quit attempts. Mobile health (mHealth) apps offer a cost-effective platform for such interventions, but research is needed to inform their optimal design and assess their acceptability, feasibility, and potential effectiveness.
Objective: This study aims to assess the feasibility, acceptability, and potential impact of a novel mHealth app for smokers who want to quit smoking someday but are ambivalent about quitting in the near term.
Methods: We enrolled adults across the United States who smoked more than 10 cigarettes a day and were ambivalent about quitting (n=60). Participants were randomly assigned to 1 of 2 versions of the GEMS app: standard care (SC) versus enhanced care (EC). Both had a similar design and identical evidence-based, best-practice smoking cessation advice and resources, including the ability to earn free nicotine patches. EC also included a series of exercises called experiments designed to help ambivalent smokers clarify their goals, strengthen their motivation, and learn important behavioral skills for changing smoking behavior without making a commitment to quit. Outcomes were analyzed using automated app data and self-reported surveys at 1 and 3 months post enrollment.
Results: Participants who installed the app (57/60, 95%) were largely female, White, socioeconomically disadvantaged, and highly nicotine dependent. As expected, key outcomes trended in favor of the EC group. Compared to SC users, EC participants had greater engagement (mean sessions 19.9 for EC vs 7.3 for SC). An intentional quit attempt was reported by 39.3% (11/28) of EC users and 37.9% (11/29) of SC users. Seven-day point prevalence smoking abstinence at the 3-month follow-up was reported by 14.7% (4/28) of EC users and 6.9% (2/29) of SC users. Among participants who earned a free trial of nicotine replacement therapy based on their app usage, 36.4% (8/22) of EC participants and 11.1% (2/18) of SC participants requested the treatment. A total of 17.9% (5/28) of EC and 3.4% (1/29) of SC participants used an in-app feature to access a free tobacco quitline. Other metrics were also promising. EC participants completed an average of 6.9 (SD 3.1) out of 9 experiments. Median helpfulness ratings for completed experiments ranged from 3 to 4 on a 5-point scale. Finally, satisfaction with both app versions was very good (mean 4.1 on a 5-point Likert scale) and 95.3% (41/43) of all respondents would recommend their app version to others.
Conclusions: Ambivalent smokers were receptive to the app-based intervention, but the EC version, which combined best-practice cessation advice with self-paced, experiential exercises, was associated with greater use and evidence of behavior change. Further development and evaluation of the EC program is warranted.
Idiographic Dynamic Modeling for Behavioral Interventions with Mixed Data Partitioning and Discrete Simultaneous Perturbation Stochastic Approximation
Institute of Electrical and Electronics Engineers, June 2023
Rachael T. Kha, Daniel E. Rivera, Predrag Klasnja, Eric Hekler
This paper presents the use of discrete simultaneous perturbation stochastic approximation (DSPSA) as a routine method to efficiently determine features and parameters of idiographic (i.e. single subject) dynamic models for personalized behavioral interventions using various partitions of estimation and validation data. DSPSA is demonstrated as a valuable method to search over model features and regressor orders of AutoRegressive with eXogenous input estimated models using participant data from Just Walk (a behavioral intervention to promote physical activity in sedentary adults); results of DSPSA are compared to those of exhaustive search. In Just Walk, DSPSA efficiently and quickly estimates models of walking behavior, which can then be used to develop control systems to optimize the impacts of behavioral interventions. The use of DSPSA to evaluate models using various partitions of individual data into estimation and validation data sets also highlights data partitioning as an important feature of idiographic modeling that should be carefully considered.
LHS712EE at BioLaySumm 2023: Using BART and LED to summarize biomedical research articles
The 22nd Workshop on Biomedical Natural Language Processing and BioNLP Shared Tasks, July 2023
Quancheng Liu, Xiheng Ren, V.G.Vinod Vydiswaran
As part of our participation in BioLaySumm 2023, we explored the use of large language models (LLMs) to automatically generate concise and readable summaries of biomedical research articles. We utilized pre-trained LLMs to fine-tune our summarization models on two provided datasets, and adapt them to the shared task within the constraints of training time and computational power. Our final models achieved very high relevance and factuality scores on the test set, and ranked among the top five models in the overall performance.
Eliciting Alternative Economic Futures with Working-Class Detroiters: Centering Afrofuturism in Speculative Design
DIS '23: Proceedings of the 2023 ACM Designing Interactive Systems Conference, July 2023
Tawanna R. Dillahunt, Alex Jiahong Lu, Joanna Velazquez
Economic crises such as the global recession and financial crisis of 2007 and 2008 and the Coronavirus disease (COVID-19) pandemic, have elevated new forms of economic cooperation. Supporting efforts in finding alternatives to capitalism requires understanding the role of design in imagining alternative economic futures and reaching those most harmed by current capitalistic models. Through a collaboration between a community organization in Detroit and a team of university researchers, we hosted and facilitated a five-week workshop series with Black and Brown working-class Detroiters where they collectively imagined alternative economic futures using speculative design. They proposed Community Capitalism, Childcare Collectives, and Village-Based Childcare as alternative economy concepts from the workshops and described their unique characteristics and traits of love, care, and inclusion. Aligning with generative justice frameworks, Detroiters prioritized sustainable families and communities. We contribute an understanding of technology’s role in the imagined economic futures, a discussion of what this means for community-involved governance, and a push for centering Afrofuturism in speculative design approaches to foster futures literacy.
How Bereaved Parents Make Meaning from Photos for Continuing Bonds
DIS '23: Proceedings of the 2023 ACM Designing Interactive Systems Conference, July 2023
Ju Yeon Jung, Tom Steinberger, John L. King, Mark S. Ackerman
The bereaved interact with photos of the deceased in various ways to maintain continuing bonds. Extant research in HCI and interaction design suggests leveraging data on photos’ contexts (e.g., time and place) to support photo use by the bereaved. Extending this research, we interviewed 17 bereaved parents to examine how the bereaved might also interact with photos’ content (e.g., color schemes, facial expressions). We characterize their interactions as processes of meaning-making that enabled them to keep their fnite photos relevant and maintain continuing bonds with their children. By identifying meaning-making processes in terms of interactions with both photos’ content and contexts, we open up possibilities for supporting photo use for the bereaved that leverage content-related data. Our fndings more broadly highlight the importance of accounting for subjective meaning-making processes in data-driven approaches to photo use in HCI and computer vision.
Bridging Nations: Quantifying the Role of Multilinguals in Communication on Social Media
Proceedings of the 17th International AAAI Conference on Web and Social Media (ACL 2023), June 2023
Julia Mendelsohn, Sayan Ghosh, David Jurgens, Ceren Budak
*Awarded Outstanding Methodology Paper Award for ICWSM 2023*
Social media enables the rapid spread of many kinds of information, from pop culture memes to social movements. However, little is known about how information crosses linguistic boundaries. We apply causal inference techniques on the European Twitter network to quantify the structural role and communication influence of multilingual users in cross-lingual information exchange. Overall, multilinguals play an essential role; posting in multiple languages increases betweenness centrality by 13%, and having a multilingual network neighbor increases monolinguals’ odds of sharing domains and hashtags from another language 16-fold and 4-fold, respectively. We further show that multilinguals have a greater impact on diffusing information is less accessible to their monolingual compatriots, such as information from far-away countries and content about regional politics, nascent social movements, and job opportunities. By highlighting information exchange across borders, this work sheds light on a crucial component of how information and ideas spread around the world.
From Dogwhistles to Bullhorns: Unveiling Coded Rhetoric with Language Models
arXiv, May 2023
Julia Mendelsohn, Ronan Le Bras, Yejin Choi, Maarten Sap
Dogwhistles are coded expressions that simultaneously convey one meaning to a broad audience and a second one, often hateful or provocative, to a narrow in-group; they are deployed to evade both political repercussions and algorithmic content moderation. For example, in the sentence 'we need to end the cosmopolitan experiment,' the word 'cosmopolitan' likely means 'worldly' to many, but secretly means 'Jewish' to a select few. We present the first large-scale computational investigation of dogwhistles. We develop a typology of dogwhistles, curate the largest-to-date glossary of over 300 dogwhistles with rich contextual information and examples, and analyze their usage in historical U.S. politicians' speeches. We then assess whether a large language model (GPT-3) can identify dogwhistles and their meanings, and find that GPT-3's performance varies widely across types of dogwhistles and targeted groups. Finally, we show that harmful content containing dogwhistles avoids toxicity detection, highlighting online risks of such coded language. This work sheds light on the theoretical and applied importance of dogwhistles in both NLP and computational social science, and provides resources for future research in modeling dogwhistles and mitigating their online harms.
Social Robotics Business and Computing: Editorial for Special Issue of Information Systems Frontiers
Information Systems Frontiers, July 2023
Lionel P. Robert Jr., Marcelo Fantinato, Sangseok You, Patrick C. K. Hung
Our society is moving towards a world with social robots, where humanoid and anthropomorphic machines can use their artificial intelligence capabilities to engage in complex social interactions and provide valuable services to humans. Unlike traditional robots, social robots are robotic technologies designed to focus on assisting humans through social interactions, rather than just physical tasks (Feil-Seifer and Matari´c, 2005). Designed to provide support and assistance, social robots can meet the specific needs of individuals in their social environments and interaction contexts. Social robots may serve myriad applications, leading to a variety of specific names such as companion robots, service robots, education robots, and therapeutic robots, and so on, each tailored to specific purposes and contexts of use. Following the examples of de Araujo et al. (2022), social robots can be employed, e.g., as service robots to support communication for isolated patients in hospitals (Arce et al., 2022), as companion robots for vulnerable people as lonely older adults (Lewis, 2014), as therapeutic robots for children with autism spectrum disorder (Cabibihan et al., 2013), and as educational robots for special teaching-learning approaches (Kim et al., 2015).
Social robots are innovative autonomous systems that feature a physical robot component, typically designed with a humanoid or anthropomorphic form (Sandini, 2019). They are connected to online services through a network infrastructure, which allows them to go beyond traditional robotic functionality (de Araujo et al., 2022). Equipped with devices such as cameras, microphones, and sensors, they can capture a user’s physical activity state (e.g., walking, standing, running, etc.), store personalized information (e.g., face, voice, location, activity pattern, etc.), and interact with humans through communication and service delivery (Wirtz et al., 2018). Empowered with characteristics such as speech, gestures, and eye-gaze, which can be customized to a particular user or social context, social robots can behave like natural partners engaging humans in social interactions (Robert, et al., 2020). Thus, social robots use body language that corresponds to the information being expressed verbally, simulating human behavior and attitude. They can be programmed to interact with humans by performing tasks that adhere to specific social cues and rules. Examples of social robots are SoftBank Robotics’ Pepper, Romeo and NAO, ASUS’s Zenbo, Misty Robotics’ Misty II, Blue Frog Robotics’ Buddy, Human Robotics’ Robios, and Ubtech Robotics’ Lynx.
Social robots often come equipped with Artificial Intelligence (AI) capabilities that enable features such as human facial, voice, and emotion recognition, providing for more immersive and interactive experiences (Sandini, et al., 2018; Wu, 2015; Chen, 2021). Along with AI, these robots rely on network data transmission protocols and cloud data storage to function optimally. They also often feature human-like personalities and characteristics to enhance human-machine communication (Wykowska et al., 2015; Sorrentino et al., 2021). The range of activities that social robots can perform depends on both the functionalities developed specifically for them and on the creativity embedded in their human-robot interaction design. Providing a fully functional social robot often requires the use of disruptive technologies, spanning both hardware and software aspects. While these disruptive technologies can bring significant benefits to society, they can also raise concerns related to privacy and confidentiality, discrimination and prejudice, freedom of expression and choice, among other issues (Denning et al., 2009; Lutz et al., 2019; Howard et al., 2018).
Unfortunately, advances in social robotics have far outpaced our understanding of the social implications of their use (You et al., 2018, 2022). Human-Robot Interaction (HRI) is a research area whose purpose is to address the large and important questions at the heart of designing and evaluating robots for use by or with humans. This special issue aims to address relevant issues of social robotics, from both a business and computing perspective, i.e., treating this area from a socio-technical perspective.
Researchers’ Experiences in Analyzing Privacy Policies: Challenges and Opportunities
Proceedings on Privacy Enhancing Technologies, July 2023
Abraham Mhaidli, Selin Fidan, An Doan, Gina Herakovic, Mukund Srinath, Lee Matheson, Shomir Wilson, Florian Schaub
Companies’ privacy policies and their contents are being analyzed for many reasons, including to assess the readability, usability, and utility of privacy policies; to extract and analyze data practices of apps and websites; to assess compliance of companies with relevant laws and their own privacy policies, and to develop tools and machine learning models to summarize and read policies. Despite the importance and interest in studying privacy policies from researchers, regulators, and privacy activists, few best practices or approaches have emerged and infrastructure and tool support is scarce or scattered. In order to provide insight into how researchers study privacy policies and the challenges they face when doing so, we conducted 26 interviews with researchers from various disciplines who have conducted research on privacy policies. We provide insights on a range of challenges around policy selection, policy retrieval, and policy content analysis, as well as multiple overarching challenges researchers experienced across the research process. Based on our findings, we discuss opportunities to better facilitate privacy policy research, including research directions for methodologically advancing privacy policy analysis, potential structural changes around privacy policies, and avenues for fostering an interdisciplinary research community and maturing the field.
Scaffolding to Support Humanities Students Programming in a Human Language Context
Language is a key topic of interest for students in the humanities -- language is the way in which humans express themselves, communicate, and make art. Computing on language (e.g., recognizing language, generating language, building bots) can be a pathway into using computing for humanities contexts. At the University of Michigan, we are developing a new program to support students in liberal arts and sciences to learn about computing, explicitly including programming. We have designed two courses for introducing computing (1) in terms of creative expression and (2) around the implications of computing on justice. In both classes, we use a scaffolded sequence of programming languages and activities to explore computing on language: (a) a teaspoon language for sentence generation and recognition, (b) a set of custom Snap blocks for sentence generation and recognition, (c) a set of custom Snap blocks for building Chatbots, and (d) an ebook activity for mapping from Snap to Python. Each language takes less than 10 minutes to introduce, with a wide variety of possible student activities (for in-class active learning or for later homework). While the tools build on each other, the earliest stage (the teaspoon language) could be used within a single class session in linguistics, communications, or other liberal arts courses.
Attachment-Informed Design: Digital Interventions That Build Self-Worth, Relationships, and Community in Support of Mental Health
DIS '23: Proceedings of the 2023 ACM Designing Interactive Systems Conference, July 2023
Gabriela Marcu, Jina Huh-Yoo
The design of digital interventions for mental health often overlooks the foundational belief in one’s own self-worth and related ability to develop close relationships with others. Attachment theory explains these interpersonal aspects, which have long been associated with positive mental health outcomes such as resilience when faced with adversity. We report on generative interviews with mentors experienced in supporting youth, as part of design work initiated by community leaders concerned about the traumatic effects of gun violence and systemic racism on local youth. These community leaders wanted to extend their existing mentoring efforts through digital technology. We identified attachment theory as the best fit for helping us understand how mentors are addressing trauma among youth, in ways that can inform how we design in support of these relationships and communities. We introduce attachment-informed design as a set of design principles that represent a novel approach to digital mental health interventions.
The Theory of Mind and Human-Robot Trust Repair
Scientific Reports, June 2023
Connor Esterwood, Lionel P. Robert
Nothing is perfect and robots can make as many mistakes as any human, which can lead to a decrease in trust in them. However, it is possible, for robots to repair a human’s trust in them after they have made mistakes through various trust repair strategies such as apologies, denials, and promises. Presently, the efficacy of these trust repairs in the human–robot interaction literature has been mixed. One reason for this might be that humans have different perceptions of a robot’s mind. For example, some repairs may be more effective when humans believe that robots are capable of experiencing emotion. Likewise, other repairs might be more effective when humans believe robots possess intentionality. A key element that determines these beliefs is mind perception. Therefore understanding how mind perception impacts trust repair may be vital to understanding trust repair in human–robot interaction. To investigate this, we conducted a study involving 400 participants recruited via Amazon Mechanical Turk to determine whether mind perception influenced the effectiveness of three distinct repair strategies. The study employed an online platform where the robot and participant worked in a warehouse to pick and load 10 boxes. The robot made three mistakes over the course of the task and employed either a promise, denial, or apology after each mistake. Participants then rated their trust in the robot before and after it made the mistake. Results of this study indicated that overall, individual differences in mind perception are vital considerations when seeking to implement effective apologies and denials between humans and robots.
