Collaborative Research: Malleable Media to Support Interaction through Bi-Directional Touch Displays

Interaction with touch screens is compelling -- because, in part, you can use a glass touchscreen to visually apprehend, reach toward, touch and manipulate an object. For some users, however, the presence of a glass barrier causes the interaction to fail completely. This is true especially for blind users.

In a project that promises to revolutionize how blind people interact with digital media, researchers will use existing and new technology to create a full-page refreshable digital braille display. In addition to presenting braille text, a large and dense array of dots will be created that can be raised individually to render tactile graphics or images for the sense of touch.

Start date: 9/1/2017

End date: 8/31/19

Read More

Some technology has already been developed for the project. However, new paradigms for use and interaction will have to be designed, supported by science grounded in haptics and HCI, the researchers said. New interaction paradigms will mix text and graphics and convey meaning through special layout, tactile symbols and extended braille elements.

“We proposed to combine our device with appropriate sensing technology to create an exciting new field of interaction paradigms with tangible media.”

Working on the project with Sile O'Modhrain, an associate professor in the School of Information and the School of Music, Theatre and Dance, will be Brent Gillespie, an associate professor of mechanical engineering in the University of Michigan College of Engineering. Also assisting is the Smith-Kettlewell Eye Research Institute.

The researchers propose to develop new interaction paradigms involving malleables that support interaction with digital media and go way beyond traditional haptic rendering through a stylus and beyond the touchscreen with rich application for both blind and sighted users.

“We have at our disposal a technology that will revolutionize access to media for tables and portable computing devices and we want to sure that such new user interfaces are immediately available to support entirely new modes of interaction for blind and sighted computer users.” 





National Science Foundation, $226,040

UMSI portion, $59,358

Mechanical Engineering portion, $166,682