A day without a search engine: An experimental study of online and offline searches

People search for information on a daily basis and for various purposes: for help with job-related tasks or schoolwork, to gather information for making purchasing decisions, or just for fun. Searching is the second most common online activity after email.

Given the popularity of online searching, the question arises as to how much time people save by using search engines for their information needs, and the extent to which online search affects search experiences and outcomes. Using a random sample of queries from a major search engine and a sample of reference questions from the Internet Public Library (IPL), this project conducted a real-effort experiment to compare online and offline search experiences and outcomes.

Start date: 10/1/2010
End date: 9/30/2013

Read More

With the advent of the Web and search engines, online searching has become a common method for obtaining information. One concern with the growing popularity of online searching is the quality of the information obtained by these searches. Web searches provide searchers with almost instantaneous access to millions of results, but the quality of the information linked to these results is uneven. Some of this information may be inaccurate or come from untrustworthy sources. While a searcher may save time by searching online, it may come at the expense of information quality.

This project addressed the issue of quality versus convenience, as researchers examined how much time people saved by searching online, and if that time was saved at the expense of quality. Researchers compared the processes and outcomes of Web searches to more traditional information searches using non-Web resources.

Using a random sample of queries from a major search engine and a sample of real reference questions from the Internet Public Library, researchers compared online and offline search experiences and outcomes. Overall, the results indicated that online searching yields a higher likelihood of search success, saves on average 10-15 minutes compared to the corresponding offline search, and is more enjoyable for the person involved. 

The findings on the quality of sources were more nuanced. While library sources were judged to be significantly more trustworthy and authoritative than the corresponding Web sources, Web sources were judged to be significantly more relevant. In addition to time saved, the project’s results suggest that the overall Web source quality is not significantly different from that of non-Web sources when questions come from search-engine queries.

Overall, this research contributes to the evaluation of search technology on search outcome and productivity in everyday life. The project also introduced experimental techniques from interactive information retrieval to experimental economics. 

UMSI PhD student Grace YoungJoo Jeon and Yong-Mi Kim (PhD ’11) also collaborated on this project.

For more information on the project, the methodologies used, and its findings, please click here.

 

Grants

Collaborative Research: School Choice and College Admissions: Theory and Experiments, National Science Foundation: $233,467 

An Experimental Study of Strategic Complementarity, Asynchronicity and Mechanism Design, National Science Foundation: $200,730


The National Science Foundation is an independent federal agency created by Congress in 1950 "to promote the progress of science; to advance the national health, prosperity, and welfare; to secure the national defense…"  

 

This project also received support from a Google Research Award, which is a one-year award structured as unrestricted gifts to universities to support the work of world-class full-time faculty members at top universities around the world.