Library Discovery Tools & Bias
Discovery layers in library catalogs offer a simple way to search across library content. But there are issues with discovery layers because the companies that make the search tools are also in the content business. And one fear is that these companies might favor their own content in the results generated by the discovery layers.
Discovery layers in library catalogs generally mean that patrons are finding reputable content because they are using the carefully curated collections of libraries. One study looked at how the adoption of a discovery tool changes the use of articles from publisher-hosted online journals. "Based on data from 33 libraries and 8,765 journals from six major publishers, their analysis showed 'an overall increase in usage for the entire set of journals in the year after implementation, though the extent of change varied by discovery service and publisher.'"
This is good news. We want students to use highly reputable, peer-reviewed articles that can be hard to find using a traditional library catalog. We know that, generally, if students find an online database too laborious (say, three clicks), they will skip it.
Discovery layers are a good thing, but is there bias? To understand if there is bias, "it helps to understand how discovery tools work. Libraries make large investments in different kinds of content, such as their subscriptions to databases of scholarly articles, or the books that fill their local catalogs. The new breed of search software hinges on building 'a very large, consolidated index that represents all of those things,' says Marshall Breeding, a consultant who specializes in library technology. Vendors of discovery tools will make deals with providers that sell content to libraries, he says, so that content can be represented in the discovery tools’ indexes and made available for search. (Beyond products from Ebsco and ProQuest, other major tools in this genre, known as 'web scale' or 'index based' discovery, include Primo, from Ex Libris, and WorldCat Discovery Services, from OCLC.)
Vendors describe their discovery tools as unbiased arbiters of information. Ebsco, for example, sells both search software and content, as does ProQuest. Asked whether Ebsco favors its own content in the results generated by its search tool, Sam Brooks, executive vice president for sales and marketing, dismissed the idea as 'competitor-driven propaganda.' He added, 'There’s no truth to that whatsoever.' Bias toward a content provider, he says, 'would be commercial suicide for any discovery vendor.'"
Although the vendors insist they are unbiased, Andrew Asher, Assessment Librarian at Indiana U. at Bloomington, took a look at the potential bias. "Mr. Asher’s experiment discovered that default settings of the tools had a major effect on what resources students chose. Working with Google Scholar, which is integrated with Google Books, students used more books. With Summon, they used a lot of shorter newspaper and magazine articles. With Ebsco Discovery Service, they used more journals, which meant they scored highest under the study’s rating rubric. (In a blog post responding to Mr. Asher’s study, ProQuest said the methodology 'inadvertently penalized' Summon, its product.)"
The bias may be inadvertent, but it does appear that the vendor created discovery tools do favor their content.
Ultimately, though, how much all this will matter is debatable. "Only about 20 percent of faculty members begin research at their libraries’ online catalogs, according to a 2012 survey by Ithaka S+R. Meanwhile, the competition for student and faculty attention has only intensified since 2004, when Google’s 'simple way to broadly search for scholarly literature' made its debut. That free service, called Google Scholar, has many fans in academe."
After noting the criticisms of Google Scholar (limited advanced search functionality, incomplete or inaccurate metadata, inflated citation counts, lack of usage statistics, and inconsistent coverage across disciplines), Asher reluctantly stated his preference for using it. "I kind of hate to say it, since I am a librarian," he says. "We pay a lot of money for discovery tools. And then I go off and just use Google Scholar."
Discovery layers in library catalogs generally mean that patrons are finding reputable content because they are using the carefully curated collections of libraries. One study looked at how the adoption of a discovery tool changes the use of articles from publisher-hosted online journals. "Based on data from 33 libraries and 8,765 journals from six major publishers, their analysis showed 'an overall increase in usage for the entire set of journals in the year after implementation, though the extent of change varied by discovery service and publisher.'"
This is good news. We want students to use highly reputable, peer-reviewed articles that can be hard to find using a traditional library catalog. We know that, generally, if students find an online database too laborious (say, three clicks), they will skip it.
Discovery layers are a good thing, but is there bias? To understand if there is bias, "it helps to understand how discovery tools work. Libraries make large investments in different kinds of content, such as their subscriptions to databases of scholarly articles, or the books that fill their local catalogs. The new breed of search software hinges on building 'a very large, consolidated index that represents all of those things,' says Marshall Breeding, a consultant who specializes in library technology. Vendors of discovery tools will make deals with providers that sell content to libraries, he says, so that content can be represented in the discovery tools’ indexes and made available for search. (Beyond products from Ebsco and ProQuest, other major tools in this genre, known as 'web scale' or 'index based' discovery, include Primo, from Ex Libris, and WorldCat Discovery Services, from OCLC.)
Vendors describe their discovery tools as unbiased arbiters of information. Ebsco, for example, sells both search software and content, as does ProQuest. Asked whether Ebsco favors its own content in the results generated by its search tool, Sam Brooks, executive vice president for sales and marketing, dismissed the idea as 'competitor-driven propaganda.' He added, 'There’s no truth to that whatsoever.' Bias toward a content provider, he says, 'would be commercial suicide for any discovery vendor.'"
Although the vendors insist they are unbiased, Andrew Asher, Assessment Librarian at Indiana U. at Bloomington, took a look at the potential bias. "Mr. Asher’s experiment discovered that default settings of the tools had a major effect on what resources students chose. Working with Google Scholar, which is integrated with Google Books, students used more books. With Summon, they used a lot of shorter newspaper and magazine articles. With Ebsco Discovery Service, they used more journals, which meant they scored highest under the study’s rating rubric. (In a blog post responding to Mr. Asher’s study, ProQuest said the methodology 'inadvertently penalized' Summon, its product.)"
The bias may be inadvertent, but it does appear that the vendor created discovery tools do favor their content.
Ultimately, though, how much all this will matter is debatable. "Only about 20 percent of faculty members begin research at their libraries’ online catalogs, according to a 2012 survey by Ithaka S+R. Meanwhile, the competition for student and faculty attention has only intensified since 2004, when Google’s 'simple way to broadly search for scholarly literature' made its debut. That free service, called Google Scholar, has many fans in academe."
After noting the criticisms of Google Scholar (limited advanced search functionality, incomplete or inaccurate metadata, inflated citation counts, lack of usage statistics, and inconsistent coverage across disciplines), Asher reluctantly stated his preference for using it. "I kind of hate to say it, since I am a librarian," he says. "We pay a lot of money for discovery tools. And then I go off and just use Google Scholar."
Comments
Post a Comment