WHAT ARE THE IMPLICATIONS FOR INFORMATION LITERACY TRAINING IN HIGHER EDUCATION WITH THE INTRODUCTION OF FEDERATED SEARCH TOOLS? A study submitted in partial fulfillment of the requirements for the degree of Master of Arts in Librarianship at THE UNIVERSITY OF SHEFFIELD by LUCY MCCASKIE September 2004
84
Embed
WHAT ARE THE IMPLICATIONS FOR INFORMATION LITERACY
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
WHAT ARE THE IMPLICATIONS FOR INFORMATION LITERACY TRAINING IN HIGHER EDUCATION WITH THE INTRODUCTION
OF FEDERATED SEARCH TOOLS?
A study submitted in partial fulfillment
of the requirements for the degree of
Master of Arts in Librarianship
at
THE UNIVERSITY OF SHEFFIELD
by
LUCY MCCASKIE
September 2004
Abstract
This dissertation investigates the implications for information literacy
training in higher education with the introduction of federated search tools, tools
which cross-search a number of databases simultaneously. The Information
Literacy Unit at the Open University suggested the research area since the
university is currently implementing this kind of tool. The focus of the
dissertation is on librarians’ experiences and opinions in this area and the
research looks at the search behaviour of users, the nature of the tool, changes
to information literacy training as a result of a federated search tool and any
other perceived benefits and disadvantages of using this kind of tool in relation
to the development of information literate students.
A qualitative research approach was used to fulfill the research objectives
and a number of methods of investigation were used to explore the research
area. The research begins with a review of the literature about information
literacy, the search behaviour of students and federated search tools. The Open
University was used as a case study to explore an institution currently
implementing a federated search tool and issues were raised in this study that
helped focus the rest of the research. The opinions and experiences of
librarians at universities currently using a federated search tool were obtained in
two different ways. Interviews were used for those universities within traveling
distance and those further afield were included in an e-mail survey.
The research found that universities had varying experiences of federated
search tools but that some common themes were: an increased use of
databases, concerns about the quality of results produced from searches and a
need for information literacy training for users. The research concludes that
federated search tools cannot prevent users from becoming information literate
but by using information literacy skills users can make better use of these tools.
Finally recommendations are made to the Open University for actions that may
assist users becoming information literate whilst using a federated search tool.
1.1 Research Origins....................................................................................... 4 1.2 Aims and Objectives of Research ............................................................. 4 1.3 Definitions.................................................................................................. 5 1.4 Research Context...................................................................................... 7
Chapter 2: Literature Review ........................................................................... 9 2.1 Information Literacy................................................................................... 9 2.2 Searching Behaviour of Higher Education Students................................ 17 2.3 Federated Search Tools .......................................................................... 19
Chapter 3: Methodology................................................................................. 24 3.1 Research Approach................................................................................. 24 3.2 Methods of Investigation.......................................................................... 25 3.3 Results Analysis ...................................................................................... 31 3.4 Further Methodological Considerations................................................... 32
Chapter 4: Open University Case Study ....................................................... 34 4.1 Aims of Open University Case Study ...................................................... 34 4.2 Case Study Evidence .............................................................................. 34 4.3 Open University Background................................................................... 35 4.4 Information Literacy at the Open University............................................. 35 4.5 Implementing the Federated Search Tool, ENCompass ......................... 38 4.6 Issues Arising from the Open University Case Study .............................. 41
Chapter 6: Results of E-mail Survey ............................................................. 58 6.1 Changes in Information Literacy Training ................................................ 58 6.2 Changes in Users’ Search Behaviour...................................................... 60 6.3 Possible Changes to Library Services and Training in the Future ........... 61
Chapter 7: Discussion of Results.................................................................. 63 7.1 Raising Awareness of Resources............................................................ 63 7.2 The Need For Information Literacy Training ............................................ 64 7.3 Recommended Use of Federated Search Tools ..................................... 65
Chapter 8: Conclusions, Recommendations and Areas for Further Research.......................................................................................................... 66
8.1 Conclusions............................................................................................. 66 8.2 Recommendations for the Open University ............................................. 67 8.3 Areas for Further Research ..................................................................... 69
2004). Information literacy development has also been considered in the
context of electronic resources (Wood et al., 1996). Whilst no research and
limited discussion has taken place about the impact of federated search tools on
information literacy, the research about information literacy is used as the
context for the research in this dissertation.
This dissertation will examine the topic of federated search tools and the
implications that they have for information literacy training in higher education by
first reviewing the literature related to information literacy and federated search
tools. The methodology for research will then be described, followed by a
presentation and discussion of the results before a series of recommendations
for universities choosing to implement a federated search tool are made in the
conclusion.
8
Chapter 2: Literature Review
This chapter explores the literature regarding information literacy and
federated search tools in order to provide a context for the research and to
illuminate the current issues and developments in these areas. The searching
behaviour of students is also examined as an issue that impacts upon both the
information literacy development of students and the need for federated search
tools.
2.1 Information Literacy
2.1.1 Information Literacy: an Introduction
Paul Zurkowski first used the term information literacy in 1974 in a report to
the National Commission on Libraries and Information Science about
establishing a national programme in the US to achieve universal information
literacy (Doyle, 1994; Bruce, 1997). The first advocates of information literacy
were mainly school librarians in the US but in the 1980s use of the term and the
concept became more widespread. Significant events in the lifespan of the
concept include the establishment of the American Library Association's (ALA)
Presidential Committee on Information Literacy in 1987 and its subsequent
report produced in 1989 (ALA, 1989). This report made a statement about the
importance of information literacy and connected it with the goals of lifelong
learning and effective citizenship. In the US, the National Forum on Information
Literacy was established in 1989 as a response to this statement and forms a
coalition of national organisations, which aims to aid information literacy
promotion and development within these organisations (NFIL, 2004). In the UK
there is no such consortium or forum, although the Charted Institute of Library
and Information Professionals (CILIP) is currently preparing a definition of
information literacy and is planning to become involved in future advocacy
(Corrall, 2004). Information literacy in the UK has been championed in the
9
higher education sector by the Society of College, National and University
Libraries (SCONUL) but despite the two conferences focusing on information
literacy held in the UK (Webber & Johnston, 2003a) developments are still
behind those in the US and Australia which has held five national information
literacy conferences and has a dedicated Australian and New Zealand Institute
for Information Literacy (ANZIIL).
2.1.2 Information Literacy: Definitions and Descriptions
There has been discussion about information literacy and whether it is a
new concept or a natural progression from the library instruction that has been
taking place since the 1960s (Grassian & Kaplowitz, 2001). Whether
information literacy is viewed as a new concept or not depends upon its
definition and individuals' understanding of the concept. There have been
numerous definitions and models of information literacy, four of which are
illustrated in table 2.1.2. These models are chosen as examples of those that
are used in a higher education context. The first is from the Association of
College and Research Libraries (ACRL, 2000), a group within the ALA. This
model is endorsed by the ALA and forms a set of standards which students
should be able to demonstrate. The table indicates only the standards but for
each there are a number of performance indicators and outcomes that can be
used to assess whether an individual has achieved that standard of information
literacy. The breadth of standards such as the ACRL model were criticised by
Owusu-Ansah (2003) who complains that the standards try to be all
encompassing, making it difficult for practical steps to be given for developing
information literate students. Owusu-Ansah also believes that the ACRL model
is beyond the boundaries of what librarians can realistically be expected to
teach.
The second model is from the Australian and New Zealand Institute for
Information Literacy (ANZIIL) a group established in 2000 which promotes
information literacy in higher education (ANZIIL, 2003). The model given
10
Table 2.1.2: A comparison of information literacy models showing attributes of the information literate individual. ACRL (ACRL, 2000) ANZIIL (Bundy, 2004) BIG6 (Big6 Associates, 2003) SCONUL (SCONUL, 1999) 1. Determines the nature and extent of the information needed.
1. Recognises the need for information and determines the nature and extent of the information needed.
1. Task definition. 2. Information seeking strategies.
1. The ability to recognise a need for information. 2. The ability to distinguish ways in which the information ‘gap’ may be addressed. 3. The ability to construct strategies for locating information.
2. Accesses information effectively and efficiently.
2. Finds needed information effectively and efficiently.
3. Location and access. 4. The ability to locate and access information.
3. Evaluates information and its sources critically and incorporates selected information into his or her knowledge base and value system.
3. Critically evaluates information and the information seeking process.
5. The ability to compare and evaluate information obtained from different sources.
4. Individually or as a member of a group, uses information effectively to accomplish a specific purpose.
4. Manages information collected or generated.
4. Use of information. 6. The ability to organise, apply and communicate information to others in ways appropriate to the situation.
5. Applies prior and new information to construct new concepts or create new understandings.
5. Synthesis. 7. The ability to synthesise and build upon existing information, contributing to the creation of new knowledge.
5. Understands many of the economic, legal, and social issues surrounding the use of information and accesses and uses information ethically and legally.
6. Uses information with understanding and acknowledges cultural, ethical, economic, legal, and social issues surrounding the use of information.
6. Evaluation.
11
(Bundy, 2004) is the second edition of the Australian and New Zealand
Information Literacy Framework, derived from the ACRL Information Literacy
Standards. Both the ACRL and ANZIIL models include information literacy
competencies at lower and higher levels. Bundy describes these attributes
as a continuum of capacities and students at higher levels would be expected
to demonstrate the standards in conjunction with one another.
Michael Eisenberg and Robert Berkowitz developed the Big6 Skills for
Information Problem-solving model in the 1980s originally for use within
secondary schools but it is now applied to all levels of education (Spitzer,
Eisenberg & Lowe, 1998; Big6 Associates, 2003). The final model from table
2.1.2 was developed by SCONUL in 1999 by a task force established to
make a statement about information skills in higher education in the UK
(SCONUL, 1999). In the UK use of the term information literacy has not
been as widespread as information skills. Webber and Johnston (2003a)
point out that the term information skills is sometimes preferred as it avoids
negative associations with literacy and illiteracy.
The SCONUL model is similar to those of the ACRL and ANZIIL in that
it shows a progression from lower level to higher-level skills. All four models
have similarities in that they describe the attributes of an information literate
person and these attributes are often comparable. Webber and Johnston
(2000) note how different definitions of information literacy tend to focus on
similar areas such as need recognition, search formulation, source selection
and interrogation, information evaluation, information synthesis and use.
There are certainly similarities between these models and Owusu-Ansah
(2003) observes how the main writers on information literacy tend to agree
more than they disagree.
Other important writers about information literacy include Doyle (Doyle,
1994) who described the attributes of an information literate person and
Bruce (1997) who examined higher educator's conceptions of information
literacy in order to understand the phenomenon and how it is experienced.
Bruce's conceptions of information literacy begin with the information
technology conception and the relationship between IT skills and information
12
literacy is one that has been discussed in the literature (Kwasnik, 1990;
SCONUL, 1999; ACRL, 2000). Generally it is agreed that some IT skills are
a necessary part of information literacy but that information literacy is a much
broader concept and the SCONUL model includes basic IT skills as a
building block upon which information skills can be constructed.
2.1.3 The Need for Information Literacy
The association between information literacy and IT skills can be
attributed to the need for information literacy that has arisen out of the
changing information and IT environments. The Internet in particular has
enhanced the need for information literacy. Users have access to large
quantities of unfiltered information with varying degrees of reliability and
usefulness, which the user needs to distinguish for his or herself (Basili,
2003). The amount of information available can lead to information overload
so users need strategies for handling information (Haban, 1990). Although
library users have always needed some information skills, in the past the
library collection would have been selected to include only those sources
considered reputable (Grassian & Kaplowitz, 2001). Now libraries often
focus on access to information rather than the holding of specific resources
so there is more pressure on the user to select appropriate materials.
The ALA report (1989) described how information use is necessary for
both personal and business needs and the need for information literacy has
also been associated with democracy and lifelong learning (Bundy, 2004).
Citizens need access to information in order to make informed decisions and
the changing economy in countries like the US, UK and Australia has placed
more emphasis on knowledge and information industries. Webber and
Johnston (2003a) describe the situation in the UK where more information
literate employees are required. The Dearing Report in the UK also
highlights the problem of the lifelong career becoming less common and
indicates the need for individuals to learn throughout their lives and to
develop new skills and knowledge (NCIHE, 1997). This report states “many
jobs require sophisticated information handling skills” and explains that this is
13
more than simply using a computer (section 4.44) but disappointing fails to
acknowledge information literacy as one of the key skills that undergraduates
should obtain at university.
Students at university need to become information literate not only for
their future careers and to participate in a democratic society but also to
study successfully. Ward (2003) points out that whilst students may come to
university with IT experience, their experience may be in using IT for
recreational purposes only. Ray and Day's (1998) examination of the use of
electronic information sources by students found that one of the barriers to
using these sources was a lack of effective information retrieval skills and
that students may avoid using these materials because they do not have the
skills necessary to use them. With more emphasis in higher education on
students learning through problem-based learning, which includes
researching individual problems and topics (Webber & Johnston, 2003a), it is
essential that students gain the skills necessary to find, evaluate and use the
information they need for their studies.
2.1.4 Information Literacy Teaching in Higher Education
In the UK, higher education has been the main area in which
information literacy movements have been taking place. The Big Blue
Project was conducted by the University of Leeds and Manchester
Metropolitan University for the Joint Information Services Committee (JISC)
in 2001-2002. The project investigated the extent of information skills
teaching in HE and post-16 education and undertook a number of case
studies of information skills programmes (Big Blue, 2004). The project found
that:
“despite the fact there is no formal coherent strategy for information
skills development and training within the higher education sector there
is a great deal of work taking place in this area.”
It also produced an Information Skills Toolkit identifying nine features of a
successful information skills programme. The project also identified some
14
issues that have been widely discussed in the literature including whether
information skills should be integrated into the curriculum, the collaboration
that is needed between library, computing and academic staff for successful
information literacy programmes and the need for assessment of the skills
obtained.
Whether information literacy should be incorporated into the curriculum
or not is an area of some disagreement. The main problem with voluntary
information literacy programmes is acknowledged by Haycock who explains
that students are busy and if they do not see any immediate need to, will not
attend these sessions. A second problem is that students may not believe
that they lack these skills and so do not see the need for this kind of training
(Brown, Murphy & Nanny, 2003). The ACRL (2000) advocates integrating
information literacy into the curriculum in order to provide opportunities for
problem-based learning, evidence-based learning and inquiry learning.
Some writers suggest that the way students are taught needs to be changed
in order to provide opportunities for them to develop information literacy skills
(ALA, 1989; Bruce, 1997; Bundy, 2004). Bundy explains how students need
a chance to engage with information sources and that there is a danger with
some educational experiences that students will develop only a surface-
learning approach. Ramsden (1992) describes the difference between deep
and surface-learning approaches and how students will use different
approaches according to the particular task that they are undertaking. For
Ramsden,
“learning in educational institutions should be about changing the ways
in which learners understand, or experience, or conceptualise the world
around them” (1992:4).
The challenge for educators is to create learning experiences that allow
students to use deep-learning approaches. This is particularly problematic
for library staff teaching information literacy because time is needed for
students to change the way they think and often time is in short supply for
these kind of programmes (Webber & Johnston, 2003b). Webber and
Johnston (2000, 2003a) are concerned about the integration of information
15
literacy into the curriculum because if it is taught in relation to other subjects,
there is a danger students may use only a surface-learning approach and
may fail to gain a coherent understanding of the concept of information
literacy and what it means to them. Webber and Johnston's experience of
teaching a credit-bearing information literacy class was an example of how
students can change their conceptions of information literacy but required
time and resources that are rarely made available to libraries. The authors
are both academics so it is possible they may have had more influence on
students than library staff. The importance of collaboration and of academics
supporting information literacy is a theme in the literature (Spitzer, Eisenberg
& Lowe, 1998; ACRL, 2000). Ward (2003) points out other important factors
in the success of information literacy programmes, such as explaining why
the training is important, teaching at the point of need and providing materials
that students can return to when needed.
Assessing whether students have developed information literacy is
another problematic area. Viewing information literacy as an area of
knowledge (Town, 2003) rather than a set of discrete skills means that
assessing information literacy is a complex task. For Doyle (1994),
measuring information literacy cannot be separated from the measurement of
the critical thinking skills and problem solving skills with which it is related.
Assessment is essential in indicating the importance and credibility of a
subject to students and Webber and Johnston (2003b) believe that
assessment should take a number of forms and should include diagnosis of a
student's knowledge, feedback on strengths and weaknesses and monitoring
of progression. This is a more complex approach than is demonstrated by
libraries who use simple multiple-choice questionnaires to assess whether a
student has acquired certain skills. It is logical to suppose that information
literacy education would have an effect on the enquiries that are received in
libraries. Parker and Waller (2002) visited Australian academic libraries
where information literacy instruction has reduced the number of enquiries or
that queries have been answered more quickly. The libraries also found that
students have become interested in using more resources but based on
anecdotal evidence, it is difficult to measure the real impact of information
literacy education and this is an area for more research.
16
2.2 Searching Behaviour of Higher Education Students
The association between information literacy and the ability of students
to use electronic resources has already been mentioned. Students' use of
electronic resources and their search behaviour has been the subject of a
number of studies and the issue is influential in the development of library
services to improve access to information. One of the reasons that students
may be unwilling to undertake information literacy education is that they are
unaware that they lack these abilities. Wood et al. (1996) conducted a study
of over a hundred students from different faculties to see how they searched
for information in particular databases. One of the findings of the study was
that:
“many students were unable to construct an adequate search query,
devise an effective search strategy or achieve an adequate search
result” (p.90).
The authors of the study related search strategies to students' sense of
satisfaction and they suggest that students may be satisfied with results that
would be unsatisfactory to an information professional. Students may have
low expectations, which are met with inadequate search strategies, so they
see no need to improve their techniques. Ray and Day (1998) also
acknowledged the problem of satisfaction in their study of the use of
electronic information sources across three universities. Ray and Day found
that whilst many respondents used electronic information sources, they
tended to use only a limited number. Barriers to use included lacking the
skills necessary to use them but interestingly the authors note that students
will learn these skills if they think the skills are transferable and if encouraged
to use sources by academic staff.
The JUSTEIS project (JISC User Surveys: Trends in Electronic
Information Sources) took place at the University of Wales and also
examined use of electronic information sources (Armstrong et al., 2001).
This study looked at a far greater number of students than the two already
mentioned but had similar findings: that students lacked coherent search
17
strategies and that some electronic information sources received minimum
usage. This study also found that students were motivated to use these
sources if they thought they would obtain better assessment grades. The
JUSTEIS project also produced some very significant statistics when
students were asked to describe how they found information. Usage of
search engines was more than double that of the library OPAC) and was the
main use of electronic resources for first-year undergraduates. Student
reliance on search engines was also found in the EDNER project (2002)
where students completed set tasks using electronic information sources. In
this study 45% of students turned to Google first with only 10% using the
library OPAC. The reasons suggested for student reliance on search
engines are that they are familiar, have produced successful results in the
past and that the time and effort required to search may be more important to
students than the relevance of the items that they retrieve. Lack of
awareness of other resources could also be a problem. Brown, Murphy and
Nanny (2003) looked at teacher training students in the US and found that
accuracy and speed were important to the students when they searched and
that they were willing to learn about library resources if they thought search
time and effort could be reduced and accuracy and relevance maximised.
The searching behaviour of students that has been demonstrated in
these studies is significant for developers of electronic information sources
and for librarians. Students do not appear to share the same values of
information professionals and are very much focused on doing what they
need to pass assessments within the minimum effort. This is not surprising
given the demands that the modern student may have upon his or her time.
Griffiths and Brophy (2002) suggest that information sources can be
improved by limited the choices that users need to make as this limits the
opportunity of mistakes being made. Koh (2003) also believes that
databases should be simplified and argues that:
“Ideally, users should not have to complete a tutorial to access
information through a database effectively; they should be able to find
the results they are looking for without having to understand the
database's underlying structure”.
18
Koh goes on to describe and promote the AARLIN (Australian Academic and
Research Library Network) project, which uses a federated search tool
designed so that students can access a range of resources through an
interface that is as easy to use as an search engine.
2.3 Federated Search Tools
2.3.1 Portals and Federated Search Tools
In the electronic information environment one of the responses to the
problem of bringing large amounts of information together has been for
libraries to introduce portals. A portal is a gateway, or a point where users
can start their search for information on the web (Miller, 2001). There are a
number of different types of portals, for example universities have been
introducing institutional portals, which can be described as “a layer which aggregates, integrates, personalises and presents
information, transactions and applications to the user according to their
role and preferences” (Dolphin, Miller & Sherratt, 2002).
A second type of portal is a subject portal and the JISC have produced
subject hubs bringing together resources in certain subject areas (Awre,
2003). A third type of portal is a federated search tool which brings together
the resources a library subscribes to and allows cross-searching of these
resources. Library management system suppliers such as Endeavor
(ENCompass), Ex Libris (Metalib), Fretwell-Downing Informatics (Zportal)
and MuseGlobal (MuseSearch) are all introducing these tools. These tools
can be seen as a progression from the cross-file searching that is available in
Dialog (Webster, 2004). They work using the Z39.50 protocol that is used by
database suppliers to enable communication between computers (Miller,
1999). This protocol is not always effective because database suppliers
have not always adopted it consistently so other protocols are also used such
as HTML, XML and SQL (Lewis, 2002). Since these tools access resources
that the library pays for, some type of authentication is required to ensure
19
only registered users of the library can use the tool. Rather than the user
needing to remember a large number of passwords, the authentication can
be matched against the LDAP (lightweight directory access protocol) of a
university so students can use their existing network usernames and
passwords (Ramsden, 2003). This technology also allows for personalisation
features such as remembering users, their favourite resources and the
previous searches they have performed. The way that results are displayed
varies between suppliers, either producing a single list with duplications
removed, or results arranged by individual resources. Luther (2003) explains
the way that results are returned in batches with those from the fastest
servers coming back first but questions the order in which results should
appear. Other features of federated search tools include allowing users to
link to the full-text of items when bibliographic databases are searched and a
level of customisation for institutions implementing these tools (Ramsden,
2003; Lewis 2003).
2.3.2 The Need for Federated Search Tools
The development of federated search tools can be seen as a response
to the needs and expectations of library users, particularly students using
academic libraries. The growth of different types of databases, produced by
different suppliers, with numerous interfaces and logins means that library
users can find it confusing when attempting to access information (Ramsden,
2003; Fryer, 2004). Commentators seem to agree that Google has changed
the expectations of library users (Luther, 2003; Miller 2004) who want quick,
easy access to information, possibly at the expense of recalling all relevant
materials and that they expect to be able to access the full-text of items
immediately (Ward, 2003). Roy Tennant explained that “only librarians like to
search; everyone else like to find” (Miller, 2004:32). Library OPACs and
web-pages have been alienating users with their use of library terminology
and by including long lists of databases that users find it difficult to select
from (Luther, 2003). The library OPAC is also better for users who are
looking for a known item, rather than those who want to research a new area
(Lewis, 2003). Webster (2004) believes that the idea of a one-stop shop is
20
not a new idea, and that libraries have traditionally been performing this role
by bringing materials together. It is now Google that is performing the role of
a one-stop shop for many users and libraries introducing federated search
tools are hoping that the tool can supplant Google (Frost, 2004; Fryer, 2004)
and return the library to the centre of students' studies.
2.3.3 Issues Arising From Federated Search Tools
There are a number of issues that have arisen due to the introduction of
federated search tools and which have been discussed in professional and
popular journals for information professionals. One of the most prominent
issues is about the effectiveness of searching with these tools and the way
that results are organised. Authors are keen to point out that these tools
cannot search any more effectively than the native interface of a database
since they are limited to using the search facility of the database itself (Lewis,
2002; Hane 2003) and that these tools are most useful for resource
discovery. Fryer (2004) thinks that whilst federated search tools are a good
point to start a sophisticated search, the user will need to search further.
Resource providers are not always happy for their databases being searched
through a federated search tool, since they do not want the native interface
they have invested in to be bypassed (Lewis, 2003). JSTOR (2004) released
a statement about these tools that gives an alternative viewpoint from the
resource provider. JSTOR is unhappy with the method it is currently
searched, involving the screen-scraping of HTML from the web-page, since
this is not very effective. The displaying of results is another area of
contention because databases have different ways of ranking results, such
as by date or relevancy and it is also impossible to completely remove all
duplications from the results set since the results are returned at different
times according to the speed of the individual servers (Hane, 2003).
Other issues that have been discussed with regard to federated search
tools include the risk of information overload for the user (Luther, 2003; Cox,
2003) and whether it is the right way to address the problems that
researchers encounter. Cox (2003) questions whether it may be more
constructive to invest more in training users to exploit the existing interfaces
21
rather than creating a tool for less skilled searching. Webster (2004) thinks
the tools are a limited solution and do not address the underlying problems of
lack of interoperability and non-standardisation of search tools. He suggests
that the separation of content from search tools so that each database could
search the whole e-content of a library, although he acknowledges that there
would be commercial barriers to doing this.
2.3.4 Federated Search Tools and Information Literacy
The discussion about federated search tools in relation to information
literacy may have been limited in the literature but some interesting points
have been raised. Luther (2003) acknowledges the concern some librarians
have that users will not know which databases they are searching with such
a tool and that the interface has been made overly simplistic but she
reiterates the belief that these tools are not for expert users. Frost (2004) is
more direct in his criticism of these tools, believing that they are “a way of
avoiding the learning process” and that part of the education process
involves students learning to improve the tools that they use. He seems to
think that using these tools prevent students from becoming information
literate and that the library should not promote 'good enough' searching. His
article did highlight the need for more investigation into the area of federated
search tools and their effect on information literacy.
Terrell (2004) produced a more in-depth paper at this year's Lifelong
Learning Conference in Queensland, as he examined the process of
federated searching in light of the ANZIIL Information Literacy Framework
(Bundy, 2004). The main concerns that Terrell raised were about the quality
of the information retrieved with these tools and whether students would fail
to demonstrate competence in standards one and two. The first standard,
'recognising the information need and determining the nature and extent of
the information needed' may be compromised if students are unaware of
which databases they are actually searching. The second standard, 'finding
the information effectively and efficiently' may not be obtained if students do
not need to think about their search strategies when using these tools.
Terrell believes that libraries which introduce federated search tools still need
22
to devote time to user instruction and documentation in order to support their
use. This seems to reflect the experience of La Trobe University in Australia
where a federated search tool was introduced as part of the AARLIN project.
One of the comments received during the pilot study at La Trobe was that
“learning how to use AARLIN was time consuming and labour intensive”
(Burke et al., 2003). The fear that these tools may be seen by some as a
way of reducing the need for information literacy training was an issue raised
in the discussion following Terrell's paper (Webber, 2004).
The issues raised by Frost and Terrell are interesting but more work is
needed to see whether the experience of those universities actually using
these tools reflects their thoughts. Two institutions that have produced
accounts of their experiences of implementing federated search tools are
Boston College, US (Tallent, 2004) and Loughborough University, which
undertook a report commissioned by the JISC (Hamblin & Stubbings, 2003).
At Boston College a small study was executed to see how students used the
tool and found that users were not interested in advanced search features,
did not want to know about the differences between databases and electronic
journals and tended not to use help screens and instructions. This
experience seems to suggest that given the choice the students were not
interested in becoming more information literate whilst using these tools. The
study at Loughborough was more positive with the amount of databases
consulted rising dramatically with the introduction of the tool and enquiries
becoming easier to answer at the enquiry desk. This experience seems to
suggest that federated search tools could be a way to promote the resources
that a library makes available to students but the authors of the report noted
that they thought there was still a need for some training.
An examination of the literature has raised a number of issues, which
have been used to focus the research area and aid the design of the
research methodology. Although a number of universities are using
federated search tools, there are limited accounts of their experiences in the
literature so the research of this dissertation can be used to build a more
comprehensive picture of the effect of federated search tools on information
literacy training.
23
Chapter 3: Methodology
This chapter describes the research approach that was chosen in order
to fulfill the dissertation objectives. The different methods of investigation are
also described together with the methods used to analyse the information
obtained. Finally other research considerations are explored including the
limitations of the research.
3.1 Research Approach
The aim of the dissertation was to investigate the implications for
information literacy training in higher education with the introduction of
federated search tools. The focus on higher education was chosen because
it is the main sector where these tools have been introduced and training in
information literacy or information skills is normally part of the service that an
academic library provides. The research approach chosen to fulfill this aim
was qualitative and was chosen for a number of reasons. Since federated
search tools are relatively new, particularly in this country, access to
quantitative data would be limited as would opportunities for gathering
quantitative data, especially during the summer when contacting and
observing students is difficult. Gathering data about the number of libraries
using these tools and the number of searches performed was unlikely to
further understanding of the implications these tools may have for information
literacy training. Focusing on librarians' and information professionals'
experience of the implementation of the tools and the concerns and issues
that they raised could provide more understanding of the effect of the tools
and hopefully inform other libraries who were planning to implement a
federated search tool.
Raising issues and concerns that may be applicable to other libraries
implementing federated search tools was a potential outcome of the research
but creating generalisable results was not a primary aim. The focus was
more on understanding issues within their context. Mellon (1990) describes
qualitative studies, or naturalistic studies, as those that view experiences
24
from the perspectives of the people involved and try to understand a situation
in-depth. The type of qualitative research undertaken was descriptive
(Gorman & Clayton, 1997) as it was involved with examining and describing
the phenomenon encountered in terms of behaviours, beliefs and attitudes
related to searching for information and information literacy. The research
was inductive because it did not start with a predefined theory but examined
specific cases and allowed conclusions to be drawn from them.
3.2 Methods of Investigation
3.2.1 Literature Review
The first phase of the research began with a review of literature about
federated search tools, information literacy and information-seeking
behaviour. Yin (1984) states that the literature review can be used to help
pose insightful questions on the topic and in this case, initial readings
informed the dissertation objectives. The literature review began in the early
stages of the dissertation but continued throughout the research process to
ensure that new articles, especially those regarding federated search tools
were included. The items for the literature review were found in a number of
different ways: from the university library, databases such as LISA and
Internet searches. Items were also recommended by those involved in the
dissertation and references from useful sources were followed up. The
search terms that proved useful during the literature search included:
metasearch, federated search, the names of federated search tools,
information literacy and information skills. The literature review was not only
useful in defining the research objectives, but also provided a context for the
research and enhanced understanding of the research findings.
3.2.2 Open University Case Study
The Information Literacy Unit at the OU suggested the area for research
and the OU itself was included as a case study in the second phase of
research because it presented an opportunity to explore the concerns and
issues raised by staff at a university currently implementing a federated
25
search tool. Yin (1984) explains that one rationale for a single case study “is
where the case represents an extreme or unique case” (p.43) and this was
the reason for choosing the OU. A case study draws on different types of
evidence in order to create a comprehensive picture of the phenomenon and
its context. In this case the sources of evidence used were interviews,
documents and a demonstration of the tool. Yin (1984) recommends
demonstrations of new technology as invaluable in furthering understanding
when the technology is the subject of research.
The staff members interviewed at the OU were recommended by a key
informant at the university. Five members of staff were interviewed, three
concerned with information literacy and two involved with the implementation
of the federated search tool, ENCompass. The interviews were unstructured
and areas for discussion were e-mailed to the interviewees before the
interviews so they were prepared. During the interviews other, unexpected
areas of interest were highlighted and these were pursued. The
implementation project was at an earlier stage than anticipated, so the areas
for discussion were altered as a result. The tool had not reached the piloting
stage so views could not be obtained about how users would search the tool.
Instead the focus of this case study was on the information literacy strategies
in place and the concerns that members of staff had about the effect that
introducing ENCompass might have upon these. Information about the
nature of the tool itself was also gathered and documents were supplied that
aided understanding of federated search tool.
The interviews took place in groups and in the workplace, so there is the
possibility that the interviewees may have felt inhibited in expressing
personal points of view but they did not appear uncomfortable with the
interview approach. The interviews were not recorded but notes were made
throughout and when the case study was completed the interviewees
checked the report to ensure that no mistakes had been made. Yin (1984)
recommends this approach to case studies to ensure construct validity. The
interviews were conducted in June and the information obtained from the OU
case study was used in conjunction with the literature review to help
formulate the research objectives.
26
3.2.3 Interviews at UK Universities
The third phase of the research focused on universities currently using
a federated search tool and two methods of data collection were used to
survey these universities. Staff at universities in the UK, which were within
traveling distance, were interviewed and an e-mail survey was send to
universities in other countries or those in the UK where interviews could not
take place. Both methods of investigation took place in July. These two
different samples will be treated separately.
The sample for universities in the UK using a federated search tool was
a purposeful sample because there are so few universities in this position.
Both Patton (2002) and Silverman (2000) explain the benefits of choosing a
purposeful sample and Patton states that:
“Cases for study... are selected because they are 'information rich' and
illuminative, that is, they offer useful manifestations of the phenomenon
of interest” (p.40).
The universities chosen in the sample were selected in a number of ways.
Interviewees at the OU suggested universities that were currently using a
federated search tool and the literature search and Internet searching
highlighted other cases. Universities that had purchased a federated search
tool but were had not made it available to students were excluded from the
sample. Individuals at the universities were contacted by e-mail in the hope
that the response rate would be higher with personal e-mails. These
individuals were chosen because the literature or Internet search showed
that they had been involved with the implementation of the federated search
tool or were responsible for information literacy. In some cases more than
one individual at each university was contacted and for those where only one
name was known, the e-mail suggested that others be included in the
interviews. Five universities were contacted and four responded, with one
response from each university. Including more individuals from each
university may have enriched the research but the demands on staff time or
27
lack of interest may have been responsible for the limited sample. Those
who did response were interested in the research area and were in a position
to offer opinions and relate relevant experiences.
Interviews were chosen as the best way to obtain data from the UK
sample because these would provide an opportunity to explore the area in
more depth than a questionnaire. There are other benefits associated with
interviewing as a method for data collection, which are listed by Gorman and
Clayton (1997) and include the immediacy of response, the opportunity for
mutual exploration, investigation of causation and personal contact. Since
the research focused on library and information professionals' own
experiences and opinions, interviews were a more personal method of data
collection and it was hoped interviewees would be more forthcoming in
interviews than questionnaires. Face-to-face interviews were chosen rather
than telephone interviews with the expectation that there would be a greater
opportunity to build rapport.
One of the weaknesses of interviews is the risk of bias and since the
researcher becomes the research tool, remaining objective is essential. The
opinions of the researcher may have an influence on the information that the
interviewee provides. In order to avoid this, the researcher expressed no
opinions during the interviews. Interviewees may have attempted to discern
the researcher's point of view from the type of questions asked, which
highlighted some weaknesses in the interview schedule. It was also clear
that information literacy was important to the researcher because it was the
subject of the research but the interviewees all had some responsibility for
information literacy or information skills instruction so were likely to share this
view.
The interviewees were all interviewed alone either in quiet offices or
study rooms where no one else could overhear, which may have allowed the
interviewees to express themselves freely without the fear that colleagues
would hear their opinions. The interviews were all recorded with the
interviewees' agreement so that quotes could be recorded accurately and
information would not be missed. Notes were also made during the
28
interviews to highlight the most important points and whilst the researcher
made notes, quiet spaces were given where interviewees could add more to
a question if they wanted. The knowledge that interviewees were being tape-
recorded may have affected the ideas and opinions that they chose to
express but to encourage more freedom of expression, all were guaranteed
anonymity and agreed to their job titles only being stated in the dissertation.
The interviews were semi-structured to ensure all important areas were
covered but to allow new, unanticipated issues to be raised. An interview
schedule was prepared but questions were sometimes rephrased and asked
in a different order according to the experiences that interviewees were
relating and so the questions remained relevant to each university. The
interview schedule was composed of four sections:
A: Search techniques and usage for the federated search tool;
B: Relationship between tool and other library services;
C: Databases and training;
D: Information literacy.
The sections were included to cover the dissertation objectives in the
following ways:
A: Objectives 1, 2, 4, 6
B: Objectives 3, 6
C: Objectives 5, 6
D: Objectives 5, 6
The questions included in the interview schedule were composed taking into
account the issues and concerns raised during the Open University case
study and the literature review. A number of questions were asked about the
type of training provided at the university so that the context for the federated
search tool and its effect on training could be understood. There were some
weaknesses with the interview schedule including question 20 about Google,
which was a confusing question and needed more clarification and
discussion before it could be answered. Sections C and D also had a
29
tendency to overlap and may have been more successful if combined into
one section with more open-ended questions. The schedule would have
benefited from a pilot but one was not conducted due to the small number of
universities with a federated search tool and unwillingness on the part of the
researcher to risk excluding useful material.
3.2.4 E-mail Survey
An e-mail survey was used to contact those universities with a
federated search tool that were located outside the UK or were too far to
travel to. Interviews may have provided more depth of material but were
impractical in these cases. It was important to include universities outside of
the UK because experiences with a number of federated search tools could
be included, whereas all the universities interviewed were using Metalib, the
dominant tool in this country. It was also useful to hear the issues and
concerns of universities in countries with different information literacy
agendas. An e-mail survey was chosen rather than a postal survey because
responses could be received faster and there were no postal costs. One
problem with e-mail surveys is that they are impersonal, so each was e-
mailed to an individual in the university who had been identified through the
literature or Internet search as having some responsibility for information
literacy education.
The survey included three questions and a request for hyper-links to
other information that was considered relevant. The number of questions
was limited to make the survey quick to answer and to encourage a high
response rate. The survey was developed after the initial university
interviews and the issues raised during these interviews informed the design.
The questions were also designed after considering the problems that had
arisen with the interview schedule, so were constructed to avoid confusion
and were open-ended to allow respondents to add as much detail as they felt
necessary. The questions were checked by the dissertation supervisor and
altered with her advice so they were more focused on key issues. Seventeen
universities currently using a federated search tool were contacted and of
these, seven responded.
30
3.3 Results Analysis
3.3.1 Open University Case Study
The three sets of results were analysed separately and analysis took
place during July and August. For the Open University case study the notes
were transcribed into a report in the days following the interviews and were
arranged by themes that had been apparent during the interviews. Since
some selection of material for recording had taken place during the
interviews this process of analysis was less complex than for the other
methods of investigation. The members of staff interviewed at the OU
checked the report and some small adjustments were made to the
ENCompass section to ensure accuracy.
3.3.2 Interviews at UK Universities
The four interviews at universities in the UK had been taped and these
tapes were transcribed and checked for errors. The researcher undertook
the transcription and the tapes were listened to several times to ensure
familiarity with the material. Silverman recommends researchers prepare
their own transcripts and listen to tapes repeatedly in order to help data
analysis (Silverman, 2000). The transcripts were then coded with colours
representing the different objectives that were being addressed in a pertinent
section. The findings for each objective were sectioned into themes before
the final results were written. The research objectives were chosen as the
organising principle for the interview results because a great deal of data had
been collected and there was a risk that the focus of the material would not
be maintained.
3.3.3 E-mail Survey
The results from the e-mail survey were gathered and each question in
turn was analysed. The questions had been focused to aid easy analysis
and since the response was small, analysis was relatively straightforward.
31
One of the responses was excluded because the university had only been
using the federated search tool for a short period of time but this respondent
had recommended some relevant articles that were included in the literature
review.
3.4 Further Methodological Considerations
3.4.1 Ethical Issues
The research in this dissertation focused on the opinions and thoughts
of individuals and so it was necessary to consider related ethical issues. All
participants in the case study, interviews and e-mail survey consented to
their responses being used for the purposes of the dissertation and were
informed of the way in which the information would be used. The information
obtained was stored privately and appears anonymously in the dissertation.
This was important because the respondents may have expressed opinions
that were sensitive or not shared by others at their university.
3.4.2 Research Limitations
There were a number of limitations with the research. The Open
University case study was a more limited study than had originally been
anticipated because the implementation of ENCompass was at an earlier
stage than expected. If the federated search tool had been in the piloting
phase then users, academics and subject librarians from the piloted
departments could have been interviewed. The interviews at universities in
the UK were limited because all the universities were using the same
federated search tool. The results obtained may therefore reflect the
implications for information literacy training with the introduction of a
particular type of federated search tool rather than any tool. This limitation
was mainly due to one tool being dominant in the country but it would have
advantageous to have found institutions using other tools and have included
them in the study. The time and travel constraints of the dissertation also
32
prohibited traveling to universities in other countries or undertaking more than
a limited number of interviews. Including an e-mail survey to universities in
other countries was one way of trying to draw upon a wider range of
experience. Interviewing more individuals at those universities that were
included in the research may also have enhanced the study. Other library
and information services staff could have been interviewed, or the research
could have been expanded to include the issues and concerns expressed by
academic staff or students. During the interviews it became clear that a
study of the search techniques of students would have been very useful but
was not possible due to the research for the dissertation occurring during the
summer vacation.
The methods of investigation that were used could have been improved
in several ways. The interview schedule for the university interviews had a
couple of problematic questions that may have been spotted during an
interview pilot. These questions were re-phrased in later interviews in an
attempt to avoid leading or confusing questions. The schedule also covered
a large number of topics and may have benefited from focusing more on the
issues relating directly to information literacy and federated search tools.
Instead this focusing occurred during the data analysis but more in-depth
discussions may have taken place with a shorter, more focused interview
schedule.
The e-mail survey would have benefited from a larger response rate.
The response may have been larger if those universities that failed to
respond were contacted again or if more individuals were contacted at each
university. Using the information from the survey together with the interviews
helped to broaden the scope of the research and was an attempt to minimize
the limitations of each method of investigation.
33
Chapter 4: Open University Case Study
4.1 Aims of Open University Case Study
1. Investigate the current and future information literacy strategies so
these can be considered in light of the introduction of ENCompass.
2. Examine the current state of the federated search tool, ENCompass
and find out about the implementation procedure and any issues that
have arisen during implementation.
3. Highlight any concerns staff may have about the implementation of
ENCompass and the effect it may have upon students and their
information literacy development.
4.2 Case Study Evidence
Interviews
Interviews were conducted with five members of staff, three associated
with information literacy and two with the implementation of ENCompass.
The interviews took place in the work environment of the interviewees.
Documents presented during the interviews
Library and Learning Resources Centre, OU. Encompass Overview.
Ramsden, A. (2004) Federated Searching and Encompass.
(Presentation slides handout for Oxford trainee librarians visit)
Other documents consulted
Open University (2004). Step Forward [Online]. Milton Keynes: OU.
http://www3.open.ac.uk/stepforward [Accessed 10 June 2004].
Parker, J. (2003). “Putting the Pieces Together: Information Literacy at
the Open University”. Library Management [Online], 24 (4/5).
http://www.emeraldlibrary.com [Accessed 10 June 2004].