Actual & Perceived Information Literacy Skills in Pre‐Service Educators Prepared by Marcy Brown Dominique Turnbow Educational Technology 690 San Diego State University May 25, 2009
Actual & Perceived Information Literacy Skills in
Pre‐Service Educators
Prepared by
Marcy Brown Dominique Turnbow
Educational Technology 690 San Diego State University
May 25, 2009
Actual & Perceived Information Literacy Skills in Pre‐Service Educators
EDTEC 690: Brown/Turnbow 2
Table of Contents Overview ....................................................................................................................................................... 4
Literature Review .......................................................................................................................................... 5
Information Literacy within Higher Education .......................................................................................... 5
Libraries’ role in information literacy ................................................................................................... 5
Instruments to Assess Information Literacy Skills and Perception ........................................................... 6
Instruments to assess information literacy skills .................................................................................. 6
Assessing perceptions using Competency Theory ................................................................................ 6
Study Instrument Design ........................................................................................................................... 7
Questions to test actual skills ............................................................................................................... 7
Questions to test perceptions of knowledge ........................................................................................ 8
Information Literacy within Teacher Education ....................................................................................... 8
Information literacy standards and accreditation ................................................................................ 9
Information literacy instruction within teacher education coursework............................................. 10
Recommendations for change ............................................................................................................ 10
Methodology ............................................................................................................................................... 10
Participants ............................................................................................................................................. 11
Instrument Development ........................................................................................................................ 11
Administration ........................................................................................................................................ 12
Contextual Factors ...................................................................................................................................... 12
Study Sample .......................................................................................................................................... 12
Instrument .............................................................................................................................................. 13
Findings ....................................................................................................................................................... 13
Participant Demographics ....................................................................................................................... 13
Actual Information Literacy Skills – Whole Number Scores ................................................................... 14
Skills Perception – Estimates of Performance ........................................................................................ 14
Correlation of Actual and Perceived Skills .............................................................................................. 15
Actual Information Literacy Skills – Score Quartile ................................................................................. 16
Skills Perception – Estimates Relative to Other Respondents ................................................................ 17
Correlation of Actual and Perceived Quartiles ....................................................................................... 17
Actual & Perceived Information Literacy Skills in Pre‐Service Educators
EDTEC 690: Brown/Turnbow 3
Comfort with Research‐Related Skills ..................................................................................................... 18
Correlation of Actual Skills and Research Comfort ................................................................................. 19
Library Use .............................................................................................................................................. 20
Correlation of Actual Skills and Library Use ............................................................................................ 20
Discussion ................................................................................................................................................... 21
Conclusion ................................................................................................................................................... 22
Appendix A: Results for Question 7 ............................................................................................................ 23
Appendix B: Results for Question 4 ............................................................................................................ 24
References .................................................................................................................................................. 25
Instrument…………………………………………………………………………………………………………………………………………….27
Actual & Perceived Information Literacy Skills in Pre‐Service Educators
EDTEC 690: Brown/Turnbow 4
Overview
Today’s college graduates must be information literate; in fact, most accrediting agencies for higher
education—among them the Western Association of Schools and Colleges (WASC) – emphasize the
criticality of this competence, regardless of the degree students earn.
But what does it mean to be information literate? Researchers offer many definitions–some abstract,
some concrete. Following is the characterization provided by the Association for College &Research
Libraries (ACRL):
Information literacy is a set of abilities requiring individuals to recognize when information is
needed and have the ability to locate, evaluate, and use effectively the needed information
(2000).
ACRL offers a detailed list of student competencies for librarians and other educators to use in planning
instruction. They are organized into five “standards”; each includes several performance indicators and
outcomes.
The researchers explored whether a pre‐service educator’s perception of his or her ability to identify
potential information sources relates to his or her actual ability to identify information sources.
Specifically, this study focused on ACRL’s Standard 2: The information literate student accesses needed
information effectively and efficiently. The specific performance indicators for this standard are below
while the indicators to which this study attended are shown in italics.
1. The information literate student selects the most appropriate investigative methods or
information retrieval systems for accessing the needed information.
2. The information literate student constructs and implements effectively‐designed search
strategies.
3. The information literate student retrieves information online or in person using a variety of
methods.
4. The information literate student refines the search strategy if necessary.
5. The information literate student extracts, records, and manages the information and its sources
information literate student reevaluates the nature and extent of the information need.
Actual & Perceived Information Literacy Skills in Pre‐Service Educators
EDTEC 690: Brown/Turnbow 5
In practice, many librarians aim interventions toward students who express lower levels of confidence in
their information literacy skills. The results of this study addressed whether or not this is a worthwhile
strategy. An alternative might be to more broadly target students—including those who appear
confident in their information literacy and library skills.
Literature Review
The researchers drew heavily from the Library &Information Science (LIS) and Education disciplines in
preparing this review. It provides an overview of how information literacy is taught within higher
education, instruments that have been used to assess skills and perception, and information literacy
within teacher education.
Information Literacy within Higher Education
Universities are turning their attention to assessing student information literacy skills on their campuses
largely due to accrediting agencies requirements (Hernon & Dugan, 2004). The methods used to deliver
instruction and assessment may be institutionally driven or instructor driven. In most cases, librarians –
as recognized experts on information finding and use – provide information literacy instruction.
Libraries’ role in information literacy
The way information literacy instruction is delivered on academic campuses is as varied as the campuses
themselves. Here are some examples of librarian‐led information literacy instruction:
• One‐Shot. “One‐shot” instruction is provided to a class as requested by a faculty member. These
sessions can be general overviews of how to use the library or focused on a specific paper or
project students need to complete for their course.
• “Just in Time.” Librarians sometimes provide several workshops to students as requested by a
faculty member. The workshops are dispersed throughout the course and cover topics that
students need when they need them.
• “First‐year experience.” This one‐unit course is usually a semester long and targets new and
transfer students. The goal is to provide students with essential skills that will help them
succeed in college, such as note taking, research, and critical thinking.
• Reference & consultations. These are the services with which most library users are familiar.
Through the reference desk or a consultation, librarians provide individual instruction with
users.
Actual & Perceived Information Literacy Skills in Pre‐Service Educators
EDTEC 690: Brown/Turnbow 6
• Online instruction. Online instruction is a new, growing area for many libraries. It takes many
forms, from a small, one‐minute video about how to search a database (Turnbow, 2009b) to a
full‐length online workshop, including assessment (Turnbow, 2009a).
Whether or not Information Literacy instruction is required in the curriculum varies across college and
university campuses. Regardless, formalized “testing” or assessment of these skills is typically not
required by the campuses. Without support at a campus‐wide level, librarians will continue to struggle
to provide an accurate picture of student information literacy skills. Despite this, many librarians have
developed and administered instruments in order to illustrate student information literacy skills – or
lack thereof – to their campus administrations. These will be discussed below.
Instruments to Assess Information Literacy Skills and Perception
Instruments to assess information literacy skills
Many instruments have been developed to measure information literacy competencies (i.e. actual skills)
based on the ACRL standards. Neely (2006) discusses the use of many of these instruments in detail. In
contrast, only a handful of studies have examined the correlation between "actual" performance on
information literacy tasks with students' perceived performance (Cameron, Wise, & Lottridge, 2007;
Geffert & Christensen, 1998; Ivanitskaya, Laus, & Marie Casey, 2004; Maughan, 2001; Ren, 2000).
Unfortunately, most of these instruments have not been validated. As far as the researchers are aware,
none of these studies included correlation analysis of participant results with perceived level of skill.
Assessing perceptions using Competency Theory
The lean research base focused on the correlation between student perceptions of their information
literacy skills and actual performance on tasks tends to be grounded in competency theory; Kruger and
Dunning (1999) are largely credited with developing this train of thought. They suggest that people who
perform at a low skill level lack the metacognitive abilities to recognize their own incompetence.
Therefore, they cannot accurately assess their own or others’ skills and tend to overestimate their
abilities. Further, people that perform at a very high skill level tend to underestimate their performance.
Kruger and Dunning detail four studies that they designed to test their predictions in regard to
competency theory. Three of those studies directly relate to the relationships explored in the current
study of perceived versus actual information literacy.
Actual & Perceived Information Literacy Skills in Pre‐Service Educators
EDTEC 690: Brown/Turnbow 7
1. In the first, 65 undergraduate students were asked to rate how funny given jokes were, as well
as rate their own ability to recognize what’s funny. The researchers found that participants were
moderately able to rate their own identification of humor, but overestimated their ability
relative to their peers (p. 1123).
2. The second study explored the logical reasoning skills of 45 undergraduate students. The
students completed 20 questions from the Law School Admissions Test, and then made three
estimates about their abilities and test performance. Participants overestimated their logical
reasoning abilities relative to their peers (pp. 1124‐1125).
3. Study 3a asked 84 undergraduates to estimate their knowledge of American standard written
grammar, to estimate their knowledge relative to their peers, and then to complete a 20‐item
grammar test. “As in studies 1 and 2, participants overestimated their ability and performance
relative to objective criteria” (p. 1125).
Kruger and Dunning conclude, “Indeed, across the four studies, participants in the bottom quartile not
only overestimated themselves, but thought they were above average” (p. 1130). They go on to say,
“Across the four sets of studies, participants in the top quartile tended to underestimate their ability
and test performance relative to their peers” (p. 1131).
In 2007, Gross and Latham used competency theory to explain the significant correlation they
discovered between incoming freshmen students’ actual performance on information literacy (IL) tasks
compared to their perceptions. Participants were 51 incoming freshman who were either in the top 25%
or bottom 25% of their high school graduating classes based in GPA and SAT scores. In addition to
testing the participants’ information literacy skills, Gross and Latham also asked them to estimate their
performance both before and after taking the test and their performance relative to other participants.
They analyzed IL scores and performance estimates by percentage, by estimated number of correct
answers, and by expectations of performance compared to others. “In all cases these variables were
positively correlated, but not to a significant degree” (343). These findings mirror those of Kruger and
Dunning as they relate to information literacy.
Study Instrument Design
Questions to test actual skills
The researchers discovered four validated instruments to test information literacy skills based on the
ACRL competencies.
Actual & Perceived Information Literacy Skills in Pre‐Service Educators
EDTEC 690: Brown/Turnbow 8
1. Information Literacy Test (ILT) developed at James Madison University (Cameron et al., 2007)
2. Research Readiness Self‐Assessment (RRSA) from Central Michigan University (Ivanitskaya et al.,
2004)
3. Project SAILS from Kent State University (Kent State University, 2000‐2009)
4. ETS iSkills – Information and Communication Technology Literacy Test (ETS, 2009)
Due to high costs and access issues, the researchers were not able to use any of these instruments to
test actual information literacy skills in this study. They instead developed an instrument largely based
on validated information literacy test items from Morner (1993) and from other instruments that were
compiled by Neely (2006).
Questions to test perceptions of knowledge
The questions that were used to test perceived skills were largely based on studies conducted by Kruger
and Dunning (1999), Ehrlinger and Dunning (2003), and Gross and Latham (2007).
Gross and Latham (2007) used a simple method to test student perception of information literacy skills.
After completing the questions that tested actual information literacy skills, 58 incoming freshman at a
large state university were asked to:
1. Give a percentile estimate of their performance.
2. Estimate how many questions they thought they got right.
3. Give a percentile estimate of their performance relative to other students in the study.
For this study, the researchers used a similar method to this one.
Information Literacy within Teacher Education
Information literacy is a core component of teacher preparation—not surprising given the growing
number of content‐based professional organizations (e.g., the National Council of Teachers of English,
the National Council of the Social Studies and the National Council of Teachers of Mathematics) that
advocate for information literacy integration within all aspects of the K‐12 enterprise. The need for pre‐
service educators to demonstrate information literacy has been formalized by the nation’s primary
accrediting body, the National Council for Accreditation of Teacher Education (NCATE).
Cited in a 2008 study by Birch, Greenfield, Janke, Schaeffer, and Woods were the results of an informal
survey of College of Education faculty at the University of Arizona. Ten faculty members who teach pre‐
Actual & Perceived Information Literacy Skills in Pre‐Service Educators
EDTEC 690: Brown/Turnbow 9
service educators were asked the question: “What information literacy skills do your students need to
prepare them to teach?” The following skills were identified (p. 370):
• Finding resources
• Organizing information
• Establishing priorities
• Maintaining research skills
• Keeping current
• Evaluating information quality
Results from this very small informal survey cannot be generalized, but do point to a general awareness
of information literacy among education faculty.
Information literacy standards and accreditation
In the 1990s, several education‐related professional organizations revised program standards for
teacher education in order to include information literacy competencies; among them were the National
Council for the Social Studies, the National Council of Teachers of English, and the International Reading
Association (Henderson & Scheffler, 2003). During the same time period, many university accrediting
bodies incorporated information literacy into accreditation standards. Agencies recognizing information
literacy included the Southern Association of Colleges and Schools, the New England Association of
Colleges and Schools, the North Central Accrediting Agency, and the Middle States Association
(Henderson & Scheffler, 2003).
The publication of the 2000 NCATE standards was a watershed for information literacy education, with
competence in information and technology literacy highly promoted/emphasized. In particular,
Standard 1 (Candidate Knowledge, Skills, and Dispositions) features the following behaviors that pre‐
service educators must display (Birch et al., 2008, p.370):
• Demonstrated ability to use tools and processes of inquiry
• Critical analysis
• Reflective practice
• Data collection
• Integration of technology and information literacy in instruction to support student learning
Actual & Perceived Information Literacy Skills in Pre‐Service Educators
EDTEC 690: Brown/Turnbow 10
These behaviors closely mirror the ACRL standards described above. Existing information literacy
instruction, if closely tied to those ACRL standards, may be sufficient to meet the needs of candidates in
teacher education programs.
Information literacy instruction within teacher education coursework
Details of specific information literacy instruction within teacher preparation programs are scarce in the
professional literature. Henderson and Scheffler (2003) describe a few such programs within teacher
education curricula, most of which were conducted under the leadership of academic libraries.
A few years later, Birch and her colleagues (2008)identified several partnerships between teacher
education departments and college or university libraries designed to meet NCATE information literacy
and technology standards: “Partnerships between teacher education faculty and education librarians
improve the likelihood that both NCATE and ACRL competencies will be integrated with and reinforced
by content instruction and performance” (p. 370). Examples include the following:
1. The Information Literacy Project, University of British Columbia. This ongoing program allows
pre‐service educators to observe both the planning and implementation of an information
literacy class before planning one independently (Asselin & Lee, 2002).
2. Westfield State College. An education librarian and two professors of education collaborated
with 200 students to create the Pre‐Service Information Literacy Model (Birch et al., 2008).
3. Wayne State University. Librarians are paired with education faculty to team teach a research
methods course for early child education graduate students (Bhavnagri & Bielat, 2005).
Recommendations for change
Even the collaborative models described above appear librarian‐led, and many are published as case
studies within library science literature. Teacher education programs must ultimately become
responsible for integrating information literacy competencies into the curricula. These programs must
design robust assessment strategies to ensure mastery of information literacy skills, and offer ongoing
professional development opportunities related to information literacy to both pre‐service educators
and faculty alike.
Methodology
The study used a closed‐question survey to explore the possible correlation between actual and
perceived information literacy skills in pre‐service educators. The survey was designed to measure both
Actual & Perceived Information Literacy Skills in Pre‐Service Educators
EDTEC 690: Brown/Turnbow 11
variables as well as collect demographic and other basic data that could be used to perform secondary
analyses if warranted. To recap, items measuring actual information literacy skills were derived from the
ACRL Information Literacy Competency Standards for Higher Education, with each mapped to a specific
performance indicator within Standard Two.
Participants
The sample for this correlation study was a convenience sample composed of students in a single, online
section of EDTEC 470: Technologies for Teaching, at San Diego State University. All are enrolled in the 5th
year credential program, seeking a variety of credentials. They complete EDTEC 470 to meet state
credentialing requirements associated with technology use and instructional integration. Sixteen (n=16)
respondents completed the survey and were included in the analysis.
Instrument Development
The instrument was developed and administered using SurveyMonkey, a subscription‐based survey‐
generation service for the web. The survey contained a total of 20 numbered items, although some were
multi‐part and/or required multiple responses. The entire survey instrument is available for review in
Appendix C.
One cluster of items was organized around demography and background. For example, respondents
were asked to indicate their educational background, undergraduate major, year bachelor’s degree was
earned (selected from a range), and the highest degree obtained. Three multi‐part items called for
respondents to categorize the frequency of their interactions with various library services such as in‐
person reference, email reference, and online tutorials. These items supported a secondary analysis of
correlation between depth and frequency of prior library interaction with actual information literacy
skills. The final background item required respondents to rate their comfort with various research tasks
such as identifying a relevant library database, using Boolean operators, or creating a reference list. The
results from this item were also useful for a secondary analysis of correlation between comfort with
research and either perceived or actual skills.
Eleven numbered items (some with multiple parts) measured each participant’s performance on one or
more ACRL performance indicators. Respondents could score a total of 29 points on this section. The
researchers divided the scores into three categories: Not Proficient (NP), Proficient (P) and Advanced
Proficiency (AP). Scores that reflected 0‐65% correct (0‐18) were NP, those that fell within 66‐90%
Actual & Perceived Information Literacy Skills in Pre‐Service Educators
EDTEC 690: Brown/Turnbow 12
correct (19‐25 points) were P, and those from 91‐100% correct (26‐29 points) were AP. This
methodology was used in a similar study by Gross and Latham (2007, p. 338).
Similar to the Gross and Latham study mentioned above, the final two items on the survey measured
each respondent’s perception of how s/he performed on the skills questions. The first asked the
respondent to estimate how many questions were answered correctly, and the second item asked the
respondent to estimate percentile compared to other classmates completing the survey.
Administration
The EDTEC 470 course instructor sent an email to all enrolled students (n=41) inviting them to complete
the survey. Included in the email message were a brief overview of the study, information about the
course for which this study was designed, dates during which the survey would be active, and an email
link to the instrument in SurveyMonkey. The instructor also informed the email recipients that any
student who completed the survey could be entered into a drawing for a $10 Starbucks gift card.
Students were given two weeks in which to submit their responses. Only five people had completed the
survey by the conclusion of week one, so a reminder email was sent by the section instructor. By this
point, the number of enrolled students had by this time dropped to 38.
Contextual Factors
This section discusses the limitations the researchers encountered with the sample and instrumentation
for this study.
Study Sample
The researchers used a convenience sample of students enrolled in a single section of EDTEC 470,
Technologies for Teaching at San Diego State University. At the time that the survey was distributed,
enrollment in the section stood at 41 students, although several students withdrew from the class
before the data collection period ended. Additionally, there was a low response rate (n=16) to the
survey. While the researchers did not perform a sample size calculation for this analysis, they estimate
that the study was underpowered. Fraenkel and Wallen suggest that “the minimum acceptable sample
size for a correlational study is considered by most researchers to be no less than 30” (2009, p. 335).
Because of the small sample size in this study, correlations may not accurately describe relationships
between variables.
Actual & Perceived Information Literacy Skills in Pre‐Service Educators
EDTEC 690: Brown/Turnbow 13
Instrument
Few instruments have been developed to study the correlation between actual and perceived
information literacy skills; thus, the researchers had to develop this instrument largely from scratch.
Preliminary analysis of the data suggests that a correlation might have been more accurately measured
if the scales for the two variables were the same. Actual information literacy was scored between 0 and
29, representing the actual number of items answered correctly by a participant on the skills portion of
the survey. Perceived information literacy was scored between 1 and 4, representing the participant’s
selection of a statement that most described how s/he thought he did on the skills portion. While a
correlation coefficient may be generated for any paired scores regardless of scale, the researchers
theorize that the coefficient for this study may have been different if participants were asked to guess
their actual score on the skills portion using the same (0‐29) scale. Similarly, participants were asked to
select a quartile when estimating how their scores compared to those of classmates completing the
survey. The correlation between perceived and actual percentile might have been more accurate if
participants were asked to guess their actual percentile.
Findings
This section details participant demographics as they relate to this study as well as results of correlation
analyses that compared participant test scores with perceptions of test performance.
Participant Demographics
Sixteen students enrolled in the section completed the survey by the end of the two‐week period. The
charts below illustrate the distribution of the undergraduate degrees earned by the participants by
broad subject area and year. Respondents were fairly well distributed between social sciences (n=6, or
38%), liberal studies (n=5, or 31%), and the humanities (n=4, or 25%). Eleven (69%) were recent
graduates, receiving their undergraduate degrees in 2004 or later. Only two respondents (13%)
completed any education beyond a bachelor’s degree (both indicated that they held a master’s or
professional degree).
Actual & Perceived Information Literacy Skills in Pre‐Service Educators
EDTEC 690: Brown/Turnbow 14
Actual Information Literacy Skills – Whole Number Scores
Scores on the skills questions (items 8 through 18) were totaled for each survey respondent, with 29
points possible. Scores ranged from a low of 14 points to a high of 24 points. The median score was 19,
falling just into the range of points considered Proficient. The mode, or most commonly occurring score,
was 17—which for this study was considered Not Proficient.
Results suggest that pre‐service educators at the post‐baccalaureate level are not necessarily
information literate. No one scored in the range for Advanced Proficiency although nine participants
(56%) scored between 66% and 90%, placing them in the Proficient category. Seven respondents (44%);
a fairly high number, scored 65% or lower, making them Not Proficient.
Skills Perception – Estimates of Performance
The final two questions on the survey (items 19 and 20) elicited the respondents’ perceptions of how
well they performed on the skills portion of the survey. Item 19 asked the respondent to estimate the
6
5
4
1
Participants: Undergraduate Degree by Subject Area
Social Sciences
Liberal Studies
Humanities
Natural Sciences
11
4
10
2
4
6
8
10
12
after 2004 2001‐2004 1986‐1990
Participants: Year Bachelor's Degree Obtained
Actual & Perceived Information Literacy Skills in Pre‐Service Educators
EDTEC 690: Brown/Turnbow 15
percentage of skills‐based questions s/he answered correctly. This closed question offered four answer
choices, scored as follows:
I think I answered 0‐25% correctly (0‐2 questions) = 1 point
26‐50% correctly (3‐5 questions) = 2 points
51‐75% correctly (6‐8 questions) = 3 points
76‐100% correctly (9‐11 questions) = 4 points
Only one respondent (n=1; 6%) selected 0‐25%, scoring 1 point; one respondent (6%) selected 76‐100%,
scoring 4 points. The rest of the participants (n=14; 88%) selected one of the two middle choices. The
answer selected most frequently was the third choice, with 10 respondents (63%) estimating that they
answered 51% to 75% of the skills questions correctly. Four respondents (25%) selected 26‐50%. That
88% of respondents placed themselves in one of the middle two quartiles might indicate that the
students in this sample neither underestimate nor overestimate their information literacy skills.
Correlation of Actual and Perceived Skills Each actual skills score (14 points through 24 points) was paired with the corresponding perception
score (1 through 4). The paired scores were entered into Excel so that a correlation coefficient
(Pearson’s r) could be generated.
Rounding to three decimal places, the correlation coefficient was 0.296. Fraenkel and Wallen state that
correlation coefficients below 0.35 show only a slight relationship between variables (p. 337).The two‐
tailed p‐value for this correlation is 0.27. Although the correlation is slightly positive, this p‐value
indicates a non‐significant relationship between actual and perceived information literacy skills in this
sample.
Actual & Perceived Information Literacy Skills in Pre‐Service Educators
EDTEC 690: Brown/Turnbow 16
Kruger and Dunning’s competency theory, described in the literature review, hypothesizes that
competent individuals tend to underrate their abilities, while less competent people tend to overrate
their abilities when performing specific tasks. Competency theory thus suggests that any correlation of
perceived and actual skills would likely be negative. This analysis did not confirm that competency
theory was at work when pre‐service educators were asked about their perceptions of their information
literacy skills.
Actual Information Literacy Skills – Score Quartile
To explore performance more systematically, scores were assigned to quartiles. Because the final score
was a whole number and there were multiple values at several of the data points, the scores did not fall
into four evenly divided groups. The Excel Quartile function was used to determine cutoff points for
each quartile.
There were six respondents (38%) who fell into the first quartile: one score at 14 points and the five
respondents who scored 17. Three respondents (19%) scored 18 or 19 points and landed in the second
quartile. There was one score each at 20, 21, and 22 points, meaning 19% fell into the third quartile.
Three respondents received 23 total points and one scored 24, placing these four respondents (25%)
into the fourth and final quartile.
00.51
1.52
2.53
3.54
4.5
0 5 10 15 20 25 30
Perceived Inform
ation Literacy
Actual Information Literacy Skills Scores
Correlation: Actual & Perceived Skillsr=0.296; p=0.27
Actual & Perceived Information Literacy Skills in Pre‐Service Educators
EDTEC 690: Brown/Turnbow 17
Skills Perception – Estimates Relative to Other Respondents
Respondents were also asked how well they thought they performed on the skills questions in
comparison to their classmates who also completed the survey. This question (item 20) was very similar
in construction and scoring to item 19:
I think I did as well as 0‐25% of the students = 1 point
26‐50% of the students = 2 points
51‐75% of the students = 3 points
76‐100% of the students = 4 points
Three respondents (19%) rated themselves in the highest quartile, scoring 4 points. Seven respondents
(44%) felt that they performed as well as 51‐75% of the students, scoring 3 points. The choice for the
second quartile – 26‐50% of the students – was selected by 5 respondents, or 31%. Only one person
(6%) estimated that they fell into the lowest quartile. The majority of participants again estimated their
relative performance in the middle two quartiles, possibly indicating a reluctance to overestimate or
underestimate performance.
Correlation of Actual and Perceived Quartiles The researchers next paired the actual quartile into which each respondent’s score fell with that
respondent’s perceived quartile. The correlation coefficient for these sets of variables was 0.514. This
can be considered a moderately positive correlation. Fraenkel and Wallen state that while correlations
of at least 0.50 allow “crude predictions…such predictions will be subject to sizable errors” (p. 337). The
p‐value for this correlation coefficient was 0.04, indicating a significant probability of a positive
6
3 3
438%
19% 19%
25%
0
1
2
3
4
5
6
7
Quartile 1 Quartile 2 Quartile 3 Quartile 4
Participant Score Quartiles (n=16)
Actual & Perceived Information Literacy Skills in Pre‐Service Educators
EDTEC 690: Brown/Turnbow 18
relationship between these variables. It appears that participants were able to positively predict into
which quartiles their scores would fall.
The fact that this correlation was stronger than the prior one suggests that students were better at
judging their skills and abilities in relation to others than they were at judging their own skills
independent of any comparison standard.
Comfort with ResearchRelated Skills
One multi‐part question on the survey (Item 7) called for respondents to rate their comfort with 10
specific research‐related activities. Each activity corresponded to at least one question on the skills
portion of the instrument. Responses were scored according to degree of comfort:
Not at all [comfortable] = 1 point
Somewhat = 2 points
Moderately = 3 points
Very = 4 points
Extremely = 5 points
With 10 activities scored at a maximum of five points each, respondents could score a composite 50
points on this research comfort scale. Scores ranged from a low of 23 to a high of 50, with the median
score at 36. The mode was also 36.
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
0 1 2 3 4 5
Perceived Qua
rtile
Actual Quartile
Correlation: Actual & Perceived Quartiler=0.514; p=0.04
Actual & Perceived Information Literacy Skills in Pre‐Service Educators
EDTEC 690: Brown/Turnbow 19
Respondents indicated the greatest degree of comfort with basic library‐related tasks, such as finding
out whether the library had a specific book. Eight people (50%) indicated that they were “extremely
comfortable” with this task. The research activity that was least comfortable for this sample asked about
the respondent’s ability to use wildcards in a database search. Nine people (56%) were “not at all
comfortable” with this task, and only one person was “extremely comfortable.”
Appendix A includes a chart that illustrates how participants’ rated their comfort with various activities.
Correlation of Actual Skills and Research Comfort The researchers paired the actual skills scores (14 points through 24 points) with the corresponding
research comfort scores, which ranged from 23 to 50. This correlation was the weakest in this study
(Pearson’s r = 0.090). The two‐tailed p‐value for this r value is 0.74.
The lack of correlation between a respondent’s information literacy skills and his or her comfort with
various research tasks has many implications for information literacy instruction. Many campuses make
information literacy instruction available through optional tutorials or workshops on specific research
tasks. When instruction is optional, students might select only those tutorials or workshops for research
tasks with which they feel uncomfortable. This analysis hints that a student’s comfort level with a task
may not indicate a level of proficiency or competence with that same task. Mandatory instruction is one
solution to the lack of correlation.
0
10
20
30
40
50
60
0 5 10 15 20 25 30
Research Com
fort Score
Actual Information Literacy Skills
Correlation: Actual Score & Research Comfortr=0.09; p=0.74
Actual & Perceived Information Literacy Skills in Pre‐Service Educators
EDTEC 690: Brown/Turnbow 20
Library Use
Library use was measured in a single question (Item 4) asking respondents to rate the frequency of
usage of three types of libraries: public, university, or specialized. For each of the three library types,
scoring was as follows:
At least one time per week = 3 points
At least one or two times per month = 2 points
At least one or two times per semester = 1 point
Can’t recall = 0 points
Never = 0 points
Appendix B includes a chart that illustrates how often participants used these types of libraries.
Points for each library type were totaled to give a composite library use score. Scores for the 16
respondents ranged from 0 to 6, with a median of 2.5 points.
Correlation of Actual Skills and Library Use For this final correlation analysis, the researchers paired each actual skills score with the respondent’s
library use score. As described above, this score ranged from 0 to 6 in the sample. Pearson’s r for this
correlation was 0.443, indicating a moderately positive correlation. For two‐tailed probability, p=0.09.
The researchers hypothesized that the frequency of library visits would have a positive impact on
information literacy skills. This correlation coefficient r indicates there is some positive relationship
0
1
2
3
4
5
6
7
0 5 10 15 20 25 30
Library Use
Actual Information Literacy Skills
Correlation: Actual Score & Library User=0.443; p=0.09
Actual & Perceived Information Literacy Skills in Pre‐Service Educators
EDTEC 690: Brown/Turnbow 21
between library usage and information literacy, although the p value implies that the significance of this
relationship may be no stronger than random.
Discussion
Overall, the results of this study do not support the results found in other correlation studies completed
by Kruger and Dunning (1999) and Gross and Latham (2007). As mentioned previously, this may be in
large part due to a low response rate and the instrument design. A study that includes a larger sample
(preferably over 30) may yield more accurate correlations. In addition, an instrument that asks
participants to rate their performance based on percentages instead of quartiles would allow the data to
be correlated more accurately.
The researchers were surprised by the number of participants (n=8) who correctly perceived the
percentage of questions they answered correctly, and the number of participants (n=7) who accurately
estimated the quartile into which they fell. Prior research related to competency theory suggests that
this number would be lower compared to the total included in the study sample. The researchers
hypothesize that participants’ perceptions might be different if the “metatopic”—in this case,
information literacy – was one in which students had more working knowledge. For example, the Kruger
and Dunning (1999) study asked participants to rate their abilities to identify humor and grammar. Many
people perceive themselves as funny and having command of the English language. This study shows
that the participants feel “somewhat” or “moderately” comfortable to perform information literacy
tasks (Appendix A). It is possible that there would be a greater difference in the actual versus perceived
performance if participants were tested on a more common topic or one in which knowledge is
considered desirable.
The research comfort score was derived from responses to the research tasks listed in Item 7. Each of
these tasks maps to at least one skills question in the survey. Further analysis of the correlation
between the tasks in Item 7 and the corresponding skills questions might also prove interesting. For
example, can a participant who feels “extremely comfortable” finding a book in a library catalog actually
do so? This research question was not within the scope for this study, but may yield interesting results.
Actual & Perceived Information Literacy Skills in Pre‐Service Educators
EDTEC 690: Brown/Turnbow 22
Conclusion
Higher education is increasingly interested in understanding students’ ability to attain information
literacy skills. Major associations in librarianship and teacher education already recognize the need for
students to learn these skills. For the past decade, librarians have been at the center of teaching and
assessing such skills. Librarians and faculty must partner to provide information literacy instruction that
is integrated into the curriculum. Identifying students’ actual information literacy skills and perceived
ability to complete such tasks will provide a place for librarians and faculty to address specific
information literacy skills in their own institutions.
EDTEC690: Brown/Turnbow 23
Appendix A: Results for Question 7
13
9
1 12 31
5
2
3 22 1
4
6 5
5
3
4
44
5
3
3
5 5
4
2
5 7 36
8
3 36
31
4 35 5
How comfortable are you doing the following things when you do research? (n=16)
extremely
very
moderately
somewhat
not at all
EDTEC690: Brown/Turnbow 24
Appendix B: Results for Question 4
1
32
44
4
1
4
3
5 5
12
0
2
4
6
8
10
12
14
16
18
Public Library Academic Library Specialized Library
During this academic year, how often have you visited one or more of the libraries listed below? (n=16)
Never
Can't recall
At least 1x or 2x/semester
At least 1x or 2x/month
At least 1x/week
EDTEC690: Brown/Turnbow 25
References
Asselin, M. M., & Lee, E. A. (2002). "I wish someone had taught me": Information literacy in a teacher education program. Teacher Librarian, 30(2), 10‐17.
Association of College & Research Libraries. (2000). Information literacy competency standards for higher education. Chicago, IL: American Library Association.
Bhavnagri, N. P., & Bielat, V. (2005). Faculty‐librarian collaboration to teach research skills: Electronic symbiosis. Reference Librarian, 43(89), 121‐138.
Birch, T., Greenfield, L., Janke, K., Schaeffer, D., & Woods, A. (2008). Partnering with librarians to meet NCATE standards in teacher education. Education, 128(3), 369‐379.
Cameron, L., Wise, S. L., & Lottridge, S. M. (2007). The development and validation of the information literacy test. College & Research Libraries, 68(3), 229‐236.
Ehrlinger, J., & Dunning, D. (2003). How chronic self‐views influence (and potentially mislead) estimates of performance. Journal of Personality and Social Psychology, 84(1), 5‐17.
ETS. (2009). iSkills assessment ‐ information and communication technology literacy test. Retrieved March 13, 2009, from http://www.ets.org/portal/site/ets/menuitem.1488512ecfd5b8849a77b13bc3921509/?vgnextoid=159f0e3c27a85110VgnVCM10000022f95190RCRD&vgnextchannel=e5b2a79898a85110VgnVCM10000022f95190RCRD
Fraenkel, J. R., & Wallen, N. E. (2009). How to design and evaluate research in education (7th ed.). New York: McGraw‐Hill.
Geffert, B., & Christensen, B. (1998). Things they carry: Attitudes toward, opinions about, and knowledge of libraries and research among incoming college students. Reference & User Services Quarterly, 37(3), 279‐289.
Gross, M., & Latham, D. (2007). Attaining information literacy: An investigation of the relationship between skill level, self‐estimates of skill, and library anxiety. Library & Information Science Research, 29(3), 332‐353.
Henderson, M. V., & Scheffler, A. J. (2003). New literacies, standards, and teacher education. Education, 124(2), 390‐395.
Hernon, P., & Dugan, R. E. (2004). Outcomes assessment in higher education : Views and perspectives. Westport, CT: Libraries Unlimited.
Ivanitskaya, L., Laus, R., & Marie Casey, A. (2004). Research readiness self‐assessment: Assessing students' research skills and attitudes. Journal of Library Administration, 41(1), 167‐183.
EDTEC690: Brown/Turnbow 26
Kent State University. (2000‐2009). Project SAILS (standardized assessment of information literacy skills). Retrieved March 13, 2009, from https://www.projectsails.org
Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: How difficulties in recognizing one's own incompetence lead to inflated self‐assessments. Journal of Personality and Social Psychology, 77(6), 1121‐1134.
Maughan, P. D. (2001). Assessing information literacy among undergraduates: A discussion of the literature and the University of California‐Berkeley assessment experience. College & Research Libraries, 62(1), 71.
Morner, C. J. (1993). A test of library research skills for education doctoral students. (Doctoral dissertation, Boston College, 1993).
Neely, T. Y. (2006). Information literacy assessment: Standards‐based tools and assignments. Chicago: American Library Association.
Ren, W. (2000). Library instruction and college student self‐efficacy in electronic information searching. Journal of Academic Librarianship, 26(5), 323.
Turnbow, D. (2009a). BENG1 tutorial. Retrieved March 13, 2009, from http://biomed.ucsd.edu/dturnbow/bioeng1_aftersessions/beng1_tutorial_aftersessions/player.html
Turnbow, D. (2009b). Getting started: PubMed. Retrieved March 13, 2009, from http://biomed.ucsd.edu/dturnbow/beng1_pubmed/
EDTEC 690 SurveyEDTEC 690 SurveyEDTEC 690 SurveyEDTEC 690 Survey
We are EDTEC graduate students enrolled this semester in a research class. Both the study we're conducting and this survey focus on
the information literacy skills of preservice educators like you. We hope you're willing to participate; you'll need about 10 to 15
minutes to answer all the questions--and both your anonymity and confidentiality are assured.
We are using a definition of information literacy provided by the Association for College & Research Libraries (ACRL):
"Information literacy is a set of abilities requiring individuals to recognize when information is needed and have the ability to locate, evaluate,
and use effectively the needed information." Basically this means that you know how and where to find and use information when you
have a specific question or research need. This survey is focused on your ability to find information.
We would like you to answer each question. However, it is never a problem and you shouldn't worry if you don't know or aren't sure of
an answer. Just select "I don't know" and move to the next question.
Everyone who completes the survey by April 17 will be entered into a drawing for a $10 Starbucks gift card. There are instructions at
the end of the survey for submitting your name for the drawing.
If you have questions about this study you may contact one of the researchers via e-mail - Dominique Turnbow or Marcy Brown.
EDTEC 690 SurveyEDTEC 690 SurveyEDTEC 690 SurveyEDTEC 690 Survey
1. Please select the category that best describes your undergraduate major.
2. When did you graduate from college with your bachelor's degree?
3. Please indicate your highest degree obtained?
4. During this academic year, how often have you visited one or more of the libraries listed below?
5. Think about your interactions with librarians while you were getting your undergraduate degree. Below, there are a variety of ways that you may have had contact with a librarian or library staff member. For each one, tell us whether or notit applies to you (choices=y/n). Then tell us how often you did this. [Choose Not applicable if your answer to the first question was No.]
Getting to know you
Please tell us a little about yourself and how you use libraries.
*
*
*
*
At least 1x/weekAt least 1x or
2x/month
At least 1x or
2x/semesterCan't recall Never
Public library nmlkj nmlkj nmlkj nmlkj nmlkj
University library nmlkj nmlkj nmlkj nmlkj nmlkj
Specialized library nmlkj nmlkj nmlkj nmlkj nmlkj
*
Did you do this? If yes, how often?
Asked a question at a service desk at the library
Asked a question via e-mail, chat, or txt message
A librarian came to my class and did a workshop
Went to the library for a workshop
Used an online tutorial on the library's web page to
learn how to search the catalog or a database
Arts (visual and performing)
nmlkj
Humanities (e.g. languages, literature, history)
nmlkj
Liberal Studies
nmlkj
Physical or Life Sciences (e.g. biology, chemistry, engineering)
nmlkj
Social Sciences (e.g. psychology, geography)
nmlkj
After 2004
nmlkj
Between 2001 and 2004
nmlkj
Between 1996 and 2000
nmlkj
Between 1991 and 1995
nmlkj
Between 1986 and 1990
nmlkj
Before 1986
nmlkj
Bachelor's degree
nmlkj
Post-graduate certificate
nmlkj
Professional/Master's degree
nmlkj
Other
nmlkj
EDTEC 690 SurveyEDTEC 690 SurveyEDTEC 690 SurveyEDTEC 690 Survey6. Think about your experiences using a library "service" for a college-level project, paper or other assignment you had to complete. Several library services (and reasons for using them) are listed below. For each one, first indicate whether or notit applies to you (choices=yes/no). Then tell us how often you took advantage of it. [Choose Not applicable if your answer to the first question was no.]
*
Did you do this? If yes, how often?
Used the library catalog to find books,
encyclopedias and handbooks
Used library databases to find scholarly journal
articles
Went to the library for help identifying relevant
research articles or books
Went to the library to get help with a reference list
or bibliography
EDTEC 690 SurveyEDTEC 690 SurveyEDTEC 690 SurveyEDTEC 690 Survey
7. How comfortable are you doing the following things when you do research?
Comfort level with research skills
* not at all somewhat moderately very extremely
Finding out if your library has a
particular booknmlkj nmlkj nmlkj nmlkj nmlkj
Understanding when to use a library
catalog or article databasenmlkj nmlkj nmlkj nmlkj nmlkj
Identifying the online databases that
match your topic of interestnmlkj nmlkj nmlkj nmlkj nmlkj
Identifying keywords/phrases for your
research topicnmlkj nmlkj nmlkj nmlkj nmlkj
Using Boolean operators (AND,OR,NOT)
as part of a database searchnmlkj nmlkj nmlkj nmlkj nmlkj
Using truncation or wildcards (*) as
part a database searchnmlkj nmlkj nmlkj nmlkj nmlkj
Modifying a database search when you
get too many or too few resultsnmlkj nmlkj nmlkj nmlkj nmlkj
Identifying the most relevant results
for your research topic after searching
a database
nmlkj nmlkj nmlkj nmlkj nmlkj
Getting the full-text of an article for
free or a small fee from a librarynmlkj nmlkj nmlkj nmlkj nmlkj
Creating a reference list or bibliography
for your paper using an appropriate
style (e.g. APA, MLA)
nmlkj nmlkj nmlkj nmlkj nmlkj
EDTEC 690 SurveyEDTEC 690 SurveyEDTEC 690 SurveyEDTEC 690 Survey
8. The service offered in the Library that allows you to get almost any publication you need is called:
9. If you are looking for a specific journal article and have the publication information, you can use:
10. What types of information can you find in the SDSU Library subscription databases? (Select all that apply.)
Doing research at SDSU
The questions on this page will ask you about doing research at San Diego State University. Remember, it is OK to select "I don't
know" if you are unsure of the answer.
*
*
*
Interlibrary loan
nmlkj
Full text
nmlkj
Reference
nmlkj
Reserves
nmlkj
I don't know
nmlkj
Citation linker
nmlkj
Digital projects page
nmlkj
Electronic course reserves (ECR)
nmlkj
None of the above
nmlkj
I don't know
nmlkj
conference proceedings
gfedc
dissertations
gfedc
full text of journal articles
gfedc
journal article citations
gfedc
all of the above
gfedc
I don't know
gfedc
EDTEC 690 SurveyEDTEC 690 SurveyEDTEC 690 SurveyEDTEC 690 Survey
11. Below are pieces of information needed for various class assignments. Match each specific information need to the type of library source most likely to meet that need. Note: you will not use every source type.
12. Which database would be most likely to contain articles about bilingual education?
Research tasks
The next two questions will ask you about different sources and why you might use them for research. Remember, it is OK to select
"I don't know" if you are unsure of the answer.
*
Sources
Magazine article about vitamin supplements
Titles of books written by B. F. Skinner available in
the library
Census data for San Diego county
Daily accounts of the Wall Street bankruptcies
*
Academic Search Premier (EBSCO)
nmlkj
Applied Science and Technology Abstracts
nmlkj
ERIC
nmlkj
Historical Abstracts
nmlkj
PubMed
nmlkj
I don't know
nmlkj
EDTEC 690 SurveyEDTEC 690 SurveyEDTEC 690 SurveyEDTEC 690 Survey
13. You searched for articles about distance education and don’t think you found enough information. What synonyms might you use to search further? (Select all that apply.)
14. Which of the following search statements would you use in a database to find records that would include either of these terms: leadership; managerial skills
15. What is your next logical step when your search yields 2,000 citations?
16. The following is a list of strategies a researcher can use to narrow or broaden a search. For each strategy, indicate which would be used to find fewer results (narrow) or find more results (broaden).
Search strategies
The questions on this page will ask you about creating search strategies to find information about a topic. Remember, it is OK to
select "I don't know" if you are unsure of the answer.
*
*
*
*
narrow (fewer results) broaden (more results) I don't know
add more search terms with OR nmlkj nmlkj nmlkj
limit to a specific format (i.e. journal articles, book
chapters, etc.nmlkj nmlkj nmlkj
limit to date published nmlkj nmlkj nmlkj
use synonyms in the same search nmlkj nmlkj nmlkj
add specific author name(s) nmlkj nmlkj nmlkj
use truncation (when you substitute the last letter(s)
of a word with a symbol like "*" or "$", e.g.
manage*)
nmlkj nmlkj nmlkj
Distance training
gfedc
Online education
gfedc
Online training
gfedc
Web-based training
gfedc
All of the above
gfedc
None of the above
gfedc
I don't know
gfedc
Leadership AND managerial skills
nmlkj
Leadership OR managerial skills
nmlkj
Leadership NOT managerial skills
nmlkj
I don't know
nmlkj
Add another term to the search using AND
nmlkj
Add another term to the search using OR
nmlkj
Add another term to the search using NOT
nmlkj
Look at all of the citations in order to be thorough
nmlkj
Look at the first few citations and just use them if they are relevant
nmlkj
I don't know
nmlkj
EDTEC 690 SurveyEDTEC 690 SurveyEDTEC 690 SurveyEDTEC 690 Survey
17. The image below is a citation from a library database. Select the "element" from the drop-down menu that best describes the part illustrated in the citation.
Citation from a library database
18. The following are citations for different types of publications. Mark which publication type matches each citation. Note: you will not use every publication type.
Managing information
The next two questions will ask you about how to create reference lists and analyze the information you find. Remember, it is OK to
select "I don't know" if you are unsure of the answer.
*
Citation element
author
title of journal
volume
pages
issue
date
title of article
*
bookbook
chapter
conference
proceeding
journal
article
magazine
articlenewspaper
I don't
know
Koepke, M. (1991). Expertise for sale: Countless
organizations use teachers as consultants. Teacher
Magazine, 3(1), 30.
nmlkj nmlkj nmlkj nmlkj nmlkj nmlkj nmlkj
Foster, N. F., & Gibbons, S. (2007). Studying Students:
The Undergraduate Research Project at the University of
Rochester. Chicago: Association of College and
Research Libraries.
nmlkj nmlkj nmlkj nmlkj nmlkj nmlkj nmlkj
Kierniesky, N. C. (2005). Undergraduate research in
small psychology departments: Two decades later.
Teaching of Psychology, 32(2), 84-91.
nmlkj nmlkj nmlkj nmlkj nmlkj nmlkj nmlkj
Lopatto, D. (2004). Survey of undergraduate research
experiences: First findings. Cell Biology Education, 3(4),
270-275.
nmlkj nmlkj nmlkj nmlkj nmlkj nmlkj nmlkj
Duignan, P. A. (2006). Key challenges for educational
leaders. In M. L. Brown, Educational Leadership: Key
Challenges and Ethical Tensions (pp. 21-40). Cambridge,
UK: Cambridge University Press.
nmlkj nmlkj nmlkj nmlkj nmlkj nmlkj nmlkj
EDTEC 690 SurveyEDTEC 690 SurveyEDTEC 690 SurveyEDTEC 690 Survey
19. There were 11 questions on this survey focused on information literacy skills. How many do you think you answered correctly?
20. Now, how well do you think you did compared to other students enrolled in yourEDTEC 470 course?
How do you think you did?
Now, we are interested in how you think you did on the research questions.
*
*
I think I answered 0-25% correctly (0-2 questions)
nmlkj
I think I answered 26-50% correctly (3-5 questions)
nmlkj
I think I answered 51-75% correctly (6-8 questions)
nmlkj
I think I answered 76-100% correctly (9-11 questions)
nmlkj
I think I did as well as 0-25% of the students.
nmlkj
I think I did as well as 26-50% of the students.
nmlkj
I think I did as well as 51-75% of the students.
nmlkj
I think I did as well as 76-100% of the students.
nmlkj