Gardner Susan

8/12/2019 Gardner Susan

1/34

8/12/2019 Gardner Susan

2/34

Loyola Marymount University

Private Jesuit and

Marymount universityin Los Angeles, CA

More than 5600

undergraduate and1800 graduatestudents

New $63 millionWilliam H. HannonLibrary

8/12/2019 Gardner Susan

3/34

Goals for Revising FEP Library

Instruction

Create standardized intro to research process

Could apply to any topic Develop learning outcomes and tie to ACRL Info

Literacy Competency Standards

Develop scoring rubric to evaluate skills

8/12/2019 Gardner Susan

4/34

Content Selection

Applied Jerilyn Veldofs One-Shot instructional

design methodology Prioritized content: need to know versus nice to

know

Task Analysis: steps, teaching points, learningobjectives

8/12/2019 Gardner Susan

5/34

Sample Task Analysis

8/12/2019 Gardner Susan

6/34

Student Learning Outcomes

1) Given a broad research topic, use the 4W questions (who,

what, where, when) to write a research question.2) Given a research topic, compile a list of search terms or

keywords.

3) Given background information about Google and the Library,

list two differences between the two related to content,organization, or quality.

4) Given a research topic and access to the library's catalog,find relevant books on your topic.

5) Given a research topic and access to a general article indexdatabase (ProQuest), find relevant articles on your topic.

8/12/2019 Gardner Susan

7/34

The Worksheet: Module 1

http//:libguides.lmu.edu/ENGL110

8/12/2019 Gardner Susan

8/34

The Worksheet: Module 2

8/12/2019 Gardner Susan

9/34

The Worksheet: Module 3

8/12/2019 Gardner Susan

10/34

The Worksheet: Module 4

8/12/2019 Gardner Susan

11/34

The Worksheet: Module 5

8/12/2019 Gardner Susan

12/34

LibGuide: Excerpt Module 1

8/12/2019 Gardner Susan

13/34

Evaluation: Developing a Rubric

Scoring rubric to assess how well students achieved

learning outcomes Analytic rubric: divides each performance into

separate facets with separate scales

Breaks down 5 learning modules into subsectionswith learning objectives and corresponding ACRLstandards

8/12/2019 Gardner Susan

14/34

Rubric: Module 1

8/12/2019 Gardner Susan

15/34

Rubric: Module 2

8/12/2019 Gardner Susan

16/34

Rubric: Module 3

8/12/2019 Gardner Susan

17/34

Rubric: Module 4 part 1

8/12/2019 Gardner Susan

18/34

Rubric: Module 4 part 2

8/12/2019 Gardner Susan

19/34

Rubric: Module 5 part 1

8/12/2019 Gardner Susan

20/34

Rubric: Module 5 part 2

8/12/2019 Gardner Susan

21/34

Calibrating the Rubric

8/12/2019 Gardner Susan

22/34

Calibrating the Rubric

Objective: reach percent-agreement consensus

among all 8 graders Several rounds of group grading, followed by

rigorous discussion

Tested sample of 20 worksheets by assigning 2independent judges to each

Reached at least 92% agreement

8/12/2019 Gardner Susan

23/34

Problem Areas and Revisions

4 Ws- changed to completion (1.a.)

Changed synonyms to keywords and added atotal of (2.b.)

Added or characteristics of (3)

Added criteria for relevancy (2.b., 4.d., 5.d.)

Clearer directions for scoring citations (4 and 5)

Blank answers (all)

8/12/2019 Gardner Susan

24/34

Design/Methodology

755 worksheets collected from Fall 2009

100 random worksheets single-graded Graders recorded scores in Google form

Transferred data to Excel file- inserted formulas for

weights Average scores for all modules/subsections

8/12/2019 Gardner Susan

25/34

Results

8/12/2019 Gardner Susan

26/34

Results: By Subgroup

8/12/2019 Gardner Susan

27/34

Discussion: Module 3

Shortened Module 3

and standardized Adapted language

from the ENGL 110textbook to talk aboutinformation quality

Made it more spatialby having visual image

8/12/2019 Gardner Susan

28/34

Discussion: Modules 1 & 2

Added additional examples to LibGuide of research

questions Changed worksheet example from eating disorders

to something neutral

Added more keyword practice tools Better reflects cyclical nature of research process-

can add keywords, change research questionthroughout

8/12/2019 Gardner Susan

29/34

Discussion: Modules 4 & 5

Changed Modules 4 and 5 to match stated learning

outcome lists citation elements in correct fields Divided Module 4 into elements for citing and

finding

Modified to only 1 book and 1 article rather than 2(relevancy over quantity)

8/12/2019 Gardner Susan

30/34

Conclusions

Some limitations of the rubric:

Gray areas in trying to judge good researchquestions and relevancy

Frustration trying to calibrate

Failure of percent agreement method to correct forchance

8/12/2019 Gardner Susan

31/34

Conclusions

Led to increased standardization in library

instruction across all freshman classes Led to greater communication of our instruction

goals

Teaching expectations clearer for librarians

8/12/2019 Gardner Susan

32/34

Conclusions

Easier to connect our learning outcomes with ACRL

standards Led to improvements in teaching material for lower-

scoring modules

New benchmark for defining success in Fall 2010

8/12/2019 Gardner Susan

33/34

Questions?

Links:

Old ENGL 110 LibGuidehttp://libguides.lmu.edu/Fall2009ENGL110

New ENGL 110 LibGuide

http://libguides.lmu.edu/ENGL110

Thank You: LMU Reference Department

Meghan Oakleaf

8/12/2019 Gardner Susan

34/34

CONTACT US:

Susan Gardner [email protected]

Elisa Slater Acosta [email protected]