Seeking Synchronicity: Evaluating Virtual Reference Transcripts Presented by Lynn Silipigni Connaway and Marie L. Radford QuestionPoint Users Group Meeting.

Seeking Synchronicity: Evaluating Virtual

Reference Transcripts

Presented by Lynn Silipigni Connaway

andMarie L. Radford

QuestionPoint Users Group MeetingJune 25, 2006

New Orleans, Louisiana

Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User, and Librarian Perspectives

• $1,103,572 project funded by:

– Institute of Museum and Library Services $684,996 grant

– Rutgers, The State University of New Jersey and OCLC Online Computer Library Center $405,076 in kind contributions

Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User, and Librarian Perspectives

Project duration10/1/2005-9/30/2007

Four phases:I. Focus group interviews*II. Analysis of 1,000+ QuestionPoint

transcriptsIII. 600 online surveys*IV. 300 telephone interviews*

*Interviews & surveys with VRS users, non-users, & librarians

Phase II:24/7 Transcript Analysis

• Generated random sample– July 7, 2004 through June 27, 2005– 263,673 sessions– 25 transcripts/month = 300 total

• 256 usable transcripts – Excluding system tests and technical problems

6 Analyses

• Geographical Distribution• Library receiving query• Library answering query

• Type of Library• Type of Questions

• Katz/Kaske Classification • Subject of Questions

• Dewy Decimal Classification• Session Duration •Interpersonal Communication

• Radford Classification

Librarian Location - Question Received

Other States = 10

United Kingdom = 1

Pennsylvania = 4

Arizona = 4

Kansas = 5

Delaware = 6

Canada = 7

New York = 7

Washington = 8

Utah = 8

North Carolina = 14

Massachusetts = 21

Australia = 36

Maryland = 47

California = 77

0 10 20 30 40 50 60 70 80 90

Number of Questionsn=255

Librarian Location - Question Referred/ Answered

Other states = 12

Germany = 1

Pennsylvania = 4

Michigan= 4

Colorado = 4

Hawaii = 6

Canada = 7

Washington = 7

North Carolina = 7

New York = 8

Connecticut = 9

Massachusetts = 10

Maryland = 35

Australia = 36

California = 88

0 10 20 30 40 50 60 70 80 90 100

Number of questionsn=238

Type of Library Receiving Question

0

10

20

30

40

50

60

70

80

90

Consortium Public University National Law State NotAvailable

K-12

Num

ber

of Q

uest

ions

n=256

Type of Question Asked

Procedural25%

Research2%

Subject Search37%

Inappropriate0%

Holdings6%

Ready Reference30%

n=273

Procedure and Subject

Language1%

Science8%

Technology5%

Arts & recreation4%

Literature5%

History & geography16%

Library procedure & miscellaneous

25%

Philosophy & psychology

1%

Compuer science, information & general

works4%

Social sciences31%

Religion0%

n=273

Dewey Decimal Classification

Arts & recreation5%

Literature6%

History & geography21%

Philosophy & psychology

1%

Compuer science, information & general

works5%

Religion0%

Social sciences42%

Technology7% Science

11%Language

2%

n=273

Service Duration

• Mean Service Duration: 13:53

• Median Service Duration: 10:37

Focus Group InterviewsReasons for Using VRS

• Convenient• Efficient• More reliable than search engines & free• Allows multi-tasking• Email follow-up & provision of transcript• Pleasant interpersonal experience

– Librarian on first name basis – more personalized• Less intimidating than physical reference desk

– Feel comfortable abruptly ending session

Focus Group InterviewsReasons for not using VRS

• Graduate students– Fear of

• Bothering librarian• Looking stupid & advisors finding out

– Questions may not be taken seriously– Potential technical problems– Bad experiences in FtF influence expectations of VRS

• Screenagers– Virtual stalkers (“psycho killers”)– Not finding a trusted librarian– Unsure of what to expect

Focus Group Interviews Challenges for Users & Non-Users

• Speed and technical problems• Delayed response time• Librarians are not in users’ libraries

– Fear of no subject expertise

• Fear of overwhelming librarian

Focus Group Interviews Suggestions from Users & Non-Users

• Inclusion of multiple languages• Access to subject specialists• Better marketing and publicity

– Information on how to connect and use VRS– Reassurance that users will not bother librarians – the

library wants the service to be used

• Faster technology• Improved interface design

– More color– More attractive

Next Steps• Conduct

– Three focus group interviews – VRS users– Online survey & telephone interviews with VRS

• Users• Non-users• Librarians

• Analyses– Gender– User Type

• Child/Young adult• Adult• Unknown

End Notes

• This is one of the outcomes from the project Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User, and Librarian Perspectives, Marie L. Radford and Lynn Silipigni Connaway, Co-Principal Investigators.

• Funded by IMLS, Rutgers University and OCLC, Online Computer Library Center.

• Project web site: http://www.oclc.org/research/projects/synchronicity/

Questions

Marie L. Radford, Ph.D.Email: mradford@scils.rutgers.eduwww.scils.rutgers.edu/~mradford

Lynn Silipigni Connaway, Ph.D.Email: connawal@oclc.orgwww.oclc.org/research/staff/connaway.htm