ARE WE GETTING WARMER? QUERY CLARIFICATION IN VIRTUAL REFERENCE Marie L. Radford Lynn Silipigni Connaway Library Research Round Table ALA Annual Conference.

ARE WE GETTING WARMER? QUERY CLARIFICATION IN VIRTUAL REFERENCE

Marie L. Radford Lynn Silipigni Connaway

Library Research Round TableALA Annual ConferenceWashington, DC

June 23, 2007

2

Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User, & Librarian Perspectives

Funded by Institute of Museum & Library Services Rutgers University & OCLC (10/05-9/07)

Four phases:I. Focus group interviewsII. Analysis of 850 QuestionPoint live chat

transcriptsIII. 600 online surveysIV. 300 telephone interviews

3

Would you clarify?

L# Time Message 1 5:55 U Hello. I need help finding

information on the following: where in baltimore, maryland can I find peppermint oil, if a 15 year old can start diving classes now, and can a 15 year old get a office job at this age?

4

A helpful answer?

2 5:55 L [A librarian will be with you in a minute.] 3 6:03 L [Librarian X – A librarian has joined the

session.] 4 6:38 L One moment please. 5 7:05 L I will see what I can find. 6 7:25 U alright 7 11:54 L After looking at a few web sites it seems

that beginning scuba diving classes start at age 12 to 16. It depends on how good a swimmer the person is. As a scuba diver myself I think the age range sounds right.

5

i don’t want scuba diving classesi want driving classes

8 12:18 U i don’t want scuba diving classes 9 12:27 U i want driving classes 10 15:21 L From the us department of labor site. The Fair

Labor Standards Act (FLSA)… 11 16:27 U So how do I go about findint a office job for my

15 year old daughter 12 16:44 U and also I want to now about

driving classes not scuba diving 14 18:36 U alright\ 15 06:13 [user has closed this session]

6

Do librarians clarify? Good news!

n=581

75%

22%

3%

0%

10%

20%

30%

40%

50%

60%

70%

80%

Yes No No clarificationnecessary

7

When librarians clarify…

50% (217) asked the highly recommended follow-up question:

“Does this completely answer your question?”

n=434

8

What do librarians clarify?

Topic Background Verify Their Understanding Search History Type of Resource Extent/Depth Referral Corrected User Misunderstanding

9

Librarians ask about TOPIC

45% (195) sought TOPIC information

“Is there a specific animal that you're researching?”

n=434

10

Librarians ask about BACKGROUND

31% (133) sought BACKGROUND information

“When is your paper due?”

“Are you in the NYC area?”

n=434

11

Librarians VERIFY

20% (87) VERIFIED their understanding

“So you are looking for the reasons why the Japanese attacked Pearl Harbor?”

n=434

12

Librarians ask about SEARCH HISTORY

19% (83) sought SEARCH HISTORY information

“Ok, and have you checked their Web sites yet?”

n=434

13

Librarians ask about TYPE OF RESOURCE

11% (49) sought TYPE OF RESOURCE information

“Are you looking for books, articles, or online information?”

n=434

14

Librarians ask about EXTENT/DEPTH

11% (48) sought EXTENT/DEPTH information

“Were you looking for in-depth history? Or a summary?”

n=434

15

REFERRAL

4% (16) asked a REFERRAL question

“Do you want me to refer your question to your library?”

n=434

16

CORRECTED MISUNDERSTANDING

2% (7) CORRECTED MISUNDERSTANDING

“By the way, what I told you about the list being in strictly chronological order was not correct; I'm sorry.”

n=434

17

How about the Users?

Users offered information about: Topic Background Extent/Depth Correct Librarian’s Misunderstanding

18

Users Clarify TOPIC

52% (69) offered TOPIC information

“I am doing marketing reseach on marylands restaurant industry”

n=132

19

Users Clarify BACKGROUND

24% (32) offered BACKGROUND information

“I am in 7th grade GT Sciene and I am Doing a 2 year research project…”

n=132

20

Users Gave EXTENT/DEPTH

20% (26) offered EXTENT/DEPTH information

“the thing is i need it to be simple and not to wordy”

n=132

21

Users CORRECT Librarian’s Misunderstanding

16% (21) CORRECTED the librarian’s misunderstanding

“No. The form says I need the correct name for the position.”

22

Users VERIFY

14% (19) VERIFIED their understanding

“So the best thing is to find it on the online catalog… is that correct?”

n=132

23

Users Offer SEARCH HISTORY

14% (18) offered SEARCH HISTORY information

“I looked in the card catalogue and didn't see it there...”

n=132

24

Users provide TYPE OF RESOURCE

9% (12) offered TYPE OF RESOURCE information

“...And [not] a book, online though”

“Yes, but I’m looking for more historical and architectural resources” n=132

25

Users ask for REFERRAL

4% (5) asked for a REFERRAL

“Can you contact me with anyone from the […] Library”

n=132

26

When did librarians & users clarify?

Surprising finding…

Two different patterns of clarification!

Librarians clarified in beginning of interaction

Users clarified in middle of interaction

27

When did they ask for clarification?

51%

32% 33%

8%

18%

2%0%

10%

20%

30%

40%

50%

60%

Beginning/Beforesearching

Middle/During searching End/After searching

Pe

rce

nta

ge

of

tra

ns

cri

pts

Librarian

User

n=581

28

Types of clarifying questions… Open or closed?

Percentage of all questions asked by librarians:

66% (554) Closed Questions

34% (284) Open Questions

n=838

29

Closed or open: Is it clear?

“Can you tell me more about what you are looking for?” An invitation to talk, but still a closed

question?

“Can I ask what grade level this is for so I don’t give you something that is too technical or too easy?” “Ok” An unhelpful, closed response to an

ostensibly open question.

30

Imposed Query

“Imposed queries emanate from both formal and informal relationships between people”

(Gross, 1999, p. 501).

“For a history project, I must do a poster diagram of the evolution of pyramids from masatbas to step pyramids to straight-sided pyramids.”

31

“User identifies an information need and asks a question” (Gross, 1995, p. 237).

“I want to plan a camping and rock climbing trip to Pilot Mountain. Do you have any recommendations?”

Self-Generated Query

32

Imposed query vs. Self-generated query

46%

24%

1%

28%

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

Unknown Self-generated query Imposed Query -School assignment

Imposed Query -Work related(n=581)

(9)

(140)

(165)

(267)

33

What makes a difference for accuracy?

Analysis of Accuracy of Ready Reference questions (n=180)

Does question clarification make a difference?

Does asking a follow-up question make a difference?

YOU BE THE JUDGE…

34

YES! Clarifying & follow-up improve accuracy!

84

47

18 80

20

40

60

80

100

120

140

160

180

Clarified Follow-up

Correct

Incorrect

35

Future Directions

Continue to collect & analyze data Online surveys

Librarian survey completed Non-user and User surveys in progress

Telephone interviews 100 with Librarians completed 100 Users in progress 100 Non-users in progress

36

END NOTES

• This is one of the outcomes from the project Seeking Synchronicity: Evaluating Virtual Reference Services from User, Non-User, and Librarian Perspectives.

• Funded by IMLS, Rutgers University, & OCLC Online Computer Library Center, Inc.

• Special thanks to Jocelyn DeAngelis Williams, Patrick Confer, Timothy Dickey, David M. Dragos, Mary Anne Reilly, Susanna Sabolcsi-Boros, Julie Strange.

• Slides available at project web site: http://www.oclc.org/research/projects/synchronicity/

37

Questions & Comments

Lynn Silipigni Connaway, Ph.D. Email: connawal@oclc.org www.oclc.org/research/staff/connaway.htm

Marie L. Radford, Ph.D. Email: mradford@scils.rutgers.edu www.scils.rutgers.edu/~mradford