Studying and Using Multiple Documents in History Effects of Discipline Expertise

7/30/2019 Studying and Using Multiple Documents in History Effects of Discipline Expertise

http://slidepdf.com/reader/full/studying-and-using-multiple-documents-in-history-effects-of-discipline-expertise 1/23

Studying and Using Multiple Documents in History: Effects of Discipline ExpertiseAuthor(s): Jean-François Rouet, Monik Favart, M. Anne Britt and Charles A. PerfettiSource: Cognition and Instruction, Vol. 15, No. 1 (1997), pp. 85-106Published by: Taylor & Francis, Ltd.

Stable URL: http://www.jstor.org/stable/3233756 .

Accessed: 10/04/2013 23:53

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of

content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms

of scholarship. For more information about JSTOR, please contact [email protected].

.

Taylor & Francis, Ltd. is collaborating with JSTOR to digitize, preserve and extend access to Cognition and

Instruction.

http://www.jstor.org

This content downloaded from 152.74.16.35 on Wed, 10 Apr 2013 23:53:31 PMAll use subject to JSTOR Terms and Conditions

http://www.jstor.org/action/showPublisher?publisherCode=taylorfrancis

http://www.jstor.org/stable/3233756?origin=JSTOR-pdf

http://www.jstor.org/page/info/about/policies/terms.jsp





http://www.jstor.org/stable/3233756?origin=JSTOR-pdf

http://www.jstor.org/action/showPublisher?publisherCode=taylorfrancis



COGNITIONND NSTRUCTION,5(1), 5-106

Copyright1997,Lawrencerlbaumssociates,nc.

StudyingandUsing MultipleDocuments n History:

Effects of Discipline Expertise

Jean-FrangoisRouetandMonikFavart

Languageand Communication aboratory

Universityof Poitiers

M. Anne Britt and CharlesA. Perfetti

LearningResearchandDevelopmentCenter

Universityof Pittsburgh

Extensiverainingn history esults n generalizednowledge f the methods ndinformationources ypicalof historyproblems,hat s, discipline xpertise.We

investigatedhe nfluence fdiscipline xpertisenstudents'eading,valuation,nd

useof multiple ocuments bouta historicalontroversy.levengraduatetudentsinpsychologyhistory ovices) nd8graduatetudentsnhistoryhistorypecialists)studied controversiesbouthehistory f thePanama anal.Foreachcontroversy,thestudentstudied set of documents, roteanopinion ssay,andevaluatedhe

documentsorusefulness nd rustworthiness.tudy trategiesidnotdiffer ignifi-cantlyacrossgroups.However,heevaluationf usefulness aried s a function fdocumentypeand students' xpertise.Furthermore,oviceandexpert tudentsdifferedntheway hey xpressednd upportednopinionntheir ssay.Wesuggestthatdiscipline xpertisehelps history tudents onnect nformationourcesand

interpretationso their epresentationf thesituation rproblem.

Historical knowledge is conveyed througha variety of texts and documents:

textbooks, historians'essays, memoirs, official documents,and other forms of

discourse. Whenstudying

a particular opic or event, the history studentmust

progressivelyconstructa mentalrepresentationrommultiplesources of informa-

Requests orreprints houldbe sent toJean-Frangois ouet,LanguageandCommunicationLabora-

tory,95 Avenue du RecteurPineau,86022 PoitiersCedex,France.E-mail:[email protected]








86 ROUET, AVART, RITT, ERFETTI

tion. Furthermore, istoryinvolves permanent econstruction ndreinterpretationof pastevents, and our knowledgeof most history topics entails some degreeof

uncertainty ndcontroversy.As aconsequence,manyhistoricalaccountsare ainted

with incompleteness,inconsistencies,or bias. Thus, skilled learningof historyincludes the ability to integrate,to complete, and to challenge the knowledge

conveyedthroughmultiplehistoricaldocuments.What s the nature f thecognitiveskills involvedin usinghistoricaldocuments?Do they developas partof students'

general iteracy,ordo they belong exclusivelyto thedisciplineof history?We examine this questionwith a studyof history disciplinespecialists,asking

how suchdisciplinespecialistsdifferfromacademically omparable onspecialistsin their

approacho a

particularistorical

problem.A

keyidea in ourexamination

is the distinctionbetweendomainanddisciplineexpertise.Academic rainingeads

to expertise in specific domains.For a historian,this domainexpertisemay be

19th-centuryAmerica,post-WorldWarI Europe,and so on. Ourassumption s

thatdiscipline specializationprovidesan overlapping butnot identical)expertisein the tools of the discipline. These include its characteristicproblem types,information ources,methodsof analysis,and standards f evidence. To capturethe character f disciplineknowledge, tis necessary ogainsome controlof domain

or topicknowledge.

Domainknowledge,not discipline knowledge,has been the objectof studyinmost researchon text understanding.Thus,for example,Voss and his colleagues

(Spilich,Vesonder,Chiesi,& Voss, 1979;Voss, Vesonder,& Spilich, 1980) have

shown that individualswith more initialknowledgeof the game of baseballhave

a bettermemoryfor verbalaccountsof a particular ame.Recht and Leslie (1988)found similarresultswith 12-year-oldreaders.They showed that the influenceof

domainknowledge s independentromreadingabilityandtypeof posttest(verbalvs. pictorial).Online studies haveshown thatdomainexpertsapplymore efficient

text comprehension trategies(Afflerbach,1989) becausethey can activate more

elaboraterepresentations f the situationorproblem Means& Voss, 1985; Voss,Greene,Post,&Penner,1983).Domainexpertsalsoapplymoreflexible ruleswhen

evaluating he importanceof statements n a scientific text (Dee Lucas& Larkin,

1986, 1988). Thus, in additionto content areaknowledge, domainexpertsmay

possessmoreelaboratemodelsof the discoursestructuresypicalof theirdiscipline

(Dillon, 1991).

History expertisehas been studiedas ablendof domainanddisciplineknOwl-edge. Wineburg(1991) observed whatwe call disciplineexpertisewhen he sug-

gested thathistoryexpertsuse specific strategieswhen studyingmultipledocu-

ments. In Wineburg's study, high school seniors and historyexperts (graduatestudents and faculty) were asked to study a set of paintings and documents

representing he battleof Lexington.Expertsshowed three distinctstrategiesnot

sharedby novices. First,they focusedon the sourceinformationattached o each








STUDYING MULTIPLEDOCUMENTSINHISTORY 87

document,even prior oreading t.Expertsused source nformation o evaluateand

interprethe documentcontent.

Bycontrast,

highschoolseniors

paidittle attention

to source information,except when they readthe textbookexcerpt, which theyfound most useful and reliable.Second, expertscorroboratednformationacross

documents.They were sensitive to discrepancies,and they establishedrelations

among different versions of the events. High schools seniors tended to consider

each source in isolationandseldom noticeddiscrepanciesacrosssources.Third,theexpertscontextualizeddocument nformation singtheirownknowledgeof the

situation.Theyused contextualknowledgeto evaluate heaccuracyandreliabilityof the documents.Carretero nd Limon(1995) found thatexperiencedhistorians

were able to draw more elaborateconclusions (comparedwith undergraduatehistory majors)from a set of texts and tables about a particularopic in Spanish

history.Thesestudiessuggestthathistorians,but notnonhistorians,andistinguish

among different types of evidence and can interpret nformationfrom amongsources.

Other tudies,however,provideevidencethat nexpert tudentshave somesense

of documentsas evidence.Perfetti,Britt,andGeorgi(1995) observed hatcollege-level studentsareable to detect bias inhistoricalaccounts,although heypay little

attention o primary vidence. Rouet, Britt,Mason,and Perfetti 1996) found that

college-level studentswithlittleexperience n historycandistinguishamongtypesof historical sources. When asked to evaluate several documentsrelated to a

historicalcontroversy, hestudents oundprimarydocuments e.g., treatyexcerpts,

correspondence)more useful andtrustworthyhanopinionatedhistoricalessays.Furthermore,he studentssometimesreferred o thetypeof documentperse (e.g.,"Itis an official document")n order ojustify theirpreference.The studentsalso

cited mostly primarydocumentswhen writinganessay aboutthecontroversy.The studies nhistory,as well asin otherexpert-novice ontrasts,have notforged

the separationbetweendiscipline specializationand domainknowledge,however.

As we noted earlier,the two overlapbut are not identical.Whetherdisciplinespecializationbringscharacteristictrategiesbeyondassociateddomainknowledgeis an interestingand difficult issue. It is interestingbecauseevidenceof discipline

knowledge would instantiatea kindof generalized earning,one thatcomes from

trainingin the methodsof a discipline. It is a difficult issue becausediscipline

knowledgecannot be separated asily fromdomainknowledge.Inthis study,our

strategy is to comparestudents of comparableacademic level-two groups of

graduatestudents-who differprimarily n the type of training hey have chosen

to take.Althoughgraduate tudentscannotbeconsidered xperts nthe usualsense,

they have been extensively exposed to problemsanddiscoursestructuresypicalof thedisciplineof history(e.g., studyingcontroversial ventsbased onconflicting

interpretations).Werefer o theseexpertstudentsasdisciplinespecialists.Students

fromsome otherdiscipline-in ourcase,psychology-are referred o asdiscipline








88 ROUET,FAVART, BRITT,PERFETTI

novices.'To controlthe effect of topicknowledge,we selected a topic unfamiliar

to all theparticipants.The experimentaddressedthree questions. First, do discipline novices and

specialistsusedifferent tudystrategies?We anticipatedhatexpertstudentswould

tend to focus on primaryevidence, as opposed to secondhandaccounts,and to

review the documents more often in order to corroborate nformationacross

sources.Second,do disciplinenovices andspecialistsevaluatedocument nforma-

tion differently?Based on the studyby Wineburg 1991), expertstudentscan be

expected to find primarydocumentsmost useful, whereas novices will rely on

textbook nformation rhistorians'accounts.However,studiesby Perfettiand his

colleagues (Perfettiet al., 1995; Rouet,Britt,et

al., 1996) suggestthat

inexperthistory students do find primarydocuments useful and trustworthywhen given

explicit studydirections.We anticipatedhatbothnovices andexpertswould find

primarydocumentsmostuseful,but thatexpertswoulduse moreelaborate riteria

to justify their evaluations.Third,do discipline novices and specialistsdiffer in

theiruse of document nformationn anessay task?We anticipatedhattheexpertstudentswould use more elaboratereasoningheuristicsandexpressa morethor-

ough examinationof the problemspace (i.e., the possible interpretations f the

events).

METHOD

Participants

The participantswere 19 graduate tudentsengaged in doctoralprogramsat the

Universityof Poitiers France)who were recruitedndividuallyandparticipated s

volunteers.Eleven of theparticipantsweregraduate tudents npsychology(disci-

pline novices),and8 weregraduate tudents n history(disciplinespecialists).The

two groupswereequivalent n age, sex, and academic evel.

Materials

The materialswere based on twocontroversies elated o thehistoryof thePanama

Canal.A controversywas definedas anissue about he causes or consequencesof

a series of events over whichhistoriansdisagree.One controversyconcerned he

justification ortheU.S. military nterventionn the 1903 PanamaRevolution; he

othercontroversynvolvedtheconsequencesof the1903Treatybetween heUnitedStatesandPanama.For eachcontroversy,we translatedhechronologyandtheset

IInmostFrenchuniversities,undergraduate rograms restronglyspecialized.








STUDYINGULTIPLEOCUMENTSNHISTORY 89

of seven documents(or study set) used in a previousstudy (Rouet, Britt,et al.,

1996).The

compositionof each

studyset was as follows:

1. Two historianessays: Historianessays (HEs)explicitly supportedone side

of thecontroversy e.g., they argued hat he interventionwas or was notjustified).The two essays gave oppositeviewpoints. They were both secondhandaccounts

and werewrittena long timeafter he events.

2. Twoparticipantaccounts:Participantccounts PAs)were writtenby peoplewho directlytook partin the events. These accounts were written at the time or

some time afterthe events.Like the HEs, the PAs were opinionated, hatis, they

explicitly supportedone side of thecontroversy.3. Twoofficialdocuments:These documentswerewrittenbefore or at the time

of the events andactuallyplayeda role in the controversy e.g., the 1846 United

States-ColombiaTreaty tating heirrightsandduties nPanama).The two official

documents ODs) were mentioned n the HEsand PAs in supportof their nterpre-tations.

4. One textbook-likeexcerpt:This passage, writtenby the researchers,was

basedon severalaccountsandgavea basicdescriptionof the events. The textbook-

like excerpt(TB) wasmostlyfactualanddidnot referexplicitlyto thecontroversy.

The study set was presentedon a computerscreen using a simple hypertext

system(fora more detailedpresentation f thesystem,see Britt,Rouet,& Perfetti,

1996). Based on studies of hypertextusability (Rouet, 1992), the system was

designedin order o be easily masteredby novice users.Thesystemfirstpresentedachronologyof the mainfactsanda list of the availabledocuments.Thedocuments

were listed in a fixed order:HEs, PAs, TB, and ODs. Each document was

representedby elementary ource nformation:Author'sstatusandname,the typeof document,andthedate(e.g., "Pres.T. Roosevelt's speech, 1904").Thepartici-

pantscould select and read the documentsone by one, in any order,and as manytimes as they needed. Each documentselectionresulted n the presentation f the

extended source (e.g., "Excerptfrom U.S. PresidentT. Roosevelt's speech to

Congress[January , 1904].The authorarguesthatthe U.S. military ntervention

in the Panamanian evolutionwasjustified.").The participants ould thendecide

either to readthe documentor to go back to thechronologyand document ist.

Procedure

The experimentwas runindividuallyandinvolved two sessions. In Session 1, we

evaluated heparticipants' nowledge ngeneralhistoryand hehistoryof Panama.

The generalhistorytest was madeof nine short-answerquestionsaboutgeneral

worldhistory (e.g., "Whatwas the WeimarRepublic?").The Panamahistorytest









was madeof 14short-answer uestions e.g., "Whocontrols hePanamaCanal?").In addition, he

participantswere asked to locate nine countrieson a blankworld

map.2Theparticipantshen reada backgroundext about hehistoryof Panamaand

answereda seriesof comprehensionandopinion questions.Session 2 was run from a few minutesto a few hours afterSession 1.First,the

participantswere trainedon the hypertext ystem using an unrelated et of docu-

ments.Theexperimentermadesurethat heparticipantswerecomfortablewith the

hypertext nterface.Participantswere thenasked to studythe two controversies.

The procedurewas identicalfor each controversyandincluded three main tasks:

studyperiod,essay writing,and documentevaluation.

Study period. Participantswere given 15 min to studythe list of facts and

the set of documentsusingthe hypertext ystem. They were asked to select each

documentat least once in order o read he extendedsource(however,theydid not

haveto read he actualdocument).Theparticipantswere told that heycould select

thedocuments n anyorderand as manytimes as theyliked.Theywere advised to

do so wisely becauseof the time constraint.The participantswere free to stop as

soon astheyfelt theyhadaninformedopinionabout hecontroversy,provided hat

theyhad selectedeachextendedsource at leastonce. Duringthe studyperiod,the

controversy tatementwas accessibleusinga "read hequestion"button ocatedoneverypage of thehypertext.

Essay writing. Afterthe studyphase,theparticipantswere given 10 minto

write aone-pagedraftessayexpressing heiropinionon thecontroversy.Thestudyset was no longer availableto the participants ecausewe wanted them to draw

argumentsrommemory.However,theparticipantsweregiven a reminder tatingthe controversyanda list of the extendedsources as well as the importantnames

mentionedin the study set. The participantswere also told that they were not

expectedto quotefromthedocuments.

Document evaluation. Theparticipantsweregiven a sheetlistingthe seven

extended sources. They were asked to rank the documentsaccordingto their

usefulness(i.e., "totheextenttheyhelpedyoubuildupan informedopinionduringthe study").The number7 was to be given to the most usefuldocument,6 to the

next most useful, and so on to 1 for the least useful document.In addition,the

participantswere askedtojustify brieflythe rankgiven to each document.Then,

theparticipantswereagain giventhe sevenextendedsourcesand askedto rank he

2The materialsare availableon request o the firstauthor.

3The timelimitsusedin Session2 were setaftera pilotstudy.Theywere used as baselines,andtheywereappliedwith some flexibilityso thatall the participantsouldcompletethetasks.








STUDYINGMULTIPLEOCUMENTSNHISTORY 91

documentsaccording o theirtrustworthinessi.e., "tothe extentyou trustwhatthe

authorsays").

Thisranking

was to beindependent

of the usefulnessranking.

The

participantswere askedagainto justify theirrankingof each document.Finally,the participantswere asked for remarksor comments about the procedureand

debriefed.

RESULTS

We firstbrieflysummarize he performance f the historynovices andspecialists

on the initial knowledgetests, which determined hat the two groupsdifferedintheirgeneralhistoryknowledgebutnotintheirknowledgeof thehistoryof Panama.

Then, we present the data corresponding o the three aspects of the students'

performancehatwereinvestigated: tudents'patternsof studytime anddocument

selection, students'evaluationsof the documentswith respectto usefulness and

trustworthiness, nd students'expressionsof and support or an opinion in their

essays.

Did Noviceand ExpertStudentsDiffernTheirInitial

Knowledge?

The historyspecialists outperformedhe novices on the general historytest (65%vs. 35%, respectively),t(17) = 4.19, p < .001. However, the specialistsand the

novices bothperformedpoorlyon the Panamahistorytest (33%and23%correct,

respectively) and did not significantlydiffer from each other, t(17) = 1.41, ns.

Finally,thespecialistsandthe novices differedonly marginally rom eachotheron

the maptest (63%vs. 82%,respectively), (17) = 1.95,p < .10. Thus,as expected,the specialistshad more generalknowledge in historythan the novices, but the

novices and the specialistshadequallylittlepreviousknowledgeof the historyofPanama.

How DidStudentsManagethe StudyPeriod?

Therewas a lot of variation n students'managementof the studyperiod,but no

consistentdifferenceacrossgroups. Studytimeranged rom4 to 20 min,'with an

averageof 11.3 min, and did not significantlydiffer acrossgroupsor problems.

Students' order of selection of documentsalso varied considerably. Only twostudents(one in each group)selectedthe documents n theirorderof presentation

40n five occasions,studentsrequestedandweregranted xtrastudytimepastthe 15-min ime limit.









on the computerscreen,andtheydid so for bothproblems.All the other students

selectedthe documents n differentorders.Therewere a few consistentpatterns

n

students' selection orders.Fifty-threepercentof the students(6 novices and 4

specialists)selected the TB in thefirst,second,or thirdpositionforbothproblems.An additional25%(4 novices and 1 specialist)did so either for Problem1 or for

Problem2. In Problem2, 68% of the studentsselected both ODs amongthe first

threedocuments.Also, moststudents 68%) ooked backat theproblemstatement

at least once duringthe studyof the documents.Seventy-twopercentof the look

backs occurredeitherat thebeginningor at the end of the study sequence.The datasuggestthatthemanagement f thestudyperiodwas little affectedby

discipline expertise.However,most

study sequencesfeaturedbasic

organizationprinciples e.g., selectingtheTB early),whichmayreflectgeneralstudyheuristics

bothin novices and in specialists.

HowDidStudentsEvaluateDocumentnformation?

Justifications or rankingswerecategorizedusingthe methoddesignedby Rouet,

Britt,et al. (1996). First,eachjustificationstatementwas segmentedinto one or

morejustificationunits.Then, each unit was assignedto one of four categories:Documentcontent(e.g., "Presents he agreementwith Colombiaand the reasons

forintervention"), ocument ource(e.g., "He s a Senatoropposedto Roosevelt"),

task (e.g., "Keydocumentof the controversy"),and otherjustifications(mostlyabsencethereof).Onethirdof thejustificationswere scored ndependentlyby two

judgeswith anagreementhigher han85%.Discrepancieswere notsystematic,and

they were resolvedthroughdiscussion.Eachjudge scored half of the remaining

protocols.

Evaluation of usefulness. Table 1 presents he median usefulnessrankof

each document n eachgroupand for eachproblem.Using Friedman'schi-square

test,we founda significantdifference n therankingsof the documentsbothin the

novice group,x2(6,N = 11) = 12.78,p < .05, andX2(6,N = 11) = 17.81,p < .01,andin thespecialistgroup,X2(6,N= 8) = 34.84,p < .001, andX2(6,N= 8) = 29.04,

p < .001, for Problems1 and2, respectively.Ingeneral, he two documents n a category HE,PA, andOD)receivedsimilar

rankings.The only exception concernedthe novice studentswho rankedHE1

consistentlymore usefulthanHE2,butonly forProblem1.Both groupsfoundODs veryuseful(medianrankingsweregreater han5 with

onlyoneexception).Thespecialists ended o find PAs moreusefulthan henovices

did.Conversely,the novices rankedHEs moreusefulthanthe specialistsdid.The

TB was foundratherusefulby bothgroups n Problem1 butnot in Problem2.








STUDYING MULTIPLEDOCUMENTSIN HISTORY 93

TABLEMedian ankingsfUsefulness s a FunctionfDocument, roup,ndProblem

Document

HEJ HE2 PAl PA2 TB OD1 OD2

Problem1

Novices 5.0 4.0 2.0 3.0 6.0 3.0 6.0

Experts 3.0 2.5 3.5 5.0 4.5 5.5 6.5

Problem2

Novices 4.0 4.0 3.0 4.0 3.0 6.0 7.0

Experts 2.0 2.5 4.0 4.5 2.0 6.0 6.5

Note. Rankingsare from 1 (minimum) o 7 (maximum).HE = historianessay; PA = participant

account;TB = textbook;OD = official document.

TABLE

Percentagef EachTypeofUsefulness ustificationCollapsed crossProblems)PerGroupndTypeof Document

JustificationType

Content Source Task Other

HistorianessaysNovices (n= 54) 74 13 11 2

Experts n= 43) 49 14 23 14

Participant ccounts

Novices (n = 58) 60 28 10 2

Experts n = 45) 24 53 18 4

Textbook

Novices (n = 26) 77 4 15 4

Experts(n= 22) 41 27 27 5

Official documents

Novices (n = 56) 36 14 41 9

Experts n = 48) 29 27 44 0

Total

Novices (n = 194) 59 16 20 4

Experts(n = 158) 35 31 28 6

Whatmakes a document useful? Table2 presentshefrequencyf each

categoryof justificationfor each type of document.A frequency analysison the

pool of justifications,collapsedacrossdocumentsandproblems,showedthatthe

distributionof justificationsdifferedsignificantlyacrossgroups,X2(3,N= 352) =21.78, p < .01. The novice studentsmostly justified their rankingsusing the

documentcontentas a criterion 60%on average).Incontrast, he specialistsused

a roughly equal proportion f eachcategoryof justifications(i.e., content,source,andtask).









The novice studentsbased most of theirjustificationson content for the TB,

HEs, andPAs. Thejustificationsusedby the specialistsvariedacrossdocuments.

They used both content and task for the TB andHEs, andmainly sourcefor the

PAs. The specialists'use of sourceas ajustification or PAs can be related o the

findingthatthespecialists oundPAs ratheruseful(see Table1).Thefact thatPAs

were writtenby majorcharacterse.g., U.S. PresidentRoosevelt)mayhavebeena

salient feature o the specialistsbutwas less so to the novices.

Finally, both groupsjustified their (usually high) rankingsof the ODs by

mentioningthe role of the document n the controversy e.g., "Treaty s the basis

everyonerefers o").Thus,most novices andspecialists dentified he ODs' specialstatus

(i.e.,documentas evidencevs. documentas accountor

interpretation).In Problem2, 42% of thejustificationsmentioned hat the TB was either too

succinctor redundantwith the chronology,comparedwith only 10% n Problem

1. Thestudents'awarenessof theTB's redundancy xplainswhy it was foundless

useful in Problem2.

Evaluation f trustworthiness. Table3 shows hemedianrustworthiness

rankingsfor Problem 1 and Problem 2. A series of Friedman'schi-square ests

showedthat trustworthinessankingsvaried across documents:novices, Problem1,X2(6,N= 11) = 41.30, p < .001; novices, Problem2, X2(6,N= 11)= 41.61, p <

.001; specialists,Problem1, X2(6,N = 8) = 21.38, p < .01; specialists,Problem2,

X2(6,N= 8) = 33.96, p < .001.Both groupsfound ODs most trustworthy.The TB was rankedas next most

trustworthy, ndthePAs andHEs wereranked easttrustworthy.Bothnovices and

specialists consistently found OD2 (1846 Treaty)more trustworthy han OD1

(militarycorrespondence).naddition, henovicesfoundPAl (SenatorCarmack'

speech) moretrustworthyhan PA2 (PresidentRoosevelt's speech).The novices

TABLEMedianRankings f Trustworthinesss a Function f Document,Group,and Problem

Document

HE1 HE2 PAl PA2 TB ODI OD2

Problem1

Novices 4.0 3.0 2.0 1.0 6.0 5.0 7.0

Experts 2.0 3.0 4.0 4.0 4.0 5.0 7.0

Problem2Novices 1.0 3.0 2.0 4.0 5.0 6.0 7.0

Experts 2.0 2.5 2.5 3.5 5.0 6.0 7.0

Note. Rankingsare from 1 (minimum)o 7 (maximum).HE = historianessay; PA = participant

account;TB = textbook;OD = official document.









TABLE

Percentagef EachTypeofTrustworthinessustificationCollapsed crossProblems)PerGroupndTypeof Document

JustificationType

Content Source Task Other

Historianessays

Novices (n = 52) 69 21 4 6

Experts n= 38) 61 29 11 0

Participant ccounts

Novices (n = 54) 37 56 2 6

Experts(n= 44) 32 59 7 2

Textbook

Novices (n = 24) 71 21 4 4

Experts n= 19) 53 32 5 11

Officialdocuments

Novices (n = 47) 21 57 6 15

Experts n = 43) 16 63 12 9

Total

Novices (n = 177) 47 41 4 8

Experts n= 144) 37 49 9 5

alsofoundthese two documentsvery untrustworthyomparedwiththespecialists.Thatthe documentauthorswere involved in the events may have led the novice

students o distrust he documentcontent.This may also explainwhy the novices

did not findPAs useful (see Table 1).

Overall,the students n bothgroupswere sensitive to the opinionexpressed n

thedocuments.Neutraldocuments, hat s, documents hatdid not takeany explicitstanceon the controversy TB, ODs), were foundmoretrustworthyhanopinion-

ateddocuments i.e., PAs, HEs).

Whatmakes a documenttrustworthy? Table4 presentshefrequencyfeach categoryof justificationfor eachtype of document.The patternof justifica-tionswas verydifferent romthe one foundwithusefulness.First, hereweremanymore justificationsbased on source information 44.5% vs. 23%). In addition,novices andspecialistsdid not differsignificantly n theirtypes of justifications,

X2(3,N= 311) = 7.07, ns.

As shownin Table4, source ustificationswere mostfrequent or PAs and ODs.Source was also the second most important ategoryfor the TB and HEs.

The datapresented n Table4 suggestthat source characteristicse.g., who the

authoris) were salient when evaluating trustworthiness,whereas task charac-

teristics-which the specialistsdid use to evaluate usefulness-were less salient.









From hesedata, he evaluationof document rustworthinesseems little influenced

bystudents' discipline expertise or by their representationof the documents'

content.Both novices andspecialistsaresensitiveto sourceparameters,or exam-

ple, whether he authorhadaninterestat stakeor whether he document s official.

Therewere somediscrepanciesn the evaluationof documentswithinacategory.Studentsjustified their preferencefor OD2 (1846 Treaty)over OD1 (military

correspondence)n Problem1 by stating hatOD1 could be biased(as one student

put it, "I do not trustthe so-called official sources of the military").Similarly,studentsjustified their preferencefor Senator Carmack(PA1) over President

Roosevelt (PA2) by statingthe position of the latter(e.g., "One knows that a

President s notgoing

todisparage

a decision takenby

himself and hisleaders").

Conversely,a few studentsquestionedSenatorCarmack' background e.g., "takes

a con stancebut is he a political opponent o Roosevelt?"). n thiscase, the lack of

detailed informationaboutthe authorcreated some uncertaintyabout the docu-

ment' trustworthiness.

Summaryof evaluationdata. ODswere oundmostuseful nd rustworthyby both novices and specialists.The specialists,however, found the PAs more

useful than the novices did, which they justified using source characteristics.Novices evaluateddocumentusefulnessusingmostlycontent nformation,whereas

the specialistsused a varietyof criteria.Both groupsfocused on sources when

evaluating trustworthiness.This finding suggests that students may focus on

differentdocument eaturesas a functionof taskrequirements.

How DidNovices and SpecialistsComposeTheirEssays?

Students'essayswerefirstanalyzedusingaframework esigned n an earlier tudy(Rouet,Marron,Mason,& Perfetti,1993).Thisframework imsatidentifying he

claims andtypesof arguments sedby students n theiressays.Eachessaywasfirst

decomposed ntoa set of argumentationnits,whichgenerally ncludeda smallset

of semanticpropositions.Then,each unit was assignedto an argument ategory:

claim,evaluation,supportingact,orpsychologicalevent.Eachcategory ncluded

several subcategories;for example, psychological events included characters'

motivations,goals,orfeelings.Inaddition,we identified"rhetorical djuncts"uch

as qualifiersor referencesto informationsources. One third of the essays were

scoredby two independentudges with a reliability(unitdefinitionandcategori-zation)higherthan 80%.The discrepancieswereresolvedthroughdiscussionand

iterativerefinementof theframework.Then,therestof theessays were scoredby

one of the judges (for more details on the methodused to parsethe essays into

argumentation nits,see Rouet,Favart,Gaonac'h,& Lacroix, 1996).









TABLENumberndPercentagefEssaysWithRespectoTypesof Claims

(Collapsed crossProblems)

Restricted

No Claim Claim Full Claim Total

n % n % n % n %

Novices 4 18 2 9 16 73 22 100

Experts 7 44 4 25 5 31 16 100

How did students express an opinion? Opinionsin each essay were as-sessed througha subcategoryof evaluationstatements alled claims. Withrespectto thepresenceandtypeof claimstatements, heessays collected in this studyfell

into threebroadcategories:

1. Full claim: Some of the students ook anexplicitstance on thecontroversy,forexample,"TheU.S. interventionwas notjustifiedbecausetherewere no signsof a revolution n Panamabeforethemarines anded" N5, P1).5

2. Restrictedclaim: Some of the students ook a moderatepositionor did not

personallyassumethe expressed position, for example, "If one considers that arevolutionwould havethreatenedransitn theIsthmus, henonemayhypocriticallyconcludethatthe interventionwasjustified"(E4, Pl).

3. No claim: Some of the studentsdid nottake a stance on thecontroversybut

ratherpresenteda generalevaluationof the controversy, or example, "TheU.S.

interventionn Panamagave birth o two opposedinterpretations"E5, P1).

The frequencyof full claims,restricted laims,andno claimessays in the two

groupsarepresented n Table 5.

A frequencyanalysison the pool of 38 essays showed that the distributionof

claims was significantlydifferent n thetwo groups,X2(2,N = 38) = 6.44, p < .02.

Mostof thenovices' essaysfell intothefull claimcategory,whereas hespecialists'

essays containedmore restricted laims or no claim at all. This resultsuggeststhat

the two groupshad differentrepresentations f the taskof expressingan opinionon the controversy.Most novice studentsexpressedan opinionabout which side

was right,whereas the specialists expressedan opinionaboutthe structure f the

problemspace (i.e., the interpretationshatcouldbe foundin the documents).Moreevidencefor differentproblemrepresentationsomes fromtheanalysisof

initial statements n each essay. Most novices' essays beganwith a claim (59%),

5In the identificationof protocols,N standsfor novice students,E standsforexpertstudents,andP

stands forproblem.








98 ROUET,FAVART,BRITT,PERFETTI

whereas 69% of the specialists' essays began with eithera contextual statement

(e.g., "OnNovember2, 1903,arevolutionbrokeoutinPanama") r acommentaryon theproblem tatement e.g.,'"The ontroversybasicallyrests on anambiguityof

the 1846 Treaty").Thus,thespecialists ended o introduceordiscuss theproblem

spacebeforegoing into furtherdetails.The distribution f initial statements claimvs. otherstatement)was significantlydifferentacrossgroups,X2(1,N= 38) = 3.89,

p< 05.

Didnovicesandspecialists sesimilareasoningeuristics?We lookedfor

reasoningheuristicssimilar o

Wineburg's 1991) contextualization, ourcing,andcorroboration euristics n students'essays.

Contextualizationeuristic. The contextualizationeuristicconsistsof

confrontingdocument nformationwith one's generalknowledge.We identified

contextualstatementsas statementspresenting elevant nformationhatcouldnot

be tracedbackto the documents.

The specialistsincludedmore contextualstatements hanthe novice students:

1.75 versus 0.68 per essay, respectively,F(1, 17) = 5.94, p < .05. Contextualstatementswere also morefrequent n Problem2 than in Problem 1: 1.75 versus

0.68 per essay on average,respectively,F(1, 17) = 4.62, p < .05. There was no

significant nteractionbetweengroupandproblem.We identifiedthreecategoriesof contextualstatements:

1. Problemcontextstatements eferred o the specificcontextof the Panama-

nianrevolution, or example,"[TheU.S.] hadneverbeen ableto reach an

agreementwith Colombia n order o buildthe canal" N2, Pl).

2. Historical context statementsreferred o generalhistoricalknowledge orprinciples, or example,"[Atthattime] the U.S. aimed at expanding heir

zone of economicandmilitary nfluence n SouthAmerica" E10, P1).3. General contextstatements eferred o principlesnot specific to historical

reasoning, or example,"Negotiationshaveto go bothways"(N10, P2).

Therewasalsoa difference nthetypeof contextual tatementsssuedbynovices

andspecialists,X2(2,N = 44) = 7.83, p < .05. Historicalcontextrepresented 5%

(16 of 29) of thespecialists'contextual tatements,as opposedto 20%(3 of 15) of

the novices' statements.The novicesrecalledeither ocal backgroundnformation(53%)orgeneralprinciples 27%).

A furtherqualitativeanalysis suggestedthat the specialists' contextualstate-

ments tended to be more elaborate.For instance, in Problem 1, a few of the

participants 1 novice and3 specialists)recalledthe U.S. role in the earlyphases









of the revolt(i.e., alocalcontextstatement).Thisargumentwaspresentednarather

straightforwardmanner

bythe novice: "Fromthe

beginning,the Panamanian

revolutionhad been set up by the U.S." (N4). In contrast,a specialistreferred o

the U.S. role in the following terms: "The U.S. presence was essential to the

outcome of the Panamanianevolution, incethePanamanianselt encouragedand

upheldbythispresencewhich,in somesense, ustifiedtheircause,andgavetotheir

movementbackingandjustification" E2). Thus,the specialistexplainedthe U.S.

background nfluence through a series of inferences about the Panamanians'

motivationsandby emphasizing hepotentialmeaningof the U.S. presenceto the

Panamanians. imilarly,another pecialistprovided actualevidencein supportof

herevaluationof U.S. involvement nthe revolution:"Thearrival f aU.S.warship,even with a small troop on the eve of the revolution and the recognition of

[Panama's] ndependence hreedays later nvolves the U.S. in thisevent"(E4).In summary,contextualstatementswerenotonly morefrequent n the special-

ists' essays butalso more focusedandmore elaborate.

Sourcing heuristic. Thesourcingheuristic onsists of payingclose attention

to the sourceof information e.g., theauthorof adocument).This source informa-

tion is vitalin anessay taskif students ntend o cite thesourceof their nformation(i.e., thespecificdocument hatcontained heargument rprovidedsupport orit).

Consequently,we considered hepresenceof references o documentsas evidence

of asourcingproductionheuristic.Onereferencewas scoredeach timeaparticipantmentioneda documentor the nameof an author.

We did notfindanydifferenceacrossgroups n thenumber rtypeof references.

Ninety-two percentof theessays containedat least one reference,and theaveragenumberof referencesperessaywas2.05. Bothgroupsreferredmainly o ODs(62%of thereferences).Thus,based on thestudyof references,we didnotfindevidence

for sourcingdifferencesbetween novices andspecialists.

Corroborationeuristic. Thecorroborationeuristic onsistsof checkingseveralsourcesof information gainsteachother.Theessaysincludedseveral ypesof corroborations s shownin Table6.

Some studentsmerely expressedthe existence of two opposingviews or inter-

pretations,whereasothersexplicitly noted the type of connection betweentwo or

more documents. Some mentioned the positive connectionsbetween consistent

sources,thatis, they tended to groupthe documentsexpressingsimilar views onthe controversy.One studentmentioneda negativeconnection; hatis, he pointedout thediscrepancyor conflictbetweentwo sources(thesetwo cases werefrom the

samestudent,one in eachproblem).Finally,somestudentsmadegeneral referencesto a groupof sourcesbackinganassertion.









TABLE

Types,Examples,ndFrequencyfCorroborationsorNovices ndExperts

Novices Experts

TypeandDefinition Example n % n %

Argumentmodel: Indicates "TheU.S. interventionn Panama

orcomparesdifferent gavebirth o two radically

interpretations opposedtheses"(E5, PI) 5 31 8 44

Positive connection:Connects "AsBunau-Varilla nd P. Moore

several sourcespositively explain" N4, P2) 5 31 6 33

Negativeconnection: "Accordingo P. Moore ...

Connectsseveral sources assertionsdeniedby R. Willman"negatively (E9, P2) - - 2 11

Generalreference: ndicates "Severaldocumentsmention he

several consistentsources Colombianpopularwill" (N8, PI) 6 38 2 11

Total 16 100 18 100

Note. E = expert;P = problem;N = novice.

Therewasless thanonecorroborationeressay,andcorroborations ereequally

frequentin both groups. Some students used a different means to corroborate

information.They pointed out that some evidence was missing or that moreevidence of a certain ypewouldhavehelped.InProblem1,forinstance,aspecialistnoted: "Whether r not the U.S. really respected he Bidlack-MallarinoTreaty s

a difficultquestion,since we do not havethis treaty n its entirety" E7).Sometimestheneedforadditional nformationwas stressedeven morespecifi-

cally, as in this specialist's essay:

The problem s to know to whatextent the Panamanianswere readyat the

time of the revolution to pay thatprice for theirindependence.We would

need to know to whatextentthe Panamanianswere againstthe Colombiangovernment,whether heywereready o sacrificeeverything nexchange or

a free country; hus,to know thePanamanians' eactions o the Treaty,and

whether n November1903they thought heyhad made a "gooddeal."(E5,

P2)

Inthis case, the specialistidentifieda potentialsourcethat was not representedn

the studyset. Although he two examples ust presented ame fromspecialists,we

also foundsimilarstatementsn two novices' essays.Thus,ourdatado notsupport

the conclusionthatcorroboratingources or identifyingmissing information s askill specific to experthistorystudents.

Complexreasoningheuristics. Ina fewessays,participantsombinedev-

eral of the heuristicsdescribedearlier.For instance,a specialist discussed two









authors'arguments boutthecontroversy:"TheU.S. Presidentas well as historian

Wilson seem to consider[past

events] as aunique

criterion.It is thepast (thenumerous nterventionsn Panama, he relationswithColombia)that ustifies the

[1903] intervention"E5, P1).

Here,contextualevents(e.g., previous nterventions)wereintegratedas partof

the argumentsproposed n two sources(i.e., Roosevelt,Wilson).Thetwo sources

(Roosevelt,aparticipant, ndWilson,a historian) orroborated ach otherto form

"oneside" of thecontroversy.Thissuggeststhatthestudenthad builtanargument

model, that is, a global representation f the claims, evidence, and information

sources (Britt,Rouet, Georgi, & Perfetti,1994). Finally andmost important, he

"side"was in turn

ntegratednto the student'sown

reasoningscheme,which led

her to conclude that: "This argumentpoorly explains the sudden necessity of

[another]ntervention."

In this case, the student used qualityof evidence as a means to evaluatethe

proposedinterpretations.Moreover,the student'ssingle threadof argumentationincluded sourcing,corroboration, nd contextualization. n the essays collected

here,such an integrateduse of reasoningheuristicswas not very frequent,and we

did not attempt o quantify hedifferences.

Summaryof essay data. The novicestudentsended o express lear-cutopinionsatthebeginningof theiressays,whereas hespecialistshedged heirclaims

or did not offer an opinion on the controversy.The specialists included more

contextualstatementsn theiressays. Theyreferredmostlyto thegeneralhistorical

context of the events, whereas the novices used various types of contextual

information.Some specialistsalso used more sophisticatedreasoningstrategies,

combiningthesourcing,corroboration, ndcontextualization euristics n a single

threadof argumentation.

DISCUSSION

We examinedthe effects of students'expertisein historyon theirevaluationand

use of documentinformation.The studentswere all graduatestudents,and theydifferedonly in theirdisciplineof specialization i.e., psychologyvs. history).We

asked the students to study a set of documents, to evaluate each document's

usefulness and trustworthiness, nd to write a draftopinion essay based on the

documents.Threequestions were addressed: a) Do discipline specialistsdiffer

from novices in theirstudy strategies?(b) Do discipline specialistsuse specificknowledgewhenevaluatingdocument nformation? nd(c) Do discipline special-ists use documents n a specificway whenwritinganessay?

Theanalysisof onlinestudyparameters id notshowany significantdifferences.

Most students constructedidiosyncratic study sequences, and they selectively









reviewed some of the documents.Novices andspecialists ended o examineeither

the TB or ODs first. These documentswere generally found most useful and

trustworthy. n addition,most studentscheckedthe problemstatementat specifictimes, eitherat the beginningor at the end of the study sequence,a monitoring

strategyobserved n othertextcomprehension tudies(Wright,1993).Theresults

suggest that,when asked to examinea set of documents,most graduate tudents

will use elaboratestudy strategies,whatever heir areaof specialization.General

literacyor academictrainingmay account for graduate tudents' elaboratestudy

strategies Anderson& Armbruster, 984).However,the analysisof online study

sequencescanonlycapture omeaspectsof students' tudystrategies.For nstance,

corroboratingdocument information

maybe done from

memorywithout

goingback andforthbetweentwodocuments.Moreover,different tudystrategiesmightstill be observed underdifferentstudy conditions(e.g., asking students to learn

directlyfrom a set of documentswithoutany backgroundnformation).There werebothconsistentsimilaritiesanddifferences n theway novices and

specialistsevaluated he documents.Ingeneral,ODs were found most useful and

trustworthy y bothgroups.The novicestudents onsistentlyranked he HEs more

useful thanthe PAs, whereasthe reversewas truefor the specialists.Finally,the

TB was found useful in Problem1 but not in Problem2, due in partto students'

findingthatthe TB was redundantwith thechronology.To the novice students,whethera documentwas useful was mostlya matterof

content. In contrast,the specialists applied multiple criteria when evaluatingdocumentusefulness.Furthermore,hespecialists'criteriavariedacrossdocument

types. Theycitedmostlycontentfor secondhandaccounts, hat s, the TB and HEs

(with manyreferencesto task),mostly source for PAs, andmostly task for ODs

(withmanyreferences o content).These resultssuggest thatnovices andspecialistsstudied the documents with

differentpurposes.The novices' main concern was to build up a mentalrepre-

sentationof whathappened n Panama.To this end, the TB andHEs were usefulsources,and content was the most salient factor of usefulness. Incontrast,history

graduatestudents focused on interpretations nd evidence. They grantedmore

importance o primary ources(e.g., PAs, ODs). They disregarded he HEs (one

specialiststated hatshe wishedshe knew the historians'credentials).Thespecial-istsalsoused a broader angeof criteriawhenevaluatingusefulness.Itis likelythat

extensive academic training allows students to build up elaboratemodels of

differenttypes of documentsand theirpotentialuse for learning,reasoning,and

argumentation. his interpretations consistentwith Dillon's (1991) proposal hat

disciplinespecialistspossesspermanentchemata hatrepresenthetypical eaturesof theirusual sourcesof information.

Novices and specialists did not differ as much when evaluatingdocument

trustworthiness.Most students ocused on sourceinformation, speciallyfor PAs

and ODs. From the data, trustworthinessmay be evaluated based on students'









generalknowledgeof information ndcommunication ules.It does not takemuch

expertisen

historyo

recognizethata

personnvolvedin acontroversial ssue

maynotprovidea veryreliableaccountof the events. To a largeextent,trustworthiness

is also relatedto the intrinsiccharacteristics f a document(e.g., content,source)rather han o the role the documentplaysin aparticularaskorproblem.Whatmayevolve withdisciplineexpertise,however, s how theevaluationof trustworthiness

influencesthe actualuse of a document.Novice students oundPAs neitheruseful

nor trustworthy.They rejectedthese documentspossibly becausethey could not

handle the bias in them.Expertstudentsalso distrustedPAs but still found them

useful.Theirexperience n usingvarious orms of evidencehelpedthemdealwith

the bias in PAs. They could formulatehypotheseson why and how a president,asenator,or a lobbyist would provide slantedaccounts,and they interpreted he

documentaccordingly.When writingan opinion essay on the controversies,most specialistsresisted

takinganexplicit stance.In a way, the task of expressinganopinionseemedmore

difficult to them than it was to the novices. The data suggest that novices and

specialists nterpreted ifferently hetaskof expressinganopinionon acontroversy.Becausetheyhad been confrontedwith similarproblemsmanytimes,thegraduate

historystudentsknewthey were asked to studyactualcontroversies hatcould not

be answered n a simple,straightforward anner.Tothem,"expressyouropinion"meant describe the problemspace, the claims, andarguments hatmay be stated.

In contrast, he novices probablyassumedthatsupportingone particular pinionwas relevant because this was the assigned purposeof their writing. To them,

"expressyour opinion"meantdecidewhichside is correctandexplain why. In that

sense the novices represented he historicalcontroversiesas they wouldrepresent

any opinion problem (except perhapscontroversiesin their own discipline of

specialization).We identifiedseveralreasoningheuristicsby analyzing heparticipants' ssays.

Thesestrategieswere similar o the heuristics dentifiedbyWineburg 1991) basedon think-aloudprotocols.The occurrenceof similarreasoningprocessesat studytime and during essay composition is an importantfinding. It indicates that

informationsources (sourcingheuristic),relations between sources (corrobora-

tion),andrelationsbetween sourcesand students'generalknowledge(contextuali-

zation) are partof the memoryrepresentationhat students build from a set of

documents.This supportsthe notion of an argumentmodel as the structuringscheme of high-level representation f multipleinformation ources (Brittet al.,

1994;Rouet, Britt,et al., 1996).

Thefindingthatcontextualizationswere morefrequentn thespecialists'essaysconfirmspreviousobservations Wineburg,1991). In addition,we founda quali-tativedifference between novices andspecialists'contextualstatements.Novices

eitherreferred o the localproblemcontextor togeneral,nonhistorical otions.The

specialists focused on historicalprinciplesor events relevantto the period.Voss









andPost(1988) havepointedout thatsolvingill-structuredroblems equiresa rich

base of declarativeknowledge.

Inourstudy,

the novices' limitedhistoricalknowl-

edge led them to mentionany information hey thoughtwas helpful,from back-

groundproblem nformation ogeneralworldknowledge.Incontrast,hespecialistswere able to draw from a richerknowledgebase and, hence, to select contextual

elementsat a moreappropriateevel of generality.Thesourcingheuristicwas equallyfrequent n novices' andspecialists'essays.

Moreover, hepattern f referenceswas similar o the one observedbyRouet,Britt,et al. (1996) with undergraduatetudentsandhistorygraduates.As in ourstudy,

Rouet, Britt,et al. foundthatstudentsreferredmostly to official documentsand

firsthandaccounts.Thus,neitheracademic evel nordisciplineexpertiseseemed to

influencethe use of explicitreferences n a draftessaywriting ask.However,there

may be more subtle differencesin the use of sources that our analysis failed to

capture.Whatdo these results ell us about earning rommultipledocuments?Knowing

how to handle document information s an advanced learningskill that manystudents ack (Guthrie,1988).To a largeextent,highereducationpromotesdocu-

ment literacy, and university graduatesmay be considereddocument experts

regardlessof theirdisciplineof specialization.However,disciplinessuch as psy-

chology andhistorydiffer n theiruse of document nformation elative o thetypesof problemswith which they deal. In thatsense, ourresults,along with those of

previousstudies(Rouet,Britt,et al., 1996;Wineburg,1994), indicatethathistorystudentspossessmorethanageneral ense of documentgenres.Theyarealsoexpertatusingdocumentsasevidence,that s, integrating ocument nformationntotheir

mental representation f a series of events. The use of documentsas evidence

requireshistorystudents o recognizethepropertiesof a particular ocument i.e.,

whattypeof informationhedocumentconveys) andhow the informationmaybe

usedwhenreasoningabout he situationorproblem.Recognizingtheproperties f

documentsandusingdocument nformationn specificways maybe consideredaparticular orm of discourse expertise, one that may be labeled documentary

expertise. Although documentaryexpertisemay take differentforms from one

disciplineto another, t seemsratherndependent f thespecificcontentarea.From

ourdata,disciplinespecialistscanapplytheirdocumentary xpertiseeven though

theyhaveverylittle initialknowledgeof theparticularituationorproblem. nsuch

acase, to borrowanexpression romVoss et al. (1983), theyarenonexpert xperts.Ourstudywas anattempto demonstratehatexpertise n thedisciplineof history

involves boththe knowledgeof specific formsof discourseand theknowledgeof

how discoursemay be used as evidencewhen solving problems.We believe thatdocumentary xpertiseplays an important ole in manyareasof problemsolving,even thoughsucha roleis mostobviousin the social sciences.We alsosuggestthat

documentaryexpertiseis useful even at intermediateevels of instruction.Text-

booksoftenincludeselecteddocuments,and he studentmustknow how toevaluate








STUDYINGMULTIPLEDOCUMENTS N HISTORY 105

themand orelate hem o theexplanation rinterpretationroposednthe textbook.

Developinginstructional

echniqueshatcan

promotehe skills of

evaluatingand

using document nformationn studentsat the secondaryor undergraduateevels

constitutesan intriguingchallengefor futurestudies.

ACKNOWLEDGMENTS

The authors hankNatashaLacroixand DanielGaonac'h or theirparticipationn

the workpresentedhere,as well as Andr6BisseretandMaraGeorgifortheiruseful

comments on an earlierdraft.

REFERENCES

Afflerbach,P. (1989). The influence of priorknowledgeon expertreaders'mainidea comprehension.

ReadingResearchQuarterly,25, 31-46.

Anderson,T. H., & Armbruster, . B. (1984). Studying.In P. D. Pearson Ed.),Handbookof readingresearch(pp.657-680). New York:Longman.

Britt,M. A., Rouet, J.-F.,Georgi,M. C., & Perfetti,C. A. (1994). Learning romhistorytexts:From

causalanalysisto argumentmodels.In G. Leinhardt, . Beck, & C. Stainton Eds.), Teachingand

learningin history(pp.47-84). Hillsdale,NJ:LawrenceErlbaumAssociates,Inc.

Britt,M. A., Rouet, J.-F.,& Perfetti,C. A. (1996).Usinghypertext o studyand reasonabouthistorical

evidence. In J.-F.Rouet,J. J.Levonen,A. P. Dillon, & R. J. Spiro(Eds.),Hypertextandcognition

(pp. 43-72). Mahwah,NJ:LawrenceErlbaumAssociates,Inc.

Carretero,M., & Limon,M. (1995, April).Uses of evidenceby historyexperts.Paperpresentedat the

annualmeetingof the AmericanEducationalResearchAssociation,San Francisco.

Dee Lucas, D., & Larkin,J. H. (1986). Novices' strategiesfor processingscientific texts. Discourse

Processes, 9, 329-354.

Dee Lucas, D., &Larkin,J. H. (1988). Novice rules forassessing mportancen scientifictexts. Journal

of MemoryandLanguage,27, 288-308.

Dillon,A.

(199 1).Readers'modelsof text structure: hecase of academicarticles. nternational ournal

of Man-MachineStudies, 35, 913-925.

Guthrie,J. T. (1988). Locating nformationn documents:Examination f a cognitivemodel.ReadingResearchQuarterly,23, 178-199.

Means,M. L., & Voss, J. F. (1985). Starwars:A developmental tudyof expertand novice knowledgestructures. ournalof MemoryandLanguage,24, 746-757.

Perfetti,C. A., Britt,M. A., & Georgi,M. C. (1995). Text-basedearningand reasoning:Studies in

history.Hillsdale,NJ: LawrenceErlbaumAssociates,Inc.

Recht,D. R., & Leslie,L. (1988). Effect of priorknowledgeongoodandpoorreaders'memoryof text.

Journalof EducationalPsychology,80, 16-20.

Rouet, J.-F. (1992). Cognitive processingof hyperdocuments:When does nonlinearityhelp? In D.

Lucarella,J. Nanard,M. Nanard,& P. Paolini(Eds.),Proceedings of theFourthACMConferenceon Hypertext pp. 131-140). New York:ACM Press.

Rouet, J.-F., Britt,M. A., Mason,R. A., & Perfetti,C. A. (1996). Using multiplesourcesof evidence

to studyhistoricalcontroversies.Journalof EducationalPsychology,88, 476-493.

Rouet, J.-F., Favart, M., Gaonac'h, D., & Lacroix, N. (1996). Writingfrom multiple documents:

Argumentation trategies n novice andexpert historystudents.In G. Rijlarsdaam& H. van den









Bergh(Eds.),Theories,modelsandmethodologiesnwriting pp.44-60). Amsterdam:Amsterdam

UniversityPress.

Rouet, J.-F., Marron,M., Mason,R. A., & Perfetti,C. A. (1993). Claimsandarguments:Aframeworkfor analyzinghistoricalarguments.Pittsburgh,PA: Universityof Pittsburgh,LearningResearch

andDevelopmentCenter.

Spilich,G. H., Vesonder,G. T., Chiesi,H. L., &Voss, J. F. (1979). Textprocessingof domain-related

information orindividualswithhighandlow domainknowledge.Journalof VerbalLearningand

VerbalBehavior, 18, 275-290.

Voss, J.F., Greene,T.R., Post,T.A., &Penner,B.C.(1983).Problem olvingskillinthe social sciences.

In G. H. Bower (Ed.), Advancesin research theory: Vol. 17. Thepsychology of learning and

motivation pp. 165-213). New York:Academic.

Voss, J.F., & Post,T. A. (1988). On thesolvingof ill-structured roblems. nM. T. H. Chi,R. Glaser,

& M. J. Farr(Eds.), The natureof expertise (pp. 261-285). Hillsdale, NJ: Lawrence Erlbaum

Associates,Inc.

Voss, J.F., Vesonder,G. T., & Spilich,G.J. (1980).Textgenerationandrecallby highknowledgeand

low knowledgeindividuals.Journalof VerbalLearningand VerbalBehavior, 19, 661-667.

Wineburg,S. S. (1991). Historicalproblem solving: A studyof the cognitive processes used in the

evaluationof documentary ndpictorialevidence. Journalof EducationalPsychology,83, 73-87.

Wineburg,S. S. (1994). Thecognitive representationf historical exts. InG. Leinhardt, . Beck, & C.

Stainton(Eds.) Teachingand learningin history(pp. 85-135). Hillsdale,NJ: LawrenceErlbaum

Associates,Inc.

Wright,P. (1993). To jump or not to jump: Strategyselection while readingelectronictexts. In C.

McKnight,A.Dillon,& J. RichardsonEds.),Hypertext:Apsychologicalperspective pp.137-152).

Chichester,England:Ellis Horwood.