Usability Evaluation The Supreme Court of Texas Sheng-Cheng Huang and Anuj A. Nanavati School of Information University of Texas at Austin November 22,

Usability EvaluationUsability EvaluationThe Supreme Court of The Supreme Court of

TexasTexas

Sheng-Cheng HuangSheng-Cheng Huang and and Anuj A. NanavatiAnuj A. Nanavati

School of InformationSchool of InformationUniversity of Texas at AustinUniversity of Texas at Austin

November 22, 2004November 22, 2004

OverviewOverview

► IntroductionIntroduction► Potential users and information needPotential users and information need► Test objectives and environmentTest objectives and environment► Test participantsTest participants► Test procedureTest procedure►Data analysisData analysis► RecommendationsRecommendations

Problem areasProblem areas Additional functionalitiesAdditional functionalities

The Supreme Court of TexasThe Supreme Court of Texas

IntroductionIntroduction

► Texas Judiciary onlineTexas Judiciary online Supreme court of TexasSupreme court of Texas Court of criminal appealsCourt of criminal appeals Court of appealsCourt of appeals Trial courtsTrial courts

► Use of the Web siteUse of the Web site Case related information – access + search + Case related information – access + search +

monitormonitor► Briefs, opinions, orders, deadlinesBriefs, opinions, orders, deadlines► Party, attorney, firm, trial courts, judgeParty, attorney, firm, trial courts, judge

Procedural and administrative court rulesProcedural and administrative court rules Contact informationContact information

► Attorney, party, judgeAttorney, party, judge

Potential user groups and Potential user groups and Information needInformation need

► User groupsUser groups AttorneyAttorney

► Trial Trial ► AppellateAppellate

JournalistJournalist► Regular monitorRegular monitor► Occasional monitorOccasional monitor► Watch dogsWatch dogs

Law scholarLaw scholar► Law studentLaw student► Law professorLaw professor► Court advisory committee memberCourt advisory committee member

Novice userNovice user► Different Information needDifferent Information need

Search targetsSearch targets Known informationKnown information ConstraintsConstraints

Test objectives and Test objectives and EnvironmentEnvironment

► ObjectivesObjectives Identify distinction between different information needsIdentify distinction between different information needs Obtain understanding of information categorization and Obtain understanding of information categorization and

organizationorganization Determine effective information deliveryDetermine effective information delivery Identify most frequently used featuresIdentify most frequently used features Identify overall satisfaction levelIdentify overall satisfaction level

► EnvironmentEnvironment User’s workplaceUser’s workplace Different computing environmentDifferent computing environment

► 4 users – high configuration laptop computer4 users – high configuration laptop computer► 2 users – personal computing environment2 users – personal computing environment

Recording softwareRecording software► MoraeMorae

Test participantsTest participants► Two user groupsTwo user groups► Total: 6 Total: 6

3 attorneys3 attorneys 3 journalists3 journalists

► Use of key informationUse of key information AttorneysAttorneys

► Case actionsCase actions► Orders/opinionsOrders/opinions► Rules and proceduresRules and procedures

JournalistJournalist► Case briefsCase briefs► Orders/opinionsOrders/opinions► Individual infoIndividual info

► Diverse age group: 26-55+Diverse age group: 26-55+► All experienced and frequent computer/Web usersAll experienced and frequent computer/Web users

Test procedureTest procedure

► Consent formConsent form► Pre-test questionnairePre-test questionnaire► 5 tasks: access + search + monitor5 tasks: access + search + monitor

1.1. Search general case information by case number/keywordsSearch general case information by case number/keywords

2.2. Search opinion or orders by case number/keywordsSearch opinion or orders by case number/keywords

3.3. Search for Texas rules of appellate procedureSearch for Texas rules of appellate procedure

4.4. Track general case development, specific actions or eventsTrack general case development, specific actions or events

5.5. Open task chosen by participants Open task chosen by participants

► Post-test questionnairePost-test questionnaire► Interview – user feedbackInterview – user feedback

Data Analysis - 1Data Analysis - 1

► Evaluation MeasuresEvaluation Measures Quantitative Quantitative

► Demographic and computer usage dataDemographic and computer usage data QualitativeQualitative

► Subjective responsesSubjective responses► Subjective ratingsSubjective ratings

► Task analysis by user groupTask analysis by user group Task 1Task 1

Data Analysis - 2Data Analysis - 2

Tasks 2 and 4Tasks 2 and 4

Tasks 3 and 5Tasks 3 and 5

Data Analysis - 3Data Analysis - 3

►Other issuesOther issues

►HTML Source AssessmentHTML Source Assessment

RecommendationsRecommendations

►Four major problem areaFour major problem area Menu designMenu design Information descriptionInformation description Information organizationInformation organization Information needsInformation needs

►Two levels of priorityTwo levels of priority►Additional functionalitiesAdditional functionalities►http://http://www.supreme.courts.state.tx.uswww.supreme.courts.state.tx.us//

Problem Area:Problem Area:Menu DesignMenu Design

► Level: 1Level: 1 Appropriate and familiar labelsAppropriate and familiar labels

►Front Desk Front Desk FAQ FAQ No sub menus for single entityNo sub menus for single entity Main menu entry for important infoMain menu entry for important info

►Contact information, FAQ Contact information, FAQ

► Level: 2Level: 2 Proper highlighting for current mouse focusProper highlighting for current mouse focus Don’t hide the remaining menuDon’t hide the remaining menu Menu categorization Menu categorization

►RulesRules

Problem Area:Problem Area:Information DescriptionInformation Description

► Level: 1Level: 1 Current and previous versions or rulesCurrent and previous versions or rules General case info upfrontGeneral case info upfront

►Party, trial courtParty, trial court

Contact information for inquiriesContact information for inquiries►Party, attorney, firmParty, attorney, firm

► Level: 2Level: 2 Rule numbers with current versionsRule numbers with current versions Clear labeling for date field in search formsClear labeling for date field in search forms Multiple input format for COA numberMultiple input format for COA number

Problem Area:Problem Area:Information OrganizationInformation Organization

►Level: 1Level: 1 Links to opinions & briefs with case infoLinks to opinions & briefs with case info The folder iconThe folder icon

►Level: 2Level: 2 Categorized labeling on the home pageCategorized labeling on the home page

►Latest news/releases, weekly orders, eventsLatest news/releases, weekly orders, events

Problem Area:Problem Area:Information NeedsInformation Needs

►Level: 1Level: 1 Party name in opinion search resultsParty name in opinion search results Time deadlines for casesTime deadlines for cases

►Level: 2Level: 2 Statistical information for a court termStatistical information for a court term

►Case trialedCase trialed►Opinions releasedOpinions released►Cases per judgeCases per judge

Additional FunctionalitiesAdditional Functionalities

► Case info in the case mail featureCase info in the case mail feature► Case search by Case search by

Law firmLaw firm AttorneyAttorney Party Party Pending casesPending cases

► Opinion search byOpinion search by KeywordsKeywords judgejudge

► Search interfaceSearch interface RulesRules E briefsE briefs

Questions or Questions or Comments?Comments?

Thank you !Thank you !