Sept. 25-26, 2008 City of St. Louis, Missouri 2008 Taxonomy Boot Camp Organizing Information for Search & Discovery San Jose McEnery Convention Center.

Sept. 25-26, 2008

City of St. Louis, Missouri

2008 Taxonomy Boot Camp

Organizing Information for Search & Discovery

San Jose McEnery Convention Center

San Jose CA

Planning Made All The Difference: City of St. Louis Case Study

Sept. 25-26, 2008 City of St. Louis, Missouri

Agenda

1. Background2. Research and Review of practices3. Information Architecture Expertise4. Metadata and Taxonomy Development

Process5. Taxonomy Testing6. Implementation7. Governance8. Conclusions


Background

Started as a Community Information Network not a municipal website in 1995.

Focus was on general useful content (history, Neighborhoods, Education, Housing, Services, and Government). Municipal information came later.

By 2006 site had over 10,000 pages of content, 120+ stand-alone dynamic applications and no overall way to stitch everything together

Organizing and finding information was an exercise in faith and frustration. Complaints became increasingly common.


Site Consolidation Goals

Decided to overhaul website and consolidate web presence using a content management solution.

Goals:1. Classification and organization of content

2. Integration of 120+ dynamic applications

3. Use standard metadata and create a Standardized Municipal Topic Classification

Challenge:1. How?


Agenda




Research & Review of Practices

No standardized US model for government available International models reviewed:

Australia: Thesaurus of Australian Government Subjects

http://www.naa.gov.au/records-management/create-capture-describe/describe/agls/index.aspx

Canada: Government of Canada Core Subject Thesaurushttp://en.thesaurus.gc.ca/intro_e.html

United Kingdom: Integrated Public Service Vocabulary (IPSV)http://www.esd.org.uk/standards/ipsv/



http://en.thesaurus.gc.ca/intro_e.html

http://www.esd.org.uk/standards/ipsv/




British IPSV as model

Top Terms

Deep Structure

http://www.esd.org.uk/standards/ipsv/viewer/

The British central government developed and promotes the Integrated Public Services Vocabulary as the standard for all British Governmental bodies.






How to Integrate Our Early Classification Effort

User Survey Services A to Z Random Site Topics Category Agreement Assessment


Survey: Category Agreement Assessment (CAA)

400+ terms on site Random presentation of

20 terms User asked to type

primary and secondary category

Results generated list used for our A to Z pages


Category Agreement Assessment:

A to Z Resulted in 45

categories Created A-Z list

Topics Categories

Beginning of a Folksonomy


Tagged sample contents with IPSV

Mapped our A to Z topics and categories against British IPSV


Findings Our Content topics mapped well to IPSV IPSV hierarchy was too deep and complex Content classification needed to expand beyond

topics Effort was beyond our team’s expertise Outside Information Architecture services was

needed to create a robust but simpler classification


Agenda




Retaining IA Expertise

Identified possible resources mentioned in reputable resources with a mandate toward standards and Dublin Core

Developed list of questions Interviewed contacts to identify good match and

clarify scope Selected Taxonomy Strategies to filter all

information and devise a well structured Information Architecture document to guide our CMS implementation


Agenda




Materials Used to Create Classification Schema

IPSV List and frequency of Search Terms User Survey Results:

Usability Assessment Results List of information people could not find on website Category Agreement Assessment (folksonomy)

A to Z content items and categories Audience/user scenarios CMS Application requirements list Site Architecture and Navigation Requirements


Summary of St. Louis Metadata

Standard Text Fields:

• Title, Description, Date, Headline, Subhead, …

• Useful for search, and for linking from one page to another.

Typical Vocabulary-based Fields:

• Access, Audience, Content Type, Organization, Topic

• Content Type provides the basis for page templates.

• Organization and Topic provide basis for dynamic navigation.

Special Vocabulary-based Fields:

• Event Type, Neighborhood, Procurement Type, Ward

• These are key to improving the dynamic navigation.

Vocabulary Name

Vocabulary Purpose and Description

Access Groups

List of group names for different access control groups. Currently only contains "Public" until we understand the various access control groups better.

Audience

Describes a small number of target audiences for content. This list may grow in the future but the amount of content on the site now does not justify it, esp. since this is likely to be highly correlated with Topic and Organization.

Content Type

List of genres of content produced by the City of St. Louis, such as press releases, RFPs, meeting agendas and minutes, etc. Page templates are based on Content Type, and many queries will look for specific content types.

Event Types

List of types of events, such as public meetings, alderman meetings, exhibitions, etc. Used to categorize calendar events so that more tailored calendar listings can be shown.

NeighborhoodSimple list of Neighborhoods within City of St. Louis. Used for geographic tagging.

OrganizationHierarchical list of entities within the government of the City of St. Louis, or affiliated with the government.

Organization Type

Simple list of different type of organizations that provide content which may be linked to from the site.

Procurement Types

Sub-content types for "RFPs, Bids and Notices."

TopicHierarchical list of topics relevant to city and county governments within the United States. This is the main subject categorization scheme.

Ward Simple list of Wards within City of St. Louis. Used for geographic tagging.


Summary of St. Louis Content Types.CVLabel Synonyms

A-Z PagesCalendars and Schedules Event Calendar, Events, Public MeetingsContact Info and Biographies Bios, DirectoryFacts, Figures, and Demographics Charts, Graphs, Population, Data, StatisticsGallery pages Video, Audio, ImagesGroup Home PagesHelp pagesInformational Pages

Job OpeningsJobs, Help wanted, job annoucements, employment opportunities

Laws, Bills, and PoliciesRegulations, Ordinances, Procedures, Rules, Laws, Legislation, Bills, Board Bills

News and Annoucements Newsletters, BulletinsMaps and GISMeeting Materials Agendas, Minutes, HandoutsOnline Forms and Applications Online Surveys, Find an Elected Official, Permits

Plans and ReportsCAFR, Comprehensive Annual Financial Reports, Budget reports, Departmental status reports, Program performance reports

Printed Forms and Templates Surveys, Permits, Tax Forms, Records Request

RFPs, Bids, and NoticesRFP, RFI, RFQ, Proposals, Bids, Contract Awards, Procurement

Subscriptions


Business & IndustryBusiness Assistance, Incentives and PromotionBusiness Regulation and LicensingBusiness TaxationLabor Market Facts & FiguresLocal EconomySmall BusinessWorkforce Training and Development

CommunityBirths, Deaths, Weddings, and other Life EventsBeautification and Nuisance ReductionCommunity FacilitiesFacts, Figures & Community SpiritHistoryNeighborhoodsNeighborhood AssociationsNeighborhood Programs & EventsSocial and Support GroupsVolunteeringNon-Governmental OrganizationsImmigration and Nationality

Education & Skills TrainingAdult and Community EducationFurther and Higher educationHomeschoolingPublic and Private SchoolsSpecial Educational Needs and Additional Support

Employment, Jobs & CareersEmployment Rights and RegulationsLocal Government Employment OpportunitiesSelf employmentUnemployment and jobseekingYouth Employment

EnvironmentEnvironmental ProtectionHazardous MaterialsWaste Management & RecyclingClimate

GovernmentLocal Government RecordsLocal Governmental UnitsComplaints and Service RequestsElections & VotingElected OfficialsFederal and State Government ServicesFees & Payment InformationLocal Government ServicesPermits, Inspections & CertificationsPolicy MakingProcurementTaxation & RevenueWards and NeighborhoodsAccount & Annual Budget & Operating PlanEmployees

HealthAnimal ControlImmunizations and Public HealthFood SafetyNutrition and Preventative Care

Housing & Community Development

Home OwnershipHome RentalHousing Financial ProgramsHome Repairs and RenovationRelocationTemporary HousingHomelessness

Leisure & CultureArts and Entertainment EventsChildren's and Teens ActivitiesLibraries, Museums & Cultural VenuesGambling and LotteriesParks and GardensPets and AnimalsShopping & DiningSports and RecreationTourism

Planning & Urban Development

Land AcquisitionPreservationPlanningProperty AssessmentZoning

Public SafetyPoliceFire Dept.Ambulance and Emergency ServicesBuilding Inspections & PermitsCourts and Justice SystemLegal AssistanceReporting & Operation Planning

Transport & infrastructureAirportsConstruction, Maintenance & ImprovementsElectric, Gas, Sewer & Water UtilitiesTelephone, Cable TV & Internet UtilitiesMotor VehiclesPublic TransportationParkingRail and Water TransportationStreets, Sidewalks & Streetlighting

St. Louis Topics.CV Before Usability Testing


Dublin Core Metadata Standard

http://www.dublincore.org Followed the Dublin Core Metadata Element, a vocabulary of fifteen properties for use in resource description

Mapped Dublin Core properties to corresponding metadata fields in the proprietary CMS

Coded to output DC metadata in source code Examples:

DC.Term Description CMS Equivalent

dc.coverage.ward

Location related to content custom metadata perhaps attached to the base template

dc.terms.audience

Audience content is geared toward

request.page.audiencecategory

dc.type Type of content (e.g. Press Release, Fiscal Report, Meeting Minutes, etc

request.Page.CategoryID

dc.subject Relevant terms from Topics.CV

Page.CategoryID


Agenda




Taxonomy Testing

3 testing approaches used:

Tagging representative content Card-sorting and use-based scenario testing with

focus groups Benchmarks and metrics to evaluate usability test

results, identify coverage gaps, and provide guidance for changes.


Tagging Representative Content

Test Cases: A to Z topics Random URLs Human Services

content pages


Card Sorting and Use-based Scenarios with Focus Groups

22 representative questions were selected 42 participants were tested Testers were asked “where would you look for …” Testers were asked to “think aloud” Pre- and post questions were asked Initial findings indicated that 8 of the 22 scenarios

may require revisions Governance Team agreed on revisions 8 of the original scenarios were re-tested using 8

new participants Modifications to the proposed taxonomy were

based on the findings and approved by Governance Group.


Task-based Card SortRepresentative Question Example: You are planning to build a new deck on your house. You want to know if you need a permit for this. Which topic would you select to get started?

Topics1. Business & Industry2. Community3. Education & Skills Training4. Employment, Jobs & Careers5. Environment6. Government7. Health8. Housing & Community Dev9. Leisure & Culture10. Planning & Urban Development11. Public Safety12. Transport & infrastructure

6. GovernmentLocal Government RecordsLocal Governmental UnitsComplaints and Service RequestsElections & VotingElected OfficialsFederal and State Government ServicesFees & Payment InformationLocal Government ServicesPermits, Inspections & CertificationsPolicy MakingProcurementTaxation & RevenueWards and NeighborhoodsAccount & Annual Budget Employees

1.Topics2.Content Types


Analysis of Task-based Card Sort

Scenario

1. Business

and Industry

2. Communi

ty

3. Education and

Training

4. Employment, Jobs, and

Careers

5. Environment

6. Government

7. Health

8. Housi

ng

9. Leisure and Culture

10. Urba

n Developme

nt and

Planning

11. Law,

Safety, and

Justice

12. Transportation,

Infrastructure, and

Utilities

1. Historic development requirements 1 1 6

2. Lead Remediation Initiatives 1 1 3 2 1

3. Jury Duty 3 5

4. Permits 1 2 2 3

5. Street light 3 5

6. Complaints 1 2 1 4

7. Assess Property 1 4 2 1

8. Census Data 2 6


Summary Usability Testing Process

Card Sorting and Use-based Scenarios Governance Team Members Involvement Re-testing and Validation Changes to Taxonomy and Metadata based

on resultsFor Example: ‘Public Safety’ became ‘Law, Safety, and Justice’ Words like “Development” and “Community”

repeated in different terms were eliminated Identified additional ‘See Also’ association


Business & IndustryBusiness Assistance, Incentives and PromotionBusiness Regulation and LicensingBusiness TaxationLabor Market Facts & FiguresLocal EconomySmall BusinessWorkforce Training and Development

CommunityBirths, Deaths, Weddings, and other Life EventsBeautification and Nuisance ReductionCommunity FacilitiesFacts, Figures & Community SpiritHistoryNeighborhoodsNeighborhood AssociationsNeighborhood Programs & EventsSocial and Support GroupsVolunteeringNon-Governmental OrganizationsImmigration and Nationality

Education & Skills TrainingAdult and Community EducationFurther and Higher educationHomeschoolingPublic and Private SchoolsSpecial Educational Needs and Additional Support

Employment, Jobs & CareersEmployment Rights and RegulationsLocal Government Employment OpportunitiesSelf employmentUnemployment and jobseekingYouth Employment

EnvironmentEnvironmental ProtectionHazardous MaterialsWaste Management & RecyclingClimate

GovernmentLocal Government RecordsLocal Governmental UnitsComplaints and Service RequestsElections & VotingElected OfficialsFederal and State Government ServicesFees & Payment InformationLocal Government ServicesPermits, Inspections & CertificationsPolicy MakingProcurementTaxation & RevenueWards and NeighborhoodsAccount & Annual Budget & Operating PlanEmployees


Housing & Community Development

Home OwnershipHome RentalHousing Financial ProgramsHome Repairs and RenovationRelocationTemporary HousingHomelessness

Leisure & CultureArts and Entertainment EventsChildren's and Teens ActivitiesLibraries, Museums & Cultural VenuesGambling and LotteriesParks and GardensPets and AnimalsShopping & DiningSports and RecreationTourism

Planning & Urban Development

Land AcquisitionPreservationPlanningProperty AssessmentZoning

Public SafetyPoliceFire Dept.Ambulance and Emergency ServicesBuilding Inspections & PermitsCourts and Justice SystemLegal AssistanceReporting & Operation Planning

Transport & infrastructureAirportsConstruction, Maintenance & ImprovementsElectric, Gas, Sewer & Water UtilitiesTelephone, Cable TV & Internet UtilitiesMotor VehiclesPublic TransportationParkingRail and Water TransportationStreets, Sidewalks & Streetlighting

Topics.CV Before Usability Testing


Finalized Topics.CV After Usability Testing

Business and IndustryBusiness Assistance, Incentives, and PromotionBusiness Regulation and LicensingBusiness TaxationLabor Market Facts and FiguresLocal EconomySmall BusinessWorkforce Training and Development

CommunityBirths, Deaths, Weddings, and other Life EventsBeautification and Nuisance ReductionCommunity FacilitiesFacts and FiguresHistoryNeighborhoodsNeighborhood AssociationsNeighborhood Programs and EventsSocial and Support GroupsVolunteeringNon-Governmental OrganizationsImmigration and Nationality

Education and Skills TrainingAdult and Community EducationFurther and Higher educationHomeschoolingPublic and Private SchoolsSpecial Educational Needs and Additional Support

Employment, Jobs, and CareersEmployment Rights and RegulationsLocal Government Employment OpportunitiesSelf employmentUnemployment and JobseekingYouth Employment

EnvironmentEnvironmental ProtectionHazardous MaterialsWaste Management and RecyclingClimate

GovernmentLocal Government RecordsLocal Government Offices, Agencies, and DepartmentsComplaints and Service RequestsElections and VotingElected OfficialsFederal and State Government ServicesFees and Payment InformationLocal Government ServicesPermits, Inspections, and CertificationsPolicy MakingProcurementTaxation and RevenueWards and NeighborhoodsAccount and Annual Budget and Operating PlanEmployees


HousingHome OwnershipHome RentalHousing Financial ProgramsHome Repairs and RenovationRelocationTemporary HousingHomelessness

Law, Safety, and JusticePoliceFire DepartmentAmbulance and Emergency ServicesCourts and Justice SystemLegal AssistanceTourism

Leisure and CultureArts and Entertainment EventsChildren and Teens ActivitiesLibraries, Museums, and Cultural VenuesGambling and LotteriesParks and GardensPets and AnimalsShopping and DiningSports and Recreation

Transportation, Infrastructure, and Utilities

AirportsConstruction, Maintenance, and ImprovementsElectric, Gas, Sewer, and Water UtilitiesTelephone, Cable TV, and Internet UtilitiesMotor VehiclesPublic TransportationParkingRail and Water TransportationStreets, Sidewalks, and Streetlighting

Urban Development and PlanningLand Acquisition, Reutilization, and DevelopmentPreservationPlanningZoning


Process Summary

Review and filtering of all relevant documentation (A to Z classification, International Best Practices, search terms and frequency, use scenarios, requirements, etc.)

= Draft Information Architecture Document

+ Sample Tagging of pages

+ Wireframes

= Proposed Information Architecture Volume 1.0

+ Taxonomy Testing

= Adopted Information Architecture Volume 1.0


Agenda




Wireframe Tagging of Page

Populated by querying calendar system with

Org =“Cultural Resources Office” ORTopic= ”Planning & Urban Development/Preservation” OR Topic=“Community/History”

WHERE Date > Today

ASCENDING ORDER BY Date

Related folders box dynamically populated from queryAROUND(“Planning & Urban Development/Preservation”, “Community/History”).

Same as See also references

Spotlights and other sidebars can be manually filled at page creation/edit time. Alternatively, they can be dynamically filled with queries like:

SELECT (Title, URL, Description,

Thumbnail) WHERETopic = AROUND

(”Planning & Urban Development/Preserv

ation”, “Community/History) AND Thumbnail not

null.

Page Index with Metadata for content placement

Office contact info dynamically populated from contacts database WHERE Org = Planning&UrbanDesign Agency/CulturalResourcesOffice

Tagging of Page:Type=Group Home PagesOrg=Planning & Urban Design Agency / Cultural Resources OfficeTopic=“Planning & Urban Development/Preservation”, “Community/History”Coverage.Wards=“MANY_WARDS”Coverage.Neighborhoods=“City Wide”


Taxonomy & Applications Integration

Challenge: How to leverage Information Architecture for outside applications (120+)?Solution: Create homepage in CMS Login Permissions managed by CMS Re-coded to have one single page output Select Metadata and taxonomy during page creationExample: Gatewood Cemetery Application data outside of CMS tables Permissions set inside CMS Display page is tagged using CMS metadata. Topic relationships visible in “Browse by Topic” and “ See Also…”

http://consulting.paperthin.com/stlouis/browseByTopics.cfm

http://consulting.paperthin.com/stlouis/browseByTopics.cfm


Browse by, See Also…

Tagged Content Page

Topic association with other facets

External Application Integrated in CMS

using Metadata and Taxonomy


Agenda


Process5. Taxonomy Testing6. Implementation 7. Governance8. Conclusions


Governance Information Architecture Governance Team (IAGT)

created to institute a process for changes. 9 members, representatives of major sections of

government (Public Safety, Community Services, Parks, Health) and Web Team Manager as Lead

IAGT Responsible for Policies Governing Process Change. Policy Documents to-date: Process Guide Editorial Rules

IAGT Responsible for Testing and Validation


Agenda


Process5. Taxonomy Testing6. Implementation 7. Governance8. Conclusions


Conclusions Goal to standardize Metadata and Taxonomy for

Municipal Government proved difficult. Taxonomy Testing was critical Testing with user groups forced customization of

taxonomy that reflected local flavor Process helped identified areas of missing content

and classification. (i.e. Included audience facet for human services type classifications)

Benefits Created: Strengthened Relationships between content Extensible to other section of city government Improved ability to find information and search More discreet filtering Established process for future site/content growth.


Thank you!

For more information contact:

Sonya PelliInternet Services Manager

Information Technology Services AgencyCity of St. Louis, Missouri

[email protected]://stlouis.missouri.org

314 622-3400 x258

Sept. 25-26, 2008 City of St. Louis, Missouri 2008 Taxonomy Boot Camp Organizing Information for Search & Discovery San Jose McEnery Convention Center.

Documents

city of

missouri background

louis case study slide

british ipsv slide

missouri survey

missouri agenda

folksonomy slide

missouri british ipsv