Taxonomy Governance Through Metrics

Taxonomy GovernanceThrough Metrics, Existing Tools & Process

November 09, 2007 @ 10:00am

Taxonomy Bootcamp, 2007

Alex Barnes

Tom Witczak

©2007 Hitachi Consulting. All Rights Reserved.

2

Session Objectives

• Understand what Governance Is

• Understand its Context

• Identify Core Taxonomy Governance Processes

• Identify Standard Processes and Tools

• Tips and Tricks for Governance


3

Taxonomy Feedback Loop

Taxonomy Model

Taxonomy

Management

Executive Sponsors form Strategy and Objectives that

are driven by metrics and business objectives

Front End:

Applications

Back End: Taxonomy

Control and Publishing

Taxonomy Change ControlExecution is driven by approved change requests

UpdatesPublished

Versions

Feedback and Requests for Change are captured through

periodic review, usability testing and feedback

Taxonomy

Users


4

Definitions

Governance is the management of

a system, usually political or

organizational, involving mutual

adjustment, negotiation, and

accommodation between the parties

involved rather than direct control.— Webster’s

Taxonomy Governance defines:

• Organizational Structure

• Processes and Procedures

• Practices, Standards and

Measures Goals are to define a repeatable,

visible, accountable, process for

submission, review, approval and

dissemination of taxonomy

changes.

Taxonomy

Practices,Standards &

Measures

Processes &Procedures

OrganizationalStructure

Tools and Solutions


5

Components

Processes and Procedures

• Steps taken when performing taxonomy

governance tasks

• “How-to” view of taxonomy governance

Organizational Structure

• Structure, membership, accountabilities,

organizations involved

• Who in the organization is responsible for each

aspect of taxonomy governance and how

delegations and escalations are carried out

Practices, Standards & Measures

• What will be done

• How it will be controlled and monitored

Tools and Solutions

• What existing tools or solutions can be used

• How can they be integrated and leveraged

Processes & Procedures

OrganizationalStructure

Practices,Standards &

Measures

Tools and Solutions


6

Core Operational Processes

Request and Track Taxonomy Changes

• Change request and status visibility

• Track changes thorough various lifecycle stages, start to finish

Assess the Impact of Change

• Assess the validity of change requests

• Impact of the requested change on taxonomy, consuming systems and processes

• Effort required to implement a change

Fix and Control the Taxonomy Baseline

• Consuming systems should know content of the taxonomy, its ownership, change history, and any relationships to downstream systems and processes.

• Implies administrative metadata, versioning, knowledge of the context in which the taxonomy is used

Validate Taxonomy Change

• Continually validate with end users for consistency and usability.

• Can be analysis, scripted tests with automated tools, or contextual inquiry with representative user populations.

• Must identify the appropriate validation method and coordinate verification efforts

Communicate Taxonomy Change

• Consumers need to know the change, the reason for change, known impacts, related changes, and their timing

AssessFix &

Control

Commun-

icateValidateRequest

& Track

observed defect observed fix


7

Organizational Structure

A

R

R

I

Executive Sponsors meet quarterly to review summary

taxonomy metrics and analysis results, and to define

and refine strategy

Business Unit Leadership team meets monthly to

review/ prioritize requests for taxonomy change; to

plan training, communications, and strategy; and to

approve release and versioning plans

Coordinates with the Business Unit Taxonomy Owners

as needed (generally online rather than face-to-face)

to assure compliance with processes and to plan the

content and timing of taxonomy releases

Execute processes, receive training, and recommend

taxonomy changes to the Taxonomy Librarian

AR C IResponsible Accountable Consulted Informed

Executive Sponsors

Business Unit Leadership

Taxonomy Librarian Operations

BU Taxonomy Owners Support Team

System Users

•Administer Taxonomy •Content Admin

•Solution Admin

•Control/Approve Changes •Coordinate Changes in

•Dependent Systems

Day-to-Day Taxonomy Management

Layered ModelIt is important to define a structure that separates policy and

strategic direction, operational coordination, and day-to-day

management and operations.

ThresholdsEscalation between layers is controlled by policies that define the

thresholds for communication, and what the content of the

communications should be. This will vary widely from one

organization to another.


8

Governance Responsibilities

Define Objectives

• For the taxonomy, its intended use, and how it

supports the organization’s strategic goals

Manage Change Requests

• Who are responsible parties for change requests,

maintenance and changes of the taxonomy

• Define the tools and processes to manage

change

• Define and track metrics on the performance of

the process

Manage the Taxonomy Baseline

• Who maintains the current model of the taxonomy

• Processes for reassignment of taxonomy

components and whom should reassign them

Coordinate Change in Related Systems

• Other systems and processes can be impacted –

coordinate change to the consuming systems and

processes

Communicate with Stakeholders

• Notify Consumers of intended changes and

current state of the taxonomy – stakeholders

may want to browse or query the taxonomy,

receive notification of planned changes

Plan and Prioritize

• Define the criteria for prioritizing requested

changes; who decides on priorities, and

escalation processes in case of disputes

Change Management

Strategy

Manage Change

Requests

Manage TaxonomyBaseline

CoordinateChanges

Communication

Pla

nnin

g


9

Delegated Management

Delegate

It’s not feasible for all taxonomy changes to

be channeled through a single individual or

group.

Governance bodies should delegate

ownership of the representative context of a

taxonomy and how changes to the parts are

made without compromising the overall

structure.

Standards

• Ownership and Reassignment

• Structural Norms and Naming

Conventions

• Versioning Criteria

• Validation and Testing

• Periodic Review and QA

Disposition

Item Action Member Action

Billing Action

Account ClosureBid Retraction

Listing Removal Auction Restart

Move Item

Suspension

Suspension

WarningPolicy Warning VeRO Warning Support Warning

Adjustment

Safety Alert

Temporary

Suspension

Indefinite

Suspension

ReinstatementWarning

30-day

Suspension

90-Day

Suspension

Collection

Account Linking

Flag Item Flag Member

Investigate

Member

Closed No-Action

Bid Retraction

Organization A

Organization B

Organization C

Organization D


10

Practices, Standards and Measures

Policies are a plan of action that guides decisions and practices based on stated objectives and strategies.

Practices are requirements that establish guidelines for all business units and aligns with defined policies.

Standards are specifications that details how to fulfill the practice; what is done? how often?

Measures are numbers or percentages that illustrate how well a practice is fulfilled.

Bu

sine

ss U

nit

Bu

sine

ss U

nit

Bu

sine

ss U

nit

Bu

sine

ss U

nit

Bu

sine

ss U

nit Standards

and

Measures

Policy

Practices


11

Metrics

Metrics can be gathered at each stage of the taxonomy life cycle, by the business unit and/or by dedicated taxonomy support staff.

Metrics can improve:

• Taxonomy and Information Architecture:

– Elimination of ambiguity

– Identification of new taxa

– Removal of redundant taxa

– Improved controlled vocabularies (labels and synonyms)

• Taxonomy change request process

• User experience in both taxonomy management tools and the operational use

• Identification of consistent patterns of issue, root cause, and change resolution to enable automated change resolution

Real-time metrics

• Usage volumes by:

–User segment

–Channel

–Geography

–Facet

• Taxonomy usage:

–Taxa that are never used

–“None of the above” responses

–Search statistics

Analytics

• Accuracy of categorization (% of categorizations

that are corrected in a later stage)

• Abandonment rate

• Taxonomy change request statistics

• Taxonomy defect metrics

• Usability test results


12

Taxonomy

Use the Existing Tools

Many of the goals and techniques of taxonomy

governance are common to other IT and data

governance methods. It’s better to reuse existing

tools rather than build or buy new ones. The

learning curve is shorter and the costs are

lower.

Lifecycle Model

• A taxonomy’s lifecycle is similar to that of

standard software change request

processes.

Baseline Control

• A taxonomy, just like code, should be

controlled and versioned.

Branching and Forking

• Ability to maintain concurrent models in

different states is common to many source

control systems.

The Draft/Review/Publish Model

• Existing systems already have workflows that

can be reused to control publication and

notification of changes.

Capturing Feedback

• Both “suggest new content” forms, customer-

service ticket-trackers and bug-tracker tools

can be used to capture user requests for

change.

Release Planning

• Existing project portfolio management

systems often include version description

and release-planning functions.

Workflow Bug Tracking

Source Mgmt.


13

Tool Requirements

Modeling

Essential features:

• Web access

• Ability to delegate ownership

• Versioning capability, both coarse and fine-

grained

• Rollback

• Flexible taxonomy metadata model

(administrative and engineering)

• Change history– traceability of changes to

CR

Publication

Must have:

• Integration with model

• Integration with change request solution

• Version description/release notes capability

• Flexibility in publication format (DB dump,

XML)

Change Request Tracking

Must have:

• Ability for end users to report issues and see

resolution status

• Resolution workflow

• Integration to model

Should have:

• Notification

• Metrics capture and reporting

Change

PublishModel


14

Summary

You now should understand Taxonomy Governance:

• Processes and Procedures

• Organizational Structure

• Practices, Standards & Measures

• Tools and Solutions

You now should understand its context:

• Taxonomy exists to support organizational goals and their supporting business

functions. As such, a taxonomy governance model is needed.

You should be able to identify core taxonomy governance processes:

• Not much is unique to taxonomy governance, as it resembles similar governance

processes (data, code, etc.).

Identify Standard Processes and Tools:

• Many opportunities for reuse with existing solutions; adoption is better with reuse.

Tips and Tricks for Governance:

• For more info, see appendices.


15

Tom Witczak

Senior Consultant

Hitachi Consulting

www.hitachiconsulting.com

Mobile: 415.412.2915

[email protected]

Inspiring your next success! ®

Alex Barnes

Senior Architect

Hitachi Consulting

www.hitachiconsulting.com

Mobile: 415.297.0712

[email protected]

Inspiring your next success! ®

Questions?

AppendicesA. Integrations

B. Organizational Change Management

C.Taxonomy Defect Types


17

Integration with Process & Workflow

Process Analysis Framework

Taxonomy does not exist in isolation. It should be informed

by a process analysis framework: a model that is used to

understand key business processes. It will serve to interpret

as-is processes and to scope subsequent process

development iterations.

Developing a process framework is an activity outside the

scope of taxonomy governance. It requires a coordinated,

global effort touching upon several facets of business

process management. These components include:

formulating a consistent strategic view of processes,

mapping current state processes, assessment and

prioritization of processes, establishing development and

modeling standards, process governance, and finally

process management. Supporting the framework are the

technologies, such as orchestration, infrastructure and how

processes are realized and presented to the user.

The process framework, as it is executed, will yield

requests for taxonomy change, potentially including the

identification of new taxonomic facets.

If the process framework includes elements of participatory

design, valuable stakeholders will actively contribute their

knowledge to the process models, translating to increased

end user adoption of the solution.

The taxonomy is exposed within the process framework

through taxonomic views that are task-specific. By some

definitions, these taxonomic views are regarded as part of

the information architecture of the solution.


18

Coordinating Change

• Metadata should exist in the taxonomy model to reflect these differences in the impact of taxonomy changes on consuming systems and processes

• When a taxonomy update is published, it must be synchronized with changes to systems and processes that are sensitive to that part of the taxonomy

Dependencies

Taxonomy is seldom exposed directly on a user interface. Instead, task-specific views of the taxonomy are shown to the user. These may have a structure that is significantly different than that of the underlying taxonomy: for example, a node may have multiple parents on the UI.

But it is important that traceability is maintained between the taxonomic views and the underlying taxonomy, so that changes to the taxonomy lead to changes in the user experience.

Taxonomic views may be surfaced in a number of ways:

• Navigation

• Workflow/routing

• Filtering

Along with the hierarchical relationships depicted in the taxonomy, user interfaces (and system behavior) may also be responsive to frequency-of-use considerations. This frequency-of-use data must be measured by observation of system usage, and can be tracked as additional metadata on taxonomy nodes.

Binding

Different systems consume taxonomy in different ways:

• Realtime: the system looks at, and responds to, the current taxonomy as it is needed, on a case-by-case basis

• Load time: the system is initialized with a snapshot of the taxonomy at the time it is started

• Compile time: the system is built in a way that incorporates knowledge of the taxonomy

While realtime binding is a desirable design goal, most systems are not this adaptable to taxonomy changes.

In addition, business processes are also sensitive to taxonomy:

• Categorization processes

• Metrics

Like systems, processes respond to taxonomy changes with varying lag times and varying costs of change

Integration with Consuming Systems

System A

System C

System BReal time

Com

pile t

ime

Load t

ime

Taxonomy


19

Organizational Change Management

Approach

The key to a successful transition from the legacy state to the

new taxonomy governance model is to:

• Understand the Organizational Landscape and develop

an a change management approach consistent with the

organization’s needs

• Establish and support Leadership and Stakeholder

Commitment to actively lead the initiative

• Develop an effective Learning strategy that educates the

organization on new processes and functionality from the

system and also builds enthusiasm, adoption and skills in

stakeholders through exceptional training.

• Creation and execution of an effective Communication

plan that delivers the right message to the right person at

the right time.

The goal of OCM is to assist and drive the following key

activities to project success:

• Methodology adoption by BU, tech team, and SMEs

• Definition of critical success factors

• Appropriate analysis of the stakeholder landscape

• Definition and execution of the communication plan to

inform all relevant stakeholders and stakeholder groups

• Communication with involved groups to make sure SME’s

are aware of tasks and complete tasks on time

Key Work Products

• OCM risk diagnostic

• Stakeholder analysis

• Champion Framework

• Channel assessment

• Management Model

• Training Approach

• Communication plan

• Leadership Action Plans


20

Taxonomy Defect Types

Code Definition

Completeness/Consistency This category of taxonomy defects is for incomplete (missing child nodes) or

inconsistent (conflicts with, duplicates or overlaps with other nodes) nodes.

Incomplete Not all of the node’s child nodes are specified.

Conflicting The node wholly or partially conflicts with another node.

Wrong parent The node is placed incorrectly in the hierarchy.

Duplicate The node is the same as all or part of another node (not mutually exclusive).

No Longer Required There is no longer a business requirement for the node to exist.

Missing A required node is not present.

Correctness This category includes defect codes for nodes that are incorrect, unverifiable or cannot

be implemented.

Incorrect (Not a Node) The requirement for the node is logically flawed, or is not needed to support the business

process.

Ambiguous The definition is ambiguous.

Dimensional Problem The node or nodes are repeated due to dimensional problems with the hierarchy.

As analysis is performed on Taxonomy Change Requests, they must be categorized. This determines resolution workflow, and provides a basis for ongoing measurement of taxonomy quality. A change request can be tagged with one or more defect types. These codes are based on software engineering and data quality defect codes.


21

Taxonomy Defect Codes, cont’d.

Code Definition

Presentation/Style This category includes taxonomy defects of an editorial nature.

Unclear Wording The wording of the definition or label is difficult to understand.

Typo/Grammar The definition or label has spelling, typographical or grammatical errors.

External Taxonomy

Change

This category includes defect codes for changes introduced by an external change control

process in response to changing business requirements or project scope.

New A new node (or nodes) has been added.

Modify An existing node (or nodes) is to be modified.

Delete An existing node (or nodes) is to be deleted.

Miscellaneous

Changes

This category includes other taxonomy changes.

Synonym A taxonomy node is identified by, or searchable by, more than one name.

Not a Node The node should not be in the taxonomy at all.

Update Metadata Change administrative metadata (ownership, related systems, etc) for a node.