Project Management and Technology don't matter

District of Columbia Public Schools | 1200 First Street, NE | Washington, DC 20002 | T 202.442.5885 | F 202.442.5026 | www.dcps.dc.gov

Project Management &Technology don’t matter

Council of Great City Schools CIO Conference3 June 2015

Introduction

2

Project Management and Technology Don't Matter.

The District of Columbia Public Schools successfully completed a major effort to replace its Student Information System (SIS). Phase 1 for 60 elementary schools was completed in only 4 months from vendor start to first go-live for SY14-15. Phase 2 for the remaining schools was completed for the 2015=2016 school year.

Key Factors:• Managing failure rather than success• DCPS in house technical expertise and leadership• Strong focus on people rather than technology

Student Information System (SIS) Replacement

District of Columbia Public Schools | 2015 CGCS CIO Conference 3

About Me – Andrew Patricio Deputy Chief for Data Systems• Reporting to Chief of Office of Data and Strategy• Worked as contractor for DCPS 2009- 2010• Started as FTE fall of 2010• Most senior technical resource at DCPS

(No official DCPS CIO)

Personal background• BS in Electrical Engineering• Previously worked in Management and

and IT Consulting• DCPS is my first stint in government• Two adorable kids



Project Management and Technology


“Project Management and Technology don’t matter”

Project Management and Technology DO matter….

...but not as much as we think



“Project Management doesn’t matter”

Good project management governs how well you succeed not if you succeedMany IT projects have been “well-managed failures”• Tracked all milestones• Had strong technical team• Still resulted in cost-overruns, delays,

poorly utilized systems

Some kind of project management is important, but details don’t matter• Project manager should understand school context, not just manage deliverables• Approach should be compatible with organization management style



“Office Space” 1999

“Technology doesn’t matter”

Technology choice matters, implementation less so• Implementation governs how well you succeed not if you succeed• Can’t fix non-scalable, non-customizable system after the fact• Many effective options available for base technology so choose

what is most compatible with skill set available to your team• Procurement is therefore critical step for technology aspect



Ensure Competent Technology – RFPCritical technological step is effective RFP processTrue competitive solicitation process resulted in multiple potential vendors with system that met requirements for DCPS• Released RFP to the street for 2 months• Vendors had opportunity to ask questions• Evaluated written proposals• Invited vendors in for demos

• 1 full day for vendor to demo SIS features to school staff focus group• 1 half day to demo specifically how system can be customized by district

• Evaluated demos and “Best and Final Offer” proposals

Decision made based on total points awarded for proposal, demos, and cost



RFP – Scope of WorkProposal requirements were intended to get a sense of vendor’s implementation skills as well as system feature set. Three areas of requirements:Vendor background and project approach • Project implementation plan• Vendor transition plan – how to transfer development over to DCPS• Vendor qualifications – company size, stability/longevity of product ,etc

Feature set out of the box• Attendance, Mark Entry, Scheduling,

Enrollment, Gradebook, etc

Customization• We were very specific about what we meant by customization: specific data model

modifications, user interface customizations, etc



RFP – Prioritized RequirementsEach detailed technical requirement was assigned a rating within a range of necessity rather than simply “optional” or “required”Provided way to convey more nuanced view of DCPS thinking



Rating DescriptionMandatory Any solution which does not offer this feature will not be considered.

Critical Solutions lacking this feature will still be considered but will be evaluated negatively.

Necessary All solutions should offer this feature. DCPS is looking for solve a specific issue that is not being addressed well or at all with current SIS. Solutions lacking this feature will be evaluated negatively but can make up for that with greatly enhanced functionality in other areas. Missing support for too many necessary features will make it much less likely that the vendor will win this contract.

Expected Best solution should offer this feature. DCPS views this as being a part of an effective system but can make do without it. Solutions lacking this feature will not be evaluated as positively but will not necessarily be evaluated negatively. Supporting these features will help make up for any lacking support for "Necessary" features.

Desired Ideal solution should offer this feature. DCPS views this as something that would help differentiate between two equally good solutions. Supporting these features will not make up for lack of support for any "Necessary" features.

Helpful Wishlist feature. DCPS is not currently encountering an issue addressed by this feature but recognizes its usefulness. These features are mentioned for completeness but will have very little bearing on the overall qualifications of a particular solution.

RFP – Prioritized Requirements – Examples



Ref # Requirement Priority Weighting

GC1 Solution must be a web-based application with an interface using the native rendering capabilities (ie HTML, CSS, Javascript, etc) of standard web browsers without requiring any plugins. Specifically any solutions requiring installation or updates of Java, Flash/Shockwave, Silverlight, or similar plugins will be rejected.

Mandatory

GC10 Every student record must have a unique, global, permanent ID generated by system

Critical

CF13.2 API to pull dynamically pull any data from backend database for use in front end customizations

Necessary

GU12.1 validates each field as it is entered rather than after an entire screen or transaction is entered

Expected

AG21 allows for the printing of attendance or hall passes to identify late arrivals and early departures

Desired

GU12.6 provides “spell checking” ability to all narrative fields Helpful

RFP – Vendor Requirements CoverageSimilarly all vendors were required to be specific in how they proposed to meet each requirement.



Rating Description

Y This feature can be demonstrated in system immediately

N System does not provide this feature

C1, C2, C3 Feature can be implemented by DCPS via increasing level of effort to customize system (C1 – few days, to C3 – few months)

UD Feature is being actively developed by vendor for next release

P Feature is currently planned to be included in future release

3 Feature provided via 3rd party software or add-on

RFP – Vendor Demo – System Features



1.5 days for each vendorFirst full day – feature demo• Ran through all features of system with school

based staff focus group• DCPS required presence of actual assigned vendor

project manager to evaluate people skillsSecond half day – customization/technical demo• Purely technical, ran through example

customizations for DCPS technical resources• Required vendor deep technical experts to be

present (not just sales) to get into details

RFP – Vendor Demo – People factors

HumilityVendor team is willing to take direction and are not wedded to their own ideas

HearingVendor team listens to what we are actually saying instead of just assuming what we are going to say

HumorVendor team is personable and easy to work with



Intangibles are just as important as system features

RFP – Technology Chosen



Result was that we were as confident about choice of technology as we could be before implementing

• Did not encounter any big surprises during technical implementation

• Minimal “no worse than STARS” worst case scenario was assured



So if not that, then what does matter?

Implementation - Manage failure


Limit scope

Renovated house on my own• Scope creep• Kept assuming best case

scenario• Project ended up taking

almost 3 years

Lesson:• Be pragmatic• Walk before you can run



“Manage Failure”

Failure is inevitable



Manage FailureCan’t do everything well. Some things are going to drop



Consciously decide what to do well and what to skimp on instead of falling into it • Things need to be done right but good enough is usually good enough• First focused on school needs then central office reporting

• Data starts at the schools: garbage data in = garbage out for reporting

Goal was being perfectly clear, not being perfect• Clarity has a certain correctness all its own• Everyone should agree on what decision was, even if they disagree with it

Facts Not OpinionsMade conscious decision to prioritize timeline over perfecting process• No business process improvement meant that requirements were stable

target• The “right process” is an opinion, the “existing process” is a fact

Manage Failure



Phase 1 – Elementary Schools onlyElementary Schools as de facto “Pilot” for Aspen at DCPSThey do everything that Middle and High schools do but much simpler• Enrollment the same• Daily instead of period attendance/scheduling• Simple rubric report card: no course marks, GPA, or graduation requirements

Our mantra for Phase 1 was “If STARS didn’t do it, Aspen wouldn’t either”• DCPS did not have the time or capacity

for deep process improvement• Familiar process in new system

= easier transition for schools



Phase 2 – All remaining schoolsScheduling was exception to “no process improvement”5 high schools with veteran staff scheduled SY15-16 courses entirely in Aspen• Hired dedicated contractors who were experts in scheduling to support pilot• Developing standard process for DCPS to use to schedule SY16-17

Remaining schools started scheduling SY15-16 in DC STARS, finished in Aspen• Scheduling from scratch in Aspen would have been overwhelming• Most schools normally copy and manually tweak schedules from year to year

Implemented remaining phase 2 requirements also using STARS as baseline• No replacement of other systems initially, eg Student Behavior Tracker• No new functionality initially, eg parent portal, student schedule requests



Reporting Platform - Phase 1SY14-15 reporting from two systemsReporting platform combines student level data from both STARS and Aspen to provide district-wide reports for certain data

Started off with elements common to ES. Data which was strictly MS/HS was deprioritized since it was still in STARS so could continue to use existing reporting



ETL

STARS

Aspen

Reporting Platform

HS/MS only reporting:Courses, GPA, etc

Elementary / District-wide reporting: Attendance, Enrollment, etc

(ETL = Extraction, Transformation, Load)

Reporting Platform - Phase 2 & beyondSY15-16 and beyond reporting Reporting platform is independent from transactional systems and thus data model/processing can be optimized for reporting.• More efficient reporting

• Create pre-calculated datasets to report from instead of re-implementing business rules in every report so not redoing work

• Correlate data from separate databases, including manual pulls• Provide more self-service access to data instead of everything is a custom query



ETLSp Ed

Aspen

Reporting Platform

All reporting

LMS

Secondary & Tertiary Processing

Implementation - District Expertise


In House expertise and project leadership1983 Mercedes diesel • Easy to understand• Can direct repairs on anything I

can’t fix myself• Much cheaper to run than

previous modern (2011) vehicle

Lesson:• Importance of understanding

things myself when working with vendors



Toxic cycle for IT projects

Well-meaning vendors, passionate about technology



Well-meaning school district personnel, passionate about education

+ =Disaster

Why?

Lost in translation


DCPS concept

I wantGrey skinned animalMust have a trunkMust have large earsMust have tusks

“What do you want?” Delivered by external resource lacking DCPS context

…

After multiple expensive iterations, still not quite right

DCPS Program Office Vendor Technical Team

Vendor’s interpretation matches requirements but does not meet need and frustration ensues. “What do you want?” is the wrong question because DCPS program client is not a technical expert and vendor is not a business expert:

No one translating what programs need.

Speaking the Same Language


…modulated by DCPS technical team familiar with DCPS context (minimal iterations)

Initial interpretation…

DCPS Technical

Actually, they probably mean an elephant….

DCPS concept

I want Grey skinned animalMust have a trunkMust have large earsMust have tusks I need to have an animal for my circus

“What do you need?”

DCPS Subject Matter Expert

In house technical experts more aware of DCPS context and can interpret requirements based on true “need” rather than stated “want”:

Programs get what they expected even if they didn’t quite know how to articulate that initially

Vendor driven projects


32District of Columbia Public Schools | 2015 CGCS CIO Conference

Vendor Business analysts,

trainers, etc

Vendor PM

Vendor Executive

Leadership

Vendor Technical

Team

DCPS Executive Leadership

DCPS SIS Program Owner

DCPS Subject Matter Experts

• Vendor does not have to live with long term consequences of decisions

• Accountable to their leadership directly

• Often not dedicated to one client• Different motivations/point of view

No dedicated DCPS resources resulted in over-reliance on vendor in previous IT projects

Single path for Accountability


33District of Columbia Public Schools | 2015 CGCS CIO Conference

DCPS Business

analysts, etc

DCPS PM

Vendor Executive

Leadership

Vendor PM and Implementation

Support

DCPS Executive Leadership

DCPS SIS Program Owner

DCPS Subject Matter Experts

• DCPS directed project manager and business analyst

• Worked directly with vendor implementation team

• Independent from vendor team• Dedicated to nothing other than

Aspen implementation• Vendor PM acted more as project

coordinator for vendor resources

Dedicated DCPS resources leading project meant that DCPS was in the drivers seat.

Implementation – People not Technology


First real computer: 386 PC • Got just before freshman year

of college• Even typing “dir” at dos

prompt was scary

Lesson:• Anything new, especially new

technology, is intimidating• Fear is the problem, not lack

of knowledge



Manage change not technologySuccess is all about managing fear:• Fear in your implementation team that they are going to fail• Fear in your user base that they won't be able to do their work• Fear in your vendor that the project won't go well and they will get blamed

But push through that and in the end it is almost always:

”It wasn’t as bad as I thought it would be”



“Fear of change” cycleWhy do people fear change? Because they are rational and rational thinking seeks to solve problems.



Encounter problem

Rational mind begins to determine solution to problem

Can I solve the problem? No need to fear

Know exactly how much to fear (bounded)

yes

no

Fear is bounded even if problem cannot be solved because you know how bad it is

“Fear of change” cycleWhat happens with the unknown? Rational mind looks for problems, when it can’t find any, imagination kicks in



Encounter unknown


But what if it’s a worse problem than that?

yes

no

Whether or not you can deal with the problem you’ve imagined, you can always imagine something worse, adding more fear each turn of the cycle so that it increases without bound until it takes on a life of its own

No data = no problem

Problem is imagined

Can I solve this problem?

Worse problem is imagined Fear increases

Break cycle with informationAdding information gives the rational mind something to chew on, cycle is broken and fear becomes bounded once more



Encounter unknown


But what if it’s a worse problem than that?

No data = no problem

Problem is imagined

Problem becomes specific

Project team gives information Fear increases

No need to fear

Know exactly how much to fear

yes

no

Can I solve this problem?

Familiar process and trusted advisorsLimited fear of change in rolling out system by limiting amount of change• “If STARS didn’t do it, Aspen wouldn’t either”

• Process was familiar so just had to train on new way to enter the same data• Initially addressed fear about how Aspen wouldn’t be worse before trying to sell

how it would be better

• Trainers assigned to “their” set of schools built relationship and trust• Biggest project expense was personnel providing direct school support (training,

helpdesk etc)• Provided human face to project:

hearing “I understand” is powerful as it counters one emotion (fear) with another (fellowship)



Reduce Fear - Focus on school support - Phase 112 out of 18 resources dedicated to direct school support



Role PeopleSIS implementation 1 PM 1 BA 1 SME School Support 9 trainer , 3 helpdeskReporting 1 DBA, 2 devsVendor 0.5 PM (2 x .25), 2 FTE, ….Existing SIS support 2 helpdesk , 1 trainer, 4 technical contractor

FY14 – 6 months (April to September 2014)Roughly two thirds of overall budget was allocated to personnel costs and most of that was for directly supporting school staff

Focus on school support – Phase 224 out of 32 resources dedicated to direct school support



Role PeopleSIS implementation 1 PM 1 BA 1 SME 1.5 devSchool Support 12 trainers , 8 helpdesk, 3 scheduling consultants, 1 mgrReporting 1 DBA, 2 devsVendor 0.5 PM (2 x .25), 2 FTE (4 x .25)

FY15 (1 Oct 2014 to 30 sept 2015) Again, almost two thirds of overall budget was allocated for personnel costs and 70% of that was for directly supporting school staff

School Staff InvolvementMade sure schools felt a part of the process• This reduced fear and increased buy-in• Schools took part in vendor demos as part of RFP process• Got feedback on system configuration from selected school staff before rolling out• Reached out to experts at the schools to help develop scheduling process

• Original plan was to have all schools start scheduling SY15-16 in Aspen from the beginning• Feedback from school schedulers resulted in changing that plan to just 5 pilot schools



+ =

Do what’s best for the project but force yourself to be in the shoes of your users/vendors/team members Don’t imagine yourself doing things, imagine them doing things• Think of a time when you did something for the first time rather than on something you

are now comfortable with• Compare actual reality vs your own initial intimidation

Be selfish but not self-centered



+ =

Summary


TakeawaysHuge range between complete failure and perfect success

What needs to be not done wrong• Project management• Technology choice

What needs to be done right• Manage failure not success (ie manage scope)• Ensure that technical expertise is available to district and project leadership is

accountable to district and not just vendor• Focus on people, view IT system implementations as change management

projects not technology projects



[email protected]



mailto:[email protected]

Original Leading Slides(Changed the order to get to the less DCPS specific slides more quickly on slideshare)


DCPS FactsLed by Chancellor Kaya Henderson since 201048000+ students (steady increase for the past 5 years) in 111 schools• 60 Elementary Schools• 11 Middle Schools• 18 Education Campuses (usually Preschool through 8th grade)• 15 High Schools• 7 other (Special Ed etc)

6800+ school staff (3600+ teachers)600+ central office staff



Project details


DC STARSCurrent SIS, called DC STARS, had been running since 2003. • Outdated technology (Java applet front end)• Difficult to customize• No longer supported by vendor



By 2013, DC STARS rigidity meant that data was fragmented among a patchwork of applications put in place to cover gaps in functionality

Made the decision early on to replace with new system rather than reinventing the wheel by developing custom solution from scratch

FlexibilityDCPS has an extremely high-caliber workforce and entrepreneurial culture• System features must keep pace with velocity of reform• System must be customizable by DCPS staff

StabilityTimeline was very short for initial Phase 1 launch but SIS is critical to operations• System cannot lose functionality from what STARS provides for schools• System vendor must have history of successful deployments to K12 districts

Replacement SIS Context



New SIS: Follett School Systems ASPEN



DC STARS

Follett ASPEN

Project TimelinePhased rollout• Phase 1 – Elementary Schools only (not Education Campuses)• Phase 2a – Schedule for upcoming year in Aspen with 5 pilot high schools • Phase 2b – All schools in Aspen for everything



Milestone Date

RFP released November 2013

Vendor demos January 2014

Vendor start April 2014

Go live for Elementary Schools only August 2014 (SY14-15)

Phase 2 scheduling pilot February 2015 (pre SY15-16)

Go live for all remaining schools July 2015 (SY15-16)

SY 2013 -2014 SY 2014-2015 SY 2015-2016

ES Design & Implementation

DC STARS for SY14-15 MS / HS Operations

MS / HS OperationsRollout

DC STARS Available for Read Only in HS MS

SY15-16 Scheduling Pilot in Aspen

High Level rollout schedule (Phased Approach)

SchoolStat: PARCC/eSIS

ES Rollout

SIS Vendor Selection

Oct Nov DecJan 2014 Feb Mar Apr May Jun Jul Aug Sep Oct Nov DecJan Feb Mar Apr May Jun Jul Aug Sep

Vendor Starts

DC STARS For ES Operations

DC STARS Available for Read Only in ES

Jan Feb Mar

ES Operations in Aspen →

MS/HS Operations in Aspen →

MS / HS Design & Implementation

MS / HS Scheduling PilotPro

ject

Rol

lout

Pha

se

and

Mile

ston

es

Out

com

e fo

r S

choo

ls &

C

entr

al

ODS/Vendor Work Streams Activities Before Vendor Selected

Key DC STARS Usage by Schools

Aspen Usage by Schools DC STARS Data Availability

SY15-16 Scheduling begins

SY15-16 Begins

PM & BA Start

55

Project Management and Technology don't matter

Software

project management doesnt

cgcs cio conference

kind of project management

good project management

base technology

elementary schools

remaining schools

sis features