Delivering World Class Software

Delivering World Class Software

Jeffrey MurrayTest Manager PowerPoint Microsoft Silicon Valley

Goals for todayThe parts of the Office Product Cycle

Measurements and quality

Who Microsoft hires

Stories

Microsoft Silicon Valley • Over 1800 employees, plus 400 vendors/contractors. Approx. 450 employees in San Francisco.

Jeffrey Murray … What’s my job?

Graduated from SUNY Albany

Write code

Manage people

Manage products

The Office Product CycleAbout every 3 years a major release of Office comes out with features aimed at increasing productivity and ease of use for our customers.

What is Office 2010?

Word Excel Outlook PowerPoint

SharePoint

Publisher Access OneNote

Groove InfoPath VisioServer

and Tools

OnlineOnline

Online

Online

How many people does it take to ship core Office?

800 Software Design Engineers (Developers)800 Software Test Engineers (Testers)400 Program Managers (Feature designers)200 Localizers, Lab Managers, etc.200 Planners, Recruiters, Sales and Marketing2400 Total

Project Management

Schedule• How much

time do we have?

Resources

• Who are they and how many?

Features

• What are you going to do and how risky is it?

Plan new features1. What is the vision for the

product?2. Make lists of features you

want to do, listen to customers, and see what is possible

3. Estimate and prioritize what is you can do in the time you have

4. Triage these until you can fit the schedule

5. Then write page 1 specs and go to step 3

New ribbonBetter graphicsAnimation painterOn line editingProjector setupSave to videoNew animation timelineSingle Document interfaceSlideShow broadcastCo-authoringNew transitionsBetter animationsSectionsNative video supportSplit videoCamera integrationHardware supportODF support

Code the features!

• Feature team makes the decisions• Must fit into allowed development time• Must be fully resourced• Responsible for getting it done

• Management will approve features via • Adds/Cuts• Product priorities and opportunities• Manage risk• 8 questions

1 Develop

er1 Tester

1 Program Manager

Feature team

Feature

crew

Feature crew

Feature crew

Feature crew

Feature

crew

Feature

crew Feature

crew

Feature

crew

Feature

crew

Feature

crew

Feature

crew

Feature

crew

Feature

crew

Feature crew

Feature crew

Feature crewFeature

crew

Feature crew

Feature crew

Feature crew

Feature

crew

Feature crew

Feature crew

Feature crew

Feature

crew

Feature

crew Feature

crew

Feature

crew

Feature

crew

Feature

crew

Feature

crew

Feature

crew

Feature

crew

Feature crew

Feature crew

Feature crewFeature

crew

Feature crew

Feature crew

Feature crew

Feature

crew

Feature crew

Feature crew

Feature crew

Feature

crew

Feature

crew Feature

crew

Feature

crew

Feature

crew

Feature

crew

Feature

crew

Feature

crew

Feature

crew

Feature crew

Feature crew

Feature crewFeature

crew

Feature crew

Feature crew

Feature crew

12 monthsPlanned testing phase, validation, user

scenarios, international, stress, security, configuration, accessibility, compatibility

etc.

Typical 28 to 32 month schedule

Plan

Code

Test

Beta

RTM

Planning Phase 4-6 monthsCreate lists and 1 page specs

Development and test estimate and risk assessmentAdds/cuts

4 MonthsFeatures ready to go

Fix last remaining important bugs

8 months: Design and implement

Unit test and validate8 questions

4 Months Beta 1 about 10,000 usersBeta 2 about 1,000,000 users

Important checkpoints

Plan

Code

Test

Beta

RTM

No coding without

dev/test/pm resource

Feature demo and

8 questions answered

Code complete: no changes without a

bug entered

Product must be internally

dogfoodable

Triage teams in

place

All metrics and goals must be

met

Metrics and Quality• Good planning is the key to good quality • “If you fail to plan you plan to fail”• Features added or changed late are always more buggy

and risky• Proper design, test, automation support produces better

code• Bug rates are good way to track quality all else

being equal• Customer feedback through Watson

Typical Office Product Bug Trend

What makes it work!• Checks and balances• Testing signs off on specs• Dev signs of on test plans• PM charged with overseeing progress of dev/test

What makes it work? (continued)• Constant and never ending improvement• Test involved earlier• Automation• Technical innovations • Auto code review• Automation validation before release to testers

• Listening to customers and competition

1 5 9 13 17 21 25 29 33 37 41 45 490

50,000100,000150,000200,000250,000300,000350,000400,000450,000

Example Watson Curve

Bucket number

Hits

Watson

We don’t have user steps or data

We know what line of code caused the crash and can often guard against it

Questions about the Office Cycle?

Microsoft wants you if?

Microsoft

Technical

Passion

Potential

Impact

Career Tips• Companies are better at identifying talent within you than

you are at bluffing your way through an interview. Make sure you are there for the right reasons and don’t hold back.

• Don’t plan your whole career all at once, you will miss out on interesting opportunities

• High tech companies need fresh idea, and that is a great open door for you

• You are a professional, act like it• When you screw up (and you will) what you do next is

critical• Ask yourself each week, what do I like about my job?• Interview the company beyond the job, a good part of your

life will be spent there.

Stories• How I got my Job at Microsoft• Copy protection• Steve’s laptop• OneNote• Office pranks• Elevator• Beach• Peanuts• Disco• balloons

www.microsoft.com/collegewww.viewmyworld.com