Product management with analytics

kpi’s and youproduct management with analytics

Who am I?Daryl Hemeon - @darylhemeonTech Architect for iServices Claims Self Service ProjectsWeb Dev for 15+ yearsI still code and when I do it scares the crap out of the CSS team

What are KPI’s?Key Performance Indicators are really Outcomes that we are interested in learning about to see if our experiments (and our products) are successful.

What do we monitor today?1. Employee Reg2. Logins3. Contact Center Calls4. Opt Out / In5. Online claims filed6. EFT Setup7. Docs Uploaded8. eSigned Auths9. Claims Filed total

10. Open Claims

How do we monitor them?1. Custom spreadsheet running macros to the

iServices database2. My personal dashboard that reads 2 different

datasources3. Product Owner(s) use magic and the phone

to get other stats

Are these the right stats?Vanity Metrics ● Logins per month (did the user accomplish their goal?)● Registrations per month (did the user then login?)

Learning Metrics - decision or actionable metrics● Messages Sent● Documents Uploaded - after a request for information

How should this work?1. Modern Toolset2. Inject into the planning process3. Inject into dev process4. Monitor post-release5. Product Owner now has the data to pivot priorities after

every sprint or iteration if they need to6. Outstanding question - Who does this work?

Modern ToolsetWe need to be using modern real-time analytics

the planning processBefore we created the map we should have voted the KPI’s or Outcomes and after the map creation adjusted them.

How does Agile help?

How does it inform UX design?

Heatmaps and click charts can inform HOW your application is actually being used and whether or not it is meeting objectives.

What is needed for us to get there?● Agile training for everyone - in progress - some people

already trained● Cost Effective Google Analytics training for analysts and

developers (free on-line, Skill Soft)● Continuous Integration for greater quality and delivery

with greater frequency - putting together a roadmap● Resources are need to monitor post release

effectiveness and then work with product owners to influence the backlog

What are we interested in for July?

● An agreement upon what stats will tell us if we are successful

● An agreement upon the goals for the first 3 months of usage

● An understanding of the long term business goals

References

● Lean UX by Jeff Gothelf

● Product Managers Guide to Analytics

● The Lean Startup by Eric Reis - discussion on Vanity Metrics

● The Marketer’s Guide to GA by the folks at Buffer

● Built to Last by Collins and Porras

● Training resources for GA

Questions?We like them.