Agile Data Rationalization for Operational Intelligence

The Briefing Room

Twitter Tag: #briefr

The Briefing Room

Welcome

Host: Eric Kavanagh

[email protected]


The Briefing Room

!   Reveal the essential characteristics of enterprise software, good and bad

!   Provide a forum for detailed analysis of today’s innovative technologies

!   Give vendors a chance to explain their product to savvy analysts

!   Allow audience members to pose serious questions... and get answers!

Mission


The Briefing Room

JANUARY: Big Data

February: Analytics

March: Open Source

April: Intelligence


The Briefing Room

Geoffrey Malafsky

Dr. Geoffrey Malafsky earned a Ph.D. in Nanotechnology from Pennsylvania State University. He was a research scientist at the Naval Research Laboratory before becoming a technology consultant in advanced system capabilities for numerous Government agencies and corporate clients. He has over thirty years of experience and is an expert in multiple fields including Nanotechnology, Knowledge Discovery and Dissemination, and Information Engineering. He founded and operated the technology consulting company TECHi2 prior to founding Phasic Systems Inc., where he is the CEO and CTO.

Agile Data Rationalization for Operational Intelligence

Dr. Geoffrey Malafsky Phasic Systems Inc www.phasicsystemsinc.com 703-945-1378

Operational Intelligence and Data Rationalization • Operational Intelligence uses real-time data collected from

operating environments feeding analytical algorithms to detect and predict problems and efficiency opportunities

•  It relies on and is vulnerable to: ▫  Data accuracy ▫  Data completeness

•  Big Data is really 2 types: ▫  Lots of data used for statistical analysis – quality is not critical ▫  Lots of data used for deterministic analysis – quality is critical

and high volume is limiting (CPU, storage, power) • Garbage in à garbage out; Big Garbage in à Galaxy Class

misinformation

2

Enabling Data Success •  Overcome typical obstacles that prevented success in the past: ▫  Organizational group rivalry , Terminology confusion , Poor knowledge sharing ,

Inflexible designs •  Rapidly build and manage data portfolio models that provides

visibility on strategy, stakeholders, designs, systems with dependencies, linkages & analysis to operational data and metadata

•  Fill the gap in identifying, understanding and practically implementing actual operational data versions with evolving standards and consolidation

•  Distinguish, design, and implement similar, supposedly similar, and operationally distinct data

•  Complement existing systems

3

Design Rationalization Issues

•  Multiple data models •  Conflicting definitions •  Similar, supposedly similar,

operationally distinct values •  Unknown business logic •  Multiple ETL mappings

System Rationalization Issues

•  Multiple database systems •  Conflicting formats •  Redundant storage •  Unsynchronized values •  Multiple integration points •  System performance

5

NKY HomeSeekers Texas

Different Meanings (Legal and Business Activities)

•  data values not metadata rule operations for application support, reporting, and decision making •  data values are out-of-synch with all forms of metadata •  data values conflict across data stores, organizational groups, and applications: syntactically

(simplest case) and semantically (most difficult) •  top-down/bottom-up approaches have failed almost universally because they rely on metadata

and silo-ed organizational groups to solve what is inherently interrelated, complex •  enterprise business goals are being hindered because of the poor data environment •  there is little impetus to correct this situation

6

Ψ-KORS Methodology: Data Rationalization and Portfolio Management •  Integrated Organization,

Process, Technology •  Synchronize metadata and

operational data •  Allow valid, multiple distinct

versions of data entities •  Cycle time in days/weeks •  Correlated products

The Ψ–KORS™ System Model 7

Point-select data models, codes, entities

Data Rationalization Design Rationalization

•  Consolidated, adaptive data models •  Standardized definitions •  Synchronized distinct operational values •  Managed business logic •  Coordinated ETL mappings

System Rationalization •  Consolidated, adaptive systems •  Common, interoperable formats •  Common storage •  Synchronized interfaces •  Coordinated integration •  Greater system performance

DataStar Discovery

DataStar Unifier

9

Position Data Model

Corporate NoSQL™


The Briefing Room

Analyst: Eric Kavanagh

Perceptions & Questions


The Briefing Room

The Information Oriented Architecture (IOA)


The Briefing Room

Are We In the Data Tower of Babel?


The Briefing Room

God came down to see what they did and said: "They are one people and have one language, and nothing will be withheld from them which they purpose to do." "Come, let us go down and confound their speech." And so God scattered them upon the face of the Earth, and confused their languages, so that they would not be able to return to each other, and they left off building the city, which was called Babel "because God there confounded the language of all the Earth".[3]

Replace ‘God’ with ‘Innovation’ and…


The Briefing Room

Modes of Transportation: I


The Briefing Room

Modes of Transportation: II


The Briefing Room

Modes of Transportation: III


The Briefing Room

Modes of Transportation: IV


The Briefing Room

!  Open-Source innovations are opening up whole new ways of capturing, storing and processing data; and many solutions are free, though you’ll need trained developers to use the free stuff

!  Because the storage game has changed so much with Hadoop, you can now store massive amounts of granular detail, relatively cheaply

!  Big Data represents a huge opportunity, but also a serious challenge for the business & IT

The New Reality: I


The Briefing Room

! NoSQL Database technologies change the game due to greatly increased speed, among other characteristics

!  Other innovations, including Massive Parallel Processing, Multi-Core Processors and In-Memory capabilities are also significant change agents

!  This opens the door to a new kind of information architecture, with even real-time capabilities

The New Reality: II


The Briefing Room

!  The cost of software is in precipitous decline, as evidenced by any number of metrics

!  In 2005, Microsoft quoted me $7,500 to host a one-hour Webcast

!  In 2007, several vendors were offering pricing in the $1,500-per-Webcast space

!  We now pay less than $500 per month for unlimited Webcasts with WebEx

The New Reality: III

The Bloor Group

!  What is the NoSQL engine you’re using?

!  Could this replace both operational and analytical Master Data Management solutions?

!  Is there any way to dynamically reconcile data models? Or must you manually do this?

!  How do you deal with very old, “black box” legacy systems?

!  Where would this sit in an information architecture?

The Bloor Group

!  How do you deal with the User Adoption issue?

!  What would a small, foothold-style engagement look like? What’s the low-hanging fruit?

!  You have a fascinating case study involving the Navy and Human Resources Data. Can you describe?

!  Some consultants, like Michael Haisten in the 1990s referred to an Enterprise Back Plane for data. That was very similar to what’s now called Data Virtualization. Do you see a comparison?

The Bloor Group

Mariah, tacked up and ready to sleigh! photo by pmarkham on Flickr Mangapps Railway Museum - 2009 photo by Peter Taylor31 xLamborghini Countach, Diablo SV and Murciélago photo by exfordy on Flickr NASA SR-71B trainer after taking on fuel photo by jamesdale10 on Flickr


The Briefing Room


The Briefing Room

Thank You for Your

Attention

Agile Data Rationalization for Operational Intelligence

Technology

distinct data

data success

data stores

critical lots of data

data portfolio models

briefrthe briefing room

multiple fields

big garbage