An Integral Open Source Software selection model with a business case on IT Infrastructure Monitoring System

An Integral Open Source Software selection model with a business case on

IT Infrastructure Monitoring System

By José Manuel López LujánMay, 2013

Docs: http://jmll.me/thesis-doc Slides: http://jmll.me/thesis-de-ppt

• Research Question

• Scope & Methodology

• Evaluation Models

• The Integral OSS Evaluation Model

• Case Study: IT infrastructure Monitoring software

• Conclusions

• Challenges

Agenda

2

“Would it be possible to identify the most important elements, Management and Technological, with the purpose of defining an Open Source Software selection model; and could this model be applied to select an IT Infrastructure Monitoring System?”

Research Question

3

• What is the need of having an enterprise-ready OSS selection model?

• What is the attractiveness of the OSS from an IT and Management perspective?

• Is there any current OSS evaluation or procurement models?

• Which criteria can be defined for OSS?

Research Question

4

• Methodology

• Literature

• The model

• The Case study

• OSS Evaluation model

• University of Toronto, I+TS Division, EIS, HIG.

• IT Infrastructure Monitoring System:

• Nagios, Zabbix and Cacti

Methodology & Scope

5

Proof by example

Evaluation ModelsName Year Source Corp/Org Method

Open Source Maturity Model(C-OSMM)

Open Business Readiness Rating (O-BRR)

Open Source Maturity Model(N-OSMM)

Qualification and Selection of Open Source Software (Q-SOSS)

Open Source Maturity Model (Q-OMM)

2003Duijnhouwer &

Widdows Cap Gemini Yes

2005Wasserman, Chan,

& Pal Open-BRR Yes

2005 Golden Navica Software Yes

2006 Atos-Origin Atos-Origin Yes

2009 QualiPSoWittmann & Nambakam, No

6

The IntegralOSS Evaluation Model

7

DefinitionThe evaluation Criteria

• Functionality

• License

• Community

• Seniority

• Performance

• Scalability

• Documentation

• Total Cost of Ownership

• Support

• Interoperability

• Security

• Roadmap

1

8

DefinitionThe evaluation Criteria

I-OSSEM O-BRR N-OSMM C-OSMM Q-SOSS Q-OMM

Functionality

License

Community

Seniority

Support

Interoperability

Security

Roadmap

Performance

Scalability

Documentation

TCO

Functionality - - Functionality Functionality

License - Licensing License/Copyright Licenses

Community/Adoption -

Developer and user Community Activity/Adoption Popularity

- - Age Maturity Number of commits

Support Support Support Support Quality test

- IntegrationCollaboration with

other products - Standards

Security - Security - -

Professionalism - Roadmap Roadmap Roadmap

Performance - Performance - -

Scalability - ModularityModularity/Code

modification -

Documentation Documentation Ease of deployment Documentation Documentation

- Training TrainingTraining/Consulting/

Support TCO/Training

1

9

Identification

RequirementsRequirementsHardware

Software

DocumentationDocumentationDocumentationDocumentation

Official

Non-Official

Relevant

Books

Support & CommunitySupport &

CommunitySupport &

CommunitySupport &

Community

Official

Non-Official

Issue tracker site

Relevant

DistributionDistributionDistribution

Source

Binaries

Platforms

GeneralGeneralGeneralGeneralGeneralGeneral

Name

Version

License

Type

Site

Language

+Services+Services+Services

Training

Support

Consulting

ArchitectureArchitectureModularity

Plugins

2

10

QualificationTotal Cost of Ownership3

11

QualificationTotal Cost of OwnershipStage Driver/Cost Initial Acquisition Year Year 2 Year 3

SelectionSelection

AcquisitionAcquisitionAcquisitionAcquisition

IntegrationIntegrationIntegration

UseUseUseUse

RetirementRetirement

Up-front evaluation study $- $- $-

Up-front proof of concept implementation $- $- $-

Software $- $- $-

Customization for business needs $- $- $-

Integration $- $- $-

Hardware $- $- $-

Migration (data and users) $- $- $-

Training $- $- $-

Process and Best practice Change $- $- $-

Cost of Support Services – in house $- $- $-

Cost of Support Services – contracted $- $- $-

Maintenance and upgrades $- $- $-

Training $- $- $-

Exit costs (in relation to hardware and software) $- $- $-

Exit costs (in relation to changeover, retraining) $- $- $-

Total Cost/Year $- $- $-

Discount factor 5%

TCO $-

3

12

ValuationCriterion Weight OSS1 % OSS2 % Perfect

Functionality

License

Community

Seniority

Support

Interoperability

Security

Roadmap

Performance

Scalability

Documentation

TCO

1 87% 78% 100%

2 100% 0% 100%

2 25% 75% 100%

2 71% 43% 100%

2 67% 67% 100%

1 80% 80% 100%

2 77% 100% 100%

1 83% 50% 100%

1 33% 33% 100%

2 25% 100% 100%

2 62% 74% 100%

1 90% 100% 100%

Final Score

Final Percentile

Final Score 12.28 12.59 19

Final Percentile 65% 66% 100%

4

13

Selection5

14

Case Study on ITInfrastructure Monitoring

• Information & Technology Services

• Enterprise Virtualization & Storage Specialist at EIS

• Hybrid: CSS and OSS

• Enterprise Monitoring at the Data Centre

• Data Centre Renovation

• Budget 5.1M CAD, ROI of two years

15

Case Study on ITInfrastructure Monitoring

• Virtual and Physical Servers

• Many devices need monitoring: PDU, UPS, sensors.

• Already implemented IT-IMS:

• Cacti

16

Questions

17

Monitoring

Measuring

“Does our current monitoring package meet all our needs? “

“Can the package be scaled up as the DC and its virtual infrastructure and systems grow? “

Controlling

Definition• Business Context

• Business Requirements

• Functional

• Non-Functional

• Technological

• The Evaluation Criteria

1

18

DefinitionWhat are the requirements for the IT

Infrastructure Monitoring System?

Technological

Support  SNMP  v1  and  v2c,  v3  Linux/Unix  pla:orms.  DB  backend  should  be  configurable  to  use  any  OS/CS  –  RDBMSHandle  64bit  valuesGather  SNMP  data  at  1min  intervalsCompiled  not  interpreted.

19

1

DefinitionWhat are the requirements for the IT

Infrastructure Monitoring System?

1

Non-Functional

User Management tools, Role based access control

Ownership of devices

Graphs should be created on demand, not in mass with every sample

Authentication framework, such as LDAP, Shibboleth

Data Importing/Exporting formats XML, CSV, XLS

Reconcile missing data and identify the network fault.

Keep at least 25 months of data at 1 minute intervals

Import/Export data and templates

Scale to thousand of devices

Handle more than 64K outstanding requests

Memory footprint should not increase with the number of devices being monitored

20

DefinitionWhat are the requirements for the IT

Infrastructure Monitoring System?

1

Functional

Network  fault  determinaQon  -­‐  Logic  hierarchyAuto  topology  creaQonInterface  Discovery  should  be  automaQc  for  switches  and  devicesTopographic  map  of  the  devices.Device  Auto  discovery  capabiliQes  Threshold  triggeredAlerQng  lists  and  scheduled  alerts  –  alert  scheduleMobile  alerQng.  SMS/Email  or  PushNoQficaQonsIndependent  probing  capabiliQesAgentless  non-­‐intrusive

21

1

Description

Cacti

0.8.8a

GNU General Public License

Monitoring System

http:///cacti.net

PHP

Network Access

Web Server (Apache), MySQL, PHP, RRDTool, net-snmp. Runs over Unix and Windows

http://docs.cacti.net/ ; http://www.cacti.net/downloads/docs/html/

http://blog.cactiusers.org/

Comprehensive Linux Install Guide by Lee Carter, Solaris Install Guide by Javier Vidal Postigo, German Install Guide by Sebastian Larisch.

Cacti 0.8 Beginner's Guide, Thomas Urban

http://forums.cacti.net; http://cacti.net/mailing_lists.php

http://blog.cactiusers.org/

http://bugs.cacti.net/

https://help.ubuntu.com/community/Cacti

svn checkout svn://svn.cacti.net/cacti; http://www.cacti.net/downloads/cacti-0.8.8a.tar.gz

Windows, Linux/Unix

PIA - Plugin Architecture

http://docs.cacti.net/plugins

http://gregsowell.com/?page_id=86; http://www.transitiv.co.uk/services/training/cacti; http://www.credativ.co.uk/services/training/monitoring/cacti/

http://www.transitiv.co.uk/services/consultancy/cacti

http://www.transitiv.co.uk/services/consultancy/cacti

IdentificationCacti

22

2

2IdentificationNagios

Sub-category Description

Name Nagios Core

Version 3.4.4

License GNU General Public License

Type Monitoring System

Site http://www.nagios.org/

Language C

Hardware Network Access

Software C Compiler, Web Server (Apache), GD Library and Unix/Linux as OS.

Official http://nagios.sourceforge.net/docs/3_0/toc.html

Non-Official http://exchange.nagios.org/directory/Documentation/Nagios-Core-Documentation

Relevant http://www.fullyautomatednagios.org/wordpress/documentation/

BooksNagios Core Administration Cookbook By: Tom Ryder; Nagios: Building Enterprise-Grade Monitoring Infrastructures for Systems and Networks, Second Edition By: David Josephsen

Official http://library.nagios.com/; http://support.nagios.com/forum/; http://support.nagios.com/wiki/index.php/Main_Page

Non-Official https://help.ubuntu.com/community/Nagios

Issue tracker site http://tracker.nagios.org/my_view_page.php

Relevant http://nagiosplugins.org/support

Source http://prdownloads.sourceforge.net/sourceforge/nagios/nagios-3.4.4.tar.gz

Platforms Linux/Unix

Modularity Plugin Architecture

Plugins http://www.nagios.org/download/plugins; http://nagiosplugins.org

Training http://www.nagios.com/services/training/

Support http://support.nagios.com/; http://support.nagios.com/wiki/index.php/Main_Page

Consulting http://www.nagios.org/support/servicepartners/

23

2IdentificationZabbix

Sub-category Description

Name Zabbix

Version 2.0.4

License GNU General Public License version 2

Type Monitoring System

Site http://www.zabbix.org/

Language C (server, proxy, agent), PHP (frontend), Java (Java gateway)

Hardware Network Access, 100MB Disk Space, 256M RAM, Pentium IV or equivalent

Software Apache Web Server, MySQL, PostgreSQL, SQLite, Oracle or IBM DB2

Official https://www.zabbix.com/wiki/doku.php; http://blog.zabbix.com/

Non-Official https://s3.amazonaws.com/analyticarts/zabbix/Zabbix2-0Manual.pdf

Relevant N/A

Books Zabbix 1.8 Network Monitoring By: Rihards Olups

Officialhttps://www.zabbix.com/forum/; https://support.zabbix.com/secure/Dashboard.jspa; https://lists.sourceforge.net/lists/listinfo/zabbix-announce; https://lists.sourceforge.net/lists/listinfo/zabbix-users

Non-Official N/A

Issue tracker site https://support.zabbix.com/browse/ZBX

Relevant N/A

Source http://sourceforge.net/projects/zabbix/files/ZABBIX%20Latest%20Stable/2.0.4/zabbix-2.0.4.tar.gz/download

Binaries http://www.zabbix.com/download.php

Platforms Cross Platform

Modularity Plugins

Training http://www.zabbix.com/business_solutions.php

Support http://www.zabbix.com/business_solutions.php

Consulting http://www.zabbix.com/business_solutions.php

24

Qualification3

DocumentationDocumentationDocumentationDocumentation

Cacti Nagios Zabbix

Overall

Technical

User

2.00 1.00 2.00

1.36 1.09 1.18

1.67 1.67 1.67

Score 1.68 1.25 1.62

RoadmapRoadmapRoadmapRoadmap

Cacti Nagios Zabbix

Roadmap

Project activity

2 0 2

3 3 3

Score 2.5 1.5 2.5

ScalabilityScalabilityScalabilityScalability

Cacti Nagios Zabbix

Overall

Linear Scalable

1.0 2.0 3.0

1.0 1.0 1.0

Score 1.0 1.5 2.0

SupportSupportSupportSupport

Cacti Nagios Zabbix

Self

Paid

Community

1 0 1

1 1 1

1 1 1

Score 3.0 2.0 3.0

FunctionalityFunctionalityFunctionalityFunctionality

Cacti Nagios Zabbix

Functional

Non-Functional

Technological

0.8 1.0 1.0

0.7 0.7 0.9

1.3 1.3 1.3

Score 0.9 1.0 1.1

SenioritySenioritySenioritySeniority

Cacti Nagios Zabbix

Lifespan 11 14 9

Score 3.0 3.0 2.0

25

3

QualificationSupport3

26

QualificationDocumentation3

27

QualificationPerformance3

28

QualificationCommunity

Data gathered from SourceForge.net, 2013

Cacti

Top 633

CR16.3

Nagios

Top 7,017

CR40.9

29

3

QualificationCommunity3

Community

OSS Package

CommunityCommunity

Type Score

Cacti

Nagios

Zabbix

ORG 1

COR 4

ORG 1

Data gathered from SourceForge.net, 2013

Zabbix

Top 109

CR8.16

30

QualificationTotal Cost of Ownership (Cacti)3

$ in Canadian Dollars (CAD)31

Stage Driver/Cost Initial Acquisition Year

Year 2 Year 3

SelectionSelection

AcquisitionAcquisitionAcquisitionAcquisition

IntegrationIntegrationIntegration

UseUseUseUse

RetirementRetirement

Up-front evaluation study $852.20 $- $-

Up-front proof of concept implementation $150.00 $- $-

Software $- $- $-

Customization for business needs $426.10 $510.00 $520.20

Integration $1,917.45 $510.00 $520.20

Hardware $1,210.00 $1,210.00 $1,210.00

Migration (data and users) $- $- $-

Training $213.05 $- $-

Process and Best practice Change $421.10 $1,086.53 $1,108.27

Cost of Support Services – in house $16,351.50 $16,678.53 $17,012.10

Cost of Support Services – contracted $- $- $-

Maintenance and upgrades $1,022.64 $5,786.09 $5,901.81

Training $1,278.30 $1,303.87 $1,329.94

Exit costs (in relation to hardware and software) $- $- $-

Exit costs (in relation to changeover, retraining) $- $- $-

Total Cost/Year $22,950.44 $26,175.29 $26,674.59

Discount factor 5%

TCO $72,073.92

QualificationTotal Cost of Ownership3

$73,144.17Zabbix

$74,422.47Nagios

$72,073.92Cacti

$ in Canadian Dollars (CAD)32

3

Valuation4

Criterion Weight Cacti % Nagios % Zabbix % Perfect %

Functionality

License

Community

Seniority

Support

Interoperability

Security

Roadmap

Performance

Scalability

Documentation

TCO

2 73% 78% 83% 100%

1 100% 100% 100% 100%

2 33% 100% 33% 100%

1 100% 100% 67% 100%

1 100% 67% 100% 100%

2 84% 89% 63% 100%

1 100% 100% 100% 100%

1 83% 50% 83% 100%

2 33% 33% 67% 100%

2 33% 50% 67% 100%

2 98% 73% 95% 100%

1 100% 33% 66% 100%

Final Score

Final Percentile

Final Score 12.77 12.99 13.15 18

Final Percentile 71% 72% 73% 100%

33

Valuation4

Cacti Nagios Zabbix

34

Selection• Insight into the deficiencies of Cacti

• Zabbix’ scalability and core features fulfill most of UofT instrumentation needs.

• A change from Cacti to Zabbix would substantially change the cost or efficacy of the Enterprise Monitoring Solution

• Zabbix implementation plus integration with the current Cacti instance, in order to work as a distributable monitoring system, and take advantage of the features of both systems.

5

35

Selection• Remarkable expertise of the HIG.

• TCO of a second package is dwarfed by the other operating costs of the Data Centre.

• Cacti scalability will improve

• Spine

• Isolated instances

5

36

• Reveals advantages and disadvantages of given OSS

• High level of detail supporting the decision.

• The best solution might come from joining forces by creating a hybrid system

• How to measure criteria and the flexibility to add or remove.

Conclussions

37

• Highlighting TCO hidden costs overlooked by Senior Management.

• Hidden costs like: man-hours invested in self-training, looking for support in the community or documentation.

• Shatters the illusion that OSS is free.

• All this to avoid mistakes that often incur in unexpected costs.

Conclussions

38

• Implementing the model for additional kinds of software.

• Implementing the model in the private sector. Downtime costs, maintenance costs, etc.

• Including Return On Investment (ROI)

• Will be proposed to the internal use of HIG and eventually to EIS.

Challenges

39

An Integral Open Source Software selection model with a business case on

IT Infrastructure Monitoring System

By José Manuel López LujánMay, 2013

Docs: http://jmll.me/thesis-doc Slides: http://jmll.me/thesis-de-ppt