Copyright © 2014 Oracle and/or its affiliates. All rights reserved. 1
Jul 28, 2015
Copyright © 2014 Oracle and/or its affiliates. All rights reserved.
Safe Harbor Statement The following is intended to outline our general product direcEon. It is intended for informaEon purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or funcEonality, and should not be relied upon in making purchasing decisions. The development, release, and Eming of any features or funcEonality described for Oracle’s products remains at the sole discreEon of Oracle.
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. Copyright © 2014 Oracle and/or its affiliates. All rights reserved.
Oracle Public Cloud Operations
Alexander Barrere Senior Systems Administrator, Global Cloud Operations April 13 2015
Copyright © 2014 Oracle and/or its affiliates. All rights reserved.
Topics ❏ Brief Overview of Oracle Cloud properEes ❏ Use and growth of ThousandEyes monitoring ❏ Deployment of Enterprise Agents infrastructure ❏ Global Enterprise Agent clusters ❏ Monitoring Cloud properEes at scale ❏ Future infrastructure endeavors
Copyright © 2014 Oracle and/or its affiliates. All rights reserved.
Oracle Service Cloud Properties
Platform as a Service (PaaS) Java, Database, Messaging, etc.
Software as a Service (SaaS)
Oracle CX Cloud, HR Cloud, etc.
Infrastructure as a Service (IaaS)
Elastic Compute, Storage, etc.
Copyright © 2014 Oracle and/or its affiliates. All rights reserved.
● Cloud Operations needed an alternative to our in-house monitoring tools ● Purchased public agents on trial basis, over-utilized them within 3 months ● Growth and use led to expand to an Enterprise Agent infrastructure solution ● Limitations to multiple agents on one host led to architecture of a cluster
environment ● Milestone to deploy to 16 Clusters worldwide by Q3 2015
The Growth and Need of Intelligent Monitoring
Copyright © 2014 Oracle and/or its affiliates. All rights reserved.
Deployment of Enterprise Agent Infrastructure ● Designed infrastructure to cater to multiple Oracle properties
○ agent overutilization for tests against sites ○ demand for monitoring increased across all of cloud
● Need for agents to simulate end-user traffic
○ Security restrictions deploying on a Public External IP space ○ Concerns for Change Management Solutions and potential
Hijacking ○ Having to use NAT to solve for IPv4 over-utilization
Prior to cluster deployment distribution of use stable utilization
Copyright © 2014 Oracle and/or its affiliates. All rights reserved.
Enterprise Network Infrastructure
Enterprise Hypervisor
ThousandEyes Network
Internet
Isolated Management Network
● Simulating ‘end-user’ experience
on the public internet ● Deployed on Oracle External
OOB network ● Only ILOM connections to
Management network ● iptables restricting connections
to egress only
Internet
Copyright © 2014 Oracle and/or its affiliates. All rights reserved.
Global Enterprise Agent Clusters ● Hypervisor: Sun servers running OEL6 virtualizing 25 guests ● 16 datacenter locations all deployed with 25 agents per site
○ 400 Enterprise Agents all running multiple tests against production sites ● Worldwide Network of agents to assure superior MTTR and high availability
and uptime for customers ● Monitoring of DNS public servers to ensure resolution times are within
adequate metrics
Copyright © 2014 Oracle and/or its affiliates. All rights reserved.
Enterprise Agent Clusters
● Designed infrastructure to support multiple Oracle properties ○ increased agent usage created high
demand for scaling ● 25 independent instances of the software
agent running
Copyright © 2014 Oracle and/or its affiliates. All rights reserved.
Monitoring Cloud Properties at Scale ● Discovering unwanted consistencies with reporting and trends
○ The deployment of the agent cluster revealed several patterns that were unexpected on a large scale
○ 24/7 monitoring of test sites to increase MTTR and Customer SLA's ● Finding unique problems with reports and trends
Copyright © 2014 Oracle and/or its affiliates. All rights reserved.
Future Infrastructure Features ● Integration of pool-based system when creating tests ● SAML authentication for Single Sign-on and Account Management ● Full IPv6 integration ● Utilizing API to automate test creation upon VIP creation ● Deploy separate management
infrastructure for internal monitors and deploying packages remotely