How Cognitive Systems like ignio are simplifying Batch Jobs Management Tom Reuner, Research VP, HfS Research [email protected]@tom_reuner @hfsresearch Web: www.hfsresearch.com | Blog: www.horsesforsources.com Webinar in collaboration with TCS November 3 rd , 2016
35
Embed
HfS Webinar Slides: How Cognitive Systems like ignio™ Simplify Batch Jobs Management
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Automation is a journey: Automation is not a quick fix; it is a journey. It takes preparation to find the right candidates and can be done effectively only by taking support from the people who are involved in the business or IT operations
War for talent: IA strategies require a unique talent set with the right mix of technical knowledge and business acumen. Scarcity of this talent is currently the biggest factor limiting the speed of execution
Finding a common language: As IA is not defined, stakeholders are struggling with blurred perceptions in the marketplace. Many tools and approaches use the automation moniker. Many stakeholders fail to understand the nuanced differences
Data curation is critical: Applying Cognitive and machine learning solutions to IA requires access to large amounts of relevant data to build reliable models
Crossing the chasm: A major challenge is to convince and align client stakeholders. In the words of one executive: “People don’t believe, people don’t trust. A lot of people are talking about automation, but few really understand it.”
Look beyond task automation: The marketing noise is largely around RPA and implicitly notions of task automation. Therefore, it can be challenging to get a sense of the bigger picture
It Is A Nascent Market But There Are Broad Lessons Already
Changing environmentü Every day changing jobs and dependenciesü Changing compute and storage infrastructure allocation
Diversityü Different applications, business units, and business processesü Different schedulers – Autosys, ControlM, TWS, Opconü Different platforms – mainframes and distributed systemsü Different environments - prod, non-prod, dev, QA, UAT
Batch Processing | Key Obstacles
Confidential 20 A Tata Consultancy Services Venture
Batch Processing | Problem Areas
Intelligent Command Center
ProactiveResilience
Agile Transformation
Improve transparency and eliminate noise
Generate proactive notifications to predict and prevent
What-If and If-what analysis
Confidential 21 A Tata Consultancy Services Venture
Case Study: How Customer Uses ignio for Batch Job Management
Context• Proactive management of batch jobs of a leading bank in the UK
Scope and Scale• 52 business units• 2500 business processes• ~23,000 batch jobs per day• ~100,000 job-job dependencies• 2 batch schedulers – OpCon and ControlM
Confidential 22 A Tata Consultancy Services Venture
Blueprint Construction
360-degree view: Graph model relating business units, to business processes to batch jobs
Nodes represent business units, business processes (streams), and jobs
Nodes and edges are associated with static and dynamic attributes (e.g., job start time, run time, end time, …)
Edges represent precedence and containment relationships
Entities, relationships and attributes are mined from batch schedulers, batch run logs, job definitions, SLA definitions, &
other data sources
Confidential 23 A Tata Consultancy Services Venture
Normal Behavior Characterization
Profile vitals and issues• Changes• Trends• Outliers• Temporal patterns
Profile dependencies• Influencers• Influencees• Cuts across technologies, and business
units
Confidential 24 A Tata Consultancy Services Venture
Suppress false alerts• Dynamic thresholds for run
time, start time and end times
Smart Triggers
Confidential 25 A Tata Consultancy Services Venture
• Computes probability of failure by analyzing past failures and SLA violations
• Computes impact by analyzing dependencies
• Reports jobs with a high failure risk
Assess and Manage Risks
Confidential 26 A Tata Consultancy Services Venture
Predict a Future Batch
• Status of scheduled jobs: running, delayed, failed jobs
• Inter-stream and inter-BU dependencies
• Anomalies and SLA violations• Critical paths and critical jobs
Confidential 27 A Tata Consultancy Services Venture
Generate Proactive Notifications
• Derive historical trends and patterns to predict future behavior
• Predict likely SLA violations • Predict time-to-saturation of
resources
Confidential 28 A Tata Consultancy Services Venture
• Predict execution behavior of jobs and business processes• Predict potential SLA violations• Identify critical jobs and paths to act upon to prevent SLA violations
What-If AnalysisDerive the impact of change• Business change
• Change in workload• Operations change
• Addition/Deletion of jobs and dependencies
• Change in schedule• Change in runtime
• Infrastructure change• Change in provisioned CPU/MIPS• Change in number of worker processes
Confidential 29 A Tata Consultancy Services Venture
If-What Analysis
Derive the plan for optimizing • Batch execution time• SLA adherence• Number of required
CPUs/MIPS• Peak MIPS usage
Confidential 30 A Tata Consultancy Services Venture
About HfS ResearchHfS Research is The Services Research Company™—the leading analyst authority and global community for business operations and IT services. The firm helps enterprises validate their global operating models with world-class research and peer networking.
HfS Research coined the term The As-a-Service Economy to illustrate the challenges and opportunities facing enterprises to re-architect their operations and thrive in this era where emerging disruptive competitors are using digital platforms and cognitive computing that can wipe out traditional enterprises overnight. HfS’ OneOfficeTM Paradigm is centered on creating the digital customer experience and an intelligent, single office to enable and support it. HfS’ vision is about helping clients achieve an integrated support operation has the digital prowess to enable its enterprise to meet customer demand - as and when that demand happens.
With specific practice areas focused on the Digitization of business processes and Design Thinking, Intelligent Automation and Outsourcing, HfSanalysts apply industry knowledge in healthcare, life sciences, retail, manufacturing, energy, utilities, telecommunications and financial services to form a real viewpoint of the future of business operations.
HfS facilitates a thriving and dynamic global community which contributes to its research and stages HfS holds several OneOfficeTM Summits each year, bringing together senior service buyers, advisors, providers and technology suppliers in an intimate forum to develop collective recommendations for the industry and add depth to the firm’s research publications and analyst offerings.
Now in its tenth year of publication, HfS Research’s acclaimed blog Horses for Sources is the most widely read and trusted destination for unfettered collective insight, research and open debate about sourcing industry issues and developments.
HfS was named Analyst Firm of the Year for 2016, alongside Gartner and Forrester, by leading analyst observer InfluencerRelations.