This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Thank you for joining us, the webinar will start at: 10:00
Pacific / 12:00 Central / 13:00 East / 18:00 UK Time Manage
Databases like Codebases
Before we start You will be on mute for the duration of the
event We are now talking so please type a message in the Questions
box in the Control Panel if you cant hear us (please check your
speakers and GoToWebinar audio settings first) There will be a Q+A
session at the end but please feel free to type your questions in
the Questions box in the Control Panel in advance A recording of
the full webinar will be put up online
Presenters Tim O'Brien Analyst, CTO Gleanster Research Yaniv
Yehuda @yanivyehuda Co-Founder & CTO at DBmaestro Presenter at
world-wide conferences: ODTUG, ilOUG Kyle Hailey @kylehhailey
Technical Evangelist at Delphix Oracle ACE, member of the OakTable
Network
About Delphix Founded in 2008, launched in 2010 CEO Jedidiah
Yueh (founder of Avamar: >$1B revenue)) Based in Silicon Valley,
Global Operations 10% of Fortune 500
About DBmaestro Founded in 2008, launched in 2010 Founded by
Yariv Tabac and Yaniv Yehuda Headquartered in Israel, Global
Operations
Manage Databases like Codebases Kyle Hailey & Yaniv
Yehuda
The Business Need Copyright@2010, Gartner RAS Core Research 80%
of unplanned downtime is due to Change 50% More than of this, half
is due to human errors 40% of changes FAIL Copyright@2008, Juniper
Networks, Inc.
Code, Databases, and Agility in 2014 May20,2014 Tim OBrien
Analyst, CTO Gleanster Research
Industry Trends Code in 2014 Distributed& AsynchronousNew
Normal Self-serviceCriticaltoProductivity IncreasingVariety
(fromJavascript,Java, .NET, Ruby,Python+everything) Everything is
Automated
Industry Trends Databases in 2014 More thanjustRelational
IncreasingVariety (Oracle,Mongo) Management? Notas Matureas Code
Oddity Big Datamarginallymore manageablethan traditionalRDBMS
Werestillwaitingforthedatabase
Code is King Modern Enterprise Fully distributed development
Continuous Deployment Pipelines Everything Automated! - DevOps,
PaaS, IaaS Every Company is a Software Company How did we get
here?
Scale Development Open Source @ Scale Google Chrome, Mozilla
Firefox Android, Apache httpd Linux*, FreeBSD Characteristics Rapid
Iterations, Frequent Releases Numerous Experiments (or Branches)
Thousands of developers OSS @ Scale established best practices now
in use.
@Scale Everything is Continuous Continuous integration and
deployment Modern set ofTools Integrated with: Source Control
Perforce, Clearcase, Git Infrastructure-as-a-Service Openstack
Platform-as-a-Service AWS, Heroku Databases? - Still an Integration
Nightmare
Case in Point Example Fortune 500 Relaunch Re-architecture:
More distributed, Move to a PaaS Scope: Everything, 2Year Migration
Project Size: 1500 Developers, Several Continents, $100M+ Results?
Code: Distributed Projects, Continuous Deployment Infrastructure:
Self-service, On-demand, Agile Database: Delays, Manual Builds,
Cost Overruns
RootCause Analysis Code is Agile Developers move FAST. Branches
are fungible, disposable, personal. Deployments managed with DevOps
tools (immediately) Traditional Databases are Not Costly to Setup
Often require Dedicated Hardware Little automation. No Change
Management The rest of the department is Virtual but our Databases
are stuck in 1998.
Worst Practices (wereall used to) Lack of Database Change
Management: Shared Developer Databases Results in Contention
Costly, Slow Environment Provisioning Testing and Staging Not
Synchronized with Production Drag on Productivity and Efficiency
Opportunity lost Weve accepted that the databases causes
delays
What Weve Seen 1. QA: Expensive, Slow 2. Development:
Bottlenecks & bugs 3. Provisioning: Delays
1. QA: Expensive, Slow : Long Build times Build Time 96% of QA
time was building environment $.04/$1.00 actual testing vs. setup
Build Build Time QA Test QA Test Build
1. QA: Expensive, Slow: bugs found late Build QA Env QA Build
QA Env QA Sprint 1 Sprint 2 Sprint 3 Bug CodeX 0 10 20 30 40 50 60
70 1 2 3 4 5 6 7 Delay in Fixing the bug Cost To Correct Software
Engineering Economics Barry Boehm (1981) Build QA Env QA
2. Development: Bottlenecks & bugs: subsets cause bugs
2. Development: Bottlenecks & bugs: subsets cause bugs
3. Provisioning Delays : weeks to months Management DBA System
Admin Storage Admin Developers Submit Request Disk Capacity?
Approve Request $$ (2 Weeks) Approve Request $$ (1 Week) Request
Additional Storage? Provision Capacity File System Configured?
Configure LUNS & Build File System Coordinate Replication w/
Infrastructure Re- Parameterize & Configure DB Mount Recovery
DB to Specific PIT Begin Work Approve Request $$ (2 Weeks) (3 Days)
(3 Days) (2 Days) (3 Days) (3 Days) .1-2 Weeks of Approvals,
Delays, and Provisioning 24 Developer Asks for DB Get Access
Manager approves DBA Request system Setup DB System Admin Request
storage Setup machine Storage Admin Allocate storage (take
snapshot)
3. Provisioning Delays : culture of no
What Weve Seen 1. QA: Higher costs more code re-work 2.
Development: Bottlenecks & bugs 3. Provisioning: Delays
Three Physical Copies Three Virtual Copies
Install Delphix on Commodity Intel Hardware Intel hardware
Allocate Any Storage to Delphix Allocate Storage Any type
One time backup of source database Database Production Instance
File systemFile system
DxFS (Delphix) Compress Data Database Production Instance Data
is compressed typically 1/3 size File system
Incremental forever change collection Database Production
Instance File system Changes Collected incrementally forever Old
data purged Time Window File system
Typical Architecture Production Instance Development Instance
QA Instance UAT Instance Database File systemFile system Database
File systemFile system Database File systemFile system Database
File systemFile system
With Delphix Database File system Production Instance Database
Development Instance Database QA Instance Database UAT
Instance
What Weve Seen With Delphix 1. QA: Low cost, fast bug detection
2. Development: Parallelized, less bugs 3. Provisioning: Fast,
Culture of Yes
1. QA Efficient : Lower cost B u i l d T i m e QA Test 1% of QA
time was building environment $.99/$1.00 actual testing vs. setup
Build Time QA Test Build
1. QA Fast : bugs found fast and fixed Sprint 1 Sprint 2 Sprint
3 Bug CodeX QA QA Build QA Env Q A Build QA Env Q A Sprint 1 Sprint
2 Sprint 3 Bug Cod e X
2. Development : Parallelize gif by Steve Karam
2. Development: Eliminate bugs
3. Provisioning: Fast, Efficient, Self Service, Culture of
Yes!
What Weve Seen With Delphix 1. QA: Low cost, fast bug detection
2. Development: Parallelized, less bugs 3. Provisioning: Fast,
Culture of Yes But How do we manage database changes as code
changes and automate deployment of changes?
Dealing with Risk Smaller and more focused changes are easier
to manage (Agile) Automation of repeating tasks lowers risk of
(human) error Development and Operations should work in synergy
(DevOps)
45 More rapid changes Reacting quickly to market needs Getting
ahead of competition Fewer changes backed out Better collaboration
Fewer defects Ultimately better service Happy customers
Profitability Why Continuous Delivery?
46 Team and process Version everything Automate your tests Fix
it, properly (no out of process changes!) Automate your deployments
Database Continuous Delivery?
47 A quick poll
48 The database holds your essential information Changes can
impact the entire system Need to be synchronized with other changes
Often overlooked Database is a Key Component
49 There is more to a database than SQL scripts Schema
structure Code Content and meta-content Internal dependencies
Ensure that changes are not made without authorization Ensure no
out-of-process changes Reaching Inside the Database
50 Old adage but true The database is often neglected and
therefore can become the weakest link Essential from a compliance
point of view Should be the strongest link The Weakest Link In a
Chain
51 Challenges Un-coordinated Process Silos in development
Deployment delays Out-of-Process Changes Errors in production Lack
of confidence in automation Code overrides Different methodologies
Lack of history Missing changes wrong versions
52 Two isolated Processes Version Control Process Development
Process Check-Out Script Modify Script Get updated Script from DB
Check-In Script Compile Script in DB Debug Script in DB ? ? ? ? A
A
53 X 1.11.1.11.11.21.31.41.51.61.7 Build Once Deploy Many Int
QA Stage Prod Database Deploy Script Environment Re-Base (due to
defects) Dev Dev Dev Model 1.1 1.2 1.2 1.3 1.3 1.4 1.4 1.5 1.5 1.6
1.6 1.7 1.11.11.41.7 1.1 1.2 1.2 1.3 1.3 1.4 1.4 1.5 1.5 1.6 1.6
1.7 1.1 1.2 1.2 1.3 1.3 1.4 1.4 1.5 1.5 1.6 1.6 1.7 Out of Process
Change X X X X X ? 1.1.1 X
54 Dealing with challenges Coordinated ProcessTraceability
Start in the Beginning No Out-of-Process Changes Impact Analysis
Automation Task Based Development Well Defined Processes
What is DBmaestro TeamWork Database Enforced Change Management
+ Database version control + Plugs into the ALM (change request,
tickets & work items) + Database merge & change impact
analysis DevOps Solution for databases + Baseline aware deployment
automation, rollback & recovery + Plugs into release management
& Continuous Delivery
How? Database version control Enforced Check Out/In Labels
Rollback/Undo Audit trail reports Database impact analysis Utilizes
version control repository information 3-way analysis Database
deployment automation API Baselines Conflict resolution Customized
business logic
57 Version Control - One Enforced Process
58 1.11.21.31.41.51.61.7 Build & Deploy On Demand * Int QA
Stage Prod Database Deploy Script Environment* Execute the same
script being executed at the Stage environment Re-Base (due to
defects) Dev Dev Dev Model 1.1 1.2 1.2 1.3 1.3 1.4 1.4 1.5 1.5 1.6
1.6 1.7 1.1 1.4 1.4 1.7 1.1.1 1.7 1.1 1.1 1.11.41.7 File Based
Version Control Out of Process Change 1.1.11.7 1.1.11.7
Safety Net For Deployment Automation Source vs. Target Action =
No Action ? Source vs. Baseline Target vs. Baseline Action = = No
Action = Deploy Changes = Protect Target Merge Changes You do not
have all of the information With Baselines and 3 way analysis the
unknown is now known Simple Compare & Sync Baseline aware
Deployment An index exists in Target (Production) but not in source
(QA). What should we do? Drop the index or not?
Benefits - Development Database change repository Follow SCM
best practices (Check-Out/Check-In) All changes are documented
Manage who can do what, where, when & why
Dev2 Dev1Merge to dev1 Trunk DB VC Fork Fork Fork Fork
Safe and Efficient Work Process Clone relevant number of
virtual copies of the Trunk Dev Team 1-n, Developer a-z, Hot fix
environment, Etc Work in parallel, save time Rely on enforced
change process Know exactly who did what, when and why Deployment
automation Save time, mitigate risk Continuous integration,
Continuous Delivery Deal with conflicts & merge them into the
Trunk Test before deploy to production Pre-prod clone