Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 1
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 4
Oracle Database 12c
Patrick Wheeler Senior Director, Product Management Oracle Database
In association with
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 5
Major Database Focus Areas
ENGINEERED SYSTEMS
BIG DATA
SOCIAL
BLOG
SMART METER
101100101001001001101010101011100101010100100101
CLOUD COMPUTING
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 6
Private Database Cloud Architectures Oracle Database 11g
Dedicated Databases
share servers and OS
Virtual Machines
share servers
Schema Consolidation
share servers, OS and database
Increasing Consolidation
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 7
Private Database Cloud Architectures Oracle Database 12c
Dedicated Databases
share servers and OS
Virtual Machines
share servers
Multitenant Database
share servers, OS and database
Increasing Consolidation
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 8
Isolation and multitenancy
Fast provisioning and cloning
Secure and highly available
No application changes
Manage many as one
Greater resource utilization
Performant and scalable
Lower IT costs
Consolidating Databases on Clouds Key requirements…
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 9
David Storage Manager
Mike IT Manager
James CIO
Laura Dev. Manager
Amy Systems Manager
The Data Center Team
Mark Security Manager
Justin DBA
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 10
Consolidating Databases v
Challenge #1
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 11
Database Consolidation Challenges
Reduce the number of separate
databases
Reduce IT costs by consolidating more
servers
Keep database applications isolated
and unchanged
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 12
Oracle Database Architecture Requires memory, processes and database files
System Resources
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 13
New Multitenant Architecture Memory and processes required at multitenant container level only
System Resources
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 14
New Multitenant Architecture Memory and processes required at multitenant container level only
System Resources
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 15
Consolidating Databases Step 1: Upgrade databases in-place
Upgrade in Place
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 16
Consolidating Databases Step 2: Plug-in upgraded databases
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 18
OLTP benchmark comparison
Only 3GB of memory vs. 20GB memory used for 50 databases
Pluggable databases scaled to over 250 while separate database instances maxed at 50
Pluggable vs Separate Databases Highly Efficient: 6x Less H/W Resource, 5x more Scalable
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 19
Managing Shared Resources Resource management in multitenant environment
High Priority
Medium Priority
Low Priority
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 21
Manage Many as One with Multitenant Backup databases as one; recover at pluggable database level
One Backup
Point-in-time recovery At pluggable database level
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 22
Manage Many as One with Multitenant One standby database covers all pluggable databases
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 23
Multitenant for Simplified Upgrades Apply changes once, all pluggable databases updated
Upgrade in-place
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 24
Multitenant for Patching Flexible choice when patching & upgrading databases
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 25
GOLD
SILVER
BRONZE
RAC, Data Guard, Daily Incrementals
Data Guard, Daily Incrementals
Weekly Full Backups
PDB migrates through SLAs as it becomes more mission critical Unprecedented Agility with Pluggable Portability
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 26
Multitenant for Provisioning
PDBs can be cloned from remote CDBs
PDBs can be cloned from within the same CDB
Fast cloning of PDBs
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 27
Multitenant for Fast Provisioning Pluggable databases can be quickly provisioned from seed
0
5
10
15
20
25
Non CDB PDB Clone PDB usingCopy-on-Write File
System
Time Taken to Provision New Database
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 28
Advantages of Multitenant Architecture Reduced CapEx & OpEx, Increased Agility, Easy Adoption
Self-contained PDB for each application Applications run unchanged Rapid provisioning (via clones) Portability (via pluggability)
Shared memory and background processes More applications per server
Common operations performed at CDB level Manage many as one (upgrade, HA, backup) Granular control when appropriate
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 29
Multitenant for Test and Development Fast, flexible copy and snapshot of pluggable databases
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 30
Multitenant. Perfect for SaaS. Multitenancy implemented by the Database, not the Application
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 31
Isolation and multitenancy
Fast provisioning and cloning
Secure and highly available
No application changes
Manage many as one
Greater resource utilization
Performant and scalable
Lower IT costs
New Multitenant Architecture Ideal for consolidating databases on Clouds
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 33
Lowering Storage Costs v
Challenge #2
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 34
Managing Storage Challenges
Compress data, without impacting
performance
Manage more data without incurring additional cost
Tier and compress data based on usage
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 36
Automatic Data Optimization Add compression and tiering policies to tables
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Confidential – Oracle Restricted 36
Oldest Data Most Recent Data
Po licy 1
Po licy 2
Compress Partitions with Row Compression if they haven’t been modified in 30 days
Compress Partitions with Hybrid Columnar Compression if they haven’t been modified in 180 days
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 37
Automatic Data Optimization A heat map tracks the activity of segments and blocks
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Confidential – Oracle Restricted 37
Oldest Data Most Recent Data
Po licy 1
Po licy 2
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 38
Automatic Data Optimization Policies are automatically applied to tables
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Confidential – Oracle Restricted 38
Oldest Data Most Recent Data
Po licy 1
Po licy 2
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 39
Automatic Data Optimization Policies are automatically applied to tables
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Confidential – Oracle Restricted 39
Oldest Data Most Recent Data
Po licy 1
Po licy 2
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 40
Automatic Data Optimization Policies are automatically applied to tables
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Confidential – Oracle Restricted 40
Oldest Data Most Recent Data
Po licy 1
Po licy 2
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 41
Automatic Data Optimization Reduce storage footprint, read compressed data faster
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Confidential – Oracle Restricted 41
Oldest Data Most Recent Data
Po licy 1
Po licy 2
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 44
Oracle Database Security Solutions
Defense-in-Depth for Maximum Security
Activity Monitoring
Database Firewall
Auditing and Reporting
DETECTIVE
Redaction and Masking
Multi-Factor
Authorization
Encryption
PREVENTIVE ADMINISTRATIVE
Data Discovery
and Classification
Vulnerability Scanning
Database Lifecycle
Management
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 45
Redacting Data v
Challenge #3
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 46
Redacting Data Challenges
Redact data in applications,
queries and reports
Secure sensitive personal
information
Avoid changing applications,
queries and reports
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 47
Soc. Sec. # 115-69-3428
DOB 11/06/71
PIN 5623
Policy enforced redaction of sensitive data
Redacting Sensitive Data Mask Application Data Dynamically
Call Center
Operator
Payroll
Processing
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 49
Maximum Availability Architecture
Active Data Guard – Data Protection, DR
– Query Offload
GoldenGate – Active-active
– Heterogeneous
RMAN, Oracle Secure Backup – Backup to tape / cloud
Active Replica
Edition-based Redefinition, Online Redefinition, Data Guard, GoldenGate – Minimal downtime maintenance, upgrades, migrations
RAC – Scalability
– Server HA
Flashback – Human error
correction
Production Site Application Continuity – Application HA
Global Data Services – Service Failover / Load Balancing
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 51
Zero Data Loss Challenge
The longer the distance, the larger the performance impact
Trade-off between Zero Data Loss and Performance
Primary Standby
Commit
Commit Acknowledge
Network Send
Network Acknowledge
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 52
Data Guard Far Sync No Compromise Between Availability and Performance
Primary Standby
Far Sync Instance
(Active)
Far Sync Instance
(Idle)
Far Sync Instance
(Active)
Far Sync Instance
(Idle)
Primary Standby
sync
async
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 53
Data Guard Far Sync Standby between continents with zero data loss
Primary Standby
Far Sync Instance
(Active)
Far Sync Instance
(Idle)
New York Bridgeport
Reading London
sync
async
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 55
Unknown Transaction State
Database and infrastructure outages can cause in-flight work to be lost, leaving users and applications in-doubt
Often leads to User pains Duplicate submissions Rebooting mid-tiers Developer pains
Leads to poor customer experiences
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 56
Unknown Transaction State
Application Servers
Database Servers
End User
Current Situation
Network Switches
User selects product from application and purchases it from the web checkout
User transaction arrives at application infrastructure. It makes it’s way through the application tiers and results in a database transaction being created
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 57
Unknown Transaction State
Application Servers
Database Servers
End User
Current Situation
Network Switches
User transaction arrives at application infrastructure and makes it’s way through application tiers and results in a database transaction being created
The transaction commits inside of the database and an acknowledgement begins it return journey
A failure in the infrastructure on its return journey means the application server never receives an acknowledgment
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 58
Unknown Transaction State
Application Servers
Database Servers
End User
Current Situation
Network Switches
The application is left in an unknown state. It returns an error to the user who may order the product for a second time resulting in them potentially being charged twice.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 59
A Solution to the Problem
• Transaction Guard: a reliable protocol and API that returns the outcome of the last transaction
• Application Continuity: safely attempts to replay in-flight work following outages and planned operations.
Two New Features
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 60
Application Continuity
Application Servers
Database Servers
End User
A reliable replay of in flight work
Network Switches
User selects product from application and purchases it from the web checkout
User transaction arrives at application infrastructure. It makes it’s way through the application tiers and results in a database transaction being created
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 61
Application Continuity
Application Servers
Database Servers
End User
A reliable replay of in flight work
Network Switches The infrastructure hosting the database fails just before the transaction is committed to the database.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 62
Application Continuity
Application Servers
Database Servers
End User
A reliable replay of in flight work
Network Switches
The jdbc driver detects the failure and checks with an available node in the cluster, using “Transaction Guard”, whether the transaction committed or needs to be replayed
If the transaction needs to be replayed, “Application Continuity” will submit all of the inflight work to a surviving node in the cluster and perform a commit. This all happens transparently to the application
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 63
Application Continuity
Application Servers
Database Servers
End User
A reliable replay of in flight work
Network Switches
The user receives confirmation that his order has been successfully completed.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 64
Simplified Analysis of Big Data Pattern Matching
Select * from
Ticker MATCH_RECOGNIZE (
…
PATTERN(X+ Y+ W+ Z+)
DEFINE X AS (price < PREV(price)),
Y AS (price > PREV(price)),
W AS (price < PREV(price)),
Z AS (price > PREV(price)
…
)
Scalable discovery of business event sequences
– Clickstream logs: sessionization, search behaviour
– Financial transactions: fraud detection, double bottom (“W”) stock analysis
– Telco: dropped calls – Medical sensors: automated medical
observations and detections
Asc
endi
ng O
rder
“W” event defined as regular expression
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 65
package pigstuff;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Iterator;
import org.apache.pig.EvalFunc;
import org.apache.pig.PigException;
import org.apache.pig.backend.executionengine.ExecException;
import org.apache.pig.data.BagFactory;
import org.apache.pig.data.DataBag;
import org.apache.pig.data.DataType;
import org.apache.pig.data.Tuple;
import org.apache.pig.data.TupleFactory;
import org.apache.pig.impl.logicalLayer.FrontendException;
import org.apache.pig.impl.logicalLayer.schema.Schema;
/**
*
* @author nbayliss
*/
public class W_FINDER extends EvalFunc<Tuple> {
private class V0Line {
String state = null;
String[] attributes;
String prev = "";
String next = "";
public V0Line(String[] atts) {
attributes = atts;
}
public String[] getAttributes() {
return attributes;
}
public void setState(String state) {
this.state = state;
}
public String getQuantity() {
return this.attributes[4];
}
public String setState(V0Line linePrev, V0Line lineNext) {
String q = this.getQuantity();
if (linePrev == null) {
prev = "";
} else {
prev = linePrev.getQuantity();
}
if (lineNext == null) {
next = "";
} else {
next = lineNext.getQuantity();
}
if (!q.isEmpty() && (prev.isEmpty() || (eq(q, prev) && gt(q, next)))) {
state = "S";
return state;
}
if (gt(q, prev) && gt(q, next)) {
state = "T";
return state;
}
if (lt(q, prev) && lt(q, next)) {
state = "B";
return state;
}
if (!q.isEmpty() && (next.isEmpty() || (gt(q, prev) && eq(q, next)))) {
state = "E";
return state;
}
if (q.isEmpty() || eq(q, prev)) {
state = "F";
return state;
}
return state;
}
private boolean eq(String a, String b) {
if (a.isEmpty() || b.isEmpty()) {
return false;
}
return a.equals(b);
}
private boolean gt(String a, String b) {
if (a.isEmpty() || b.isEmpty()) {
return false;
}
return Double.parseDouble(a) > Double.parseDouble(b);
}
private boolean lt(String a, String b) {
if (a.isEmpty() || b.isEmpty()) {
return false;
}
return Double.parseDouble(a) < Double.parseDouble(b);
}
public String getState() {
return this.state;
}
}
BagFactory bagFactory = BagFactory.getInstance();
@Override
public Tuple exec(Tuple input) throws IOException {
long c = 0;
String line = "";
String pbkey = "";
V0Line nextLine;
V0Line thisLine;
V0Line processLine;
V0Line evalLine = null;
V0Line prevLine;
boolean noMoreValues = false;
String matchList = "";
ArrayList<V0Line> lineFifo = new ArrayList<V0Line>();
boolean finished = false;
DataBag output = bagFactory.newDefaultBag();
if (input == null) {
return null;
}
if (input.size() == 0) {
return null;
}
Object o = input.get(0);
if (o == null) {
return null;
}
//Object o = input.get(0);
if (!(o instanceof DataBag)) {
int errCode = 2114;
String msg = "Expected input to be DataBag, but"
+ " got " + o.getClass().getName();
throw new ExecException(msg, errCode, PigException.BUG);
}
DataBag bag = (DataBag) o;
Iterator<Tuple> valueIt = bag.iterator();
while (!finished) {
if (valueIt.hasNext()) {
Tuple nextTuple = valueIt.next();
line = nextTuple.get(0).toString();
String ordk = nextTuple.get(1).toString();
String part = nextTuple.get(2).toString();
String supp = nextTuple.get(3).toString();
String q = nextTuple.get(4).toString();
String ship = nextTuple.get(5).toString();
pbkey = nextTuple.get(6).toString();
thisLine = new V0Line(new String[]{line, ordk, part, supp, q, ship, pbkey});
if (lineFifo.isEmpty()) {
thisLine.setState("S");
} /*else {
if (!lineFifo.get(0).getGroup().equals(thisLine.getGroup())) {
thisLine.setState("S");
}
} */
lineFifo.add(0, thisLine);
if (!valueIt.hasNext()) {
noMoreValues = true;
thisLine.setState("E");
}
} else {
thisLine = null;
}
String evalState = null;
if (lineFifo.size() == 3) {
processLine = lineFifo.get(1);
if (processLine.getState() == null) {
prevLine = lineFifo.get(0);
nextLine = lineFifo.get(2);
processLine.setState(prevLine, nextLine);
}
evalLine = lineFifo.remove(2);
evalState = evalLine.getState();
} else {
if (noMoreValues) {
evalLine = lineFifo.remove(lineFifo.size() - 1);
evalState = evalLine.getState();
}
}
if (evalState != null) {
matchList += evalState;
if (matchList.length() > 5) {
matchList = matchList.substring(1);
}
if (matchList.equals("SBTBE")
|| matchList.equals("TBTBE")
|| matchList.equals("SBTBT")
|| matchList.equals("TBTBT")) {
//String[] atts = evalLine.getAttributes();
c++;
}
}
if (lineFifo.isEmpty()) {
finished = true;
}
}
Tuple outputTuple = TupleFactory.getInstance().newTuple(3);
outputTuple.set(0, line);
outputTuple.set(1, pbkey);
outputTuple.set(2, new Long(c));
return outputTuple;
}
@Override
public Schema outputSchema(Schema input) {
Schema.FieldSchema linenumber = new Schema.FieldSchema("linenumber", DataType.CHARARRAY);
Schema.FieldSchema pbykey = new Schema.FieldSchema("pbykey", DataType.CHARARRAY);
Schema.FieldSchema count = new Schema.FieldSchema("count", DataType.LONG);
Schema tupleSchema = new Schema();
tupleSchema.add(linenumber);
tupleSchema.add(pbykey);
tupleSchema.add(count);
return new Schema(tupleSchema);
}
}
Pattern Matching Finding Double Bottom (W)
SELECT first_x, last_z
FROM ticker MATCH_RECOGNIZE (
PARTITION BY name ORDER BY time
MEASURES FIRST(x.time) AS first_x,
LAST(z.time) AS last_z
ONE ROW PER MATCH
PATTERN (X+ Y+ W+ Z+)
DEFINE X AS (price < PREV(price)),
Y AS (price > PREV(price)),
W AS (price < PREV(price)),
Z AS (price > PREV(price) AND
z.time - FIRST(x.time) <= 7 ))
250+ Lines of Java and PIG 12 Lines of SQL
20x less code, 5x faster
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 66
Security & Compliance
Big Data
Consolidation
Data Optimization
Application Development
High Availability
In-Memory
Performance & Scalability
Data Warehousing
Plug into the Cloud
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 67
The Customer’s View v
Oracle Database 12c
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 69
Foundation of Private and Public Clouds