Oracle In-Memory Data Processing...Oracle In-Memory Data Processing Doug Hood @ScalableDBDoug Consulting Member of Technical Staff Product Manager for Oracle TimesTen, Oracle InMemory

Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|

OracleIn-MemoryDataProcessing

DougHood@ScalableDBDougConsultingMemberofTechnicalStaffProductManagerforOracleTimesTen,OracleInMemoryandOracleNoSQLDec12,2018


OracleDatabaseIn-MemoryEnablingReal-TimeAnalytics

AndyRivenesDatabaseIn-MemoryProductManagement

Oracle Corporation

Email:[email protected]:@TheInMemoryGuy

Blog:blogs.oracle.com/in-memory


SafeHarborStatementThefollowingisintendedtooutlineourgeneralproductdirection.Itisintendedforinformationpurposesonly,andmaynotbeincorporatedintoanycontract.Itisnotacommitmenttodeliveranymaterial,code,orfunctionality,andshouldnotberelieduponinmakingpurchasingdecisions.Thedevelopment,release,andtimingofanyfeaturesorfunctionalitydescribedforOracle’sproductsremainsatthesolediscretionofOracle.

3


WhatareOracle’sIn-MemoryDatabases

4


In-MemoryDatabaseTechnologyAcrossTiers

Application Application Application

Application Application Application

TimesTen In-MemoryDatabase• EmbeddableIn-MemoryDatabaseforApplicationTier• PrimaryUsecase: Latency-criticalcustomOLTP

qMicrosecondResponseTime• StandaloneDatabaseorasApplication-TierCacheforOracleDatabase

OracleDatabaseIn-Memory• Dual-FormatIn-MemoryDatabase• PrimaryUsecase: RealTimeAnalyticsonanysource

qBillionsofRows/Secanalyticdataaccess• Faster mixed-workloadenterpriseOLTP• Storage-Tiering:Combinesbestofmemory,flash,disk• Transparent:packagedappsrunwithnochanges

In-MemoryRowStore

In-MemoryColumnStore

ApplicationTier

DatabaseTier

5


WhatisDatabaseIn-Memory

6


RowFormatDatabasesvs.ColumnFormatDatabases

RowsStoredContiguously

§ Transactions runfasteronrowformat– Example:QueryorInsertasalesorder– Fastprocessingfewrows,manycolumns

ColumnsStoredContiguously

§ Analytics runfasteroncolumnformat– Example:Reportonsalestotalsbyregion– Fastaccessingfewcolumns,manyrows

SALES

SALES

7

UntilNowMustChooseOneFormatandSufferTradeoffs

QueryQuery

Que

ry


Breakthrough:DualFormatDatabase

• BOTH rowandcolumnformatsforsametable

• Simultaneouslyactiveandtransactionallyconsistent

• Analytics&reportingusenewin-memoryColumnformat

• OLTPusesprovenrowformat

8

BufferCacheNewIn-MemoryColumnStore

SALES SALESRow

FormatColumnFormat

SALES


Howeasyisittogetstarted

9


DatabaseIn-Memory:TrivialtoImplement

10

SimpletoConfigure

NocomplexsetupSetcolumnstoresize

DeclareIn-Memorytables

FullFunctionality

NoSQLrestrictions

100%Compatible

Noapplicationchanges

Nodatamigration

EasytoDeploy


Oracle In-Memory: Simple to Setup

1. Configure Memory Capacity• inmemory_size = XXX GB

2. Configure tables or partitions to be in memory• alter table | partition … inmemory;

3. Later drop analytic indexes to speed up OLTP

11


WhyAreAnalyticQueriesFasterInTheIn-MemoryColumnStore?

12


ScanningandfilteringdatamoreefficientlyDatabaseIn-MemoryTechnology

StorageIndexesCompression

Scan&filterdataincompressedformat

Accessonlythecolumnsyouneed

ColumnarFormat SIMDVectorProcessing

ProcessmultiplecolumnvaluesinasingleCPUinstruction

VectorRegister

Loadmultipleregionvalues

VectorCompareallvaluesan1cycle

CPUCA

CACA

CA

Min1Max3

Min4Max7

Min8Max12

Pruneoutanyunnecessarydatafromthecolumn

13


• Speedofmemory• ScanandFilteronlytheneededColumns

• VectorInstructions

OptimizerEnhancementsImprovesallaspectsofanalyticqueries

DataScansVe

ctorRegisterCPU

CA

CACA

CA

In-MemoryAggregation

•CreateIn-MemoryReportOutlinethatisPopulatedduringFastScan

•RunsReportsInstantly

Joins

•ConvertStarJoinsinto10XFasterColumnScans

•Searchlargetableforvaluesthatmatchsmalltable

HASHJOIN

TableA TableB

SALES

STATE=CA

14


DatabaseIn-MemoryandotherOracleDatabasefeatures

15


DatabaseIn-Memory:ScaleOut

16

Scale-Up

•Scale-UponlargeSMPs•NUMAOptimized

•Scale-OutAcrossServerstoGrowMemoryandCPUs

• In-MemoryQueriesParallelizedAcrossServers

Scale-Out CombinewithFlashandDisk

DISKPCIFLASHDRAM

ColdDataHottestData

ActiveData

•Easilyplacedataonmostcosteffectivetier

•SimultaneouslyAchieve:•Speed ofDRAM• I/OsofFlash•Cost ofDisk

Copyright©2018, Oracleand/oritsaffiliates.Allrightsreserved.| 17

RAC

ASM

RMAN

Data Guard & GoldenGate

• Pure In-Memory format does not change Oracle’s storage format, logging, backup, recovery, etc.

• All Oracle’s proven availability technologies work transparently

• Protection from all failures- Node, site, corruption, human error, etc.

DatabaseIn-Memory:IndustrialStrengthAvailability


ExtendIn-MemoryAnalyticsintoStorage

18

In-MemoryColumnarscans

In-FlashColumnarscans

• ExadataautomaticallytransformstabledataintoIn-MemoryDBcolumnarformatsinExadataFlashCache– Enablesfastvectorprocessingforstorageserverqueries

• AdditionalcompressionforOLTPcompressedoruncompressedtablesinflash– newin18.1

• Enablesdictionarylookupandavoidsprocessingunnecessaryrows

• SmartScanresultssentbacktodatabaseinIn-MemoryColumnarformat– ReducesDatabasenodeCPUutilization

• Uniquely optimizesnextgenerationFlashasmemory

Upto1.5TBDRAM perServer

Upto25.6TBFlash perServer


HowarecustomersusingDatabaseIn-Memory

19


AT&TWiFi– DataWarehouse• BusinessObjectsreports100X faster

• ETLprocessesimprovedby50% faster

• NochangestoSAPBusinessObjectsreports

Villeroy&Boch– SAPBW• SAPBWCOPAqueries30– 33X faster

• SAPTransactionlistqueries4– 4,800Xfaster

• Avoidedexpensive&riskyupgradetoS4/Hana

20

BOSCH – SAPCRM• Dropped allcustomindexes

• Analyticqueries2-20Xfaster,DML2-3X faster

• Nochangestoapplicationrequired

DieMobiliar– MixedWorkload• Analyticqueries50-200X faster

• Databasesizereduced considerably

• PhaseoutofNetezzaandmainframesystems

DatabaseIn-MemoryReferences


MankindPharma– MixedWorkload• Analyticalreports11x faster

• DroppingindexesimprovedOLTP• 90% reduction indatabasesize

Lufthansa – ReportingApplication• Analyticqueriesupto100x faster

• Improveddataingestperformance• Reductionindatabasesize

21

• LION – SAPERP• Analyticqueries4X faster

• Transactions2X faster• Analyticqueriesnowpossibleon100sofMillionsofPoint-of-SalesTransactions

ShanghaiCustoms– MixedWorkload• ProcessesClearance43x Faster

• ImprovesDeclaration-ServicesEfficiency• ReducedCosts

DatabaseIn-MemoryCustomers


Canweseeanexample

22


TemperatureTrendsDemo


WherecanIgetmoreinformation

24


JointheConversation

25

https://twitter.com/TheInMemoryGuyhttps://blogs.oracle.com/In-Memory/

WhitePapers(otn.com)• OracleDatabaseIn-MemoryWhitePaper• OracleDatabaseImplementationandUsageWhitePaper

• OracleDatabaseIn-MemoryAggregationPaper• WhentouseOracleDatabaseIn-Memory• OracleDatabaseIn-MemoryAdvisor

Videos• OracleDatabaseIn-MemoryYouTubeChannel• oracle.comPoweringtheReal-TimeEnterpriseoracle.com/us/corporate/events/dbim/index.htmlReal-TimeAnalyticsDemo

•AskTOMDBIMOfficeHoursSessions

AdditionalQuestions• In-Memoryblog:blogs.oracle.com/In-Memory• Myemail:[email protected]

https://www.facebook.com/OracleDatabasehttp://www.oracle.com/goto/dbim.html

AdditionalResources

https://twitter.com/db_inmemory

Oracle In-Memory Data Processing...Oracle In-Memory Data Processing Doug Hood @ScalableDBDoug Consulting Member of Technical Staff Product Manager for Oracle TimesTen, Oracle InMemory

Documents