Top Banner
© 2011 SAP AG. All rights reserved. 1 RKT Welcome and Introduction Welcome to day 1 of the Course
97
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 1RKT

Welcome and Introduction

Welcometo day 1 of the Course

Page 2: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 2RKT

DAY 1 Learning Objectives: Overview SAP HANA

After completing this unit you will be able to:• Explain the architectural principles for SAP HANA• Understand function and roles of included components• Explain SAP’s stepwise approach towards SAP HANA• Understand the deployed SAP HANA scenarios• Explain SAP HANA sizing

Page 3: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 3RKT

SAP‘s Realization of In-Memory Computing: SAP HANA

SAP Note No. 1523337: SAP In-Memory Database 1.0: Central Note

Page 4: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 4RKT

Terminology

HANA In-memory

Column-Store

Sybase

OLAP

OLTP

BWA

liveCache

Replication

Row-Store

IMCE ?

??

?

?

?

??

? ETL

?

Realtime

NewDB

Page 5: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 5RKT

Pivotal Moment in Technology

MultiCore Technology

Affordable Memory

In-Memory Computing

Innovation

The advances in hardware of the last two decades allow to fundamentally rethink this basic design goal

Page 6: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 6RKT

PrerequisitesHardware and Software Innovations

Innovations inHardware-technology

~1 TB RAM / Server

Significant Reduction in cost / GB

Multi-Core Architecture

Massive scaling with blade servers

Row- and column based data store

Compression

Partitioning

No pre-calculated aggregates required

Innovations inSAP-Software-technology

SAP HANA’s strength comes from innovations in hardware and software

Important to understand them for understanding performance

Page 7: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 7RKT

Row and Column-StoreWhat is the difference and what is better?

Column wisestores tables by column

Columns can be accessed in one read.

Row wisestores tables by row

Data records are available as complete tuples in one read.

Tuple1

Tuple2Att1 Att2

Att3 Att5

Att4

Page 8: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 8RKT

- The application needs to process only one single record at one time (many selects and /or updates of single records). - The application typically needs to access the complete record - The columns contain mainly distinct values so compression rate would be low - Neither aggregations nor fast searching are required - The table has a small number of rows (for example configuration tables)

- Calculations are typically executed on single or few columns only - The table is searched based on values of a few columns - The table has a big number of columns- The table has a big number of rows and columnar operations are required (aggregate, scan, etc.) - High compression rates can be achieved because the majority of the columns contain only few distinct values (compared to number of rows)

Column wiseRow wise

Tuple1

Tuple2Att1 Att2

Att3 Att5

Att4

HANA ArchitectureRow Store vs. Column Store

SAP Standard: „It depends.“

Page 9: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 9RKT

Schematic View

Product Stock

A 5

B 4

C 3

D 4

E 2

F 2

G 2

H 4

I 4

J 4

K 5

L 5

M 4

N 3

O 3

P 2

mapping to memory

A B C D E F G H I J K L M N O P 5 4 3 4 2 2 2 4 4 4 5 5 4 3 3 2

A 5 B 4 C 3 D 4 E 2 F 2 G 2 H 4 I 4 J 4 K 5 L 5 M 4 N 3 O 3 P 2

orrowwise

columnwise

memory address

memory address

Page 10: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 10RKT

Schematic View & Compression

Compression rate depends on data distribution

Decompression is not necessarily required for all operations:

i.e. max or avg

Direct update into compressed storage is expensive:

Split into delta and main storage with periodic merges

Take Aways:

A B C D E F G H I J K L M N O P 5 4 3 4 2 2 2 4 4 4 5 5 4 3 3 2

A K L B D H I J M C N O E F G P 5 5 5 4 4 4 4 4 4 3 3 3 2 2 2 2

sort

Page 11: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 11RKT

Row vs. Column Store

Compression

Performance

Page 12: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 12RKT

Example: Row vs. Column Store Compression

Page 13: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 13RKT

Example: Delta vs. Main Store

Del

ta m

erg

e

Page 14: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 14RKT

Column Store: Compression by Column

HANA Database Bluebook (page 73 ff. - Data Compression in Column Store).

For more information regarding compression types, see:

Page 15: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 15RKT

Example 1: Compression Ratio

Traditional DB SAP HANA DB

~ 1/7

~ 1/7

Page 16: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 16RKT

Customer Example: Data Compression

Page 17: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 17RKT

Example: Performance Comparison (Rev. 25)Row vs. Column Store

Primary index & filter

Indexed column

Non-indexed column

Page 18: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 18RKT

Example: Performance Comparison (Rev. 25)Row vs. Column Store

Primary index & filter

Indexed column

Non-indexed column

Page 19: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 19RKT

In Memory Computing – a paradigm shift

Programming against a new scarce resource…

Core

CPU

Performance bottleneck today:CPU waiting for data to beloaded from memory into cache

Performance bottleneck in the past: Disk I/O

Disk

CPU Cache

Main Memory

… requires cache-conscious data-structures and algorithms.

Page 20: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 20RKT

Advantage: Parallelization

Column-based storage also simplifies parallel execution using multiple processor cores

Operations on different columns can easily be processed in parallel

execution of operations on one column can be parallelized by dividing the column into multiple sections that are processed by different processor cores

Page 21: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 21RKT

Impact: ERP Performance Gain

Assumption: I exchange the DB of my current ERP system with an SAP HANA DB.

Question: „Will the SAP response times decrease by a factor of 10?“

For certain jobs or transactions, yes possibly. Candidates for accelerator scenario.

On average, no.On average, database time contributes 30 – 40 % to the response time. Even, if ab SAP HANA DB reduces the db-time to nothing, there is no factor of 10.

Page 22: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 22RKT

SAP-Presentation

SAP-Presentation

SAP-Database

SAP-Database

R/3-Application

R/3-Application

R/3 Design: Three-Tier Architecture

Scalability

Page 23: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 23RKT

Further SW side optimization for speed required

Delegation of data intense operations to the in-memory data layer

Application Layer

Data Layer

Today‘s applicationsexecute many data intense operations in the application layer

High performant appsdelegate data intenseoperations to in-memory data layer

In-Memory Computing Imperative:

Avoid movement of detailed data – calculate first, then move results

Page 24: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 24RKT

SAP HANA DB Architecture

Not

yet

Page 25: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 25RKT

The SAP HANA DB has a persistence layer for data and a persistence layer for logs.

If power of SAP HANA DB server is lost data are not lost.

Backup & recovery of database is possible.

Is the SAP HANA DB a Memory-Only-DB?

No!

Page 26: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 26RKT

HANA Side-by-Side Scenarios

SAP ERPSAP CRM

SAP BW

ReplicationETL

BI ClientsS

QL

MD

X

BIC

S

HANA Studio

HANA DB

HANAModeling

In-M

emory

3rd Party

SAP Solution Manager

DataServices

LT R

ep

lic

ato

r

HANAAdmin.

1b) Accelerator

SAP ERP

HANA DB

In-Mem

ory

Primary Database

ABAP Layer

Update+ Read

Replicate+ Read

HANA Studio

HANAModeling

HANAAdmin.

1a)

LT Replicator

BI Clients

Non-SAP DB

SAP Kernel

Page 27: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 27RKT

Replication Scenarios

SAP Landscape Transformation:

Real time replication based on DB Trigger technology

BOBJ DataServices:

ETL-Tool for periodic data transfer

(Sybase Replication:)

Real time replication based on DB log mining

Others: - Via ABAP report and usage of secondary database connection- Direct Extractors

Page 28: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 28RKT

BW on HANA

SAP Solution Manager

SAP BW(>= BW 7.3 SP 5)

HANA DB

In-Mem

ory

HANA Studio

HANAModeling

HANAAdmin.

BI Clients

Page 29: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 29RKT

HANA deployment options for different use cases

Content

Reports that are re-built on HANA using a BI frontend.

Accelerated Apps

Speed up existing business suite functionality/applications through a side-by-side scenario with HANA.

Products on In-Memory

Existing applications and strategic investment areas that may be disruptive.

Technology Platform

Customer specific development.

BW

Traditional Extractors

SAPBusiness

Suite

Traditional DB

SAP HANA

BI Clients

SQ

L

MD

X

BIC

S

3rd Party

Non SAPData Source

In-Mem

ory

Computing Engine

Data Modeling

ETL

SAP HANA

In-Mem

ory

Computing Engine

Replication

Page 30: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 30RKT

Sizing of SAP HANA DB

Components to be sized Memory, CPU, Disk Space

• Driving Factor:

Amount of Data loaded into SAP HANA DB

Not included: Disk Space for Backup and other HA features

For SAP HANA Landscape there are additional requirements for other involved components:

SLT, BOBJ, … Infrastructure, i.e. Network Bandwidth

Page 31: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 31RKT

SAP HANA DB Architecture

Page 32: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 32RKT

Hardware Partners

SAP HANA-certified SAP hardware partners:CISCODELLFujitsu Hewlett Packard HitachiHuaweiIBM NEC VCE

For further information see service.sap.com/pam

SAP HANA Applianace only certified for Intel-CPUs:

SAP HANA Appliance only released for one Operating System:

LINUX SUSE SLES11/X86_64 64BIT 

Page 33: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 33RKT

SAP HANA scalability Scales from very small servers to very large clusters

Single Server •2 CPU 128GB to 8 CPU 4TB •Single SAP HANA deployments for data marts or accelerators •Support for high availability and disaster recovery

Scale Out Cluster •2 to n servers per cluster •Each server is either 4 CPU/512GB or 8 CPU/1TB •Largest certified configuration: 56 servers •Largest tested configuration: 100+ servers •Support for high availability and disaster recovery

Cloud Deployment •SAP HANA instances can be deployed to AWS •Limited to developer license

Page 34: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 34RKT

SAP HANA License

SAP Note 1634687

“Starting with SPS03, SAP HANA Database (hereafter database) is equippedwith a licensing mechanism, which only permits the operation of thedatabase under a valid license.”

Licensing is based on memory consumption

Page 35: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 35RKT

Terminology

HANA In-memory

Column-Store

Sybase

OLAP

OLTP

BWA

liveCache

Replication

Row-Store

IMCE !

!!

!

!

!

!!

! ETL

!

Realtime

NewDB

Page 36: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 36RKT

Questions

Page 37: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 37RKT

You should now be able to• Explain the architectural principles for SAP HANA• Understand function and roles of included components• Explain SAP’s stepwise approach towards SAP HANA• Understand the deployed SAP HANA scenarios• Explain SAP HANA sizing

DAY 1 Summary: Overview SAP HANA

Page 38: 1 HANA Overview

Database Fundamentals

Page 39: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 39RKT

DAY 1 Learning Objectives: DB Fundamentals

After completing this unit you will be able to:

• Explain the purpose and the principles of a database

• Understand the possibilities to access and manage a database

• Use the Structured Query Language (SQL) for simple database

queries

Page 40: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 40RKT

Why databases? (1)

Data hold our knowledge as well as

information on individuals, processes and

technology.

It is one of today‘s most valuable goods.

Concepts are needed on how to deal with

the exploding volume of data.

http://www.oreilly.de/catalog/9780596527341/toc.html

Page 41: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 41RKT

Why databases (2)?

Naively, programs were used to store title, artist, genre and their relations in several files directly on top of the file system.

Data redundancy

Difficulty in accessing data

Data isolation

Integrity problems

Concurrent access

Security issues

Music collection with 1,000,000

songs

P1

P2

F1

F2

P3

Page 42: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 42RKT

Why databases (3)?

Naively, programs were used to store title, artist, genre and their relations in several files directly on top of the file system.

Data redundancy Data in several files are redundant.

Difficulty in accessing data Fragmented data are hard to access.

Data isolation Data are fragmented to multiple places.

Integrity problems Aborted operations may have changed data.

Concurrent access Multiple users want to access the same data.

Security issues Data has to be secured from unauthorized access.

Music collection with 1,000,000

songs

P1

P2

F1

F2

P3

Page 43: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 43RKT

Transactions and the ACID principle

A Transaction is an indivisible unit of processes such as reads and writes.

ACID is an acronym for Atomicity, Consistency, Isolation and Durability

• Atomicity: A Transaction fails even if only one part of it fails.

• Consistency: A Transaction brings the database from one consistent state to another.

• Isolation: Transactions cannot interfere with each other.

• Durability: Data changed by committed Transaction are persisted, even after a power loss or error.

Page 44: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 44RKT

Tables

• Using tables is the best way to store structured data.

• They consist of rows and columns.

• Columns possess headers defining the data type stored in them and the name with which the column can be addressed.

• Rows hold the actual information which are stored in the database.

• Each row is uniquely identified by a Primary Key that is represented by one or more columns.

• A Foreign Key points to a column or a set of column in another (referenced) table.

1

2

3

4

0

P F

Page 45: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 45RKT

Relational database

Relational databases consist of at least one table modeling important aspects of real (business) life.

→ Typically more than one table is contained in a database.

→ Tables can be interconnected (in a non-tree-like structure)

SONGS

ARTISTS

Page 46: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 46RKT

Components of a database

Data Files

Direct User Queries

Application Programs

Page 47: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 47RKT

Components of a database

Data Files

Query Processor

Direct User Queries

Application Programs

Page 48: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 48RKT

Components of a database

Data Files

Query Processor

Direct User Queries

Application Programs

File Manager

Database Manager

Page 49: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 49RKT

Components of a database

Data Files

Query Processor

Direct User Queries

Application Programs

File Manager

Database Manager

Page 50: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 50RKT

Components of a database

Data Files

Query Processor

Direct User Queries

Application Programs

File Manager

Authorization Tables

Database Manager

Concurrent Access Tables

Page 51: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 51RKT

Components of a database

Data Files

Query Processor

Direct User Queries

Application Programs

Database Administrator

DDL Processor

Database Description Tables

File Manager

Authorization Tables

Database Manager

Concurrent Access Tables

Page 52: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 52RKT

Components of a database

Data Files

Query Processor

Parameter Files

Direct User Queries

Application Programs

Database Administrator

DDL Processor

Database Description Tables

File Manager

Authorization Tables

Database Manager

Concurrent Access Tables

Page 53: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 53RKT

Issues and Solutions

• Data redundancy & isolation: Information is stored in just one file

• Concurrent user access: Usage of locking tables to manage the access

• Integrity Problems: Introduction of transactions

• Difficulty in accessing the data: In the database only one entry point is

existing to access the data

• Data Security: Ensured by the CIA Triad

Page 54: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 54RKT

Query Execution Plan

Show me the song

with ID 454

Jump to table SONGS, search for record with ID 454 and give back all its fields!

The path how a query is being executed to return the correct result.

SONGS

Page 55: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 55RKT

Query Execution Plan

Show me the song

with ID 454

Jump to table SONGS, search for record with ID 454 and give back all its fields!

Show me all my songs sung by Queen

The path how a query is being executed to return the correct result.

An access plan can reach any order of complexity depending on query including e.g. joins, sorts, etc.

SONGS

ARTISTS

Page 56: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 56RKT

Query processing

1) Coming from a graphical frontend, The query is translated into an expression understandable by the database, e.g. SQL.

2) Most probably, several ways exist to obtain the correct results from the database, including or not including indices, joins, full table scans, etc. The combinations of these primitive operations are called the query execution or access plan.

3) In order to obtain the result with the most performant plan, the query is optimized with regards to computation time, memory consumption and thread communication.

4) Finally, the query is evaluated via the access path generated by the optimizer.

Show me all my songs sung by Queen

Page 57: 1 HANA Overview

SAP HANA Database Fundamentals

Page 58: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 58RKT

DAY 1 Learning Objectives: SAP HANA DB Fundamentals

After completing this unit you will be able to:

• Explain how the principles provided in the previous unit are realized in

an SAP HANA Database

• Describe the main components of the SAP HANA DB

• Understand and describe the SAP HANA DB specifics

Page 59: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 59RKT

Components of SAP HANA Database

Index Server

Preprocessor Server

Statistics Server

Name Server

SAP HANA Database System

XS Engine

Client Application

SAP HANA Studio

Page 60: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 60RKT

Index Server

Database Client request

SAP HANA Database Index Server

Connection and session management

Page 61: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 61RKT

Index Server

Database Client request

SAP HANA Database Index Server

Connection and session management

Request Processing and Execution Control

Page 62: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 62RKT

Index Server

Database Client request

SAP HANA Database Index Server

Connection and session management

Request Processing and Execution Control

Data processing

Page 63: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 63RKT

Index Server

Database Client request

SAP HANA Database Index Server

Connection and session management

Request Processing and Execution Control

Data processing

Persistence Layer

Disk Storage

Page 64: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 64RKT

Client interfaces

SQL :• Support both column and row based tables

• Support for defining and querying temporal tables

• The Column Store of the SAP HANA database supports special types of SQL views that are called column views

• Several SQL statements were extended to allow the specification of additional parameters.

• Support for full text search on database tables and views

• Planning Engine provides planning commands that are invoked by planning applications using SQL extensions

MDX: • MDX can be used to connect a variety of

analytics applications including for example SAP Business Objects products and clients such as Microsoft Excel.

BICS: • BICS is a replacement for MDX for

BusinessObjects Enterprise 4.0

REST: • XS Engine has a REST interface for searching

the data in the SAP HANA database

Page 65: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 65RKT

Client Library

The following client libraries are provided for accessing the SAP HANA database via SQL or MDX:

The SAP HANA database JDBC driver for Java clients

The SAP HANA database ODBC driver for Windows clients, especially for MS Office integration

The SAP HANA database specific Database Shared Library (DBSL) for running the ABAP application server on The SAP HANA database

The SAP HANA database client module for Python, implementing the Python DB API. The SAP-internal test framework for the SAP HANA database and applications on top is for example written in Python.

ODBO (OLE DB for OLAP) provider for MDX, using a partner product from Simba Technologies

The SQLDBC library which is the basis for the MDX provider and for the Python integration

Page 66: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 66RKT

MVCC: Multi Version Concurrency Control

© SAP AG 2011

Time

T0 Read D

All transactions want to access the same entry with value D

T2 Read ???

T3 Read A

T1 Change D -> A

There must be a mechanism which synchronizes theses accesses

Commit

Page 67: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 67RKT

MVCC: Multi Version Concurrency Control

update personalset name = ‘Michael'where userid = '4711'and name = 'McEwen'

select *from personalwhere name like 'M%'

PNR NAME

T1

T2

MVCC transaction level Snapshot

4711 McEwen 5200 Miller

Version n

MVCC statement level Snapshot

commit

select *from personalwhere name like 'M%'

PNR NAME

T2’

select *from personalwhere name like 'M%'

PNR NAME

select *from personalwhere name like 'M%'

PNR NAME

4711 McEwen 5200 Miller

4711 McEwen 5200 Miller

© SAP AG 2011

4711 Michael 5200 Miller

Page 68: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 68RKT

Row Store

Stores table by row, data records are available as complete lines in one read.

ID Artist Genre Title

1 Adele soul Rolling In The Deep

2 Lady Gaga pop Born this way

3 Katy Perry house E.T.

4 Jason Mraz pop I am yours

2 Lady Gaga popBorn this

way

3 Katy Perry house E.T.

4 Jason Mraz pop I am yours

1 Adele soulRolling In The Deep

Single-record access:

SELECT * FROM Tophits where ID = ‘2’

Page 69: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 69RKT

Write OperationsMainly go into “Transactional

Version Memory”

“INSERT” also writes to Persisted Segment

Read Operations

Write Operations

Transactional Version Memory

Main Memory

Persisted Segment

Data that may be

seen by all active

transactions

Recent versions of changed records

Version Memory Consolidation

Version Consolidation Moves “visible version”

from Transaction Version Memory into Persisted Segment (based on Commit ID)

Clears “outdated” record versions from Transactional Version Memory

Memory Handling Row store tables are

linked list of memory pages

Pages are grouped in segments

Page size: 16 KB

Persisted Segment Contains data that may be seen by any

ongoing transaction Data that has been committed before

any active transaction was started

Row Store Architecture

Page 70: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 70RKT

Indexes for Row Store Tables

Each row-store table has a primary indexPrimary index maps ROW ID primary key of tableROW ID: a number specifying for each record its memory segment and page

How to find the memory page for a table record?A structure called “ROW ID” contains the segment and the page for the recordThe page can then be searched for the records based on primary keyROW ID is part of the primary index of the table

Secondary indexes can be created if needed

Persistence of indexes in row storeIndexes in row store only exist in memoryNo persistence of index dataIndex definition stored with table metadataIndexes filled on-the-fly when system loads tables into memory on system start-up

Page 71: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 71RKT

Column Store

Stores table by column, columns can be accessed in one read..

ID Artist Genre Title hits

1 Adele soulRolling In The Deep

352

2 Lady Gaga popBorn this

way216

3 Katy Perry house E.T. 183

4 Jason Mraz pop I am yours 96

1

2

3

4

Adele

Lady Gaga

Katy Perry

Jason Mraz

soul

pop

house

pop

Rolling In The Deep

Born this way

E.T.

I am yours

352

216

183

96

Single-scan aggregation:

SELECT SUM(hits) FROM Tophits

Result: 847

Page 72: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 72RKT

Column store architecture

SQL Processor or Calc Engine

Column Store Optimizer(Join Engine)

query

OLAP Optimizer(OLAP Engine)

Column Store executor

Page 73: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 73RKT

Storage Separation (Main & Delta) Enables high compression and high write

performance at the same time

Delta Merge Operation See next slide

Read Operations

Write Operations

Main

Main Memory

Delta

Write optimized

Compressed and

Read optimized

Read Operations Always have to read from both

main & delta storages and merge the results.

Engine uses multi version concurrency control (MVCC) to ensure consistent read operations.

Data Compression in Main Storage Compression by creating

dictionary and applying further compression methods

Speed up Data load into CPU cache Equality check Search

The compression is computed during delta merge operation.

Write Operations Only in delta storage because write

optimized. The update is performed by inserting a new

entry into the delta storage.

Column store architecture

Page 74: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 74RKT

Delta Merge

PurposeTo move changes in delta storage into the compressed and read optimized main storage

CharacteristicsHappens asynchronouslyEven during merge operation the columnar table will be still available for read and write operations

To fulfill this requirement, a second delta and main storage are used internally

Read Operations

Write Operations

Main

Before Merge

Delta

Read Operations

Write Operations

MainNew

After Merge

DeltaNew

Read Operations

Write Operations

Main

During Merge

Main New

DeltaNew

Delta

Merge Operations

Page 75: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 75RKT

Data Compression

Column “Artist” (uncompressed)

sort

ed

Value ID implicity given by sequence in which values are stored

Value-ID sequenceOne element for each row in column

ID Artist Genre Title

1 Adele soul Rolling In The Deep

2 Lady Gaga pop Born this way

3 Katy Perry house E.T.

4 Jason Mraz pop I am yours

5 Adele soul someone like you

6 Lady gaga pop Pockerface

7 Katy Perry pop Part of me

8 Bruno Mars popJust the Way You

Are

9 Jason Mraz Pop Lucky

10 Katy Perry pop Wide awake

Value ID value

1 Adele

2 Bruno Mars

3 Jason Mraz

4 Katy Perry

5 Lady gaga

Dictionary

row id

Value id

1 1

2 5

3 4

4 3

5 1

6 5

7 4

8 2

9 3

10 4Po

int i

nto

dict

iona

ry Inverted index

Value ID rows

1 1,5

2 8

3 4,9

4 3,7,10

5 2,6

Page 76: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 76RKT

Data Compression

For Main Storage:

Value ID Sequence compressed with different methods during delta merge operation. Best Compression rates applies for sorted table

For dictionary contains strings, values are further compressed

For Delta storage:

The values in the delta dictionary are stored in an unsorted array where new values are appended at the end. The position in this array is the value ID

Please refer to Chapter 8.5 of SAP Architecture Bluebook for more details

Page 77: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 77RKT

Valid

0

1

1

0

1

1

0

1

0

1

row id

Value id

1 1

2 5

3 4

4 3

5 1

6 5

7 4

8 2

9 3

10 4

Column Store Data Access

Tophit

Select * from Tophit where Artist = ‘Katy Perry’

Binary search on dictionary

Valid

0

1

1

0

1

1

0

1

0

1

Consistent View Filter(bit vector)

Full column scan on Value ID sequence for rows with value 4

Search by attribute value

Binary search on dictionary

Consistent View Filter(bit vector)

ID Artist Genre Title

1 Adele soulRolling In

The Deep

2 Lady Gaga pop Born this way

3 Katy Perry house E.T.

4 Jason Mraz pop I am yours

5 Adele soul someone like you

6 Lady gaga pop Pockerface

7 Katy Perry pop Part of me

8 Bruno Mars pop

Just the Way You

Are

9 Jason Mraz Pop Lucky

10 Katy Perry pop Wide awake

row id

Value id

1 1

2 5

3 4

4 3

5 1

6 5

7 4

8 2

9 3

10 4

Search by attribute value with inverted index

Page 78: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 78RKT

Column Store Data Access

Access by Row ID Column ‘Genre’ of table

Value ID

Value

1 house

2 pop

3 soul

… …

DictionaryValue ID Sequence

Row ID

Value ID

1 3

2 2

3 1

4 2

5 3

6 2

7 2

8 2

9 2

10 2

Result

ID Artist Genre Title

3 Katy Perry pop E.T.

10 Katy Perry pop Wide awake

Page 79: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 79RKT

Persistence Layer in SAP HANA DB

Why does an in-memory database need a persistence layerMain Memory is volatile. What happens uponDatabase restart?Power outage?... Data need to be stored in a non-volatile way

Backup and restore (see unit “SAP HANA Operations”)

HANA DB offers one persistence layer which is used by row store and column storeRegular “savepoints” full persisted image of DB at time of savepointLogs capturing all DB transactions since last savepoint (redo logs and undo logs written) restore DB from latest savepoint onwardsCreate Snapshots ( backup)

Purpose and Scope

© SAP AG 2011

Page 80: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 80RKT

SAP HANA Database

Memory

Data

Persistent Storage

Regular automatic savepoints

Information aboutdata changes

LogVolume

DataVolumes

© SAP AG 2011

Persistence Layer in SAP HANA DB

Page 81: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 81RKT

DAY 1 Summary: SAP HANA DB Fundamentals

You should now be able to:

• Explain how the principles provided in the previous unit are realized in

an SAP HANA Database

• Describe the main components of the SAP HANA DB

• Understand and describe the SAP HANA DB specifics

Page 82: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 82RKT

Today´s Agenda

Lessons Topics Time

Welcome Welcome, Introduction of Participants, Review Agenda and Topics

09:30 -10:00

Overview SAP HANA History (TREX, BWA, MaxDB…), Components, High-level Architecture and Overview, Roadmap, Scenarios, Business and Value proposition, Sizing, License

10:00 – 11:30

Lunch 11:30 – 13:00

Database Fundamentals Database basics (purpose, principles, components), Accessing and Managing a Database

13:00 – 14:30

Coffee Break 14:30 – 15:00

SAP HANA Database Fundamentals

Realization of Database Principles in SAP HANA DB, SAP HANA DB Components and Specifics

15:00 – 16:00

SAP HANA Studio Installation of SAP HANA Studio 16:00 -17:00

Page 83: 1 HANA Overview

Installation

SAP HANA Studio

Page 84: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 84RKT

DAY 1Learning Objectives: SAP HANA Studio Installation

After completing this unit, you will be able to:• Install the SAP HANA Studio• Connect to a SAP HANA DB• Do some basic configuration

Page 85: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 85RKT

Goal

Install two versions of SAP HANA Studios on your PC/Laptop

Method

Individual work with support of experienced participants

Tasks and Questions to be answered:

•Install latest available SAP HANA Studio from SAP Service-MarketplaceQ: What is the latest version?

•Install latest available development version by configuring automatic software updates

Q: What is the latest version?

•Connect to an internal SAP HANA DB

Q: Can I run two SAP HANA Studios in parallel?

•Change default number of rows to be retrieved when performing a select-statement

Q: What is the default value?

Duration

60 minutes (45 minutes for tasks, 15 minutes for discussion)

Exercise: Installation of SAP HANA Studio

Page 86: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 86RKT

Additional Information

Location for Studio Updates:

http://bsl8213.wdf.sap.corp:1080/studio/

Logon Details for SAP HANA DB:

see flipchart

Page 87: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 87RKT

SAP HANA Studio First Start

The SAP HANA Studio is an Eclipse based tool delivered with SAP HANA and provides an environment for administration , modeling and data provisioning.

Page 88: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 88RKT

SAP HANA StudioPlain Administration Console

Page 89: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 89RKT

SAP HANA StudioConfigured Administration Console

Systems

Page 90: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 90RKT

Tasks and Questions to be answered:

• Install latest available SAP HANA Studio from SAP Service-MarketplaceQ: What is the latest version?

Solution: Installation of SAP HANA Studio

Goto: service.sap.com/hana Software Download SAP HANA Database - Studio Installation Guide

Page 91: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 91RKT

Tasks and Questions to be answered:

• Install latest available SAP HANA Studio from SAP Service-MarketplaceQ: What is the latest version?

• Install latest available development version by configuring automatic software updates

Q: What is the latest version?

Exercise: Installation of SAP HANA Studio

Configure automated updates: Window Preferences

Page 92: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 92RKT

Tasks and Questions to be answered:

• Install latest available SAP HANA Studio from SAP Service-MarketplaceQ: What is the latest version?

• Install latest available development version by configuring automatic software updates

Q: What is the latest version?

Exercise: Installation of SAP HANA Studio

How to find installed SAP HANA Studio version:

1) From within the SAP HANA Studio: Help About SAP HANA Studio

Page 93: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 93RKT

Exercise: Installation of SAP HANA Studio

How to find installed SAP HANA Studio version:

2) On OS-level

check file „manifest“ in

SAP HANA Studio

installation path

Page 94: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 94RKT

Tasks and Questions to be answered:

• Connect to an internal SAP HANA DB

Exercise: Installation of SAP HANA Studio

Right mouse click in Navigator pane Add system

Page 95: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 95RKT

Tasks and Questions to be answered:

• Connect to an internal SAP HANA DB

Q: Can I run two SAP HANA Studios in parallel? YES, when using a different workspace.

Exercise: Installation of SAP HANA Studio

How to switch the workspace:

1) File Switch Workspace

Other…

2) hdbstudio.exe –data <path_to_a_new_workspace> (add it to the hdbstudio.ini file (two lines))

Page 96: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 96RKT

• Change default number of rows to be retrieved when performing a select-statement

Q: What is the default value?

Exercise: Installation of SAP HANA Studio

Window Preferences:

Page 97: 1 HANA Overview

© 2011 SAP AG. All rights reserved. 97RKT

You should now be able to• Install the SAP HANA Studio• Connect to a SAP HANA DB• Do some basic configuration

DAY 1Summary: SAP HANA Studio Installation