© 2011 SAP AG. All rights reserved. 1 RKT Welcome and Introduction Welcome to day 1 of the Course
© 2011 SAP AG. All rights reserved. 1RKT
Welcome and Introduction
Welcometo day 1 of the Course
© 2011 SAP AG. All rights reserved. 2RKT
DAY 1 Learning Objectives: Overview SAP HANA
After completing this unit you will be able to:• Explain the architectural principles for SAP HANA• Understand function and roles of included components• Explain SAP’s stepwise approach towards SAP HANA• Understand the deployed SAP HANA scenarios• Explain SAP HANA sizing
© 2011 SAP AG. All rights reserved. 3RKT
SAP‘s Realization of In-Memory Computing: SAP HANA
SAP Note No. 1523337: SAP In-Memory Database 1.0: Central Note
© 2011 SAP AG. All rights reserved. 4RKT
Terminology
HANA In-memory
Column-Store
Sybase
OLAP
OLTP
BWA
liveCache
Replication
Row-Store
IMCE ?
??
?
?
?
??
? ETL
?
Realtime
NewDB
© 2011 SAP AG. All rights reserved. 5RKT
Pivotal Moment in Technology
MultiCore Technology
Affordable Memory
In-Memory Computing
Innovation
The advances in hardware of the last two decades allow to fundamentally rethink this basic design goal
© 2011 SAP AG. All rights reserved. 6RKT
PrerequisitesHardware and Software Innovations
Innovations inHardware-technology
~1 TB RAM / Server
Significant Reduction in cost / GB
Multi-Core Architecture
Massive scaling with blade servers
Row- and column based data store
Compression
Partitioning
No pre-calculated aggregates required
Innovations inSAP-Software-technology
SAP HANA’s strength comes from innovations in hardware and software
Important to understand them for understanding performance
© 2011 SAP AG. All rights reserved. 7RKT
Row and Column-StoreWhat is the difference and what is better?
Column wisestores tables by column
Columns can be accessed in one read.
Row wisestores tables by row
Data records are available as complete tuples in one read.
Tuple1
Tuple2Att1 Att2
Att3 Att5
Att4
© 2011 SAP AG. All rights reserved. 8RKT
- The application needs to process only one single record at one time (many selects and /or updates of single records). - The application typically needs to access the complete record - The columns contain mainly distinct values so compression rate would be low - Neither aggregations nor fast searching are required - The table has a small number of rows (for example configuration tables)
- Calculations are typically executed on single or few columns only - The table is searched based on values of a few columns - The table has a big number of columns- The table has a big number of rows and columnar operations are required (aggregate, scan, etc.) - High compression rates can be achieved because the majority of the columns contain only few distinct values (compared to number of rows)
Column wiseRow wise
Tuple1
Tuple2Att1 Att2
Att3 Att5
Att4
HANA ArchitectureRow Store vs. Column Store
SAP Standard: „It depends.“
© 2011 SAP AG. All rights reserved. 9RKT
Schematic View
Product Stock
A 5
B 4
C 3
D 4
E 2
F 2
G 2
H 4
I 4
J 4
K 5
L 5
M 4
N 3
O 3
P 2
mapping to memory
A B C D E F G H I J K L M N O P 5 4 3 4 2 2 2 4 4 4 5 5 4 3 3 2
A 5 B 4 C 3 D 4 E 2 F 2 G 2 H 4 I 4 J 4 K 5 L 5 M 4 N 3 O 3 P 2
orrowwise
columnwise
memory address
memory address
© 2011 SAP AG. All rights reserved. 10RKT
Schematic View & Compression
Compression rate depends on data distribution
Decompression is not necessarily required for all operations:
i.e. max or avg
Direct update into compressed storage is expensive:
Split into delta and main storage with periodic merges
Take Aways:
A B C D E F G H I J K L M N O P 5 4 3 4 2 2 2 4 4 4 5 5 4 3 3 2
A K L B D H I J M C N O E F G P 5 5 5 4 4 4 4 4 4 3 3 3 2 2 2 2
sort
© 2011 SAP AG. All rights reserved. 11RKT
Row vs. Column Store
Compression
Performance
© 2011 SAP AG. All rights reserved. 12RKT
Example: Row vs. Column Store Compression
© 2011 SAP AG. All rights reserved. 13RKT
Example: Delta vs. Main Store
Del
ta m
erg
e
© 2011 SAP AG. All rights reserved. 14RKT
Column Store: Compression by Column
HANA Database Bluebook (page 73 ff. - Data Compression in Column Store).
For more information regarding compression types, see:
© 2011 SAP AG. All rights reserved. 15RKT
Example 1: Compression Ratio
Traditional DB SAP HANA DB
~ 1/7
~ 1/7
© 2011 SAP AG. All rights reserved. 16RKT
Customer Example: Data Compression
© 2011 SAP AG. All rights reserved. 17RKT
Example: Performance Comparison (Rev. 25)Row vs. Column Store
Primary index & filter
Indexed column
Non-indexed column
© 2011 SAP AG. All rights reserved. 18RKT
Example: Performance Comparison (Rev. 25)Row vs. Column Store
Primary index & filter
Indexed column
Non-indexed column
© 2011 SAP AG. All rights reserved. 19RKT
In Memory Computing – a paradigm shift
Programming against a new scarce resource…
Core
CPU
Performance bottleneck today:CPU waiting for data to beloaded from memory into cache
Performance bottleneck in the past: Disk I/O
Disk
CPU Cache
Main Memory
… requires cache-conscious data-structures and algorithms.
© 2011 SAP AG. All rights reserved. 20RKT
Advantage: Parallelization
Column-based storage also simplifies parallel execution using multiple processor cores
Operations on different columns can easily be processed in parallel
execution of operations on one column can be parallelized by dividing the column into multiple sections that are processed by different processor cores
© 2011 SAP AG. All rights reserved. 21RKT
Impact: ERP Performance Gain
Assumption: I exchange the DB of my current ERP system with an SAP HANA DB.
Question: „Will the SAP response times decrease by a factor of 10?“
For certain jobs or transactions, yes possibly. Candidates for accelerator scenario.
On average, no.On average, database time contributes 30 – 40 % to the response time. Even, if ab SAP HANA DB reduces the db-time to nothing, there is no factor of 10.
© 2011 SAP AG. All rights reserved. 22RKT
SAP-Presentation
SAP-Presentation
SAP-Database
SAP-Database
R/3-Application
R/3-Application
R/3 Design: Three-Tier Architecture
Scalability
© 2011 SAP AG. All rights reserved. 23RKT
Further SW side optimization for speed required
Delegation of data intense operations to the in-memory data layer
Application Layer
Data Layer
Today‘s applicationsexecute many data intense operations in the application layer
High performant appsdelegate data intenseoperations to in-memory data layer
In-Memory Computing Imperative:
Avoid movement of detailed data – calculate first, then move results
© 2011 SAP AG. All rights reserved. 24RKT
SAP HANA DB Architecture
Not
yet
© 2011 SAP AG. All rights reserved. 25RKT
The SAP HANA DB has a persistence layer for data and a persistence layer for logs.
If power of SAP HANA DB server is lost data are not lost.
Backup & recovery of database is possible.
Is the SAP HANA DB a Memory-Only-DB?
No!
© 2011 SAP AG. All rights reserved. 26RKT
HANA Side-by-Side Scenarios
SAP ERPSAP CRM
SAP BW
ReplicationETL
BI ClientsS
QL
MD
X
BIC
S
HANA Studio
HANA DB
HANAModeling
In-M
emory
3rd Party
SAP Solution Manager
DataServices
LT R
ep
lic
ato
r
HANAAdmin.
1b) Accelerator
SAP ERP
HANA DB
In-Mem
ory
Primary Database
ABAP Layer
Update+ Read
Replicate+ Read
HANA Studio
HANAModeling
HANAAdmin.
1a)
LT Replicator
BI Clients
Non-SAP DB
SAP Kernel
© 2011 SAP AG. All rights reserved. 27RKT
Replication Scenarios
SAP Landscape Transformation:
Real time replication based on DB Trigger technology
BOBJ DataServices:
ETL-Tool for periodic data transfer
(Sybase Replication:)
Real time replication based on DB log mining
Others: - Via ABAP report and usage of secondary database connection- Direct Extractors
© 2011 SAP AG. All rights reserved. 28RKT
BW on HANA
SAP Solution Manager
SAP BW(>= BW 7.3 SP 5)
HANA DB
In-Mem
ory
HANA Studio
HANAModeling
HANAAdmin.
BI Clients
© 2011 SAP AG. All rights reserved. 29RKT
HANA deployment options for different use cases
Content
Reports that are re-built on HANA using a BI frontend.
Accelerated Apps
Speed up existing business suite functionality/applications through a side-by-side scenario with HANA.
Products on In-Memory
Existing applications and strategic investment areas that may be disruptive.
Technology Platform
Customer specific development.
BW
Traditional Extractors
SAPBusiness
Suite
Traditional DB
SAP HANA
BI Clients
SQ
L
MD
X
BIC
S
3rd Party
Non SAPData Source
In-Mem
ory
Computing Engine
Data Modeling
ETL
SAP HANA
In-Mem
ory
Computing Engine
Replication
© 2011 SAP AG. All rights reserved. 30RKT
Sizing of SAP HANA DB
Components to be sized Memory, CPU, Disk Space
• Driving Factor:
Amount of Data loaded into SAP HANA DB
Not included: Disk Space for Backup and other HA features
For SAP HANA Landscape there are additional requirements for other involved components:
SLT, BOBJ, … Infrastructure, i.e. Network Bandwidth
© 2011 SAP AG. All rights reserved. 31RKT
SAP HANA DB Architecture
© 2011 SAP AG. All rights reserved. 32RKT
Hardware Partners
SAP HANA-certified SAP hardware partners:CISCODELLFujitsu Hewlett Packard HitachiHuaweiIBM NEC VCE
For further information see service.sap.com/pam
SAP HANA Applianace only certified for Intel-CPUs:
SAP HANA Appliance only released for one Operating System:
LINUX SUSE SLES11/X86_64 64BIT
© 2011 SAP AG. All rights reserved. 33RKT
SAP HANA scalability Scales from very small servers to very large clusters
Single Server •2 CPU 128GB to 8 CPU 4TB •Single SAP HANA deployments for data marts or accelerators •Support for high availability and disaster recovery
Scale Out Cluster •2 to n servers per cluster •Each server is either 4 CPU/512GB or 8 CPU/1TB •Largest certified configuration: 56 servers •Largest tested configuration: 100+ servers •Support for high availability and disaster recovery
Cloud Deployment •SAP HANA instances can be deployed to AWS •Limited to developer license
© 2011 SAP AG. All rights reserved. 34RKT
SAP HANA License
SAP Note 1634687
“Starting with SPS03, SAP HANA Database (hereafter database) is equippedwith a licensing mechanism, which only permits the operation of thedatabase under a valid license.”
Licensing is based on memory consumption
© 2011 SAP AG. All rights reserved. 35RKT
Terminology
HANA In-memory
Column-Store
Sybase
OLAP
OLTP
BWA
liveCache
Replication
Row-Store
IMCE !
!!
!
!
!
!!
! ETL
!
Realtime
NewDB
© 2011 SAP AG. All rights reserved. 36RKT
Questions
© 2011 SAP AG. All rights reserved. 37RKT
You should now be able to• Explain the architectural principles for SAP HANA• Understand function and roles of included components• Explain SAP’s stepwise approach towards SAP HANA• Understand the deployed SAP HANA scenarios• Explain SAP HANA sizing
DAY 1 Summary: Overview SAP HANA
Database Fundamentals
© 2011 SAP AG. All rights reserved. 39RKT
DAY 1 Learning Objectives: DB Fundamentals
After completing this unit you will be able to:
• Explain the purpose and the principles of a database
• Understand the possibilities to access and manage a database
• Use the Structured Query Language (SQL) for simple database
queries
© 2011 SAP AG. All rights reserved. 40RKT
Why databases? (1)
Data hold our knowledge as well as
information on individuals, processes and
technology.
It is one of today‘s most valuable goods.
Concepts are needed on how to deal with
the exploding volume of data.
http://www.oreilly.de/catalog/9780596527341/toc.html
© 2011 SAP AG. All rights reserved. 41RKT
Why databases (2)?
Naively, programs were used to store title, artist, genre and their relations in several files directly on top of the file system.
Data redundancy
Difficulty in accessing data
Data isolation
Integrity problems
Concurrent access
Security issues
Music collection with 1,000,000
songs
P1
P2
F1
F2
P3
© 2011 SAP AG. All rights reserved. 42RKT
Why databases (3)?
Naively, programs were used to store title, artist, genre and their relations in several files directly on top of the file system.
Data redundancy Data in several files are redundant.
Difficulty in accessing data Fragmented data are hard to access.
Data isolation Data are fragmented to multiple places.
Integrity problems Aborted operations may have changed data.
Concurrent access Multiple users want to access the same data.
Security issues Data has to be secured from unauthorized access.
Music collection with 1,000,000
songs
P1
P2
F1
F2
P3
© 2011 SAP AG. All rights reserved. 43RKT
Transactions and the ACID principle
A Transaction is an indivisible unit of processes such as reads and writes.
ACID is an acronym for Atomicity, Consistency, Isolation and Durability
• Atomicity: A Transaction fails even if only one part of it fails.
• Consistency: A Transaction brings the database from one consistent state to another.
• Isolation: Transactions cannot interfere with each other.
• Durability: Data changed by committed Transaction are persisted, even after a power loss or error.
© 2011 SAP AG. All rights reserved. 44RKT
Tables
• Using tables is the best way to store structured data.
• They consist of rows and columns.
• Columns possess headers defining the data type stored in them and the name with which the column can be addressed.
• Rows hold the actual information which are stored in the database.
• Each row is uniquely identified by a Primary Key that is represented by one or more columns.
• A Foreign Key points to a column or a set of column in another (referenced) table.
1
2
3
4
0
P F
© 2011 SAP AG. All rights reserved. 45RKT
Relational database
Relational databases consist of at least one table modeling important aspects of real (business) life.
→ Typically more than one table is contained in a database.
→ Tables can be interconnected (in a non-tree-like structure)
SONGS
ARTISTS
© 2011 SAP AG. All rights reserved. 46RKT
Components of a database
Data Files
Direct User Queries
Application Programs
© 2011 SAP AG. All rights reserved. 47RKT
Components of a database
Data Files
Query Processor
Direct User Queries
Application Programs
© 2011 SAP AG. All rights reserved. 48RKT
Components of a database
Data Files
Query Processor
Direct User Queries
Application Programs
File Manager
Database Manager
© 2011 SAP AG. All rights reserved. 49RKT
Components of a database
Data Files
Query Processor
Direct User Queries
Application Programs
File Manager
Database Manager
© 2011 SAP AG. All rights reserved. 50RKT
Components of a database
Data Files
Query Processor
Direct User Queries
Application Programs
File Manager
Authorization Tables
Database Manager
Concurrent Access Tables
© 2011 SAP AG. All rights reserved. 51RKT
Components of a database
Data Files
Query Processor
Direct User Queries
Application Programs
Database Administrator
DDL Processor
Database Description Tables
File Manager
Authorization Tables
Database Manager
Concurrent Access Tables
© 2011 SAP AG. All rights reserved. 52RKT
Components of a database
Data Files
Query Processor
Parameter Files
Direct User Queries
Application Programs
Database Administrator
DDL Processor
Database Description Tables
File Manager
Authorization Tables
Database Manager
Concurrent Access Tables
© 2011 SAP AG. All rights reserved. 53RKT
Issues and Solutions
• Data redundancy & isolation: Information is stored in just one file
• Concurrent user access: Usage of locking tables to manage the access
• Integrity Problems: Introduction of transactions
• Difficulty in accessing the data: In the database only one entry point is
existing to access the data
• Data Security: Ensured by the CIA Triad
© 2011 SAP AG. All rights reserved. 54RKT
Query Execution Plan
Show me the song
with ID 454
Jump to table SONGS, search for record with ID 454 and give back all its fields!
The path how a query is being executed to return the correct result.
SONGS
© 2011 SAP AG. All rights reserved. 55RKT
Query Execution Plan
Show me the song
with ID 454
Jump to table SONGS, search for record with ID 454 and give back all its fields!
Show me all my songs sung by Queen
The path how a query is being executed to return the correct result.
An access plan can reach any order of complexity depending on query including e.g. joins, sorts, etc.
SONGS
ARTISTS
© 2011 SAP AG. All rights reserved. 56RKT
Query processing
1) Coming from a graphical frontend, The query is translated into an expression understandable by the database, e.g. SQL.
2) Most probably, several ways exist to obtain the correct results from the database, including or not including indices, joins, full table scans, etc. The combinations of these primitive operations are called the query execution or access plan.
3) In order to obtain the result with the most performant plan, the query is optimized with regards to computation time, memory consumption and thread communication.
4) Finally, the query is evaluated via the access path generated by the optimizer.
Show me all my songs sung by Queen
SAP HANA Database Fundamentals
© 2011 SAP AG. All rights reserved. 58RKT
DAY 1 Learning Objectives: SAP HANA DB Fundamentals
After completing this unit you will be able to:
• Explain how the principles provided in the previous unit are realized in
an SAP HANA Database
• Describe the main components of the SAP HANA DB
• Understand and describe the SAP HANA DB specifics
© 2011 SAP AG. All rights reserved. 59RKT
Components of SAP HANA Database
Index Server
Preprocessor Server
Statistics Server
Name Server
SAP HANA Database System
XS Engine
Client Application
SAP HANA Studio
© 2011 SAP AG. All rights reserved. 60RKT
Index Server
Database Client request
SAP HANA Database Index Server
Connection and session management
© 2011 SAP AG. All rights reserved. 61RKT
Index Server
Database Client request
SAP HANA Database Index Server
Connection and session management
Request Processing and Execution Control
© 2011 SAP AG. All rights reserved. 62RKT
Index Server
Database Client request
SAP HANA Database Index Server
Connection and session management
Request Processing and Execution Control
Data processing
© 2011 SAP AG. All rights reserved. 63RKT
Index Server
Database Client request
SAP HANA Database Index Server
Connection and session management
Request Processing and Execution Control
Data processing
Persistence Layer
Disk Storage
© 2011 SAP AG. All rights reserved. 64RKT
Client interfaces
SQL :• Support both column and row based tables
• Support for defining and querying temporal tables
• The Column Store of the SAP HANA database supports special types of SQL views that are called column views
• Several SQL statements were extended to allow the specification of additional parameters.
• Support for full text search on database tables and views
• Planning Engine provides planning commands that are invoked by planning applications using SQL extensions
MDX: • MDX can be used to connect a variety of
analytics applications including for example SAP Business Objects products and clients such as Microsoft Excel.
BICS: • BICS is a replacement for MDX for
BusinessObjects Enterprise 4.0
REST: • XS Engine has a REST interface for searching
the data in the SAP HANA database
© 2011 SAP AG. All rights reserved. 65RKT
Client Library
The following client libraries are provided for accessing the SAP HANA database via SQL or MDX:
The SAP HANA database JDBC driver for Java clients
The SAP HANA database ODBC driver for Windows clients, especially for MS Office integration
The SAP HANA database specific Database Shared Library (DBSL) for running the ABAP application server on The SAP HANA database
The SAP HANA database client module for Python, implementing the Python DB API. The SAP-internal test framework for the SAP HANA database and applications on top is for example written in Python.
ODBO (OLE DB for OLAP) provider for MDX, using a partner product from Simba Technologies
The SQLDBC library which is the basis for the MDX provider and for the Python integration
© 2011 SAP AG. All rights reserved. 66RKT
MVCC: Multi Version Concurrency Control
© SAP AG 2011
Time
T0 Read D
All transactions want to access the same entry with value D
T2 Read ???
T3 Read A
T1 Change D -> A
There must be a mechanism which synchronizes theses accesses
Commit
© 2011 SAP AG. All rights reserved. 67RKT
MVCC: Multi Version Concurrency Control
update personalset name = ‘Michael'where userid = '4711'and name = 'McEwen'
select *from personalwhere name like 'M%'
PNR NAME
T1
T2
MVCC transaction level Snapshot
4711 McEwen 5200 Miller
Version n
MVCC statement level Snapshot
commit
select *from personalwhere name like 'M%'
PNR NAME
T2’
select *from personalwhere name like 'M%'
PNR NAME
select *from personalwhere name like 'M%'
PNR NAME
4711 McEwen 5200 Miller
4711 McEwen 5200 Miller
© SAP AG 2011
4711 Michael 5200 Miller
© 2011 SAP AG. All rights reserved. 68RKT
Row Store
Stores table by row, data records are available as complete lines in one read.
ID Artist Genre Title
1 Adele soul Rolling In The Deep
2 Lady Gaga pop Born this way
3 Katy Perry house E.T.
4 Jason Mraz pop I am yours
2 Lady Gaga popBorn this
way
3 Katy Perry house E.T.
4 Jason Mraz pop I am yours
1 Adele soulRolling In The Deep
Single-record access:
SELECT * FROM Tophits where ID = ‘2’
© 2011 SAP AG. All rights reserved. 69RKT
Write OperationsMainly go into “Transactional
Version Memory”
“INSERT” also writes to Persisted Segment
Read Operations
Write Operations
Transactional Version Memory
Main Memory
Persisted Segment
Data that may be
seen by all active
transactions
Recent versions of changed records
Version Memory Consolidation
Version Consolidation Moves “visible version”
from Transaction Version Memory into Persisted Segment (based on Commit ID)
Clears “outdated” record versions from Transactional Version Memory
Memory Handling Row store tables are
linked list of memory pages
Pages are grouped in segments
Page size: 16 KB
Persisted Segment Contains data that may be seen by any
ongoing transaction Data that has been committed before
any active transaction was started
Row Store Architecture
© 2011 SAP AG. All rights reserved. 70RKT
Indexes for Row Store Tables
Each row-store table has a primary indexPrimary index maps ROW ID primary key of tableROW ID: a number specifying for each record its memory segment and page
How to find the memory page for a table record?A structure called “ROW ID” contains the segment and the page for the recordThe page can then be searched for the records based on primary keyROW ID is part of the primary index of the table
Secondary indexes can be created if needed
Persistence of indexes in row storeIndexes in row store only exist in memoryNo persistence of index dataIndex definition stored with table metadataIndexes filled on-the-fly when system loads tables into memory on system start-up
© 2011 SAP AG. All rights reserved. 71RKT
Column Store
Stores table by column, columns can be accessed in one read..
ID Artist Genre Title hits
1 Adele soulRolling In The Deep
352
2 Lady Gaga popBorn this
way216
3 Katy Perry house E.T. 183
4 Jason Mraz pop I am yours 96
1
2
3
4
Adele
Lady Gaga
Katy Perry
Jason Mraz
soul
pop
house
pop
Rolling In The Deep
Born this way
E.T.
I am yours
352
216
183
96
Single-scan aggregation:
SELECT SUM(hits) FROM Tophits
∑
Result: 847
© 2011 SAP AG. All rights reserved. 72RKT
Column store architecture
SQL Processor or Calc Engine
Column Store Optimizer(Join Engine)
query
OLAP Optimizer(OLAP Engine)
Column Store executor
© 2011 SAP AG. All rights reserved. 73RKT
Storage Separation (Main & Delta) Enables high compression and high write
performance at the same time
Delta Merge Operation See next slide
Read Operations
Write Operations
Main
Main Memory
Delta
Write optimized
Compressed and
Read optimized
Read Operations Always have to read from both
main & delta storages and merge the results.
Engine uses multi version concurrency control (MVCC) to ensure consistent read operations.
Data Compression in Main Storage Compression by creating
dictionary and applying further compression methods
Speed up Data load into CPU cache Equality check Search
The compression is computed during delta merge operation.
Write Operations Only in delta storage because write
optimized. The update is performed by inserting a new
entry into the delta storage.
Column store architecture
© 2011 SAP AG. All rights reserved. 74RKT
Delta Merge
PurposeTo move changes in delta storage into the compressed and read optimized main storage
CharacteristicsHappens asynchronouslyEven during merge operation the columnar table will be still available for read and write operations
To fulfill this requirement, a second delta and main storage are used internally
Read Operations
Write Operations
Main
Before Merge
Delta
Read Operations
Write Operations
MainNew
After Merge
DeltaNew
Read Operations
Write Operations
Main
During Merge
Main New
DeltaNew
Delta
Merge Operations
© 2011 SAP AG. All rights reserved. 75RKT
Data Compression
Column “Artist” (uncompressed)
sort
ed
Value ID implicity given by sequence in which values are stored
Value-ID sequenceOne element for each row in column
ID Artist Genre Title
1 Adele soul Rolling In The Deep
2 Lady Gaga pop Born this way
3 Katy Perry house E.T.
4 Jason Mraz pop I am yours
5 Adele soul someone like you
6 Lady gaga pop Pockerface
7 Katy Perry pop Part of me
8 Bruno Mars popJust the Way You
Are
9 Jason Mraz Pop Lucky
10 Katy Perry pop Wide awake
Value ID value
1 Adele
2 Bruno Mars
3 Jason Mraz
4 Katy Perry
5 Lady gaga
Dictionary
row id
Value id
1 1
2 5
3 4
4 3
5 1
6 5
7 4
8 2
9 3
10 4Po
int i
nto
dict
iona
ry Inverted index
Value ID rows
1 1,5
2 8
3 4,9
4 3,7,10
5 2,6
© 2011 SAP AG. All rights reserved. 76RKT
Data Compression
For Main Storage:
Value ID Sequence compressed with different methods during delta merge operation. Best Compression rates applies for sorted table
For dictionary contains strings, values are further compressed
For Delta storage:
The values in the delta dictionary are stored in an unsorted array where new values are appended at the end. The position in this array is the value ID
Please refer to Chapter 8.5 of SAP Architecture Bluebook for more details
© 2011 SAP AG. All rights reserved. 77RKT
Valid
0
1
1
0
1
1
0
1
0
1
row id
Value id
1 1
2 5
3 4
4 3
5 1
6 5
7 4
8 2
9 3
10 4
Column Store Data Access
Tophit
Select * from Tophit where Artist = ‘Katy Perry’
Binary search on dictionary
Valid
0
1
1
0
1
1
0
1
0
1
Consistent View Filter(bit vector)
Full column scan on Value ID sequence for rows with value 4
Search by attribute value
Binary search on dictionary
Consistent View Filter(bit vector)
ID Artist Genre Title
1 Adele soulRolling In
The Deep
2 Lady Gaga pop Born this way
3 Katy Perry house E.T.
4 Jason Mraz pop I am yours
5 Adele soul someone like you
6 Lady gaga pop Pockerface
7 Katy Perry pop Part of me
8 Bruno Mars pop
Just the Way You
Are
9 Jason Mraz Pop Lucky
10 Katy Perry pop Wide awake
row id
Value id
1 1
2 5
3 4
4 3
5 1
6 5
7 4
8 2
9 3
10 4
Search by attribute value with inverted index
© 2011 SAP AG. All rights reserved. 78RKT
Column Store Data Access
Access by Row ID Column ‘Genre’ of table
Value ID
Value
1 house
2 pop
3 soul
… …
DictionaryValue ID Sequence
Row ID
Value ID
1 3
2 2
3 1
4 2
5 3
6 2
7 2
8 2
9 2
10 2
Result
ID Artist Genre Title
3 Katy Perry pop E.T.
10 Katy Perry pop Wide awake
© 2011 SAP AG. All rights reserved. 79RKT
Persistence Layer in SAP HANA DB
Why does an in-memory database need a persistence layerMain Memory is volatile. What happens uponDatabase restart?Power outage?... Data need to be stored in a non-volatile way
Backup and restore (see unit “SAP HANA Operations”)
HANA DB offers one persistence layer which is used by row store and column storeRegular “savepoints” full persisted image of DB at time of savepointLogs capturing all DB transactions since last savepoint (redo logs and undo logs written) restore DB from latest savepoint onwardsCreate Snapshots ( backup)
Purpose and Scope
© SAP AG 2011
© 2011 SAP AG. All rights reserved. 80RKT
SAP HANA Database
Memory
Data
Persistent Storage
Regular automatic savepoints
Information aboutdata changes
LogVolume
DataVolumes
© SAP AG 2011
Persistence Layer in SAP HANA DB
© 2011 SAP AG. All rights reserved. 81RKT
DAY 1 Summary: SAP HANA DB Fundamentals
You should now be able to:
• Explain how the principles provided in the previous unit are realized in
an SAP HANA Database
• Describe the main components of the SAP HANA DB
• Understand and describe the SAP HANA DB specifics
© 2011 SAP AG. All rights reserved. 82RKT
Today´s Agenda
Lessons Topics Time
Welcome Welcome, Introduction of Participants, Review Agenda and Topics
09:30 -10:00
Overview SAP HANA History (TREX, BWA, MaxDB…), Components, High-level Architecture and Overview, Roadmap, Scenarios, Business and Value proposition, Sizing, License
10:00 – 11:30
Lunch 11:30 – 13:00
Database Fundamentals Database basics (purpose, principles, components), Accessing and Managing a Database
13:00 – 14:30
Coffee Break 14:30 – 15:00
SAP HANA Database Fundamentals
Realization of Database Principles in SAP HANA DB, SAP HANA DB Components and Specifics
15:00 – 16:00
SAP HANA Studio Installation of SAP HANA Studio 16:00 -17:00
Installation
SAP HANA Studio
© 2011 SAP AG. All rights reserved. 84RKT
DAY 1Learning Objectives: SAP HANA Studio Installation
After completing this unit, you will be able to:• Install the SAP HANA Studio• Connect to a SAP HANA DB• Do some basic configuration
© 2011 SAP AG. All rights reserved. 85RKT
Goal
Install two versions of SAP HANA Studios on your PC/Laptop
Method
Individual work with support of experienced participants
Tasks and Questions to be answered:
•Install latest available SAP HANA Studio from SAP Service-MarketplaceQ: What is the latest version?
•Install latest available development version by configuring automatic software updates
Q: What is the latest version?
•Connect to an internal SAP HANA DB
Q: Can I run two SAP HANA Studios in parallel?
•Change default number of rows to be retrieved when performing a select-statement
Q: What is the default value?
Duration
60 minutes (45 minutes for tasks, 15 minutes for discussion)
Exercise: Installation of SAP HANA Studio
© 2011 SAP AG. All rights reserved. 86RKT
Additional Information
Location for Studio Updates:
http://bsl8213.wdf.sap.corp:1080/studio/
Logon Details for SAP HANA DB:
see flipchart
© 2011 SAP AG. All rights reserved. 87RKT
SAP HANA Studio First Start
The SAP HANA Studio is an Eclipse based tool delivered with SAP HANA and provides an environment for administration , modeling and data provisioning.
© 2011 SAP AG. All rights reserved. 88RKT
SAP HANA StudioPlain Administration Console
© 2011 SAP AG. All rights reserved. 89RKT
SAP HANA StudioConfigured Administration Console
Systems
© 2011 SAP AG. All rights reserved. 90RKT
Tasks and Questions to be answered:
• Install latest available SAP HANA Studio from SAP Service-MarketplaceQ: What is the latest version?
Solution: Installation of SAP HANA Studio
Goto: service.sap.com/hana Software Download SAP HANA Database - Studio Installation Guide
© 2011 SAP AG. All rights reserved. 91RKT
Tasks and Questions to be answered:
• Install latest available SAP HANA Studio from SAP Service-MarketplaceQ: What is the latest version?
• Install latest available development version by configuring automatic software updates
Q: What is the latest version?
Exercise: Installation of SAP HANA Studio
Configure automated updates: Window Preferences
© 2011 SAP AG. All rights reserved. 92RKT
Tasks and Questions to be answered:
• Install latest available SAP HANA Studio from SAP Service-MarketplaceQ: What is the latest version?
• Install latest available development version by configuring automatic software updates
Q: What is the latest version?
Exercise: Installation of SAP HANA Studio
How to find installed SAP HANA Studio version:
1) From within the SAP HANA Studio: Help About SAP HANA Studio
© 2011 SAP AG. All rights reserved. 93RKT
Exercise: Installation of SAP HANA Studio
How to find installed SAP HANA Studio version:
2) On OS-level
check file „manifest“ in
SAP HANA Studio
installation path
© 2011 SAP AG. All rights reserved. 94RKT
Tasks and Questions to be answered:
• Connect to an internal SAP HANA DB
Exercise: Installation of SAP HANA Studio
Right mouse click in Navigator pane Add system
© 2011 SAP AG. All rights reserved. 95RKT
Tasks and Questions to be answered:
• Connect to an internal SAP HANA DB
Q: Can I run two SAP HANA Studios in parallel? YES, when using a different workspace.
Exercise: Installation of SAP HANA Studio
How to switch the workspace:
1) File Switch Workspace
Other…
2) hdbstudio.exe –data <path_to_a_new_workspace> (add it to the hdbstudio.ini file (two lines))
© 2011 SAP AG. All rights reserved. 96RKT
• Change default number of rows to be retrieved when performing a select-statement
Q: What is the default value?
Exercise: Installation of SAP HANA Studio
Window Preferences:
© 2011 SAP AG. All rights reserved. 97RKT
You should now be able to• Install the SAP HANA Studio• Connect to a SAP HANA DB• Do some basic configuration
DAY 1Summary: SAP HANA Studio Installation