VectorWise The world’s fastest database GIUA, 13 September 2011
VectorWiseThe world’s fastest database
GIUA, 13 September 2011
© 2011 Ingres Corporation
DBT-3 Database Schema
© 2011 Ingres Corporation
DBT-3 Data
© 2011 Ingres Corporation
What is VectorWise?
Started as an academic project– Centrum Wiskunde & Informatica (CWI)
• Python• MonetDB• X100 VectorWise
Adopted as an Ingres community project Joint venture company set up between CWI and
Ingres Corp. Now wholly owned by Ingres Corp.
© 2011 Ingres Corporation
What is VectorWise for?
Data warehousing Data marts Data mining Online Analytical Processing (OLAP) Business Intelligence
© 2011 Ingres Corporation
VectorWise Technology
On Chip ComputingT
ime
/ Cyc
les
to
Pro
cess
Data Processed
DISK
RAM
CHIP
10GB2-3GB40-400MB
2-20
15
0-2
50
Mil
lion
s
Vector Processing
Breakthrough technology
© 2011 Ingres Corporation
On Chip Computing
Processing in Chip Cache CPU cache access is more efficient than RAM cache
access
Tim
e / C
ycle
s t
o P
roce
ss
Data Processed
DISK
RAM
CHIP
10GB2-3GB40-100MB
2-2
01
50-
25
0M
illio
ns
© 2011 Ingres Corporation
Vector Processing
verses
=
1 x 1 = 1
2 x 2 = 4
3 x 3 = 9
4 x 4 = 16
5 x 5 = 25
6 x 6 = 36
7 x 7 = 49
8 x 8 = 64
.
.
.
n x n = n2
1 x 1
2 x 2
3 x 3
4 x 4
5 x 5
6 x 6
7 x 7
8 x 8
.
.
.
n x n
1
4
9
16
25
36
49
64
.
.
.
n2
SISDSingle
InstructionSingle
Dataprocessed
SIMDSingleInstructionMultipleDataprocessed
© 2011 Ingres Corporation
VectorWise Technology
Automatic Compression
Updateable Column Store
Automatic Storage Indexes
Minimize IO
Innovations on industry proven techniques
© 2011 Ingres Corporation
Updateable Column Store
Only access relevant data
Efficient incremental update enabled– Traditionally a weakness of column stores
Cust_Num Cust_surname Cust_first_name Cust_DOB Cust_Sex Cust_Add_1 Cust_Addr_2 Cust_City Cust_State
46328927956 Jones Steven 17-JAN-1971 M 333 StKilda Rd Melbourne Vic
98679975745 Smith Leonard 04-APR-1964 M 147 Trafalgar Road Birmingham England
52634346735 Rogers Cindy 11-MAR-1980 F Belmont Rail Service421 Station St
Belmont CA
346737347347 Andrews Jenny 14-SEP-1977 F 117 West 42nd St New York NY
88673477347 Cooper Sheldon 30-JUN-1980 M Ingres Corporation Level 2, 426 Argello St Redwood City CA
34673447568 Kollwitz Rolf 22-DEC-1975 M IBM Headquarters 123 Mount View Crs Atlantic City PN
99554443044 Wong Penny 13-NOV-1981 F Ming On Tower 1 177 Moa Tzu Tung Rd Beijing China
© 2011 Ingres Corporation
Automatic Compression
Vectorized compression– Compressed on disk– Decompression for data processing in CPU cache– Compressed in RAM
Column based compression with multiple algorithms– Automatically determined by VectorWise
© 2011 Ingres Corporation
Compression Methods
Run Length Encoding– Efficient if many duplicate adjacent tuple values are present– Such as in ordered columns with few unique values
Patched Frame Of Reference– Encodes values as a small difference from a page-wide base value– PFOR is effective on any data distribution with some value distribution locality
Delta encoding on top of PFOR– Integers are made smaller by considering the differences between subsequent
values– Highly effective on ordered data
PDICT dictionary encoding– Efficient in case the value distribution is dominated by a limited amount of very
frequent values– Is currently the only one that applies to character data types
© 2011 Ingres Corporation
Automatic Storage Indexes
Stores min/max value per data block Automatically created Automatically maintained Enables efficient identification of candidate data
blocks
© 2011 Ingres Corporation
authpass
IVWLOG
User Interface (SQL, ABF, OpenROAD,
JAVA, etc.)
DASServer (iigcd)
CommunicationsServer (iigcc)
Archiver (iiacp)
Recovery Server (iircp)
JournalsJournals
JournalsJournals
IngresTransaction
Log File
VectorWise Instance Architecture
Name Server (iigcn)
User Interface (SQL, ABF, OpenROAD,
etc.)
DBMS Server (iidbms)
Locks Log Buffers
iix100 Server VectorWiseData Store
IVWMemory
IVW LOCK
Databases
Databases
Databases
© 2011 Ingres Corporation
Operating System
Currently available on 64-bit Linux and Windows Runs on
– RedHat– Fedora– CentOS– SuSE11– Ubuntu– Works on other Linux flavours– Windows 2008– Windows 7
© 2011 Ingres Corporation
Hardware Requirements
Fast multi-core CPUs Memory
– 2 Gbytes for OS + IVW requirements + other apps– Minimum 8 Gbytes
Disk– Lots
© 2011 Ingres Corporation
And now live
© 2011 Ingres Corporation