REAL-LIFE BENCHMARK: SQL Server 2016 - D ONE · REAL-LIFE BENCHMARK: SQL Server 2016 Apr 2016, ... SAP HANA Candidate when staying with ... Bulk load into column store oriented tables

REAL-LIFE BENCHMARK: SQL Server 2016

Apr 2016, Zurich

Microsoft Launch Event, Zurich, 2016-04-06

Joonas Asikainen, Ruben Wolff, Simon Hefti, Hans Peter Gränicher

POC/Migration

Summary

Results

Benchmark Approach

Introduction

Results of a real-life benchmark: SQL Server 2016 | Microsoft Launch Event| (c) 2016 D|ONE 3

INTRODUCTION


Motivation

▪ Data Driven Value Creation is a key success factor in all industries

▪ Typical question: which analytical database will deliver the performance and features we need over the next 3 - 5 years?

▪ This is not a theoretical, but a very practical question

▪ As we are practical, we want to find out


Client Background

▪ Health Insurance

▪ Initiative “Information Driven Enterprise” since 2013

▪ 3 Mio insured people

▪ ~100 target tables, ~200 source tables several billion rows

7

Multi-layered, holistic, dimensional model, used by all users (enterprise wide)

Data Model


Architecture

Analytics/Reporting

Source

SRC1

SRC2

SRC3

INB UDPSTG UDM

SRC4

SRC5

PWX CDC

BU UsersUpdates Data BenefitETL Chain

Migration Focus

Web Service

Why bother, we have a running system

9

More Performance = More Insight

Leverage * Insight = Impact

Results of a real-life benchmark: SQL Server 2016 | Microsoft Launch Event| (c) 2016 D|ONE

BENCHMARK APPROACH

JunMayAprMarFebJan

2015 2016

DecNov

Assessment Go: POC

POC of Winning DB Go

Production

GoParallel Operations

In cloud5 contendersSynthetic data On premise

Winner vs Sybase IQFull productive data set

Daily load on both systems Automated test frameworkFull test end user tools


Process

11

Candidates for Analytical Database Upgrade

12

Sybase IQ 15.4 needs be updated as must-have requirements can not be met. Possible options:


Sybase IQ 16 Logical candidate to consider

SAP HANA Candidate when staying with same vendor

Oracle 12c Inhouse know how, widely used, used for staging area

SQL Server 2014 Inhouse know how, widely used

SQL Server 2016 Promising - to be looked at if considering SQL Server

Claim(150M)

Case(100 M)

CaseClaim Facility

Activity(500 M)

Date

Member(20 M)

Detail

Territory

Package Network NetworkGroup

Query of a specific Key Information Asset

Typical user queries

Claim(150M)

Facility

Member(20 M)

Claim(150M)

Facility

Member(20 M)

Activity(500 M) Detail

Claim(150M)

Facility

Member(20 M)

Case(100 M)

CaseClaim Territory

Queries



▪ Platform

Agile testing setup in the cloud: full benchmarking in a few weeks

Compare results against on-premises setup with Sybase IQ for baseline

▪ Data

Synthetic data

12 table structures & relations duplicated from the production system

Row count equal to production

Same grouping characteristics (distinct values)

14

Approach


Approach

▪ Hardware

Test on-premise Sybase 15.4 (155 GB RAM, 16 Cores)Test in cloud for Sybase 16, Oracle 12c, SQL Server (120 GB RAM, 16 Cores)Test in cloud for SAP HANA ONE (240 GB RAM, 16 Cores)

▪ Test Procedure

Key Information Asset based queries of varying complexity and filteringRun the query cycles multiple samples for averaging

RESULTS

Results


Typical user queries

SQL16 HANA SY15 SY16 SQL14 ORA

1.1s 25s 3.4s 6.9s 4.1s 43s

1.0 22 2.9 6.0 3.6 38

Queries for specific Key Information Asset

SQL16 HANA SY15 SY16 SQL14 ORA

2.8m 3.2m 5.6m 7.7m 26m 128m

1.0 1.2 2.0 2.8 9.5 46Factor:

Product:

Time:


▪ SQL Server 2016 with astonishingly clear winWith synthetic data against all contenders. With live data confirmed against Sybase (other contenders were not pursued further)

▪ SAP HANA performs similarly well to SQL Server 2016 for some queries, yet has more non-performing outliers(Also to be considered: with higher need of memory)

▪ Improvement from SQL Server 2014 to 2016 is impressive

▪ Aggregate Awareness: Oracle & SQL Server improved by factor > 10

Observations

POC & MIGRATION

JunMayAprMarFebJan

2015 2016

DecNov

Assessment Go: POC SQL Server 2016

POC SQL Server 2016 Go

Production

GoParallel Operations

In cloud6 contendersSynthetic data On premise

SQL Server 2016 vs Sybase IQFull productive data set Daily load on SQL Server

2016 Automated test frameworkFull test end user tools


Process

20


Migration-worthy SQL Server Features

▪ Aggregate awarenessfor performance boost

▪ AD-governed Row Level SecurityActive Directory Integration simplifies permission administration

▪ Monitor & SuspendQuery store allows us to “monitor & suspend”, i.e., identify users causing unnecessary DB load, suspend and educate them

▪ Semi-automated Query OptimizationAggregate Awareness together with the query store capabilities allows us to build a semi automated query optimization process


Findings during POC

▪ No show stoppersThese could be e.g. migration not possible, data not usable, security not manageable, RC not stable enough

▪ Performance gains reaffirmed using live dataSelect performance as measured in cloud is also achieved in prod system

▪ Lower storage cost due to high compressionFactor of 7 smaller on the column store tables compared to Sybase IQ

▪ Getting rid of long-runnersSome long running queries in Sybase went from 8h to seconds on SQL Server


Useful features

▪ Management/monitoring functionalityIt shows that SQL Server is a mature product. Even though SQL Server 2016 is new, the management and monitoring functions have carried over from earlier versions, or improved

▪ Management Studio is a nice development environmentDevelopers and DBAs tend to like it

▪ Snapshots instead of backupsAbility to take quick snapshots instead of full backups

▪ Microsoft TAP (Technology Adoption Program)Useful questions answered quickly during migration


Issues During Migration

▪ Linux based ETL - no bulk loadIn the current setup, the ETL server does not support bulk inserts because there is no native client. Our ODBC throughput is around 60’000 rows/sec.

▪ Bulk insertBulk load into column store oriented tables in combination with unique/foreign key constraints does not (yet) work with Community Technology Preview 3.2

▪ UTF8 Not supported Like - really?

▪ T/SQL differencesKey [word] escapingException / Transaction handling differencesCount(*) vs Count_Big(*)

▪ From 0 to parallel run in 60 daysAfter 60 PD’s effort we have Sybase IQ and MS SQL successfully running in parallel

▪ Expected to Go Live by mid-May Given no show-stoppers arise & stakeholders build confidence in SQL Server 2016→ Planned on … May Friday the 13th


Summary

Dr. Joonas Asikainen

[email protected] 71Zürich 8040 / Switzerland Tel: +41 44 435 10 10

Ruben Wolff

[email protected] 71Zürich 8040 / Switzerland Tel: +41 44 435 10 10

REAL-LIFE BENCHMARK: SQL Server 2016 - D ONE · REAL-LIFE BENCHMARK: SQL Server 2016 Apr 2016, ... SAP HANA Candidate when staying with ... Bulk load into column store oriented tables

Documents