Overview of the Hive Stinger Initiative

Overview of the Hive Stinger Initiative

Eric N. Hanson

Principal Software Development Engineer

Microsoft HDInsight Team

30 June 2014

What is Stinger? Umbrella term for…

• Faster query in Hive• ORC• Vectorization• Tez

• Better language features for analysis• Window functions etc.

Why Stinger?

• Hive has good functionality

• But it started out sloooowww

• Need to speed it up• keep it competitive • make it fun to use

ORC

• A good columnstore format

• Run length encoding, value encoding, dictionary encoding

• Layers stream compression over the top

• Written by Owen O’Malley

• http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.0.0.2/ds_Hive/orcfile.html

Using ORC

• create table Tbl (col int) stored as orc;

• orc.compress default ZLIB

• See http://www.slideshare.net/oom65/orc-andvectorizationhadoopsummit

TPC-DS File Sizes

Page 6*Courtesy of Hortonworks

Vectorization

Page 7

How the code works (simplified)

Page 8

class LongColumnAddLongScalarExpression {int inputColumn;int outputColumn;long scalar;void evaluate(VectorizedRowBatch batch) {

long [] inVector =((LongColumnVector) batch.columns[inputColumn]).vector;

long [] outVector = ((LongColumnVector) batch.columns[outputColumn]).vector;

if (batch.selectedInUse) {for (int j = 0; j < batch.size; j++) {

int i = batch.selected[j];outVector[i] = inVector[i] + scalar;

} } else {

for (int i = 0; i < batch.size; i++) {outVector[i] = inVector[i] + scalar;

} }

}}

}

No method callsLow instruction countCache locality to 1024 valuesNo pipeline stallsSIMD in Java 8

Vectorization and Compilation

• Vectorization “instructions” generated from templates

• Example’s:– Int add col-col

– Int add col-scalar

– Int add scalar-col

–Double add col-col

–Double add col-scalar

–Double add scalar-col

–And hundreds more!

• Pre-compilation of expressions

• Reduces # of function calls and instructions at runtime

• Expressions like (a + 2) / b are interpreted with these primitives

Example of vectorized template code

} else {

if (batch.selectedInUse) {

for(int j = 0; j != n; j++) {

int i = sel[j];

outputVector[i] = vector1[i] <OperatorSymbol> vector2[i];

}

} else {

for(int i = 0; i != n; i++) {

outputVector[i] = vector1[i] <OperatorSymbol> vector2[i];

}

}

}

Using vectorization in Hive

• set hive.vectorized.execution.enabled = true;

• Run query over ORC

• Only works for scalar types

• https://cwiki.apache.org/confluence/display/Hive/Vectorized+Query+Execution

• ~5X CPU reduction

Apache Tez (“Speed”)• Replaces MapReduce as primitive for Pig, Hive, Cascading etc.

– Smaller latency for interactive queries

– Higher throughput for batch queries

– 22 contributors: Hortonworks (13), Facebook, Twitter, Yahoo, Microsoft

YARN ApplicationMaster to run DAG of Tez Tasks

Task with pluggable Input, Processor and Output

Tez Task - <Input, Processor, Output>

Task

ProcessorInput Output

*Courtesy of Hortonworks

Tez: Building blocks for scalable data processing

Classical ‘Map’ Classical ‘Reduce’

Intermediate ‘Reduce’ for

Map-Reduce-Reduce

Map Processor

HDFS Input

Sorted Output

Reduce Processor

Shuffle Input

HDFS Output

Reduce Processor

Shuffle Input

Sorted Output

*Courtesy of Hortonworks

Hive – MR Hive – Tez

Hive-on-MR vs. Hive-on-TezSELECT a.x, AVERAGE(b.y) AS avg

FROM a JOIN b ON (a.id = b.id) GROUP BY a

UNION SELECT x, AVERAGE(y) AS AVG

FROM c GROUP BY x

ORDER BY AVG;

SELECT a.state

JOIN (a, c)SELECT c.price

SELECT b.id

JOIN(a, b)GROUP BY a.state

COUNT(*)AVERAGE(c.price)

M M M

R R

M M

R

M M

R

M M

R

HDFS

HDFS

HDFS

M M M

R R

R

M M

R

R

SELECT a.state,c.itemId

JOIN (a, c)

JOIN(a, b)GROUP BY a.state

COUNT(*)AVERAGE(c.price)

SELECT b.id

Tez avoids unneeded writes to HDFS

*Courtesy of Hortonworks

Tez Sessions

… because Map/Reduce query startup is expensive

• Tez Sessions–Hot containers ready for immediate use

–Removes task and job launch overhead (~5s – 30s)

• Hive–Session launch/shutdown in background (seamless, user not aware)

–Submits query plan directly to Tez Session

Native Hadoop service, not ad-hoc

*Courtesy of Hortonworks

Stinger Phase 3: Interactive Query In Hadoop

Page 16

Hive 10 Trunk (Phase 3)Hive 0.11 (Phase 1)

190xImprovement

1400s

39s

7.2s

TPC-DS Query 27

3200s

65s

14.9s

TPC-DS Query 82

200xImprovement

Query 27: Pricing Analytics using Star Schema Join Query 82: Inventory Analytics Joining 2 Large Fact Tables

All Results at Scale Factor 200 (Approximately 200GB Data)

*Courtesy of Hortonworks

How you can use Stinger enhancements

• Use Hive 13

• Use ORC: create table … stored as ORC

• Enable vectorization: set hive.vectorized.execution.enabled=true

• Enable Tez: set hive.execution.engine=tez

• See http://hortonworks.com/hadoop-tutorial/supercharging-

interactive-queries-hive-tez/

Reference(s)

• Stinger overview, Strata, fall 2013: http://www.slideshare.net/alanfgates/strata-stingertalk-oct2013?qid=09d16028-bd7e-47d8-8438-34f3242c6f0e&v=qf1&b=&from_search=1

Slides marked “Courtesy of Hortonworks” are from Hortonworks talks