JANUS: Fast and Flexible Deep Learning via Symbolic Graph Execution of Imperative Programs Eunji Jeong, Sungwoo Cho, Gyeong-In Yu, Joo Seong Jeong, Dong-Jin Shin, Byung-Gon Chun 1 Demo
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
JANUS: Fast and Flexible Deep Learningvia Symbolic Graph Execution of Imperative Programs
Eunji Jeong, Sungwoo Cho, Gyeong-In Yu,Joo Seong Jeong, Dong-Jin Shin, Byung-Gon Chun
1Demo
2
Introduction Challenge Solution Results
Images From:http://www.mdpi.com/https://adeshpande3.github.io/A-Beginner%27s-Guide-To-Understanding-Convolutional-Neural-Networks/Going Deeper with Convolutions, 2014, https://towardsdatascience.com/learn-how-recurrent-neural-networks-work-84e975feaaf7Short-Term Load Forecasting Using EMD-LSTM Neural Networks with a Xgboost Algorithm for Feature Importance Evaluation, Energies 2017https://skymind.ai/wiki/generative-adversarial-network-ganhttps://en.wikipedia.org/wiki/Reinforcement_learninghttps://medium.com/@Petuum/intro-to-dynamic-neural-networks-and-dynet-67694b18cb23
Deep Neural Networks
Images From:http://www.mdpi.com/https://adeshpande3.github.io/A-Beginner%27s-Guide-To-Understanding-Convolutional-Neural-Networks/Going Deeper with Convolutions, 2014, https://towardsdatascience.com/learn-how-recurrent-neural-networks-work-84e975feaaf7Short-Term Load Forecasting Using EMD-LSTM Neural Networks with a Xgboost Algorithm for Feature Importance Evaluation, Energies 2017https://skymind.ai/wiki/generative-adversarial-network-ganhttps://en.wikipedia.org/wiki/Reinforcement_learninghttps://medium.com/@Petuum/intro-to-dynamic-neural-networks-and-dynet-67694b18cb23
3
Introduction Challenge Solution Results
Deep Learning (DL) Frameworks
Deep Neural Networks
4
2.0
Introduction Challenge Solution Results
1.x
Images From:http://www.mdpi.com/https://adeshpande3.github.io/A-Beginner%27s-Guide-To-Understanding-Convolutional-Neural-Networks/Going Deeper with Convolutions, 2014, https://towardsdatascience.com/learn-how-recurrent-neural-networks-work-84e975feaaf7Short-Term Load Forecasting Using EMD-LSTM Neural Networks with a Xgboost Algorithm for Feature Importance Evaluation, Energies 2017https://skymind.ai/wiki/generative-adversarial-network-ganhttps://en.wikipedia.org/wiki/Reinforcement_learninghttps://medium.com/@Petuum/intro-to-dynamic-neural-networks-and-dynet-67694b18cb23
Symbolic DL Frameworks Imperative DL Frameworks
Deep Neural Networks
Symbolic DL Frameworks
✓ Build a Symbolic Graph✓ Execute the Graph
def build_graph(g): x = g.placeholder(float) linear = g.add(g.mul(W, x), b)
build_graph(graph)run_graph(graph, x_data)
Imperative DL Frameworks
✓ Directly Execute the Computations
x
Mul
Add
W
b
5
def linear(x): return W * x + blinear(x_data)
2.01.x
Introduction Challenge Solution Results
Symbolic DL Frameworks
+ Easy to Optimize+ Compiler Optimization+ Parallel Execution of Operations+ Deploy on GPU, Cluster, Mobile,...
- Decoupled View:Hard to Program & Debug
Imperative DL Frameworks
+ Direct Execution:Easy to Program & Debug
- Hard to Optimize
Pros
Cons
6
Introduction Challenge Solution Results
JANUS: Combining the Best of Both Worlds
7
Imperative DL Program
def foo(x): tmp = mul(3, x) return add(tmp, 2)
TransparentConversion
Symbolic DL Graph
x
Mul
Add
3
2
“Easy Programmability” “High Performance”
Introduction Challenge Solution Results
8
Introduction Challenge Solution Results
Imperative DL Programwith Dynamic Features
for item in sequence: state = Cell(state, item) outputs += [state] Symbolic DL Graph
?● Dynamic Control Flow● Dynamic Types● Impure Functions● ...
9
Introduction Challenge Solution Results
Symbolic DL Graph
state
CellSwitch
Mergei<N
Next ● Correct● Slow
?Imperative DL Programwith Dynamic Features
for item in sequence: state = Cell(state, item) outputs += [state]
Symbolic DL Graph10
Introduction Challenge Solution Results
Symbolic DL Graph
state
CellSwitch
Mergei<N
Next ● Correct● Slow
● Fast● Incorrect
Cell
state
Cell
Cell
?Imperative DL Programwith Dynamic Features
for item in sequence: state = Cell(state, item) outputs += [state]
Solution: Speculative Graph Generation and Execution
● [Performance] Speculatively Specialize the Graph○ Make reasonable assumptions based on the execution history (Profiling)○ Run specialized graph (Common Case)
● [Correctness] Validate Assumptions○ Fallback if an assumption is broken (Rare Case)
11
Introduction Challenge Solution Results
Imperative DL Program
for item in sequence: state = rnn(state, item) outputs += [state]
Imperative Executor
Pre-defined DL Operations .Python Interpreter .
Overall Workflow on JANUS
12
Introduction Challenge Solution Results
Imperative DL Program
for item in sequence: state = rnn(state, item) outputs += [state]
Imperative Executor
Pre-defined DL Operations .Python Interpreter .
Profiler
len:3
13
Overall Workflow on JANUSIntroduction Challenge Solution Results
Symbolic Graph Executor
Imperative DL Program
for item in sequence: state = rnn(state, item) outputs += [state]
Symbolic DL Graph
Pre-defined DL Operations .Python Interpreter .
14
Overall Workflow on JANUS
Cell
state
Cell
Cell
len == 3?
Assert
GraphGenerator
len:3
Introduction Challenge Solution Results
Cell
state
Cell
Cell
len == 3?
Assert
Symbolic Graph Executor
Imperative DL Program
for item in sequence: state = rnn(state, item) outputs += [state]
Symbolic DL Graph
Pre-defined DL Operations .Python Interpreter .
15
Overall Workflow on JANUS
AssumptionFailure
len:3
Introduction Challenge Solution Results
Symbolic Graph Executor
Cell
state
Cell
Cell
len == 3?
Assert
Imperative DL Program
for item in sequence: state = rnn(state, item) outputs += [state]
Symbolic DL Graph
Pre-defined DL Operations .Python Interpreter .
16
Overall Workflow on JANUS
len:3
Introduction Challenge Solution Results
Imperative Executor
Imperative DL Program
for item in sequence: state = rnn(state, item) outputs += [state]
Pre-defined DL Operations .Python Interpreter .
Profiler
17
Overall Workflow on JANUS
len:?
Introduction Challenge Solution Results
Symbolic Graph Executor
Imperative DL Program
for item in sequence: state = rnn(state, item) outputs += [state]
Symbolic DL Graph
Pre-defined DL Operations .Python Interpreter .
GraphGenerator
18
Overall Workflow on JANUS
state
CellSwitch
Merge
i<N
Next
len:?
Introduction Challenge Solution Results
ImageNet Test Error with ResNet50
19
ImperativeSymbolic
Time
JANUS
36 GPUs
3.4x Faster Convergence
Introduction Challenge Solution Results
CNN
GAN
DRL
TreeNN
RNN
Normalized Training Throughput
20
LeNetResNet-50
Inception-v3LSTM
LMTreeRNN
TreeLSTMA3CPPO
ANPIX2PIX
Single Machine
Imp.
Symbolic
47.6x over Imperative
96.0% ofSymbolic
Imperative JANUS
Introduction Challenge Solution Results
Thank You!
21
Related Documents
