Ko, Youngjoong September 9, 2015 Dept. of Computer Engineering, Dong-A University Deep Learning for NLP - Word Embedding - 1. Basic Concepts of Neural Network (NN) 2. Why do we need Deep Learning? 3. Learning Representation for NLP 4. Tools for Word Embedding - Word2Vector - Ranking-based 2 Contents
25
Embed
Deep Learning for NLPnlp.skku.edu/talks/DL-WordEmbedding(Youngjoong Ko).pdf · 2019-09-10 · 45 Tools for Word Embedding Word2Vec 4 F½ 46 Tools for Word Embedding Word2Vec parameters
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Ko, Youngjoong
September 9, 2015
Dept. of Computer Engineering, Dong-A University
Deep Learning for NLP - Word Embedding -
1. Basic Concepts of Neural Network (NN) 2. Why do we need Deep Learning? 3. Learning Representation for NLP 4. Tools for Word Embedding - Word2Vector - Ranking-based
2
Contents
3
Basic Concepts of NN
� Perceptron
Basic Concepts of NN
4
� Illustration Example (Apple Tree)
Basic Concepts of NN
5
� Illustration Example (Apple Tree)
Basic Concepts of NN
6
� Illustration Example (Apple Tree)
Basic Concepts of NN
7
� Illustration Example (Apple Tree)
� Multilayer Neural Network
8
Basic Concepts of NN
� Multilayer Neural Network
9
Basic Concepts of NN
� Training (Weight Optimization)
10
Basic Concepts of NN
� Training (Weight Optimization)
11
Basic Concepts of NN
)
� Training (Activation Functions)
12
Basic Concepts of NN
� Training (Activation Functions)
� Scoring Functions (Softmax)
13
Basic Concepts of NN
� Learning: Backpropagation � Calculate error at the output � Back-propagation = gradient descent + chain rule
14
Basic Concepts of NN
� Learning: Backpropagation
15
Basic Concepts of NN
� Learning: Backpropagation
16
Basic Concepts of NN
� Learning: Backpropagation � Calculate error at the output
17
Basic Concepts of NN
� Learning: Backpropagation � Calculate error at the output
18
Basic Concepts of NN
� Neural Network-Core Components
19
Basic Concepts of NN
� Neural Network-Process
20
Basic Concepts of NN
� Why was not old NN successful?
21
Why? Deep Learning
� Neural Network-Process � , parameter � Parameter Local Minima