Deep Multi-Task Learning with Shared Memory Pengfei Liu, Xipeng Qiu, Xuanjing Huang EMNLP2016 reading group presenter: ryosuke miyazaki
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
DeepMulti-TaskLearningwithSharedMemoryPengfei Liu,Xipeng Qiu,Xuanjing HuangEMNLP2016readinggrouppresenter:ryosukemiyazaki
AbstractDuetothelargenumberofparametersneuralmodelsneedalarge-scalecorpus. →unsupervisedpre-trainingiseffectiveMulti-tasklearningalsoimprovethefinalperformance.ThispaperproposeLSTMwithexternalmemoryformulti-tasklearning.
Model:ME-LSTM
Keyvector,Erasevector,Addvector
Model:ME-LSTM
Readingoperation
Ksegment,Mdimensionsperonesegment
,
Model:ME-LSTM
DeepFusionstrategy
Model:ME-LSTM
Writingoperation
Twoarchitectures
ARC-1 ARC-2
TrainingTask-specificoutputlayer
Linearcombinationofcostfunction
λm istheweightsforeachtaskm
Experiment:textclassification
Result:Movie
Result:Product
Analysis:Visualizedeepfusiongate
Sentimentscore
Dimensionsofdeepfusiongategt
Activate→ black
Analysis:Visualizedeepfusiongate
Conclusion・ Thispaperproposetwodeeparchitecturesformulti-tasklearning.・ Theydesignanexternalmemorytostoretheknowledgebyrelatedtasks.・ Deepfusionstrategyenablingthemodeltogivesharedinformation.
Related Documents
