• Machine learning • Databases • Algorithms and systems • Statistics and optimization Four year PhD programme Courses + PhD dissertation (No previous MSc required) http://datascience.inf.ed.ac.uk/ Discovering new patterns and knowledge from data • Big data • Natural language processing • Computer vision • Speech processing at the University of Edinburgh
85
Embed
Edinburgh MT lecture 3: Latent Variable Models and Word Alignment
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
• Machine learning • Databases • Algorithms and systems • Statistics and optimization
Four year PhD programme Courses + PhD dissertation (No previous MSc required)
http://datascience.inf.ed.ac.uk/
Discovering new patterns and knowledge from data
• Big data • Natural language processing • Computer vision • Speech processing
at the University of Edinburgh
The biggest revolution in the technological landscape for fifty years
Now accepting applications! Find out more and apply at:
pervasiveparallelism.inf.ed.ac.uk
• • 4-year programme: 4-year programme: MSc by Research + PhDMSc by Research + PhD
• Collaboration between: ▶ University of Edinburgh’s School of Informatics ✴ Ranked top in the UK by 2014 REF
▶ Edinburgh Parallel Computing Centre ✴ UK’s largest supercomputing centre
• Full funding available
• Industrial engagement programme includes internships at leading companies
• Research-focused: Work on your thesis topic from the start
• Research topics in software, hardware, theory and
application of: ▶ Parallelism ▶ Concurrency ▶ Distribution
Latent Variable Models and Word Alignment
INFR11062 spring 2015lecture 3
Goal
•Write down a model over sentence pairs.
•Learn an instance of the model from data.
•Use it to predict translations of new sentences.
Goal
•Write down a model over sentence pairs.
•Learn an instance of the model from data.
•Use it to predict translations of new sentences.
虽然 北 风 呼啸 , 但 天空 依然 十分 清澈 。Although north wind howls , but sky still very clear .
However , the sky remained clear under the strong north wind .
IBM Model 1
虽然 北 风 呼啸 , 但 天空 依然 十分 清澈 。Although north wind howls , but sky still very clear .
However , the sky remained clear under the strong north wind .
IBM Model 1
p(f ,a|e)Let’s write a simple model in terms of word-to-word alignments:
虽然 北 风 呼啸 , 但 天空 依然 十分 清澈 。Although north wind howls , but sky still very clear .
IBM Model 1
虽然 北 风 呼啸 , 但 天空 依然 十分 清澈 。Although north wind howls , but sky still very clear .
�
IBM Model 1
虽然 北 风 呼啸 , 但 天空 依然 十分 清澈 。Although north wind howls , but sky still very clear .
�
IBM Model 1
虽然 北 风 呼啸 , 但 天空 依然 十分 清澈 。Although north wind howls , but sky still very clear .
�
IBM Model 1
p(English length|Chinese length)
虽然 北 风 呼啸 , 但 天空 依然 十分 清澈 。Although north wind howls , but sky still very clear .
�
IBM Model 1
p(English length|Chinese length)
虽然 北 风 呼啸 , 但 天空 依然 十分 清澈 。Although north wind howls , but sky still very clear .
�
IBM Model 1
虽然 北 风 呼啸 , 但 天空 依然 十分 清澈 。Although north wind howls , but sky still very clear .
�
IBM Model 1
p(Chinese word position)
虽然 北 风 呼啸 , 但 天空 依然 十分 清澈 。Although north wind howls , but sky still very clear .
�
IBM Model 1
虽然 北 风 呼啸 , 但 天空 依然 十分 清澈 。Although north wind howls , but sky still very clear .
�
IBM Model 1
However
虽然 北 风 呼啸 , 但 天空 依然 十分 清澈 。Although north wind howls , but sky still very clear .
�
IBM Model 1
However
p(English word|Chinese word)
虽然 北 风 呼啸 , 但 天空 依然 十分 清澈 。Although north wind howls , but sky still very clear .
�
IBM Model 1
However
虽然 北 风 呼啸 , 但 天空 依然 十分 清澈 。Although north wind howls , but sky still very clear .
�
IBM Model 1
However
虽然 北 风 呼啸 , 但 天空 依然 十分 清澈 。Although north wind howls , but sky still very clear .
�
IBM Model 1
However ,
虽然 北 风 呼啸 , 但 天空 依然 十分 清澈 。Although north wind howls , but sky still very clear .
�
IBM Model 1
However ,
虽然 北 风 呼啸 , 但 天空 依然 十分 清澈 。Although north wind howls , but sky still very clear .
�
IBM Model 1
However , the
虽然 北 风 呼啸 , 但 天空 依然 十分 清澈 。Although north wind howls , but sky still very clear .
�
IBM Model 1
However , the sky remained clear under the strong north wind .
IBM Model 1
French, Englishsentence lengths
alignment of French word at position i
French, Englishword pair
p(f ,a|e) = p(I|J)IY
i=1
p(ai|J) · p(fi|eai)
IBM Model 1
p(despite| )虽然
p(however| )
p(although| )
p(northern| )
p(north| )北
虽然
虽然
北
IBM Model 1
p(despite| )虽然
p(however| )
p(although| )
p(northern| )
p(north| )北
虽然
虽然
北
???
???
???
???
???
IBM Model 1
p(despite| )虽然
p(however| )
p(although| )
p(northern| )
p(north| )北
虽然
虽然
北
???
???
???
???
???
✓{
• Optimization
虽然 北 风 呼啸 , 但 天空 依然 十分 清澈 。Although north wind howls , but sky still very clear .
However , the sky remained clear under the strong north wind .