Computing & Information Sciences Kansas State University Rabat, Morocco Human Language Technologies Workshop Human Language Technologies Human Language Technologies (HLT) Workshop 2006 (HLT) Workshop 2006 Classification-Based Contextual Classification-Based Contextual Correction of Mistranslations: Correction of Mistranslations: A Machine Learning Approach A Machine Learning Approach William H. Hsu Joint work with: Waleed Al-Jandal, Martin S. R. Paradesi, Tejaswi Pydimarri, Chris Meyer Thursday, 01 June 2006 Laboratory for Knowledge Discovery in Databases Kansas State University http://www.kddresearch.org/KSU/CIS/HLT-Specialized- 20060601.ppt
27
Embed
Computing & Information Sciences Kansas State University Rabat, MoroccoHuman Language Technologies Workshop Human Language Technologies (HLT) Workshop.
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Computing & Information SciencesKansas State University
Rabat, MoroccoHuman Language Technologies Workshop
Human Language TechnologiesHuman Language Technologies(HLT) Workshop 2006(HLT) Workshop 2006
Classification-Based Contextual Correction Classification-Based Contextual Correction of Mistranslations:of Mistranslations:
A Machine Learning ApproachA Machine Learning Approach
Human Language TechnologiesHuman Language Technologies(HLT) Workshop 2006(HLT) Workshop 2006
Classification-Based Contextual Correction Classification-Based Contextual Correction of Mistranslations:of Mistranslations:
A Machine Learning ApproachA Machine Learning Approach
William H. HsuJoint work with:
Waleed Al-Jandal, Martin S. R. Paradesi, Tejaswi Pydimarri, Chris MeyerThursday, 01 June 2006
Laboratory for Knowledge Discovery in DatabasesKansas State University
• Hard to “game” system (i.e., change machine output so that BLEU goes up, but quality doesn’t)
Reference (human) translation: The U.S. island of Guam is maintaining a high state of alert after the Guam airport and its offices both received an e-mail from someone calling himself the Saudi Arabian Osama bin Laden and threatening a biological/chemical attack against public places such as the airport .
Machine translation: The American [?] international airport and its the office all receives one calls self the sand Arab rich business [?] and so on electronic mail , which sends out ; The threat will be able after public place and so on the airport to start the biochemistry attack , [?] highly alerts after the maintenance. Adapted from Knight (2003)
Reference translation 1: The U.S. island of Guam is maintaining a high state of alert after the Guam airport and its offices both received an e-mail from someone calling himself the Saudi Arabian Osama bin Laden and threatening a biological/chemical attack against public places such as the airport .
Reference translation 3: The US International Airport of Guam and its office has received an email from a self-claimed Arabian millionaire named Laden , which threatens to launch a biochemical attack on such public places as airport . Guam authority has been on alert .
Reference translation 4: US Guam International Airport and its office received an email from Mr. Bin Laden and other rich businessman from Saudi Arabia . They said there would be biochemistry air raid to Guam Airport and other public places . Guam needs to be in high precaution about this matter .
Reference translation 2: Guam International Airport and its offices are maintaining a high state of alert after receiving an e-mail that was from a person claiming to be the wealthy Saudi Arabian businessman Bin Laden and that threatened to launch a biological and chemical attack on the airport and other public places .
Machine translation: The American [?] international airport and its the office all receives one calls self the sand Arab rich business [?] and so on electronic mail , which sends out ; The threat will be able after public place and so on the airport to start the biochemistry attack , [?] highly alerts after the maintenance.
Computing & Information SciencesKansas State University
Rabat, MoroccoHuman Language Technologies Workshop
BBiillingual ingual EEvaluation valuation UUnderstudy (BLEU) [3]:nderstudy (BLEU) [3]:Tracking Human JudgmentTracking Human Judgment
BBiillingual ingual EEvaluation valuation UUnderstudy (BLEU) [3]:nderstudy (BLEU) [3]:Tracking Human JudgmentTracking Human Judgment
R2 = 88.0%
R2 = 90.2%
-2.5
-2.0
-1.5
-1.0
-0.5
0.0
0.5
1.0
1.5
2.0
2.5
-2.5 -2.0 -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 2.0 2.5
Human Judgments
NIS
T S
core
Adequacy
Fluency
Linear(Adequacy)Linear(Fluency)
(va
ria
nt
of
BL
EU
)
Courtesy G. Doddington (NIST)
Computing & Information SciencesKansas State University
Rabat, MoroccoHuman Language Technologies Workshop
BBiillingual ingual EEvaluation valuation UUnderstudy (BLEU) [4]:nderstudy (BLEU) [4]:Metrics in ActionMetrics in Action
BBiillingual ingual EEvaluation valuation UUnderstudy (BLEU) [4]:nderstudy (BLEU) [4]:Metrics in ActionMetrics in Action
枪手被警方击毙。 (Foreign Original)
the gunman was shot to death by the police . (Reference Translation)
the gunman was police kill . #1wounded police jaya of #2the gunman was shot dead by the police . #3the gunman arrested by police kill . #4the gunmen were killed . #5the gunman was shot to death by the police . #6gunmen were killed by police ?SUB>0 ?SUB>0 #7al by the police . #8the ringer is killed by the police . #9police killed the gunman . #10
green = 4-gram match (good!)red = word not matched (bad!)
Computing & Information SciencesKansas State University
Rabat, MoroccoHuman Language Technologies Workshop
The frequency of named-entity phrases in news text reflects the significance of the events they are associated with. So the news most likely be reported in many languages.
For example:
Translating Named Entity Phrases [1]:Translating Named Entity Phrases [1]:Arabic-English ApplicationArabic-English Application
Translating Named Entity Phrases [1]:Translating Named Entity Phrases [1]:Arabic-English ApplicationArabic-English Application
The Arabic newspaper article is about negotiations between the US and North Korean authorities regarding the search for the remains of US soldiers who died during the Korean war.
[Knight & Al-Onaizan, 2001]
Computing & Information SciencesKansas State University
Rabat, MoroccoHuman Language Technologies Workshop
Generate ranked list of translation candidates Bilingual resources: parallel corpus Monolingual resources
Re-score list of candidates using different monolingual clues
Translating Named Entity Phrases [2]:Translating Named Entity Phrases [2]:Two-Phase ApproachTwo-Phase Approach
Translating Named Entity Phrases [2]:Translating Named Entity Phrases [2]:Two-Phase ApproachTwo-Phase Approach
[Knight & Al-Onaizan, 2001]
Computing & Information SciencesKansas State University
Rabat, MoroccoHuman Language Technologies Workshop