KIWI A Multi-Lingual Usage Consultation Tool based on Internet Searching

KIWIA Multi-Lingual Usage Consultation Tool

based on Internet Searching

Kumiko TANAKA-IshiiMasato YAMAMOTOHiroshi NAKAGAWA

Language Informatics Laboratory,University of Tokyo

How do you say

「無線 LAN 」 in French?

Never be found in dictionaries…

wireless

=

What could be done

1. Look up part of key in the dictionary

　　　無線 → sans fil

2. Enter the translation into search engine

1. Look up part of key in the dictionary

　　　無線 = sans fil

2. Enter the translation into search engine

le reseau sans fil le net sans fil l’acces sans fil

Top 20

What could be done

1. Look up part of key in the dictionary

　　　無線 = sans fil

2. Enter the translation into search engine

les reseaux sans fil l’internet sans fil

Sum up top 500

What could be done

Similar Ocassions….• Up to date expression 　 * sans fil les reseaux sans fil• Commonness of expression (le reseaux)/(les reseaux) sans fil 　• Simple Q&A 　　 * Zidane• Grammar check　　　 -noun gender un/une langage, un/une langue -preposition discuter ? -articles 　　 du/de Japon, du/de Nancy

Have clues but can’t remember exactly…

Our Idea• Multiple candidates Which one? 　　 • Minority candidate 300th candidate?

　 Impossible to manually scan 500 candidates!

A tool for scanning search engine results

Kiwi

Related Work 1 ： www.webcorp.org.uk

ー The Web as Corpus ー1999

• English only• Sum up fixed length words• Slow!!

Related Work 1 ： www.webcorp.org.uk

ー The Web as Corpus ー

•Compare the frequency of 2 phrases •Multilingual

Related work 2 ： Google Fight, Google Duel

Related work 2 ： Google Fight, Google Duel

•Comparison of two phrases only

Kiwi’s characteristics

•Flexible query 　　 - comparison A/B　　 - wild card 　 *A A*B B*　　

•Multilingual aspect -String based processing　　　 Language dependent analysis 　

User has clues

English webuser36.5%globstats 2002

Online Language Populations

http://www.glreach.com/globstats

The Process

1. Obtain search results

summaries only

2. Extract candidates 　 *A, A*

3. Order candidates

• Frequent

• Moderately long

• Various succeeding characters 　

Characteristics of candidates 　　 (at entry being A *) 　

• Frequent

• Moderately long

• Various succeeding characters 　Extraction

Ordering} }

Characteristics of candidates 　　 (at entry being A *) 　

Extraction: number of succeeding character kind

n a u r et _

a

l

human *

_

cut

longer context

increase

decrease increase

Branching degreedecreases

cut

is

Ordering

-Shorter more frequent

　 Ex. “international” includes “in” 　　　

Eval-fun (candidate)　　　 = freq （ candidate)× log (length(candidate) + 1) 　

　　　　　　 Empirically defined

Examples

german_demo_viewlet_swf.html

German

“Atemwegssyndrom”

Other candidates

・ Respiratorische Syndrom

・ oder Chronische

Gesundheits ・ Erkrankung

Etc…

Japanese

“ 重症急性呼吸器”（ SARS)

Other candidates

・シックハウス (Sick Building)

・エコノミークラス (Economy class)

・慢性疲労 (Chronic Fatigue)

Etc…

French

“Respiratoire Aigu Sévère ”

Other candidates

・ de Marfan ・ Prémenstruel ・ de la class

économique

Etc…

Chinese

“ 嚴重急性呼吸道”

Other candidates

・經前 ・電腦視覺 ・腕道 ・睡眠呼吸中止 ・後天免疫缺乏

Etc…

Korean

“ 중증급성호흡기”

Other candidates

・급성호흡기 ・만성피로 ・ 과민성 대장

Etc…

Evaluation: English collocations

Kiwi ： 1000 match totalized: examine top n　　　　　　　 (exact match)Baseline ： Search engine results: top n　　　　　　 (included or not)

and so on in spite of

and so * * spite of

tail head

300 300

Results

③ 　 upper bound of Kiwi 　　　

n = 1 n = 10 n = 1000head

head

tail

tail

Results③ ー②　 Extraction error② ー① Ordering error

＋ test set problem} Ex. 　 be anxious for 　　　　　　　 to

≠search engine　

n = 1 n = 10 n = 1000head

head

tail

tail

No. matches

Rankin

g

Insufficient

Sufficient

Data amountRank transition of best & correct candidate

EvaluationUsing different search engines

Answer in Top 1

in Top 10 In Candidate

Mean Reciprocal

Ranking

AltaVista head

AllTheWeb head

Google head

77.0%74.8%76.4%

93.3%91.5%92.7%

97.0%97.6%97.2%

0.830.800.82

AltaVista tail

AllTheWeb tail

Google tail

78.5%73.6%75.8%

92.8%93.2%

93.8%

96.3%97.8%98.1%

0.850.800.82

Red score is the best score

n = 1 n = 10 n=1000

Obtain Q&A answers from search engine results 　

Related work 3 ： NL based on search engineEx. 　 Q&A Brill et al.(2002)

already totalized results

What does this mean to NL?

Comparison of Results

head tail

Top n candidates

1 3 10 1 3 10

Using different search engine

AltaVista-

AllTheWeb87.4% 72.9

%56.2% 84.5% 71.0% 57.8%

AllTheWeb-

Google86.2% 75.1

%59.4% 87.0% 75.8% 61.8%

Google-

AltaVista87.8% 72.2

%57.9% 82.3% 75.5% 59.9%

Using different segment of search results (AltaVista)

1st 1000 match–

Next 1000 match

91.1% 69.1%

60.0% 87.0% 70.9% 59.9%

Conclusion

• Usage consultation tool

　　 up-to-date expression, grammar check

• Totalize search engine results

• Multi-lingual & flexible entry

• String based candidate extraction and ordering

• Evaluation

Thank you!

Demonstration at ACL

(demo session & Univ.Tokyo booth)

Please come!