------ Speech and Language Processing An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition Second Edition Daniel Jurafsky Stanford University James H. Martin University of Colorado at Boulder PEARSON Prpntice Hall Upper Saddle River, New Jersey 07458
15
Embed
Second Edition and Language Processing An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition Second Edition ition Daniel Jurafsky Stanford
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
------
Speech and Language Processing An Introduction to Natural Language Processing,
Computational Linguistics, and Speech Recognition
Second Edition
ition
Daniel Jurafsky Stanford University
;econd Edition James H. Martin University of Colorado at Boulder
PEARSON
PrpnticeHall
Upper Saddle River, New Jersey 07458
Summary of Contents Foreword .......................•...................................xxiii Preface .......•..................................................... xxv About the Authors .xxxi
2.1.1 Basic Regular Expression Patterns . 18 2.1.2 Disjunction, Grouping, and Precedence 21 2.1.3 A Simple Example ..... 22 2.1.4 A More Complex Example . . . . . . . 23 2.1.5 Advanced Operators . 24 2.1.6 Regular Expression Substitution, Memory, and ELIZA 25
2.2 Finite-State Automata . 26 2.2.1 Use of an FSA to Recognize Sheeptalk 27 2.2.2 Formal Languages . 30 2.2.3 Another Example . 31 2.2.4 Non-Deterministic FSAs . 32 2.2.5 Use of an NFSA to Accept Strings . 33 2.2.6 Recognition as Search . 35 2.2.7 Relation of Detelministic and Non-Deterministic Automata 38
2.3 Regular Languages and FSAs 38 2.4 Summary . 41
ix
x Contents
Bibliographieal and Historieal Notes 42 Exereises 42
3 Words and Transducers 45 3.1 Survey of (Mostly) English Morphology 47
3.2 Finite-State Morphologieal Parsing 52 3.3 Construetion of a Finite-State Lexieon 54 3.4 Finite-State Transdueers . 57
3.4.1 Sequential Transdueers and Determinism 59 3.5 FSTs for Morphologieal Parsing 60 3.6 Transdueers and Orthographie RuJes 62 3.7 The Combination of an FST Lexieon and Rules 65 3.8 Lexieon-Free FSTs: The Porter Stemmer 68 3.9 Word and Sentenee Tokenization 68
3.9.1 Segmentation in Chinese . 70 3.10 Deteetion and Correetion of Spelling Errors 72 3.11 Minimum Edit Distanee . 73 3.12 Human Morphologieal Proeessing 77 3.13 Summary . 79 Bibliographieal and Historieal Notes 80 Exereises 81
4 N-Grams 83 4.1 Word Counting in Corpora 85 4.2 Simple (Unsmoothed) N-Grams . 86 4.3 Training and Test Sets . 91
4.3.1 N-Gram Sensitivity to the Training Corpus 92 4.3.2 Unknown Words: Open Versus C10sed Voeabulary Tasks 95
4.7.1 Advaneed: Details of Computing Katz Baekoff a and P* 107 4.8 Praetieal Issues: Toolkits and Data Formats . 108 4.9 Advaneed Issues in Language Modeling . 109
4.9.1 Advaneed Smoothing Methods: Kneser-Ney Smoothing 109 4.9.2 Class-Based N-Grams . 111 4.9.3 Language Model Adaptation and Web Use ..... 112
4.9.4 Using Longer-Distance Information: A Brief Summary. 4.10 Advanced: Information Theory Background . . . . . . . . . . .
4.10.1 Cross-Entropy for Comparing Models . 4.11 Advanced: The Entropy of English and Entropy Rate Constancy 4.12 Summary . Bibliographical and Historical Notes Exercises .
5 Part-of-Speech Tagging 5.1 (Mostly) English Word Classes 5.2 Tagsets for English . 5.3 Part-of-Speech Tagging . 5.4 Rule-Based Part-of-Speech Tagging . 5.5 HMM Part-of-Speech Tagging .. ,
5.5.1 Computing the Most Likely Tag Sequence: An Examp1e 5.5.2 Formalizing Hidden Markov Model Taggers 5.5.3 Using the Viterbi Algorithm for HMM Tagging 5.5.4 Extending the HMM Algorithm to Trigrams .
5.6 Transformation-Based Tagging .... 5.6.1 How TBL Rules Are Applied 5.6.2 How TBL Rules Are Learned
5.7 Evaluation and Error Analysis . 5.7.1 Error Analysis .
5.8 Advanced Issues in Part-of-Speech Tagging 5.8.1 Practical Issues: Tag Indeterminacy and Tokenization . 5.8.2 Unknown Words . 5.8.3 Part-of-Speech Tagging for Other Languages 5.8.4 Tagger Combination .
5.9 Advanced: The Noisy Channel Model for Spelling . 5.9.1 Contextual Spelling Error Correction
5.10 Summary . Bibliographical and Historical Notes Exercises .
6 Hidden Markov and Maximum Entropy Models 6.1 Markov Chains . 6.2 The Hidden Markov Model . 6.3 Likelihood Computation: The Forward Algorithm 6.4 Decoding: The Viterbi Aigorithm . 6.5 HMM Training: The Forward-Backward Algorithm 6.6 Maximum Entropy Models: Background
6.6.1 Linear Regression . 6.6.2 Logistic Regression . 6.6.3 Logistic Regression: Classification 6.6.4 Advanced: Learning in Logistic Regression
7.4 Aeoustie Phoneties and Signals 230 7.4.1 Waves .................. 230 7.4.2 Speech Sound Waves .......... 231 7.4.3 Frequeney and Amplitude; Piteh and Loudness 233 7.4.4 Interpretation of Phones from a Waveform 236 7.4.5 Speetra and the Frequeney Domain 236 7.4.6 The Souree-Filter Model ........ 240
8.3.3 Tune . 8.3.4 More Sophisticated Models: ToBI . 8.3.5 Computing Duration from Prosodie Labels 8.3.6 Computing FO from Prosodie Labels .... 8.3.7 Final Result of Text Analysis: Internal Representation
8.4 Diphone Waveform Synthesis . 8.4.1 Steps for Bui1ding a Diphone Database . 8.4.2 Diphone Concatenation and TD-PSOLA for Prosody
9.5 The Lexicon and Language Model 9.6 Search and Decoding . 9.7 Embedded Training . 9.8 Evaluation: Word Error Rate 9.9 Summary . Bibliographical and Historical Notes Exercises .
18.3.1 Store and Retrieve Approaches . 18.3.2 Constraint-Based Approaches .
18.4 Unification-Based Approaches to Semantic Analysis. 18.5 Integration of Semantics into the Earley Parser 18.6 Idioms and Compositionality 18.7 Summary........... BibJiographicaJ and Historical Notes Exercises .
19 LexicaI Semantics 19.1 19.2
19.3 19.4
19.5 19.6 19.7
Word Senses . Relations Between Senses . 19.2.1 Synonymy and Antonymy 19.2.2 Hyponymy . 19.2.3 Semantic Fields . WordNet: A Database of LexicaJ Relations Event Participants . 19.4.1 Thematic Roles . 19.4.2 Diathesis Alternations . 19.4.3 Problems with Thematic Roles . 19.4.4 The Proposition Bank 19.4.5 FrameNet . 19.4.6 Selectional Restrictions Primitive Decomposition Advanced: Metaphor Summary .
21.4.1 Five Types of Referring Expressions . 698 21.4.2 Information Status . 700
21.5 Features for Pronominal Anaphora Resolution 701 21.5.1 Features for Filtering Potential Referents 701 21.5.2 Preferenees in Pronoun Interpretation .. 702
21.6 Three Algorithms for Anaphora Resolution .... 704 21.6.1 Pronominal Anaphora Baseline: The Hobbs Algorithm 704 21.6.2 A Centering Aigorithm for Anaphora Resolution . . . 706 21.6.3 A Log-Linear Model for Pronominal Anaphora Resolution 708 21.6.4 Features for Pronominal Anaphora Resolution . 709
22 Information Extraction 725 22.1 Named Entity Recognition . 727
22.1.1 Ambiguity in Named Entity Recognition 729 22.1.2 NER as Sequence Labeling . 729 22.1.3 Evaluation of Named Entity Recognition 732 22.1.4 Practica1 NER Architectures . . . . . . . 734
22.2 Relation Detection and Classification . . . . . . . 734 22.2.1 Supervised Learning Approaches to Relation Analysis 735 22.2.2 Lightly Supervised Approaches to Relation Analysis 738 22.2.3 Evaluation of Relation Analysis Systems 742
23 Question Answering and Summarization 765 23.1 Information Retrieval . . . . . . 767
23.1.1 The Vector Space Model . . 768 23.1.2 Term Weighting . 770 23.1.3 Term Selection and Creation 772 23.1.4 Evaluation of Information-Retrieval Systems 772 23.1.5 Homonymy, Polysemy, and Synonymy 776 23.1.6 Ways to Improve User Queries . 776
25.2 Classieal MT and the Vauguois Triangle 867 25.2.1 Direct Translation . 868 25.2.2 Transfer . 870 25.2.3 Combined Direct and Transfer Approaches in Classic MT 872 25.2.4 The Interlingua Idea: Using Meaning 873
25.3 StatisticaJ MT . 874 25.4 P(FIE): The Phrase-Based Translation Model 877 25.5 Alignment in MT . 879
25.5.1 IBM Modell . 880 25.5.2 HMM AJignment . 883
25.6 Training Alignment Models 885 25.6.1 EM für Training Alignment Models 886
25.7 Symmetrizing Alignments for Phrase-Based MT 888 25.8 Decoding for Phrase-Based Statistical MT 890 25.9 MT Evaluation . 894
25.9.1 Using Human Raters . 894 25.9.2 Automatie Evaluation: BLEU 895
25.10 Advaneed: Syntaetie Models for MT . 898 25.11 Advaneed: IBM Model 3 and Fertility 899
25.1 1.1 Training for Model 3 ..... 903 25.12 Advaneed: Log-Linear Models for MT 903 25.13 Summary . 904 Bibliographical and Historieal Notes 905 Exercises . 907