A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation Michael Hausman A master’s project report submitted to the Graduate Facility of the University of Colorado at Colorado Springs in partial fulfillment of the requirements for the degree of Masters of Engineering in Software Engineering Department of Computer Science 2011
106
Embed
A Genetic Algorithm Using Semantic Relations for Word ...kalita/work/StudentResearch/Haus... · A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation Michael
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
A Genetic Algorithm Using Semantic
Relations for Word Sense Disambiguation
Michael Hausman
A master’s project report submitted to the Graduate Facility of the
University of Colorado at Colorado Springs
in partial fulfillment of the requirements for the degree of
Masters of Engineering in Software Engineering
Department of Computer Science
2011
This master’s project report for
Masters of Engineering in Software Engineering
degree by
Michael Hausman
has been approved for the
Department of Computer Science
By
_________________________ Jugal Kalita, Chair
_________________________ Edward Chow
_________________________ Al Brouillette
__________ Date
Hausman iii
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Abstract
Word Sense Disambiguation is a formal way of saying, “Which dictionary definition is correct in
context?” Humans are adept at extracting the context of a sentence and applying it to every word. If
the sentence is “I like to swim next to the river bank,” then the word “bank” means “sloping land” and
does not mean “a financial institution.” Humans know this because the words “river” and “swim” have
very little to do with finance. Humans have years of knowledge and experience to quickly contextualize
the meaning of every word. A machine, however, has a much harder time finding the correct meaning.
It takes thousands of computations for even the simplest algorithms, which are not very accurate. Even
so, many applications such as language translators are still available and sold today. Language
translation relies heavily on word sense disambiguation. For this reason many translated sentences do
not make much sense. Solving word sense disambiguation would help with many applications such as
language translation.
This project explores solutions to the word sense disambiguation dilemma. There are a variety
of tools such as WordNet and SemCor referenced in this project. WordNet is a lexical database that
investigates several relations for comparing words and glosses of a word. SemCor is a collection of text
tagged with the proper part of speech and definitions. This researcher uses the semantic relations from
WordNet and examples from SemCor to measure which definitions are most likely to be correct in the
context of the communication.
A genetic algorithm employs these measurements to find the optimal set of definitions across
several sentences. Then, the researcher compares the algorithm to other word sense disambiguation
algorithms.
Hausman iv
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Table of Contents
Abstract ............................................................................................................................................ iii
Table of Figures ............................................................................................................................... vii
Table of Tables ................................................................................................................................ viii
Table of Equations ............................................................................................................................ ix
Table of Examples .............................................................................................................................. x
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Table of Figures
Figure 1: An example graph indicating the various sample areas .............................................................. 41
Figure 2: The graph of the frequency semantic relation using a noun‐noun part of speech combination 42
Figure 3: The graph of the frequency semantic relation using an adjective‐adjective part of speech combination ................................................................................................................................................ 42
Figure 4: The graph of the hypernym semantic relation using a noun‐noun part of speech combination 44
Figure 5: The graph of the hypernym semantic relation using a verb‐verb part of speech combination .. 44
Figure 6: The graph of the coordinate sister semantic relation using a noun‐noun part of speech combination ................................................................................................................................................ 45
Figure 7: The graph of the coordinate sister semantic relation using a verb‐verb part of speech combination ................................................................................................................................................ 46
Figure 8: The graph of the domain semantic relation using a noun‐noun part of speech combination .... 47
Figure 9: The graph of the domain semantic relation using an adverb‐adverb part of speech combination .................................................................................................................................................................... 47
Figure 10: The graph of the synonym semantic relation using a verb‐verb part of speech combination . 48
Figure 11: The graph of the synonym semantic relation using an adjective‐adjective part of speech combination ................................................................................................................................................ 49
Figure 12: The graph of the antonym semantic relation using a noun‐noun part of speech combination 50
Figure 13: The graph of the antonym semantic relation using an adjective‐adjective part of speech combination ................................................................................................................................................ 50
Figure 14: The graph indicating the shape of the ideal cost function ........................................................ 51
Figure 15: The graph indicating the shape of Cost Function Method 1 ..................................................... 52
Figure 16: A graph with an example regression equation .......................................................................... 53
Figure 17: A graph indicating an example of the regression of multiple regressions for frequency ......... 54
Figure 18: The graph indicating the shape of Cost Function Method 2 ..................................................... 55
Figure 19: A graph showing an example of the weaknesses of multiple hypernym regressions ............... 56
Figure 20: A graph showing an example using Cost Function Method 3 ................................................... 57
Figure 21: The graph indicating the shape of Cost Function Method 3 ..................................................... 58
Figure 22: The graph indicating the shape of Cost Function Method 4 ..................................................... 60
Hausman viii
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Table of Tables
Table 1: The results from SemEval 2 ........................................................................................................... 73
Table 2: The results from SemEval 3 ........................................................................................................... 74
Table 3: The coarse results from SemEval 2007 ......................................................................................... 75
Table 4: The SemCor file letter indicating type of resource the original text came from. ......................... 79
Table 5: The various SemCor files ............................................................................................................... 79
Table 6: The values for Cost Function Method 3 used in this project ........................................................ 92
Table 7: The values for Cost Function Method 4 used in this project ........................................................ 94
Table 8: The values for Cost Function Method 5 used in this project ........................................................ 94
Table 9: The results of one run of every SemCor file using all the parts of speech.................................... 99
Hausman ix
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Table of Equations
Equation 1 and Example 1: Three Path‐Based Methods ............................................................................ 16
Equation 2 and Example 2: Three Information‐Based Methods ................................................................ 17
Equation 3: Cost function for Zhang’s solution ........................................................................................... 20
Equation 4: Semantic Relation Equation for Frequency ............................................................................. 28
Equation 5 and Example 4: Semantic Relation Equation and Example for Hypernyms ............................. 30
Equation 6 and Example 5: Semantic Relation Equation and Example for Coordinate Sisters .................. 31
Equation 7 and Example 6: Semantic Relation Equation and Example for Domain ................................... 31
Equation 8 and Example 7: Semantic Relation Equation and Example for Synonyms ............................... 33
Equation 9 and Example 8: Semantic Relation Equation and Example for Antonyms ............................... 34
Equation 10: The Weighted Probability Equation ...................................................................................... 38
Equation 11: The equation for Cost Function Method 2 ............................................................................ 54
Equation 12: The equation for Cost Function Method 3 ............................................................................ 57
Equation 13: The equation for sense distribution error ............................................................................. 59
Equation 14: The equation for Cost Function Method 4 ............................................................................ 59
Equation 15: The equation for semantic relation distribution error .......................................................... 61
Equation 16: The equation for Cost Function Method 5 ............................................................................ 61
Equation 17: The equation for comparing the current solution to the optimal solution ........................... 66
Hausman x
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Table of Examples
Equation 1 and Example 1: Three Path‐Based Methods ............................................................................ 16
Equation 2 and Example 2: Three Information‐Based Methods ................................................................ 17
Example 3: Example using the Frequency Semantic Relation Equation ..................................................... 29
Equation 5 and Example 4: Semantic Relation Equation and Example for Hypernyms ............................. 30
Equation 6 and Example 5: Semantic Relation Equation and Example for Coordinate Sisters .................. 31
Equation 7 and Example 6: Semantic Relation Equation and Example for Domain ................................... 31
Equation 8 and Example 7: Semantic Relation Equation and Example for Synonyms ............................... 33
Equation 9 and Example 8: Semantic Relation Equation and Example for Antonyms ............................... 34
Example 9: An example using the Weighted Probability Equation ............................................................ 38
Hausman 11
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Chapter 1: Introduction
This project explores word sense disambiguation. A word sense is a definition or meaning of a
word. Disambiguation serves to “remove all ambiguity.” Therefore, this research explores the technical
way of saying, “Which dictionary definition is correct in context?” For example, let us suppose that the
application is a GPS in a car that uses verbal locations. If the user says, “I want to go to the bank,” where
is the location? The noun “bank” has several meanings. Some of these possibilities are below. This
could be a financial institution to deposit a check. This could also be the river bank next to the house.
From this sentence alone, the GPS cannot tell the difference. It could just assume the most commonly
used meaning and point to the financial institution. However, if the next sentence is, “I want to go for
a swim,” then the assumption would be wrong. To understand which location is the destination, the
GPS would need to know the correct sense of the location. To understand which sense is correct, the
GPS needs to understand the context.
Possible senses of the noun “bank” (Princeton University, 2010) 1. a financial institution that accepts deposits and channels the money into lending activities 2. sloping land (especially the slope beside a body of water) 3. a supply or stock held in reserve for future use (especially in emergencies)) 4. a building in which the business of banking transacted 5. an arrangement of similar objects in a row or in tiers 6. a container (usually with a slot in the top) for keeping money at home 7. a long ridge or pile 8. the funds held by a gambling house or the dealer in some gambling games 9. a slope in the turn of a road or track; the outside is higher than the inside in order to reduce
the effects of centrifugal force 10. a flight maneuver; aircraft tips laterally about its longitudinal axis (especially in turning)
Word sense disambiguation is not as simple as it sounds. In order for a machine to understand
which sense is correct, it must correlate several things. Some of these are: A) how many senses are
possible for this word? B) How do those senses correlate to other words? and C) What is important in a
Hausman 12
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
sentence? In the preceding example, the machine would need to understand that swimming involves
water, that water implies sense 2, and the location of the desired body of water. If the GPS were to
accept any sentence, then the knowledge base required to connect every word would be immense. It
requires several years for humans to make sense and then making meaning of the simplest sentences.
To achieve this with a machine is significantly more complex.
The fictional GPS is not the only application of word sense disambiguation. If word sense
disambiguation were solved, then many applications would be possible. This could include computers
that actually talk and interact with humans as seen in science fiction movies. This also includes language
translators such as Google Translate (Google, 2010).
At present, many translated sentences do not make much sense or are hard to understand
because words are missing and the context is often incorrect. Just knowing the correct sense would
help convey the correct context after the translation. Other applications that benefit from word sense
disambiguation include text classification, automatic summaries, or anywhere text or language is
analyzed. This project solely focuses on the problem of word sense disambiguation.
The first step to word sense disambiguation is to understand how words relate within a specific
context. Humans understand the context of a word by looking at the surrounding words. Humans also
compare words in several different ways. For example, a human knows that a lake, a river, and the
ocean are all related because they are all bodies of water. Swimming is a water sport, so water must be
involved. Many people visit rivers that are near hills and mountains in order to enjoy the scenery.
Therefore, if the word “bank” were in the middle of the statement, then the correct sense of the word
bank must be 2. It probably is not a financial institution. Perhaps a machine could try the same
technique. The first step is to examine the surrounding words and compare them with each other. A
semantic relation allows the machine to compare two words in a specific way. For example, a hypernym
of a word is a more generic way of saying that word. A more generic way of saying “lake”, “river” and
Hausman 13
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
“ocean” is “body of water.” This means that “lake”, “river”, and “ocean” all share similarity through the
hypernym semantic relation. “Hillside” and the second sense of “bank” are also similar via a hypernym
semantic relation. Semantic relations provide a way for a machine to compare words with each other.
Exploring Semantic relations is the primary lens for this project. This exploration compares
several words. Earlier, the hypernym semantic relation showed some similarity between several words.
The words “lake”, “river”, and “ocean” were similar because their hypernym was “body of water.”
However, it takes more hypernyms to compare “hillside” and “bank.” This implies that the first set is
more related to each other than the second set. It also means that a portion of the project is examining
how to properly measure each relation to account for this varying similarity. Not all words have
hypernyms. To account for this, there are several semantic relations in this project: frequency,
hypernym, coordinate sister, domain, synonym, and antonym. Each of these semantic relations has
different ways of comparing two words.
As the title suggests, semantic relations are only part of the project. A genetic algorithm uses
these semantic relations to provide the senses for a given text. A genetic algorithm uses Darwin’s
theory of evolution to evolve a solution over time for optimization problems. The advantage of a
genetic algorithm is that it does not need to compare every possible solution. This is important because
there are billions of possible sense combinations for a paragraph of text. The disadvantage of this
method is that the solution may not be the best possible combination, but it should be a “good” solution
if the genetic algorithm is performed correctly.
Chapter One of this report has introduced the concept of word sense disambiguation. Chapter
Two describes several techniques which other researches use in word sense disambiguation. Chapter
Three describes all of the tools and resources that this project uses to provide all of the sense and
semantic relation information related to this study. Chapter Four explains what the semantic relations
are and how the project measures the similarity between two words with each relation. Chapter Five
Hausman 14
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
explains the genetic algorithm in this project. The major portion of this chapter explains how the
genetic algorithm transforms these measures to make word sense disambiguation an optimization
problem. Chapter Six compares the results to several researchers and competitions. Chapter Seven
concludes by suggesting possible areas for research in the future.
Hausman 15
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Chapter 2: Background
There are various ways to attempt word sense disambiguation. There are also various ways to
classify each attempt. One classification is supervised versus unsupervised. A supervised algorithm
depends on some training data to learn or compare information. The algorithm then uses that
knowledge to more accurately tag other text. An unsupervised system does not require any training
data. Typically the supervised approaches are more accurate if the training data are available.
However, this classification is extremely generic. Zesch and Gurevych classify four categories of
approaches in their paper: path‐based, information content based, gloss based, and vector based (Zesch
& Gurevich, 2010). These classifications give a very high‐level indication on how early researchers
approached the problem. Many of the newer researchers often combine several of these approaches to
achieve better results.
2.1 Path‐Based Approaches
Path‐based approaches measure the length of the path between two words. The shorter the
path, the more related the two words are. This method relies on a resource that supplies the paths.
This is usually a graph‐like structure, like WordNet (Princeton University, 2010). Many early researchers
focus on a single relation, like hypernyms, and optimize for that relation. What hypernyms are is not as
important at this point as the fact that hypernyms create a tree. Once the relation is in a tree structure,
then someone can measure the paths in the tree. Rada et al. begin by measuring the hypernym edges
between the words (Rada, Mili, Bicknell, & Blettner, 1989). Then, Leacock and Chodorow normalize the
length by accounting for the tree depth (Leacock & Chodorow, 1998). Wu and Palmer also use this idea
of depth in their equation (Wu & Palmer, 1994). The difference is that Wu and Palmer use the depth to
Hausman 16
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
the “lowest common subsumer,” which is where the hypernyms of two words intersect. This creates
three different measurements for the “hypernym tree” alone. An example and the two equations are
below. The equations are the main part of both Rada’s and Leacock’s methods. The majority of their
papers describe equations performance for various text examples. However the path is measured, a
path‐based method takes the measurement and optimizes for the shortest path between all words.
Equation 1 and Example 1: Three Path‐Based Methods
2.2 Information‐Based Methods
Information‐based methods take into account how much information the two words share. The
more information the two words share, the more similar the two words are. For example, start with the
hypernym tree structure from before. Just as before, the only important fact is that the relation creates
a tree structure. Since it is a tree structure, someone can measure the number of nodes that both
words share. The more nodes two words share, the more related the two words are. Resnik adapted
this idea and defines the “lowest common subsumer” as the point where the two words intersect in the
hypernym tree (Resnik, 1995). All the words from the top of the tree to the lowest common subsumer
are common subsumers. Resnik adds the probability of all the subsumers appearing in a corpus as the
Depth
w
w
w
w
w
w
w
w
W1
W2
LCS Depth
Length
𝑅𝑎𝑑𝑎(𝑤", 𝑤%) = 𝐿𝑒𝑛𝑔𝑡ℎ 𝑅𝑎𝑑𝑎(𝑤", 𝑤%) = 3
𝐿𝐶(𝑤", 𝑤%) = −log 𝐿𝑒𝑛𝑔𝑡ℎ2 ∗ 𝐷𝑒𝑝𝑡ℎ
𝐿𝐶(𝑤", 𝑤%) = 0.426
𝑊𝑃(𝑤", 𝑤%) =2 × 𝐿𝐶𝑆 𝐷𝑒𝑝𝑡ℎ
𝐿𝑒𝑛𝑔𝑡ℎ + 2 × 𝐿𝐶𝑆 𝐷𝑒𝑝𝑡ℎ
𝑊𝑃(𝑤", 𝑤%) = 0.727 Length: length of the path between the words Depth: the longest depth to the words LCS Depth: the depth to the lowest common
subsumer
Hausman 17
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
measure of similarity. Jiang and Conrath modify Resnik’s idea to include the distance between the word
and the lowest common subsumer (Jiang & Conrath, 1997). Lin begins with Jiang’s hypothesis and uses
the universal measure from information theory instead (Lin, 1998). All three of these measures use the
same idea, which is adding the probability of a word appearing in a corpus. The difference is in the
equations. An algorithm depending on the amount of shared information indicates an information‐
based approach.
Equation 2 and Example 2: Three Information‐Based Methods
2.3 Gloss Based Methods
Gloss‐based methods rely on the definition of the word. Dictionaries describe definitions with
different words. A dictionary will describe two similar words with other similar words. Keep track of all
the similar words between two senses, and the overlap becomes a measurement. Lesk is famous for
using this concept to tell a “pine cone” from an “ice cream cone” (Lesk, 1986). The words “pine” and
“cone” both have definitions that contain the words “tree” and “fruit.” This is a completely different
overlap than the senses of “ice cream” and “cone.” Simply pick the senses that provide the most
overlap with each other. There are, however, drawbacks to this method. If the glosses are not
descriptive enough, or if there are many false positives, this method will fail. Banerjee and Pederson
attempt to account for both problems with their Adapted Lesk Algorithm (Patwardhan, Banerjee, &
and attribute. Gloss‐based methods depend on the overlap between the glosses of two words.
2.4 Vector Based Methods
Vector‐based methods take each individual measurement and represent it as a vector. The
cosine of the angle between the vectors is an indication of how related the two concepts are. If the
angle is really large, then the two measurements are not very related. If the angle is very small, then the
two measurements are related. The best example given by Zesch (Zesch & Gurevich, 2010) is from
Patwardahan et al. (Patwardhan, Using WordNet‐Based Context Vectors to Estimate the Semantic
Relatedness of Concepts, 2006). They start with two words and find all of the glosses that match using
the Adapted Lesk Algorithm (Patwardhan, Banerjee, & Pedersen, An Adapted Lesk Algorithm for Word
Sense Disambiguation Using WordNet, 2002). Then they say that each word is a dimension. Each vector
is a gloss of the one of the relations from either word. The angle of the vector is the number of words
that overlap in the glosses of the Adapted Lesk Algorithm. The strength of this concept is that every
relation turns into a vector. Since every relation is a vector, it is easy to compare two words. However,
turning everything into a vector is a problematic task.
2.5 Using Multiple Approaches
Most of the preceding examples from early researchers have the same thought, especially in the
development of path‐based and information‐based methods. For each word, compare the word with
the surrounding words within one equation. The sense with the highest score is selected. Many of the
later researchers start to attempt using several approaches. The example cited here precedes the
Hausman 19
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Adapted Lesk Algorithm (Patwardhan, Banerjee, & Pedersen, An Adapted Lesk Algorithm for Word Sense
Disambiguation Using WordNet, 2002). This method starts with Lesk and expands on it by using several
semantic relations. One of these semantic relations, hypernyms, is the main relation in the path and
information examples. Patwardahan et al. even tried to combine Adapted Lesk with the earlier
information‐based methods (Patwardhan, Banerjee, & Pedersen, Using Measures of Semantic
Relatedness for Word Sense Disambiguation, 2003). They found that the equation from Jiang and
Conrath worked best in their situation (Jiang & Conrath, 1997). Then they moved on to the vector‐
based method described above, which starts with Adapted Lesk. All of these examples yield better
results with the combination of approaches.
Basile et al. realize that every part of speech has different relationships with other parts of
speech (Basile, Degemmis, Gentile, Lops, & Semeraro, 2007). A changing a noun has a different effect
on another noun than on a verb for a given relation. Also, one approach may work better on a specific
part of speech. With this in mind, Basile et al. use a different approach depending on the part of speech.
Nouns use a modified Leacock measure as a starting point (Leacock & Chodorow, 1998). Their
algorithm uses an extra Gaussian factor for the distance between the words in the disambiguation text
and a factor for the frequency in WordNet. Verbs use a similar approach, except with a different
Gaussian factor. Adjectives and Adverbs use the Adapted Lesk Algorithm (Patwardhan, Banerjee, &
Pedersen, An Adapted Lesk Algorithm for Word Sense Disambiguation Using WordNet, 2002). The
underlying concept of this application is that using a different approach for every part of speech can
improve the overall score.
2.6 A Genetic Algorithm Approach
This paper employs a genetic algorithm, and therefore it follows to investigate other proposed
genetic algorithm approaches as well. The genetic algorithm approach for word sense disambiguation
Hausman 20
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
using the Zhang et al. formula is the conceptual base for this paper [15]. Zhang and fellow researchers
focus only upon nouns, but their approach can easily apply to other parts of speech.
This paper also focuses on the most important part of a genetic algorithm: the cost function.
The cost function determines how the genetic algorithm compares solutions. If a genetic algorithm
cannot compare solutions correctly, then the rest of the algorithm is irrelevant. Zhang explains the core
of the algorithm in two sentences. These two sentences prove that the approach is extremely simple
beyond the cost function. The cost function begins with the hypernym tree equation from Wu and
Palmer (Wu & Palmer, 1994). This equation is an input to an equation that accounts for the domain of a
word. The domain is the word or collection to which a word belongs. After this, the results are
weighted based on the frequency of the word. The frequency comes from statistics found in WordNet
(Princeton University, 2010). The significance of this cost function is that it relies on several semantic
relations at the same time.
Equation 3: Cost function for Zhang’s solution
𝐶𝑜𝑠𝑡(𝑤",𝑤%)= ;𝑆𝑒𝑛𝑠𝑒𝐶𝑛𝑡(𝑤")𝑇𝑜𝑡𝑎𝑙𝐶𝑛𝑡(𝑤")
+ 𝑆𝑒𝑛𝑠𝑒𝐶𝑛𝑡(𝑤%)𝑇𝑜𝑡𝑎𝑙𝐶𝑛𝑡(𝑤%)<
∗ =1 +𝑊𝑃(𝑤",𝑤%)
2 , 𝐷𝑜𝑚(𝑤") = 𝐷𝑜𝑚(𝑤%)𝑊𝑃(𝑤",𝑤%)
2 , 𝑂𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
SenseCnt(w): The number of times this was referenced in WordNet corpus TotalCnt(w): The total number of references for this word in WordNet corpus SenseTotal(w): The total number of senses for this word
Sense(w): The sense number of the word currently in use Dom(w*): the domain of the word given
Hausman 21
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
2.7 Main Ideas behind the Approach in this Project
There are a variety of ways to solve word sense disambiguation. To combine every possible
approach is not the best option. However, this author has chosen to investigate examples from various
sources. Zhang’s idea of using several semantic relations in a genetic algorithm is very appealing
(Zhang, Zhou, & Martin, 2008). However, it may make more sense to use more semantic relations. It
also may help to examine each relation independently before combining them into a large cost function
equation. Many of those semantic relations should probably begin with equations that other
researchers have developed, such as the hypernym equation from Wu and Palmer (Wu & Palmer,
1994). It is possible that some of these relations correlate better than others, so adjusting for this may
make a more accurate solution. Basile’s idea of separating the parts of speech also seems cogent
(Basile, Degemmis, Gentile, Lops, & Semeraro, 2007). For example, there are only hypernyms of nouns
and verbs. It is for this reason that Zhang only looks at nouns in their solution. The melding of the
preceding concepts provides the starting point of this project.
Hausman 22
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Chapter 3: Tools/Resources
This project uses a variety of tools and resources that are referenced throughout the project.
The tools and resources include: WordNet, a C# interface to WordNet, a part of speech tagger from
Tokyo University, SemCor, and SemEval. Some of these tools, WordNet for example, provide all the
definitions and relations. Some resources, like SemCor, provide examples of correctly translated text.
The sections below explain what each tool/resource is and how this project uses them.
3.1 WordNet
WordNet is a publicly available lexical database developed by Princeton University (Miller). It
only defines nouns, verbs, adjectives, and adverbs. There are 206941 words across 117659 SynSets,
which are groups of synonyms, in WordNet 3.0. This means that there are 117659 unique definitions
available. Each of these SynSets relates to other SynSets in various ways, hence the “net” in WordNet.
This project uses many of these relations and definitions to disambiguate text. These relations include
disport, sport, cavort, gambol, frisk, romp, run around, lark about
𝑆𝑦𝑛(𝑤",𝑤%) = G1, 𝑆𝑦𝑛(𝑤") ∋ 𝑤%0, 𝑂𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒 w1 and w2: the two words being
compared Syn(w*): all of the synonyms of the
word Syn(w1) ∋ w2: if w2 is any of the
synonyms of w1
Hausman 34
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Equation 9 and Example 8: Semantic Relation Equation and Example for Antonyms
1. good, goodness a. evil, evilness
2. pure a. defiled, maculate
3. vague a. clear, distinct
𝐴𝑛𝑡(𝑤",𝑤%) = G1, 𝐴𝑛𝑡(𝑤") ∋ 𝑤%0, 𝑂𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒 w1 and w2: the two words being
compared Ant(w*): all of the antonyms of the
word Ant(w1) ∋ w2: if w2 is any of the
antonyms of w1
Hausman 35
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Chapter 5: A Word Sense Disambiguation Genetic Algorithm
Word Sense Disambiguation for all words in a file creates a large solution space. For example,
suppose that there exists a file with five paragraphs of information. Each of these paragraphs could
have 100 words, which means the file has 500 words. Each word can have many definitions, so assume
there are five definitions per word. This makes 5100 = 7.888 * 1069 possible word sense combinations for
a paragraph or 5500 = 3.055*10349 word sense combinations for the entire file. This solution space is far
too large to check every word sense combination. If there is a way to measure the accuracy of a word
sense combination, genetic algorithms provide a way to check a subset of these solutions in order to
obtain a good solution. This solution will not be “the solution,” but the solution should be close.
The genetic algorithm is based on Darwin’s theory of evolution to solve optimization problems.
The general idea is to start with a set of solutions, using that set of solutions to make better solutions,
and only keeping the best solutions. In genetic algorithms, chromosomes define all of the information
necessary to define a solution. The individual pieces of the chromosome are genes that define a certain
aspect of the solution. In this case, the chromosome defines the word senses for the file. Each gene
represents the chosen word sense for a single word. The generic process of creating “better”
chromosomes, or solutions, with genetic algorithms is below.
1. Start with a set of solutions (1st Generation) 2. Take original “parent” solutions and combine them with each other to create a new set of
“child” solutions (Mating) 3. Introduce some random changes in case solutions are “stuck” or are all the same (Mutation) 4. Somehow measure the solutions (Cost Function) to evaluate the best solution 5. Repeat starting with Step 2
Hausman 36
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Genetic Algorithm implementations can vary in many respects. The preceding steps describe
the most common case at a very high level. Most genetic algorithms have three main parts. They are:
mating, mutation, and cost function. The preceding steps above do not describe how to implement
mating, mutation, and cost function. The implementation of these portions has different strengths and
weaknesses depending upon the problem. It also does not describe how to select chromosomes for
each step. One should expect the implementation to change for every problem, and most authors will
have a cost function, a mating phase, and a mutation phase as part of their genetic algorithm.
5.1 Evolution of the Cost Function
The cost function is possibly the most important part of a genetic algorithm. It is the portion of
the algorithm that determines which solution is the “correct” or “better” solution. How to determine
the best answer depends upon the problem and on the approach. If the cost function’s approach
incorrectly compares solutions, then the genetic algorithm will lead to an increasingly incorrect answer.
For example, if the problem is to find the highest point on a mountain, the cost function needs to
correctly determine which point is higher in elevation. Assume that the cost function compares two
areas according to the slope of the hillside under the assumption that the slope on the top of the
mountain is zero. In this case, the genetic algorithm will not be able to correctly compare two
mountains or plateaus on the map. In addition, it does not matter how well the rest of the genetic
algorithm functions because the results will most likely be incorrect. If the genetic algorithm is to find a
“good” answer, then the cost function must be able to determine which answers are correct in most
cases.
For this project, the cost function must be able to compare the definitions of a word and
determine which definition is most likely to be correct. All of the preceding semantic relation equations
are different measurements that can be part of the cost function. Each of these measurements has its
strengths and weaknesses. A large part of this author’s project involves examining these measurements
Hausman 37
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
and determining how to use them most accurately. The sections below begin by examining each
semantic relation. The author then incorporates them into a cost function and analyzes the results. This
leads to several variations of the cost function using the knowledge and results of the last cost function.
The last cost function is the one the genetic algorithm uses.
5.1.1 Notes on Sampling
Earlier in this report, the author explored SemCor and WordNet. The SemCor files list the
correct WordNet senses for the given text. Since this project needs to compare incorrect senses with
correct senses, there is a need for many other possible sense combinations. Throughout this project, a
solution is a specific selection of sense combinations. The number of senses that match the senses in
the SemCor file indicate how “good” the solution is. With that in mind, the SemCor selection is the best
solution, and the worst solution matches none of the SemCor senses. The author has noted a variety of
solutions and several levels of correctness.
The problem surfaces when examining the number of possibilities. Each SemCor file contains
approximately 2000 words. Of these, 1100 to 1300 words have a WordNet definition. These definitions
may be a noun, verb, adjective, or adverb. Assuming that the part of speech is already identified for
each word, each of these defined words has the following average number of definitions: noun 1.24,
verb 2.17, adjective 1.4, and adverb 1.25. If every defined word were a noun, the number would be
1.241200 = 1.2765*10112 possible combinations per SemCor file. If there were an equal distribution for
each part of speech, this would be 1.5151200 = 3.1272*10216 possible combinations per SemCor file. This
is an inordinate number of combinations. To further clarify this, imagine that if a solution were
calculated every nanosecond; it would require 9.9164*10199 years to investigate each combination. It
follows that a subset of solutions must represent all of the solutions. The selected subset areas are:
1. Near the correct solution 2. Randomly picked solutions 3. Near the most frequent senses
Hausman 38
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
4. Random solutions weighted towards the most frequent senses 5. Min/Max areas for each semantic relation via a Mutation method
The author has selected the five primary areas to represent all of the possible solutions. Each
area is a starting point. Every solution begins in one area and randomly changes a percentage of the
word senses toward another area. The first area is near the correct solution. These answers give insight
regarding how to reach the correct solution. The second area contains randomly selected senses to
provide variety. The third area is located near the most frequent senses since the most frequent sense
is a common baseline. The fourth area contains “weighted” randomly selected solutions. This provides
answers between the randomly selected area and the most frequent area. The weighted random
equation and an example are formulated below. The last area describes the boundaries of each
individual semantic relation. A function that eventually became one of the mutation functions defines
these boundaries by building a solution to match the given value for a semantic relation. (See the
mutation section for further explanation.) Using 3000 solutions and using these five areas gives variety
and focus to address most of the cost function behavior.
w and s: the word and sense TotalSense(w): the number of senses for this word
Equation 10: The Weighted Probability Equation
If the word has 4 senses, the probability of returning the 1st sense is:
Example 9: An example using the Weighted Probability Equation
Hausman 39
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
5.1.2 Semantic Relation Investigation
Studying semantic relations behavior is the first step to making the cost function. Each semantic
relation is a small part of the cost function. The behavior of the semantic relations in this section has a
major influence on the cost function variations in the following sections. Therefore, the coverage and
representation for each semantic relation is crucial to the process. The variety and boundaries of the
semantic relation indicate where and how each semantic relation influences the solution. This includes
the parts of speech which the semantic relation represents. To illustrate, hypernyms are available solely
for nouns and verbs in WordNet, and therefore the relation is useless when comparing adjectives and
adverbs. In addition, the hypernym trees for nouns do not overlap with hypernym trees for verbs. For
this reason the results for every part of speech combination is unique within each semantic relation.
The cost function can then account for each semantic relation and part of speech combination
separately.
A second thing to note is that each semantic relation compares two words in order to return a
value. There are several possible word pair combinations in the given text. The challenge is to
determine which combinations are important. Various researchers try different techniques. Zhang finds
every word combination in a paragraph (Zhang, Zhou, & Martin, 2008). The idea is that a single
sentence does not contain enough information to properly determine the correct word sense with
semantic relations. A paragraph would have more information because it has several related sentences.
Zhang’s solution contains one drawback. Very large paragraphs have an extremely large number of
combination pairs. In addition, the SemEval competitions do not indicate where the paragraph
boundaries are located. The competitors in many recent SemEval competitions use a sliding window
technique for this reason (Patwardhan, Banerjee, & Pedersen, UMND1: Unsupervised Word Sense
Disambiguation Using Contextual Semantic Relatedness, 2007) (Bosch, Hoste, Daelemans, & Den, 2004)
(Mihalcea & Csomai, 2005). The first step is to remove all the words that don’t have definitions in
Hausman 40
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
WordNet. Then use all the combinations “n” words before and after the target word. The number of
words before and after varies from three words to five words in many papers. This project checks for
words ten words before and after the target word. This is a compromise between a paragraph and the
sliding window technique. Each word has a window with 45 possible combinations while a paragraph
can easily have thousands of possible combinations. Note that each combination uses the semantic
relation equations below to return a value. The overall semantic relation behavior is the average of all
the individual values.
One of the most transparent ways to view behavior is by means of a graph. With graphs of the
semantic relation, it is easier to see the estimated solution space of the relation, the boundaries of the
relation results, the variety of solutions that give a specific result from a semantic relation equation, and
the coverage. Therefore, a scatter plot with 3000 different solutions is available for every semantic
relation. These graphs have the semantic relation as the independent variable. The accuracy of a
solution is the independent variable. In several graphs, each of the individual sample areas is visible.
The graph below uses frequency and indicates the location of five sample areas. The sections below do
not show the individual areas. However, all these areas are very important for finding the boundaries of
each semantic relation.
Hausman 41
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Figure 1: An example graph indicating the various sample areas
5.1.2.1 Frequency Behavior
The graphs for two part of speech combinations for frequency are included below. All part of
speech combinations have approximately the same shape. The lowest frequencies have very low
accuracy, but more than zero. This implies that some words only have one definition or use the least
frequent definition. The highest frequencies have ~75% accuracy. This solution would be the typical
“baseline” solution recognized by many authors. The area between them contains a distinct point
where a small range of frequencies contain “good” solutions. However, this small range of frequencies
also contains the widest range of accuracies. This means that it will take more than frequency to sort
out the subset of solutions in this range. The five sample areas are clear and appear to cover several
boundaries of the semantic relation. All in all, frequency looks like a promising semantic relation as an
individual measurement.
1) Correct Area
5) Min Mutation Area
4) Weighted Random Area
2) Random Area
3) Most Frequent Area
5) Max Mutation Area
Average Solution Value
Hausman 42
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Figure 2: The graph of the frequency semantic relation using a noun‐noun part of speech combination
Figure 3: The graph of the frequency semantic relation using an adjective‐adjective part of speech combination
Distinct Maximum
Large Range at Correct Solution
Distinct Maximum
Large Range at Correct Solution
Hausman 43
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
5.1.2.2 Hypernym Behavior
There are only two part of speech combinations that work for hypernyms: comparing two nouns
or comparing two verbs. In both cases the sample areas are not as distinct. The noun‐noun
combinations appear to have more solutions on the left side of the correct solution. This may explain
why many researchers who use hypernyms can achieve greater accuracy by maximizing hypernyms
scores. The bulk of the solutions have a lower hypernym score and other researchers don’t access these
exceptions very often. In either case, the noun‐noun hypernym graph has a distinct point around the
correct solution near the middle of the range. The verb‐verb hypernym graph does not display the same
distinct point that the noun‐noun hypernym displays. It appears to be inversely proportional to
accuracy. This implies that actions are not as related as the subjects. Both graphs indicate that there is
a relation between hypernyms and accuracy. Both graphs also indicate that the score around the
correct solution has the widest range of accuracies. Of the two graphs, the noun‐noun version looks
more useful.
Hausman 44
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Figure 4: The graph of the hypernym semantic relation using a noun‐noun part of speech combination
Figure 5: The graph of the hypernym semantic relation using a verb‐verb part of speech combination
Distinct Maximum
Very Large and Dense Range at Correct Solution
Distinct Maximum
Very Large and Very Dense Range at Correct Solution
Hausman 45
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
5.1.2.3 Coordinate Sister Behavior
Both coordinate sisters and hypernyms rely on the same hypernym tree, so both of them also
have the same part of speech limitations. Coordinate sisters can only compare two nouns or two verbs.
In this case, the graphs are somewhat similar. The noun‐noun combination has a distinct point around
the correct solution in the middle of the coordinate sister range. The verb‐verb version has the distinct
point near the beginning of the range. The verb‐verb graph indicates the correct solution is not as close
to zero as the hypernym graphs. Both graphs show that the area around the solution has a majority of
the solutions and the widest range of accuracies. Overall, the coordinate sister results are slightly
different than the hypernym results, which may prove useful later on.
Figure 6: The graph of the coordinate sister semantic relation using a noun‐noun part of speech combination
Distinct Maximum Very Large and Very
Dense Range at Correct Solution
Hausman 46
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Figure 7: The graph of the coordinate sister semantic relation using a verb‐verb part of speech combination
5.1.2.4 Domain Behavior
Few words have a domain in WordNet. This fact is evident in the graphs for domain because the
possible values show up as columns. Perhaps a different SemCor file that is more domain specific may
have a larger variety of results, but br‐g23 only has a few. A majority of the given solutions have an
average domain value below 0.001. In addition, the correct solution is not unique among the given
solutions. This makes it very hard to see a correlation between domain and accuracy if one indeed
exists. Overall, domain does not look promising at this point. At least it does not look promising for this
SemCor example.
Distinct Maximum
Very Large Range at Correct Solution
Hausman 47
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Figure 8: The graph of the domain semantic relation using a noun‐noun part of speech combination
Figure 9: The graph of the domain semantic relation using an adverb‐adverb part of speech combination
Not Very Distinct Maximum
Very Large Range at Correct Solution
Not Very Distinct Maximum
Very Large Range at Correct Solution
Hausman 48
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
5.1.2.5 Synonym Behavior
Synonyms show similar issues that domain did, except on a lesser scale. There simply are not
enough synonyms in the br‐g23 SemCor file to provide a variety of results. In the verb‐verb example
below, a large majority of the answers have approximately the same two values. The larger group
happens to contain the correct answer. There is a distinct maximum point, but this is a 50% chance in
the first place. As for the adjective‐adjective example, it does have more than two solutions. However,
it shows very little correlation between synonyms and accuracy. It is unlikely that synonyms will be
useful in the cost function later on.
Figure 10: The graph of the synonym semantic relation using a verb‐verb part of speech combination
Distinct Maximum
Very Small Variation
Hausman 49
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Figure 11: The graph of the synonym semantic relation using an adjective‐adjective part of speech combination
5.1.2.6 Antonym Behavior
If synonyms do not look very promising, then antonyms will probably have the same results.
The two are extremely similar in nature, so each of them should have similar strengths and weaknesses.
Antonyms do not have sufficient variety in the results to be reliable. None of the solutions have an
antonym average above 0.005! There is no distinct point near the correct solution. The correlation
between antonyms and accuracy is very hard to determine. Overall, antonyms will perform as badly as
synonyms and will probably not be useful in the cost function.
Not Very Distinct Maximum
Very Small Correlation
Hausman 50
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Figure 12: The graph of the antonym semantic relation using a noun‐noun part of speech combination
Figure 13: The graph of the antonym semantic relation using an adjective‐adjective part of speech combination
Very Small Variation
Very Small Correlation
Not Very Distinct Maximum
Very Small Correlation
Hausman 51
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
5.1.3 The Optimal Cost Function
At this point there are several semantic relation equations and graphs. The author’s plan is to
use those equations in such a way so that the best solutions have the highest costs while the lower
accuracy solutions have lower costs. The goal of the genetic algorithm is to aim for the highest cost
solution in every generation. Ideally, the highest cost possible will be the correct solution and nothing
but the correct solution. A different way of describing it is, “The accuracy of the solutions is
proportional to the cost of the solution with little or no variance near the maximum solution.” The
scatter plot of the cost functions should resemble the one below.
Figure 14: The graph indicating the shape of the ideal cost function
5.1.4 Cost Function Method 1: Simple Addition
The easiest way to create a cost function is to add the average values of each semantic relation.
This provides a good starting point. It is not, and was never intended to be, the final cost function. It
serves to determine how much the cost function needs to change in order to make an optimal cost
Max Cost is Correct Solution
Very Little Variation in Accuracy near
Max Cost
Hausman 52
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
function. This provides a baseline to determine whether the future solutions actually improve beyond
the simplest solution. The resulting graph is below.
Figure 15: The graph indicating the shape of Cost Function Method 1
The result is not ideal. The highest cost solution is not the correct solution, but variance at this
highest point is small. The result takes the same shape as the frequency semantic relation. This is not
surprising considering the fact that frequency has the highest average values by a significant amount.
This also means that the answer this cost function converges to is the most frequent sense. Considering
this is the most common baseline, this answer is not an acceptable one.
5.1.5 Cost Function Method 2: Regression
Thorough examination of the semantic relations will reveal three main points. The maximum
score in all the semantic relations is not the correct answer. For example, the maximum frequency
score gives an accuracy of ~75%. If each semantic relation had the maximum point as the correct
solution, then the addition of the semantic relations should be the correct solution. Some semantic
Maximum Cost, ≈75% Accuracy
Correct Answer Not the Maximum Cost
Hausman 53
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
relations appear to relate to accuracy better than others. Antonyms, for example, do not appear to
have an effect on accuracy. Those semantic relations which do not relate as well have a higher chance
of influencing the cost function to an incorrect solution. The semantic relations vary a little between
different SemCor files. A good cost function would need to account for all of these points in order to
succeed.
One possibility is polynomial regression. Take the semantic relation values and accuracies and
fit a 6th order polynomial to the result. Then use that equation to transform the average semantic
relation value. This would make the maximum value at or close to the correct answer. To account for
the multiple SemCor files, simply make a regression of the regressions. Then this average regression
would account for as many of the SemCor files as possible. Multiply by the R2 value of this average
regression to adjust for semantic relation variance. A step by step example is below.
1. Take several samples for one SemCor file 2. Find the regression for one semantic relation.
Figure 16: A graph with an example regression equation
Hausman 54
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
3. Find the regression of regressions of that semantic relation from multiple SemCor files.
Figure 17: A graph indicating an example of the regression of multiple regressions for frequency
4. Use the R2 value as a multiplier for the semantic relation 5. Calculate the total weighted cost with the following equation. It adds the transformed semantic
relation value for every word combination.
An examination of the resulting cost function is better, according to the following graph. The
correct solution has a higher cost than the first method, but it still is not the highest. There appear to be
S: The current solution Word1 and Word2: Every word in the solution SemRel: Every semantic relation SemRelEqn: Use the words in the correct semantic relation equation Avg: Find the average value then transform with the regression equation
Equation 11: The equation for Cost Function Method 2
Hausman 55
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
several answers that have a higher cost. Also, there is a large variance near the maximum with as low as
65% accuracy. This method remains insufficient.
Figure 18: The graph indicating the shape of Cost Function Method 2
5.1.6 Cost Function Method 3: Proportional Placement in a Range
The regression for some semantic relations, such as frequency, work fairly well. However,
regression has its weaknesses. Any extraneous points tend to have a large influence on the regression.
The maximum value of the curve does not match the correct answer. In most cases it only varies a
slightly as in Figure 18.
The results between the SemCor files make the difference here. Hypernyms, for example, do
not always have the same range of values for every SemCor file as shown in the graph below. The graph
illustrates some of the files that have the best R2 values for the hypernym noun‐noun regression. The
correct hypernym value for one SemCor file can be the worst answer for another file. In this case, 0.26
is the best answer for br‐g16, and it is the worst answer for br‐e30. This skews the cost function despite
Correct Answer Near Maximum Cost
Maximum Cost, Several Accuracies
Hausman 56
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
the fact that the original regressions fit better than most of the semantic relations. As a result of this
skewing, one might determine that something other than regression works better for hypernyms.
Figure 19: A graph showing an example of the weaknesses of multiple hypernym regressions
The graph above illustrates that regression between files fails. It also implies a trend about
hypernyms. The ranges change, but the correct answer is always at or near the middle of that range.
For frequency, the correct answer is closer to the maximum value of the range. In the preceding
samples, one of the areas is the boundaries of a semantic relation. The function that locates these
boundaries is already available, so it is possible for the genetic algorithm to find the range. Since
dynamically finding the range is not an issue, an alternate possibility is to determine what proportion in
that range is typically correct. Then an equation based on these proportions will provide a better
transformation than would regression. The steps and example below illustrate how to find these
equations.
1. Take several samples for one SemCor file 2. Record the min, max, and ratio of the correct answer for each Semantic Relation
Correct Solution Varies Same Cost, Opposite Effect Between Files
Hausman 57
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
3. Find the average and standard deviation of the correct solution ratio for each semantic relation and part of speech combination from multiple SemCor files
4. Use four lines to transform the results. The lines connect five crucial points: the min, the max, the average of the correct solution, and the two points twice the standard deviation away from the average. Note that the genetic algorithm will find the range dynamically. A graphical example is below.
Figure 20: A graph showing an example using Cost Function Method 3
5. Calculate the total cost with the following equation. It adds the transformed semantic relation value for every word combination.
S: The current solution Word1 and Word2: Every word in the solution SemRel: Every semantic relation SemRelEqn: Use the words in the correct semantic relation equation Avg: Find the average value then transform with the proportional equation
Equation 12: The equation for Cost Function Method 3
(Avg, stdDev2)
(Avg – 2*stdDev, 0.9*stdDev2)
(Min, 0) (Max, 0)
(Avg + 2*stdDev, 0.9*stdDev2)
Hausman 58
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Both regression and this technique find the most likely values that contain the correct solution.
The difference is that this technique takes into account the changing range between the SemCor files.
As such, this method has fewer solutions that have a higher score than the correct solution, as shown
below. Just like regression, however, there is a large variance in accuracies around the highest cost.
More exploration is necessary to eliminate these false positives.
Figure 21: The graph indicating the shape of Cost Function Method 3
5.1.7 Cost Function Method 4: Add Sense Distribution
The method above is very promising. Its main weakness is that the highest cost can be a variety
of answers, ranging anywhere from a very inaccurate answer to the correct answer. All of these
answers provide average semantic relation scores that are in the optimum ranges. However, the lower
accuracy answers have a different sense distribution than the correct answer. They contain a different
Correct Answer Near the Maximum Cost
Maximum Cost, Several Accuracies
Hausman 59
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
number of senses that use the first sense and a different number of senses that use the second sense.
The steps below describe how to find and apply these distributions.
1. Take several SemCor files and find the average percentage of each sense 2. Find the error using the following equation. It adds the error for each sense.
3. Multiply the proportional range weighted cost function with this error value
The premise behind this additional measurement is to force high cost solutions to have optimal
semantic relation scores and the proper sense distribution. As a result, many of the answers that did
not have the proper sense distribution have lower scores. This separates a large part of the variation at
the maximum cost function score, as shown below. The correct solution is also close to the maximum
score. Overall, this is much closer to the optimal cost function shape than many of the previous
techniques. This looks promising when one assumes that the subset of examples matches the real
S: The current solution Sense: The sense number SenseCnt: Find the current number of the given sense TotalCnt: The expected number of the given sense Abs: Absolute value
Equation 13: The equation for sense distribution error
S: The current solution Word1 and Word2: Every word in the solution
SemRel: Every semantic relation SemRelEqn: Use the words in the correct semantic relation equation Avg: Find the average value then transform with the proportional
equation Equation 14: The equation for Cost Function Method 4
Hausman 60
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
solution space. At this point in the project, the author began to investigate the rest of the genetic
algorithm.
Figure 22: The graph indicating the shape of Cost Function Method 4
5.1.8 Cost Function Method 5: Add Semantic Relation Distribution
Despite what the sample solutions show, the genetic algorithm finds solutions that have better
scores than the correct solution. However, these solutions do not exhibit the same accuracy as the
correct solution. These are select solutions that are not part of the sample space and are “optimal” as
far as the cost function is concerned. The last modification accounts for the sense distribution, which
helps remove unwanted solutions such as these. Applying the same concept on the semantic relation
level may remediate these new solutions. The steps for applying this concept are below.
1. Take several SemCor files and find the average semantic relation value for each sense
Correct Answer Near the Maximum Cost Maximum Cost,
Less Accuracy
Hausman 61
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
2. Find the error using the following equation. It adds the error for each sense.
3. Multiply the semantic relation by this error when calculating the cost function
The concept of applying the distribution on the semantic relation level helps, but not
significantly. Some experiments indicate that this improves the accuracy by only 1% to 2%, a small
percentage compared with the error. This implies that the semantic relation distribution does not
matter as much or that the rest of the cost function already accounts for most of these solutions. At the
very least, this last modification lacks what is necessary to account for the incorrect solutions which the
SemRel: The current semantic relation Sense: The sense number SemVal: The average semantic relation value for this sense SemOpt: The optimal semantic relation value for this sense Abs: Absolute value
Equation 15: The equation for semantic relation distribution error
S: The current solution Word1 and Word2: Every word in the solution SemRel: Every semantic relation SemRelEqn: Use the words in the correct semantic relation equation Avg: Find the average value then transform with the proportional
equation Equation 16: The equation for Cost Function Method 5
Hausman 62
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
5.2 Mating
The concept underlying mating is to combine two “parent” solutions somehow and create a
“child” solution. This “child” solution should be similar to the solutions before and can potentially be
better than the parents. Any weaker solutions will eventually “die” and will not mate in the next
solution, as in Darwin’s theory of survival of the fittest. In genetic algorithms terms, this is “elitism.”
Over several generations, the good portions of all the ancestors should collect together into one very
high scoring child.
There are hundreds of ways two parents can mate. In this project, the author has chosen a
dominant gene approach that has worked well in the past (Hausman, A Dominant Gene Genetic
Algorithm for a Transposition Cipher in Cryptography, 2009) (Hausman, A Dominant Gene Genetic
Algorithm for a Substitution Cipher in Cryptography, 2009). The concept is to take the cost function and
apply it on the gene level, or to the individual portions of the solution. This provides a way to determine
which genes are strongest or more “dominant.” These strong genes should have a better chance of
matching the correct solution. Then any children will inherit the dominant genes from both parents.
This makes the child stronger than either parent since the child inherits only the optimal parts of the
solution. Over several generations, this mating technique approaches a strong solution faster than
many other approaches.
In this project, a solution is the current set of senses for each word. Each gene represents the
sense for a single word. To apply the cost function at the gene level, one must keep track of the
semantic relation scores for each word. This means that each word has a contribution to the total cost
function. The words that contribute the most are dominant genes. Since the maximum cost should
represent the best solution, those dominant genes are most likely correct.
There are two ways to mate solutions in this project. When two parents mate, they randomly
choose one of the two methods. Both methods focus on dominant genes. The first method begins with
Hausman 63
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
the best genes and then combines the mid‐range genes. The second method starts with the mid‐range
genes and then moves on to the best genes.
5.2.1 Mating Method 1: Mate Top Third
As the title suggests, this mating method sorts the genes by their cost, divides this group into
three, and starts with the highest scoring genes. The top third originate from the first parent. Following
this, the child inherits the top two thirds of the second parent. Any remaining genes are derived from
the first parent. The theory is that the child will inherit the best genes while any other genes are
secondary. This formula focuses on the best possible gene distribution. The detailed steps and example
2. Place Dominant Genes Based on First Parent Child: a1 a2 * * * * …
3. Fill in Blanks from Second Parent Child: a1 a2 * b4 * b6 …
4. Fill in any Remaining Blanks from First Parent Child: a1 a2 a3 b4 a5 b6 …
5.3 Mutation
The concept underlying mutations is to give a genetic algorithm a second chance. In many
solutions, there are local maximum solutions that mating can find and get stuck at the local maximum.
For example, if the challenge is to find the highest point on a map, each hill and mountain is a local
maximum solution. The mating function can “get stuck” at the second highest mountain. Had the
mating function turned the other direction at a valley, it would have found the higher mountain. The
mutation function randomly changes some of the solutions so mating may find the other mountain over
time.
In the dominant gene approach, mutations have two objectives. The first objective is to provide
alternate avenues and second chances by providing alternate dominant genes. The second objective is
Hausman 65
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
to focus on ways to improve the lower cost or recessive genes. The mating function examines only a
part of the solution and does not improve genes that are not already strong. Eventually the mating
function will use all the available “strong” genes and not be able to move toward a stronger solution.
The mutations can modify a recessive gene to make it stronger. This new gene may be strong enough to
become a new dominant gene, a factor which leads to better solution overall. The dominant gene
approach needs to focus on the dominant and recessive genes in order to be successful.
There are four main mutation functions in the sections below. When a solution mutates, it
randomly chooses one of the four mutation functions. In some implementations, the mutated solutions
replace the original solution. In this case, the mutations return a mutated clone to prevent original
solutions from becoming worse. This makes it harder to find alternate avenues, but the solution never
becomes weaker.
5.3.1 Mutation Function 1: Random Mutation
One of the most popular mutations in many genetic algorithms is a random mutation. It is
quick, easy, and finds solutions that are not possible through other means. In this case, it is the main
way to find alternate avenues and paths not naturally found by improving recessive genes. Most of the
time the results are a weaker gene, but this mutation helps in the long run. The steps for using this
mutation are below.
1. Randomly pick a percentage between 0% and 20% 2. Randomly pick that percentage of words from the solution 3. For each of those words, randomly pick one of the available senses
5.3.2 Mutation Function 2: Semantic Relation Score Mutation
The semantic relation score mutation is perhaps the most useful mutation of all the mutations.
This started out as a function to find the boundaries of a semantic relation in the samples section. A
slight modification allows this mutation to modify a solution so it has or is near the given semantic
Hausman 66
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
relation score. This makes this very useful for moving a solution towards the optimal semantic relation
values, thus creating stronger solutions. It also provides a way for the genetic algorithm to establish the
range of values necessary for the cost function. The steps to the semantic relation score mutation are
below.
1. Start with the given semantic relation average 2. Find the percentage this average is off from the optimal semantic relation score using the
equation below.
3. Randomly pick the comparison percentage of words 4. If the comparison was negative, the semantic relation cost needs to be increased
a. If the semantic relation is frequency, then randomly pick a sense lower than the current sense. Otherwise use step b.
b. Look at each sense starting at the first sense. Stop and select that sense when the sense increases the semantic relation cost.
5. If the comparison was positive, the semantic relation cost needs to be decreased a. If the semantic relation is frequency, then randomly pick a sense higher than the current
sense. Otherwise use step b. b. Look at each sense starting at the first sense. Stop and select that sense when the sense
decreases the semantic relation cost.
5.3.3 Mutation Function 3: Sense Distribution
The cost function has the following main parts: a semantic relation portion, a sense distribution
portion, and a semantic relation distribution portion. The mutation above covers the semantic relation
portion, so it follows that the second mutation focuses upon the sense distribution. On a higher level,
this mutation finds the current distribution and shuffles the lowest cost genes around to match the
optimal sense distribution. This process achieves two goals. It gives some recessive genes a chance to
become dominant genes and makes the correct distribution to maximize the cost function. This makes
𝐶𝑜𝑚𝑝𝑎𝑟𝑖𝑠𝑜𝑛(𝑠) = 𝐴𝑣𝑔(𝑠) − 𝑂𝑝𝑡𝑖𝑚𝑎𝑙(𝑠)𝑂𝑝𝑡𝑖𝑚𝑎𝑙(𝑠)
S: The current semantic relation Avg: the average value of the semantic relation for this solution Optimal: the optimal value for this semantic relation
Equation 17: The equation for comparing the current solution to the optimal solution
Hausman 67
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
the sense distribution mutation very useful if it were run several times during the genetic algorithm
process. The steps to the sense distribution mutation are below.
1. Find the number of words extra or missing for each sense compared to the optimal sense distributions
2. Find the genes that have the lowest gene cost 3. Look at each sense distribution
a. If the current sense distribution has extra words, move the lowest cost genes to a sense distribution needing words
5.3.4 Mutation Function 4: Semantic Relation Distribution
The last area of the cost function, not covered by a mutation function, is the semantic relation
distribution. The concept is very similar to the sense distribution mutation. Find the current distribution
for the given semantic relation and shuffle the recessive genes to match the optimal distribution. This
maximizes the cost function by incorporating the proper semantic relation distributions. It also provides
recessive genes a chance to become dominant genes, just as in the sense distribution mutation. The
steps to the semantic relation distribution are below.
1. Compare each average semantic relation value for each sense to the optimal semantic relation value for that sense. This should result in the number of words that need to change for each sense.
2. Find the genes that have the lowest gene cost 3. Look at each sense distribution
a. If the current sense distribution has extra words, move the lowest cost genes to a sense distribution needing words
5.4 Main Genetic Algorithm Function
Earlier the author provided the generic main steps to create a genetic algorithm. The preceding
sections explain how the three main parts (cost function, mating, and mutation) function. However,
none of these sections explain the size of a generation, where the range of values for the cost function
originates, and how chromosomes for the mutation and mating are selected. Details such as this are the
responsibility of the main genetic algorithm function. This function is the center of the genetic
algorithm. The step by step process of this algorithm is below.
Hausman 68
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
1. Read in the given text, initialize all the WordNet semantic relation information, and assign the parts of speech with a part of speech tagger
2. Initialize the generation pool. Note that this pool does not keep track of duplicate solutions. 3. Initially find the range of possible scores for each semantic relation using the semantic relation
score mutation. Each semantic relation has an upper and lower boundary. There will be one process for each boundary. This process is below. The cost function relies on these ranges for comparing solutions.
a. Start with a solution using the first sense for all words (or the given solution if started from step 5)
b. Mutate that solution towards the given boundary (maximum or minimum possible value)
c. Repeat step b until there is a repeat solution or until 25 mutations are performed d. Report the current semantic relation score as the given boundary.
4. Add all of the solutions from step 3 into the generation pool. On top of that use the weighted random function from the sampling section to create 25 new solutions.
5. If the generation number is divisible by 10, repeat step 3 using the top solution. Sometimes the new starting point finds a slightly wider semantic relation range.
6. Mate the top 5 solutions as parent 1 with randomly chosen solutions from the generation pool as parent 2. After this mate two randomly chosen solutions from the generation pool 20 times. Add all children to the generation pool.
7. Mutate the top 5 solutions. Then mutate 20 randomly chosen solutions from the generation pool. Add all new solutions to the generation pool.
8. Reduce the current generation pool by only keeping the top 25 solutions. 9. Repeat steps 5‐8 for 25 generations.
5.5 Notes about Speed
Many of the sections above describe complex algorithms containing a large number of
combinations. There are 25 generations with each generation creating 50 new solutions. The cost
function for each solution uses several semantic relations. Each semantic relation is runs on every word
pair combination in a window with 45 possible combinations. The number of windows is the total
number of words in a solution. Every time the cost function ranges change, all of the solutions must
rerun the cost function. The semantic relation score mutation in particular manipulates genes one at a
time, which changes the score of the entire solution. There are a very large number of calculations for
this genetic algorithm.
To deal with the number of calculations, there are several architectural choices and
modifications to the code to reduce the impact of the changes. All of the information is stored in RAM
Hausman 69
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
to reduce the time it takes to look at the disk, which makes the program run with ~700MB of memory.
The cost function tracks the information in various data structures and minimizes the number of
recalculations mutations and semantic relation range changes cause. All mating and mutation
operations run in parallel to take advantage of multicore processors. However, all of these
improvements are limited. The number of calculations alone causes this program to run for a few
minutes for each SemCor file. Many of the SemEval competitions have more words, so the process will
take longer. This is not a real time solution for large groups of text.
Hausman 70
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Chapter 6: Results and Analysis
This project has compared the results within three main areas. The first area focused upon a
comparison against a colleague working on the same problem. The colleague, Michael Billot, has
implemented a page rank algorithm to solve word sense disambiguation. The second area is the
“Genetic Word Sense Disambiguation” study by Zhang (Zhang, Zhou, & Martin, 2008). This paper was
the starting point for this study, and it has significantly influenced the conceptual framework of the
author’s approach. The author has included several changes which are based on reports from other
studies, but their influence has not been as significant. The third area is the SemEval competitions.
Many researchers who did and did not participate in the competitions use these results as a comparison
against other algorithms. All three comparisons should provide significant insight to how well this
algorithm functions.
6.1 Measuring the Results
When the first SemEval competition occurred in 1998, the sponsors sought to compare the
results from all the competitors. They devised three categories for the competition: coverage, recall,
and precision. Coverage is the ratio of how many words the competitor answered to the total number
of words overall. This reflects how much of the solution a competitor answered. Recall is the ratio of
the total number of senses correct within the total number of words provided. This indicates how well a
solution was answered. Precision is the total number of senses correct within the total number of
words answered. This indicates how well an algorithm answers when it does give a sense. SemEval
2007 introduced two different ways to measure a result because some senses in WordNet are too subtle
for even humans to disambiguate. The first method uses the exact sense as a fine word assessment, as
Hausman 71
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
was done in the previous competitions. The second method combines subtly different senses into one
possible answer as a coarse words assessment. The sponsors evaluated the SemEval 2007 competition
by using the same three numbers separately for both the fine and coarse word assessments.
6.2 Comparison to Michael Billot
Michael Billot is the author’s colleague at the University of Colorado at Colorado Springs. His
solution uses the page rank algorithm to disambiguate verbs. In his study, he used several walkthroughs
for Zelda as the input. He then evaluated the results by hand, a method which presents obvious
difficulties. Unless the verb is clearly a sense, which is rare, it is hard to prove one sense is correct. Even
during the SemEval competitions, the sponsors only have the competitors attempt to find the sense of a
word when over 90% of a group of professional linguists agree. Even if the project uses the same text,
there is no clear indication as to whether the author has chosen the same senses as did Billot, or
whether the sense is correct. With that in mind, comparing against Billot may be a problematic
challenge.
Billot’s page rank algorithm has a 46.4% accuracy according to his study (Billot, 2010).
Considering that he supplies a sense for every verb, this 46.4% figure is the recall and the precision. The
value for the first sense baseline is 77.4%. Since evaluation by hand is problematic at best, this author
did not attempt to disambiguate the walkthrough. However, this author does have the results for
SemCor. For a first sense baseline, the 77.4% baseline is higher than the average baseline, but it is still
within acceptable range. The average coverage for verbs is 99.26%. This is very high, which makes the
recall and precision are very close. The average recall is 49.24% and the average precision is 49.61%.
These percentages suggest that this author’s project evaluates the results more accurately, but the
results are still too close to be certain. The most effective way to tell for sure is for Billot to provide
results for the SemCor files as well.
Hausman 72
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
6.3 Comparison to Zhang
Zhang’s study, “Genetic Word Sense Disambiguation”, represents the most similar approach to
this project. However, Zhang only evaluates nouns and has not participated in any of the SemEval
competitions. Zhang does, however, provide references to several SemCor files. Overall, Zhang’s
coverage is 100%, so his recall and precision results are the same. This project purposely ignores
pronouns, resulting in average coverage of 89.25%. Within all the SemCor files, Zhang has a
recall/precision of 71.96%. This project has a precision of 62.13% for nouns. Zhang reports a 70%
accuracy on 51 files. This project reports a 70% precision on nine files. This author’s conclusion is that
that this project is not as successful as was Zhang’s. However, one should keep in mind that this
author’s project solved for all words, a significantly more complex study overall.
6.4 SemEval
The SemEval competitions occur every three years. The competitions are devoted to language
processing and typically incorporate an “all words” task. This task requires performing word sense
disambiguation on all the words they have tagged. They then use this information to provide recall,
precision, and coverage for each competing algorithm. Many researchers compare themselves to the
competitors in the SemEval competitions.
6.4.1 SemEval 2
In the SemEval‐2 competition in 2001, there were 22 systems participating for the English all
words task (ACL‐SIGLEX, 2011). The baseline for using the first sense was 57.0%. The best system had
100% coverage and a 69.8% precision. This author’s project achieved 95.12% coverage and a 52.29%
precision. The comparison shows a 4.71% precision below the baseline and a 17.51% precision below
the best system. The winner was an outlier in this competition. The second place system had a
Hausman 73
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
precision 5.3% below the first place winner. If this project were competing, it would be in sixth place.
This performance lies somewhere between the middle ranks and the top ranks.
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Chapter 7: Conclusions and Future Research
This project has attempted to solve for word sense disambiguation. Solving word sense
disambiguation would allow several applications, like language translation, to work much more
accurately. To accomplish this, the author investigated several semantic relations and transformed
them into an optimization problem. This way a genetic algorithm could solve the problem. The genetic
algorithm used a dominant gene technique to converge on an answer. The author then compared the
results to several other researchers, including the competitors in the SemEval competitions. If this
algorithm had competed in the SemEval competitions, it would have rank in the middle. This means
that it is not the optimal algorithm, but it still works well enough to be of interest since it is not in the
lower ranks.
7.1 Algorithm Weaknesses
One of the reasons why the results are not 100% has to do with the available semantic relations.
An in‐depth investigation of some of the words reveals that there is simply not enough information
provided in context. Some words have a correct sense of something other than the most common
sense, yet frequency is the only semantic relation that applies to the word. Frequency directs the result
to the incorrect sense. False positives and misleading information do have an effect, but a large
proportion of the words have insufficient information in order to provide acceptable results. In most of
these cases, the information supplied by WordNet will not help. This includes several other semantic
relations that were not part of this project. The author cursorily explored Lesk, but upon reflection it
proved unhelpful. It is significant that many other algorithms appear to have similar problems. One of
the top competitors in the third SemEval competition, Sense Learner, openly admits that they do have
Hausman 77
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
any information on 15% of the words (Mihalcea & Csomai, 2005). These words simply chose the most
frequent definition as a default. Regardless, something needs to account for these words.
This algorithm does appear to work fairly well with the available WordNet information. Many
other researchers assume that the highest scores work while this algorithm attempts to adjust for the
weaknesses within each semantic relation. However, there still are possible improvements around
semantic relations in the cost function. One of the more significant problems is how large the 90%
window is for some semantic relations. If there were a way to dynamically adjust the maximum point
and reduce the deviation, the answers would improve. At this writing, both the distributions and the
relations rely on averages. The distributions require more investigation and adjustments. Perhaps the
distributions would function more effectively if they accounted for the various parts of speech
individually. One can only surmise in this regard, but it seems logical that additional percentage points
make a significant difference in the SemEval competitions.
7.2 Future Possibilities
There were two tool changes mentioned earlier that may foreshadow future research. The first
was importing the “WordNet Lexical Markup Framework” from the SemEval 2010 competition. This
would allow the author’s project to compare the SemEval 2010 results and to look at results in different
languages. Since one possible application of word sense disambiguation is machine text translation, the
performance in other languages would be very helpful to know. The second change is continuing
investigation of OntoNotes. The semantic and lexical relations will be different for OntoNotes. If the
plans for exploring OntoNotes take place, the research results will be more useful than those relations in
WordNet. At the very least, the algorithm will be able to compete in SemEval in the future if the
competition replaces WordNet. These two changes would help in the future and may provide insight for
other changes to this algorithm.
Hausman 78
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
The top‐rated algorithms in the SemEval competition typically account for a wider range of
information. This algorithm uses semantic relation information from WordNet. Their algorithms literally
look at everything they could think of. This includes information such as: A) “Is this sense typically the
end of a sentence?” or B) “Is this word followed by a noun?” and C) “Does this word end in ‘ing’ all the
time?” This information feeds into various types of algorithms many of which are extremely
complicated. Outstanding competitors have used multiple information sources. One option that may
help this algorithm is the use of more than WordNet semantic relations.
A final option is exploration of punctuation. For example, examine the statement, “Let’s go eat,
grandma!” If no comma separated eat and Grandma, the sentence would take on a most horrific
meaning. Sentence sense changes with punctuation. This author suggests that exploration of the field
of punctuation research in computer systems language development studies may grow into a field that
significantly changes the focus of the discipline and provides data to scaffold word sense disambiguation
on a most significant scale.
Hausman 79
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Appendix
The following sections contain several of the detailed results developed throughout this project.
This includes a short table of the types of SemCor files, the proportion statistics for Cost Method 3, the
distributions for Cost Method 4, the distributions for Cost Method 5, and results for every SemCor file.
Appendix A: SemCor Files
Each SemCor file has a similar name: br‐*##. The “br” is short for the brown corpus. The “*”
represents a letter that indicates what genre of text is tagged. The “##” represents a number that
separates different files in a genre.
Table 4: The SemCor file letter indicating type of resource the original text came from. Informative Prose (374 samples) Imaginative Prose (126 samples) Letter Genre Letter Genre A Press: Reportage K General Fiction B Press: Editorial L Mystery and Detective Fiction C Press: Reviews M Science Fiction D Religion N Adventure and Western Fiction E Skill and Hobbies P Romance and Love Story F Popular Love R Humor G Belles Letters, Biography, Memoirs, etc. H Miscellaneous J Learned
Table 5: The various SemCor files File Author Source A01
Atlanta Constitution Political Reportage A02 Dallas Morning News Political Reportage A03 Chicago Tribune Political Reportage A04 Christian Science Monitor Political Reportage A05 Providence Journal Political Reportage A06 Newark Evening News Political Reportage A07 New York Times Political Reportage A08 Times‐Picayune, New Orleans Political Reportage A09 Philadelphia Inquirer Political Reportage A10 Oregonian, Portland Political Reportage A11 Sun, Baltimore Sports Reportage A12 Dallas Morning News Sports Reportage
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
File Author Source A13 Rocky Mountain News Sports Reportage A14 New York Times Sports Reportage. A15 St. Louis Post‐Dispatch Sports Reportage A16 Chicago Tribune Society Reportage A17 Rocky Mountain News Society Reportage A18 Philadelphia Inquirer Society Reportage A19 Sun, Baltimore Spot News A20 Chicago Tribune Spot News A21 Detroit News Spot News A22 Atlanta Constitution Spot News A23 Oregonian, Portland Spot News A24 Providence Journal Spot News A25 San Francisco Chronicle Spot News A26 Dallas Morning News Financial Reportage A27 Los Angeles Times Financial Reportage A28 Wall Street Journal Financial Reportage A29 Dallas Morning News Cultural Reportage. A30 Los Angeles Times Cultural A31 Miami Herald Cultural Reportage A32 San Francisco Chronicle Cultural Reportage A33 Washington Post Cultural Reportage A34 New York Times News of the Week in Review A35 James J. Maguire A Family Affair A36 William Gomberg Unions and the Anti‐Trust Laws A37 Time National Affairs A38 Sports Illustrated A Duel Golfers Will Never Forget A39 Newsweek Sports A40 Time People. Art & Education A41 Robert Wallace This Is The Way It Came About A42 Newsweek National Affairs A43A U. S. News & World Report Better Times for Turnpikes A43B U. S. News & World Report A Plan to Free U. S. Gold Supply A44A John Tebbel Books Go Co‐operative A44B Gilbert Chapman Reading and the Free Society B01
Atlanta Constitution Editorials B02 Christian Science Monitor Editorials B03 Detroit News Editorials B04 Miami Herald Editorials B05 Newark Evening News Editorials B06 St. Louis Post‐Dispatch Editorials B07 New York Times Editorials B08 Atlanta Constitution Columns B09 Christian Science Monitor Columns
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
File Author Source B10 Sun. Baltimore Columns B11 Los Angeles Times Columns B12 Newark Evening News Columns B13 Times‐Picayune, New Orleans Columns B14 Atlanta Constitution Columns B15 Providence Journal Letters to the Editor B16 Chicago Tribune Voice of the People B17 Newark Evening News What Readers Have to Say B18 New York Times Letters to the Times B19 Philadelphia Inquirer The Voice of the People B20 Nation Editorials B21A Gerald W. Johnson The Cult of the Motor Car B21B James Deakin How Much Fallout Can We Take B22 Commonweal Week by Week B23A William F. Buckley, Jr. We Shall Return B23B James Burnham Tangle in Katanga B24 Time Reviews B25A Alexander Werth Walkout in Moscow B25B Peter Solsich, Jr. The Armed Superpatriots B26 National Review To the Editor B27 Saturday Review Letters to the Editor C01
Chicago Daily Tribune Reviews C02 Christian Science Monitor Reviews C03 New York Times Reviews C04 Providence Journal Reviews C05 Christian Science Monitor Reviews C06 Wall Street Journals Reviews C07 New York Times Reviews C08 Providence Journal Reviews C09 New York Times Reviews C10 Providence Journal Reviews C11 New York Times Reviews C12 Christian Science Monitor Reviews C13 Wall Street Journal Reviews C14 New York Times Reviews C15 Life Reviews C16 Saturday Review Reviews C17 Time Reviews D01
William Pollard Physicist and Christian D02 Schubert Ogden Christ Without Myth D03 Edward E. Kelly Christian Unity in England D04 Jaroslav Pelikan The Shape of Death D05 Perry Miller Theodore Parker: Apostasy With in Liberalism
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
File Author Source D06 A Howard Kelly Out of Doubt into Faith D06B Shirley Schuyler Not as the World Giveth D06C Nathanael Olson Are You in Orbit? D07 Peter Eldersveld Faith Amid Fear D08 Schuyler Cammann The Magic Square of Three D09 Eugene E. Golay Organizing the Local Church D10 Huston Smith Interfaith Communication: The Contemporary Scene D11 Paul Ramsey War & the Christian Conscience D12 Kenneth Underwood and Widen
Jacobson Probing the Ethics of Realtors
D13A Donald H. Andrews The New Science & the New Faith D13B George Bo Longstreet The Seeming Impossible D14 Kenneth S. Latourette Christianity in a Revolutionary Age D15 Ernest Becker Zen: A Rational critique D16A Anonymous What the Holy Catholic Bible Teach D16B Harold Brenneman Notice You May Do As You Please D17A Anonymous Guideposts: 15th Anniversary Issue D17B J. I. Rivero The Night Our Paper Died E01A
Ben Welder Henri de Courcy: Jr. Mr. Canada E0lB Joe Welder The Mark of the Champion E02A Dorothy Schroeder Plant a Carpet of Bloom E02B Anonymous Avocado is Something Special E03 D. F. Martin Will Aircraft or Missiles Win Wars? E04A Harris Goldsmith The Schnabel Pro Arte Trout E04B Robert C. Marsh The True Sound of a Solid Second E04C R.D.D. Review of Adam, Giselle E05A Paul Nigro The Younger Generation E05B Patricia Barney Use of Common Sense Makes Dogs Acceptable E05C Anonymous The Malady Lingers On E06 Joseph E. Choate The American Boating Scene E07 Paul Larson and Gordon Odegard How to Design Your Interlocking Frame E08 Don Francisco Formulas and Math Every Hot Rodder Should Know E09A Don McMahan The Week at Ben White Raceway E09B Edith Shaw The Picture at Del Mar E10 Larry Koller The New Guns of 61 E11 Idwal Jones Santa Cruz Run E12 Julia Newman Travel and Camera USA E13 Robert Deardorff Step by Step through Istanbul E14 Ann Carnahan Nick Manero's Cook‐out Barbecue Book E15A Anonymous Pottery from Old Molds E15B Anonymous Knitting Knacks E16 Hal Kelly Build Hotei E17A Anonymous This is the Vacation Cottage You Can Build
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
File Author Source E17B Patrick K. Snook Care and Basic Use of the Drill Press E18A Lura W. Watkins The Bridge Over the Merrimac E18B Boyd B. Stutler Veteran Philippi Bridge E19 Booth Hemingway and Stuart H.
Brown How to Own a Pool and Like It.
E20 Anonymous What You Should Know About Air Conditioning E21 Richard McCosh Recreation Site Selection E22A Roy Harris Roy Harris Salutes Serge Prokofieff E22B Helen Havener A 30 Years War E23 Norman Kent The Watercolor Art of Roy M. Mason E24 Bonnie Prudden The Dancer & the Gymnast E25 Walter Ho Buchsbaum Advances in Medical Electronics E26 Bern Dibner Oersted & the Discovery of Electromagnetism E27A Mike Bay What Can Additives Do for Ruminants? E27B James S. Boyd Which Feed Bunk for You E28 John R. Sargent Where to Aim Your Planning E29 Edward A. Walton On Education for the Interior Designer E30 Anonymous The Attack on Employee Services E31A Jim Dee Expanding Horizons E31B George Laycock The Challenge E32 E. J. Tangerman Which Way Up. Technical or Management? E33A Robert Gray Fifty Houses, One Tank E33B Chet Cunningham Truck Talk E34 Anonymous The New Look in Signs E35 Anonymous The Industrial Revolution in Housing E36 Ethel Norling Renting a Car in Europe F01
Rosemary Blackmon How Much Do You Tell When You Talk?
F02 Glenn Infield America's Secret Poison Gas Tragedy F03 Nathan Rapport I've Been Here Before F04 Ruth F. Rosevear North Country School Cares for the Whole Child F05 Richard S. Allen When Fogg Flew the Mail F06A Alice Ho Austin Let's Discuss Retirement F06B Harold P. Winchester What It Means to be Creative F07A Marvin Sentnor and Stephen Hult How to Have a Successful Honeymoon F07B Ho Walter Yoder Attitudes Toward Nudity F08 Philip Reaves Who Rules the Marriage Bed? F09A David Martinson Fantastic Life & Death of the Golden Prostitute. F09B Isel D. Rugget When It Comes to Carpets F10 Jack Kaplan Therapy by Witchcraft F11 Lillian Pompian Tooth‐Straightening Today F12 Marian Neater New Methods of Parapsychology. F13 Orlin J. Scoville Part‐time Farming
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
File Author Source F14 Harold Rosenberg The Trial and Eichmann F15 John A. O'Brien Let's Take Birth Control Out of Politics F16 James Boylan Mutiny F17 John Harnsberger and Robert P.
Wilkins Transportation on the Northern Plains
F18 Bell I. Willy Home Letters of Johnny Reb & Billy Yank Fl9 Tristram P. Coffin Folklore in the American Twentieth Century F20 Kenneth Allsop The Bootleggers and Their Era F21A Joseph Bernstein Giant Waves F21B L. Don Leet Introduction F21C L. Don Leet The Restless Earth and Its Interiors F22 Booton Herndon From Custer to Korea, the 7th Cavalry F23 Barry Goldwater A Foreign Policy for America F24 Peter J. White Report on Laos F25 David Boroff Jewish Teen‐Age Culture F26 Amy Lathrop Pioneer Remedies from Western Kansas F27 Creighton Churchill A Notebook for the Wines of France F28 Frank O. Gatell Doctor Palfrey Frees His Slaves F29 Douglass Cater The Kennedy Look in the Arts F30 Frederic A Birmingham The Ivy League Today F31 Edward Do Radin Lizzie Borden: The Untold Story F32 Florence M. Read The Story of Spelman College F33 James Be Conant Slurs and Suburbs F34 Frederic R. Senti and W. Dayton
Maclay Age‐old uses of Seeds and Some New Ones
F35 Ramon F. Adams The Old‐time Cowhand F36 Robert Easton and Mackenzie Brown Lord of Beasts F37 Samuel M. Cavert On the Road to Christian Unity F38 Robert Smith Baseball in America F39 Clark E. Vincent Unmarried mothers F40 William Greenleaf Monopoly on Wheels F41 George W. Oakes Turn Right at the Fountain F42 James Baldwin Nobody Knows My Name F43 Frank Getlein and Harold C. Gardiner Movies, Morals, and Art F44 Gibson Winter The Suburban Captivity of the Churches F45 Paul C. Phillips The Fur Trade F46 Russell Baker An American in Washington F47 Clara L. Simerville Home Visits Abroad F48 Paul Ramsey Christian Ethics & the Sit‐In G01 Edward P. Lawton Northern Liberals & Southern Bourbons G02 Arthur S. Miller Toward a Concept of National Responsibility G03 Peter Wyden The Chances of Accidental War G04 Eugene Burdick The Invisible Aborigine
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
File Author Source G05 Terence O'Donnell Evenings at the Bridge G06A Ruth Berges William Steinberg, Pittsburgh's Dynamic Conductor G06B Henry W. Koller German Youth Looks to the Future G07 Richard B. Morris Seven Who Set Our Destiny G08 Frank Murphy New Southern Fiction: Urban or Agrarian? G09 Selma J. Cohen Avant‐Garde Choreography G10 Clarence Streit How the Civil War Kept You Sovereign G11 Frank Oppenheimer Science and Fear G12 Tom F. Driver Beckett by the Madeleine G13 Charles Glicksberg Sex in Contemporary Literature G14 Helen H. Santmeyer There Were Fences G15 Howard Nemerov Themes and Methods: Early Storie of Thomas Mann G16 John F. Hayward Mimesis & Symbol in the Arts G17 Randall Stewart A Little History, a Little Honesty G18 Charles W. Stork Verner von Heidenstam Gl9 R. F. Shaw The Private Eye G20 Dan McLachlan, Jr. Communication Networks & Monitoring G21 Brainard Cheney Christianity & the Tragic Vision G22 Kenneth Reiner Coping with Runaway Technology G23 William C. Smith Why Fear Ideas G24 Sanchia Thayer Personality & Moral Leadership G25 Stanley Parry The Restoration of Tradition G26 Selma Fraiberg Two Modern Incest Heroes G27 Matthew Josephson Jean Hélion. The Return from Abstract Art G28 Arlin Turner William Faulkner, Southern Novelist G29 Anonymous References for the Good Society G30 Norwood R. Hanson Copernican & Keplerian Astronomy G31 Irving Fineman Woman of Valor: Life of Henrietta Szold 1860‐1945 G32 Finis Farr Frank Lloyd Wright G33 Virgilia Peterson A Matter of Life and Death G34 Harry Golden Carl Sandburg G35 Dwight D. Eisenhower Peace With Justice G36 DeWitt Copp & Marshall Peck Betrayal at the UN G37 Gordon L. Hall Golden Boats from Burma G38 Bertrand A. Goldgar The Curse of Party G39 Edward Jablonski Harold Arlen Happy with the Blues G40 Gene Fowler Skyline: A Reporter's Reminiscences of the 1920s. G41 Lillian R. Parka and Frances S.
Leighton My Thirty Years Backstairs at the White House
G42 Harold D. Lasswell Epilogue G43 Robert E. Lane The Liberties of Wit G44 Newton Stallknecht Ideas and Literature G45 W. A. Swanberg Citizen Hearst: A Biography of W. R. Hearst
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
File Author Source G46 Henry R. Winkler George Macaulay Trevelyan G47 Carry Davis The World Is My Country G48 Francis F. McKinney Education in Violence G49 Paul van K. Thomson Francis Thompson, a Critical Biography G50 Curtis C. Davis The King's Chevalier G51 Ilka Chase The Carthaginian Rose G52 Robert L. Duncan Reluctant General G53 Bertram Lippincott Indians, Privateers, and High Society G54 Mabel W. Wheaton & LeGette Blythe Thomas Wolfe & His Family G55 Ralph E. Flanders Senator from Vermont,. 112 G56 Keith F. McKean The Moral Measure of Literature G57 Robin M. Williams, Jr. Values & Modern Education in the United States G58 North Callahan Daniel Morgan G59 Esther R. Clifford A Knight of Great Renown G60 Gertrude Berg & Cherney Berg Molly and Me G61 Donald A. White Litus Saxonicum G62 C. H. Cramer Newton D. Baker G63 George Steiner The Death of Tragedy G64 Mark Eccles Shakespeare in Warwickshire G65 Timothy P. Donovan Henry Adams & Brooks Adams G66 Van Wyck Brooks From the Shadow of the Mountain G67 Mark Schorer Sinclair Lewis: An American Life G68 Harris F. Fletcher The Intellectual Development of John Milton G69 Mark R. Hillegas Dystopian Science Fiction G70 Joseph W. Krutch If You Don't Mind My Saying So G71 Joseph Frank André Malraux: The Image of Man G72 J W. Fulbright For a Concert of Free G73 Carolyn See The Jazz Musician as Patchen's Hero G74 John McCormick The Confessions of Jean Jacques Krim G75 George Garrett A Wreath for Garibaldi H01
U. S Dep't of Commerce Handbook of Federal Aids to Communities
H02 U. S. Dep't of State An Act for International Development H03 U. S. 87th Congress House Document No. 487 H04 R. I. Legislative Council State Automobiles & Travel Allowances H05 R. I. Leglelative Council Taxing of Movable Tangible Property H06 R. I. Development Council Annual Report, 1960 H07 R. I. Legislative Council linlform Fiscal Year for Municipalities H08 John A. Notte, Jr. R. I. Governor's Proclamations H09 U. S. 87th Congress Public Laws 295, 300, 347 H10 U. S. Dep't of Defense Medicine in National Defense H11 U. S. Dep't of Commerce 1961 Reaearch Highlights, Nat'1 Bureau of Standards H12 U. S. 87th Congress Legislation on Foreign Rels
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
File Author Source H13 U. S. 87th Congreas Congressional Record: Extension of Remarks. May 2,
1961 H14 U. S. Dep't of Health, Education &
Welfare Grants‐in‐Aid and Other Financial Assistance Programs
H15 U. S. Office of Civil and Defence Mobilization
The Family Fallout Shelter
H16 U. S. Reports Cases AdJudged in the Supreme Court, October Tenm 1960
H17 U. S. Reports Cases AdJudged in the Supreme Court, October Tenm 1959‐60
H18 Dean Rusk The Department of State Hl9 Peace Corps Fact Book H20 U. S. Dep't of Agriculture Development Program for the National Forests H21 Dwight D. Eisenhower Public Papers, 1960‐61 H22 U. S. Dep't of State U. S. Treatiea and Other International Agreements H23 U. S. Federal Communications
H24 U. S. Tresaury Dep't Your Federal Income Tax H25 Guggenheim Foundation Report of the Secretary Gen'1 H26 Anonymous A Brief Background of Brown & Sharpe H27A Robert Leeson Leesona Corporation President's Report H27B Leesona Corporation More Efficient Production for Expanding Textile
Markets H28 Carleton College Carleton College Bulletin H29 Sprague Electric Company Sprague Log H30 Carnegie Foundation Annual Report of Year Ending June 30, 1961 J01
Cornell H. Mayer Radio. Emission of the Moon and Planets
J02 R. C. Binder et al. 1961 Heat Transfer & Fluid Mechanics Institute J03 Harry H. Hull Normal Forces & Their Thermodynamic Significance J04 James A. Ibers et al. Proton Magnetic Resonance Study J05 John R. Van Wazer, ed. Phosphorus and Its Compounds J06 Francis J. Johnston & John E. Willard Exchange Reaction Between C12 and CC14 J07 J. F. Vedder Micrometeorites J08 LeRoy Fothergill Biological Warfare J09 M. Yokayama et al Chemical & Serological Characteristics J10 B. J. D. Meeuse The Story of Pollination J11 Clifford H Pope The Ciant Snakes J12 Richard F McLaughlin et al. A Study of the Subgross Pulmonary Anatomy J13 S. Idell Pyle et al. Onsets, Completions & Spans J14 Jacob Robbins et al. The Thyroid‐Stimulating Hormone J15 J. W. C. Hagstrom et. al. Debilitating Muscular Weakness Jl6 A. N. Nagaraj & L. M. Black Localization of Wound‐Tumor Virus Antigen J17 E. Gellhorn Prolegomena to a Theory of the Emotions
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
File Author Source J18 Kenneth Hoffman & Ray Kunze Linear Algebra Jl9 Frederick Mosteller et al. Probability with Statistical Applications J20 R. P. Jerrard Inscribed Squares in Plane Curves J21 C. R. Wylie, Jr. Line Involutions in S3 J22 Max F. Millikan & Donald L.
Blackmer, editors The Emerging Nations
J23 Joyce O. Hertzler American Social Institutions J24 Howard J. Parad Preventive Casework: Problems & Implications J25 Sister Claire M. Sawyer Some Aspects of Fertility of a Tri‐Racial Isolate J26 Frank Lorimer Demographic Information on Tropical Africa J27 Dale L. Womble Functional Marriage Course for the Already Married J28 William H. Ittelson & Samuel B.
Kutash, editors Perceptual Changes in Psychopathology
J29 Jesse W. Grimes & Wesley Allinsmith Compulsivity, Anxiety & School Achievement J30 Raymond J. Corsini Roleplaying in Business & Industry J31 Harold Searles Schizophrenic Communication J32 Hugh Kelly & Ted Ziehe Glossary Lookup Made Easy J33 Ralph Bc Long The Sentence & Tts Parts J34 H.A. Cleason Review of African Language Studies J35 A. T. Kroeber Semantic Contribution of Lexicostatistics J36 D. F. Fleming The Cold War & Its Origins J37 Douglas Ashford Elections' in Morocco: Progress or Confusion J38 Committee for Economic
Development Distressed Areas in a Growing Economy
J39 William O'Connor Stocks, Wheat & Pharaohs J40 James J. O'Leary The outlook for Interest Rates in 1961 J41 Allan J. Braff & Roger F. Miller Wage‐Price Policies Under Public Pressure J42 Morton A. Kaplan ~ Nicholas
Katzenbach The Political Foundation of Internationa1 Law
J43 Wallace Mendelson Justices Black & Frankfurter J44 J. Mitchell Reese, Jr, Reorganization Transfers J45 Albert Schreiber et al. Defense Procurement & Small Business J46 Irving Perluss Agricultural Labor Disputes in California 1960 J47 William S. Ragan Teaching America's Children. J48 Paul Cooke Desegregated Education in the Middle‐South Region J49 Robert J. Havighurst Social‐Class Influences on American Education J50 James C. Bonbright Principles of Public Utility Rates J51 Irving L. Horowitz Philosophy, Science & the Sociology of Knowledge J52 Brand Blanshard The Emotive Theory J53 William S. Haymond Is Distance an Original Factor in Vision? J54 Chester G. Starr The Origins of Greek Civilization 1100‐650 B. C J55 Jim B. Pearson The Maxwell Land Grant J56 Edwin L. Bigelow & Nancy H. Otis Manchester, Vermont, A Pleasant Land J57 J. H. Hexter Thomas More: on the Margins of Modernity
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
File Author Source J58 John M, Ray Rhode Island's Reactions to John Brown's Raid J59 Clement Greenberg Collage J60 Robert A. Futterman The Future of Our Cities J61 Allyn Cox Completing & Restoring the Capitol Frescos J62 Jimmy Ernst A Letter to Artists of the Soviet Union J63 John H. Schaar Escape from Authority, Perspectives of Erich Fromm J64 Katherine G. McDonald Figures of Rebellion J65 Samuel Hynes The Pattern of Hardy's Poetry J66 Kenneth Rexroth Disengagament: The Art of the Beat Generation J67 William Whallon The Diction of Beowulf J68 Charles R. Forker The Language of Hands in Great Expectations J69 I. B. M. Corporation IBM 7070, Autocoder Reference Manual J70 Ross E. McKinney & Howard Edde Aerated Lagoon for Suburban Sewage Disposal J71 Thomas D. McGrath Submarine Defense J72 Mellon Institute Annual Report; 1960, Independent Research J73 Nat'l Research Council Directory of Continuing Numerical Data Projects J74 Harlan W. Nelson Food Preservation by Ionizing Radiation J75 W. K. Asbeck Forces in Coatings Removal Knife Cutting Method J76 Joel Frados, editor Survey of Foamed Plastics J77 William D. Appel, editor 1961 Technical Manual of American Ass'n of Textile
Chemists & Colorists J78 Paul J. Dolon & Wilfrid F. Niklas Gain & Resolution of Fiber Optic Intensifier J79 Rutherford Aris The O'ptim.A1 Design of Chemical Reactors J80 C. J. Savant Jr. & R. C. Howard Principles of Inertial Navigation K01
Christopher Davis First Family K02 Clayton C. Barbeau The Ikon K03 Tristram Coffin Not to the Swift K04 W. E. B. Du Bois Worlds of Color K05 David Stacton The Judges of the Secret Court K06 Louis Zara Dark Rider K07 Francis Pollini Night K08 Guy Endore Voltaire! Voltaire! K09 Howard Fast April Morning K10 Gladys H. Barr The Master of & Geneva K11 Robert Penn Warren Wilderness K12 Gerald Green The Heartless Light Kl3 William Maxwell The Chateau K14 Irving Stone The Agony & the Ecstasy K15 Ann Hebson The Lattimer Legend K16 Stephen Longstreet Eagles Where I Walk Kl7 Leon Uris Mila 8 K18 John Dos Passos Midcentury K19 Robert J Duncan The Voice of Strangers
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
File Author Source K20 Guy Bolton The Olympians K21 Bruce Palmer My Brother's Keeper K22 John Cheever The Brigadier & the Golf Widow K23 Prieda Arkin The Tight of the Sea K24 W. H. Gass The Pedersen Kid K25 Arthur Miller The Prophecy K26 Jane G. Rushing Against the Moon K27 E. Lucas Myers The Vindication of Dr. Nestor K28 Sallie Bingham Moving Day K29 Marvin Schiller The Sheep's in the Meadow. L01
Winfred Van Atta Shock Treatment L02 A. A. Fair Bachelors Get Lonely L03 Amber Dean Encounter With Evil L04 David Alexander Bloodstain L05 Brett Halliday The Careless Corpse L06 Thomas B. Dewey Hunter at Large L07 Genevieve Golden Deadlier Than the Male L08 Dell Shannon The Ace of Spades L09 Mignon G. Eberhart The Cup, the Blade or the Swords L10 Harry Bleaker Impact L11 Hampton Stone The Man Who Looked Death in the Eye L12 Whit Masterson Evil Come, Evil Go L13 Dolores Hitchens Footsteps in the Night Ll4 Frances & Richard Lockridge Murder Has Its Points L15 Doris M. Disney Mrs. Meeker's Money L16 Alex Gordon The Cipher L17 Brent James Night of the Kill L18 George H. Coxe Error of Judgment L19 Brad Williams Make a Killing L20 Ed Lacy Death by the Numbers L21 Helen McCloy The Black Disk L22 S. L. M. Barlow Monologue of Murder L23 J. W. Rose Try My Sample Murders L24 Fredric Brown The Murders M01
Robert Heinlein Stranger in a Strange Land M02 Philip J. Farmer The Lovers M03 James Blish The Star Dwellers M04 Jim Harmon The Planet with No Nightmare M05 Anne McCaffrey The Ship Who Sang M06 Cordwainer Smith A Planet Named Shayol N0l
Wayne D. Overholser The Killer Marshall N02 Clifford Irving The Valley N03 Cliff Farrell The Trail of the Tattered Star
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
File Author Source N04 James D. Horan The Shadow Catcher N05 Richard Ferber Bitter Valley N06 Thomas Anderson Here Comes Pete Now N07 Todhunter Ballard The Night Riders N08 Mary Savage Just for Tonight N09 Jim Thompson The Transgressors N10 Joseph Chadwick No Land Is Free N11 Gene Caesar Rifle for Rent N12 Edwin Booth Outlaw Town N13 Martha F. McKeown Mountains Ahead N14 Peter Field Rattlesnake Ridge N15 Donald J. Plantz Sweeney Squadron N16 Ralph J. Salisbury On the Old Sante Fe Trail to Siberia N17 Richard S. Prather The Bawdy Beautiful N18 Peter Bains With Women Education Pays off Nl9 David Jackson The English Gardens N20 T. C. McClary The Flooded Dearest N21 C. T. Sommers The Beautiful Mankillers of Eromonga N22 Gordon Johnson A Matter of Curiosity. N23 Wheeler Hall Always Shoot to Kill N24 T. K. Brown III The Fifteenth Station N25 Wesley Newton Aid & Comfort to the Enemy N26 Paul Brock Toughest Lawman in the Old West N27 James Hines & James Morris Just Any Girl N28 Ralph Grimshaw Mrs. Hacksaw, New Orleans Society Killer N29 Harlan Ellison Riding the Dark Train Out P01
Octavia Waldo A Cup of the Sun P02 Ann Ritner Seize a Nettle P03 Clark McMeekin The Fairbrothers P04 B. J. Chute The Moon & the Thorn P05 Allan R. Bosworth The Crows of Edwina Hill P06 Richard Tiernan Land of the Silver Dollar P07 Vina Delmar The Big Family P08 R. Leslie Course With Gall & Honey P09 Jesse Hill Ford Mountains of Gilead P10 Jay Williams The Forger P11 Bessie Breuer Take Care of My Roses P12 Morley Callaghan A Passion in Rome P13 Frank B. Hanes The Fleet Rabble P14 Livingston Biddle, Jr. Sam Bentley's Island P15 Loretta Burrough The Open Door P16 Margery F. Brown A Secret Between Friends P17 Al Hine The Huntress
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
File Author Source P18 Anonymous No Room in My Heart to For Give Pl9 Anonymous This Cancer Victim May Ruin My Life P20 Spencer Norris Dirty Dog Inn P21 Elizabeth Spencer The White Azalea P22 Anonymous A Husband Stealer from Way Back P23 Barbara Robinson Something Very Much in Common P24 Samuel Elkin The Ball Player P25 William Butler The Pool at Ryusenji P26 Ervin D. Krause The Snake P27 Lee McGiffin Measure of a Man P28 Carol Hoover The Shorts on the Bedroom Floor P29 Robert Carson My Hero R01
Anita Loos No Mother to Guide Her R02 Jean Mercier Whatever You Do, Don't Panic R03 Patrick Dennis Little Me R04 Edward Streeter The Chairman of the Bored R05 Evan Esar Humorous English R06 James Thurber The Future, If Any, of Comedy R07 John H. Wildman Take It Off R08A Leo Lemon Catch Up With R08B Leo Lemon Something to Talk About R09 S. J. Perelman The Rising Gorge
Appendix B: Proportional Placement Statistics
The statistics below come from 40 randomly picked SemCor files. See Cost Function Method 3
in the Cost Function section for more details about these numbers. The POS‐POS column represents the
parts of speech of the word pair combination. (NN = Noun, JJ = Adjective, VB = Verb, RB = Adverb)
Table 6: The values for Cost Function Method 3 used in this project Semantic Relation POS‐POS Min Max Average Standard Deviation Frequency NN‐JJ 0.792254615 0.918299831 0.864021328 0.026870632 Frequency NN‐RB 0.808804114 0.919316349 0.862272345 0.027024865 Frequency NN‐NN 0.775888726 0.910315112 0.849278482 0.028957857 Frequency NN‐VB 0.775888726 0.910315112 0.849278482 0.028957857 Frequency NN‐VB 0.762541491 0.897634401 0.826850626 0.029391924 Frequency VB‐JJ 0.777054072 0.921952008 0.848030644 0.032030693 Hypernym VB‐VB 0 0.173931701 0.049567971 0.033316047 Frequency JJ‐RB 0.804506863 0.962140412 0.87887566 0.033752072
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Appendix C: Sense Distribution Statistics
The statistics below come from the “Brown 1” SemCor files. See Cost Function Method 4 in the
Cost Function section for more details about these numbers. Note that this table assumes the most
common sense is sense 0.
Table 7: The values for Cost Function Method 4 used in this project Sense Value Sense Value Sense Value 0 0.745002967 17 0.000179044 34 0 1 0.143447519 18 0.000144862 35 0 2 0.051918954 19 0.000114338 36 0 3 0.024731353 20 0.000141419 37 0 4 0.013769042 21 7.17999E‐05 38 0 5 0.006769587 22 4.29754E‐05 39 0 6 0.004881351 23 7.21787E‐05 40 0 7 0.002858328 24 1.13685E‐05 41 0 8 0.001849772 25 0 42 0 9 0.000993421 26 3.30413E‐05 43 0 10 0.000760418 27 1.15856E‐05 44 0 11 0.000593476 28 1.10957E‐05 45 0 12 0.000572185 29 1.18544E‐05 46 0 13 0.000436927 30 2.06378E‐05 47 0 14 0.000215302 31 0 48 0 15 0.000170978 32 0 49 0 16 0.000152483 33 9.73795E‐06
Appendix D: Semantic Relation Distribution Statistics
The statistics below come from 40 randomly picked SemCor files. See Cost Function Method 5
in the Cost Function section for more details about these numbers. Note that this table assumes the
most common sense is sense 0.
Table 8: The values for Cost Function Method 5 used in this project Type Sense Value Type Sense Value Frequency 0 0.781961482 Synonym 0 0.708127392 Frequency 1 0.126728106 Synonym 1 0.14722019 Frequency 2 0.043575442 Synonym 2 0.058309409
Hausman 95
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Type Sense Value Type Sense Value Frequency 3 0.020242591 Synonym 3 0.035265127 Frequency 4 0.011330363 Synonym 4 0.02488885 Frequency 5 0.005382083 Synonym 5 0.008951525 Frequency 6 0.003820112 Synonym 6 0.006850447 Frequency 7 0.002217267 Synonym 7 0.003343709 Frequency 8 0.001386978 Synonym 8 0.002257055 Frequency 9 0.000719614 Synonym 9 0.000917186 Frequency 10 0.000550252 Synonym 10 0.000415363 Frequency 11 0.000428899 Synonym 11 0.001272953 Frequency 12 0.000401828 Synonym 12 0 Frequency 13 0.000303701 Synonym 13 0.000649926 Frequency 14 0.000142087 Synonym 14 0.000374687 Frequency 15 0.000122769 Synonym 15 0 Frequency 16 9.93788E‐05 Synonym 16 6.48281E‐05 Frequency 17 0.000126437 Synonym 17 0.000302171 Frequency 18 9.51612E‐05 Synonym 18 0.000120573 Frequency 19 7.45493E‐05 Synonym 19 0 Frequency 20 9.84838E‐05 Synonym 20 0.000268828 Frequency 21 4.59405E‐05 Synonym 21 5.53748E‐05 Frequency 22 2.64911E‐05 Synonym 22 0 Frequency 23 4.80187E‐05 Synonym 23 7.21131E‐05 Frequency 24 6.51078E‐06 Synonym 24 0 Frequency 25 0 Synonym 25 0 Frequency 26 2.37718E‐05 Synonym 26 0 Frequency 27 6.93931E‐06 Synonym 27 0 Frequency 28 6.74259E‐06 Synonym 28 0 Frequency 29 7.47323E‐06 Synonym 29 0 Frequency 30 1.47131E‐05 Synonym 30 0.000272292 Frequency 31 0 Synonym 31 0 Frequency 32 0 Synonym 32 0 Frequency 33 5.81201E‐06 Synonym 33 0 Frequency 34 0 Synonym 34 0 Frequency 35 0 Synonym 35 0 Frequency 36 0 Synonym 36 0 Frequency 37 0 Synonym 37 0 Frequency 38 0 Synonym 38 0 Frequency 39 0 Synonym 39 0 Frequency 40 0 Synonym 40 0 Frequency 41 0 Synonym 41 0
Hausman 96
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Type Sense Value Type Sense Value Frequency 42 0 Synonym 42 0 Frequency 43 0 Synonym 43 0 Frequency 44 0 Synonym 44 0 Frequency 45 0 Synonym 45 0 Frequency 46 0 Synonym 46 0 Frequency 47 0 Synonym 47 0 Frequency 48 0 Synonym 48 0 Frequency 49 0 Synonym 49 0 Hypernym 0 0.780778412 Antonym 0 NaN Hypernym 1 0.127113255 Antonym 1 NaN Hypernym 2 0.045911925 Antonym 2 NaN Hypernym 3 0.021512378 Antonym 3 NaN Hypernym 4 0.010228877 Antonym 4 NaN Hypernym 5 0.006096897 Antonym 5 NaN Hypernym 6 0.003875595 Antonym 6 NaN Hypernym 7 0.001693082 Antonym 7 NaN Hypernym 8 0.001142567 Antonym 8 NaN Hypernym 9 0.000634825 Antonym 9 NaN Hypernym 10 0.000521345 Antonym 10 NaN Hypernym 11 0.000139175 Antonym 11 NaN Hypernym 12 0.000156363 Antonym 12 NaN Hypernym 13 0.000138005 Antonym 13 NaN Hypernym 14 0 Antonym 14 NaN Hypernym 15 2.79886E‐05 Antonym 15 NaN Hypernym 16 9.4065E‐06 Antonym 16 NaN Hypernym 17 1.02999E‐05 Antonym 17 NaN Hypernym 18 0 Antonym 18 NaN Hypernym 19 0 Antonym 19 NaN Hypernym 20 0 Antonym 20 NaN Hypernym 21 9.6045E‐06 Antonym 21 NaN Hypernym 22 0 Antonym 22 NaN Hypernym 23 0 Antonym 23 NaN Hypernym 24 0 Antonym 24 NaN Hypernym 25 0 Antonym 25 NaN Hypernym 26 0 Antonym 26 NaN Hypernym 27 0 Antonym 27 NaN Hypernym 28 0 Antonym 28 NaN Hypernym 29 0 Antonym 29 NaN Hypernym 30 0 Antonym 30 NaN
Hausman 97
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Type Sense Value Type Sense Value Hypernym 31 0 Antonym 31 NaN Hypernym 32 0 Antonym 32 NaN Hypernym 33 0 Antonym 33 NaN Hypernym 34 0 Antonym 34 NaN Hypernym 35 0 Antonym 35 NaN Hypernym 36 0 Antonym 36 NaN Hypernym 37 0 Antonym 37 NaN Hypernym 38 0 Antonym 38 NaN Hypernym 39 0 Antonym 39 NaN Hypernym 40 0 Antonym 40 NaN Hypernym 41 0 Antonym 41 NaN Hypernym 42 0 Antonym 42 NaN Hypernym 43 0 Antonym 43 NaN Hypernym 44 0 Antonym 44 NaN Hypernym 45 0 Antonym 45 NaN Hypernym 46 0 Antonym 46 NaN Hypernym 47 0 Antonym 47 NaN Hypernym 48 0 Antonym 48 NaN Hypernym 49 0 Antonym 49 NaN Coordinate Sister 0 0.731642312 Domain 0 NaN Coordinate Sister 1 0.140189319 Domain 1 NaN Coordinate Sister 2 0.059664121 Domain 2 NaN Coordinate Sister 3 0.026321333 Domain 3 NaN Coordinate Sister 4 0.014048575 Domain 4 NaN Coordinate Sister 5 0.008563011 Domain 5 NaN Coordinate Sister 6 0.007495793 Domain 6 NaN Coordinate Sister 7 0.002041513 Domain 7 NaN Coordinate Sister 8 0.002195311 Domain 8 NaN Coordinate Sister 9 0.00124681 Domain 9 NaN Coordinate Sister 10 0.000714507 Domain 10 NaN Coordinate Sister 11 0.000830106 Domain 11 NaN Coordinate Sister 12 0.001219296 Domain 12 NaN Coordinate Sister 13 0.000939302 Domain 13 NaN Coordinate Sister 14 0.000651397 Domain 14 NaN Coordinate Sister 15 0.000227097 Domain 15 NaN Coordinate Sister 16 0.000161478 Domain 16 NaN Coordinate Sister 17 0.000455964 Domain 17 NaN Coordinate Sister 18 0.000287357 Domain 18 NaN Coordinate Sister 19 0.000232809 Domain 19 NaN
Hausman 98
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Type Sense Value Type Sense Value Coordinate Sister 20 0.000430405 Domain 20 NaN Coordinate Sister 21 0 Domain 21 NaN Coordinate Sister 22 0 Domain 22 NaN Coordinate Sister 23 0.000109042 Domain 23 NaN Coordinate Sister 24 0 Domain 24 NaN Coordinate Sister 25 0 Domain 25 NaN Coordinate Sister 26 0 Domain 26 NaN Coordinate Sister 27 0 Domain 27 NaN Coordinate Sister 28 0.000117354 Domain 28 NaN Coordinate Sister 29 4.43262E‐05 Domain 29 NaN Coordinate Sister 30 0.000171463 Domain 30 NaN Coordinate Sister 31 0 Domain 31 NaN Coordinate Sister 32 0 Domain 32 NaN Coordinate Sister 33 0 Domain 33 NaN Coordinate Sister 34 0 Domain 34 NaN Coordinate Sister 35 0 Domain 35 NaN Coordinate Sister 36 0 Domain 36 NaN Coordinate Sister 37 0 Domain 37 NaN Coordinate Sister 38 0 Domain 38 NaN Coordinate Sister 39 0 Domain 39 NaN Coordinate Sister 40 0 Domain 40 NaN Coordinate Sister 41 0 Domain 41 NaN Coordinate Sister 42 0 Domain 42 NaN Coordinate Sister 43 0 Domain 43 NaN Coordinate Sister 44 0 Domain 44 NaN Coordinate Sister 45 0 Domain 45 NaN Coordinate Sister 46 0 Domain 46 NaN Coordinate Sister 47 0 Domain 47 NaN Coordinate Sister 48 0 Domain 48 NaN Coordinate Sister 49 0 Domain 49 NaN
Appendix E: SemCor Results
Below are the results for every SemCor file that tags nouns, verbs, adjectives, and adverbs. The
authors ignore all files only tagging verbs. See the results section for more details.
Hausman 99
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
References
ACL‐SIGLEX. (2011, March). Retrieved from SENS EVAL: http://www.senseval.org
Basile, P., Degemmis, M., Gentile, A. L., Lops, P., & Semeraro, G. (2007). The JIGSAW Algorithm for Word Sense Disambiguation and Semantic Indexing of Documents. Proceedings of the 10th Congress of the Italian Association for Artificial Intelligence on AI*IA 2007: Artificial Intelligence and Human‐Oriented Computing, (pp. 314‐325). Rome, Italy.
Billot, M. (2010). Combing Lexical Resources for Text Analysis of Game Walkthroughs. Summer 2010 National Science Foundation REU. Colorado Springs.
Bosch, B. D., Hoste, V., Daelemans, W., & Den, W. V. (2004). GAMBL, Genetic Algorithm Optimization of Memory‐Based WSD. Proceedings of ACL/SIGLEX Senseval‐3, (pp. 108‐112).
Gerber, M. (2010, October). Retrieved from NLP Software: http://links.cse.msu.edu:8000/members/matt_gerber/index.php/NLP_software#WordNet_API
Google. (2010). Retrieved from Google Translate: http://translate.google.com/?hl=en&tab=wT#
Hausman, M. (2009). A Dominant Gene Genetic Algorithm for a Substitution Cipher in Cryptography. CS591 at the University of Colorado. Colorado Springs.
Hausman, M. (2009). A Dominant Gene Genetic Algorithm for a Transposition Cipher in Cryptography. CS‐571 at the University of Colorado. Colorado Springs.
Jiang , J. J., & Conrath, D. W. (1997). Semantic Similarity Based on Corpus Statistics and Lexical Taxonomy . In International Conference Research on Computational Linguistics , (pp. 19‐33).
Leacock, C., & Chodorow, M. (1998). Combining Local Context and WordNet Similarity for Word Sense Identification. In C. Fellbaum (Ed.), WordNet: An Electronic Lexical Database (pp. 305‐332). MIT Press.
Lesk, M. (1986). Automatic Sense Disambiguation using Machine Readable Dictionaries: How to Tell a Pine Cone from an Ice Cream Cone. Proceedings of the 5th Annual International Conference on Systems Documentation, (pp. 24‐26). Toronto, Ontario, Canada.
Lin, D. (1998). An Information‐Theoretic Definition of Similarity. In Proceedings of the 15th International Conference on Machine Learning, (pp. 296‐304).
Mihalcea, R., & Csomai, A. (2005). SenseLearner: Word Sense Disambiguation for All Words in Unrestricted Text. Proceedings of the ACL 2005 on Interactive Poster and Demonstration Sessions, (pp. 53‐56). Stroudsburg, PA.
Miller, G. A. (n.d.). WordNet: A Lexical Database for English. Communications of the ACM, 38(11), 39‐41.
Hausman 105
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Navigli, R. (2007). SemEval‐2007 Task 07: Coarse‐Grained English All‐Words Task. SemEval 2007.
Patwardhan, S. (2006). Using WordNet‐Based Context Vectors to Estimate the Semantic Relatedness of Concepts. Proceedings of the EACL, (pp. 1‐8).
Patwardhan, S., Banerjee, S., & Pedersen, T. (2002). An Adapted Lesk Algorithm for Word Sense Disambiguation Using WordNet. Proceedings of the Third International Conference on Computational Linguistics and Intelligent Text Processing, (pp. 136‐145). London, UK.
Patwardhan, S., Banerjee, S., & Pedersen, T. (2003). Using Measures of Semantic Relatedness for Word Sense Disambiguation. Proceedings of the Fourth International Conference on Intelligent Text Processing and Computational Linguistics, (pp. 241‐257). Mexico City, Mexico.
Patwardhan, S., Banerjee, S., & Pedersen, T. (2007). UMND1: Unsupervised Word Sense Disambiguation using Contextual Semantic Relatedness. Proceedings of the 4th International Workshop on Semantic Evaluations, (pp. 390‐393). Prague, Czech Republic.
Patwardhan, S., Banerjee, S., & Pedersen, T. (2007). UMND1: Unsupervised Word Sense Disambiguation Using Contextual Semantic Relatedness. Proceedings of the 4th International Workshop on Semantic Evaluations, (pp. 390‐393). Prague, Czech Republic.
Princeton University. (2010). About WordNet. Retrieved from About WordNet: http://wordnet.princeton.edu
Princeton University. (2011, March). Retrieved from The SemCor Corpus: http://multisemcor.fbk.eu/semcor.php
Rada, R., Mili, H., Bicknell, E., & Blettner, M. (1989, Jan/Feb). Development and Application of a Metric on Semantic Nets. Systems, Man and Cybernetics, IEEE Transactions, 19(1), 17‐30.
Raytheon BBN Technologies, the University of Colorado, the University of Pennsylvania, and the University of Southern California's Information Sciences Institute . (2011, March). Retrieved from OntoNotes: http://www.bbn.com/ontonotes/
Resnik, P. (1995). Using Information Content to Evaluate Semantic Similarity in a Taxonomy. Proceedings of the 14th International Joint Conference on Artificial Intelligence, (pp. 448‐453).
Snyder, B., & Palmer, M. (2004). The English all‐words task. Proceedings of the 3rd International Workshop on the Evaluation of Systems for the Semantic Analysis of Text, (pp. 41‐43). Barcelona, Spain.
Tsuruoka, Y., & Tsujii, J. (2005). Bidirectional Inference with the Easiest‐First Strategy for Tagging Sequence Data. HLT/EMNLP, (pp. 467‐474).
Wu, Z., & Palmer, M. S. (1994). Verb Semantics and Lexical Selection. Proceedings of the 32th Annual Meeting on Association for Computational Linguistics, (pp. 133‐138). Las Cruces, New Mexico.
Hausman 106
A Genetic Algorithm Using Semantic Relations for Word Sense Disambiguation
Zesch, T., & Gurevich, I. (2010). Wisdom of Crowds Versus Wisdom of Linguists: Measuring the Semantic Relatedness of Words. Natural Language Engineering, 16(01), 1351‐3249.
Zhang, C., Zhou, Y., & Martin, T. (2008). Genetic Word Sense Disambiguation Algorithm. Intelligent Information Technology Applications, (pp. 123‐127).
Zhong, Z., Ng, H. T., & Chan, Y. S. (2008). Word sense disambiguation using OntoNotes: an empirical study. Proceedings of the Conference on Empirical Methods in Natural Language Processing, (pp. 1002‐‐1010). Honolulu, Hawaii.