A generalized framework for anaphora resolution in Indian …sriparna/papers/general.pdf · 2018. 10. 1. · tion technique. A recent study for feature selection in anaphora resolution
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Knowledge-Based Systems 109 (2016) 147–159
Contents lists available at ScienceDirect
Knowle dge-Base d Systems
journal homepage: www.elsevier.com/locate/knosys
A generalized framework for anaphora resolution in Indian languages
Utpal Kumar Sikdar, Asif Ekbal ∗, Sriparna Saha
Department of Computer Science and Engineering Indian Institute of Technology Patna, India
a r t i c l e i n f o
Article history:
Received 16 May 2015
Revised 27 June 2016
Accepted 28 June 2016
Available online 6 July 2016
Keywords:
Multiobjective optimization (MOO)
Single objective optimization (SOO)
Conditional random field (CRF)
Support vector machine (SVM)
a b s t r a c t
In this paper, we propose a joint model of feature selection and ensemble learning for anaphora reso-
lution in the resource-poor environment like the Indian languages. The proposed approach is based on
multi-objective differential evolution (DE) that optimises five coreference resolution scorers, namely Muc ,
Bcub , Ceafm , Ceafe and Blanc . The main goal is to determine the best combination of different mention
classifiers and the most relevant set of features for anaphora resolution. The proposed method is evalu-
ated for three leading Indian languages, namely Hindi, Bengali and Tamil. Experiments on the benchmark
datasets of ICON-2011 Shared Task on Anaphora Resolution in Indian Languages show that our proposed
approach attains good level of accuracies, which are often better with respect to the state-of-the-art sys-
tems. It achieves the F-measure values of 71.89%, 59.61%, 52.55% 34.45% and 72.52% for Muc , Bcub , Ceafm ,
Ceafe and Blanc , respectively, for Bengali language. For Hindi we obtain the F-measure values of 33.27%,
63.06%, 49.59%, 49.06% and 55.45% for Muc , Bcub , Ceafm , Ceafe and Blanc metrics, respectively. In order
to further show the efficacy of our proposed algorithm, we evaluate with Tamil, a language that belongs
to a different family. This shows the F-measure values of 31.79%, 64.67%, 46.81%, 45.29% and 52.80% for
Muc , Bcub , Ceafm , Ceafe and Blanc metrics, respectively. Experiments on Dutch show the F-measure
values of 17.67%, 74.43%, 58.08%, 59.21% and 55.58% for Muc , Bcub , Ceafm , Ceafe and Blanc metrics,
omplex, large and multi-modal landscapes, and provides near-
ptimal solutions. Parameters in the search space are encoded
n the form of chromosomes. Each chromosome is denoted by
D -dimensional parameter vector X i,G = [ x 1 ,i,G , x 2 ,i,G , . . . , x D,i,G ] , i = , 2 , . . . , NP, where NP is the number of solutions in the popu-
ation. For multiobjective version more than one objective or fit-
ess functions are associated with each chromosome. The algo-
ithm generates new parameter vector by adding the weighted dif-
erence of any two vectors to a third one, and this operation is
alled the mutation. The parameters of the mutated vectors are
ixed with the parameters of another predetermined vector, the
arget vector, to yield a new vector known as the trial vector. The
rocess of parameter mixing is often referred to as crossover. Se-
ection operation refers to the process of selecting the effective so-
utions. In this process the trial vectors are merged to the current
opulation and then ranked based on the concept of domination
nd non-domination. In the next generation we select NP num-
er of chromosomes from the ranked solutions using the crowd-
ng distance sorting algorithm. The process of selection, crossover
nd mutation continues for a fixed number of generations or till a
ermination condition is satisfied. The crowding distance d i of each
oint i in the non-dominated front I [8] is computed as follows:
• For i = 1 , . . . , I, initialize d i = 0 .
• For each objective function f k , k = 1 , . . . , K, do the following:
• Sort the set I according to f k in ascending order.
• Set d 1 = d | I| = ∞ .
• For j = 2 to (| I| − 1) , set d j = d j + ( f k ( j+1) − f k ( j−1) ) .
.4.2. Problem formulation
Suppose, there are N classifiers (denoting mention detection
odels) and M features. Weights for combining N classifiers are
enoted by W 1 , . . . , W N , where A = { W i : i = 1 ; N} . Feature values
orrespond to B = { F i : i = 1 ; M} . The problem of ensemble con-
truction and feature selection can be stated as follows: Compute
he appropriate weights for each classifier to combine the outputs
f various mention detectors, and determine the subset of features
′ ⊆ B such that when the anaphora resolver is trained on this sub-
et of features using the combined model for mention detection
hould have optimized some metrics. In our proposed MOO based
ifferential evolution (DE) setting, we optimize five objective func-
ions, namely the F-measure values corresponding to Muc , Bcub ,
eafm , Ceafe and Blanc scorers. All these metrics represent sig-
ificantly different behaviors. Our multi-objective DE based joint
odel selects the best weights for combining the outputs of clas-
ifiers for mention detection, and chooses the subset of features
best features) to maximize the five metrics simultaneously. Details
f the procedure for joint modelling are described below:
.4.3. Problem encoding and population initialization:
Problems are encoded as real-valued strings of length D (also
alled chromosomes) equals to N + M, where N is the number of
ention classifiers and M is the number of available features used
o construct the anaphora resolution system. A collection of such
ype of D length chromosomes is called a population. Size of the
opulation (total number of chromosomes) is denoted by NP . All
he NP number of chromosomes are randomly initialized with the
eal values between 0 to 1. A fitness function is associated with
ach chromosome which is having N number of classifiers and M
umber of features. More the weight of a classifier denotes more
onfidence for markable (mention) selection from the respective
odel. If the value of the bit is ≥ 0.5 then it represents that the
espective feature is used for training of anaphora resolver; oth-
rwise the feature is not used. An example of a chromosome is
hown in Fig. 3 that shows four classifiers and five features. Here,
alues of the first four bits denote the weights by which the classi-
Bengali (4 solutions): GEN − F P P − T P P − RP − SM − SCF −D − SK − CC − NP SM, GEN − F P P − SP P − T P P − SCF − MT − SK −P SM − P NSM, F P P − T P P − RP − AF − SCF − MT − AP F − SD − SK − SM − P NSM and RF − AF − MT − SD − SK
Hindi (5 solutions): GEN − NA − T P P − RP − AF − SCF − MT −D − SK − NP SM − P N SM, N A − T P P − RP − AF − SM − SCF − AP F −K − CC, F P P − SP P − T P P − AF − SM − AP F − MD − SD − SK −P SM − P NSM, GEN − NA − SP P − T P P − RP − AF − SM − AP F −D − SK and NA − RP − SM − MT − AP F − SK − NP SM.
Tamil (5 solutions): LRM − NA − T P P − RP − SCF − MT − AP F −D − MD, LRM − NA − SP P − AF − SM − AP F − MD, LRM − RP − AF −CF − MD − GEN, SK − SCF − SD − MD − GEN and LRM − F P P − P P − SCF − MD
Dutch (3 solutions): MD − SP P − T P P − SCF − AP F − P SM − NSM − GEN, SD − MD − T P P − SCF − AP F − P SM − P NSM −EN and MD − SP P − SCF − AP F − P SM − P NSM
Results of experiments for all these feature combinations are
resented in Tables 7–10 for Bengali, Hindi, Tamil and Dutch, re-
pectively. Results reported in these tables are generated based
n the solutions obtained on the first rank of the final Pareto
ront. The final score for each evaluation metric corresponds to the
ighest value obtained in any of the solutions of the final Pareto
ront. The best performance achieved by four solutions reported in
able 7 correspond to 71.89%, 59.61%, 52.55%, 34.45% and 72.52%
or Muc , Bcub , Ceafm , Ceafe and Blanc , respectively, for Bengali.
or Hindi the best performance achieved corresponds to the F-
easures of 33.23%, 63.06%, 4 9.59%, 4 9.06% and 55.45% for Muc ,
cub , Ceafm , Ceafe and Blanc , respectively. For Tamil the obtained
-measure values are 31.79%, 64.67%, 46.81%, 45.29% and 52.80%
or Muc , Bcub , Ceafm , Ceafe and Blanc scorers, respectively. Ex-
eriments on Dutch demonstrate the F-measure values of 17.67%,
4.43%, 58.08%, 59.21% and 55.58% for Muc , Bcub , Ceafm , Ceafe
nd Blanc scorers, respectively. A closer analysis of these results
hows the effectiveness of the proposed approach. It shows consis-
ent performance improvements in all the settings over the system
hat makes use of single mention classifier, and anaphora resolver
s trained either with all the features (c.f. ‘All Features’ columns)
r the optimized feature set (c.f. ‘Selected Features’ columns). The
ptimized feature set denotes the set of relevant features as deter-
ined by the multi-objective DE based feature selection technique,
hen it is executed in isolation (i.e., not with ensemble construc-
ion for mention detection). In two other experiments we also ob-
erve that our approach attains better accuracy over the system
hat uses the combined model for mention detection, but anaphora
esolver is trained either with all the available features or the op-
imized feature set. Therefore, we can argue that our proposed
oint model for feature selection and ensemble learning works ef-
ectively for all the languages. We have also conducted ANOVA
3] analysis to exhibit that performance increment by our proposed
pproach is statistically significant.
.1. Comparisons with SOO based joint model and isolated models of
nsemble construction and feature selection
In order to compare with the MOO based technique, we im-
lement a joint model using the concept of single objective DE
been reported in [6] , where they used dependency structures for
anaphora resolution. Their experiments were carried out on differ-
ent datasets, which are not publicly available, and therefore, the
comparison can’t be directly made. Comparisons with our existing
works [32,33] also show that joint modeling is more effective.
In [13] , a named entity recognition (NER) system was devel-
oped for friction domain. Fiction based domain has a complex con-
text in locating the corresponding named entities (NEs), specifi-
cally whereby its characters could be represented in diverse spec-
trum, ranging from living things (animals, plants, and person) to
non-living things (vehicle, furniture). This NER system was evalu-
ated on Fables and Fairy tales corpus. It was also shown that the
application of anaphora resolution does not play a significant role
in the proposed NER system. The proposed NER system was ap-
plied on the original corpus and the corpus after application of
anaphora resolution. Though there are some differences in recall
and precision values but F-measure values remain unaltered. As
mentioned in [13] the proposed system was for detecting NEs from
friction domain, and because of this authors were not able to com-
pare it with the other existing NER systems. Hence, we are also
not able to compare directly as our corpus and languages are not
suited for the system proposed in [13] .
In [12] , a dialogue system is developed. In this paper, a
cognitively-inspired representational model has been proposed to
address the research question of how to enable dialogue systems
to capture the meaning of spontaneously produced linguistic in-
puts without explicit syntactic expectations. The proposed model
is cognitively-inspired and thus it integrates insights from be-
havioural and neuroimaging studies on working memory opera-
tions and language-impaired patients (i.e., Brocas aphasics). As this
paper deals with dialogue system so comparison with the existing
approach would not be feasible.
In [19] , a spoken dialogue system is developed. This paper pro-
poses a technique to improve the performance of spoken dia-
logue systems that not only consider knowledge about the seman-
tic frames used by systems to understand the spoken language but
also employ knowledge about the words in the system application
domain that are used to complete frame slots. As this system deals
with the spoken dialogue system, and hence it would not be feasi-
ble to compare our proposed system with this.
3.3. Complexity
The complexity of the algorithm mainly depends on the size
of the training data, number of available features and the num-
ber of mention detection models etc. Let us assume that D, NP and
G Max represent the length of the chromosome, number of chromo-
somes in a population and the maximum number of generations,
respectively. The complexity can then be represented as: O( D ×NP × G Max ). For ensemble and feature selection in Bengali, average
learning time for each chromosome is almost 21 s. Hence, the total
time required is equal to 21 ∗40 ∗50 s (50-size of the population).
Once optimal weights and features are selected by multiobjective
DE, 80 s are required for testing. For Hindi, the time for learning
is 32 ∗40 ∗50 with a rate of 32 s for each chromosome. The testing
ime for all the five solutions is 155 s. Experiments were carried
ut on a Linux environment with machine having 8 GB memory,
ntel(R) Core(TM) i7-4510U CPU @ 2.00 GHz and the cache size of
MB.
.4. Challenges of NLP in Indian languages
India is a multilingual country with a very rich cultural history.
ndian languages are derived from most of the existing language
amilies. Although Indian languages have very old literary tradition,
echnological developments are of recent origin. Indian languages
re not resource-rich in nature. The greatest bottleneck in devel-
ping NLP systems involving Indian languages are the availability
f resources and tools such as annotated corpora, PoS tagger, mor-
hological analyzer, named entity tagger etc. in the required mea-
ure. Developing good accurate anaphora resolution system is lim-
ted by all these constrains. Indian languages are morphologically
ery rich. Most of the times, same word has different meanings.
ence it is very difficult to determine the actual semantic mean-
ng of a word. For example ‘kAshI’ is a Bengali word which may
orrespond to three different concepts. It denote a person name or
ocation name or a disease name. In some languages, gender infor-
ation has great influence, while on others it does not have any
ffect. For Hindi depending upon gender verb form changes, but
or Bengali it does not change. Pronouns may also appear in differ-
nt forms. For example ‘Ami’ is a pronoun in Bengali, and this can
e written at least in 10 different forms. Same word may appear
n different forms. Often, a word that appears as a part of a mark-
ble may not be a part in some other places. This ambiguity affects
he performance of mention detection model which, in turn, affects
he overall performance of anaphora resolution. Three languages
hat we have dealt with in our work have dissimilar characteristics.
uch peculiarities involved in Indian languages make anaphora res-
lution task a more complex problem to solve. All these challenges
ake it more harder to come up with a robust set of features for
he task.
. Conclusion
In this paper we propose a joint model for ensemble learning
nd feature selection for anaphora resolution. The algorithm can
etermine the best weights for combining the decisions of various
ention classifiers and at the same time it can also find out the
ost relevant set of features for anaphora resolution. The system
s optimized by implementing a multi-objective verion of DE, and
he five well-known coreference resolution evaluation metrics are
ptimized. The proposed model has been evaluated for the less-
esourced Indian languages, namely Bengali, Hindi and Tamil. Us-
ng the same configuration, we also apply our model to a com-
letely new language, namely Dutch and achieve good accuracies
or all the scorers.
Evaluation on a benchmark setup shows that the performance
chieved by our proposed model is encouraging, often better com-
ared to the state-of-the-art systems. In the current setting we
sed only decision tree as the machine learning algorithm. Exper-
ments with other machine learning algorithms such as maximum
U. Kumar Sikdar et al. / Knowledge-Based Systems 109 (2016) 147–159 159
e
f
s
h
fi
o
R
[
[
[
[
[
[
[
[
[
[
[
[
[
[
[
[
[
[
[
[
ntropy and support vector machine would be another direction
or future work. We would also like to concentrate on porting the
ystems to other domains (e.g., biomedical texts). In this work we
ave evaluated on one of the non-Indian languages to show the ef-
cacy of our proposed approach. In future we would like to carry
ut experiments on other international languages.
eferences
[1] H. Adel , H. Schütze , Using mined coreference chains as a resource for a se-mantic task, in: Proceedings of the 2014 Conference on Empirical Methods in
Natural Language Processing (EMNLP), Association for Computational Linguis-
tics, Doha, Qatar, 2014, pp. 1447–1452 . [2] S. Agarwal , M. Srivastava , P. Agarwal , R. Sanyal. , Anaphora resolution in hindi
documents, in: Proceedings of Natural Language Processing and KnowledgeEngineering (IEEE NLP-KE), Beijing, China, 2007, pp. 452–458 .
[3] T.W. Anderson , S.L. Scolve , Introduction to the Statistical Analysis of Data, 1978 .[4] A. Bagga , B. Baldwin , Algorithms for scoring coreference chains, in: In The First
International Conference on Language Resources and Evaluation Workshop onLinguistics Coreference, 1998, pp. 563–566 .
[5] P. Chaimongkol , A. Aizawa , Y. Tateisi , Corpus for Coreference Resolution on
Moreno, Jan Odijk, and Stelios Piperidis, editors, Proceedings of the Ninth In-ternational Conference on Language Resources and Evaluation (LREC’14), Euro-
pean Language Resources Association (ELRA), Reykjavik, Iceland, 2014 . [6] P. Dakwale , V. Mujadia , D.M. Sharma , A Hybrid Approach for Anaphora Reso-
lution in Hindi, in: Proceedings of the Sixth International Joint Conference on
Natural Language Processing, Asian Federation of Natural Language Processing,Nagoya, Japan, 2013, pp. 977–981 .
[7] K. Deb , Multi-Objective Optimization Using Evolutionary Algorithms, John Wi-ley & Sons, Inc., New York, NY, USA, 2001 .
[8] K. Deb , A. Pratap , S. Agarwal , T. Meyarivan , A fast and elitist multiobjectivegenetic algorithm: NSGA-II, IEEE Trans. Evol. Comput. 6 (2) (2002) 181–197 .
[9] S.L. Devi , V.S. Ram , P.R. Rao , A generic anaphora resolution engine for indian
languages, in: Proceedings of COLING-14, 2014, pp. 1824–1833 . [10] A. Ekbal , S. Saha , Joint model for feature selection and parameter opti-
mization coupled with classifier ensemble in chemical mention recognition,Knowl.Based Syst. 85 (2015) 37–51 .
[11] M. García , P. Gamallo , Entity-centric coreference resolution of person entitiesfor open information extraction, Procesamiento del Lenguaje Natural 53 (2014)
25–32 .
[12] M. Gnjatovic , V. Delic , Cognitively-inspired representational approach to mean-ing in machine dialogue, Knowl. Based Syst. 71 (2014) 25–33 .
[13] H.-N. Goh , L.-K. Soon , S.-C. Haw , Automatic dominant character identificationin fables based on verb analysis - empirical study on the impact of anaphora
resolution., Knowl. Based Syst. 54 (2013) 147–162 . [14] C. Hardmeier , Discourse in Statistical Machine Translation. A Survey and a Case
Study, 2012 PhD thesis .
[15] V. Hoste , Optimization Issues in Machine Learning of Coreference Resolution,Antwerp University, 2005 Phd thesis .
[16] A . Iftene , M.A . Moruz , E. Ignat , Using anaphora resolution in a question an-swering system for machine reading evaluation, in: Working Notes for CLEF
2013 Conference, Valencia, Spain, 2013, pp. 23–26 . 2013 [17] J.D. Lafferty , A. McCallum , F.C.N. Pereira , Conditional random fields: proba-
bilistic models for segmenting and labeling sequence data, in: ICML, 2001,
pp. 282–289 . [18] S. Loáiciga , E. Wehrli , Rule-based pronominal anaphora treatment for machine
translation, in: Proceedings of the Second Workshop on Discourse in MachineTranslation, Association for Computational Linguistics, Lisbon, Portugal, 2015,
pp. 86–93 . [19] R. López-Cózar , Using knowledge on word-islands to improve the performance
of spoken dialogue systems, Knowl. Based Syst. 88 (2015) 223–243 . 20] X. Luo , On coreference resolution performance metrics, in: Proceedings of the
Conference on Human Language Technology and Empirical Methods in Nat-
ural Language Processing, HLT ’05, Association for Computational Linguistics,Stroudsburg, PA, USA, 2005, pp. 25–32 .
[21] C. Ma , J.R. Doppa , J.W. Orr , P. Mannem , X. Fern , T. Dietterich , P. Tadepalli ,Prune-and-score: learning for greedy coreference resolution, in: Proceedings
of the 2014 Conference on Empirical Methods in Natural Language Process-ing (EMNLP), Association for Computational Linguistics, Doha, Qatar, 2014,
pp. 2115–2126 .
22] V. Ng , C. Cardie , Improving machine learning approaches to coreference reso-lution, in: Proceedings of the 40th Annual Meeting of the Association for Com-
putational Linguistics, 2002, pp. 104–111 . 23] M. Novk , Utilization of anaphora in machine translation, in: WDS’11 Pro-
ceedings of Contributed Papers, Part I, Matfyzpress, Praha, Czechia, 2011,pp. 155–160 .
[24] M. Novk , D. Oele , G. van Noord , Comparison of coreference resolvers for deepsyntax translation, in: Proceedings of the Second Workshop on Discourse in
Machine Translation, Association for Computational Linguistics, Lisboa, Portu-gal, 2015, pp. 17–23 .
25] J.R. Quinlan , Programs for Machine Learning, Morgan Kaufmann Publishers Inc.,San Francisco, CA, USA, 1993 .
26] A. Rahman , V. Ng , Coreference resolution with world knowledge, in: Proceed-ings of the 49th Annual Meeting of the Association for Computational Lin-
guistics: Human Language Technologies - Volume 1, HLT ’11, Association for
Computational Linguistics, Stroudsburg, PA, USA, 2011, pp. 814–824 . [27] M. Recasens , E. Hovy , A deeper look into features for coreference resolution,
Springer Berlin Heidelberg, Berlin, Heidelberg, 2009, pp. 29–42 . 28] M. Recasens , E.H. Hovy , Blanc: implementing the rand index for coreference
evaluation, Natural Lang. Eng. 17 (4) (2011) 485–510 . 29] S. Saha , A. Ekbal , O. Uryupina , M. Poesio. , Single and multi-objective optimiza-
tion for feature selection in anaphora resolution, in: Proceedings of the fifth
International Joint Conference in Natural Langauge Processing (IJCNLP 2011),2011, pp. 93–101 .
30] H. Schmolz , Anaphora resolution and text retrieval, De Gruyter Empirische Lin-guistik / Empirical Linguistics 3. De Gruyter Mouton, Berlin, 2015 .
[31] A. Senapati , U. Garain , Guitar-based pronominal anaphora resolution in ben-gali, in: ACL (2), The Association for Computer Linguistics, 2013, pp. 126–130 .
32] U. Sikdar , A. Ekbal , S. Saha , O. Uryupina , M. Poesio , Adapting a state-of-the-art
anaphora resolution system for resource-poor language, in: Proceedings of theSixth International Joint Conference on Natural Language Processing, Asian
Federation of Natural Language Processing, 2013, pp. 815–821 . [33] U.K. Sikdar , A. Ekbal , S. Saha , O. Uryupina , M. Poesio , Differential evolu-
34] L. Sobha , B. N.Patnaik , Vasisth: An anaphora resolution system for indian lan-
guages, in: Proceedings Artificial and Computational Intelligence for Decision,Control and Automation in Engineering and Industrial Applications (ACIDCA),
Monastir, Tunisia, 20 0 0 . [35] W.M. Soon , H.T. Ng , D.C.Y. Lim , A machine learning approach to coreference
resolution of noun phrases, Comput. Ling. 27 (4) (2001) 521–544 . 36] J. Steinberger , M. Poesio , M.A. Kabadjov , K. Jeek , Two uses of anaphora resolu-
tion in summarization, in: Information Processing and Management: an Inter-
national Journal, 2007, pp. 1663–1680 . [37] R. Storn , K. Price , Differential evolution a simple and efficient heuristic for
global optimization over continuous spaces., J. Global Optim. 11 (4) (1997)341–359 .
38] D.-H. Tran , M.-Y. Cheng , D. Prayogo , A novel multiple objective symbiotic or-ganisms search (mosos) for timecostlabor utilization tradeoff problem., Knowl-
edge-Based Systems 94 (2016) 132–145 .
39] B. Uppalapu , D.M. Sharma , Pronoun resolution for hindi, in: Proceedings ofDAARC, 2009 .
40] O. Uryupina , Knowledge Acquisition for Coreference Resolution, University ofthe Saarland, 2007 Phd thesis .
[41] V.N. Vapnik , The nature of statistical learning theory, Springer-Verlag NewYork, Inc., New York, NY, USA, 1995 .
42] Y. Versley , S.P. Ponzetto , M. Poesio , V. Eidelman , A. Jern , J. Smith ,X. Yang , A. Moschitti , Bart: a modular toolkit for coreference resolution, in:
HLT-Demonstrations ’08 Proceedings of the 46th Annual Meeting of the Asso-
ciation for Computational Linguistics on Human Language Technologies, 2008,pp. 9–12 .
43] M. Vilain , J. Burger , J. Aberdeen , D. Connolly , L. Hirschman , A model-theoreticcoreference scoring scheme, in: Proceedings of the 6th Conference on Message
Understanding, MUC6 ’95, Association for Computational Linguistics, Strouds-burg, PA, USA, 1995, pp. 45–52 .
44] C. Walker , S. Strassel , J. Medero , K. Maeda. , Ace 2005 Multilingual Training Cor-
pus, Linguistic Data Consortium, Philadelphia, 2006 . 45] R. Weischedel, S. Pradhan, L. Ramshaw, M. Palmer, N. Xue, M. Marcus, A.
Taylor, C. Greenberg, E. Hovy, R. Belvin, A. Houston, 2008, Ontonotes release2.0:ldc2008t04 philadelphia penn.: Linguistic data consortium.
46] S. Wiseman , A.M. Rush , S.M. Shieber , J. Weston , Learning anaphoricity and an-tecedent ranking features for coreference resolution, in: Proceedings of the
53rd Annual Meeting of the Association for Computational Linguistics and the
7th International Joint Conference on Natural Language Processing of the AsianFederation of Natural Language Processing, ACL 2015, July 26–31, 2015, Beijing,
China, Volume 1: Long Papers, 2015, pp. 1416–1426 . [47] I.H. Witten , E. Frank , Data Mining: Practical Machine Learning Tools and Tech-
niques, (Morgan Kaufmann Series in Data Management Systems), MorganKaufmann Publishers Inc., San Francisco, CA, USA, 2005 .
48] A.-C. Zavoianu , E. Lughofer , W. Koppelstätter , G. Weidenholzer , W. Amrhein ,
E. Klement , Performance comparison of generational and steady-state asyn-chronous multi-objective evolutionary algorithms for computationally-inten-
sive problems, Knowl.-Based Syst. 87 (2015) 47–60 .