Don’t compare Apples to Oranges - Extending GERBIL for a fine grained NEL Evaluation Jörg Waitelonis, Henrik Jürges , Harald Sack Hasso-Plattner-Institute for IT-Systems Engineering, University of Potsdam Semantics 2016, Leipzig, Germany, September 12-15th, 2016
23
Embed
Jörg Waitelonis, Henrik Jürges and Harald Sack | Dont compare Apples to Oranges - Extending GERBIL for a fine grained NEL evaluation
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Don’t compare Apples to Oranges -Extending GERBIL for a fine grained NEL Evaluation
Jörg Waitelonis, Henrik Jürges, Harald Sack
Hasso-Plattner-Institute for IT-Systems Engineering, University of Potsdam
Semantics 2016, Leipzig, Germany, September 12-15th, 2016
□ Annotator results are nearly the same on entities of different
popularity
□ Enable specific analyses and optimization of annotators
□ Enable users to select the tools that performs best for a specific
domain
■ Future work:
□ Keep up with GERBIL development, increase performance
□ More measurements, e. g. max_recall
□ Dataset remixing ≙ assemble new customized datasets
– E. g. Unpopular companies
Summary & Future work
Chart 21
Henrik Jürges Semantics 2016Leipzig, Germany
Don’t compare Apples to Oranges: Extending GERBIL for a fine grained NEL Evaluation
[1] Giuseppe Rizzo, Amparo Elizabeth Cano Basave, Bianca Pereira, and Andrea
Varga. Making Sense of Microposts (#Microposts2015) Named Entity rEcognition and Linking (NEEL) Challenge. In 5th Workshop on Making Sense of Microposts (#Microposts2015), pages 44–53. CEUR-WS.org, 2015
[2] M. Röder, R. Usbeck, and A.-C. Ngonga Ngomo. Gerbil’s new stunts: Semantic
[3] M. van Erp, P. Mendes, H. Paulheim, F. Ilievski, J. Plu, G. Rizzo, and J. Waitelonis.
Evaluating entity linking: An analysis of current benchmark datasets and a roadmap for doing a better job. In Proc. of 10th edition of the Language Resources and Evaluation Conference, Portoroz, Slovenia, 2016.
References
Chart 22
Henrik Jürges Semantics 2016Leipzig, Germany
Don’t compare Apples to Oranges: Extending GERBIL for a fine grained NEL Evaluation