ArguminSci A Tool for Analyzing Argumentation and ...

Anne Lauscher, ArguminSci

ArguminSci A Tool for Analyzing Argumentation and Rhetorical Aspects in Scientific Writing

Anne Lauscher, Goran Glavaš and Kai Eckert@ArgMining 2018

1


The exponential growth of scientific outputfrom 1980 to 2012(Bornmann and Lutz, 2015)

2


Computational models are already in place for many rhetorical analysis tasks ...

● citation context analysis (e.g., Jha et al., 2017)

● discourse analysis (e.g., Teufel et al., 1999; Liakata et al., 2010)

● ...

3


Computational models are already in place for many rhetorical analysis tasks ...

● citation context analysis (e.g., Jha et al., 2017)

● discourse analysis (e.g., Teufel et al., 1999; Liakata et al., 2010)

● ...

... and downstream applications.● Summarization (e.g., Cohan and Goharian, 2015)

● Research trend prediction (e.g., McKeown et al., 2016)

● Semantometrics (Herrmannova and Knoth, 2016)

● ...

4


Scientific publications are inherently argumentative (Gilbert, 1976)

„tools of persuasion“(Gilbert, 1977)

Carefully composed of different rhetorical layers(„Scitorics“)

5


”In general, our OMR preserves the highfrequency content of the motion quite well, since inverse rate control is directed by Jacobian values.”

6


”In general, our OMR preserves the highfrequency content of the motion quite well[claim], since inverse rate control is directed by Jacobian values [data].”

7


”In general, our OMR preserves the highfrequency content of the motion quite well[claim], since inverse rate control is directed by Jacobian values [data].”

8

• Subjective Aspect: advantage• Discourse Role: outcome• Summary Relevance: relevant (Fisas et al., 2016)


ArguminSci aims to support a holistic analysis of scientific publications in terms of scitorics

9


ArguminSci

1. Motivation

2. System Overview

3. Conclusion

10


ArguminSci

1. Motivation

2. System Overview

○ Annotation Tasks and Data Set

○ Annotation Models

○ Interfaces

3. Conclusion

11


System Overview:Annotation Tasks and Data Set

12


Annotation Tasks

Discourse Role ClassificationBackground, Challenge, Approach, Future Work, Outcome, Unspecified

Subjective Aspect ClassificationAdvantage, Disadvantage, Novelty, Common Practice, Limitations, None

Summary Relevance ClassificationTotally irrelevant, Should not appear, May appear, Relevant, Very relevant, None

Citation Context IdentificationB-Citation Context, I-Citation Context, Outside

Argument Component IdentificationB-I-O annotation scheme with three types of argumentative components: Own claim, Background claim, and Data

13

Sentence-levelClassification

Token-levelSequence-tagging


Dr. Inventor Corpus (Fisas et al., 2016)

Scientific discourse rolesBackground, Challenge, Approach, Future Work, Outcome

Subjective aspects and novelty classesAdvantage, Disadvantage, Novelty, Common Practice, Limitations

Summary relevance grading + SummariesTotally irrelevant, should not appear, may appear, relevant, very relevant

Citation purposeCriticism, Comparison, Basis, Use, Substantiation, Neutral

14

Sentence-levelannotations

Token-levelannotations


Extension of the corpus with fine-grained argumentative structures(Lauscher et al. 2018, derived from Toulmin, 2003; Dung 1995; Bench-Capon, 1998)

15

An argumentative statement in question related to the

background of the presented work, such as common

practices in the field or related studies.

Background Claim

Own Claim

An argumentative statement in question directly

related to the author’s own work.

Data A fact that serves as evidence in favor or against a claim.

“SSD is widely adopted in games, virtual reality, and other realtime applications due to its ease of implementation and low cost of computing.”


System Overview:Annotation Models

16


(I,OC)(B,OC) (I,OC) (I,OC)

best

Model ArchitectureToken-level tasks

17

Given a sequence of inputs x,

assign a sequence of tags y.

RNN RNNRNNRNN

Our Model performs

RNN RNN RNN RNN| | | | | | | | | | | | | | | | | | | | | | | |

Token-level classifier


…OUTCOME

best

Model ArchitectureSentence-level tasks

RNN RNNRNNRNN

Our Model performs

RNN RNN RNN RNN| | | | | | | | | | | | | | | | | | | | | | | |

Attention

| | | | | |

Sentence-level classifier

18

…


Model Performances

Granularity Task F1 (%)

Token-levelArgument Component Identification 43.8

Citation Context Identification 47.0

Sentence-level

Discourse Role Classification 42.7

Subjective Aspect Classification 18.8

Summary Relevance Classification 33.5

Evaluated on a held-out test set (2874 sentences)


Model Performances

Granularity Task F1 (%)

Token-levelArgument Component Identification 43.8

Citation Context Identification 47.0

Sentence-level

Discourse Role Classification 42.7

Subjective Aspect Classification 18.8

Summary Relevance Classification 33.5

Evaluated on a held-out test set (2874 sentences)

Models can be exchanged


System Overview:ArguminSci’s Interfaces

21


System Overview:ArguminSci’s Interfaces

● Command Line Interface● RESTful Application Programming Interface● Web Application

22






ArguminSci

1. Motivation

2. System Overview

3. Conclusion

27

Anne Lauscher, ArguminSci 28

The rhetorical aspects of scientific writing should be studied holistically in order to understand a publication, i.e. a scientific argument, as a whole

ArguminSci illustrates this idea by providing multiple rhetorical analysis perspectives




FW: Expose training phase, extend with other annotation layers and schemes




FW: Expose training phase, extend with other annotation layers and schemes

Thank you

https://github.com/anlausch/ArguminSci

http://data.dws.informatik.uni-mannheim.de/arguminsci/


References

T. J. Bench-Capon, “Specification and implementation of Toulmin dialogue game,” in Proceedings of JURIX, 1998, vol. 98, pp.

5–20.

A. Cohan and N. Goharian, „Scientific article summarization using citation-context and article's discourse structure“. arXiv

preprint arXiv:1704.06619, 2017.

P.H. Dung, "On the acceptability of arguments and its fundamental role in nonmonotonic reasoning, logic programming and

n-person games." Artificial intelligence vol. 77, no. 2, pp. 321-357, 1995.

S. Eger, J. Daxenberger, and I. Gurevych, “Neural End-to-End Learning for Computational Argumentation Mining,” in

Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vancouver,

Canada, 2017, pp. 11–22.

B. Fisas, H. Saggion, and F. Ronzano, “On the Discoursive Structure of Computer Graphics Research Papers.,” in LAW@

NAACL-HLT, 2015, pp. 42–51.

31


References

B. Fisas, F. Ronzano, and H. Saggion, “A Multi-Layered Annotated Corpus of Scientific Papers.,” in LREC, 2016.

G. Nigel Gilbert, “The transformation of research findings into scientific knowledge”, Social Studies of Science, vol. 6, no. 3-4,

pp. 281–306, 1976..

G. Nigel Gilbert, “Referencing as persuasion,” Social Studies of Science, vol. 7, no. 1, pp. 113–122, 1977.

D Herrmannova and P Knoth, “Semantometrics: Towards fulltext-based research evaluation“, in Proceedings of the Joint

Conference on Digital Libraries (JCDL), IEEE/ACM, 2016, pp. 235-236.

R. Jha, A. A. Jbara, V. Qazvinian, and D.R. Radev, “NLP-driven citation analysis for scientometrics.“, in Natural Language

Engineering, 2017, vol. 23., no. 1, pp. 93-130.

32


References

C. Kirschner, J. Eckle-Kohler, and I. Gurevych, “Linking the Thoughts: Analysis of Argumentation Structures in Scientific

Publications,” in Proceedings of the 2nd Workshop on Argumentation Mining held in conjunction with the 2015 Conference of the

North American Chapter of the Association for Computational Linguistics – Human Language Technologies (NAACL HLT 2015), 2015, pp. 1–11.

M. Liakata, S. Saha, S. Dobnik, C. Batchelor, and D. Rebholz-Schuhmann, “Automatic recognition of conceptualization zones

in scientific articles and two life science applications,” Bioinformatics, vol. 28, no. 7, pp. 991–1000, Apr. 2012.

A. Lauscher, G. Glavaš, S. P. Ponzetto, and K. Eckert, “Investigating convolutional networks and domain-specific embeddings for semantic classification of citations,” in Proceedings of WOSP 2017, Toronto, 2017a, vol. tba, p. tba.

A. Lauscher, G. Glavaš, and K. Eckert, “University of Mannheim @ CLSciSumm-17: Citation-Based Summarization of

Scientific Articles Using Semantic Textual Similarity,” in 2nd Joint Workshop on Bibliometric-enhanced Information Retrieval and Natural Language Processing for Digital Libraries 2017 [?], tba, 2017b, p. tba.

33


References

S. Teufel and M. Moens, “Summarizing Scientific Articles: Experiments with Relevance and Rhetorical Status,” Comput. Linguist., vol. 28, no. 4, pp. 409–445, Dec. 2002.

S. Teufel, A. Siddharthan, and C. Batchelor, “Towards discipline-independent argumentative zoning: evidence from chemistry

and computational linguistics,” in Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing:

Volume 3-Volume 3, 2009, pp. 1493–1502.

S. Teufel, J. Carletta, and M. Moens, “An annotation scheme for discourse-level argumentation in research articles,” in

Proceedings of the ninth conference on European chapter of the Association for Computational Linguistics, 1999, pp. 110–117.

K. McKeown et al., “Predicting the impact of scientific concepts using full-text features,” J Assn Inf Sci Tec, vol. 67, no. 11, pp. 2684–2696, Nov. 2016.

S. E. Toulmin, The Uses of Argument. Cambridge University Press, 2003.

34

ArguminSci A Tool for Analyzing Argumentation and ...

Documents