Text Mining and Software Engineering: An Integrated Source ... · artifacts and knowledge resources [29, 30]. From a maintainer’s perspective, exploring [15] and linking these artifacts

Text Mining and Software Engineering: An IntegratedSource Code and Document Analysis Approach∗

Rene Witte and Qiangqiang LiInstitut fur Programmstrukturen und Datenorganisation (IPD)

Fakultat fur InformatikUniversitat Karlsruhe (TH), Germany

Yonggang Zhang and Juergen RillingDepartment of Computer Science and Software Engineering

Concordia University, Montreal, Canada

Abstract

Documents written in natural languages constitute a major part of the artifacts produced duringthe software engineering lifecycle. Especially during software maintenance or reverse engineering,semantic information conveyed in these documents can provide important knowledge for the softwareengineer. In this paper, we present a text mining system capable of populating a software ontologywith information detected in documents. A particular novelty is the integration of results fromautomated source code analysis into a natural language processing (NLP) pipeline, allowing tocross-link software artifacts represented in code and natural language on a semantic level.

1 Introduction

With the ever increasing number of computers and theirsupport for business processes, an estimated 250 billionlines of source code were being maintained in 2000,with that number rapidly increasing [28]. The relativecost of maintaining and managing the evolution of thislarge software base represents now more than 90% ofthe total cost [26] associated with a software product.One of the major challenges for software engineerswhile performing a maintenance task is the need tocomprehend a multitude of often disconnected artifactscreated originally as part of the software developmentprocess [12]. These artifacts include, among others,source code and corresponding software documents,e.g., requirements specifications, design descriptions,and user’s guides. From a maintainer’s perspective, itbecomes essential to establish and maintain the seman-

∗This paper is a postprint of a paper submitted to and accepted forpublication in the IET Software Journal, Vol. 2, No. 1, 2008,and is subject to IET copyright [http://www.iet.org]. The copyof record is available at http://link.aip.org/link/?SEN/2/3/1.

tic connections among all these artifacts. Automatedsource code analysis, implemented in integrated devel-opment environments like Eclipse, has improved soft-ware maintenance significantly. However, integratingthe often large amount of corresponding documenta-tion requires new approaches to the analysis of naturallanguage documents that go beyond simple full-textsearch or information retrieval (IR) techniques [1].

In this paper, we propose a Text Mining (TM) ap-proach to analyse software documents at a semanticlevel. A particular feature of our system is its use offormal ontologies (in OWL-DL format) during boththe analysis process and as an export format for theresults. In combination with a source code analysissystem for populating code-specific parts of the ontol-ogy, we can now represent knowledge concerning bothcode and documents in a single, unified representation.This common, formal representation supports furtheranalysis of the knowledge base, like the automatic es-tablishment of traceability links. A general overview ofthe proposed process is shown in Figure 1: An existingontology of the software domain, including concepts of

1

R. Witte, Q. Li, Y. Zhang, J. Rilling

Figure 1: Ontological Text Mining of software documents for software engineering

programming languages and software documentation,is automatically populated through the analysis of ex-isting source code and corresponding documentation.The knowledge contained in this populated ontologycan then be used by a software maintainer through “Se-mantic Web”-enabled clients utilizing ontology queriesand reasoning.

Our research presents a novel strategy to employknowledge obtained from source code analysis for theanalysis of corresponding natural language documents.The approach of combining both now allows softwareengineers to derive actionable knowledge from the di-verse knowledge resources through ontology queriesand automated reasoning, extending the applications ofour research far beyond simple information extractionor retrieval.

This paper is structured as follows: In the next sec-tion, we motivate the need for integrating text miningand software engineering and describe our solution,an ontology-based program comprehension environ-ment. Section 3 describes our text mining system forthe analysis of software documents. Results from ourevaluation are presented in Section 4. We then showhow to apply our system for the analysis of a largeopen source system, uDig, in Section 5. Related workis discussed in Section 6, followed by conclusions inSection 7.

2 Ontology-Based ProgramComprehension Environment

In this section, we first present a brief motivation andoverview of our ontology-based software environment

and then discuss our main resource, an OWL-DL ontol-ogy, in detail.

2.1 Software Engineering and NaturalLanguage Processing (NLP)

As software ages, the task of maintaining it becomesmore complex and more expensive. Software mainte-nance, often also referred to as software evolution, con-stitutes a majority of the total cost occurring during thelife span of a software system [26, 28]. Software main-tenance is a difficult task complicated by several factorsthat create an ongoing challenge for both the researchcommunity and tool developers [10, 24]. These mainte-nance challenges are caused by the different represen-tations and interrelationships that exist among softwareartifacts and knowledge resources [29, 30]. From amaintainer’s perspective, exploring [15] and linkingthese artifacts and knowledge resources becomes a keychallenge [1]. What is needed is a unified represen-tation that allows a maintainer to explore, query, andreason about these artifacts, while performing theirmaintenance tasks [21].

Information contained in software documents is im-portant for a multitude of software engineering tasks(e.g., requirements engineering, software maintenance),but within this paper, we focus on a particular use case:the concept location and traceability across differentsoftware artifacts. From a maintainer’s perspective,software documentation contains valuable informationof both functional and non-functional requirements, aswell as information related to the application domain.This knowledge often is difficult or impossible to ex-tract only from source code [16]. It is a well-known

2

Text Mining and Software Engineering

fact that even in organizations and projects with ma-ture software development processes, software artifactscreated as part of these processes end up to be discon-nected from each other [1]. As a result, maintainershave to spend a large amount of time on synthesizingand integrating information from various informationsources in order to re-establish the traceability linksamong these artifacts.

Our approach is based on a common formal represen-tation of both source code and software documentationbased on an ontology in OWL-DL format [13]. In-stances are populated automatically through automaticsource code analysis (described in [34]) and text mining(described in this paper). The resulting, populated on-tology can serve as a flexible knowledge base for furtheranalysis. Employing queries, based on query languageslike nRQL or SPARQL, together with automated rea-soning provides a flexibility of code-document analysistasks that goes far beyond current approaches workingon source code or documentation in isolation.

2.2 System Architecture and ImplementationOverview

In order to utilize the structural and semantic informa-tion in various software artifacts, we have developed anontology-based program comprehension environment,which can automatically extract concept instances (e.g.,classes, methods, variables) and their relations fromsource code and documents (Figure 2).

2.2.1 Software Ontology

An important part of our architecture is a software on-tology that captures major concepts and relations inthe software maintenance domain (right side in Fig-ure 2). Ontologies are a commonly used technique forknowledge representation; here, we rely on the morerecently introduced Web Ontology Language (OWL),which has been standardized by the World Wide WebConsortium (W3C).1 In particular, we make use of thesub-format OWL-DL, which is based on DescriptionLogics (DL) as the underlying formal representation.This format provides maximal expressiveness whilestill being complete and decidable, which is importantas our environment relies on an automated reasonerfor several knowledge manipulation tasks. For moredetails on DL, we refer the reader to [4].

Our software ontology consists of two sub-ontologies: a source code and document ontology,

1OWL Web Ontology Language Guide, http://www.w3.org/TR/owl-guide/

which represent information extracted from source codeand documents, respectively.

2.2.2 Semantic Web Infrastructure

As a standardized Web format, OWL ontologies aresupported by a large number of (open source) tools andlibraries, which allows us to create complex knowledge-based systems based on existing Semantic Web [3]infrastructure.

The software ontology was created using the OWLextension of Protege,2 a free ontology editor. Racer [9],an ontology inference engine, is integrated to providereasoning services. Racer is a highly optimized DLsystem that supports reasoning about instances, whichis particularly useful for the software maintenance do-main, where a large amount of instances needs to behandled efficiently. Other services provided by Racerinclude terminology inferences (e.g., concept consis-tency, subsumption, classification, and ontology consis-tency) and instance reasoning (e.g., instance checking,instance retrieval).

2.2.3 Ontology Population Subsystems

One of the major challenges in software analysis is thelarge amount of information that has to be explored andanalyzed as part of typical maintenance activities. Man-ually adding information about instances (e.g., concretevariables or methods in source code, sentences and con-cepts in documentation) is not a feasible solution. Thus,a prerequisite for our ontology-based approach is thedevelopment of automatic ontology population systems,which create instances for the concepts modeled in theontology based on an analysis of existing source codeand its documentation.

The automatic ontology population (Figure 2, mid-dle) is handled by two subsystems: The source codeanalysis system and the text mining system. The sourcecode ontology population subsystem is based on JDT,3

which is a Java parser provided by Eclipse. JDT readsthe source code and performs common tokenizationand syntax analysis to produce an Abstract Syntax Tree(AST). Our population subsystem traverses the ASTcreated by the JDT compiler to identify concept in-stances and their relations, which are then passed to anOWL generator for ontology population. More detailson this subsystem are available in [34]; the text miningsubsystem is described in Section 3 below.

2Protege ontology editor, http://protege.stanford.edu/3Eclipse Java Development Tools (JDT), http://www.eclipse.org/

jdt/

3

R. Witte, Q. Li, Y. Zhang, J. Rilling

Racer

Semantic Web Infrastructure

RDF/OWL APIsProtege

Eclipse IDE

Query InterfacenRQL/JavaScript

OntologyManagement

SOUND Plug-in Ontology BrowserDocument Navigator

SoftwareArtifact

Source Code

Document

…

Source Code AnalysisSystem

Text Mining System

Ontology Population

SoftwareOntology

Source Code

Ontology

DocumentOntology

Figure 2: Ontology-based program comprehension environment overview

2.2.4 Client Integration and Querying

End users, in this case software engineers, need accessto the knowledge contained in the populated ontology.We believe this access should be delivered within thetools commonly used to develop software, providingcontext-sensitive guidance directly relevant to the taskat hand. Towards this goal, we developed a query inter-face for our system in form of a plug-in that providesOWL integration for Eclipse, a widely used software de-velopment platform. Through this query interface, theexpressive query language nRQL provided by Racercan be used to query and reason over the populatedontology. Additionally, we integrated a scripting lan-guage, which provides a set of built-in functions andclasses using the JavaScript interpreter Rhino.4 Thislanguage simplifies querying the ontology for softwareengineers not familiar with DL-based formalisms. Weshow examples on ontology queries in Section 5 below.

2.3 Software Document Ontology

The documentation ontology (see Figure 3 for an ex-cerpt) consists of a large body of concepts that are ex-pected to be discovered in software documents. Theseconcepts are based on various programming domains,including programming languages, algorithms, datastructures, and design decisions such as design patternsand software architectures. Additionally, the softwaredocumentation sub-ontology has been specifically de-signed for automatic population through a text miningsystem by adapting the ontology design requirements

4Rhino JavaScript interpreter, http://www.mozilla.org/rhino/

discussed in [33] for the software engineering domain.In particular, we included the following parts:

Text Model: represents the structure of documents, i.e.,it contains concepts for sentences, paragraphs, andtext positions, as well as NLP-related conceptsthat are discovered during the analysis process,like noun phrases (NPs) and coreference chains.These are required for anchoring detected entities(populated instances) in their originating docu-ments. Noun phrases form the grammatical basisfor finding named entities, which add the seman-tics to the detected information by labeling NPswith ontological concepts, like “class,” “method,”or “variable.”

Coreference chains connect all entities in a docu-ment that are semantically equivalent. For exam-ple, an entity, like a single method “suite()” canappear multiple times within a document, withdifferent textual representations (e.g, “the suite()method,” “this method,” “it”). A coreferencechain connects these lexically different, but se-mantically equivalent occurrences.

Lexical Information: facilitates the detection of enti-ties in documents. Examples are names of com-mon design patterns (“Bridge,” “Adapter,” etc.),programming language-specific keywords (“int,”“extends,” etc.), and architectural styles (“layeredarchitecture,” “client/server,” etc.).

Lexical Normalization Rules: these rules transformentity names as they appear in a document to acanonical name that can be used for ontology pop-

4

Text Mining and Software Engineering

Figure 3: Excerpt of the software documentation sub-ontology used for software document analysis and NLPresult export

ulation. For example, the same method name canbe referenced by a multitude of textual representa-tions, like “the suite() method” or “method suite().”Finding the canonical name “suite()” from thesedifferent representations is important for furtherautomated analyses, e.g., document/source codetraceability. This is achieved through normaliza-tion rules, which are specific for a certain ontologyclass.

Relations: between the classes, including the onesmodeled in the source code ontology. Ontologyrelations allow us to automatically restrict NLP-detected relations to semantically valid ones. Forexample, a relation like <variable> imple-ments <interface>, which can result fromparsing a grammatically ambiguous sentence, canbe filtered out since it is not supported by the on-tology.

Source Code Entities: that have been automaticallypopulated through source code analysis can also

be utilized for detecting corresponding entities indocuments, as we describe below.

How these different pieces of knowledge contribute toa detailed semantic analysis of software documents isdescribed in the next section.

3 Ontology Population throughText Mining

We developed our text mining system for populatingthe software documentation ontology based on theGATE5 (General Architecture for Text Engineering)framework [7]. Our system is component-based, utiliz-ing both standard tools shipped with GATE and customcomponents developed specifically for software textmining. An overview of the workflow is shown inFigure 4.

5GATE, http://gate.ac.uk/

5

R. Witte, Q. Li, Y. Zhang, J. Rilling

considering ontology relations and properties

populated subset of,

specific NLP resultsas well as document−

Gazetteer: assign ontology classes

OWL Ontology Export

Grammar: Named Entity recognition

NLP preprocessing: Tokenisation, Noun Phrase detection etc.

Coreference Resolution: determine identical individuals

Normalization: get representational individuals in canonical form

Relation detection: establish relations with syntactical rules

assign ontology classes to document entities

consider ontological hierarchies in grammar rules

look up synonym relations to find synonyms

look up ontology properties with rules for establishing the canonical form

Populated Ontology for Processed Documents

initial population

Morphological analysis, Deep Syntactic Analysis: SUPPLE

Instantiated Source Code Ontology

Complete Instantiated Software Ontology

Figure 4: Workflow of the software text mining subsystem

3.1 Preprocessing

The processing pipeline starts with a number of stan-dard preprocessing steps (Figure 4, top left). Thesegenerate annotations and data structures required forthe more involved natural language analysis tasks per-formed later on.

The first step is Tokenization. Similarly to program-ming languages, tokenization splits the input text intoindividual tokens, separated by whitespace or specialcharacters. Sentence Splitting then detects sentenceboundaries between tokens using a set of rules, e.g., toavoid false splits at abbreviations or other occurrencesof a full stop not indicating an end-of-sentence. Part-of-Speech (POS) Tagging works on a sentence basisand adds a POS tag to each token, identifying its class(e.g., noun, verb, adjective). Here, we use the Heppletagger included in the GATE distribution. Based onthe POS tags, Chunking modules analyze the text forNoun Phrase (NP) and Verb Group (VG) chunks. Fordetecting NPs, we use the open source MuNPEx chun-ker6 and for detecting VGs, we rely on the verb groupermodule that comes with GATE.

For more details on these steps, we refer the reader

6Multi-lingual Noun Phrase Extractor (MuNPEx), http://www.ipd.uni-karlsruhe.de/∼durm/tm/munpex/

to the GATE user’s guide.7

3.2 Ontology Initialization

While analysing documents specific to a source codebase, our text mining system can take instances detectedby the automatic code analysis into account. This isachieved in two steps: first, the source code ontology ispopulated with information detected through static anddynamic code analysis [34]. This steps adds instanceslike method names, class names, or detected designpatterns to the software ontology. In a second step,we use this information as additional input to the On-toGazetteer component for named entity recognition.

3.3 Named Entity Detection

The basic process in GATE for recognizing entities of aparticular domain starts with the gazetteer component.It matches given lists of terms against the tokens of ananalysed text and, in case of a match, adds an anno-tation named Lookup whose features depend on thelist where the match was found. Its ontology-awarecounterpart is the OntoGazetteer, which incorporatesmappings between its term lists and ontology classesand assigns the proper class in case of a term match. For

7GATE user’s guide, http://gate.ac.uk/sale/tao/index.html

6

Text Mining and Software Engineering

NP

DET MOD HEAD

the getNumber method

Ontology class "Method"

Method instance "getNumber"

Figure 5: Combining ontology and noun phrase chunksfor named entity detection

example, using the instantiated software ontology, thegazetteer will annotate the text segment method witha Lookup annotation that has its class feature setto “Method.” Here, incorporating the results from au-tomatic code analysis can significantly boost recall (cf.Section 4), since entity names in the software domaintypically do not follow naming rules.8

In a second step, grammar rules written in the JAPE9

language are used to detect and annotate complexnamed entities. Those rules can refer to the Lookupannotation generated by the OntoGazetteer, and alsoevaluate the ontology directly. For example, when per-forming a comparison like class=="Keyword" ina grammar rule, the complete concept hierarchy in theontology is considered in this comparison and as a re-sult also includes a match for a Java keyword, since aJava keyword is a subclass of the concept “Keyword”in the ontology. This feature significantly reduces theoverhead for grammar development and testing.

The developed JAPE rules combine ontology-basedlookup information with noun phrase (NP) chunks todetect semantic units. NP chunking is performed us-ing the MuNPEx chunker,10 which relies mostly onpart-of-speech (POS) tags, but can also take the lookupinformation into account. This way, it can preventbad NP chunks caused by mis-tagged software enti-ties (e.g., method names or program keywords taggedas verbs). Essentially, we combine two complemen-

8In many other domains, like in biology, an entity’s type can be(partially) derived from its lexical form, e.g., every noun endingwith the letters -ase must be an enzyme. But no such rules existin the software domain for, e.g., method or variable names.

9JAPE is a regular-expression based language for writing gram-mars over annotation graphs, from which finite-state transduc-ers are generated by a GATE component.

10Multi-Lingual Noun Phrase Extractor (MuNPEx), http://www.ipd.uka.de/∼durm/tm/munpex/

tary approaches for entity detection: A keyword-basedapproach, relying on lexical information stored in thedocumentation ontology (see above). For example, thetext segment “the getNumber() method. . .” will be an-notated with a lookup information indicating that theword method belongs to the ontology class “Method.”Likewise, the same segment will be annotated as a sin-gle noun phrase, showing determiner (“the”), modifier(“getNumber()”), and head noun (“method”). Usingan ontology-based grammar rule implemented in JAPE,we can now combine these two sources of informationand semantically mark the NP as a method (Figure 5).Similar rules are used to detect variables, class names,design patterns, or architectural descriptions. Note thatthis approach does not need to know about “getNum-ber()” being a method name; this fact is derived from acombination of grammatical (NP chunks) and lexical(ontology) information.

The second approach relies on source code analysisresults stored in the initialized software ontology (seeSection 3.7). Every method, class, package, etc. namewill be automatically represented by an instance in thesource code sub-ontology and can thus be used by theOntoGazetteer for entity detection. This applies also incase when these instances appear outside a grammaticalconstruct recognized by our hand-crafted rules. Thisis especially useful for analysing software documentsin conjunction with their source code, the primary sce-nario our system was designed for.

3.4 Coreference Resolution

We use a fuzzy set theory-based coreference resolutionsystem [31] for grouping detected entities into corefer-ence chains. Each chain represents an equivalence classof textual descriptors occurring within or across docu-ments. Not surprisingly, our fuzzy heuristics developedoriginally for the news domain (e.g., using WordNet)were particularly ineffective for detecting coreferencein the software domain. Hence, we developed an ex-tended set of heuristics dealing with both pronominaland nominal coreferences.

For nominal coreferences (i.e., references betweenfull noun phrases, excluding pronouns), we rely onthree main heuristics. The first is based on simple stringequality (ignoring case). The second heuristic estab-lishes coreference between two entities if they becomeidentical when their NPs’ HEAD and MOD slots areinverted, as in “the selectState() method” and “methodselectState()”. The third heuristic deals with a numberof grammatical constructs often used in software docu-ments that indicate synonymous entities. For example,

7

R. Witte, Q. Li, Y. Zhang, J. Rilling

Table 1: Lexical normalization rules for various ontology classes

Ontology Class H DH MH(cM) MH(cH) DMH(cM) DMH(cH)

Class H H H lastM H lastMMethod H H H lastM H lastMPackage H H H lastM H lastMOO Object H H H lastM H lastMLayeredArchitecture H H MH MH MH MHLayer H H MH MH MH MHAbstractFactory H H MH MH MH MHOO Interface H H H lastM H lastM

in the text fragment “. . . we have an action class calledViewContentAction, which is invoked.” we can identifythe NPs “an action class” and “ViewContentAction” asbeing part of the same coreference chain. This heuris-tic only considers entities of the same ontology class,connected by a number of pre-defined relation words(e.g., “named”, “called”), which are also stored in theontology.

For pronominal resolution (i.e., references includinga pronoun), we implemented a number of simple sub-heuristics dealing only with 3rd person singular andplural pronouns: it, they, this, them, and that. The lastthree can also appear in qualified form (this method,that constructor). We employ a simple resolution al-gorithm, searching for the closest anaphorical referentthat matches the case and, if applicable, the semanticclass.

3.5 Normalization

Normalization needs to decide on a canonical namefor each entity, like a class or method name. This isimportant for ontology population, as an instance, likeof the ontology class “Method,” should reflect onlythe method name, omitting any additional grammaticalconstructs like determiners or possessives. Thus, anamed entity like “the static TestCase() class” has tobe normalized to “TestCase” before it can become aninstance (ABox) of the concept Class (TBox) in thepopulated ontology.

This step is performed through a set of lexicalnormalization rules, which are stored with their cor-responding classes in the software document sub-ontology, allowing us to inherit rules through subsump-tion. Table 1 shows a number of these rules for var-ious ontology classes: D, M, H refer to determiner,modifier, and head, respectively, and c(x) denotes theontology class of a particular slot; the table entry deter-mines what part of a noun phrase is selected as thenormalized form, which is then stored as a feature

instanceName in the entity’s annotation, as shownin Figure 6.

3.6 Relation Detection

The next major step is the detection of relations be-tween entities, e.g., to find out which interface a class isimplementing, or which method belongs to which class.Relation detection in our system is again achieved bycombining two complementary approaches, as shownin Figure 7: a set of hand-crafted grammar rules im-plemented in JAPE and a deep syntactic analysis usingthe SUPPLE parser. Afterwards, detected relations arefiltered through the software ontology to erase seman-tically invalid results. We now describe these steps indetail.

3.6.1 Rule-Based Relation Detection

Similarly to entity recognition, rule-based relation de-tection is performed in a two-step process, starting withthe verb groups (VGs) detected in the preprocessingstep discussed above. In addition, our ontology con-tains lexical information for the modeled relations (cf.Section 2.3), e.g., “implements” for the class-interfacerelation or “provides” for the class-method relation.This allows to detect candidate relation words in a textusing the OntoGazetteer component. The intersectionof both—VGs containing a detected relation word—arecandidates for further processing. For example, given asentence fragment like “the Test class provides a staticsuite() method,” the token “provides” would be markedas both a VG (containing only a single, active verb) anda relation candidate (for the class-method relation).

In a second step, hand-crafted grammatical rulesfor relation detection are run over the relation candi-dates to find, e.g., relations between classes and de-sign patterns or classes and methods. These rulesare also implemented in JAPE and make use ofthe subsumption hierarchy in the ontology. For ex-

8

Text Mining and Software Engineering

Figure 6: Java class entity detected through NLP displayed in GATE showing the normalized instanceName forontology export

ample, the above sentence would be matched bythe simple rule <Software-Entity> relation<Software-Entity>. Using the voice infor-mation (active/passive) provided by the VG chun-ker, we can then assign subject/object slots for theentities participating in a relation. The final re-sult of this step for the above example is the re-lation provides(Class:Test, Method:staticsuite()).

Figure 7: Finding relations between software entities

3.6.2 Deep Syntactic Analysis

For a deep syntactic analysis, we currently employthe SUPPLE parser [8], which is integrated intoGATE through a wrapper component. SUPPLE isa general-purpose bottom-up chart parser for feature-based context-free phrase structure grammars, imple-mented in Prolog. It produces syntactic as well as se-mantic annotations for a given sentence. Grammars areapplied sequentially. After each layer of grammaticalanalysis, only the best parse is selected for the next step.Thus, SUPPLE avoids the multiplication of ambiguitiesthroughout the grammatical and semantical analysis,which comes with the trade-off of losing alternativesthat could be resolved in later analysis steps. The identi-fication of verbal arguments and attachment of nominaland verbal post-modifiers, such as prepositional phrasesand relative clauses, is done conservatively. Instead ofproducing all possible analyses or using probabilities togenerate the most likely analysis, SUPPLE only offersa single analysis that spans the input sentence only ifit can be relied on to be correct, so that in many casesonly partial analyses are produced. This strategy gen-erally has the advantage of higher precision, but at theexpense of lower recall. SUPPLE outputs a logicalform, which is then matched with the entities detected

9

R. Witte, Q. Li, Y. Zhang, J. Rilling

Figure 8: Example for a syntax tree generated by the SUPPLE parser

previously to obtain predicate-argument structures.An example for a parse tree generated by SUPPLE

can be seen in Figure 8. Here, the sentence “Test-Suite now provides a constructor TestSuite (Class the-Class).” was analysed. Based on this analysis, wedetect a relation word “provides” (modeled in the ontol-ogy, cf. Section 3.6.1) and can thus extract a predicate-argument structure by identifying the correspondingsubject “TestSuite” and object “a constructor. . .” (ac-tive/passive voice is taken into account using the verbgroup (VG) analysis step described above).

3.6.3 Result Integration

The results from both rule- and parser-based relationdetection form the candidate set for the ontology re-lation instances created for a given text (cf. Figure 7).As both approaches may result in false positives, e.g.,through ambiguous syntactical structures or rule mis-matches, we prune the set by checking each candidaterelation for semantic correctness using our softwareontology. As each entity participating in a relation hasa corresponding ontology class (see Figure 9), we canquery the ontology to check whether the detected re-

lation (or one of its supertypes) exists between theseclasses. This way, we can filter out relations like a vari-able “implementing” an interface or a design patternbeing “part-of” a class, thereby significantly improvingprecision (cf. Section 4).

Note that relation detection and filtering is one par-ticular example where an ontology delivers additionalbenefit when compared with classical NLP techniques,like plain gazetteering lists or statistical/rule-based sys-tems [33].

Figure 9: Ontology excerpt: Relations between the con-cepts “Class” and “Method”

10

Text Mining and Software Engineering

Figure 10: Text Mining results by ontology population: a detected Java Constructor instance browsed in SWOOPshowing information about coreferences and corresponding sentences

3.7 Ontology Export

Finally, the instances found in the document and therelations between them are exported to an OWL-DLontology. Note that entities provided by applying thesource code analysis step are only exported in the docu-ment ontology if they have also been detected in a text(cf. Figure 4).

In our implementation, ontology population is doneby a custom GATE component, the OwlExporter,which is application domain-independent. It col-lects two special annotations, OwlExportClass andOwlExportRelation, which specify instances ofclasses and relations (i.e., object properties), respec-tively. These must in turn be created by application-specific components, since the decisions as to whichannotations have to be exported, and what their OWLproperty values are, depend on the domain.

The class annotation carries the name of the class,a name for the instance (the normalized name createdpreviously), and the GATE internal ID of an annotationrepresenting the instance in the document. If there areseveral occurrences of the same entity in the document,the final representation annotation is chosen from theones in the coreference chain by the component creat-ing the OwlExportClass annotation. In case of the

software text mining system, a single representative hasto be chosen from each coreference chain. Rememberthat one chain corresponds to a single semantic unit,so the final, exported ontology must only contain oneentry for, e.g., a method, not one instance for everyoccurrence of that method in a document set. We selectthe representative using a number of heuristics, basi-cally assuming that the longest NP that has more slots(DET, MOD, HEAD) filled is also the most salient one.

From this representative annotation, all further in-formation is gathered. After reading the class name,the OwlExporter queries the ontology via the Jena11

framework for the class properties and then searches forequally named features in the representation annotation,using their values to set the OWL properties.

An example for an exported instance (an OWL in-dividual) can be seen in Figure 10. Note the detailedsemantic information obtained about the textual refer-ence “TestSuite.” Besides the semantic type (ontologyclass Constructor), the text mining system recorded thedifferent occurrences of this entity throughout a docu-ment. In particular, the last sentence “This constructoradds all the methods. . .” contains knowledge importantfor a software maintainer that would not have been de-

11Jena Semantic Web Framework for Java, http://jena.sf.net/

11

R. Witte, Q. Li, Y. Zhang, J. Rilling

tected by simple full-text search or IR methods, as itdoes not contain the string “TestSuite” itself. This il-lustrates the importance of advanced NLP for softwaredocuments, in this case, coreference resolution.

4 Evaluation

In this section, we present an evaluation of our ap-proach. We first introduce our analysis strategy andmetrics for readers that are unfamiliar with the evalua-tion of text mining systems in Sections 4.1 and 4.2. Thedifferent parts of the system—for entity recognition,entity normalization, and relation detection—are thenevaluated in detail in Sections 4.3–4.5.

4.1 Evaluation Strategy and Corpus

Generally, the effectiveness of a text mining system ismeasured by running it against a corpus (set of docu-ments), where the expected results are known. This al-lows to compare the output of a system (the reply) withthe correct, expected results (the model). Of course,since no system is available to produce 100% correctresults, the expected reply has to be provided by manu-ally annotating the documents from the corpus. Thisso-called gold standard then serves as the reference forautomated evaluation metrics, allowing a comparisonof different approaches on the same data and ensuringrepeatability. The downside is that manual annotationis time-consuming and thus expensive, and can alsosuffer from errors introduced by annotators.

So far, we evaluated our text mining subsystem ontwo collections of texts: a set of 5 documents (7743words) from the Java 1.5 documentation for the Col-lections framework12 and a set of 7 documents (3656words) from the documentation of the uDig13 geo-graphic information system (GIS). The document setswere chosen because of the availability of the corre-sponding source code. Both sets were manually an-notated for named entities, including their ontologyclasses and normalized form, as well as relations be-tween the entities.

4.2 Evaluation Metrics

The metrics precision, recall and F-measure are com-monly used in the evaluation of NLP systems. Theyhave been adapted from their Information Retrieval(IR) [5] counterparts. Precision and recall are based on

12Java Collections Framework Documentation, http://java.sun.com/j2se/1.5.0/docs/guide/collections/index.html

13uDig GIS Documentation, http://udig.refractions.net/

the notion of entities that have been correctly found bya system (correct), entities that are missing, and entitieswrongly detected by a system, i.e., spurious entities orfalse positives. To capture partially correct results, i.e.,entities where gold standard and system response over-lap without being coextensive (matching 100%), a thirdcategory partial is introduced in addition to correct andspurious results.

Following the definition implemented by the GATEannotation evaluation tool,14 we can define precision asthe ratio of correctly detected entities over all retrievedentities:

Precision =Correct+ 1

2 Partial

Correct+Spurious+ 12 Partial

(1)

Sometimes the error rate is used instead of precision,which is simply defined as 1/Precision. Similarly, re-call is the ratio of correctly detected entities over allcorrect entities:

Recall =Correct+ 1

2 Partial

Correct + Missing+ 12 Partial

(2)

The F-measure combines both precision and recall us-ing their harmonic mean:

F-Measure =(β 2 +1)Precision ·Recall(β 2Recall)+Precision

(3)

Here, β is an adjustable weight to favour precision overrecall (with β = 1, both are weighted equally).

For more details on the evaluation of text mining andNLP, we refer to the evaluation section in the GATEdocumentation, “Performance Evaluation of LanguageAnalysers,” as well as Section 8.1 in [17].

4.3 Named Entity Recognition Evaluation

For NE detection, we computed the precision, recall,and F-measure results using the output of our systemwith the gold standard. A named entity was onlycounted as correct if it matched both the textual de-scription and ontology class. Table 2 shows the resultsfor two experiments: first running only the text miningsystem over the corpora (left side) and second, perform-ing the same evaluation after running the code analysis,using the populated source code ontology as an addi-tional resource for NE detection as described above.As can be seen, the text mining system achieves a veryhigh precision (90%) in the NE detection task, with arecall of 62%. With the imported source code instances,

14GATE documentation, http://gate.ac.uk/documentation.html

12

Text Mining and Software Engineering

Table 2: Evaluation results: Entity recognition and normalization performance

Text Mining Only With Source Code OntologyCorpus P R F A P R F A

Java Collections 0.89 0.67 0.69 75% 0.76 0.87 0.79 88%uDig 0.91 0.57 0.59 82% 0.58 0.87 0.60 84%Total 0.90 0.62 0.64 77% 0.67 0.87 0.70 87%

Table 3: Evaluation results: Relation detection performance

Before Filtering After FilteringCorpus P R F P R F ∆P

Text Mining OnlyJava Collections 0.35 0.24 0.29 0.50 0.24 0.32 30%uDig 0.46 0.34 0.39 0.55 0.34 0.42 16%Total 0.41 0.29 0.34 0.53 0.29 0.37 23%

With Source Code OntologyJava Collections 0.14 0.36 0.20 0.20 0.36 0.25 30%uDig 0.11 0.41 0.17 0.24 0.41 0.30 54%Total 0.13 0.39 0.19 0.22 0.39 0.23 41%

these numbers become reversed: the system can nowcorrectly detect 87% of all entities, but with a lowerprecision of 67%.

The drop in precision after code analysis is mainlydue to two reasons. Since names in the software do-main do not have to follow any naming conventions,simple nouns or verbs often used in a text will be mis-tagged after being identified as an entity appearing in asource code. For example, the Java method sort fromthe collections interface will cause all instances of theword “sort” in a text to be marked as a method name.Another precision hit is due to the current handling ofclass constructor methods, which are typically identi-cal to the class name. Currently, the system cannotdistinguish the class name from the constructor name,assigning both ontology classes (i.e., Constructor andOO CLass) for a text segment, where one will alwaysbe counted as a false positive.

Both cases require additional disambiguation strate-gies when importing entities from source code analysis,which are currently under development. However, thecurrent results already underline the feasibility of ourapproach of integrating code analysis and NLP.

4.4 Entity Normalization Evaluation

We also evaluated the performance of our lexical nor-malization rules for entity normalization, since cor-rectly normalized names are a prerequisite for the cor-rect population of the result ontology. For each entity,

we manually annotated the normalized form and com-puted the accuracy A as the percentage of correctlynormalized entities over all correctly identified entities.Table 2 shows the results for both the system running intext mining mode alone and with additional source codeanalysis. As can be seen from the table (columns A),the normalization component performs rather well with77% and 87% accuracy for the analysis with/withoutsource code ontology import, respectively.

4.5 Relation Detection Evaluation

Not surprisingly, relation detection was the hardest sub-task within the system. Like for entity detection, weperformed two different experiments, with and withoutsource code analysis results. Additionally, we evaluatethe influence of the semantic relation filtering step us-ing our ontology as described above. The results aresummarized in Table 3. As can be seen, the currentcombination of rules with the SUPPLE parser achievesonly average performance. However, the increase inprecision (∆P) when applying the filtering step usingour ontology is significant: upto 54% better than with-out semantic filtering.

The errors leading to missing and spurious relationsare mainly due to the unchanged SUPPLE parser rules,which have not yet been adapted to the software domain.Also, the conservative approach of the SUPPLE parserwhen linking grammatical constituents, especially theattachment of prepositional phrases, leads to further

13

R. Witte, Q. Li, Y. Zhang, J. Rilling

Figure 11: Linking the automatically populated source code and software documentation ontologies

missing predicate-argument structures. We are cur-rently experimenting with different parsers (RASP andMiniPar) and are also adapting the SUPPLE grammarrules in order to improve the detection of predicate-argument structures.

5 Applications

In this section, we show how a software engineer canapply the developed system for program comprehen-sion tasks [34]. In particular, we focus on a specific usecase, the recovery of traceability links [23].

5.1 Linking Software and DocumentationOntology

Having both source code and documents representedin the form of an ontology allows us to link instancesfrom source code and documentation using existingapproaches from the field of ontology alignment [22].Ontology alignment techniques try to match ontolog-ical information from different sources on conceptualor/and instance levels. Since our documentation on-tology and source code ontology share many conceptsfrom the programming language domain, such as Classor Method, the problem of conceptual alignment hasbeen minimized. This research therefore focuses moreon matching instances that have been discovered bothfrom source analysis and text mining (Figure 11).

As described above, our text mining system can ad-ditionally take the results of the source code analysisas input when detecting named entities. This allowsus directly connect instances from the source code anddocument ontologies. For example, our source codeanalysis tool may identify c1 and c2 as classes, andthis information is used by the text mining system toidentify the named entities c′1,c

′2 and their associated

information in the documents. As a result, source codeentities c1 and c2 are now linked to their occurrences inthe documents (c′1 and c′2), as well as other informationabout the two entities mentioned in the document, suchas design patterns, architectures, etc.

5.2 Querying the Combined Ontology

After source code and documentation ontology arelinked, users can perform ontological queries on bothdocuments and source code regarding properties ofthe classes c1 and c2. For example, a user can exe-cute a query to retrieve document passages that de-scribe both c1 and c2 or design pattern descriptionsreferring to the class that contains the class currentlyanalyzed. Note that the alignment process might alsoidentify inconsistencies—the documentation might lista method for a different class, for example—which aredetected through the alignment process and registeredfor further review by the user. In addition, users canalways manually define new concepts/instances andrelations in both ontologies to establish the links that

14

Text Mining and Software Engineering

Figure 12: Manually adding relations to the result on-tologies

cannot be detected by the automated alignment. Forexample, as Figure 12 shows, the text mining systemmay detect an instance of DesignPattern, d p1. A usercan now manually create the relations between the pat-tern and classes that contribute to it (e.g., c1, c2, and c3)through our query interface. The newly created linksthen become an integrated part of the ontology, and canbe used to, for example, retrieve all documents relatedto the pattern (i.e., s1, s2, . . ., sn).

Furthermore, documents can not only be linked tosource code, but also to design-level concepts that re-late to particular reverse engineering tasks. For ex-ample, in contrast to the serialized view of softwaredocuments, i.e., sentence by sentence, or paragraph byparagraph, the formal ontological representation of soft-ware documentation also provides the ability to createhierarchical document views (or slices), comparable tocontext-sensitive automatic summarization [32]. Usingthe classification service of the ontology reasoner, onecan classify document pieces that relate to a specificconcept or a set of concepts (Figure 13). For exam-ple, the Visitor Pattern documents can be consideredas all text paragraphs that describe/contain informationrelated to the concept “Visitor pattern.” The newly es-tablished concept VisitorPatternDoc can then be usedto retrieve paragraphs that relate to the visitor pattern:

VisitorPatternDoc≡ Paragraphu∃contains.Visitor

Similarly, a new concept HighLevelDoc can also be de-fined to retrieve all documents that contain high-leveldesign concepts, such as Architecture or DesignPat-tern. The ontology reasoner can automatically classifydocuments according to the concept definition:

HighLevelDoc≡ DocumentFile

u∃contains.(ArchitecturetDesignPattern)

Figure 13: Recombining information from differentdocuments describing a given software en-tity

5.3 Case Study

We have been performing additional experiments onthe large open source Geographic Information System(GIS), uDig. The uDig system is implemented as aset of plug-ins that provides geographic informationmanagement integration on top of the Eclipse platform.Links between the uDig implementation and its docu-mentation (see Section 4) are recovered by first perform-ing source code analysis to populate the source codeontology. The resulting ontology contains instancesof Class, Method, Field, etc., and their relations, suchas inheritance and invocation, which are then used toinitialize the documentation ontology as described inSection 3.2. Through the text mining subsystem, alarge number of Java language concept instances (in-dividuals) are discovered in the documents, as well asdesign-level individuals, such as design patterns or ar-chitectural styles [27]. The ontology alignment rulesare then applied to link both the documentation ontol-ogy and the source code ontology. Part of our initialresult is shown in Figure 14; the corresponding sen-tences are:15

Sentence 2544: “For example if the class Feature-Store is the target class and the object that is clicked onis a IGeoResource that can resolve to a FeatureStorethen a FeatureStore instance is passed to the operation,not the IGeoResource.”

Sentence 712: “Use the visitor pattern to traversethe AST.”

Figure 14 shows that in the uDig documents,our text mining system was able to discoverthat a sentence (sentence 2544) contains boththe class instance 4098 FeatureStore and4100 IGeoResource. Both of these instances

can be linked to the instances in source code ontol-ogy, org.geotools.data.FeatureStore and

15Numbers in instance names are internal IDs generated by theontology population process (see Section 3.7).

15

R. Witte, Q. Li, Y. Zhang, J. Rilling

Figure 14: Automatic recovery of traceability links between source code and documentation

net.refractions.udig.catalog.IGeoResour-ce, respectively. In addition, in another sentence (sen-tence 712), a class instance ( 719 AST) and a designpattern instance ( 718 visitor pattern) are also identi-fied. Similarly, instance 719 AST can then be linked tothe net.refractions.udig.catalog.util.ASTinterface in the source code ontology.

After the source code ontology and documentationontology are linked, queries regarding the source codeentities, design level concepts, and their occurrencesin documents can be performed using the reasoningservices provided by our ontology reasoner, Racer. Forexample, during the comprehension of the class Fea-tureStore, a reverse engineer may want to study theclasses that are related to FeatureStore. Within thesource code ontology, a query can be performed to re-trieve all classes that contain methods that have calledclass FeatureStore. In addition to these types of sourcecode queries, the reverse engineer can perform queriesthat cross the boundaries between source code and doc-umentation. Such type of queries are enabled due tothe already established links between the source codeand documentation ontology.

The linked source code and documentation ontolo-gies also provide us with the capability to combinesemantic information from both software implementa-tion and documentation. For example, our text miningsystem has detected that class AST is potentially a partof a Visitor pattern (Figure 14). In order to retrieveall documented information related to the detected pat-tern, the following query can be used to retrieve all textparagraphs that describe the sub-classes of AST:

1 var query = new Query(); // define a new query2 query.declare( ”P”, ”C” ); // declare two query variables3 query. restrict ( ”P”, ”Paragraph” ); // P is a paragraph4 query. restrict ( ”C”, ”Class” ); // C is a class5 query. restrict ( ”C”, ”hasSuper”, // C is a sub−class of AST6 ”net. refractions .udig.catalog. util .AST” );7 query. restrict ( ”P”, ”contains”, ”C” ); // P contains C8 query.retrieve ( ”P” ); // this query only retrieves P9 var result = ontology.query(query); // perform the query

This query combines two parts of the ontology: first,the programming language semantics, such as the in-heritance relation between query variable C and theclass AST, and second, the structural information ofdocumentation, such as the containing relation betweena paragraph P and the class C. The result of this querytherefore contains all text paragraphs that describe thesub classes of AST, i.e., the Visitor pattern. It has tobe noted that the role contains is a transitive relationto describe the document structure. The ontology rea-soner can automatically resolve the transitivity fromParagraph to Sentence, and from Sentence to Class.

Summary. In this section, we presented an initialevaluation of recovering traceability links betweensource code and documentation on a large open sourcesoftware system. We have demonstrated the use of au-tomated reasoning to retrieve documented informationwith regard to a specific reverse engineering task andinfer implicit relations in the linked ontologies.

6 Related Work and Discussion

In this section, we present work similar to ours, struc-tured into three areas: (i) applications of NLP to soft-ware documents, (ii) other approaches for recovering

16

Text Mining and Software Engineering

traceability links between code and documentation; and(iii) ontological approaches to the software engineeringdomain.

NLP for Software Documentation. Very little previ-ous work exists on text mining software documents.Most of this research has focused on analysing texts atthe specification level, e.g., in order to automaticallyconvert use case descriptions into a formal representa-tion [11, 20] or detect inconsistent requirements [14].In contrast, we aim to support the complete softwaredocumentation life-cycle, from white papers, designand implementation documents to in-line code texts(e.g., JavaDoc). To the best of our knowledge, therehas been so far no attempt to automatically cross-linkentities (e.g., methods, design patterns, architectures)detected by text mining software documents with corre-sponding entities found by source code analysis, whichis an important contribution of our work. Likewise,we are not aware of previous approaches that includeknowledge derived from analysing source code into thenatural language analysis of the corresponding docu-mentation, which we believe has potential far beyondthe first experiments described in this paper.

Code/Document Traceability. There exists some re-search in recovering traceability links between sourcecode and design documents using Information Retrievaltechniques. The IR models used include traditional vec-tor space and probabilistic models [1, 2], as well aslatent semantic indexing (LSI) [18,19]. The approachby [2] applies both a probabilistic and a vector spaceinformation retrieval model in two case studies to traceC++ source code onto manual pages and Java code tofunctional requirements. The approach in [18] uses LSIto extract the meaning (semantics) of the documenta-tion and source code, and then use this information toidentify traceability links based on similarity measures.

In contrast with these IR approaches, our work alsotakes advantage of structural and semantic informa-tion in both the documentation and the source code bymeans of text mining and source code parsing. Theresulting analysis is much more fine-grained than IRapproaches, identifying individual words and phrasesin a document. IR approaches, like the one by Antoniolet al. [2], work by linking a class to a whole manualpage, whereas we can link a class to its precise occur-rence within a page, including those occurrences whereit is only mentioned by a pronoun. The same appliesto the LSI approach by Marcus et al. [19], who gener-ate links for whole documents or document parts (e.g.,

sections), not individual words or phrases like our sys-tem. Likewise, links are established with much largerstructures on the source code side—files or classes, notindividual methods or variables. As the results of thesemethods are unstructured clusters, advanced queriesand reasoning are also not possible.

Software Ontologies Ontologies as a formal knowl-edge representation mechanism have been applied tomany areas in software engineering [6]. Common toall of these approaches is that their main intent is tosupport in one form or another the conceptualization ofknowledge, mainly by standardization of terminology,and to support knowledge sharing based on a commonunderstanding. These approaches fall short on adopt-ing and implementing automated population tools tosupport analysis tasks using the ontology as a knowl-edge base. They also lack the use of reasoning servicesto infer implicit knowledge. Finally, there is somework targeting ontology learning from software docu-mentation. Our work differs in that we construct theontology specifically for text mining, whereas the workby Sabou [25] applies text mining for ontology learning.We believe that current methods for automated ontol-ogy construction do not provide the necessary level ofdetail as discussed in Section 2.3. However, a futurecombination of both approaches might be a promisingtarget of research.

7 Conclusions and Future Work

We presented a text mining system for the software do-main that is capable of extracting entities from softwaredocuments. The system’s output is a populated OWL-DL ontology containing normalized instances and theirrelations. The system is novel in two important aspects:First, it employs a formal ontology, based on descrip-tion logics, both as a processing resource for the variousNLP components and the result export format. Second,as the system is part of a larger ontology-based programcomprehension environment, it can incorporate resultsfrom automated source code analysis subsystems in itsNLP processing pipeline.

The ontological foundation allows for important im-provements in software engineering, as it supportsqueries and reasoning services on semantic knowledgeautomatically derived from large amounts of documen-tation in natural language form. We previously showedhow automated reasoning can support a software main-tainer when performing knowledge-intensive tasks, likearchitectural recovery or source code security analy-

17

R. Witte, Q. Li, Y. Zhang, J. Rilling

sis [34]. We are also currently experimenting withontology alignment strategies to automatically detectinconsistencies between code and its correspondingdocumentation.

The ontological representation allows to performhigh-level software analysis tasks that include bothcode and documents within the same data structure.One obvious application area is the recovery of trace-ability links, as shown in Figure 14. This allows, forthe first time, the automatic establishment and analysisof semantic links, down to the level of individual wordsin documents and variables in code, which is of highimportance for the software industry. Here, we see thesystematic evaluation of automatic traceability analysissystems as the next major step to be carried out by thecommunity. A possible approach is the definition ofa yearly “competition,” with shared data, tasks, andevaluation metrics for all participants, similar to theones sponsored by the U.S. NIST within the fields oftext retrieval (TREC)16 and automatic summarization(DUC).17 Possible tasks could include the automaticrecovery of traceability links between source code anddocuments, or the analysis of existing links for incon-sistencies. To allow the automatic evaluation of systemresults, a manual “gold standard” would need to bedeveloped, together with suitable performance metrics(e.g., adapted versions of the common precision/recallmeasures). Such competitions are routinely used withinthe field of NLP to assess the state of the art within agiven task, as well as for the evaluation of scientificprogress achieved by new methods. The experiencefrom these competitions show that, although involv-ing a major community-wide investment, the gainedinsights for a given research problem is well worth theeffort.

Besides improving the individual components as dis-cussed in the evaluation section, we plan to extend oursystem to explicitly deal with documents associatedwith the different steps in the software life-cycle, fromwhite papers and requirements over design and imple-mentation documents to user’s guides and source codecomments. This will allow us to trace concepts and enti-ties across the different states of software developmentand different levels of abstraction.

References

[1] Giuliano Antoniol, Gerardo Canfora, GerardoCasazza, and Andrea De Lucia. Information re-

16Text REtrieval Conference (TREC), http://trec.nist.gov/17Document Understanding Conference (DUC), http://duc.nist.gov

trieval models for recovering traceability linksbetween code and documentation. In Proc. ofIEEE Intl. Conf. on Software Maintenance, SanJose, CA, USA, 2000.

[2] Giuliano Antoniol, Gerardo Canfora, GerardoCasazza, Andrea De Lucia, and Ettore Merlo. Re-covering Traceability Links between Code andDocumentation. IEEE Transactions on SoftwareEngineering, 28(10):970–983, October 2002.

[3] Grigoris Antoniou and Frank van Harmelen. ASemantic Web Primer. MIT Press, 2004.

[4] Franz Baader, Diego Calvanese, Deborah L.MacGuinness, Daniele Nardi, and Peter F. Patel-Schneider, editors. The Description Logic Hand-book: Theory, Implementation and Applica-tions. Cambridge University Press, second edi-tion, 2007.

[5] Ricardo Baeza-Yates and Berthier Ribeiro-Neto.Modern Information Retrieval. Addison-WesleyLongman Limited, 1999.

[6] Coral Calero, Francisco Ruiz, and Mario Piattini,editors. Ontologies for Software Engineering andSoftware Technology. Springer-Verlag Berlin Hei-delberg, 2006.

[7] H. Cunningham, D. Maynard, K. Bontcheva, andV. Tablan. GATE: A framework and graphicaldevelopment environment for robust NLP toolsand applications. In Proc. of the 40th AnniversaryMeeting of the ACL, 2002.

[8] R. Gaizauskas, M. Hepple, H. Saggion, M. A.Greenwood, and K. Humphreys. SUPPLE: Apractical parser for natural language engineeringapplications. In Proc. of the 9th Intl. Workshopon Parsing Technologies (IWPT2005), Vancouver,2005.

[9] Volker Haarslev and Ralf Moller. RACER SystemDescription. In Proceedings of International JointConference on Automated Reasoning (IJCAR),pages 701–705, Siena, Italy, June 18–23 2001.Springer-Verlag Berlin.

[10] IEEE. IEEE Standard for Software Maintenance.IEEE 1219, 1998.

[11] M.G. Ilieva and O. Ormandjieva. Automatic tran-sition of natural language software requirementsspecification into formal presentation. In 10thInternational Conference on Applications of Nat-ural Language to Information Systems (NLDB),volume 3513 of LNCS, pages 392–397, Alicante,Spain, June 15–17 2005. Springer.

[12] D. Jin and J. Cordy. Ontology-Based SoftwareAnalysis and Reengineering Tool Integration: TheOASIS Service-Sharing Methodology. In 21st

18

Text Mining and Software Engineering

IEEE International Conference on Software Main-tenance (ICSM), 2005.

[13] P. N. Johnson-Laird. Mental Models: Towardsa Cognitive Science of Language, Inference andConsciousness. Harvard University, Cambridge,Mass., 1983.

[14] Leonid Kof. Natural language processing: Ma-ture enough for requirements documents analy-sis? In 10th International Conference on Applica-tions of Natural Language to Information Systems(NLDB), volume 3513 of LNCS, pages 91–102,Alicante, Spain, June 15–17 2005. Springer.

[15] T. C. Lethbridge and A. Nicholas. Architectureof a Source Code Exploration Tool: A SoftwareEngineering Case Study. Technical Report TR-97-07, Department of Computer Science, Universityof Ottawa, 1997.

[16] M. Lindvall and K. Sandahl. How well do ex-perienced software developers predict softwarechange? Journal of Systems and Software,43(1):19–27, 1998.

[17] Christopher D. Manning and Hinrich Schutze.Foundations of Statistical Natural Language Pro-cessing. The MIT Press, 1999.

[18] Andrian Marcus and Jonathan I. Maletic. Recover-ing Documentation-to-Source-Code TraceabilityLinks using Latent Semantic Indexing. In Proc. of25th Intl. Conf. on Software Engineering, 2002.

[19] Andrian Marcus, Jonathan I. Maletic, and An-drey Sergeyev. Recovery of Traceability Links be-tween Software Documentation and Source Code.Intl. J. of Software Engineering and KnowledgeEngineering, 15(5):811–836, 2005.

[20] Vladimir Mencl. Deriving behavior specificationsfrom textual use cases. In Proceedings of Work-shop on Intelligent Technologies for Software En-gineering, pages 331–341, Linz, Austria, 2004.Oesterreichische Computer Gesellschaft.

[21] W. Meng, J. Rilling, Y. Zhang, R. Witte, andP. Charland. An Ontological Software Compre-hension Process Model. In 3rd Int. Workshop onMetamodels, Schemas, Grammars, and Ontolo-gies for Reverse Engineering (ATEM 2006), pages28–35, Genoa, Italy, October 1st 2006.

[22] N. F. Noy and H. Stuckenschmidt. OntologyAlignment: An annotated Bibliography. In Se-mantic Interoperability and Integration, SchlossDagstuhl, Germany, 2005.

[23] Juergen Rilling, Rene Witte, and YonggangZhang. Automatic Traceability Recovery: An On-tological Approach. In International Symposiumon Grand Challenges in Traceability (GCT’07),Lexington, Kentucky, USA, March 22–23 2007.

[24] C. Riva. Reverse Architecting: An Industrial Ex-perience Report. In 7th IEEE Working Conferenceon Reverse Engineering (WCRE), pages 42–52,2000.

[25] Marta Sabou. Extracting Ontologies from Soft-ware Documentation: a Semi-Automatic Methodand its Evaluation. In ECAI-2004 Workshopon Ontology Learning and Population, Valencia,Spain, 2004.

[26] R. Seacord, D. Plakosh, and G. Lewis. Mod-ernizing Legacy Systems: Software Technologies,Engineering Processes, and Business Practices.SEI Series in SE. Addison-Wesley, 2003.

[27] M. Shaw and D. Garlan. Software Architecture:Perspectives on an Emerging Discipline. PrenticeHall, 1996.

[28] I. Sommerville. Software Engineering. Addison-Wesley, 6th edition, 2000.

[29] M. A. Storey, S. E. Sim, and K. Wong. A Col-laborative Demonstration of Reverse Engineeringtools. ACM SIGAPP Applied Computing Review,10(1):18–25, 2002.

[30] C. Welty. Augmenting Abstract Syntax Trees forProgram Understanding. In Proc. of Int. Conf. onAutomated Software Engineering, pages 126–133.IEEE Comp. Soc. Press, 1997.

[31] Rene Witte and Sabine Bergler. Fuzzy Corefer-ence Resolution for Summarization. In Proceed-ings of 2003 International Symposium on Refer-ence Resolution and Its Applications to QuestionAnswering and Summarization (ARQAS), pages43–50, Venice, Italy, June 23–24 2003. UniversitaCa’ Foscari.

[32] Rene Witte and Sabine Bergler. Next-GenerationSummarization: Contrastive, Focused, and Up-date Summaries. In International Conference onRecent Advances in Natural Language Processing(RANLP 2007), Borovets, Bulgaria, September27–29 2007.

[33] Rene Witte, Thomas Kappler, and ChristopherJ. O. Baker. Ontology Design for BiomedicalText Mining. In Semantic Web: RevolutionizingKnowledge Discovery in the Life Sciences, chap-ter 13, pages 281–313. Springer, 2007.

[34] Rene Witte, Yonggang Zhang, and JuergenRilling. Empowering Software Maintainers withSemantic Web Technologies. In E. Franconi,M. Kifer, and W. May, editors, 4th European Se-mantic Web Conference (ESWC 2007), number4519 in LNCS, pages 37–52, Innsbruck, Austria,June 2007. Springer-Verlag Berlin Heidelberg.

19