Semantically enhanced sequential patterns for content adaptation on the web

Semantically enhanced sequential patterns forcontent adaptation on the webMehdi Adda1,3, Petko Valtchev1, Rokia Missaoui2, Chabane Djeraba3

1DIRO Université de MontréalCP 6128 succ Centre-Ville Montréal QC H3C 3J7 Canada

2Département d’informatique et d’ingénierie Université du Québec en OutaouaisCP 1250 succ B Gatineau QC Canada J8X 3X7 Canada

3 LIFL - UMR CNRS 8022 - Bâtiment M3 59655 Villeneuve d’Ascq Cédex - FRANCEaddamehd,[email protected], [email protected], [email protected]

Abstract— Content adaptation on the Web is aimed at reducingthe total amount of information served by a site or applicationto the items matching the needs/preferences/tastes of a specificuser. As a major trend thereof, recommender systems typicallyrely on the computation of a relevance score for content objectto the anticipated user’s needs. Using the navigation patterns fora user or a group of users as a support for the relevance guessis a classical approach to recommendation that breaks downto discovering, or mining, associations among content objectsbased on the way these are navigated through. As feeding domainknowledge into the mining process has proven to both increasethe precision of these associations and ease their interpretation,we consider the "extreme" case, i.e., the availability of a full-scaledomain ontology whose concepts and properties characterizethe content objects and their relationships. We thus tacklethe problem of frequent pattern extraction from sequences ofcontent objects corresponding to user sessions, which are furtherdescribed within a domain ontology. Here we define a theoreticalframework for the resolution of the mining task which comprisesa pair of languages for data and pattern description, respectively,and a hierarchically-organized pattern space split into levels. Thelatter underlies an Apriori-like level-wise method for frequentpattern generation and evaluation.

I. INTRODUCTION

The spectacular growth of the Web along the last decadehas led to a situation where a user is faced with a hugequantity of information whereas only a tiny part of it might berelevant to the user’s needs (preferences, tastes, sensibilities,etc.). Even on a single site scale, the variety of content objectsserved by an application may require a specific mechanism tohelp the user find items of interest. Unlike search engines,which require a good knowledge of object descriptions interms of keywords, content adaptation mechanisms in generaland recommender systems in particular are less intrusive.Adaptation relies on the assessment of the relevance degreeof an object with respect to the anticipated user’s needs whichremain, in most cases, implicit. Therefore, the relevance isapproximated based on data about the adequacy of the objectto other (similar) users or of other (similar) objects to thesame user. In the later case, objects are compared based onthe heuristic guess that likeness will reflect similar relevancedegrees. Likeness scores are obtained in various manners, suchas: (i) similarity computation from structured descriptions

of objects, (ii) association extraction from co-occurrences ofobjects in user logs or (iii) aggregation of direct user voteson the relevance of particular objects.

We are currently investigating an association-based recom-mendation approach whose key task is the mining of meaning-ful associations from the raw data about user clicks (or object"visits"). Unlike the classical association mining task [1], theapproach relies on additional knowledge about the domainunderlying the served content (e.g., e-tourism, on-line sale ofPCs and appliances, etc.). Our hypothesis is that incorporatingsuch knowledge in pattern extraction should help increase therelevance of the discovered associations while granting themhigher degree of interpretability. Ontologies have become thestandard way for expressing knowledge about a domain andthe key to interoperability on the Web. Therefore, a growingnumber of information providers on the Web power their appli-cations with an ontology describing the content that is served.Hence, in our approach we hypothesize that an ontology isavailable and retrieve two types of knowledge about contentobjects: membership to generic concepts (e.g., categories ofproducts to be sold on-line at various levels of abstraction)and existence of inter-object links (e.g., compatibility, part-of,bonus product links, etc.). The data we work on is thereforederived from the raw object sequences by integrating semanticelements, such as the existing links between objects of thesequence and, later on, the domain concepts to which objectsbelong to, and the concept roles underlying the links.

The corresponding pattern mining problem amounts to acombination of structured pattern mining (sequences [2], treesor graphs) and generalized pattern mining [3] which, to thebest of our knowledge, has not been delt with before. Thekey difficulty here is the complex inner structure of thepatterns, which vary on two dimensions: the is-a hierarchyof the ontology and the network that roles induce on top ofit, resulting in a combinatorial explosion in the space of allpotential patterns. Hence the design of efficient and effectivemining methods require insightful procedures for pattern spacetraversal, candidate generation and frequency tests. In [4], wehave presented the main elements of our theoretical frame-work, i.e., two descriptive languages, a generality relation

on patterns, level-wise splitting of the pattern space and analgorithm that takes advantage of that splitting “à la Apriori”.The present work completes the theoretical bases and exploresthe application of semantically enhanced patterns for contentadaptation on the web. Topics such as efficiency, scalabilityand definition of reduced representations of the interestingpatterns such as closed and maximal ones, are left for a futuredevelopment.

The rest of the paper is organized as follows. Section IIsummarizes related work on sequential pattern mining. SectionIII presents the theoretical basis of an approach to mine anew category of sequential patterns: semantically enhancedsequential patterns. A new recommendation strategy basedon the mined patterns is presented in Section V. SectionVI gives an example that illustrates the way our algorithmxPMiner works and presents some advantages of exploitingthe mined patterns in a recommendation process comparedto the classical approaches usually used in recommendationsystems. Concluding remarks and discussions on future workare given in Section VII.

II. RELATED WORK

Apriori is a classical algorithm for frequent pattern min-ing [1]. Although it has been defined for plain itemsets,the underlying strategy has been later on adapted for dataformats involving some inner structure, such as sequences,trees, graphs, etc.

Sequential pattern mining problem was first introduced byAgrawal and Srikant [2] where patterns involve items only.The initial definition was extended to cover domain categories(classes). This leads to the notion of generalized sequentialpatterns [3]. Mannila et al. [5] proposed a new approachin which patterns are composed of episodes. An episode isdefined as a collection of events that occur relatively closeto each other in a given partial order. Instead of miningsimple patterns, Pinto et al. have proposed multidimensionalsequential patterns [6], [7] where sequential patterns aregenerated from multidimensional data. Bettini et al. [8] andWang et al. have addressed the problem of sequential meta-pattern mining on object sequences [9], [10], where a meta-pattern is a pattern of patterns.

Existing approaches are generally limited to mine objectsand/or classes while other domain knowledge such as inter-object links and inter-class relations are not taken into account.Our preliminary studies (see [11] and [4]) explore associationrule mining based on domain ontology links and relations asa mechanism for system recommendation and semanticallyenhanced sequential pattern extraction. By including linksand relations in the mining process, complex structures –like graphs– need then to be handled like in graph miningfield where notions and algorithms for discovering frequenttopological substructures over a set of graphs are studied.As in sequential pattern mining, graph mining relies mostlyon the Apriori algorithm proposed in association rule mining[1]. For example, Kuramochi and Karypis [12] describe analgorithm, called FSG, which incrementally mines frequent

subgraphs by adding at each pass a new edge. Inokuchi etal. [13] present another Apriori based algorithm called AGMwhere a new node is added at each pass. Some other studieswere concerned with mining frequent subtrees using Apriorischeme, and algorithms are proposed such as TreeMiner [14]and FreeTreeMiner [15]. Other graph mining algorithms donot rely on Apriori principle and use depth-first or breadth-first search strategies. This is the case for Gspan proposed byYan et Han [16] and DSPM proposed by Cohen et al. [17].Algorithms such as Closegraph [18], CMTreeMiner [19] andSpin [20] allow the mining of frequent and closed subgraphs.

Before presenting our own approach, we go back to Aprioriwhich – more than a concrete method – represents a strategyfor pattern mining that we have adopted in our knowledge-richcontext. Indeed, the algorithm could be seen as a combinatorialgeneration procedure that uses additional validity criteria. Inthis sense, it represents a traversal of the pattern space ina top-down, breadth-first search. The space is layered, withlayers corresponding to pattern cardinality. Moreover, to movebetween levels, Apriori uses optimization tricks: new levelelements are generated by combining two elements from thelevel immediately above while interestingness measures (e.g.,support) use already tested candidates from the same level.Finally, two languages are implicit in the approach: the dataand the pattern ones. In the classical version, both are made ofitemsets and hence are indistinguishable. Both relations thatare inherent to such pairs of languages, instantiation linkingdata records (transactions) to patterns that "characterize" them,and generality between patterns, reduce to mere set-inclusion.The next sections explain how the above elements are spelledin our own environment.

III. PATTERNS LANGUAGES AND RELATIONS

The presentation of the descriptive formats for data andpatterns starts with a brief recall of some basic terms fromthe ontology engineering field followed by the description ofthe underlying languages with their syntax and semantics.

A. Ontologies and ontological languages

An ontology is an explicit conceptualization of a specificdomain, for instance, the tourism domain. In particular, itallows for a representation of domain concepts and theirrelations, or roles, as well as for the representation of someinstances of those concepts that are further related by linksinstantiating generic roles. Roles are usually directed andbehave like functions: a role maps the instances of a sourceconcept to those of a target one. For instance, Figure 1 depictsa partial view of an ontology, called Travel1, as shownby the Protégé plug-in ezOWL. Concepts like Destination,Activity and Accommodation are shown. Furthermore, roleslike hasActivity and hasAccommodation are expressed in theontology, although the selected view does not show them.Instances and their links, for example, a Destination objectcalled Montréal with an Accommodation called Hotel CrownePlaza may appear in the ontology.

1http://protege.stanford.edu/plugins/owl/owl-library/travel.owl

Ontologies have recently gained interest with the emer-gence of the Semantic Web [21], and some standardizationefforts are underway in the field of ontological languages(e.g., DAML+OIL [22], OWL [23]). OWL (Ontology WebLanguage) is a de facto standard for ontologies on the web.Its formal background lays in the description logic (DL)field [24]. This means that an ontology Ω is expressed using alanguage whose building blocks are a set of (atomic) conceptnames TC and role names TR. These are further combinedinto more complex descriptions by a set of constructors, suchas conjunction, disjunction, role restrictions, etc. Individualsare introduced as a set of identifiers OΩ (or simply O).

Although ontological languages provide powerful mecha-nisms for reasoning such as subsumption computation fordescriptions, we limit our usage of an ontology to its strict rep-resentation functions. Hence we assume an asserted generality(is-a) relationship between descriptions, i.e., both concepts androles, in Ω, say ⊆Ω, and a computed subsumption relationshipvΩ that extends ⊆Ω to all descriptions of the language behindΩ. For example, just like Hotel is a subclass of Accommoda-tion, hasHotel is a sub-role of hasAcccommodation. Moreover,we use two functions to expressed the connections between theroles and the concepts they describe. Thus, given a relationr ∈ TR, dom(r) provides the most general concepts thatare specified as having the role r, while ran(r) provides themost general concepts that have been specified by the ontologydesigner as target concepts. In a restricted version, ran(r, c)where c ∈ TC , the latter function only computes the mostgeneral target concepts whenever the source is c. Furthermore,as ⊆Ω is usually a partial order, it makes sense to speakabout predecessors (predΩ(ci)) and successors (succΩ(ci)) ofa concept/role in the respective is-a hierarchy. Descriptionswhich are neither in predecessor nor in successor relationshipsare incomparable, and the underlying relationship will bedenoted by ⊥. The ontology-related notations are summarizedin Table I.

The descriptions of an ontology language such as OWLare typically provided with an abstract denotational semanticswhich requires an interpretation domain and an interpretationfunction. However, for our purposes, we limit that domain tothe set of all instances O, which means the semantics of aconcept c is the set of all individuals that are explicitly statedto belong to c or to a sub-concept thereof. Similarly, a roler is interpreted as a set of links between individuals from O.These concrete semantics are jointly denoted [ ]Ω.

In the remainder of this article we will use OWL terms likeclass, object and relation to indicate concept, individual androle respectively.

B. Descriptions of data and patterns

The raw data we process consists of sequences of objectURLs as recorded in the Web site logs. These are easilytranslated into object identifiers from the ontology. Thus, onthe ontology side, a data entry belongs to Oω = (O×O× ...),the set of all sequences on O. Although patterns could bedescribed at the object level, these would have only limited

Expression DefinitionΩ domain ontologyTC set of all ontology concept names c1, c2, ..., cn

TR set of all ontology relation names rvΩ concept subsumption relation on Ωdom(r) domain concepts of the relation rran(r) range concepts of the relation rran(r, c) range of r whenever its domain restricts to csuccΩ(ci) set of successors of concept ci in ΩpredΩ(ci) set of predecessors of concept ci in Ωci ⊥ cj ci, cj are incomparable if ci is neither parent nor

child of cj

ri ⊥ rj relations ri and rj are incomparable if ri is neithersuccessor nor predecessor of rj

Max(c1, c2, set of concepts ci which are incomparable..., cn) and have highest levels of abstraction

TABLE IBASIC ONTOLOGICAL DEFINITIONS

interest since for a large number of objects in O, the chancesof finding frequent combinations are relatively low. In contrast,moving to the class level increases substantially those chances.Therefore, we shall be looking exclusively at patterns whoseelements are ontology classes. Moreover, relations betweenclasses are incorporated in order to add more semantic preci-sion to the resulting patterns. To that end, the initial objectsequences are first extended by incorporating all the validlinks from the ontology. For conciseness reasons, we limit theinserted links to those which are "co-linear" with the orderamong objects in s (the source object of the link is before thetarget one).

1) Pattern syntax: Assume a set of object (ground) se-quences D and let DΩo

be the set of extended object sequenceswhereas extended means adding the links to the initial struc-ture as described above.

Then, let ΓO denote the universe of all possible objectsequences (see class sequences for a formal definition). Letalso, ΓΩ be the universe of all extended class sequences (calledsimply sequences whenever confusion is excluded) that maybe produced on top of an ontology Ω. An extended sequence isthe pattern we are looking for. It is made of a basic part, whichis an ordinary list, and an extension, which is a set of relationsgiven with the indices of connected classes in the sequence.Formally speaking, the abstract syntax of our pattern languageis defined as follows:

Definition 3.1: An extended class sequence is a pair S =〈str, rel〉 where:• str = 〈ci〉i=1..n is a sequence of concept names from TC

such that: ci 6= ci+1 for 1 ≤ i < n;• rel = r(l,m) | 1 ≤ l ≤ m ≤ n is a set of relations

(triplets) such that:– r ∈ TR is an ontology role name;– ∃c ∈ dom(r) such that cl vΩ c, which we will

denote by cl vΩ dom(r);– cm vΩ ran(r, cl).

In the remainder of this paper we denote 〈ci〉i=1..n by S.str,r(l,m) | 1 ≤ l ≤ m ≤ n by S.rel and ci by S(i).

Fig. 1. Partial view of the travel ontology: nodes in the graph represent domain concepts and the labels associated with some of them represent the numberof sub-concepts not shown in the figure. Lines represent the binary relationship is-a between concepts.

For example, assume that on Figure 1, a role hasActivityis defined between classes Destination and Activity,and that hasAccommodation is a role of Destinationwhose co-domain is Accommodation. Consider now thesequence on Figure 2 in which three classes, Destination,Accommodation, Activity, drawn as ovals, are given inthe order of their appearance (to be read from left to right)and connected by the respective roles. The corresponding ex-tended class sequence, say S1, will be represented as follows:〈〈Destination,Accommodation,Activity〉,hasAccommodation(1, 2), hasActivity(1, 3)〉.

Destination Accommodation Activity

hasAccommodation

hasActivity

Central_Florida Marriott_Orlando_World_Center_Resort Surfline

hasHotel

hasActivity

Fig. 2. Example of an extended class sequence.

Pattern language is endowed with the empty element S∅ =〈ε,∅〉, where ε is the empty sequence.

2) Pattern semantics: As for the semantics of the abovepattern constructs, one has to ask the question which dataentries do they represent? Thus, a pattern has to be interpretedin terms of extended object sequences, which means given Ωand O, our interpretation domain is ΓO.

Now, the function assigning to a pattern the set of all

object sequences that are "summarized" by that pattern isbased on the following intuition. First, the structure of theobject sequence is reflected in the pattern in a possibly reducedmanner. Reduction here means that several objects from thedata may be represented by a single common class in thepattern. In addition, not all objects need to be representedin the pattern, some may simply vanish. Similarly, relationallinks may be jointly represented in the pattern as a relation,or simply vanish. To spell it differently, given a pattern andan object sequence, all classes from the pattern need to havesome "basis" objects in the sequence and the object packetsrepresenting class bases follow the order among classes in thepattern (i.e., pattern and sequence are aligned). Moreover, foreach relation from the pattern, at least one link exists betweenthe respective bases of classes adjacent to the relation.

More rigorously, we may express the instanciation betweena sequence and a pattern as the existence of a partial mappingfrom the sequence objects to the pattern classes such that:

• an object is only mapped to a class it is an instance of,• the order in the object sequence is preserved: if the

mapping is defined for two objects, then their imageclasses are necessarily in the same order,

• the mapping is surjective,• for each relation in the pattern, there is a link which

is an instance of the relation whose adjacent objectsare respectively mapped to the adjacent classes of therelation,

Clearly, the above mapping is not necessarily unique for a pairof an object sequence and a pattern. Formally, the mapping isdefined as follows:

Definition 3.2: Given s ∈ ΓO, S ∈ ΓΩ, S represents s(s ∈ [S]ΓΩ ) if there exist a set I ⊆ [1..|s|] and a surjectivemonotonously non-decreasing map: ψ : I → [1..|S|] such that:• ∀i ∈ I, s(i) ∈ [S(ψ(i))]Ω;• ∀r(i1, i2) ∈ S.rel,∃(j1, j2) ∈ (I)2 such that ψ(j1) =i1, ψ(j2) = i2, and ∃r′(j1, j2) ∈ s.rel, such that r′ vΩ r.

Based on pattern semantics, a generality relationship be-tween patterns can be established. Thus, given a couple ofextended sequences S1,S2 in ΓΩ, S1 is more general thanS2, denoted S2 ≤ S1, if and only if [S2]Ω ⊆ [S1]Ω. Inother terms, generalization amounts to the inclusion of patterninterpretations.

3) Pattern subsumption: Generalization relation definition,such as given above, is impractical as the set of all objetsequences is not available. Instead, like in FOL [24], we definea syntactically-bound relation called subsumption which isprovably equivalent to semantic generalization. The definitionwe use is close to the one provided for instanciation whichunderscores the similarity of the structures between extendedobject sequences and patterns.

Definition 3.3: Given S1,S2 ∈ (ΓΩ)2, S2 subsumes S1

(S1 vΓΩ S2) if there exist a set I ⊆ [1..|S1|] and a surjectivemonotonously non-decreasing map: ψ : I → [1..|S2|] suchthat:• ∀i ∈ I,S1(i) vΩ S2(φ(i));• ∀r(i1, i2) ∈ S2.rel, ∃(j1, j2) ∈ (I)2 such that ψ(j1) =i1, ψ(j2) = i2, and ∃r′(j1, j2) ∈ S1.rel, such that r′ vΩ

r.Definition 3.3 basically settles pattern subsumption on a graphmorphism basis. Moreover, unlike generalization relations onsimilar knowledge formalisms (e.g., conceptual graphs [25]),the above relationship is a partial order, i.e., S1 vΓΩ S2

and S2 vΓΩ S1 imply S1 = S2. Figure 3 illustrates patternsubsumption according to the ontology travel. While V iews Aand C show two patterns that are subsumed by the patternpresented in V iew D according to the above definition,V iew B shows a pattern where conditions of Definition 3.3are not satisfied. In fact, the sequence order in the two patternsis not the same.

It may now be stated formally that subsumption can be usedto test generalization.

Proposition 3.4: Let S1, S2 be two extended sequencesfrom ΓΩ such that S1 vΓΩ S2, then S1 ≤ S2.

C. Pattern operations

Although the subsumption computation does not requireanymore the availability of the entire universe of extendedobject sequences, it is still tricky to check whether two patternsare comparable, mainly because of the constraints related tothe ground morphism. To ease that, we further decomposethe morphism into a sequence of simple transformations that

Capital LuxuryHotel Museum

hasLuxuryHotel

hasMuseum

Destination Accommodation Activity

hasAccommodation

hasActivity

Campground Contact

hasContact

Capital LuxuryHotelMuseum

hasLuxuryHotel

hasMuseum

Campground Contact

hasContact

Capital LuxuryHotel MuseumCampground Contact

hasContact

(View A)

(View B)

(View C)

(View D)

hasAccommodation

Fig. 3. Subset of an extended class sequence interpretation

turn the more general pattern into the more specific one. Thisderivational model of subsumption relies on the definition offour operations that jointly represent a complete and consistentcalculus for subsumption. In other terms, two patterns are ingenerality relation if and only if one of them can be obtainedfrom the other one by applying a finite number of operationsof the set.

The four operations consists in, respectively, specializing asingle occurrence of a class in the sequence (i.e., replacingit by a more specific class), adding a new class at a specificlocation of the sequence, specializing an existing occurrenceof a relation, and adding a new relation between two classesof the sequence. They are described in the remainder of thissection.

Specialize a class: replace the class at position j of asequence S = 〈str, rel〉 with a class c from TC provided thatc is more specific than S(j) in the ontology Ω, c vΩ S(j),c is comparable with the actual ranges of all the propertiesr(_, j) from rel, and c is different from the neighbor classesin the sequence. Formally, we have the following:

Definition 3.5: Let S ∈ ΓΩ, c ∈ TC . The operationsplCls(S, c, j) yields a sequence 〈str, rel〉 with:• str = 〈S(1), ...,S(j − 1), c,S(j + 1), ...,S(|S|)〉;• rel = S.rel,

where the following conditions are satisfied1) c vΩ S(j);2) c 6= S(j − 1);3) c 6= S(j + 1);4) ∀r(i, j) ∈ S.rel : c vΩ ran(r,S(i)), 1 ≤ i < j ≤ |S|.For example, by replacing Activity with a

more specific class Sightseeing in the pattern〈〈Destination,Activity〉, 〉 we obtain the pattern〈〈Destination, Sightseeing〉, 〉.

Add a class: insert a class c from TC into an extendedsequence S at a position j is possible provided that j is withinthe range of the sequence size and c is not identical to itsneighbor classes in the sequence.

Definition 3.6: Let S ∈ ΓΩ, c ∈ TC . The operationaddCls(S, c, j) yields a sequence 〈str, rel〉 with:

• 1 ≤ j ≤ |S|+ 1;• c 6= S(j − 1);• c 6= S(j + 1);• str = 〈S(1), ...,S(j − 1), c,S(j), ...,S(|S|)〉;•

rel =

r(l,m)|r(l,m) ∈ S.rel, 1 ≤ l < m < j∪r(l,m+ 1)|r(l,m) ∈ S.rel, 1 ≤ l < j ≤ m∪r(l + 1,m+ 1)|r(l,m) ∈ S.rel, 1 ≤ j ≤ l < m

For instance : addCls(〈〈Destination,Activity〉,hasActivity(1, 2)〉,Hotel, 2) =〈〈Destination,Hotel, Activity〉, hasActivity(1, 3)〉.

Specialize a relation: replace a relation occurrence r(i, j)of an extended sequence S = 〈str, rel〉 with another relationoccurrence r′(i, j) provided that r′ is a specialization of r.This is captured by the following definition:

Definition 3.7: Let S ∈ ΓΩ, r ∈ TR. The operationsplRel(S, r, i, j, r′) yields a sequence 〈str, rel〉 with:• str = S.str;• r(i, j) ∈ S.rel;• rel = S.rel−r(i, j)∪r′(i, j), provided that r′ vΩ r.For instance : splRel(〈〈Capital,Museum〉, hasActivity

(1, 2)〉, hasActivity, 1, 2, hasMuseum) =〈〈Capital,Museum〉, hasMuseum(1, 2)〉.

Add a relation: Adding a relational link r, from TR,between the classes at positions i and j, respectively, of anextended sequence S is possible provided that these positionsare properly ordered and lay within the range of the sequencesize, as well as the corresponding sequence members are atleast sub-classes of a valid pair of domain and range classesfor r in the ontology Ω, and that there is no relation in thesequence with the same indices, which is a specialization ofr.

Definition 3.8: Let S ∈ ΓΩ, r ∈ TR. The operationaddRel(S, r, i, j) yields a sequence 〈str, rel〉 with:• str = S.str;• rel = S.rel + r(i, j),

where the following conditions are satisfied1) 1 ≤ i ≤ j ≤ |S|;2) S(i) vΩ dom(r);3) S(j) vΩ ran(r,S(i));4) @r′ ∈ S.rel such that r′ vΩ r.For example, an insertion of the relation

hasMuseum into the pattern yielded by theprevious operation has the following effect:addRel(〈〈Destination,Museum〉, 〉, hasMuseum, 1, 2) =〈〈Destination, Museum〉, hasMuseum(1, 2)〉.

The above operations do cover all the possible cases ofspecialization among patterns. However, an effective genera-tion procedure using the Apriori level-wise traversal principlerequires a precise definition of what level is. Unlike regular

itemsets where the level notion relies on the size in numberof items, there is no straightforward way of defining levelsin our patterns space. In the next paragraph we proposesuch definition exploring the key fact that levels are actuallyanother way of defining the precedence order of the generalityrelationship between patterns.

D. Rank-based level partition of the pattern space

To motivate our level construct, observe that in the Booleanlattice of all itemsets, the immediate successors of a pattern pw.r.t. the generality relationship are all super-itemsets p′ thathave exactly one item more than p. Thus, they lay in the levelk+1 of the lattice, given that p is on level k, i.e., has k items.Thus, in the Apriori traversal of the lattice, one moves from afrequent p to the test of all or at least some of its successorsby, basically, adding a single new item to p and looking at theresulting candidates2. A similar principle could be imagined inour case, provided we can specify how generality precedencecould looks like. To that end, we need to define elementary oratomic versions of the above operations, i.e., transformationsthat turn a pattern into an immediate successor of its.

The goal is, given an operation, to make sure no patternmay be more specific than the argument pattern while strictlymore general than the operation result. Intuitively, this can beachieved by limiting the scope of both operation sorts, i.e., ad-dition and specialization of elements. Thus, the specialization-bound operations will only perform a single down step inthe respective generalization hierarchies, that is immediatesuccessors of classes and relations could only be used insubstitutions. Similarly, for addition, only maximally generalclasses and relations could be used.

Although plausible, the above principles does not admit awell-founed definition of levels in the pattern space. Indeed,while the Boolean lattice of itemsets shows a highly regularstructure, allowing easy computation of the level to which apattern belongs, this is not the case in our pattern space wherethe generalization hierarchies of the ontology may introduceirregularities. Hence, our approach is to construct the definitionof a level on top of a measure of generality for patterns. Themeasure, just like the itemset size in the case of Apriori, is tobe a linear extension of the pattern space order, but, in additionto that need to reflect the potentially irregular character of thespace whenever induced by an ontology.

Clearly, the measure should reflect the cumulated specificityof the pattern elements, relations and classes, in their respec-tive hierarchies. To that end, first, individual measures for bothelement sorts much be defined. With no surprise, these willhave to depend on the location of the elements within thehierarchy, i.e., will reflect their "depth" in the partial orderthe structure represents. There is no standard definition of thedepth. From a graph theory point of view, one may chosebetween the shortest or longest path in the precedence DAG

2Actually, the generation of candidates is slightly more complex as databasejoins are performed on patterns of the level k, but end-effect is neverthelessexactly as indicated.

of the partial order leading to the node and starting with a topmost element (typically there is a unique Thing class of anontology). As we need to reflect relative generality, we wouldneed monotony for the measure, that is more specific patternswill need to have greater depth values. Hence we chose thelongest path metric as the shortest one is not necessarilymonotonous.

Mathematically, a step-wise definition for both measures,called height may be provided.

Definition 3.9: Let c in TC , r in TR. The height functionsfor classes and relations, hc

Ω and hrΩ, respectively, are defined

on Ω as follows:• hc

Ω : TC → N is defined as follows:

hcΩ(c) =

1, if predΩ(c) = ∅maxhc

Ω(c′)|c′ vΩ c+ 1, otherwise.

• hrΩ : TR → N is defined as follows:

hrΩ(r) =

1, if predΩ(r) = ∅maxhr

Ω(r′)|r′ vΩ r+ 1, otherwise.To spell it differently, the above definition assures that each

element has a height value that is at least a unit bigger thanthe values of its predecessors.

We can now define a rank function underlies the levelsstructure within the pattern space 〈ΓΩ,vΓΩ〉. The rank valuereflects the generality of a sequence by mixing the generalityof member classes and relations. As the combination is addi-tive, it indirectly reflects the number of classes and relationsin the pattern as well. Technically speaking, the rank of anextended class sequence ρ : ΓΩ → N, is the sum of the heightsof its classes and of the included relations:

ρ(S) =∑

c∈S.str

hcΩ(c) +

∑r∈S.rel

hrΩ(r).

For example, and according to the ontologyused in this work, the rank of the pattern S =〈〈Destination,Hotel〉, hasHotel(1, 2)〉 is: ρ(S) = 5as Destination has a depth of one, while Hotel and hasHotelare both of depth two in their respective hierarchies.

The rank, as defined above, is monotonously decreasingwith the generalization relation as indicated by the nextproposition.

Proposition 3.10: Let S1, S2 be two extended sequencesfrom ΓΩ. S1 vΓΩ S2 entails ρ(S1) ≥ ρ(S2).

We may now define the level k in 〈ΓΩ,vΩ〉 as beingcomposed of all the sequences of rank k. It is noteworthythat there may be natural numbers k for which the level k isempty.

When we go back to the elementary operations, we shallbe looking at those that not only produce an immediatesuccessor patterns, but also allow for an only unit increasein the pattern rank. This means that the result will lay on thenext generality level, something which is not insured by onlyrequiring precedence between argument and result.

The new operations are merely constrained versions of thesedefined in the previous section. We name them by addinga simple ’E’ (for elementary) suffix to the original names,yielding addClsE(), addRelE(), splClsE() and splRelE().

For class additions, we shall only authorize maximallygeneral classes that satisfy the conditions of Definition 3.6.This gives the following definition.

Definition 3.11: Let S ∈ ΓΩ, c ∈ TC . The opera-tion addClsE(S, c, j) is defined as: addClsE(S, c, j) =addCls(S, c, j) only for the classes c from the set Max(TC)−S(j − 1),S(j). Otherwise it is undefined.

For example: addClsE(〈〈Destination, Activity〉,hasActivity(1, 2)〉, Accommodation, 2) =〈〈Destination, Accommodation, Activity〉,hasActivity(1, 3)〉.

For elementary concept specialization, in turn, only re-placement with immediate successor from the ontology areadmitted. Moreover, to respect our level constraint, the heightof this successor must be exactly one plus the rank of theoriginal concept.

Definition 3.12: Let S ∈ ΓΩ, c ∈ TC . The opera-tion splClsE(S, c, j) is defined as: splClsE(S, c, j) =splCls(S, c, j) only for the classes c from the setMax(succΩ(S(j))) s.t. hc

Ω(c) = hcΩ(S(j)) + 1. Otherwise

it is undefined.For instance: splClsE(〈〈Destination,Activity〉, 〉,

Sightseeing, 2) = 〈〈Destination, Sightseeing〉, 〉.

Equivalently, for relation additions, we shall only authorizegeneral relations that satisfy the conditions of Definition 3.8.

Definition 3.13: Let S ∈ ΓΩ, r ∈ TR. The opera-tion addRelE(S, r, i, j) is defined as: addRelE(S, r, i, j) =addRel(S, r, i, j) only for the relation r from the setMax(TR). Otherwise it is undefined.

For instance: addRelE(〈〈Capital, LuxuryHotel〉, 〉,hasAccommodation, 1, 2) = 〈〈Capital, LuxuryHotel〉,hasAccommodation(1, 2)〉.

For relation specialization only replacement with immediatesuccessors from the relation is-a hierarchy are admitted, andonly for those whose height is greater with a difference ofone.

Definition 3.14: Let S ∈ ΓΩ, r, r′ ∈ (TR)2. The operationsplRelE(S, r, i, j, r′) is defined as: splRelE(S, r, i, j, r′) =splRel(S, r, i, j, r′) only for the relations r′ from the setMax(succΩ(r)) s.t. hr

Ω(r′) = hrΩ(r) + 1. Otherwise it is

undefined.For instance: splRelE(〈〈Capital,Museum〉, hasActivity

(1, 2)〉, hasActivity, 1, 2, hasSightseeing) = 〈〈Capital,Museum〉, hasSightseeing(1, 2)〉

Finally, while the above restrictions on the global traversalprinciples limit the number of generations of a specific pattern(it can only be generated by a subset of its immediate

predecessors in the pattern space), there could still be a largenumber of such generations. To further reduce redundancy,we add additional constraints to the admissible operationsof candidate pattern generation. Ideally, a there should be asingle, canonical generating predecessor, although this seemout of our reach for the time being. Nevertheless, by restrictingthe place within the sequence where operations may apply,we further diminish the combinatorics. Indeed, we set thesequence end as the unique place where the operations must beperformed. More specifically, only final classes will be eligiblefor specialization while new classes could only be added at theend of the sequence. For relations, that means that the targetclass must necessarily be the final one of the sequence.

The resulting operations, called canonical, work as restric-tions on the respective elementary ones. They are namedby adding a ’C’ suffix (for canonical) to the base name:addClsC(), splClsC(), addRelC() and splRelC().

Instead of providing separate definitions of those operations,we refer the reader to Algorithms 4, 5, 6, and 7, respectively,which describe the computation behind each of them.

IV. FREQUENT PATTERN MINING

Mining sequential patterns from the extended sequencesDΩo

amounts to finding all the members of ΓΩ having supportgreater than minsup, a user provided threshold. The targetset is thus Γs

Ω = S ∈ ΓΩ|supp(S) ≥ minsup wherethe support of S is the size of the subset of DΩo

passes theinstanciation test for S.

In the following paragraphs we present our mining methodthat, similarly to Apriori, performs a top-down level-wisesearch through the pattern space. We start with a description ofthe instanciation test (see Section III-B.1), i.e., the procedurethat determines whether an extended object sequence is a partof the interpretation of a (candidate for frequent) pattern.

A. Instantiation test

The goal is to check whether a class pattern represents agiven object sequence. To that end, an algorithm that closelyfollows Definition 3.2 is used. Hence, its principle may besummarized as follows: for each class and each relationoccurrence from the pattern look for a supporting set, orimage, in the extended object sequence and make sure imagesfor concepts and relations concord. Classes are mapped tocontiguous subsequences of their instances whereas relationsare supported by at least one link of the same relation label.In addition, such a link should (i) start at an object from theimage of the source class of the relation occurrence and (ii)arrive at an object from the target class image.

Technically speaking, in order to construct the graph mor-phism, the two sequences, s and S, are traversed in parallel.For each class c of the sequence S, its image in s isdetermined, i.e., the subsequence of yet unprocessed objectsthat are members of c. It is noteworthy that this sequenceis both contiguous and maximal. Moreover, its determinationis a tentative process as it also depends on the matchingbetween relations. This means that several attempts on several

places in s may be necessary before the correct image ofc is discovered, or, alternatively the absence of such imageis established in which case the entire matching fails. Moreprecisely, once a candidate image subsequence is determined,it is further checked for relation match, i.e., it is looked atwhether all the incoming relations for c have an equivalentamong the relations of s.rel that are incoming for at leastone object from the target image subsequence. To that end, amarking mechanism is used to delimit the current candidatesubsequence. Failure to find the match of even a single relationfor c invalidates the candidate image and the search for a newcandidate subsequence must go further in the sequence s.

Algorithme 1 Instantiation procedure1: procedure INST(S : pattern; s: extended object sequence)2: i← 1; j ← 1;3: Reset(Images);4: while i ≤ |S| do5: while s(j) 6∈ [S(i)]Ω and j ≤ |s| do6: j + +;7: end while8: if j > |s| then9: return false

10: end if11: while s(j) ∈ [S(i)]Ω and j ≤ |s| do12: Images[i]← Images[i] ∪ j;13: j + +;14: end while15: concord← true;16: for all r(k, i) in S.rel do17: if @r′(m,n) ∈ s.rel s. t. r′ vΩ r, m ∈

Images[k], n ∈ Images[i] then18: concord← false;19: Images[i]← ∅20: end if21: end for22: if concord then23: i+ +;24: end if25: end while26: return true27: end procedure

The pseudo-code of the above test is provided in Algo-rithm 1 which is a greedy procedure. Moreover, Algorithm1 can be easily adapted for testing subsumption betweenpatterns, by essentially replacing the object sequence by a classone and the ontology membership test for (object, class) pairsby a subsumption test for class pairs vΩ. The instanciationtest is a key building block of our frequent pattern miningmethod that is described in the following paragraph.

B. The xPMiner algorithm

Our mining algorithm, called xPMiner, is a top-down level-wise miner similar in spirit to Apriori. This means that atthe (k + 1) − th level, xPMiner uses the frequent patterns

generated at level k to compose candidates and then checkstheir frequency in order to establish which ones are to be kept.The pseudo-code of xPMiner is given in Algorithm 2.

Algorithme 2 xPMiner1: Input:2: Ω; . Domain ontology3: D; . Set of object sequences

4: Output:5: Γs

Ω; . Set of frequent patterns

6: Initialization:7: DΩo

← ISEQTRANS(D); . Set of extended objectsequences

8: ΓΩ1 ← 〈〈c〉, ∅〉|c ∈Max(TC);9: ΓΩ1 ← CANDTEST(ΓΩ1 );

10: Method:11: for k = 2; ΓΩk−1 6= ∅; k + + do12: ΓΩk

← ∅;13: for all S ∈ ΓΩk−1 do14: ΓΩk

← ΓΩk∪ ADDCLS(S) ∪ SPLCLS(S) ∪

ADDREL(S) ∪ SPLREL(S)15: end for16: ΓΩk

← CANDTEST(ΓΩk);

17: end for18: return (∪kΓΩk

);

The algorithm works on an input made of an ontology Ωand a set of object sequences D. At its initialization step, ittransforms the sequences into extended object sequences, i.e.,it adds all the links from the ontology that connect two objectsin the same order source target as they appear in the sequence(primitive ISEQTRANS(D)3). Moreover, the structure carryingthe candidate patterns is initialized with all patterns of rankone, i.e., the patterns made of a single top most concept.These candidates are then evaluated for frequency in the databy carrying out instanciation tests for each entry in DΩo

(primitive CANDTEST() described in Algorithm 3).At each subsequent iteration step, the algorithm performs

two main operations. First, using the four auxiliary functionsrepresenting the four canonical operations, the candidates ofrank k + 1 are generated from those of rank k. The fouroperations are class append (Algorithm 4), relation addition(Algorithm 5), class specialization (Algorithm 6) and relationspecialization (Algorithm 7). Then, the frequency of the candi-dates is evaluated by confronting those to the dataset by meansof the CANDTEST().

With such a procedure, a question must be asked aboutits capacity to retrieve exactly the target pattern set, that isall frequent patterns with respect to minsupp. Although bothconsistency (no infrequent is retrieved) and completeness (allfrequent are retrieved) are at stake, due to the specific nature of

3Pseudo-code skipped since straightforward.

Algorithme 3 Candidate test1: procedure CANDTEST(P : set of patterns)2: ΓΩtemp

← ∅;3: DΩo

: set of extended object sequences;4: for all S ∈ P do5: count← 0;6: for all s ∈ DΩo do7: if Inst(s,S) then8: count+ +;9: end if

10: end for11: if count ≥ minsup then12: ΓΩtemp ← ΓΩtemp ∪ S;13: end if14: end for15: return ΓΩtemp

16: end procedure

the computation, only completeness needs to be examined. Thefollowing Proposition 4.1 states the completeness of xPMiner.

Algorithme 4 Generate patterns by adding a class1: procedure ADDCLS(S : pattern)2: k ← |S|;3: Φ← ∅;4: for all c ∈Max(TC − (S(k))) do5: Φ← Φ ∪ 〈S.str&c,S.rel〉;6: end for7: return Φ8: end procedure

Algorithme 5 Generate patterns by adding a relation1: procedure ADDREL(S : pattern)2: k ← |S|;3: Φ← ∅;4: for all r ∈Max(TR) s. t. S(k) vΩ ran(r) do5: for all i s.t. S(k) vΩ ran(r,S(i)) do6: if @r′(i, k) ∈ S.rel s.t. r′ vΩ r then7: Φ← Φ ∪ 〈S.str,S.rel ∪ r(i, k)〉;8: end if9: end for

10: end for11: return Φ12: end procedure

Proposition 4.1: For an arbitrary natural number k, if pat-terns of rank k + 1 exist, then any of them can be generatedby at least one pattern of rank k by means of the canonicaloperations.

A corollary of the above proposition is that all the patternsmay be generated by an unconstrained level-wise search(induction on k). Frequent patterns are contiguous subset ofthe space (an order filter in technical terms) so for any frequent

Algorithme 6 Generate patterns by specializing a class1: procedure SPLCLS(S : pattern)2: k ← |S|;3: Φ← ∅;4: for all c ∈ Max(succΩ(S(k))) s.t. hc

Ω(c) =hc

Ω(S(j)) + 1 do5: if (c 6= S(j − 1)) and (c 6= S(j + 1)) then6: Φ← Φ ∪ 〈S.str[(k)←c],S.rel〉;7: end if8: end for9: return Φ

10: end procedure

Algorithme 7 Generate patterns by specializing a relation1: procedure SPLREL(S : pattern)2: k ← |S|;3: Φ← ∅;4: for all r(i, k) ∈ S.rel do5: for all r′ ∈Max(succΩ(r)) s.t. hr

Ω(r′) = hrΩ(r)+

1 do6: Φ ← Φ ∪ 〈S.str,S.rel + r′(i, k) −

r(i, k)〉;7: end for8: end for9: return Φ

10: end procedure

pattern, all the super-patterns are also frequent. Hence there isalways a frequent pattern of rank k above any of rank k + 1that will enable the generation of the former.

An example of the application of our method is providedin the next section which is dedicated to its application asrecommendation tool.

V. RECOMMENDATION STRATEGY

A recent state-of-art for recommendation systems is pre-sented by G. Adomavicius and A. Tuzhilin in [26]. Accordingto the authors, depending on the portion of domain knowledgeused to represent data, such systems can be split into twocategories: item-based and concept-based recommendationsystems. In item-based recommendation systems, items (ob-jects) frequently accessed together by a user or a group ofusers are recommended when at least one of them is selectedby a new user. For concept-based recommendation systems,access frequencies are sought on the set of domain concepts atdifferent levels of abstraction, and then, instances of conceptsappearing together are recommended.

A. Concept-based versus item-based recommendation

Generally speaking, concept-based systems offer more flex-ibility compared to item-based systems due to the varyingabstraction levels of concepts in concept hierarchies. However,both categories of systems present specific limitations thatdecrease their overall precision rates. Precision is the capacity

of a recommendation system to suggest objects that highlyreflect the behavior which is the source of the patterns used inthe current recommendation process. For instance, item-basedsystems suffer from the “new item problem” which basicallymeans object not yet rated/visited are difficult to recommendwith precision. Concept-based systems, in turn, suffer fromlow precision problem due to the difficulty to choose amongthe set of concept instances, those which are more likely tointerest the user.

To illustrate the lack of precision of concept-based systems,we present a realistic and simple example. The scenario isas follows: we consider a set of three users such thateach of them has travelled to a city and visited a museumthere. The pairs of concerned objects are respectively:〈Montreal, The_Montreal_Museum_of_Fine_Arts〉,〈Paris, Louvre〉, 〈London,British_Museum〉. Now,suppose there is a fourth user which has just visited the cityNew_Y ork. The question is to find out the items whichare most likely to be relevant to this user according to thenavigation behavior of the first three users. We observe that thefirst three users have explored museums located in the city theyvisited. Intuitively, the fourth user should get a suggestionto visit museums located in New_Y ork city, such asAmerican_Folk_Art, American_Numismatic_Society,and Bronx_Museum_of_the_Arts. However, suchsuggestion and reasoning are not evident neither for item-based systems nor for concept-based systems. In fact, foritem-based systems, the object New_Y ork was not yetvisited and thus no recommendation can be made. In the caseof concept-based systems, the behavior of the three userscan be summarized by the pattern 〈City,Museum〉. Thispattern may suggest all instances of concept Museum in theontology travel whether they are situated in New_Y ork cityor not.

B. Our approach

The above example shows the limitations of recommenda-tion systems based either on items or domain concepts. Toovercome such limitations, one should consider patterns athigh level of abstraction and in the same time keep a highprecision. It is noticeable that concepts of high level of ab-straction as well as relationships between concepts (and hencetheir instances) bring added value to recommendation systems.In this section, we propose a third category of recommendationsystems based on both concepts and relations belonging to adomain ontology. Patterns composed of concepts and relationsof different levels of abstraction are discovered using the algo-rithm xPMiner as shown in Section IV-B. For example, in thesituation presented above, user behavior will be representedwith the pattern 〈City,Museum〉, hasMuseum(1, 2)〉 andfor the user consulting New_Y ork object only instances ofconcept Museum involved in a relation link hasMuseumstarting at the New_Y ork object in ontology Travel will besuggested. Hence, only museums situated in New_Y ork willbe recommended to the new user. We believe that the proposedapproach combines the precision of item-based systems and

the flexibility of concept-based systems.In the remainder of the section, we present a recommen-

dation algorithm SemRAr based on semantically enhancedsequential patterns. Its mostly original feature is that objectsare recommended according to the way they are connectedwith other objects in the ontology. More specifically, the algo-rithm considers the sequence of objects already visited by theuser (the user session) and tries to match it against the knownpatterns of user behavior. Whenever a prefix of a patternmatches completely the user session its remaining suffix isused to determine the class of the objects to recommend, i.e.,the target class, as well as all the links from the session objectsto the objects of the target class that might hold.

The pseudo-code of SemRAr is provided in Algorithm 8.

Algorithme 8 SemRAr1: Input:2: Ω . Domain ontology3: Γs

Ω . Frequent patterns generated by xPMiner4: OI . List of last visited objects

5: Output:6: Ho . Hashtable of objects to suggest

7: Initialization:8: Ho ← ∅

9: Method:

10: s← ISEQTRANS(Oi);11: for all S ∈ Γs

Ω do12: i ← COVERS(S, s, Images);13: if i < |S.str| then14: Ho ← Ho∪ GETINSTS(Ω,S, OI , i, Images);15: end if16: end for17: return Ho;

To find out the objects to be recommended, algorithmSemRAr seeks for a subset of frequent patterns that coverthe sequence of last visited objects. The pattern of an objectsequence by a pattern is very similar to the instanciationbetween extended object sequences and patterns. Indeed, foreach object sequence O in Oω, there is an associated extendedobject sequence s that may be tested for instanciation. Thus,a pattern S covers O if and only if it summarizes s or,alternatively, s is an instance of S.

Definition 5.1: Given S ∈ ΓΩ, Oi in Oω and s the asso-ciated extended object sequence, S covers Oi (S Oi) iffs ∈ [S]ΓΩ .

For instance, the object sequence O1 =〈Surfline, Paris, Louvre〉 is covered by the pattern S1 =〈〈City,Museum,LuxuryHotel〉, hasMuseum(1, 2),hasLuxuryHotel(1, 3)〉.

Given the above observations, it is not surprising that thealgorithm that computes coverage follows the same control

structure as Algorithm 1. The only noticeable difference isthat instead of a Boolean output, the new algorithm yields aninteger number which is the index of the first class in S.strthat was not matched within s. Moreover, as a byproduct, thealgorithm also exports the Images structure which embedsthe mapping between S and s. The latter is necessary for thecomputation of the set of suggested objects.

Algorithme 9 Coverage computation1: procedure COVERS(S : pattern; s: extended object se-

quence)2: i← 1; j ← 1;3: Reset(Images);4: while i ≤ |S| do5: while s(j) 6∈ [S(i)]Ω and j ≤ |s| do6: j + +;7: end while8: if j > |s| then9: return i

10: end if11: while s(j) ∈ [S(i)]Ω and j ≤ |s| do12: Images[i]← Images[i] ∪ j;13: j + +;14: end while15: concord← true;16: for all r(k, i) in S.rel do17: if @r′(m,n) ∈ s.rel s. t. r′ vΩ r, m ∈

Images[k], n ∈ Images[i] then18: concord← false;19: Images[i]← ∅20: end if21: end for22: if concord then23: i+ +;24: end if25: end while26: return i27: end procedure

Once a pattern S is partially aligned with the sequence s,the resulting mapping may be used in the computation of thetarget object set to recommend. To that end, all the instancesof the first class from S that was not matched to s, say c,are considered as candidates. This corresponds to the intuitionthat the pattern represents a scenario which is unfolding andthat the current step corresponds to the class that has yet tobe “realized”.

Among all instances of c, only those having at least onelink to objects from s.str in concordance with the relationsfrom S.rel are kept. The underlying test uses the partialmapping between both structure (Images established duringinstanciation test) to check the concordance condition. Thetest may have two outcomes: either there are some objectsthat satisfy the conditions (in the extreme case all objects do,e.g., as no relations exist for c in S.rel) or there no suchobjects. In the later case, the pattern does not contribute to

the recommendation.The test is described in Algorithm 10.

Algorithme 10 GETINSTS(Ω: the ontology, S: a pattern, O:the object sequence, i : index of the first unmatched concept,Images : the partial mapping)

1: Ot ← ∅;2: for all o ∈ ([S(i)]Ω - asSet(O) ) do3: for all r(k, i) ∈ S.rel do4: if ∃j s. t. j ∈ Images[k] and (s(j), o) ∈ [r]Ω then5: Ot ← Ot ∪ o;6: end if7: end for8: end for9: return Ot

For example, if we consider the sequences O1

and S1 given above as being the list of lastvisited objects and the test pattern, respectively,then the set of objects to be recommended will be:Le_Petit_Manoir,Royal_Garden, Le_Walt, Clarion_St.James which are instances of hasLuxuryHotelinvolved in the relational link hasLuxuryHotel with theinstance Paris.

VI. EXAMPLE OF APPLICATION

In this section, we illustrates the way xPMiner operatesand show how the resulting semantically enhanced sequentialpatterns can be used in a recommendation process.

A. Patterns extracted with xPMiner

As public domain datasets for web usage of ontology-powered sites are not availabble yet, we used in the vali-dation of our method a synthetic set of object sequences.The underlying ontology, Travel, is a public domain. TableII shows a set of eleven object sequences, while Table IIIshows the correspondence between the individual objects andtheir membership classes in Travel. Moreover, the minimum(absolute) support is set to 6 (minsup = 6).

Id Object sequence1 〈France, Grenoble, Paris, Louvre〉2 〈Morocco, Rabat, Sofitel_Diwan_Rabat〉3 〈Coonabarabran, Warrumbungle_National_Park, GUMIN

_GUMIN_HOMESTEAD〉4 〈Australia, Cairns, Clifton_Beach, Cape_Y ork_Safari,

Montra_Trilogy〉5 〈Canada, Forillon, Marche_2Tiers, The_2Campers〉6 〈Italy, Roma, Mausoleum_of_Augustus, Grand_Hotel_

Plaza〉7 〈Bulgaria, Sophia, The_National_Gallery, Hotel_Anel〉8 〈Algeria, T izi, Djurdjura, Hotel_El_Arz, Tala_Guilef〉9 〈Paris, Rodin, Royal_Garden〉10 〈Cairns, Gazelle_Tanzania_Safari, Cifton_Beach〉11 〈Teritoires_Nord_Ouest, Wood_Buffalo, Little_Buffalo〉

TABLE IISEQUENCES FORMED OF OBJECTS BELONGING TO THE TRAVEL

ONTOLOGY

In the following, for the sake of simplicity, instead ofpresenting all the patterns at different levels, we follow theevolution of a single pattern starting from the initializationphase.

Class InstancesCountry Australia, Algeria, Bulgaria, Canada, Italy,

MoroccoCapital Paris, Rabat, Roma, Sophia

City Grenoble, CairnsTown Coonabarabran, T izi

Museum Louvre, Mausoleum_of_Augustus, Rodin,The_National_Gallery

Hotel Montra_Trilogy, Hotel_El_ArzLuxuryHotel Sofitel_Diwan_Rabat, Grand_Hotel_Plaza,

Royal_Garden, HotelAnelNationalPark Warrumbungle_National_Park, Forillon,

Wood_Buffalo, DjurjuraCampground GUMIN_GUMIN_HOMESTEAD, Little

_Buffalo, The_2CampersBeach Clifton_BeachSafari Gazelle_Tanzania_Safari, Cape_Y ork

_SafariHiking Marche_2TiersSkiing Tala_Guilef

TABLE IIICLASS/INSTANCE MAPPING

At the first step, the candidate patterns have onlyone concept: S1 = 〈〈Accommodation〉, 〉 , S2 =〈〈AccommodationRating〉, 〉, S3 = 〈〈Activity〉, 〉,S4 = 〈〈Contact〉, 〉, S5 = 〈〈Destination〉, 〉. Then, thecandidate test procedure (Algorithm 1) calculates for eachcandidate the number of its instances in the object sequences.For example, S1 fails the test with the first and the tenthsequences but succeeds the instantiation with all the nineremaining sequences. The support of the five patterns ispresented in Table IV. From this table, we observe that S2

and S4 have a null support. In fact, it is easy to see thatneither AccommodationRating in S2 nor Contact in S4

have instances in the set of object sequences. At this point,the anti-monotony property of non-frequency ensures that allpatterns that can be generated from a non-frequent pattern arenon-frequent. All other candidates have a frequency greaterthan the fixed minsup 6. Thus, S1, S3, and S5 are marked asfrequent and will be considered a seed set for the next pass.

Pattern SupportS1 9S2 0S3 8S4 0S5 11

TABLE IVPATTERNS OF RANK 1 AND THEIR ASSOCIATED SUPPORT

Each 1 − frequent pattern is tentatively extendedby means of canonical operations thus yieldingthe set of 2 − candidates. For instance, concept

specialization on S5 produces the following set of2 − candidates: 〈〈BackpakerDestination〉, 〉,〈〈Beach〉, 〉, 〈〈BudgetHotelDestination〉, 〉,〈〈FamillyDestination〉, 〉, 〈〈QuietDestination〉, 〉,〈〈RetireeDestination〉, 〉, 〈〈RuralArea〉, 〉 and〈〈UrbanArea〉, 〉. Alternatively, by concept insertion,the following set of 2 − candidates is obtained:〈〈Destination, Accommodation〉, 〉, 〈〈Destination,AccommodationRating〉, 〉, 〈〈Destination, Contact〉,〉 and 〈〈Destination, Activity〉, 〉. The list of all2− candidates is presented in Table V.

Pattern Support〈〈BackpakerDestination〉, 〉 0〈〈Beach〉, 〉 2〈〈BudgetHotelDestination〉, 〉 0〈〈FamillyDestination〉, 〉 0〈〈QuietDestination〉, 〉 0〈〈RetireeDestination〉, 〉 0〈〈RuralArea〉, 〉 4〈〈UrbanArea〉, 〉 9〈〈Destination, Accommodation〉, 〉 9〈〈Destination, AccommodationRating〉, 〉 0〈〈Destination, Contact〉, 〉 0〈〈Destination, Activity〉, 〉 8

TABLE VCANDIDATES OF RANK 2 AND THEIR ASSOCIATED SUPPORT

(2-CANDIDATES)

It is noteworthy that operations involving relations arenot applicable until the step two of the process. Now, the2 − candidates are filtered through the instantiation testin order to determine 2 − frequent patterns. We observethat only patterns 〈〈Destination, Accommodation〉, 〉,〈〈UrbanArea〉, 〉 and 〈〈Destination, Activity〉, 〉 arefrequent (see Table VI). These patterns, having rank 2, arethen used to generate 3− frequent patterns.

Pattern Support〈〈UrbanArea〉, 〉 9〈〈Destination, Accommodation〉, 〉 9〈〈Destination, Activity〉, 〉 8

TABLE VIFREQUENT PATTERNS OF RANK 2 AND THEIR ASSOCIATED SUPPORT

(2-FREQUENTS)

As an illustration, consider the extension of the pattern〈〈Destination,Activity〉, 〉. Candidates generated byconcept specialization (on Activity) are: 〈〈Destination,Adventure〉, 〉, 〈〈Destination, Relaxation〉, 〉,〈〈Destination, Sightseeing〉, 〉, 〈〈Destination, Sports〉,〉, 〈〈Activity, Accommodation〉, 〉. Observe that severalconcept insertions generate the patterns: 〈〈Destination,Activity, AccommodationRating〉, 〉, 〈〈Destination,Activity, Contact〉, 〉, and 〈〈Destination, Activity,Destination〉, 〉. The pattern 〈〈Destination, Activity〉,hasActivity(1, 2)〉 is the only one obtained by relation

insertion. Table VII provides the frequency of the candidatesat the 3rd step.

Pattern Support〈〈City〉, 〉 8〈〈Destination, Accommodation〉, 9hasAccommodation(1, 2)〉〈〈UrbanArea, Activity〉, 〉 9〈〈UrbanArea, Accommodation〉, 〉 8〈〈Destination, Activity〉, hasActivity(1, 2)〉 7〈〈Destination, Activity, Accommodation〉, 〉 6

TABLE VIIFREQUENT PATTERNS OF RANK 3 AND THEIR ASSOCIATED SUPPORT

At the fourth iteration, after the generation and the subse-quent instantiation test a set of eight 4− frequent patterns isobtained (See Table VIII).

Pattern Support〈〈City, Activity〉, 〉 6〈〈UrbanArea, Accommodation〉, 8hasAccommodation(1, 2)〉〈〈UrbanArea, Hotel〉, 〉 6〈〈UrbanArea, Soghtseeing〉, 〉 6〈〈UrbanArea, Activity〉, hasActivity(1, 2)〉 7〈〈Destination, Hotel〉, hasAccommodation(1, 2)〉 6〈〈Destination, Activity, Accommodation〉, 6hasAccommodation(1, 3)〉〈〈Destination, Activity, Accommodation〉, 6hasActivity(1, 2)〉

TABLE VIIIFREQUENT PATTERNS OF RANK 4 AND THEIR ASSOCIATED SUPPORT

Table IX and Table X present frequent patterns of range 5and 6 respectively. Table XI presents the list of 7−candidateswith their associated support whereas the seventh step is thefinal one.

Pattern Support〈〈UrbanArea, Hotel〉, hasAccommodation(1, 2)〉 6〈〈UrbanArea, Soghtseeing〉, hasActivity(1, 2)〉 6〈〈Destination, Hotel〉, hasHotel(1, 2)〉 6〈〈Destination, Activity, Accommodation〉,hasActivity(1, 2), hasAccommodation(1, 3)〉 6

TABLE IXFREQUENT PATTERNS OF RANK 5 AND THEIR ASSOCIATED SUPPORT

Actually, the process will stop for one of the followingreasons: (i) there is no more frequent patterns, (ii) no newcandidate can be generated. In our case, the mining processstops at the 7-th iteration because all 7− candidates are notfrequent enough.

B. Instance recommendation

In this subsection we highlight the advantages of using thediscovered patterns to suggest/recommend objects accordingto the last visited ones. The basis of recommendation systemsis to apply data analysis techniques to generate a list of

Pattern Support〈〈UrbanArea, Hotel〉, hasHotel(1, 2)〉 6

〈〈UrbanArea, Sightseeing〉, hasSightseeing(1, 2)〉 6

TABLE XFREQUENT PATTERNS OF RANK 6 AND THEIR ASSOCIATED SUPPORT

Pattern Support〈〈UrbanArea, LuxuryHotel〉, hasHotel(1, 2)〉 4〈〈UrbanArea, Hotel, AccommodationRating〉, 0hasHotel(1, 2)〉〈〈UrbanArea, Hotel, Contact〉, hasHotel(1, 2)〉 0〈〈UrbanArea, Hotel, Destination〉, hasHotel(1, 2)〉 0〈〈UrbanArea, Hotel, Accommodation〉, hasHotel 0(1, 2)〉〈〈UrbanArea, Hotel, Activity〉, hasHotel(1, 2)〉 1〈〈UrbanArea, Museum〉, hasSightseeing(1, 2)〉 4〈〈UrbanArea, Safari〉, hasSightseeing(1, 2)〉 1〈〈UrbanArea, Sightseeing, AccommodationRating〉, 0hasSightseeing(1, 2)〉〈〈UrbanArea, Sightseeing, Contact〉, 0hasSightseeing(1, 2)〉〈〈UrbanArea, Sightseeing, Destination〉, 0hasSightseeing(1, 2)〉〈〈UrbanArea, Sightseeing, Accommodation〉, 1hasSightseeing(1, 2)〉〈〈UrbanArea, Sightseeing, Activity〉, 4hasSightseeing(1, 2)〉

TABLE XICANDIDATE PATTERNS OF RANK 7 AND THEIR ASSOCIATED SUPPORT

recommended content products for each user according tohis/her past behavior. In our approach, the past behavior isrepresented by means of sequential patterns that combineconceptual and relational information.

In the following, we use our example to illustrate theitem-based, concept-based, and concept and relation-basedrecommendation approaches. In the first approach, items (ob-jects) frequently accessed together by many users are rec-ommended when at least one of them is selected by a newuser. From the object sequences of Table II the only objectsthat appear more than once are Paris and Cairns with afrequency less than minsup. In the concept-based approach,access frequencies are computed on the set of domain con-cepts at different levels of abstraction, and then, instances ofconcepts appearing together are recommended. The patternsbased on concepts are identical to those mined in Subsec-tion VI-A except for the fact that they do not cover ontol-ogy relations. For instance, the patterns 〈UrbanArea,Hotel〉and 〈UrbanArea, Sightseeing〉 correspond respectivelyto the patterns 〈〈UrbanArea,Hotel〉, hasHotel(1, 2)〉,and 〈〈UrbanArea, Sightseeing〉, hasSightseeing(1, 2)〉.In our approach, concepts and inter-concept relationships areintegrated into the pattern mining and hence the recommen-dation processes as shown in Subsection VI-A.

Because of the essential role of object co-occurrences initem-based recommendation systems, an object will not besuggested if it is not frequently visited. For example, in TableXII there is no object to be recommended for a sequence of

accessed objects made of Montreal and Central_Florida.In fact, these objects do not appear at all in the initialdata sequences. In contrast, a concept-based recommendationstrategy would find some objects to recommend. These are theinstances of concepts Hotel and Sightseeing. However, indoing that, no distinction is made between the hotels in Mon-treal and elsewhere. The same holds for Sightseeing whoseinstances further belong to either Safari or Museum, andwill be recommended regardless of whether they are locatedin the Montreal area or not. In fact, even if it is an immediateassociation for human minds, there is no information in thepattern structure that could help deduce the geographical linksbetween urban areas, hotels and sightseeing. As a result, therecommendation quality would suffer.

With our ontology-based approach, the information thatavoids inconsistent recommendations of this sort is pro-vided by inter-concept relations. For instance, to helpusers selecting Montreal, our approach would use theabove patterns 〈〈UrbanArea,Hotel〉, hasHotel(1, 2)〉and 〈〈UrbanArea, Sightseeing〉, hasSightseeing(1, 2)〉to recommend The_Montreal_Museum_of_Fine_Artsand Pointe_à_Callière, both instances of Museum,and Le_Saint_Malo and Pierre_du_Calvet, instancesof Hotel. All of them are connected to Montrealwithin Travel. For users selecting Florida_Beach, no6 − Frequent (see Table X) patterns match the choice.However, if patterns from Table IX are considered,then the pattern 〈〈Destination,Activity, Accommodation〉,hasActivity(1, 2), hasAccommodation(1, 3)〉 suggests aninstance of Surf — subclass Activity — that is associatedto Florida_beach by a hasSurf link. The other pattern,〈〈Destination,Hotel〉, hasHotel(1, 2)〉, however, fails.

Another advantage of using the proposed patternsis to spot possibly incoherent data in the miningprocess. Such incoherent data remain out of thereach of purely concept-based pattern discovery.For example, if we consider the object sequence〈Cairns,Gazelle_Tanzania_Safari, Cifton_Beach〉in concept-based pattern discovery, this sequence will beconsidered as element of the interpretation set of the pattern〈City, Safari〉. However, it seems logical to rule outsuch a sequence because there is no apparent link betweenGazelle_Tanzania_Safari and Cairns (but this hypothesismight be wrong).

VII. DISCUSSION AND FUTURE WORK

In this paper, we have investigated the problem of miningfrequent patterns from sequences of objects described bymeans of a domain ontology. We proposed a language todescribe the target patterns with its syntax and semantics. Thestructure of the underlying pattern space was studied and amining procedure to explore it in a level-wise manner wasdefined. The design is based on a careful analysis of theApriori framework, and the procedure is complete in the sensethat it discovers all the frequent patterns.

Last object visitedRecommendations

Classic approaches Ontology-basedItem-based

Generalized xPMiner

Montreal All direct and indirect instances The_Montreal_Museum_of_Fine_Arts,

of classes Hotel and Sightseeing, Pointe_à_Callièrefrom the ontology Travel Le Saint_Malo, Pierre

_du_CalvetCentral_Florida All direct and indirect instances The_Westin_Diplomat

_Resort_and_Spaof classes Activity and SurflineAccommodation from theontology Travel

TABLE XIIOBJECTS SUGGESTED ACCORDING TO DIFFERENT RECOMMENDATION STRATEGIES

Devising an effective mining procedure for the new familyof patterns is only the first step of our study. Indeed, as itis well known that level-wise algorithms may perform poorlyon some datasets. The next goal will be to improve successrate by designing algorithms that look in the pattern space ina depth-first way. Another topic to tackle is the definition ofreduced representations of the frequent pattern family such asclosed or maximal patterns.

Recently, the xPMiner has been implemented as a plug-infor the Protégé4 ontology manipulation platform. Since theultimate goal of our study is recommendation, the miningcomponent will be integrated into a complete architecturefor extracting and exploiting behavioral patterns (e.g., forpredicting next hit/choice/move operations).

REFERENCES

[1] R. Agrawal and R. Srikant, “Fast algorithms for mining associationrules,” In Proc. 20th International Conference on Very Large Data Bases,VLDB94, 1994.

[2] ——, “Mining sequential patterns,” Proceedings of the Eleventh Inter-national Conference on Data Engineering, IEEE Computer Society., pp.3–14, 1995.

[3] R. Srikant and R. Agrawal, “Mining sequential patterns: Generaliza-tions and performance improvements,” Proc. 5th Int. Conf. ExtendingDatabase Technology, EDBT, Avigon, France, vol. 1057, pp. 3–17, 1996.

[4] M. Adda, P. Valtchev, R. Missaoui, and C. Djeraba, “On the discovery ofsemantically enhanced sequential patterns,” IEEE Proceedings. FourthInternational Conference on Machine Learning and Applications, pp.383–390, December 2005.

[5] H. Mannila, H. Toivonen, and A. Verkamo, “Discovery of frequentepisodes in event sequences,” Data Mining Knowledge Discovery jour-nal, vol. 1, no. 3, pp. 259–289, July 1997.

[6] H. Pinto, “Multi-dimensional sequential pattern mining,” M.Sc. thesis,Computing Science, Simon Fraser University, April 2001.

[7] H. Pinto, J. Han, J. Pei, K. Wang, Q. Chen, and U. Dayal, “Multi-dimensional sequential pattern mining,” Proc. Int. Conf. on Informationand Knowledge Management (CIKM’01), Atlanta, GA, November 2001.

[8] C. Bettini, X. Wang, S. Jajodia, and L. Jia-Ling, “Discovering tempo-ral relationships with multiple granularities in time sequences,” IEEETransations on Knowledge and Data Engineering, vol. 10, no. 2, 1998.

[9] W. Wang, J. Yang, and P. S. Yu, “Meta-patterns: Revealing hidden peri-odic patterns,” Proceedings of the 2001 IEEE International Conferenceon Data Mining, 2001.

[10] J. Yang, W. Wang, and P. S. Yu, “Discovering high order periodicpatterns,” Discovering High Order Periodic Patterns. Knowledge andInformation Systems, Publisher: Springer-Verlag London Ltd, pp. 243 –268, May 2004.

4http://protege.stanford.edu

[11] M. Adda, R. Missaoui, P. Valtchev, and C. Djeraba, “Recommendationstrategy based on relation rule mining,” IJCAI-2005 Workshop onIntelligent Techniques for Web Personalization, 2005.

[12] M. Kuramochi and G. Karypis, “An efficient algorithm for discover-ing frequent subgraphs,” IEEE Transactions on Knowledge and DataEngineering, vol. 16, no. 9, pp. 1038–1051, 2004.

[13] A. Inokuchi, T. Washio, and H. Motoda, “An apriori-based algorithm formining frequent substructures from graph data,” PKDD ’00: Proceedingsof the 4th European Conference on Principles of Data Mining andKnowledge Discovery, pp. 13–23, 2000.

[14] M. J. Zaki, “Efficiently mining frequent trees in a forest,” KDD ’02:Proceedings of the eighth ACM SIGKDD international conference onKnowledge discovery and data mining, Edmonton, Alberta, Canada, pp.71–80, 2002.

[15] Y. Chi, Y. Yang, and R. R. Muntz, “Indexing and mining free trees,”ICDM2003 : Proceedings of the International Conference on DataMining, 2003.

[16] X. Yan and J. Han, “gspan: Graph-based substructure pattern mining,”ICDM ’02: Proceedings of the 2002 IEEE International Conference onData Mining (ICDM’02), p. 721, 2002.

[17] M. Cohen and E. Gudes, “Diagonally subgraphs pattern mining,” Pro-ceedings of the 9th ACM SIGMOD Workshop on Research issues in datamining and knowledge discovery, 2004.

[18] X. Yan and J. Han, “Closegraph: Mining closed frequent graph patterns,”KDD’03 : Proceedings of the ninth ACM SIGKDD international con-ference on Knowledge discovery and data mining, pp. 286–295, 2003.

[19] Y. Chi, Y. Yang, and R. R. Muntz, “Cmtreeminer: Mining both closedand maximal frequent subtrees,” PAKDD’04 : The Eighth Pacific-AsiaConference on Knowledge Discovery and Data Mining, 2004.

[20] J. Huan, W. Wang, J. Prins, and J. Yang, “Spin: Mining maximal frequentsubgraphs from graph databases,” in Proceedings of the 10th ACMSIGKDD International Conference on Knowledge Discovery and DataMining, pp. 581–586, 2004.

[21] T. Berners-Lee, J. Hendler, and O. Lassila, “The semantic web,” Scien-tific American, May 2001.

[22] D. J. Commette, “Daml+oil language,”http://www.daml.org/2001/03/daml+oil-index.html, March 2001.

[23] D. L. McGuinness and F. van Harmelen, “Owl web ontology languageoverview,” http://www.w3.org/TR/owl-features/, 2004.

[24] F. Baader, D. Calvanese, D. McGuinness, D. Nardi, and P. Patel-Schneider, “The description logic handbook : Theory, implementationand applications,” Cambridge University Press, p. 574, 2003.

[25] G. W. Mineau, M. Bernard, and J. F. Sowa, “Conceptual graphs forknowledge representation,” Lecture Notes in Artificial Intelligence 699,Springer-Verlag, Berlin, 1993.

[26] G. Adomavicius and A. Tuzhilin, “Toward the next generation ofrecommender systems: a survey of the state-of-the-art and possibleextensions,” IEEE Transactions on Knowledge and Data Engineering,vol. 17, pp. 734–749, June 2005.