Dealing with incomplete information in a fuzzy linguistic recommender system to disseminate information in university digital libraries

Knowledge-Based Systems 23 (2010) 32–39

Contents lists available at ScienceDirect

Knowledge-Based Systems

journal homepage: www.elsevier .com/locate /knosys

Dealing with incomplete information in a fuzzy linguistic recommender systemto disseminate information in university digital libraries

C. Porcel a,*, E. Herrera-Viedma b

a University of Jaén, Department of Computer Science, Jaén, Spainb University of Granada, Department of Computer Science and Artificial Intelligence, Granada, Spain

a r t i c l e i n f o

Article history:Available online 5 August 2009

Keywords:Recommender systemsFuzzy linguistic modelingUniversity digital librariesIncomplete fuzzy linguistic preferencerelation

0950-7051/$ - see front matter � 2009 Elsevier B.V. Adoi:10.1016/j.knosys.2009.07.007

* Corresponding author.E-mail addresses: [email protected] (C. Po

(E. Herrera-Viedma).

a b s t r a c t

As in the Web, the growing of information is the main problem of the academic digital libraries. Thus,similar tools could be applied in university digital libraries to facilitate the information access by the stu-dents and teachers. In [46] we presented a fuzzy linguistic recommender system to advice researchresources in university digital libraries. The problem of this system is that the user profiles are provideddirectly by the own users and the process for acquiring user preferences is quite difficult because itrequires too much user effort. In this paper we present a new fuzzy linguistic recommender system thatfacilitates the acquisition of the user preferences to characterize the user profiles. We allow users toprovide their preferences by means of incomplete fuzzy linguistic preference relation. We include toolsto manage incomplete information when the users express their preferences, and, in such a way, we showthat the acquisition of the user profiles is improved.

� 2009 Elsevier B.V. All rights reserved.

1. Introduction

Digital libraries are information collections that have associatedservices delivered to user communities using a variety of technol-ogies [8,15,48]. Therefore, digital libraries are the logical exten-sions of physical libraries in the electronic information society.These extensions amplify existing resources and services. As such,digital libraries offer new levels of access to broader audiences ofusers and new opportunities for the library. In practice, a digital li-brary makes its contents and services remotely accessible throughnetworks such as the Web or limited-access intranets [39,50].

As digital libraries become commonplace and as their contentsbecome more varied, the users expect more sophisticated servicesfrom them [8,15,48,50]. A service that is particularly important isthe selective dissemination of information or filtering, to help theusers to access interesting information for them. Users developinterest profiles and as new materials (books, papers, reports,and so on) are added to the collection, they are compared to theprofiles and relevant items are sent to the users [39].

Moreover, digital libraries have been applied in a lot of contextsbut in this paper we focus on an academic environment. UniversityDigital Libraries (UDL) provide information resources and servicesto students, faculty and staff in an environment that supportslearning, teaching and research [11].

ll rights reserved.

rcel), [email protected]

Recommender systems are becoming popular tools for reducinginformation overload and to improve the sales in e-commerce websites [7,9,35,40,49]. The use of this kind of systems allows to rec-ommend resources interesting for the users, at the same time thatthese resources are inserted into the system. In the UDL frame-work, recommender systems [7,49] can be used to help users(teachers, students and library staff) to find out and select theirinformation and knowledge sources [43].

Generally, in a recommender system the users’ informationpreferences can be used to define user profiles that are appliedas filters to streams of documents [7,47,49]. In [45,46] we devel-oped some recommender systems in an academic context. For in-stance, in [45] we proposed a fuzzy linguistic recommendersystem for a technology transfer office which helps researchersand environment companies allowing them to obtain informationautomatically about research resources (calls or projects) in theirinterest areas; in [46] we proposed a fuzzy linguistic recommendersystem to achieve major advances in the activities of UDL, whichrecommends researchers specialized resources and complemen-tary resources related with their respective research areas. Theproblem of both recommender systems is that users must directlyspecify their user profiles by providing their preferences on all top-ics of interest and it requires too much user effort.

In this paper, we focus on the idea of that a recommender sys-tem could be seen as a decision support system (DSS) [37,38,44],where the solution alternatives are the digital resources insertedinto the library, and the criteria to satisfy are the user profiles.The proper use of these recommendation systems is essential to

C. Porcel, E. Herrera-Viedma / Knowledge-Based Systems 23 (2010) 32–39 33

provide real personalized services, and it can substantially reduceinformation overload and increase user satisfaction. Therefore, ithas become an important area in information systems and decisionsupport research [37,38,44]. So, the activity of a recommender sys-tem can be seen as a group decision making (GDM) problem, so wecan adopt the typical representation formats used in GDM, as forexample, fuzzy preference relations [19,20,28,32,41]. This repre-sentation format presents a high expressivity and some interestingproperties that allow us to work easily. However, in real worldproblems it is common to find situations in which users are notable to provide all the preference values that are required, andthen, we have to deal with incomplete fuzzy preference relations[1–3,25,26,41].

The aim of this paper is to present a new fuzzy linguistic recom-mender defined in a UDL framework which overcomes the problemof user profile characterization observed in the recommender sys-tems defined in [45,46]. In order to improve the system perfor-mance, we propose an alternative way to obtain accurate anduseful knowledge about the user preferences. This new recom-mender system allows users to provide their preferences by meansof incomplete fuzzy linguistic preference relations [1], and in sucha way, we facilitate users the expression of their preferences and,consequently, the determination of user profiles process. The rec-ommender system is able to complete the incomplete preferencerelations using the tools proposed in [1,2,26]. Each user profile iscomposed of both user preferences on topics of interest and userpreferences on collaboration possibilities with other users. Then,the recommender system is able to recommend both research re-sources and collaboration possibilities to the users of a UDL. Asin [45,46] we define this recommender system in a multi-granularfuzzy linguistic context [10,12,22,27,32,42]. In such a way, weincorporate in the recommender system flexible tools to handlethe information by allowing to represent the different conceptsof the system with different linguistic label sets.

The rest of the paper is set out as follows. Section 2 presents thepreliminaries necessary to develop the proposed model. Section 3presents the new recommender system to the dissemination ofknowledge in a UDL. Section 4 reports the system evaluation andthe experimental results. Finally, our conclusions are pointed outin Section 5.

2. Preliminaries

2.1. Recommender systems

Recommender systems could be defined as systems that pro-duce individualized recommendations as output, or have the effectof guiding the user in a personalized way to interesting or usefulobjects in a large space of possible options [6].

It is a research area that offers tools for discriminating betweenrelevant and irrelevant information by providing personalizedassistance for continuous information accesses [43,49]. Automaticfiltering services differ from retrieval services [23,24,29–31] in thatin filtering the corpus changes continuously, the users have longtime information needs (described by means of user profiles) in-stead of introducing a query into the system, and their objectiveis to remove irrelevant data from incoming streams of data items[17,39,49]. A result from a recommender system is understood asa recommendation, an option worthy of consideration, while a re-sult from an information retrieval system is interpreted as a matchto the user’s query [7]. However both systems present some anal-ogies, and in this sense they could be considered a DSS [44]. In bothcases, the solution alternatives would be the documents to recom-mend or retrieve and the criteria to satisfy would be the user pro-files and user queries, respectively.

A variety of techniques have been proposed as the basis for rec-ommender systems [7,17,40,49]; all of these techniques have ben-efits and disadvantages. The use of an hybrid approach is proposedto smooth out the disadvantages of each one of them and to exploittheir benefits [5,13,16]. In these kind of systems, the users’ infor-mation preferences can be used to define user profiles that are ap-plied as filters to streams of documents. The construction ofaccurate profiles is a key task and the system’s success will dependon a large extent on the ability of the learned profiles to representthe user preferences [47].

The recommendation activity is followed by a relevance feed-back phase. Relevance feedback is a cyclic process whereby theusers feed back into the system decisions on the relevance of re-trieved documents and the system uses these evaluations to auto-matically update the user profiles [17,49].

2.2. The 2-tuple fuzzy linguistic approach

The fuzzy linguistic modeling (FLM) is a tool based on the con-cept of linguistic variable[52] which has given very good results formodeling qualitative information in many problems, e.g., in deci-sion making [20], quality evaluation [33,34], models of informationretrieval [23,24,29–31], political analysis [4], etc.

The 2-tuple FLM [21] is a continuous model of representationof information that allows to reduce the loss of information typi-cal of other fuzzy linguistic approaches (classical and ordinal[18,52]).

Let S ¼ fs0; . . . ; sgg be a linguistic term set with odd cardinality,where the mid term represents a indifference value and the rest ofthe terms are symmetrically related to it. We assume that thesemantics of labels is given by means of triangular membershipfunctions and consider all terms distributed on a scale on whicha total order is defined, si 6 sj () i 6 j. In this fuzzy linguisticcontext, if a symbolic method [18,20] aggregating linguistic infor-mation obtains a value b 2 ½0; g�, and b R f0; . . . ; gg; then anapproximation function is used to express the result in S. b is rep-resented by means of 2-tuples ðsi;aiÞ, si 2 S and ai 2 ½�:5; :5Þwheresi represents the linguistic label of the information, and ai is anumerical value expressing the value of the translation from theoriginal result b to the closest index label, i, in the linguistic termset ðsi 2 SÞ. This 2-tuple representation model defines a set oftransformation functions between numeric values and 2-tuplesDðbÞ ¼ ðsi;aÞ and D�1ðsi;aÞ ¼ b 2 ½0; g� [21].

The computational model is defined by presenting a negationoperator, comparison of 2-tuples and aggregation operators [21].Using functions D and D�1 that transform without loss of informa-tion numerical values into linguistic 2-tuples and viceversa, any ofthe existing aggregation operators can be easily extended fordealing with linguistic 2-tuples. Some examples are

Definition 1 (Arithmetic mean). Let x ¼ fðr1;a1Þ; . . . ; ðrn;anÞg be aset of linguistic 2-tuples, the 2-tuple arithmetic mean xe iscomputed as,

xe r1;a1ð Þ; . . . ; rn;anð Þ½ � ¼ DXn

i¼1

1n

D�1 ri;aið Þ !

¼ D1n

Xn

i¼1

bi

!: ð1Þ

Definition 2 (Weighted average operator). Let x ¼ fðr1;a1Þ; . . . ;

ðrn;anÞg be a set of linguistic 2-tuples and W ¼ fw1; . . . ; wng betheir associated weights. The 2-tuple weighted average xw is

xw r1;a1ð Þ; . . . ; rn;anð Þ½ � ¼ D

Pni¼1D

�1 ri;aið Þ �wiPni¼1wi

!

¼ D

Pni¼1bi �wiPn

i¼1wi

� �: ð2Þ

Table 1Linguistic hierarchies.

Level 1 Level 2 Level 3

l(t,n(t)) l(1,3) l(2,5) l(3,9)l(t,n(t)) l(1,7) l(2,13)

Fig. 1. Linguistic hierarchy of 3, 5 and 9 labels.

34 C. Porcel, E. Herrera-Viedma / Knowledge-Based Systems 23 (2010) 32–39

Definition 3 (Linguistic weighted average operator). Let x ¼ fðr1;

a1Þ; . . . ; ðrn;anÞg be a set of linguistic 2-tuples and W ¼ fðw1;aw1 Þ;

. . . ; ðwn;awn Þg be their linguistic 2-tuple associated weights. The

2-tuple linguistic weighted average xwl is

xwl r1;a1ð Þ; w1;aw

1

� �� �. . . rn;anð Þ; wn;aw

n

� �� �� �¼ D

Pni¼1bi � bWiPn

i¼1bWi

!;

ð3Þ

with bi ¼ D�1ðri;aiÞ and bWi¼ D�1ðwi;aw

i Þ.

In any fuzzy linguistic approach, an important parameter todetermine is the ‘‘granularity of uncertainty”, i.e., the cardinalityof the linguistic term set S. When different experts have differentuncertainty degrees on the phenomenon or when an expert hasto assess different concepts, then several linguistic term sets witha different granularity of uncertainty are necessary [22,32]. In [22]a multi-granular 2-tuple FLM based on the concept of linguistichierarchy is proposed.

A linguistic hierarchy (LH), is a set of levels lðt;nðtÞÞ, where eachlevel t is a linguistic term set with different granularity nðtÞ fromthe remaining of levels of the hierarchy. The levels are orderedaccording to their granularity, i.e., a level t þ 1 provides a linguisticrefinement of the previous level t. We can define a level from itspredecessor level as lðt;nðtÞÞ ! lðt þ 1;2 � nðtÞ � 1Þ. Table 1 showsthe granularity needed in each linguistic term set of the level tdepending on the value n(t) defined in the first level (3 and 7,respectively).

A graphical example of a linguistic hierarchy is shown in Fig. 1.In [22] a family of transformation functions between labels

from different levels was introduced:

Definition 4. Let LH ¼S

t lðt;nðtÞÞ be a linguistic hierarchy whoselinguistic term sets are denoted as SnðtÞ ¼ fsnðtÞ

0 ; . . . ; snðtÞnðtÞ�1g. The

transformation function between a 2-tuple that belongs to level tand another 2-tuple in level t0–t is defined as

TFtt0 : l t;nðtÞð Þ ! l t0; nðt0Þð Þ;

TFtt0 snðtÞ

i ;anðtÞ�

¼ DD�1 snðtÞ

i ;anðtÞ�

� nðt0Þ � 1ð ÞnðtÞ � 1

0@

1A:

As it was pointed out in [22] this family of transformation func-tions is bijective. This result guarantees that the transformationsbetween levels of a linguistic hierarchy are carried out without lossof information.

2.3. Incomplete fuzzy preference relations

Definition 5. A fuzzy preference relation P on a set of alternativesX ¼ fx1; . . . ; xng is a fuzzy set on the product set X � X, i.e., it ischaracterized by a membership function lP : X � X ! ½0;1�:

When cardinality of X is small, the preference relation may beconveniently represented by the n� n matrix P ¼ ðpijÞ, beingpij ¼ lPðxi; xjÞ ð8i; j 2 f1; . . . ; ngÞ interpreted as the preferencedegree or intensity of the alternative xi over xj, where

� pij ¼ 1=2 indicates indifference between xi and xj,� pij ¼ 1 indicates that xi is absolutely preferred to xj,� and pij > 1=2 indicates that xi is preferred to xj.

However, as we have mentioned, our system integrates themulti-granular FLM based on 2-tuples, so we must define a linguis-tic preference relation as follows:

Definition 6. Let X ¼ fx1 . . . ; xng a set of alternatives and S alinguistic term set. A linguistic preference relation P ¼ pijð8i; j 2f1; . . . ; ngÞ on X is

lP : X � X ! S� 0:5;0:5½ Þ; ð4Þ

where pij ¼ lPðxi; xjÞ is a 2-tuple which denotes the preference de-gree of alternative xi regarding to xj.

As aforementioned, in many real world GDM problems the ex-perts are often not able to provide all the preference values thatare required. In order to model these situations, we use incompletefuzzy preference relations [1–3,25,26,41].

Definition 7. A function f : X ! Y is partial when not everyelement in the set X necessarily maps onto an element in the setY. When every element from the set X maps onto one element ofthe set Y, then we have a total function.

Definition 8. A two-tuple fuzzy linguistic preference relation P ona set of alternatives X with a partial membership function is anincomplete two-tuple fuzzy linguistic preference relation.

3. A recommender system for the dissemination of informationin UDL using incomplete linguistic preference relations

The UDL staff manage and spread many information about re-search resources such as electronic books, electronic papers, elec-tronic journals, official dailies and so on [8,48]. Nowadays, thisamount of information is growing up and they are in need of auto-mated tools to filter and spread that information to the users in asimple and timely manner. On the other hand, UDL users needtools to help them to insert their preferences to form accurateprofiles.

In this section we present a new fuzzy linguistic recommendersystem in which the user profiles are obtained from user prefer-ences represented by incomplete fuzzy linguistic preference rela-tions [1]. This proposal contributes with some advantages with

UsersResources

Resourcerepresentations

VRi

Acquiring user'spreferences

Incompletepreferencerelation P

Computingmissing

information

P* Aggregation VUx

Recommendations

Matchingprocess

Fig. 2. Operating scheme.

C. Porcel, E. Herrera-Viedma / Knowledge-Based Systems 23 (2010) 32–39 35

regard to previous systems [46,45] because it facilitates theexpression of their preferences to the users and reduces the usereffort to characterize their user profiles. It is applied to adviseUDL users on the best research resources that could satisfy theirinformation needs in UDL. Moreover, the system recommends col-laboration possibilities to meet another researchers of relatedareas which could collaborate with them in projects or interestworks. In such a way, this new recommender system improvesthe services that a UDL provides to the users, because it is easierto obtain the knowledge about the users and it allows to decreasethe time cost to establish the user profiles.

In Fig. 2 we can see the basic operating scheme, which is ex-plained in the following subsections.

3.1. Information representation

In the proposed system, the user-system communication is car-ried out by using a multi-granular fuzzy linguistic approach[22,32], in order to allow a higher flexibility in the communicationprocesses of the system. The system uses different label setsðS1; S2; . . .Þ to represent the different concepts to be assessed inits filtering activity. These label sets, Si, are chosen from those labelsets that compose a LH, i.e., Si 2 LH. We should point out that thenumber of different label sets that we can use is limited by thenumber of levels of LH, and therefore, in many cases the label setsSi and Sj can be associated to a same label set of LH but with differ-ent interpretations, depending on the concept to be modeled. Wetake into account the following concepts that can be assessed inthe system:

� Importance degree of a discipline with respect to a resourcescope or user preferences (S1).

� Relevance degree of a resource for a user (S2).� Compatibility degree between two users (S3).� Preference degree of a resource regarding another one (S4).

Following the linguistic hierarchy shown in Fig. 1, in our systemwe use the level 2 (5 labels) to assign importance and preferencedegrees (S1 ¼ S5 and S4 ¼ S5), and the level 3 (9 labels) to assignrelevance and compatibility degrees (S2 ¼ S9 and S3 ¼ S9). Usingthis LH, the linguistic terms in each level are

� S5 ¼ fb0 ¼ Null¼ N;b1 ¼ Low¼ L;b2 ¼Medium¼M;b3 ¼ High¼H;b4 ¼ Total¼ Tg

� S9 ¼ fc0 ¼ Null ¼ N; c1 ¼ Very Low ¼ VL; c2 ¼ Low ¼ L; c3 ¼More Less Low ¼MLL; c4 ¼Medium ¼M; c5 ¼¼More Less HighMLH; c6 ¼ High ¼ H; c7 ¼ Very High ¼ VH; c8 ¼ Total ¼ Tg

3.1.1. Resources representationThe considered resources are journal articles, conference contri-

butions, book chapters, books or edited books. Once the librarystaff insert all the available information about a new resource,the system obtains an internal representation mainly based inthe resource scope. We use the vector model [36] to represent theresource scope. Thus, to represent a resource i, we use a classifica-tion composed by 25 disciplines (see Fig. 3). In each position westore a linguistic 2-tuple value representing the importance degreeof the resource scope with respect to the discipline represented bythat position:

VRi ¼ VRi1;VRi2; . . . ; VRi25ð Þ: ð5Þ

Then, each component VRij 2 S1, with j ¼ f1; . . . ; 25g, indicatesthe linguistic importance degree of the discipline j with regard tothe resource i. These importance degrees are assigned by thelibrary staff when they add a new resource.

3.1.2. User profilesThe user profiles are composed of two kinds of user preferences:

(1) User preferences on topics of interest, and(2) User preferences on collaboration possibility with other

users.

The main contribution of this proposal is how users providetheir preferences on topics of interest used to represent the sourceresources. In previous proposals [45,46] we represented such userpreferences using the vector model [36]. The problem is that theusers must insert or edit all the features corresponding to the dis-ciplines, i.e., in our case 25 categories. Thus, in previous proposalswe worked with vectors composed of 25 positions (each one corre-sponding to a discipline), but there could exist cases in which thisnumber could be greater. In such a way, users have to perform agreat effort to provide their preferences about topics of interest.To reduce this effort and make the process for acquiring the userpreferences easier, in this model we propose an alternative methodto obtain the user preferences on topics of interest.

We ask users to provide their preferences on some research re-sources, usually a limited number of resources, four or five. Thechoice of research resources is made by the personal staff tankinginto account the relevance supplied by the users. As in [41] we pro-pose users to represent their preferences by means of incompletefuzzy linguistic preference relations. Then, the system presentsusers only a selection of the most representative resources, and

Fig. 3. Interface to define the disciplines of the resource scope.

36 C. Porcel, E. Herrera-Viedma / Knowledge-Based Systems 23 (2010) 32–39

the users provide their preferences about these resources by meansof an incomplete fuzzy preference relation. Furthermore, accordingto results presented in [2], it is enough that the users provide onlya row of the preference relation. Then, we use the method pro-posed in [2] to complete the relations. Once the system completesthe fuzzy linguistic preference relation provided by the user, it ispossible to obtain a vector representing the user preferences onthe topics of interest. Next, we explain this process in detail:

(1) Acquiring the user preferences on a limited number ofresearch resources: At the beginning, the main goal is tohelp the users to provide their preferences assuring thatthese preferences are as consistent as possible. The systemshows users the five most representative resources, R ¼fr1 . . . ; r5g, and asks them to express their preferences bymeans of an incomplete fuzzy linguistic preference relation(see Fig. 4). The users only fill those preferences that theywish, assigning labels of S4. In the preference relation, eachpreference value pij represents the linguistic preferencedegree of resource i over the resource j according to the userfeeling. As aforementioned, the simplest case would be toprovide a relation with only one row of preference values:

P ¼

� p12 p13 p14 p15

x � x x x

x x � x x

x x x � x

x x x x �

0BBBBBB@

1CCCCCCA: ð6Þ

Then, the system completes the preference relation P using themethod proposed in [2], and obtains the relation P�:

P� ¼

� p12 p13 p14 p15

p�21 � p�23 p�24 p�25

p�31 p�32 � p�34 p�35

p�41 p�42 p�43 � p�45

p�51 p�52 p�53 p�54 �

0BBBBBB@

1CCCCCCA; ð7Þ

where p1j 2 S4 are the degrees inserted by the user about the pref-erences of the resource x1 with respect to xj, pii represents indiffer-ence, and each p�ij is the estimated degree for the user about his/herpreference of the resource xi with respect to xj.

(2) In order to obtain user preferences on topic of interest,i.e., user preference vector, firstly we calculate the userpreference degrees on each considered resource accordingto the preference relation P�, and secondly, we use thispreference degrees together with the vectors that representeach research resource to obtain the user preference vector.The preference degrees coincides with the dominancedegrees of a linguistic preference relation [19]. To obtainthem we propose the application of the arithmetic meanxe (Definition 1). Then, the preference degree of theresource i for the expert called DGi, is computed as follows:

DGi ¼ xe p�i1; . . . ; p�i5� �

: ð8Þ

Then, to obtain the user preference vector x, i.e. VUx ¼ ðVUx1;

VUx2; . . . ; VUx25Þ, from the aggregation of the vectors that representsthe characteristics of the chosen research resources, i.e., fVR1; . . . ;

VR5g, weighted by means of the user preference degreesfDG1; . . . ; DG5g. To do that, we use the linguistic weighted averageoperator defined in Definition 3, and then each positionk ¼ f1; . . . ; 25g of the vector VUx, is computed as follows:

VUxk ¼ xwl VR1k;DG1ð Þ; . . . ; VR5k;DG5ð Þ½ �: ð9Þ

On the other hand, to complete the user profile, the system asksevery user to express his/her collaboration preferences, i.e., if he/she wants to receive recommendations on collaboration possibili-ties with others users. This could help users to develop multi-dis-ciplinar works or participate in collaborative research projects [46].They should respond to this question with ‘‘Yes” or ‘‘No”.

3.2. Recommendation strategy

In this phase the system generates the recommendations to de-liver the information resources to the fitting users. This process isbased on a matching process developed between user profiles andresource representations [17,36]. To do that, we can use differentkinds of similarity measures, such as euclidean distance or CosineMeasure. Particularly, we use the standard cosine measure [36]. Asthe components of the vectors used to represent user profiles andresearch resources are 2-tuple linguistic values, then we define thecosine measure in a 2-tuple linguistic context. Given two vectors of2-tuple linguistic values:

Fig. 4. Interface to define the user preferences.

Table 2Contingency table for the resources.

Selected Not selected Total

Relevant Nrs Nrn NrIrrelevant Nis Nin Ni

Total Ns Nn N

C. Porcel, E. Herrera-Viedma / Knowledge-Based Systems 23 (2010) 32–39 37

V1 ¼ v11;av11ð Þ; v12;av12ð Þ; . . . ; v125;av125ð Þð Þ;

and

V2 ¼ v21;av21ð Þ; v22;av22ð Þ; . . . ; v225;av225ð Þð Þ;

then the linguistic similarity between both, called rlðV1;V2Þ 2 S1 isdefined as

rl V1;V2ð Þ

¼ D g �P25

k¼1 D�1 v1k;av1kð Þ � D�1 v2k;av2kð Þ�

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiP25k¼1 D�1 v1k;av1kð Þ� 2

r�

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiP25k¼1 D�1 v2k;av2kð Þ� 2

r0BB@

1CCA;ð10Þ

where g is the granularity of S1 and ðvik;avikÞ is the 2-tuple linguisticvalue of term k in the vectorðViÞ.

When a new resource i is inserted into the system, we calculatethe linguistic similarity measures, rlðVRi;VUjÞ, between the repre-sentation vector of this new resource ðVRiÞ and all the user prefer-ence vectors, fVU1; . . . ; VUmg; where m is the number of users inthe system. These user preference vectors are obtained as we haveindicated in Section 3.1.2.

Then, if rlðVRi;VUjÞP w, the user j is selected to receive recom-mendations about resource i. Previously, we have defined a linguis-tic threshold value ðwÞ to filter the output of the system. Next, thesystem applies to each rlðVRi;VUjÞ the transformation function de-fined in Definition 4, to obtain the relevance degree of the resourcei for the user j, expressed using a label of the set S2.

The collaboration preferences provided by the users are used toclassify the selected users in two sets, collaborators UC and non-collaborators UN . For the users of UN the system has finished therecommendation process, and therefore it sends them the resourceinformation together with its linguistic relevance degree.

For the users in UC the system calculates the collaboration pos-sibilities. To do it, between each two users x; y 2 UC , the systemperforms the following steps:

(1) Calculate the linguistic similarity measure between bothusers, rlðVUx;VUyÞ.

(2) Obtain the linguistic compatibility degree between bothusers, which must be expressed in S3. To do that, we applythe transformation function defined in 4 on rlðVUx;VUyÞ.

Finally the system sends to the users of UC the resource infor-mation, its calculated linguistic relevance degree and the collabo-ration possibilities characterized by its linguistic compatibilitydegrees.

4. Experiment and evaluation

In this section we present the evaluation of the proposed sys-tem. The main focus in evaluating the system is to determinate if

it fulfills the proposed innovations, that is, the recommended infor-mation is useful and interesting for the users, reducing the effortand making easier the process for acquiring the user’s preferences.Now we have implemented a trial version, in which the systemworks only with a few researchers.

4.1. Evaluation metrics

In the scope of recommender systems, precision, recall and F1are measures widely used to evaluate the quality of the recom-mendations [9,14,43,51]. We use them to compare the new pro-posal with previous systems. To calculate these metrics we needa contingency table to categorize the items with respect to theinformation needs. The items are classified both as relevant orirrelevant, and selected (recommended to the user) or not selected.The contingency table (see Table 2) is created using these fourcategories.

Precision is defined as the ratio of the selected relevant items tothe selected items, that is, it measures the probability of a selecteditem be relevant:

P ¼ Nrs

Ns: ð11Þ

Recall is calculated as the ratio of the selected relevant items tothe relevant items, that is, it represents the probability of a rele-vant items be selected:

R ¼ Nrs

Nr: ð12Þ

F1 is a combination metric that gives equal weight to both pre-cision and recall:

F1 ¼ 2� R1 � P1

R1 þ P1: ð13Þ

4.2. Experimental results

The purpose of the experiment is to test the performance of theproposed system, so we compared the recommendations made bythe system with the information provided by the library staff. Whenthe users receive a recommendation, they provide a feedback to thesystem assessing the relevance of the recommended resource, i.e.,they provide their opinions about the recommendation supplied

38 C. Porcel, E. Herrera-Viedma / Knowledge-Based Systems 23 (2010) 32–39

by the system. If they are satisfied with the recommendation, theyprovide a higher value. We use that feedback information to evalu-ate the system, applying the measures described in the previoussection.

We considered a data set with 30 research resources of differentareas, collected by the library staff from different informationsources. These resources were included into the system followingthe indications described in Section 3.1.1. Initially we limited theseexperiments to 6 users; all of them completed the registration pro-cess and they inserted their preferences about the five most rele-vant resources presented by the system (like in Fig. 4).

From this information provided by the users, the system buildsthe user profiles. These user profiles obtained from the providedpreferences and the resources previously inserted, constitutedour training data set. Then, we added 20 new resources that consti-tuted the test data set. The system filtered these 20 resources andrecommended each one to the suitable users. To obtain data tocompare, the 20 new resources also were recommended usingthe advices of the library staff.

For example, for the user 1, the system selected 4 resources asrelevant. However, from the information provided by the librarystaff and the user feedback, we could see that the system selected1 irrelevant resource for user 1, and it didn’t select 2 resources thatlibrary staff considered relevant for the user 1. Then, to build thecontingency table, we compared the recommendations provided

Table 4Detailed experiment results for the recommendations.

Precision (%) Recall (%) F1 (%)

User 1 75,00 60,00 66,67User 2 66,67 66,67 66,67User 3 75,00 50,00 60,00User 4 33,33 50,00 40,00User 5 80,00 66,67 72,73User 6 75,00 75,00 75,00

Average 67,50 61,39 63,51

Table 3Experimental contingency table.

User 1 User 2 User 3 User 4 User 5 User 6

Nrs 3 2 3 1 4 3Nrn 2 1 3 1 2 1Nis 1 1 1 2 1 1Nr 5 3 6 2 6 4Ns 4 3 4 3 5 4

Fig. 5. Experim

by the system with the recommendations provided by the librarystaff and the relevance degrees inserted by the users. With thisinformation, we build the contingency table for the recommendedresources. It is shown in Table 3.

From this contingency table, we obtain the corresponding pre-cision, recall and F1 which are shown in Table 4. The average ofprecision, recall and F1 metrics are 67.50%, 61.39% and 63.51%,respectively. The Fig. 5 shows a graph with the precision, recalland F1 values for each user. These values reveal a good perfor-mance of the proposed system, and therefore, a good usersatisfaction.

5. Conclusions

Digital libraries can serve as powerful tools for universities toreach out and expand their sphere of influence in the society.UDL provides effective channels for the dissemination of researchinformation. But users of UDL need tools to assist them in theirprocesses of information gathering because of the large amountof information available on these systems, as for example, recom-mender systems [43].

We have proposed a multi-granular linguistic recommendersystem in this research topic [46]. However, the process for acquir-ing the user profiles requires great effort, and sometimes, it is com-plicated by the great quantity of information that the user has togive to characterize their feeling on topics of interest. In this paperwe have proposed a new method to overcome this problem. Usersdo not directly provide the user preference vectors that character-ize their profiles. They provide preferences on some research re-sources and from this information we calculate their respectivepreference vectors on topics of interest. Furthermore, to facilitatethe process for acquiring the user preferences on the resourceswe allow users to provide their preferences by means of incom-plete fuzzy linguistic preference relations. The user profile is com-pleted with user preferences on the collaboration possibilities withother users. Therefore, this recommender system acts as a decisionsupport system that makes decisions about both the resources thatcould be interesting for a researcher and his/her collaboration pos-sibilities with other researchers to form interesting workinggroups. The experimental results show the user satisfaction withthe received recommendations.

Acknowledgements

This paper has been developed with the financing of FEDERfunds in FUZZYLING project (TIN2007-61079), PETRI project(PET2007-0460), and project of Ministry of Public Works (90/07).

ent results.

C. Porcel, E. Herrera-Viedma / Knowledge-Based Systems 23 (2010) 32–39 39

References

[1] S. Alonso, F.J. Cabrerizo, F. Chiclana, F. Herrera, E. Herrera-Viedma, Groupdecision-making with incomplete fuzzy linguistic preference relations,International Journal of Intelligent Systems 24 (2) (2009) 201–222.

[2] S. Alonso, F. Chiclana, F. Herrera, E. Herrera-Viedma, J. Alcalá-Fdez, C. Porcel, Aconsistency-based procedure to estimating missing pairwise preferencevalues, International Journal of Intelligent Systems 23 (2008) 155–175.

[3] S. Alonso, E. Herrera-Viedma, F. Chiclana, F. Herrera, Individual and socialstrategies to deal with ignorance situations in multi-person decision making,International Journal of Information Technology and Decision Making 8 (2)(2009) 313–333.

[4] B. Arfi, Fuzzy decision making in politics. A linguistic fuzzy-set approach(LFSA), Political Analysis 13 (1) (2005) 23–56.

[5] C. Basu, H. Hirsh, W. Cohen, Recommendation as classification: using socialand content-based information in recommendation, in: Proceedings of theFifteenth National Conference on Artificial Intelligence, 1998, pp. 714–720.

[6] R. Burke, Hybrid recommender systems, User Modeling and User-AdaptedInteraction 12 (4) (2002) 331–370.

[7] R. Burke, Hybrid web recommender systems, in: P. Brusilovsky, A. Kobsa, W.Nejdl, (Eds.), The Adaptive Web, LNCS 4321, 2007, pp. 377–408.

[8] J. Callan et al., Personalisation and recommender systems in digital libraries,Joint NSF–EU DELOS Working Group Report, 2003.

[9] Y. Cao, Y. Li, An intelligent fuzzy-based recommendation system for consumerelectronic products, Expert Systems with Applications 33 (2007) 230–240.

[10] S.L. Chang, R.C. Wang, S.Y. Wang, Applying a direct multigranularity linguisticand strategy-oriented aggregation approach on the assessment of supplyperformance, European Journal of Operational Research 177 (2) (2007) 1013–1025.

[11] H. Chao, Assessing the quality of academic libraries on the web: thedevelopment and testing of criteria, Library & Information Science Research24 (2002) 169–194.

[12] Z. Chen, D. Ben-Arieh, On the fusion of multi-granularity linguistic label sets ingroup decision making, Computers and Industrial Engineering 51 (3) (2006)526–541.

[13] M. Claypool, A. Gokhale, T. Miranda, Combining content-based andcollaborative filters in an online newpaper, in: Proceedings of the ACMSIGIR-99 Workshop on Recommender Systems-Implementation andEvaluation, 1999.

[14] C.W. Cleverdon, E.M. Keen, Factors determining the performance of indexingsystems. Test results, vol. 2. ASLIB Cranfield Research Project, Cranfield,Bedford, England, 1966.

[15] M.A. Gonçalves, E.A. Fox, L.T. Watson, N.A. Kipp, Streams, structures, spaces,scenarios, societies (5 s): a formal model for digital libraries, ACM Transactionson Information Systems 22 (2) (2004) 270–312.

[16] N. Good, J.B. Schafer, J.A. Konstan, A. Borchers, B.M. Sarwar, J.L. Herlocker, J.Riedl, Combining collaborative filtering with personal agents for betterrecommendations, in: Proceedings of the Sixteenth National Conference onArtificial Intelligence, 1999, pp. 439–446.

[17] U. Hanani, B. Shapira, P. Shoval, Information filtering: overview of issues,research and systems, User Modeling and User–Adapted Interaction 11 (2001)203–259.

[18] F. Herrera, E. Herrera-Viedma, Aggregation operators for linguistic weightedinformation, IEEE Transactions on Systems Man and Cybernetics Part A:Systems and Humans 27 (1997) 646–656.

[19] F. Herrera, E. Herrera-Viedma, Choice functions and mechanisms for linguisticpreference relations, European Journal of Operational Research 120 (2000)144–161.

[20] F. Herrera, E. Herrera-Viedma, J.L. Verdegay, Direct approach processes ingroup decision making using linguistic OWA operators, Fuzzy Sets andSystems 79 (1996) 175–190.

[21] F. Herrera, L. Martı́nez, A 2-tuple fuzzy linguistic representation model forcomputing with words, IEEE Transactions on Fuzzy Systems 8 (6) (2000) 746–752.

[22] L. Martı́nez, F. Herrera, A model based on linguistic 2-tuples for dealing withmultigranularity hierarchical linguistic contexts in multiexpert decision-making, IEEE Transactions on Systems, Man and Cybernetics. Part B:Cybernetics 31 (2) (2001) 227–234.

[23] E. Herrera-Viedma, Modeling the retrieval process of an information retrievalsystem using an ordinal fuzzy linguistic approach, Journal of the AmericanSociety for Information Science and Technology 52 (6) (2001) 460–475.

[24] E. Herrera-Viedma, An information retrieval system with ordinal linguisticweighted queries based on two weighting elements, International Journal ofUncertainty, Fuzziness and Knowledge-Based Systems 9 (2001) 77–88.

[25] E. Herrera-Viedma, S. Alonso, F. Chiclana, F. Herrera, A consensus model forgroup decision making with incomplete fuzzy preference relations, IEEETransactions on Fuzzy Systems 15 (5) (2007) 863–877.

[26] E. Herrera-Viedma, F. Chiclana, F. Herrera, S. Alonso, Group decision makingmodel with incomplete fuzzy preference relations based on additiveconsistency, IEEE Transactions on Systems, Man and Cybernetics. Part B:Cybernetics 37 (1) (2007) 176–189.

[27] E. Herrera-Viedma, O. Cordón, M. Luque, A.G. López, A.M. Muñoz, A model offuzzy linguistic IRS based on multi-granular linguistic information,International Journal of Approximate Reasoning 34 (3) (2003) 221–239.

[28] E. Herrera-Viedma, F. Herrera, F. Chiclana, A consensus model for multipersondecision making with different preference structures, IEEE Transactions onSystems Man and Cybernetics Part A: Systems and Humans 32 (3) (2002) 394–402.

[29] E. Herrera-Viedma, A.G. López-Herrera, A model of information retrievalsystem with unbalanced fuzzy linguistic information, International Journal ofIntelligent Systems 22 (11) (2007) 1197–1214.

[30] E. Herrera-Viedma, A.G. López-Herrera, M. Luque, C. Porcel, A fuzzy linguisticIRS model based on a 2-tuple fuzzy linguistic approach, International Journalof Uncertainty Fuzziness and Knowledge-Based Systems 15 (2) (2007) 225–250.

[31] E. Herrera-Viedma, A.G. López-Herrera, C. Porcel, Tuning the matchingfunction for a threshold weighting semantics in a linguistic informationretrieval system, International Journal of Intelligent Systems 20 (9) (2005)921–937.

[32] E. Herrera-Viedma, L. Martı́nez, F. Mata, F. Chiclana, A consensus supportsystem model for group decision-making problems with multi-granularlinguistic preference relations, IEEE Transactions on Fuzzy Systems 13 (5)(2005) 644–658.

[33] E. Herrera-Viedma, G. Pasi, A.G. López-Herrera, C. Porcel, Evaluating theinformation quality of web sites: a methodology based on fuzzy computingwith words, Journal of the American Society for Information Science andTechnology 57 (4) (2006) 538–549.

[34] E. Herrera-Viedma, E. Peis, Evaluating the informative quality of documents inSGML-format using fuzzy linguistic techniques based on computing withwords, Information Processing and Management 39 (2) (2003) 233–249.

[35] M.H. Hsu, A personalized English learning recommender system for ESLstudents, Expert Systems with Applications 34 (2008) 683–688.

[36] R.R. Korfhage, Information Storage and Retrieval, Wiley Computer Publishing,New York, 1997.

[37] T.-P. Liang, Recommendation systems for decision support: an editorialintroduction, Decision Support Systems 45 (2008) 385–386.

[38] J. Malinowski, T. Weitzel, T. Keim, Decision support for team staffing: anautomated relational recommendation approach, Decision Support Systems 45(2008) 429–447.

[39] G. Marchionini, Research and development in digital libraries. <http://ils.unc.edu/march/digital_library_R_and_D.html>. (accessed 04.09).

[40] L. Martı́nez, M. Barranco, L.G. Pérez, M. Espinilla, A knowledge basedrecommender system with multigranular linguistic information,International Journal of Computational Intelligence Systems 1 (3) (2008)225–236.

[41] L. Martı́nez, L.G. Pérez, M. Barranco, M. Espinilla, Improving the effectivenessof knowledge based recommender systems using incomplete linguisticpreference relations, International Journal of Uncertainty, Fuzziness andKnowledge-Based Systems 16 (2) (2008) 33–56.

[42] L. Martı́nez, L.G. Pérez, M. Barranco, A multi-granular linguistic based-contentrecommendation model, International Journal of Intelligent Systems 22 (5)(2007) 419–434.

[43] A.W. Neumann, Recommender Systems for Information Providers: DesigningCustomer Centric Paths to Information, Physica-Verlag, Heidelberg, 2009.

[44] G. Pasi, Modeling users’ preferences in systems for information access,International Journal of Intelligent Systems 18 (7) (2003) 793–808.

[45] C. Porcel, A.G. López-Herrera, E. Herrera-Viedma, A recommender system forresearch resources based on fuzzy linguistic modeling, Expert Systems withApplications 36 (2009) 5173–5183.

[46] C. Porcel, J.M. Moreno, E. Herrera-Viedma, A multi-disciplinar recommendersystem to advice research resources in university digital libraries. ExpertSystems with Applications, doi:10.1016/j.eswa.2009.04.038.

[47] L.M. Quiroga, J. Mostafa, An experiment in building profiles in informationfiltering: the role of context of user relevance feedback, Information Processingand Management 38 (2002) 671–694.

[48] M.E. Renda, U. Straccia, A personalized collaborative digital libraryenvironment: a model and an application, Information Processing andManagement 41 (2005) 5–21.

[49] P. Reisnick, H.R. Varian, Recommender systems, Special issue ofCommunications of the ACM 40 (3) (1997) 56–59.

[50] L. Ross, P. Sennyey, The library is dead, long live the library! The practice ofacademic librarianship and the digital revolution, The Journal of AcademicLibrarianship 34 (2) (2008) 145–152.

[51] B. Sarwar, G. Karypis, J. Konstan, J. Riedl, Analysis of recommendationalgorithms for e-commerce. in: Proceedings of ACM E-Commerce 2000Conference, 2000, pp. 158–167.

[52] L.A. Zadeh, The concept of a linguistic variable and its applications toapproximate reasoning, Part I. Information Sciences 8 (1975) 199–249;L.A. Zadeh, The concept of a linguistic variable and its applications toapproximate reasoning, Part II, Information Sciences 8 (1975) 301–357;L.A. Zadeh, The concept of a linguistic variable and its applications toapproximate reasoning, Part III, Information Sciences 9 (1975) 43–80.