Utilizing local evidence for blog feed search Yeha Lee • Seung-Hoon Na • Jong-Hyeok Lee Received: 18 March 2011 / Accepted: 8 August 2011 / Published online: 26 August 2011 Ó Springer Science+Business Media, LLC 2011 Abstract Blog feed search aims to identify a blog feed of recurring interest to users on a given topic. A blog feed, the retrieval unit for blog feed search, comprises blog posts of diverse topics. This topical diversity of blog feeds often causes performance deterioration of blog feed search. To alleviate the problem, this paper proposes several approaches based on passage retrieval, widely regarded as effective to handle topical diversity at document level in ad-hoc retrieval. We define the global and local evidence for blog feed search, which correspond to the document-level and passage-level evidence for passage retrieval, respectively, and investigate their influence on blog feed search, in terms of both initial retrieval and pseudo-relevance feedback. For initial retrieval, we propose a retrieval framework to integrate global evidence with local evidence. For pseudo-relevance feed- back, we gather feedback information from the local evidence of the top K ranked blog feeds to capture diverse and accurate information related to a given topic. Experimental results show that our approaches using local evidence consistently and significantly out- perform traditional ones. Keywords Blog feed search Blog distillation Passage-based retrieval Pseudo-relevance feedback A preliminary version of this work was presented in Lee et al. (2009). Y. Lee (&) J.-H. Lee Division of Electrical and Computer Engineering, POSTECH, Pohang, South Korea e-mail: [email protected]J.-H. Lee e-mail: [email protected]S.-H. Na Department of Computer Science, National University of Singapore, Singapore, Singapore e-mail: [email protected]123 Inf Retrieval (2012) 15:157–177 DOI 10.1007/s10791-011-9176-6
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Utilizing local evidence for blog feed search
Yeha Lee • Seung-Hoon Na • Jong-Hyeok Lee
Received: 18 March 2011 / Accepted: 8 August 2011 / Published online: 26 August 2011� Springer Science+Business Media, LLC 2011
Abstract Blog feed search aims to identify a blog feed of recurring interest to users on a
given topic. A blog feed, the retrieval unit for blog feed search, comprises blog posts of
diverse topics. This topical diversity of blog feeds often causes performance deterioration
of blog feed search. To alleviate the problem, this paper proposes several approaches based
on passage retrieval, widely regarded as effective to handle topical diversity at document
level in ad-hoc retrieval. We define the global and local evidence for blog feed search,
which correspond to the document-level and passage-level evidence for passage retrieval,
respectively, and investigate their influence on blog feed search, in terms of both initial
retrieval and pseudo-relevance feedback. For initial retrieval, we propose a retrieval
framework to integrate global evidence with local evidence. For pseudo-relevance feed-
back, we gather feedback information from the local evidence of the top K ranked blog
feeds to capture diverse and accurate information related to a given topic. Experimental
results show that our approaches using local evidence consistently and significantly out-
perform traditional ones.
Keywords Blog feed search � Blog distillation � Passage-based retrieval �Pseudo-relevance feedback
A preliminary version of this work was presented in Lee et al. (2009).
Y. Lee (&) � J.-H. LeeDivision of Electrical and Computer Engineering, POSTECH, Pohang, South Koreae-mail: [email protected]
Many users have been using blogs (or weblogs) to express their thoughts or opinions about
a wide range of topics including political issues, product reviews and diary-like private
posts. As the number of blog users has increased, the importance of blogs as an information
source has risen. As a result, the need for a customized elaborate search system, which
aims to find useful information in the blogosphere, has grown. Several commercial search
engines such as Google1 and Technorati2 have started to provide blog search services.
Nowadays, it has become common for blog users to search for blog feeds (e.g. RSS,
ATOM) relevant to topics that interest them, and then subscribe to the feeds using a feed
reader such as an RSS reader. In this scenario, a key issue is how to identify blog feeds that
are relevant and dedicated to a given topic. This task is blog feed search, which is one of
the most important blog search services. The Blog Distillation task of TREC Blog Track
(Macdonald et al. 2008; Ounis et al. 2009; Macdonald et al. 2010) also reflects the
increasing interest in blog feed search.
A straightforward approach for blog feed search would be to apply existing retrieval
models developed in ad-hoc retrieval. For example, we can view a blog feed as a virtual
document by concatenating all constituent posts belonging to the blog feed, and then
readily apply existing retrieval models without any modification. In fact, most previous
work on blog feed search used this approach as the baseline system (Macdonald et al.
2008; Ounis et al. 2009; Macdonald et al. 2010).
However, blog feed search has some characteristics that limit the performance of the
straightfoward approach. First, the retrieval unit is a blog feed, which is an aggregation of
its constituent posts, not a single blog post. In this regard, blog feed search should consider
how to model the relationship between the relevance of blog posts and the blog feed, in
response to a given topic. Second, most blog feeds contain topically diverse blog posts,
depending on a blogger’s interest. In other words, a blog feed generally addresses a large
number of topics. The topical diversity of blog feeds makes it difficult for blog feed search
systems to find out which blog feeds are relevant to users’ information needs. Third, blog
feed search has to deal with more noisy data than traditional search tasks. The blog corpus
is not as topically coherent as the news corpus, and may also have non-topical contents
such as spam blogs and blog comment spam, which advertise commercial products and
services (Kolari et al. 2006). Therefore, feed search techniques should be robust to this
noisy environment.
Among the above characteristics, this paper focuses on the performance deterioration
caused by the topical diversity of blog feeds. To mitigate this problem, our approaches are
motivated by the passage retrieval technique, which is one of the most effective techniques
to deal with topical diversity at document level for ad-hoc retrieval. We introduce globalevidence and local evidence for evaluating the relevance of a blog feed in response to
a query. These two types of evidence correspond to document-level and passage-level
evidence for passage retrieval, respectively. Whereas global evidence is derived from all
the constituent posts within a feed, local evidence is defined using a few blog posts that are
highly relevant to a query.
Different from most previous studies that use only global evidence to estimate the
relevance of a blog feed, we explicitly define and take advantage of the local evidence on
both initial retrieval and pseudo-relevance feedback (PRF). For initial retrieval, we propose
The statistical significance at the 0.05 and 0.01 level is indicated by � and } for an improvement from thebaseline (GLD), respectively. The best performance is shown in bold
4 LLD also outperforms LSD.
164 Inf Retrieval (2012) 15:157–177
123
Figure 1b shows the influence of the parameter T on the performance of blog feed
search when using GSD?LSD. T controls how many blog posts are used for the local
evidence of a blog feed in response to a given query. We can obtain the best performance
when T is set to 2. This reveals that using a few highly relevant posts within a blog feed is
effective in evaluating the local evidence.
3.3 Comparison with other approaches
In the experiments, we showed that the use of local evidence is quite helpful in improving
the performance of blog feed search. Some previous researchers had already utilized
similar methods.
Macdonald and Ounis (2008), motivated by the Voting Model for the expert search task,
suggested expCombSUM. In expCombSUM, the highly relevant posts have a large effect
0 0.2 0.4 0.6 0.8 10.15
0.20
0.25
0.30
0.35
0.40
0.45
alpha
MA
P
07 MAP08 MAP09 MAP
0 2 4 6 8 10 12 150.20
0.25
0.30
0.35
0.40
0.45
The number of T
MA
P
07 MAP08 MAP09 MAP
(a)
(b)
Fig. 1 MAP scores for varying the parameters, a and T, under the GSD?LSD retrieval model. a MAPscores for varying the weight parameter a, b MAP scores for varying the value of T
Inf Retrieval (2012) 15:157–177 165
123
on the relevance score of a blog feed. Due to its weighted approach using query-relevant
scores, expCombSUM plays a similar role to local evidence. Elsas et al. (2008) also
proposed the Entry Centrality Component as a part of the Small Document Model. The
component estimates a probability distribution to measure the similarity between a blog
post and its feed, and controls the weight of each post to evaluate the relevance between its
feed and a query.
However, these approaches are different from ours in some respects. They consider all
constituent posts within a blog feed. Although the posts are differently weighted, the
approaches can be regarded as using weighted global evidence. In contrast, our model
actively finds local evidence corresponding to the passage-level evidence for passage
retrieval. Furthermore, whereas their approaches can only be applied in SDM, our model
provides a more flexible and expanded framework, in the sense that two types of evidence
can be estimated regardless of representation methods (e.g. LDM or SDM).
One of the most similar approaches to our model is the PCS-GR model suggested by
Seo and Croft (2008). PCS-GR is an approach combining their Global Representation and
Pseudo-Cluster based Selection, corresponding to our GLD?LSD approach. Like our
results, they showed that the combining approach results in significant improvements in
their well-designed experiments. However, our motivation is different from theirs.
Whereas they introduced a combining approach to penalize topically-diverse feeds, we
proposed a combining approach to avoid ‘‘over-penalizing’’ topically-diverse feeds. The
local evidence of a blog feed plays a similar role to the passage-level evidence of passage
retrieval. In addition, our approach provides a general framework by integrating global and
local evidence, including PCS-GR as a special case (i.e. GLD?LSD).
4 Feedback model for blog feed retrieval
In the previous section, we showed how local evidence is explored for the initial retrieval
of blog feed search, and verified that local evidence is helpful in improving retrieval
performance. In this section, we further explore local evidence in terms of PRF, and
propose novel feedback approaches based on local evidence.
4.1 Limitations of naive feedback approaches
Before addressing our feedback methods, we present two naive approaches for PRF and
show why they are not desirable.
Because the retrieval unit of blog feed search is a blog feed, not a document, a blog feed
is also a natural feedback unit. In this regard, a naive feedback model is an All-Postsapproach, which chooses all constituent posts in the top-ranked feeds as feedback docu-
ments. However, due to the topical diversity of the blog feed, even if a blog feed is
relevant, it does not mean that all of its constituent posts are relevant to a query. Fur-
thermore, if some of the top-ranked feeds chosen for the feedback are irrelevant, almost all
of the posts within them could be irrelevant. Therefore, the All-Posts approach has
potentially high risk of selecting many irrelevant posts, which decrease the precision of the
feedback information.
Another naive model is a Post-Level approach, which applies the traditional feedback
approach to blog feed search. The approach first performs a post-level retrieval and then
uses the top-ranked posts as feedback documents, without considering which feed they
come from. Unlike the All-Posts approach, the Post-Level one does not suffer from the low
166 Inf Retrieval (2012) 15:157–177
123
precision of feedback information. However, the feedback information can be biased
toward a dominant aspect within the top-ranked posts. In other words, the Post-Level
approach may suffer from ‘‘aspect recall’’ (Kurland et al. 2005), one of the important
properties which determines feedback quality.
With regard to query expansion for blog feed search, previous work has addressed some
properties of blog feed search queries: ‘‘. . . Given the nature of feed search, queries maydescribe more general and multifaceted topics, likely to stimulate discussion over time. If aquery corresponds to a high-level description of some topic, there might be a widevocabulary gap between the query and the more nuanced and faceted discussion in blogposts’’ (Elsas et al. 2008).
This property can make the aspect-recall problem of the Post-Level approach more
serious, because the vocabulary gap may make the top N ranked documents more likely to
be biased to a certain aspect of a given query. As a result, the feedback documents selected
using the Post-Level approach will cover only a few aspects of a query.
4.2 Feed based selection
A blog feed consists of posts with diverse topics depending on the bloggers’ interests or
inclinations. Thus, for a given query, the blog posts from different feeds may present
different perspectives or facets of a topic, although they address information about the
same topic. In other words, all (unknown) aspects of a query are scattered over all the
relevant feeds, and their relevant posts. Therefore, if we gather information from various
blog feeds, we can obtain more diverse information about a query so that it can cover the
various aspects of the query topic, and this leads to the improved performance of PRF.
However, this approach can have the same problem as the All-Posts approach. To solve
this problem, motivated by passage-based feedback, we propose Feed-Based Selectionwhich first selects as many feeds as possible for PRF, and then gathers only a few posts
within each of them, in order of the relevance between posts and the query. In other words,
Feed-Based Selection uses local evidence on the top-ranked blog feeds. This method
corresponds to passage-based feedback in ad-hoc retrieval where the scope of the feedback
context is narrowed into the passage, rather than using the entire document context.
The Feed-Based Selection has two important characteristics that allow it to handle the
problems of two naive approaches, All-Posts and Post-Level. First, it only uses the highly
relevant posts of a top-ranked feed (local evidence), not entire posts (global evidence). In
contrast with the All-Posts approach, it can alleviate the low precision problem caused by
the topical diversity of a blog feed. Second, it collects more diverse information from as
many feeds as possible. As a result, it allows a system to learn much more about the aspects
of a query than the Post-Level approach, and leads to an increase in aspect recall.
Similar to the initial retrieval model presented in Sect. 2, one of the most important
issues is how to define the local evidence of each blog feed. We propose two approaches
for defining local evidence: Fixed Feed Based Selection and Weighted Feed BasedSelection.
4.3 Fixed feed based selection (FFBS)
FFBS uses the top K ranked feeds to gather feedback documents. FFBS considers the top
K ranked feeds as equally relevant to a given query regardless of their relevance to the
query indicated by the relevance score.
Inf Retrieval (2012) 15:157–177 167
123
Let FBFFBS be a set of blog posts chosen by using FFBS. We can define FBFFBS as
The statistical significance at the 0.05 level is indicated by �, § and } for an improvement from the baseline,the Post-Level selection (TOP-10), and the All-Posts selection, respectively. The best performance is shownin bold
Inf Retrieval (2012) 15:157–177 169
123
over the baseline on MAP. This means that the feed-based approaches (FFBS and WFBS)
are effective to improve the performance of PRF.
Furthermore, FFBS and WFBS show better performance than the two naive approaches:
All-Posts (Feed3All-Posts and Feed5All-Posts) and Post-Level (TOP-10). To see whether
the improvement is statistically significant, we again performed the Wilcoxon signed rank
test, and attached § and } only when they showed significant results over All-Posts and
Post-Level, respectively.5 We found that the majority of runs of FFBS and WFBS show
statistically significant improvements over both of the naive approaches.
The All-Posts and Post-Level methods did not show reliable performance. They did not
show any improvement over the baseline for most topic sets. First, the failure of the All-
Posts approach provides good evidence that it suffers from low precision of feedback
information. In particular, for the 08 topics, the top K feeds used for PRF are likely to
contain many irrelevant feeds, because the initial performance for the 08 topics is relatively
low. Thus, as K increases for the 08 topics, the feedback documents constructed using the
All-Posts include too many irrelevant documents to improve the performance of PRF.
Actually, when using K = 5, the performance deteriorated more seriously than when
K = 3. This result explains why we need to use local evidence for PRF.
Second, for the 07 and 08 topics, the failure of the Post-Level approach supports our
proposal for the feed-level selection. Post-Level suffers from low aspect recall so that it
can only cover a few relevant aspects of a query. In contrast, our approaches enable the
system to increase the aspect recall, because the feedback documents are chosen from
various feeds which reflect the diverse aspects relevant to a query. Finally, this leads to the
improved performance of the feedback model.
We compare our approaches with the top 3 performing runs6 of the TREC 07, 08 and 09
Distillation task in Table 4. The results are obtained from (Macdonald et al. 2008; Ounis
et al. 2009; Macdonald et al. 2010). Our feedback approaches significantly and consistently
improve the results of the best runs for all tasks. In particular, for the 07 task, WFBS-5-10
achieved about a 6% increase of the MAP score over the TREC ’07 best run. FFBS-3-3
accomplished more than a 2% increase of the MAP score over the TREC ’08 best run.
WFBS-3-10 also increased the MAP score by 12% over the TREC ’09 best run.
Note that in Table 4, we only quote the official results of the top performing runs from
TREC, and we did not implement them. Furthermore, we did not apply the significance test
such as the Wilcoxon signed rank test between our methods and the TREC runs. Therefore,
it is unclear what caused the difference in performances between our methods and the runs.
The performance differences might be caused by several factors such as the method for
preprocessing documents, the way for selecting parameters or the effectiveness of each
algorithm for blog feed search. It will be valuable to implement the top performing
algorithms and directly compare the results. We leave this issue for a remaining work.
5.3 Influence of K and M on performance
Figure 2 shows the performance of FFBS and All-Posts according to varying K and
M parameters.
5 FFBS or WFBS were compared with All-Posts using the same K. That is, FFBS-5-2 and WFBS-5-10 werecompared with Feed5All-Posts.6 The runs are the automatic title-only runs sorted by MAP.
170 Inf Retrieval (2012) 15:157–177
123
The FFBS methods show more reliable and better curves than All-Posts for all the topic
sets. In particular, for the 08 and 09 topics, the performance gap between FFBS methods
and All-Posts was very big for large values of K. From these results, we can again verify
the effectiveness of local evidence to improve the performance of PRF.
The best parameter range of K for each method was different for each topic set. For the
07 topics, the best MAPs were found at relatively large K values between 5 and 7, while the
MAP scores at small K(B2) were not good. However, for the 08 and 09 topics, the trend for
K is reversed, where the best MAP scores are obtained at relatively small K values between
1 and 3, while MAP scores at large K(C5) decreased seriously.
Note that the performance curves are more robust on the 07 topics than the 08 and 09
topics for all methods including All-Posts. In other words, on the 07 topics, even at large
K values, the MAP for each method did not seriously decrease, while on the others, when
K C 5, the MAP of all methods decreased sharply.
One possible explanation for the differing trend and robustness between 07 topics and
08, 09 topics can be obtained by comparing the performance of the initial retrieval for each
topic set. From Table 3, we already saw that Pr@10 on the 07 topics is much better than
those on the 08 and 09 topics. That is, the number of relevant feeds in the top-ranked ones
will be more for the 07 topics than for the 08 and 09 topics. This may mean that the
deterioration of the precision from using more feeds is not severe, resulting in reliable
MAP scores. In contrast, for the 08 and 09 topics, when using a relatively large K value
(about 5), the top K ranked feeds are likely to be irrelevant due to low Pr@10, so that the
precision seriously decreases, causing a low MAP score.
When using M = 2, we obtained the most reliable performance, for all the topic sets,
among the three values.7 The results for M = 1 and M = 5 are on a case-by-case basis
Table 4 The performance of the top 3 performing runs for the TREC 07, 08 and 09 Distillation tasks
For comparison purposes, the performance of our approaches are included. The best performance is shownin bold
7 Even though we did not plot the case of M = 3, the curve is similar to that of M = 2.
Inf Retrieval (2012) 15:157–177 171
123
according to each topic set. For example, consider when M = 5. For the 07 topics, its MAP
is the best, compared with the MAPs when M = 1 or M = 2. On the other hand, for the 08
topics, its MAP becomes worse than when M = 1 or M = 2.
1 2 3 4 5 6 7 8 9 100.390
0.395
0.400
0.405
0.410
0.415
0.420
0.425
0.430
0.435
K (The number of Feeds)
MA
P
M = 1M = 2M = 5All−PostsBaseline
1 2 3 4 5 6 7 8 9 100.26
0.27
0.28
0.29
0.30
0.31
0.32
0.33
K (The number of Feeds)
MA
P
M = 1M = 2M = 5All−PostsBaseline
1 2 3 4 5 6 7 8 9 100.34
0.35
0.36
0.37
0.38
0.39
0.40
K (The number of Feeds)
MA
P
M = 1M = 2M = 5All−PostsBaseline
(a)
(b)
(c)
Fig. 2 The MAP scores of FFBSaccording to varying K andM parameters, usingGSD?LSD as the retrievalmodel, compared to the baselineand All-Posts. a The MAP scoresfor 07 topics. b The MAPscores for 08 topics. c The MAPscores for 09 topics
172 Inf Retrieval (2012) 15:157–177
123
5.4 Comparison of cluster centroid algorithm
As discussed in Sect. 4, our feed based selection is derived by considering the aspect recall.
There is existing work for the ad-hoc retrieval task related to increasing the aspect recall.
This work is called the Cluster Centroid approach (Shen and Zhai 2005). Cluster Centroid
clusters the feedback documents to maximize the diversity of the feedback information (i.e.
aspect recall). Cluster Centroid consists of the following 3 steps: 1) Group the top
N documents into K clusters, 2) Select a centroid document from each resulting cluster, and
3) Use all such K centroid documents for feedback documents. Since Cluster Centroid does
not use any information about the relationship between the posts and their feed, it can be
viewed as an automatic method to construct feeds by regarding a cluster as a pseudo feed.
We re-implemented the Cluster Centroid method in the same setting used in their exper-
iment, by using the K-Medoid clustering algorithm (Kaufman and Rousseeuw 1990), and
J-Divergence (Lin 1991) as the distance function between clusters. For a fair comparison to
the previous section, we fix the number of clusters K to 10.
Table 5 shows the results of Cluster Centroid, according to the number of top posts
N for clustering, where the feedback model is GSD?LSD. Note that when N = 10, the
Cluster Centroid corresponds to TOP-N, since each post creates a separate cluster. The
Cluster Centroid method outperforms the baseline for some N values by about 0.2%, 0.9%
and 2.6% for the 07, 08 and 09 topics, respectively. This result of Cluster Centroid is
important, because it confirms the view we previously discussed on the aspect recall, i.e.,
using diverse information is helpful to improve the performance of PRF for blog feed
search.
Our approaches are still notable, due to the improvements over Cluster Centroid. In
particular, for the 07 and 08 topics, our best approaches show about 3.4% and 1.5%
increases of MAP over Cluster Centroid, respectively. From these results, we can verify
that the feed-level information used in our methods is important for improving the retrieval
performance, because it captures a realistic structure between posts and feeds that Cluster
Centroid cannot automatically recognize.
Table 5 The performance of K = 10 Cluster Centroid with N under the GSD?LSD method
N 2007 (951–995) 2008 (1,051–1,100) 2009 (1,101–1,150)
The best performance is shown in bold. For comparison purposes, the performances of the baseline and ourbest approaches for each task are included
Inf Retrieval (2012) 15:157–177 173
123
6 Related work
Since the TREC blog distillation task was introduced, many approaches have been sug-
gested for blog feed search. Most approaches are motivated by other well-studied retrieval
tasks such as the expert search task (Soboroff and de Vries 2007) and the resource selection
task in distributed information retrieval.
Elsas et al. (2008) and Arguello et al. (2008), (2009) treated blog feed search as a
resource ranking problem by using the ReDDE federated search algorithm (Si and Callan
2003). They proposed two blog representations based on granularity, and also suggested a
query expansion approach using Wikipedia for blog feed search. For PRF, Elsas et al.
(2008) proposed knowledge-intensive feedback, using Wikipedia as external knowledge.
Despite its notable results, their approach is not a closed solution that only uses the given
test collection, which is different from our approaches.
Seo and Croft (2008), (2009) dealt with blog feed search by using cluster-based retrieval
for distributed information retrieval. They also divided blog sites into three types based on
topical diversity, and considered several methods for penalizing blog sites with diverse
topics.
Macdonald and Ounis (2008) and He et al. (2009) regarded blog feed search as an
expert finding task. They used the adaptable Voting Model for the expert search task
(Macdonald and Ounis 2006), and proposed several techniques that aim to boost blog feeds
where a blogger has shown a central or recurring interest in a topic area. Carman et al.
(2009) also used a similar approach, using the Voting Model. In contrast to Macdonald and
Ounis’s work, they used non-content features for each blog in addition to existing content-
level features, and applied the Learning-to-Rank (Yue et al. 2007) approach to combine the
features and obtain a single retrieval function.
Nunes et al. (2009) suggested several strategies using temporal features for blog feed
search. They examined whether or not the maximum temporal span covered by the relevant
posts is a positive criterion in the feed search, and also investigated how the dispersion of
relevant blog posts in a blog feed would impact this task. Wang et al. (2009) proposed a
reduced document model by indexing text between certain tags, and used the PageRank of
a blog feed with its query likelihood score. Balog et al. (2008) and Weerkamp et al. (2008)
proposed two language models based on expert finding techniques, and some blog-specific
features such as document structure, social structure, and temporal structure.
7 Conclusion and future work
In this paper, we have addressed several approaches for initial retrieval and pseudo-
relevance feedback on blog feed search. Our key concern was the topical diversity of a
blog feed. Motivated by passage retrieval techniques, we presented global and local evi-
dence of blog feeds, corresponding to the document-level and passage-level evidence of
passage retrieval. We estimated global evidence using all constituent posts within a blog
feed, and local evidence using highly relevant posts within a blog feed in response to a
given query. We proposed a series of methods for evaluating the relevance between a blog
feed and a given query, using the two types of evidence.
In addition, we investigated the pseudo-relevance feedback method for blog feed search.
Our feedback approaches, motivated by passage-based feedback, gathered feedback
information using the local evidence of top K ranked feeds. The proposed methods have
two advantages. First, the usage of various feeds enables the feedback model to locate the
174 Inf Retrieval (2012) 15:157–177
123
feeds that discuss different aspects of the topic of a given query. In other words, it increases
the aspect recall of feedback information. Second, the usage of the local evidence provides
the feedback model with information relevant to a query. That is, it increases the precision
of feedback information. Experimental results on TREC distillation for the 07, 08 and 09
topics showed that the proposed feedback approach significantly and consistently out-
performed the baseline.
Many studies remain for future work. First, for the initial retrieval, we used a simple
uniform distribution as P(D|L) in (3). It would be interesting to investigate other methods
to estimate P(D|L) such as Entry Centrality (Elsas et al. 2008). Furthermore, we would like
to investigate other techniques for blog feed search such as link analysis and temporal
profiling. These techniques have the potential to improve the performance of blog feed
search. Second, for pseudo-relevance feedback, we will explore a probabilistic approach
for selecting the relevant local posts, instead of using the current threshold-driven method.
References
Allan, J. (1995). Relevance feedback with too much data. In Proceedings of the 18th annual internationalACM SIGIR conference on research and development in information retrieval (pp. 337–343).
Allan, J., Connell, M. E., Croft, W. B., Feng, F. F., Fisher, D., & Li, X. (2001). INQUERY and TREC-9. InProceedings of the ninth text REtrieval conference (pp. 551–562).
Arguello, J., Elsas, J. L., Callan, J., & Carbonell, J. G. (2008). Document representation and query expansionmodels for blog recommendation. In Proceedings of the 2nd international conference on weblogs andsocial media.
Arguello, J., Elsas, J. L., Yoo, C., Callan, J., & Carbonell, J. G. (2009). Document and query expansionmodels for blog distillation. In Proceedings of the seventeenth text REtrieval conference.
Balog, K., de Rijke, M., & Weerkamp, W. (2008). Bloggers as experts: Feed distillation using expertretrieval models. In Proceedings of the 31st annual international ACM SIGIR conference on researchand development in information retrieval (pp. 753–754).
Bendersky, M., & Kurland, O. (2010). Utilizing passage-based language models for ad hoc documentretrieval. Information Retrieval, 13, 157–187.
Callan, J. P. (1994). Passage-level evidence in document retrieval. In Proceedings of the 17th annualinternational ACM SIGIR conference on research and development in information retrieval(pp. 302–310).
Carman, M., Keikha, M., Gerani, S., Gwadera, R., Taibi, D., & Crestani, F. (2009). University of Lugano atTREC 2008 blog track. In Proceedings of the seventeenth text REtrieval conference.
Elsas, J. L., Arguello, J., Callan, J., & Carbonell, J. G. (2008). Retrieval and feedback models for blog feedsearch. In Proceedings of the 31st annual international ACM SIGIR conference on research anddevelopment in information retrieval (pp. 347–354).
He, B., Macdonald, C., Ounis, I., Peng, J., & Santos, R. L. (2009). University of Glasgow at TREC 2008:Experiments in blog, enterprise, and relevance feedback tracks with terrier. In Proceedings of theseventeenth text REtrieval conference.
Kaszkiel, M., & Zobel, J. (1997). Passage retrieval revisited. In Proceedings of the 20th annual internationalACM SIGIR conference on Research and development in information retrieval (pp. 178–185).
Kaszkiel, M., & Zobel, J. (2001). Effective ranking with arbitrary passages. Journal of the American Societyfor Information Science and Technology, 52(4), 344–364.
Kaufman, L., & Rousseeuw, P. J. (1990). Finding groups in data: an introduction to cluster analysis. NewYork: Wiley.
Kolari, P., Java, A., & Finin, T. (2006). Characterizing the splogosphere. In Proceedings of the 3rd annualworkshop on weblogging ecosystem: Aggregation, analysis and dynamics, 15th World Wid Webconference.
Kurland, O., Lee, L., & Domshlak, C. (2005). Better than the real thing?: Iterative pseudo-query processingusing cluster-based language models. In Proceedings of the 28th annual international ACM SIGIRconference on research and development in information retrieval (pp. 19–26).
Inf Retrieval (2012) 15:157–177 175
123
Lafferty, J., & Zhai, C. (2001). Document language models, query models, and risk minimization forinformation retrieval. In Proceedings of the 24th annual international ACM SIGIR conference onresearch and development in information retrieval (pp. 111–119).
Lavrenko, V., & Croft, W. B. (2001). Relevance based language models. In Proceedings of the 24th annualinternational ACM SIGIR conference on research and development in information retrievalpp. 120–127.
Lee, Y., Na, S. H., & Lee, J. H. (2009). An improved feedback approach using relevant local posts for blogfeed retrieval. In Proceeding of the 18th ACM conference on Information and knowledge management(pp. 1971–1974).
Lin, J. (1991). Divergence measures based on the shannon entropy. IEEE Transactions on InformationTheory, 37, 145–151.
Macdonald, C., & Ounis, I. (2006a). Voting for candidates: Adapting data fusion techniques for an expertsearch task. In Proceedings of the 15th ACM international conference on Information and knowledgemanagement (pp. 387–396).
Macdonald, C., & Ounis, I. (2006b). The TREC Blogs06 collection: Creating and analysing a blog testcollection. Techincal report, Department of Computing Science, University of Glasgow.
Macdonald, C., & Ounis, I. (2008). Key blog distillation: Ranking aggregates. In Proceeding of the 17thACM conference on Information and knowledge management (pp. 1043–1052).
Macdonald, C., Ounis, I., & Soboroff, I. (2008). Overview of the TREC-2007 blog track. In Proceedings ofthe sixteenth text REtrieval conference.
Macdonald, C., Ounis, I., & Soboroff, I. (2010). Overview of the TREC-2009 Blog Track. In Proceedings ofthe eighteenth text REtrieval conference.
Na, S. H., Kang, I. S., Lee, Y., & Lee, J. H. (2008a). Applying complete-arbitrary passage for pseudo-relevance feedback in language modeling approach. In Proceedings of the 4th Asia informationretrieval conference on information retrieval technology (pp. 626–631).
Na, S. H., Kang, I. S., Lee, Y., & Lee, J. H. (2008b). Completely-arbitrary passage retrieval in languagemodeling approach. In Proceedings of the 4th Asia information retrieval conference on informationretrieval technology (pp. 22–33).
Nunes, S., Ribeiro, C., David, G. (2009). FEUP at TREC 2008 blog track: Using temporal evidence forranking and feed distillation. In Proceedings of the seventeenth text REtrieval conference.
Ounis, I., Macdonald, C., & Soboroff, I. (2009). Overview of the TREC-2008 blog track. In Proceedings ofthe seventeenth text REtrieval conference.
Rocchio, J. J. (1971). Relevance feedback in information retrieval. In G. Salton (Ed.), The SMART retrievalsystem: Experiments in automatic document processing, Prentice-Hall series in automatic computation(Chap. 14, pp. 313–323). Englewood Cliffs, NJ: Prentice-Hall.
Salton, G., Allan, J., & Buckley, C. (1993). Approaches to passage retrieval in full text information systems.In Proceedings of the 16th annual international ACM SIGIR conference on research and developmentin information retrieval (pp. 49–58).
Seo, J., & Croft, W. B. (2008). Blog site search using resource selection. In Proceeding of the 17th ACMconference on Information and knowledge management (pp. 1053–1062).
Seo, J., & Croft, W. B. (2009). UMass at TREC 2008 blog distillation task. In Proceedings of the seven-teenth text REtrieval conference.
Shen, X., & Zhai, C. (2005). Active feedback in ad hoc information retrieval. In Proceedings of the 28thannual international ACM SIGIR conference on research and development in information retrieval(pp. 59–66).
Si, L., & Callan, J. (2003). Relevant document distribution estimation method for resource selection. InProceedings of the 26th annual international ACM SIGIR conference on research and development ininformaion retrieval (pp. 298–305).
Soboroff, I., & de Vries, A. P. (2007). Overview of the TREC 2006 enterprise track. In Proceedings of thefifteenth text REtrieval conference.
Wang, J., Sun, Y., Mukhtar, O., & Srihari, R. (2009). TREC 2008 at the University at Buffalo: Legal andblog track. In Proceedings of the seventeenth text REtrieval conference.
Weerkamp, W., Balog, K., & de Rijke, M. (2008). Finding key bloggers, one post at a time. In Proceeding ofthe 2008 conference on ECAI 2008: 18th European conference on artificial intelligence (pp. 318–322).
Yu, S., Cai, D., Wen, J. R., & Ma, W. Y. (2003). Improving pseudo-relevance feedback in web informationretrieval using web page segmentation. In Proceedings of the 12th international conference on WorldWide Web (pp. 11–18).
Yue, Y., Finley, T., Radlinski, F., & Joachims, T. (2007). A support vector method for optimizing averageprecision. In Proceedings of the 30th annual international ACM SIGIR conference on research anddevelopment in information retrieval (pp. 271–278).
176 Inf Retrieval (2012) 15:157–177
123
Zhai, C., & Lafferty, J. (2001). Model-based feedback in the language modeling approach to informationretrieval. In Proceedings of the tenth international conference on information and knowledge man-agement (pp. 403–410).
Zhai, C., & Lafferty, J. (2004). A study of smoothing methods for language models applied to informationretrieval. ACM Transactions on Information Systems, 22(2), 179–214.