Unsupervised Transfer Classification Application to Text Categorization Tianbao Yang, Rong Jin, Anil Jain, Yang Zhou, Wei Tong Michigan State University
Dec 18, 2015
Unsupervised Transfer Classification
Application to Text Categorization
Tianbao Yang, Rong Jin, Anil Jain, Yang Zhou, Wei Tong
Michigan State University
Overview
Introduction Related Work Unsupervised Transfer Classification
Problem Definition Approach & Analysis
Experiments Conclusions
Introduction
Classification: supervised learning semi-supervised learning
What if No label information is available? impossible but not with
some additional information
supervised
semi-supervised
unsupervised classification
Introduction
Unsupervised transfer classification (UTC) a collection of training examples and their
assignments to auxiliary classes to build a classification model for a target
class….
auxiliary class 1
auxiliary class K
target class
No Labeled training examples
prior
conditional probabilities
Introduction: Motivated Examples
Image Annotationsky
1
sun
0
1 1
0 1
water
0
0
1
0 0 1
grass?
?
?
?
Social Tagging
phone
verizon
apple
1
0 0
1 1 0
0
0
1
0
1
1
?
?
?
?
How to predict an annotation word/social tag that does not appear in the training data ?
?//// / / ?
auxiliary classes
auxiliary classes
target classestarget classes
Related Work
Transfer Learning transfer knowledge from source domain to
target domain similarity: transfer label information for
auxiliary classes to target class difference: assume NO label information for
target class
Multi-Label Learning, Maximum Entropy Model
Unsupervised Transfer Classification Data
for auxiliary class
target class
target class label
target classification model
Goal
Prior probability
conditional probabilities
Class Information
Examples
Auxiliary Classes
assignments to auxiliary classes
Maximum Entropy Model (MaxEnt)
Favor uniform
distribution
Favor uniform
distribution
Feature statistics computed
from conditional model
Feature statistics computed
from training data: the jth feature function
Generalized MaxEnt
With a large probability
Equality constraints
Inequality constraints
Generalized MaxEnt
Generalized MaxEnt
is unknown for target class is unknown for target class
How to extend generalized MaxEnt to unsupervised transfer classification ?
Estimating feature statistics of target class from those of the auxiliary classes
Unsupervised Transfer Classification
~~
Unsupervised Transfer Classification Build up Relation between Auxiliary
Classes and Target Class
Independence Assumption
Unsupervised Transfer Classification Estimating feature statistics for the
target class by regression
Feature Statistics for
Auxiliary Classes
Feature Statistics for
Auxiliary Classes
Feature Statistics for Target Class
Feature Statistics for Target Class
Class Informati
on
Class Informati
on
Unsupervised Transfer Classification Dual problem
: function of U; definition can be found in paper
Consistency Result
With a large probability
The optimal dual solution using the label information for the target class
The dual solution obtained by the proposed approach
Experiments
Text categorization Data sets: multi-labeled data
Protocol: leave one-class out as the target class
Metric: AUC (Area under ROC curve)
Experiments: Baselines
cModel train a classifier for each auxiliary class linearly combine them for the target class cLabel predict the assignment of the target class for training
examples by linearly combining the labels of auxiliary classes
train a classifier using the predicted labels for target class
GME-avg use generalized maxent model compute the feature statistics for the target class by
linearly combining those for the auxiliary classes
Proposed Approach: GME-Reg
Experiment (I)
Estimate class information from training data
Estimate class information from training data
Compare to the classifier of the target class learned by supervised learning
Experiment (I)
1500 2500
Experiment (II)
Obtain class information from external sources
Datasets: bibtex and delicious bibsonomy www.bibsonomy.org/tagsbibtex ACM DL www.portal.acm.orgbibtex deli.cio.us www.delicious.com/tag
delicious
Experiment (II)
Comparison with Supervised Classification
650
1000~1200
Conclusions
A new problem: unsupervised transfer classification
A statistical framework for unsupervised transfer classification based on generalized maximum entropy robust estimate feature statistics for target class provable performance by consistency analysis
Future Work relax independence assumption better estimation of feature statistics for target
class
Thanks
Questions ?