Introduction to Supervised Machine Learning Concepts PRESENTED BY B. Barla Cambazoglu February 21, 2014.
Post on 17-Dec-2015
214 Views
Preview:
Transcript
I n t r o d u c t i o n t o S u p e r v i s e dM a c h i n e L e a r n i n g C o n c e p t s
P R E S E N T E D B Y B . B a r l a C a m b a z o g l u F e b r u a r y 2 1 , 2 0 1 4⎪
2
Guest Lecturer’s Background
3
Lecture Outline
Basic concepts in supervised machine learning Use case: Sentiment-focused web crawling
Basic Concepts
5
What is Machine Learning?
Wikipedia: “Machine learning is a branch of artificial intelligence, concerning the construction and study of systems that can learn from data.”
Arthur Samuel: “Field of study that gives computers the ability to learn without being explicitly programmed.”
Tom M. Mitchell: “A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E.”
6
Unsupervised versus Supervised Machine Learning
Unsupervised learning› Assumes unlabeled data (the desired output is not known)› Objective is to discover the structure in the data
Supervised learning› Trained on labeled data (the desired output is known)› Objective is to generate an output for previously unseen input data
7
Supervised Machine Learning Applications
Common› Spam filtering› Recommendation and ranking› Fraud detection› Stock price prediction
Not so common› Recognize the user of a mobile device based on how he holds and moves the phone› Predict whether someone is a psychopath based on his twitter usage› Identify whales in the ocean based on audio recordings› Predict in advance whether a product launch will be successful or not
8
Terminology
Instance Label Feature Training set Test set Learning model Accuracy
Toy problem: To predict the income level of a person based on his/her facial attributes.
9
Instances
10
Categorical Labels
11
Numeric Labels
$12K$11K$9K$8K$7K$5K$1K $2K $3K $4K
12
Features
Blonde No White No Male 5cm
Bald No White Yes Male 0cm
White No Black Yes Male 3cm
Dark Yes White No Female 12cm
13
Training Set
Blonde No White No Male 5cm
Bald No White Yes Male 0cm
White No Black Yes Male 3cm
Dark Yes White No Female 12cm
14
Test Set
Dark No White No Female 14cm
Dark No White Yes Male 6cm
Dark No Black No Male 6cm
Dark Yes White No Female 15cm
15
Training and Testing
Model
Training
Testing
Prediction
Test instanceSet of training instances
16
Accuracy
Actual labels
Predicted labels
Accuracy = # of correct predictions / total number of predictions = 2 / 4 = 50%
17
Precision and Recall
In certain cases, there are two class labels and predicting a particular class correctly is more important than predicting the other.
A good example is top-k ranking in web search.
Performance measures:› Recall
› Precision
18
Some Practical Issues
Problem: Missing feature values
Solution:› Training: Use the most frequently observed (or
average) feature value in the instance’s class.
› Testing: Use the most frequently observed (or average) feature value in the entire training set.
Problem: Class imbalance
Solution› Oversampling: Duplicate the training
instances in the small class
› Undersampling: User fewer instances from the bigger class
19
Majority Classifier
Training: Find the class with the largest number of instances.
Testing: For every test instance, predict that class as the label, independent of the features of the test instance.
Model
PredictionTesting
Class
Size 13 8 4
20
k-Nearest Neighbor Classifier
Training: None! (known as a lazy classifier).
Testing: Find the k instances that are most similar to the test instance and use majority voting to decide on the label.
k = 3
21
Decision Tree Classifier
Training: Build a tree where leaves represent labels and branches represent features that lead to those labels.
Testing: Traverse the tree using the feature values of the test instance.
Black White
Black Not blackYesNo
22
Naïve Bayes Classifier
Training: For every feature value v and class c pair, we compute and store in a lookup table the conditional probability P(v | c).
Testing: For each class c, we compute:
P( | ) = 0.40
P( | ) = 0.65
P( | ) = 0.78
23
Other Commonly Used Classifiers
Support vector machines Boosted decision trees Neural networks
Use Case:Sent iment-Focused Web Crawl ing
G. Vural, B. B. Cambazoglu, and P. Senkul, “Sentiment-focused web crawling”, CIKM’12, pp. 2020-2024.
25
Problem
Early discovery of the opinionated content in the Web is important.
Use cases› Measuring brand loyalty or product adoption› Politics› Finance
We would like to design a sentiment-focused web crawler that aims to maximize the amount of sentimental/opinionated content fetched from the Web within a given amount of time.
26
Web Crawling
Subspaces› Downloaded pages› Discovered pages› Undiscovered pages
27
Sentiment-Focused Web Crawling
Challenge: to predict the sentimentality of an “unseen” web page, i.e., without having access to the page content.
28
Features
Assumption: Sentimental pages are more likely to be linked by other sentimental pages.
Idea: Build a learning model using features extracted from› Textual content of referring pages› Anchor text on the hyperlinks› URL of the target page
29
Labels
Our data (ClueWeb09-B) lacks ground-truth sentiment scores. We created a ground-truth using the SentiStrength tool.
› Assigns a sentiment score (between 0 and 8) to each web page as its label.
A small scale user-study is conducted with three judges to verify the suitability of this ground-truth.› 500 random pages sampled from the collection.› pages are labeled as sentimental or not sentimental.
Observations› 22% of the pages are labeled as sentimental.› High agreement between judges: the overlap is above 85%.
30
Learner and Performance Metric
As the learner, we use the LibSVM software in the regression mode.
We rebuild the prediction model at regular intervals throughout the crawling process.
As the main performance metric, we compute the total sentimentality score accumulated after fetching a certain number of pages.
31
Evaluated Crawlers
Proposed crawlers› based on the average
sentiment score of referring page content
› based on machine learning
Oracle crawlers› highest sentiment score› highest spam score› highest PageRank
Baseline crawlers › random› indegree-based› breadth first
32
Performance
top related