Machine Learning Introduction Machine learning All the techniques that we have seen until now allow us to build intelligent systems The limitation of these systems is that they only can solve the problems their are programmed for But we only should consider a system intelligent if is also able to observe its environment and learn from it The real intelligence resides in adaptation, to be able to integrate new knowledge, to solve new problems, to learn from mistakes BY: $ \ C (LSI-FIB-UPC) Artificial Intelligence Term 2009/2010 1 / 28
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Machine Learning Introduction
Machine learning
All the techniques that we have seen until now allow us to buildintelligent systems
The limitation of these systems is that they only can solve theproblems their are programmed for
But we only should consider a system intelligent if is also able toobserve its environment and learn from it
The real intelligence resides in adaptation, to be able to integrate newknowledge, to solve new problems, to learn from mistakes
It obtains general knowledge from specific informationThe knowledge obtained is newIts not truth preserving (new information can invalidate the knowledgeobtained)It has not well founded theory
Deductive reasoning
It obtains general knowledge from general knowledgeThe knowledge is not new (it is implicit in the initial knowledge)New knowledge can not invalidate the knowledge already obtainedIts basis is mathematical logic
Each example is labeled with the concept it belongs toLearning is performed by contrast among conceptsA set of heuristics allows to generate different hypothesisThere is a criteria of preference (bias) that allows to choose the mostsuitable hypothesis for the examplesResult: The concept or concepts that describe better the examples
Examples are not labeledWe want to discover a suitable way to cluster the objectsLearning is based on the discovery of similarity/dissimilarity amongexamplesA heuristic preference criteria will guide the searchResult: A partition of the examples and a characterization of thepartitions
One of the first algorithms for building decision trees is ID3 (Quinlan1986)
It is in the family of algorithms for Top Down Induction DecisionTrees (TDIDT)
ID3 performs a search using a Hill-Climbing strategy in the space ofdecision trees
For each level of the tree an attribute is chosen and the set ofexamples is split using the values of the attribute. This process isrepeated recursively for each partition
The selection of the attribute is performed using an heuristic function
Information theory studies among other things the coding of messagesand the cost of their transmition
If we define a set of messages M = {m1,m2, ...,mn}, each one withprobability P(mi ), we can define the quantity of information (I ) thata message M contains as:
I (M) =n∑
i=1
−P(mi )log(P(mi ))
This value can be interpreted as the information needed todiscriminate the messages from M (Number of bit necessary to codethe messages)
Compute the quantity of information of the examples (I)foreach attribute in A do
Compute the entropy (E) and the information gain (G)
Pick the attribute that maximizes G (a)Delete a from the list of attributes (A)Generate a root node for the attribute aforeach partition generated by the values of the attribute a do
Treei=ID3(X (a=vi ), C(a=vi ),A-a)generate a new branch with a=vi and Treei
1 Blue Blonde Tall +2 Blue Dark Medium +3 Brown Dark Medium −4 Green Dark Medium −5 Green Dark Tall +6 Brown Dark Small −7 Green Blonde Small −8 Blue Dark Medium +