Current clustering techniques

Guided By: Presented By:Prof. Prashant G. Ahire Miss.Poonam Kshirsagar

Roll No. 204

A SURVEY OF CLUSTERING TECHNIQUES

FOR BIG DATA ANALYSIS

Agenda

Problem Definition ObjectiveLiterature Survey Big Data and it’s Analytics Challenges Cluster Criterion To Benchmark Clustering Methods Proposed System ELM ELM Feature Mapping Process ELM K-mean Algorithm Advantages Disadvantages Conclusion

Problem Definition:

Among various challenges in analyzing big data the major issue is to design and develop the new techniques for clustering.

Cloud computing can be used for big data analysis but there is problem to analyze data on cloud environment as many traditional algorithms cannot be applied directly on cloud environment and also there is an issue of applying scalability on traditional algorithms, delay in result produced and accuracy of result produced.

These issues can be addressed by clustering techniques.

Objectives:

The objectives of the thesis are as follows:

To study the existing clustering techniques for analyzing big data.

To propose and design an efficient clustering technique for big

data analysis.

Literature Survey:Topic Name Keywords Abstract Author Name

A Survey of Clustering Algorithms for Big Data: Taxonomy and Empirical Analysis

Clustering algorithms, unsupervised learning, big data

we highlighted the set of clustering algorithms that are the best performing for big data.

ADIL FAHAD 1,4 , NAJLAA ALSHATRI 1 , ZAHIR TARI 1 , (Member, IEEE)

Clustering in extreme learning machine feature space

ELM means The good properties of the ELM feature mapping, the clustering problem using ELM feature mapping techniques is studied in this paper.

Qing He a, n , Xin Jin a,b , Changying Du a,b , Fuzhen Zhuang a

A Hybrid Approach for Efficient Clustering of Big Data

big data, Basic K-Means Algorithm using MapReduce,, Basic DBSCAN Algorithm using MapReduce

This is presents a theoretical overview of some of current clustering techniques used for analyzing big data

Saurabh Arora, Department of Computer Science and Engineering ,Thapar UniversityPatiala,India,

A Survey of Clustering Techniques for Big Data Analysis

Big data, Clustering Techniques, Data Mining

In this paper we have discussed some of the current big data mining clustering techniques.

Saurabh Arora, Inderveer, dept. of CS

What is Big data ?

Big Data means data that’s too big, too fast , or too hard for existing tools to process.

Too big : Peta byte-scale collection of data.

Too fast: Processed quickly.

Too hard: It is a catch all for data that doesn’t fit neatly into an existing processing tools.

Fig. Evolution Of big data

Continue…

Fig.: Significant growth of Big data

Continue…

Big Data Analytics Challenges:

The main challenges for big data analytics are listed below :

Volume of data is large and also varies so challenge is how to deal with it.

Analysis of all data is required or not.

All data needs to be stored or not.

To analyze which data points are important and how to find them.

How data can be used in the best way.

What is Cluster ?

Clustering is a division of data into groups of similar objects.

Each group, called cluster.

Cluster consists of objects that are similar between themselves

and dissimilar to objects of other groups.

It is one of the major techniques used for data mining.

Criterion To Benchmark Clustering Methods:

Volume : Refers large amount of data Criteria: (i) Size of the dataset

(ii) Handling high dimensionality (iii) Handling outliers/ noisy data

Velocity : Refers speed of processing data. Criteria : (i) Complexity of algorithm (ii) The run time performances

Variety: Refers to the ability to handle different types of data (i) Type of dataset

(ii) Clusters shape.

Comparative Analysis of Current Clustering Techniques

Partition Clustering Techniques

1.K-mean and variant partitioning techniques: Example : K-MCI algorithm

2.Other Partitioning Techniques:Example : Cuckoo search

Hierarchical Clustering Techniques

Example : ACA-DTRS FACA-DTRS

Density Based Clustering Techniques

Example : DMM clustering algorithm DBCURE Algorithm

Generic Clustering Techniques:

Example : BRICH Algorithm

Proposed System:

In the partitioning clustering techniques K-Means is being

used for past so many years.

Now days but ELM K-means or ELM FCM is best suited

among all

Methods as it finds best quality clusters and in less

computation time.

ELM feature is easy to implement and it works well for

big datasets.

Fast learning speed.

Ease of implementation.

Minimal human intervention.

ELM tends to have better scalability.

Extreme Learning Machine

ELM Feature Mapping Process

Where,1. G(ai,bi,x) is the output of the i th hidden node 2. ai is a d-dimensional weight vector between the d input nodes and the i th hidden-node 3. bi is the bias of ith hidden-node.

ELM will map the data into the L-dimensional ELM feature space H, and L is the number of the hidden nodes used in the feature mapping process

Fig.: ELM Feature Mapping Process

Continue…

• K-Means clustering problem can be described as follows:

• Given a set of observations (x1,x2,……xm) where each observation is a d-dimensional real vector

• k-Means clustering aims to partition the m observations into k sets

• so as to minimize the within-cluster sum of squares (WCSSs):

Where,

μi is theme an of point sin Si.

Continue…

ELM k-Means algorithm Input: k : the number of clusters, L : the number of the hidden-layer nodes, D : a data set containing m objects. Output : A set of k clusters. Method : 1: Mapping the original data object sin D into the ELM feature space H using h(x)=[H1(x),….,hi(x),…hl(x)]T ;2: Arbitrarily choose k objects from H as the initial cluster centres; 3: repeat 4: (Re) assign each object to the cluster to which the object is the most similar , based on the mean value of the object sin the cluster; 5: Update the cluster means , i.e. , calculate the mean value of the objects for each cluster; 6: until no change in the cluster centres or reached the maximal iteration number limit.7: return A set of k clusters.

Advantages:

ELM features are easy to implement and ELM K-means produce better results than Mercer kernel based methods.

The mapping is very intuitive and straight forward

Disadvantages

Number of nodes should be greater than 300 else

performance is not optimal.

After studying these techniques it is observed

that still new methodologies are required for

analyzing big data as these techniques could are

not so efficient for analyzing real time and online

streaming data

Conclusion:

we have studied various clustering techniques

which are currently used for analyzing big data. All

these recent techniques are compared on the basis of

execution time and cluster quality and their merits

and demerits are provided.