Counting sort(Non Comparison Sort)

Sorting Algorithms

2

Comparison Sorting A comparison sort is a type of sorting algorithm that only

reads the list elements through a single comparison operation (often a "less than or equal to" operator or a three-way comparison) and determines which of two elements should occur first in the final sorted list.

What is common to all these algorithms?

– Make comparisons between input elements like:

ai < aj, ai ≤ aj, ai = aj, ai ≥ aj, or ai > aj

3

Examples of Comparison Sorting1. Bubble sort

2. Insertion sort

3. Selection sort

4. Quick sort

5. Heap sort

6. Merge sort

7. Odd-even sort

8. Cocktail sort

9. Cycle sort

10. Merge insertion (Ford-Johnson) sort

11. Smoothsort

12. Timsort

4

Limitations of Comparison Sorting

There are fundamental limits on the performance of comparison sorts.

To sort n elements, comparison sorts must make (n log n) comparisons in the worst case.

That is a comparison sort must have a lower bound of (Ω n log n) comparison operations, which is known as linear

or linearithmic time. This is a consequence of the limited information available

through comparisons alone

Why does it takes so much of time? A decision tree is used to represent the comparisons of a

sorting algorithm. Assume that all inputs are distinct. A decision tree compares all possible inputs to each other to determine the sequence of outputs.

Decision Tree for three numbers a1, a2, a3 : If at the root, a1 ≤ a2 g ο left and compare a2 to a3, otherwise go to right and compare a1 to a3. Each path represents a different ordering of a1, a2, a3.

Why does it takes so much of time? This type of decision tree will have n! leaves. Any comparison based sorting algorithm will have to go

through the steps in the decision tree as a minimum

Why does it takes so much of time? So for n inputs, the tree must have n! leaves. A binary tree of

height h has no more than 2h leaves. Now n ! <= 2h

Taking log on both side: lg(n !) <= h According to stirling’s approximation we have:

So it can be written h= (Ω n log n). This means we need to do at least n log n comparisons to reach the bottom of the tree

8

How Fast Can We Sort?

• Selection Sort, Bubble Sort, Insertion Sort, Cocktail sort,

Cycle sort: O(n2)

• Smooth sort, Odd-even sort, Timsort: O(n)

• Heap Sort, Merge sort: O(n log n)

• Quicksort:

– Average: O(n log n)

– Best: O(n log n) (simple partition) or O(n) (three-way

partition and equal keys)

9

Non Comparison Sorting There are some sorting algorithms that perform sorting without

comparing the elements rather by making certain assumptions about the data. They are called the non comparison sorting.

Non comparison sorting include:

1. Counting sort (indexes using key values)

2. Radix sort (examines individual bits of keys)

3. Bucket sort (examines bits of keys) These are Linear sorting algorithms. Linear sorts are NOT

“comparison sorts”. They make certain assumptions about the data. These type of

sorting algorithm does not need to go through the comparison decision tree.

Counting sort Counting sort assumes that each of the n input elements is an

integer in the range 0 to k. that is n is the number of elements and k is the highest value element.

Consider the input set : 4, 1, 3, 4, 3. Then n=5 and k=4 Counting sort determines for each input element x, the number of

elements less than x. And it uses this information to place element x directly into its position in the output array. For example if there exits 17 elements less that x then x is placed into the 18th position into the output array.

The algorithm uses three array:

Input Array: A[1..n] store input data where A[j] {1, 2, 3, …, k}

Output Array: B[1..n] finally store the sorted data

Temporary Array: C[1..k] store data temporarily

Counting Sort1. Counting-Sort(A, B, k)

2. Let C[0…..k] be a new array

3. for i=0 to k

4. C[i]= 0;

5. for j=1 to A.length or n

6. C[ A[j] ] = C[ A[j] ] + 1;

7. for i=1 to k

8. C[i] = C[i] + C[i-1];

9. for j=n or A.length down to 1

10. B[ C[ A[j] ] ] = A[j];

11. C[ A[j] ] = C[ A[j] ] - 1;

Counting Sort1. Counting-Sort(A, B, k)

2. Let C[0…..k] be a new array

3. for i=0 to k [Loop 1]4. C[i]= 0;

5. for j=1 to A.length or n [Loop 2]6. C[ A[j] ] = C[ A[j] ] + 1;

7. for i=1 to k [Loop 3]8. C[i] = C[i] + C[i-1];

9. for j=n or A.length down to 1 [Loop 4]10. B[ C[ A[j] ] ] = A[j];

11. C[ A[j] ] = C[ A[j] ] - 1;

Counting-sort example

A:

B:

1 2 3 4 5

C:

1 2 3 4

6 7 8

5 5 3 3 0 0 2 2 3 3 0 0 3 3

0

5

1 2 3 4 5 6 7 8

Executing Loop 1

A:

B:

1 2 3 4 5

C:

1 2 3 4

6 7 8

5 5 3 3 0 0 2 2 3 3 0 0 3 3

0 0 0 0 0 0 0 0 0 0

0

0 0

5

1 2 3 4 5 6 7 8

Executing Loop 2

A:

B:

1 2 3 4 5

C:

1 2 3 4

6 7 8

5 5 3 3 0 0 2 2 3 3 0 0 3 3

0 0 0 0 1 0 0 0 0

0

0 0

5

1 2 3 4 5 6 7 8

Executing Loop 2

A:

B:

1 2 3 4 5

C:

1 2 3 4

6 7 8

5 3 3 0 0 2 2 3 3 0 0 3 3

0 0 0 0 1 1 0 0 0 0

0

1

5

1 2 3 4 5 6 7 8

Executing Loop 2

A:

B:

1 2 3 4 5

C:

1 2 3 4

6 7 8

5 5 3 0 0 2 2 3 3 0 0 3 3

0 0 0 0 1 1 1 0 0

0

1 1

5

1 2 3 4 5 6 7 8

Executing Loop 2

A:

B:

1 2 3 4 5

C:

1 2 3 4

6 7 8

5 5 3 3 0 2 2 3 3 0 0 3 3

1 0 0 1 1 1 1 0 0

0

1 1

5

1 2 3 4 5 6 7 8

Executing Loop 2

A:

B:

1 2 3 4 5

C:

1 2 3 4

6 7 8

5 5 3 3 0 0 2 3 3 0 0 3 3

1 1 0 0 2 1 1 0 0

0

1 1

5

1 2 3 4 5 6 7 8

Executing Loop 2

A:

B:

1 2 3 4 5

C:

1 2 3 4

6 7 8

5 5 3 3 0 0 2 2 3 0 0 3 3

1 1 0 0 2 2 2 0 0

0

1 1

5

1 2 3 4 5 6 7 8

Executing Loop 2

A:

B:

1 2 3 4 5

C:

1 2 3 4

6 7 8

5 5 3 3 0 0 2 2 3 3 0 3 3

2 0 0 2 2 2 2 0 0

0

1 1

5

1 2 3 4 5 6 7 8

Executing Loop 2

A:

B:

1 2 3 4 5

C:

1 2 3 4

6 7 8

5 5 3 3 0 0 2 2 3 3 0 0 3

2 2 0 0 2 2 3 0 0

0

1 1

5

1 2 3 4 5 6 7 8

End of Loop 2

A:

B:

1 2 3 4 5

C:

1 2 3 4

6 7 8

5 5 3 3 0 0 2 2 3 3 0 0

2 2 0 0 2 2 0 0

0

1 1

5

1 2 3 4 5 6 7 8

3 3

3 3

Executing Loop 3

A:

B:

1 2 3 4 5

C:

1 2 3 4

6 7 8

5 5 3 3 0 0 2 2 3 3 0 0

2 0 2 2 0 0 0

1 1 5

1 2 3 4 5 6 7 8

3 3

3 3

C: 2 2 0 0 1 1 3 3

1 2 3 40 5

2 2 2

Executing Loop 3

A:

B:

1 2 3 4 5

C:

1 2 3 4

6 7 8

5 5 3 3 0 0 2 2 3 3 0 0

2 2 2 2 0 0 0

1 1 5

1 2 3 4 5 6 7 8

3 3

3 3

C: 2 2 4 0 0 1 1 3 3

1 2 3 40 5

2 2

Executing Loop 3

A:

B:

1 2 3 4 5

C:

1 2 3 4

6 7 8

5 5 3 3 0 0 2 2 3 3 0 0

4 2 2 0 0 0

1 1 5

1 2 3 4 5 6 7 8

3

3 3

C: 2 2 4 4 0 0 1 1 7

1 2 3 40 5

2 2

2 2

Executing Loop 3

A:

B:

1 2 3 4 5

C:

1 2 3 4

6 7 8

5 5 3 3 0 0 2 2 3 3 0 0

2 2 0 0

1 1 5

1 2 3 4 5 6 7 8

7

3 3

C: 2 2 4 4 7 1 1

1 2 3 40 5

2 2

2 2

4 4

7 7

Executing Loop 3

A:

B:

1 2 3 4 5

C:

1 2 3 4

6 7 8

5 5 3 3 0 0 2 2 3 3 0 0

2 2

0

1

5

1 2 3 4 5 6 7 8

3 3

C: 2 2 4 4 8

1 2 3 40 5

2 2

2 2

4 4

7 7

7 7 7

7 7

End of Loop 3

A:

B:

1 2 3 4 5

6 7 8

5 5 3 3 0 0 2 2 3 3 0 0

1 2 3 4 5 6 7 8

3 3

C: 2 2 4 4

1 2 3 40 5

2 2 7 7 7 7 8 8

Executing Loop 4

A:

B:

1 2 3 4 5

6 7 8

5 5 3 3 0 0 2 2 3 3 0 0

1 2 3 4 5 6 7 8

3

C: 2 2 4 4

1 2 3 40 5

2 2 7 7 7 8 8

Executing Loop 4

A:

B:

1 2 3 4 5

6 7 8

5 5 3 3 0 0 2 2 3 3 0 0

1 2 3 4 5 6 7 8

C: 2 2 4 4

1 2 3 40 5

2 2 7 7 7 8 8

3

J=8, then A[ j ]=A[8]=3And B[ C[ A[j] ] ]=B[ C[ 3 ] ]=B[ 7]So B[ C[ A[j] ] ] ←A[ j ]=B[7]←3

A:

B:

1 2 3 4 5

6 7 8

5 5 3 3 0 0 2 2 3 3

1 2 3 4 5 6 7 8

C: 2 4 4

1 2 3 40 5

2 2 7 7 8 8

3 3 0

6 6

3 3

Executing Loop 4

J=8, then A[ j ]=A[8]=3Then C[ A[j] ] = C[ 3 ] =7So C[ A[j] ] = C[ A[j] ] -1=7-1=6

Executing Loop 4

A:

B:

1 2 3 4 5

6 7 8

5 5 3 3 0 0 2 2

1 2 3 4 5 6 7 8

C: 4 4

1 2 3 40 5

2 2 7 7 8 8

3 3

6

3 3

0 0

1 1

0 0

3

Executing Loop 4

A:

B:

1 2 3 4 5

6 7 8

5 5 3 3 0 0

1 2 3 4 5 6 7 8

C: 4

1 2 3 40 5

2 2 7 7 8 8

3 3

5 5

3 3

0 0

0 0

2

1 1

3 3

3 3

Executing Loop 4

A:

B:

1 2 3 4 5

6 7 8

5 5 3 3

1 2 3 4 5 6 7 8

C: 3 3

1 2 3 40 5

2 2 7 7 8 8

3 3

3 3

0 0

0 0

0

1

3 3

3 3

5 5

2 2

Executing Loop 4

A:

B:

1 2 3 4 5

6 7 8

5 5

1 2 3 4 5 6 7 8

C:

1 2 3 40 5

2 2 7 7 8 8

3 3

3 3

0 0

0 0

0

0 0

3 3

3 3

5 3 3

0 0 2 2

3

Executing Loop 4

A:

B:

1 2 3 4 5 6 7 8

1 2 3 4 5 6

7 8

C:

1 2 3 40 5

2 2 7 7 8

3 3

3 3

0 0

0 0

0 3 3

3 3

3 3

0 0 2 2

3 3 5

4 4

3 3

0 0

Executing Loop 4

A:

B:

1 2 3 4 5 6 7 8

1 2 3 4 5 6

7 8

C:

1 2 3 40 5

2 2 7 7 7 7

3 3

3 3

0 0

0 0

0 3 3

3 3

3

0 0 2 2

3 3 2

3 3

0 0

5 5

4 4

5 5

End of Loop 4

A:

B:

1 2 3 4 5 6 7 8

1 2 3 4 5 6 7 8

C:

1 2 3 40 5

2 2 7 7

3 3

3 3

0 0

0 0

0 3 3

3 3 0 0 2 2

3 3

3 3

0 0

5 5

4 4

5 5

2 2 7 7

2 2

Sorted data in Array B

Time Complexity Analysis1. Counting-Sort(A, B, k)

2. Let C[0…..k] be a new array

3. for i=0 to k [Loop 1]4. C[i]= 0;

5. for j=1 to A.length or n [Loop 2]6. C[ A[j] ] = C[ A[j] ] + 1;

7. for i=1 to k [Loop 3]8. C[i] = C[i] + C[i-1];

9. for j=n or A.length down to 1 [Loop 4]10. B[ C[ A[j] ] ] = A[j];

11. C[ A[j] ] = C[ A[j] ] - 1;

Loop 2 and 4 takes O(n) time

Loop 1 and 3 takes O(k) time

Time Complexity Analysis

• So the counting sort takes a total time of: O(n + k)• Counting sort is called stable sort.– A sorting algorithm is stable when numbers with

the same values appear in the output array in the same order as they do in the input array.

Counting Sort Review

• Why don’t we always use counting sort?– Depends on range k of elements.

• Could we use counting sort to sort 32 bit integers? Why or why not?

Counting Sort Review• Assumption: input taken from small set of numbers of size k• Basic idea: – Count number of elements less than you for each element.– This gives the position of that number – similar to selection

sort.• Pro’s:– Fast– Asymptotically fast - O(n+k)– Simple to code

• Con’s:– Doesn’t sort in place.– Requires O(n+k) extra storage.

MD. Shakhawat Hossain

Student of Computer Science & Engineering Dept.

University of Rajshahi