Data structures and Algorithms Sorting

Data structures and AlgorithmsSorting

Pham Quang Dung

Hanoi, 2012

Pham Quang Dung () Data structures and Algorithms Sorting Hanoi, 2012 1 / 38

Outline

1 Introduction to Sorting

2 Insertion Sort

3 Selection Sort

4 Bubble Sort

5 Merge Sort

6 Quick Sort

7 Heap Sort

8 Counting Sort

9 Radix Sort

10 Bucket Sort


Introduction to sorting

Put elements of a list in a certain order

Designing efficient sorting algorithms is very important for otheralgorithms (search, merge, etc.)

Each object is associated with a key and sorting algorithms work onthese keys.

Two basic operations that used mostly by sorting algorithms

Swap(a, b): swap the values of variables a and bCompare(a, b): return

true if a is before b in the considered orderfalse, otherwise.

Without loss of generality, suppose we need to sort a list of numbersin nondecreasing order


A sorting algorithm is called in-place if the size of additional memoryrequired by the algorithm is O(1) (which does not depend on the sizeof the input array)

A sorting algorithm is called stable if it maintains the relative orderof elements with equal keys

A sorting algorithm uses only comparison for deciding the orderbetween two elements is called Comparison-based sortingalgorithm


Outline


2 Insertion Sort

3 Selection Sort

4 Bubble Sort

5 Merge Sort

6 Quick Sort

7 Heap Sort

8 Counting Sort

9 Radix Sort

10 Bucket Sort


Insertion Sort

At iteration k , put the kth element of the original list in the rightorder of the sorted list of the first k elements (∀k = 1, . . . , n)

Result: after kth iteration, we have a sorted list of the first kth

elements of the original list

1 vo i d i n s e r t i o n s o r t ( i n t a [ ] , i n t n ) {i n t k ;

3 f o r ( k = 2 ; k <= n ; k++){i n t l a s t = a [ k ] ;

5 i n t j = k ;wh i l e ( j > 1 && a [ j −1] > l a s t ) {

7 a [ j ] = a [ j −1] ;j−−;

9 }a [ j ] = l a s t ;

11 }}

Listing 1: insertion sort


Outline


2 Insertion Sort

3 Selection Sort

4 Bubble Sort

5 Merge Sort

6 Quick Sort

7 Heap Sort

8 Counting Sort

9 Radix Sort

10 Bucket Sort


Selection Sort

Put the smallest element of the original list in the first position

Put the second smallest element of the original list in the secondposition

Put the third smallest element of the original list in the third position

...

1 vo i d s e l e c t i o n s o r t ( i n t a [ ] , i n t n ) {f o r ( i n t k = 1 ; k <= n ; k++){

3 i n t min = k ;f o r ( i n t i = k+1; i <= n ; i++)

5 i f ( a [ min ] > a [ i ] )min = i ;

7 swap ( a [ k ] , a [ min ] ) ;}

9 }

Listing 2: selection sort


Outline


2 Insertion Sort

3 Selection Sort

4 Bubble Sort

5 Merge Sort

6 Quick Sort

7 Heap Sort

8 Counting Sort

9 Radix Sort

10 Bucket Sort


Bubble sort

Pass from the beginning of the list: compare and swap two adjacentelements if they are not in the right order

Repeat the pass until no swaps are needed

1 vo i d b u b b l e s o r t ( i n t a [ ] , i n t n ) {i n t swapped ;

3 do{swapped = 0 ;

5 f o r ( i n t i = 1 ; i < n ; i++)i f ( a [ i ] > a [ i +1]){

7 swap ( a [ i ] , a [ i +1]) ;swapped = 1 ;

9 }}wh i l e ( swapped == 1) ;

11 }

Listing 3: bubble sort


Outline


2 Insertion Sort

3 Selection Sort

4 Bubble Sort

5 Merge Sort

6 Quick Sort

7 Heap Sort

8 Counting Sort

9 Radix Sort

10 Bucket Sort


Merge sort

Divide-and-conquer

Divide the original list of n/2 into two lists of n/2 elements

Recursively merge sort these two lists

Merge the two sorted lists


Merge sort

vo i d merge ( i n t a [ ] , i n t L , i n t M, i n t R) {2 // merge two s o r t e d l i s t a [ L . .M] and a [M+1. .R ]

i n t i = L ;// f i r s t p o s i t i o n o f the f i r s t l i s t a [ L . .M]4 i n t j = M+1;// f i r s t p o s i t i o n o f the second l i s t a [M+1. .R ]

f o r ( i n t k = L ; k <= R; k++){6 i f ( i > M) {// the f i r s t l i s t i s a l l scanned

TA[ k ] = a [ j ] ; j ++;8 } e l s e i f ( j > R) {// the second l i s t i s a l l scanned

TA[ k ] = a [ i ] ; i ++;10 } e l s e {

i f ( a [ i ] < a [ j ] ) {12 TA[ k ] = a [ i ] ; i ++;

} e l s e {14 TA[ k ] = a [ j ] ; j ++;

}16 }

}18 f o r ( i n t k = L ; k <= R; k++)

a [ k ] = TA[ k ] ;20 }


Merge sort

vo i d me rg e s o r t ( i n t a [ ] , i n t L , i n t R) {2 i f ( L < R) {

i n t M = (L+R) /2 ;4 merge s o r t ( a , L ,M) ;

me rg e s o r t ( a ,M+1,R) ;6 merge ( a , L ,M,R) ;

}8 }


Outline


2 Insertion Sort

3 Selection Sort

4 Bubble Sort

5 Merge Sort

6 Quick Sort

7 Heap Sort

8 Counting Sort

9 Radix Sort

10 Bucket Sort


Quick sort

Pick an element, called a pivot, from the original list

Rearrange the list so that:

All elements less than pivot come before the pivotAll elements greater or equal to to pivot come after pivot

Here, pivot is in the right position in the final sorted list (it is fixed)

Recursively sort the sub-list before pivot and the sub-list after pivot


Quick sort

vo i d q u i c k s o r t ( i n t a [ ] , i n t L , i n t R) {2 i f ( L < R) {

i n t i nd e x = (L+R) /2 ;4 i n d e x = p a r t i t i o n ( a , L ,R , i nd e x ) ;

i f ( L < i n d e x )6 q u i c k s o r t ( a , L , index −1) ;

i f ( i nd e x < R)8 q u i c k s o r t ( a , i nd e x +1,R) ;

}10 }

Listing 4: Quick sort algorithm


Quick sort

1 i n t p a r t i t i o n ( i n t a [ ] , i n t L , i n t R , i n t i n d e xP i v o t ) {i n t p i v o t = a [ i n d e xP i v o t ] ;

3 swap ( a [ i n d e xP i v o t ] , a [R ] ) ; // put the p i v o t i n the end o f the l i s ti n t s t o r e I n d e x = L ; // s t o r e the r i g h t p o s i t i o n o f p i v o t at the

end o f the p a r t i t i o n p rocedu r e5

f o r ( i n t i = L ; i <= R−1; i++){7 i f ( a [ i ] < p i v o t ) {

swap ( a [ s t o r e I n d e x ] , a [ i ] ) ;9 s t o r e I n d e x++;

}11 }

swap ( a [ s t o r e I n d e x ] , a [R ] ) ; // put the p i v o t i n the r i g h t p o s i t i o nand r e t u r n t h i s p o s i t i o n

13

r e t u r n s t o r e I n d e x ;15 }

Listing 5: partition


Outline


2 Insertion Sort

3 Selection Sort

4 Bubble Sort

5 Merge Sort

6 Quick Sort

7 Heap Sort

8 Counting Sort

9 Radix Sort

10 Bucket Sort


Heap sort

Sort a list A[1..N] in nondecreasing order

1 Build a heap out of A[1..N]

2 Remove the largest element and put it in the Nth position of the list

3 Reconstruct the heap out of A[1..N − 1]

4 Remove the largest element and put it in the N − 1th position of thelist

5 ...


Heap sort - Heap structure

Shape property: Complete binary tree with level L

Heap property: each node is greater than or equal to each of itschildren (max-heap)

10

9 7

4 5 6 1

3 2 1 2 3 4 5 6 7 8 9

10 9 7 4 5 6 1 3 2


Heap sort

Heap corresponding to a list A[1..N]

Root of the tree is A[1]Left child of node A[i ] is A[2 ∗ i ]Right child of node A[i ] is A[2 ∗ i + 1]Height is logN + 1

Operations

Build-Max-Heap: construct a heap from the original listMax-Heapify: repair the following binary tree so that it becomesMax-Heap

A tree with root A[i ]A[i ] < max(A[2 ∗ i ],A[2 ∗ i + 1]): heap property is not holdSubtrees rooted at A[2 ∗ i ] and A[2 ∗ i + 1] are Max-Heap


Heap sort

vo i d h e a p i f y ( i n t a [ ] , i n t i , i n t n ) {2 // a r r a y to be h e a p i f i e d i s a [ i . . n ]

i n t L = 2∗ i ;4 i n t R = 2∗ i +1;

i n t max = i ;6 i f ( L <= n && a [ L ] > a [ i ] )

max = L ;8 i f (R <= n && a [R ] > a [max ] )

max = R;10 i f (max != i ) {

swap ( a [ i ] , a [ max ] ) ;12 h e a p i f y ( a , max , n ) ;

}14 }

Listing 6: heapify


Heap sort

1 vo i d bu i ldHeap ( i n t a [ ] , i n t n ) {// a r r a y i s a [ 1 . . n ]

3 f o r ( i n t i = n /2 ; i >= 1 ; i−−){h e a p i f y ( a , i , n ) ;

5 }}

7

vo i d heap So r t ( i n t a [ ] , i n t n ) {9 // a r r a y i s a [ 1 . . n ]

bu i l dHeap ( a , n ) ;11 f o r ( i n t i = n ; i > 1 ; i−−){

swap ( a [ 1 ] , a [ i ] ) ;13 h e a p i f y ( a , 1 , i −1) ;

}15 }

Listing 7: heapSort


Outline


2 Insertion Sort

3 Selection Sort

4 Bubble Sort

5 Merge Sort

6 Quick Sort

7 Heap Sort

8 Counting Sort

9 Radix Sort

10 Bucket Sort


Counting sort

Input: array of n integers a1, . . . , an, where 0 6= ai 6= k with k = O(n)

Main idea:

For each element x , compute the rank r [x ] as number of elements ofthe array which is less than or equal to xPlace x at position r [x ]


Counting sort


Counting sort

1 vo i d c oun t i n gSo r t ( i n t a [ ] , i n t r [ ] , i n t c [ ] , i n t n , i n t k ) {// a [ 1 . . n ] i s the a r r a y to be s o r t e d

3 // n i s the number o f e l ement s o f the a r r a y a// k i s the maximum of a , and 0 i s minimum of a

5

// count r [ i ] − the number o f e l ement s o f a hav ing v a l u e i7 f o r ( i n t i = 0 ; i <= k ; i++) r [ i ] = 0 ;

f o r ( i n t i = 1 ; i <= n ; i++) r [ a [ i ] ]++;9 // compute rank r [ x ] o f x

f o r ( i n t x = 1 ; x <= k ; x++) r [ x ] = r [ x ] + r [ x−1] ;11

// s o r t13 f o r ( i n t i = n ; i >= 1 ; i−−){

c [ r [ a [ i ] ] ] = a [ i ] ; / / p l a c e a [ i ] i n i t s r i g h t p o s i t i o n15 r [ a [ i ] ] = r [ a [ i ] ] −1 ;// reduce rank o f a [ i ] by one

}17 }


Outline


2 Insertion Sort

3 Selection Sort

4 Bubble Sort

5 Merge Sort

6 Quick Sort

7 Heap Sort

8 Counting Sort

9 Radix Sort

10 Bucket Sort


Radix sort

Not comparison-based sorting algorithm

Each key (integer) is represented by a sequence of d numerical digitsin a given radix (e.g., radix is 10: adad−1 . . . a1)

Principle

Take the least significant digit of each keyGroup the keys based on that digit while keep the original order of keys(stable sort)Repeat the grouping process with each more significant digit


Radix sort

Algorithm 1: RadixSort(A, d)

1 foreach i ∈ 1..d do2 Sort A based on the i th digits of keys using stable sort;


Radix sort

Example

A[1..10] = 2980, 0020, 0242, 3002, 1145, 1045, 2626, 1005, 3180, 4146

1 Step 1: 2980,0020,3180,0242,3002,1145,1045,1005,2626,4146

2 Step 2: 3002,1005,0020,2626,0242,1145,1045,4146,2980,3180

3 Step 3: 3002,1005,0020,1045,1145,4146,3180,0242,2626,2980

4 Step 4: 0020,0242,1005,1045,1145,2626,2980,3002,3180,4146


Radix sort

vo i d r a d i x s o r t 1 0 ( l ong a [ ] , i n t n ) {2 i n t max = −10000;

f o r ( i n t i = 0 ; i < n ; i++) i f (max < a [ i ] ) max = a [ i ] ;4 l ong tmp [MAX SIZE ] ;

i n t exp = 1 ;6 wh i l e (max/ exp > 0) {

i n t b i n s z [ 1 0 ] = {0} ;8 i n t i d x [ 1 0 ] ;

f o r ( i n t i = 0 ; i < n ; i++){10 i n t c = ( a [ i ] / exp ) % 10 ;

b i n s z [ c ]++;12 tmp [ i ] = a [ i ] ;

}14 i d x [ 0 ] = 0 ;

f o r ( i n t i = 1 ; i < 10 ; i++)16 i d x [ i ] = i d x [ i −1] + b i n s z [ i −1] ;

f o r ( i n t i = 0 ; i < n ; i++){18 i n t c = ( tmp [ i ] / exp )%10 ;

a [ i d x [ c ] ] = tmp [ i ] ;20 i d x [ c ]++;

}22 exp = exp ∗10 ;

}24 }


Outline


2 Insertion Sort

3 Selection Sort

4 Bubble Sort

5 Merge Sort

6 Quick Sort

7 Heap Sort

8 Counting Sort

9 Radix Sort

10 Bucket Sort


Bucket sort

Assume a uniform distribution on the input

The input (array A[1..n]) is generated by a random process thatdistributes uniformly and independently over the interval [0,1)

Main idea

Interval [0,1) is divided into n equal-size subintervals (or buckets)Distribute A[1..n] into the bucketsSort elements in each bucketConcatenate the sorted lists of the buckets in order to establish thesorted list of the original array


Bucket sort

Algorithm 2: BUCKET-SORT(A)

1 Let B[0..n − 1] be a new array;2 foreach i = 0, . . . , n − 1 do3 Make B[i] an empty list;

4 foreach i = 1..n do5 Insert A[i ] into list B[bnA[i ]c];6 foreach i = 0, . . . , n − 1 do7 Sort list B[i ] with insertion sort;

8 Concatenate the list B[0],B[1], . . . ,B[n − 1] together in order;


Bucket sort - Analysis

Analysis the average-case running time

Let ni be the random variable denoting the number of element placedin bucket B[i ]

T (n) = Θ(n) +∑n−1

i=0 O(n2i )

Average-case running time is E [T (n)] = E [Θ(n) +∑n−1

i=0 O(n2i )] =Θ(n) +

∑n−1i=1 O(E [n2i ])

We’ll prove that E [n2i ] = 2− 1n (next slide)


Bucket sort - Analysis

Xij =

{1, if A[j ] falls in bucket i0, otherwise

ni =∑

j = 1nXij

E [n2i ] = E [(∑n

j=1 Xij)2] =

E [∑n

j=1 X2ij +

∑1≤j≤n

∑k∈{1,...,n}\{j} XijXik ] =∑n

j=1 E [X 2ij ] +

∑1≤j≤n

∑k∈{1,...,n}\{j} E [XijXik ]

Xij is 1 with probability 1n and 0 otherwise, therefor

E [X 2ij ] = 12 ∗ 1

n + 02 ∗ (1− 1n ) = 1

n

When k 6= j , Xij and Xik are independent, henceE [Xij ∗ Xik ] = E [Xij ]E [Xik ] = 1

n ∗1n = 1

n2

Finally, we haveE [n2i ] = 2− 1

n ⇒ E [T (n)] = Θ(n) + n ∗ O(2− 1n ) = Θ(n)


Data structures and Algorithms Sorting

Documents