Quick Sort Cmput 115 - Lecture 13 Department of Computing Science University of Alberta ©Duane Szafron 2000 Some code in this lecture is based on code.

Quick SortQuick Sort

Cmput 115 - Lecture 13

Department of Computing Science

University of Alberta©Duane Szafron 2000

Some code in this lecture is based on code from the book:Java Structures by Duane A. Bailey or the companion structure package

Revised 1/26/00

©Duane Szafron 1999

2

About This LectureAbout This Lecture

In this lecture we will learn about a sorting algorithm called the Quick Sort.

We will study its implementation and its time and space complexity.

©Duane Szafron 1999

3

OutlineOutline

The Quick Sort Algorithm

Quick Sort - Arrays

Time and Space Complexity of Quick Sort

©Duane Szafron 1999

4

The Sort ProblemThe Sort Problem

Given a collection, with elements that can be compared, put the elements in increasing or decreasing order.

60 30 10 20 40 90 70 80 500 1 2 3 4 5 6 7 8

10 20 30 40 50 60 70 80 900 1 2 3 4 5 6 7 8

©Duane Szafron 1999

5

Quick Sort Algorithm 0Quick Sort Algorithm 0

Our initial goal is to move one element, called the pivot, to its correct final position so that all elements to the left of it are smaller than it and all elements to the right of it are larger than it.

We will call this operation partition().

We select the left element as the pivot.

60 30 10 20 40 90 70 80 500 1 2 3 4 5 6 7 8

rlp

©Duane Szafron 1999

6

Quick Sort Algorithm 1Quick Sort Algorithm 1

Find the rightmost element that is smaller than the pivot element.

Exchange the elements and increment the left.

60 30 10 20 40 90 70 80 500 1 2 3 4 5 6 7 8

lp rr

50 30 10 20 40 90 70 80 600 1 2 3 4 5 6 7 8

rl p

©Duane Szafron 1999

7

Quick Sort Algorithm 2Quick Sort Algorithm 2

Find the leftmost element that is larger than the pivot element.

Exchange the elements and decrement the right.

50 30 10 20 40 60 70 80 900 1 2 3 4 5 6 7 8

lp r

50 30 10 20 40 90 70 80 600 1 2 3 4 5 6 7 8

rl pl

©Duane Szafron 1999

8

Quick Sort Algorithm 3Quick Sort Algorithm 3

Find the rightmost element that is smaller than the pivot element.

Since the right passes the left, there is no element and the pivot is the final location.

50 30 10 20 40 60 70 80 900 1 2 3 4 5 6 7 8

r lp r

©Duane Szafron 1999

9

Quick Sort Algorithm 4Quick Sort Algorithm 4

To complete the sort, recursively partition the sub-list to the left of the pivot and the sub-list to the right of the pivot.

©Duane Szafron 1999

10

QuickSort Algorithm - ArraysQuickSort Algorithm - Arrays

public static void quickSort(Comparable anArray[ ], int size) {

// pre: 0 <= size <= anArray.length// post: values in anArray[0..size - 1] are in// ascending order

quickSortR(anArray, 0, size - 1);}

©Duane Szafron 1999

11

QuickSortR Algorithm - ArraysQuickSortR Algorithm - Arrays

private static void quickSortR(Comparable anArray[], int left, int right) {

// pre: left <= right// post: anArray[left..right] are ascending

int pivot;

if (left >= right)return;

pivot = partition(anArray, left, right);quickSortR(anArray, left, pivot - 1);quickSortR(anArray, pivot + 1, right);

}

©Duane Szafron 1999

12

Partition Algorithm - ArraysPartition Algorithm - Arraysprivate static int partition(Comparable anArray[], int left, int right) {// pre: left <= right// post: data[left] is in the correct sort location and its // index is returned.

while (true) {while ((left < right) &&

(anArray[left].compareTo(anArray[right]) < 0))right--; //find rightmost element less than left

if (left < right) swap(anArray, left++, right);else return left; //pivot is now in final locationwhile ((left < right) &&

(anArray[left].compareTo(anArray[right]) < 0))left++; //find leftmost element greater than right

if (left < right) swap(anArray, left, right--);else return right; //pivot is now in final location

} }

code based on Bailey pg. 89

©Duane Szafron 1999

13

QuickSort Calling SequenceQuickSort Calling Sequence

qSR(a, 0, 8)– p(a,0,8) -> 5– qSR(a, 0,4)

• p(a,0,4) -> 4• qSR(a,0,3)

– p(a,0,3) -> 3– qSR(a,0,2)

• p(a,0,2) -> 1

• qSR(a, 0, 0)

• qSR(a, 2, 2)

– qSR(a,4,3)

• qSR(a,4,3)

– qSR(a, 6, 8)• p(a, 6, 8) -> 6• qSR(a, 6, 5)• qSR(a, 7, 8)

– p(a,7,8) -> 7– qSR(a, 7,6)– qSR(a, 8, 8)

©Duane Szafron 1999

14

Quick Sort Trace 1Quick Sort Trace 1

qSR(0,8)

0 1 2 3 4 5 6 7 860 30 10 20 40 90 70 80 50

0 1 2 3 4 5 6 7 860 30 10 20 40 90 70 80 50

0 1 2 3 4 5 6 7 850 30 10 20 40 90 70 80 60

p(0,8)0 1 2 3 4 5 6 7 8

60 30 10 20 40 90 70 80 50

0 1 2 3 4 5 6 7 850 30 10 20 40 90 70 80 60

0 1 2 3 4 5 6 7 850 30 10 20 40 60 70 80 90

0 1 2 3 4 5 6 7 850 30 10 20 40 60 70 80 90

0 1 2 3 4 5 6 7 850 30 10 20 40 60 70 80 90

qSR(0,4)0 1 2 3 4 5 6 7 8

50 30 10 20 40 60 70 80 90

©Duane Szafron 1999

15

Quick Sort Trace 2Quick Sort Trace 2

qSR(0,4)

0 1 2 3 4 5 6 7 850 30 10 20 40 60 70 80 90

p(0,4)0 1 2 3 4 5 6 7 8

50 30 10 20 40 60 70 80 90

0 1 2 3 4 5 6 7 850 30 10 20 40 60 70 80 90

0 1 2 3 4 5 6 7 840 30 10 20 50 60 70 80 90

0 1 2 3 4 5 6 7 840 30 10 20 50 60 70 80 90

0 1 2 3 4 5 6 7 840 30 10 20 50 60 70 80 90

qSR(0,3)0 1 2 3 4 5 6 7 8

40 30 10 20 50 60 70 80 90

p(0,3)0 1 2 3 4 5 6 7 8

40 30 10 20 50 60 70 80 90

©Duane Szafron 1999

16

Quick Sort Trace 3Quick Sort Trace 3

p(0,3) - cont0 1 2 3 4 5 6 7 8

40 30 10 20 50 60 70 80 90

0 1 2 3 4 5 6 7 840 30 10 20 50 60 70 80 90

0 1 2 3 4 5 6 7 820 30 10 40 50 60 70 80 90

0 1 2 3 4 5 6 7 820 30 10 40 50 60 70 80 90

0 1 2 3 4 5 6 7 820 30 10 40 50 60 70 80 90

qSR(0,2)0 1 2 3 4 5 6 7 8

20 30 10 40 50 60 70 80 90p(0,2)

0 1 2 3 4 5 6 7 820 30 10 40 50 60 70 80 90

0 1 2 3 4 5 6 7 820 30 10 40 50 60 70 80 90

0 1 2 3 4 5 6 7 810 30 20 40 50 60 70 80 90

0 1 2 3 4 5 6 7 810 30 20 40 50 60 70 80 90

©Duane Szafron 1999

17

Quick Sort Trace 4Quick Sort Trace 4

0 1 2 3 4 5 6 7 810 30 20 40 50 60 70 80 90

0 1 2 3 4 5 6 7 810 20 30 40 50 60 70 80 90

0 1 2 3 4 5 6 7 810 20 30 40 50 60 70 80 90

qSR(0,0)

0 1 2 3 4 5 6 7 810 20 30 40 50 60 70 80 90

qSR(2,2)0 1 2 3 4 5 6 7 8

10 20 30 40 50 60 70 80 90qSR(4,3)

0 1 2 3 4 5 6 7 810 20 30 40 50 60 70 80 90

qSR(6,8)0 1 2 3 4 5 6 7 8

10 20 30 40 50 60 70 80 90

©Duane Szafron 1999

18

Quick Sort Trace 5Quick Sort Trace 5

qSR(7,8)0 1 2 3 4 5 6 7 8

10 20 30 40 50 60 70 80 90

qSR(6,8)

0 1 2 3 4 5 6 7 810 20 30 40 50 60 70 80 90

p(6,8)0 1 2 3 4 5 6 7 8

10 20 30 40 50 60 70 80 90

0 1 2 3 4 5 6 7 810 20 30 40 50 60 70 80 90

0 1 2 3 4 5 6 7 810 20 30 40 50 60 70 80 90

qSR(6,5)0 1 2 3 4 5 6 7 8

10 20 30 40 50 60 70 80 90

p(7,8)0 1 2 3 4 5 6 7 8

10 20 30 40 50 60 70 80 90

0 1 2 3 4 5 6 7 810 20 30 40 50 60 70 80 90

©Duane Szafron 1999

19

Quick Sort Trace 6Quick Sort Trace 6

0 1 2 3 4 5 6 7 810 20 30 40 50 60 70 80 90

0 1 2 3 4 5 6 7 810 20 30 40 50 60 70 80 90

qSR(7,6)0 1 2 3 4 5 6 7 8

10 20 30 40 50 60 70 80 90qSR(8,8)

0 1 2 3 4 5 6 7 810 20 30 40 50 60 70 80 90

©Duane Szafron 1999

20

Counting ComparisonsCounting Comparisons

How many comparison operations are required for a quick sort of an n-element collection?

The recursive sort method calls partition() once, and then divides the collection.

if (left >= right)return;

pivot = partition(anArray, left, right); quickSortR(anArray, left, pivot - 1); quickSortR(anArray, pivot + 1, right);

Every call to partition() for a list of size k > 1, performs k-1 comparisons.

©Duane Szafron 1999

21

Comparisons - Best Case 1Comparisons - Best Case 1

In the best case, the pivot is always in the middle of the list being sorted.

Count comparisons, C(k), for k = 15:

C(15) = 14 + C(7) + C(7) = 14 + (6+2+2) + (6+2+2) = 34 In general, we see that C(k) = k-1 + 2*C((k-1)/2) for k =

2m-1

k=7

k=3

k=1

k=3

k=1 k=1 k=1

k=7

k=3

k=1

k=3

k=1 k=1 k=1

6

14

6

2 22 2

best casecomparisons

per call topartition

n=k=15

©Duane Szafron 1999

22

Comparisons - Best Case 2Comparisons - Best Case 2

To find a general formula for C(n), we need to solve the recurrence relation:

C(n) = 2*C((n-1)/2) for n >= 3

Solving recurrence relations is beyond the scope of this course and is considered in C204.

The solution of this recurrence relation is:

C(n) = (n+1) log(n+1) - 2n for n >= 3

Therefore, in the best case, the total number of comparisons is: (n+1)*log(n+1) - 2n = O(n log(n))

©Duane Szafron 1999

23

Check the Recurrence RelationCheck the Recurrence Relation

Check that the solution is valid:

C(n) = (n+1) log(n+1) - 2n for n >= 3, Checking: C(3) = 4*log(4) - 6 = 4*2 - 6 = 8 - 6 = 2 Checking: C(7) = 8*log(8) - 14 = 8*3 - 14 = 24 - 14 = 10 Checking: C(15) = 16*log(16)-30 = 16*4-30 = 64-30 = 34 Checking: C(8) = 9*log(9) - 16 ~ 9*3.17 - 16 ~ 12.53 ~ 13

k=3

k=1 k=1

k=4

k=1 k=2

k=0 k=1

2

7

3

1

n=k=8Number of comparisons in this partitioning

©Duane Szafron 1999

24

Prove Recurrence Solution is ValidProve Recurrence Solution is Valid Prove that a solution to: C(n) = n-1 + 2*C((n-1)/2) is:

C(n) = (n+1) log(n+1) - 2n for n >= 3

Note that: C((n-1)/2) = [(n-1)/2 + 1]*log [(n-1)/2 + 1]

-2[(n-1)/2] = ((n+1)/2) *log ((n+1)/2) - (n-1)

Therefore: 2*C((n-1)/2) + (n-1) =

2*{((n+1)/2)*log ((n+1)/2) - (n-1)} + (n-1) =

(n+1)*log ((n+1)/2) - 2*(n-1) + (n-1) =

(n+1)*{log (n+1) - log (2)} - 2n + 2 + n -1 =

(n+1)*log (n+1) - (n+1)*log (2) - n + 1 =

(n+1)*log (n+1) - (n+1)*1 - n + 1 =

(n+1)*log (n+1) - n + 1 - n + 1 =

(n+1)*log (n+1) - 2n = C(n)

©Duane Szafron 1999

25

Comparisons - Worst CaseComparisons - Worst Case In the worst case, the pivot is at one end of the list so a list

of size k gets divided into a list of size 0 and a list of size k-1.

In this case there are n calls to partition().

Since there are k-1 comparisons in partition() for a list of size k, we have a total number of comparisons:

(n-1) + (n-2) + … + 1 = (n-1)*n/2 = O(n2)

The bad news is that the worst case occurs when the list is sorted (or near sorted). Why? Think about the consequences of this!

©Duane Szafron 1999

26

Comparisons - Average CaseComparisons - Average Case The average case must be between the best case and the

worst case, but since the best case is O(n log(n)) and the worst case is so the average case is O(n2), some analysis is necessary to find the answer.

Analysis yields a complex recurrence relation.

The average case number of comparisons is approximately: 1.386*n*log(n) - 2.846*n

Therefore, the average case time complexity is: O(n log(n)).

©Duane Szafron 1999

27

Counting AssignmentsCounting Assignments How many assignment operations are required for a quick sort of

an n-element collection?

In general there are far fewer assignments than comparisons, since there are 3 assignments per swap, and swap is not called for every comparison:

while ((left < right) &&(anArray[left].compareTo(anArray[right]) < 0))

right--;if (left < right) swap(anArray, left++, right);else return left;

Therefore we ignore the assignments.

©Duane Szafron 1999

28

Time Complexity of Quick SortTime Complexity of Quick Sort

Best case O(n log(n)) for comparisons.

Worst case O(n2) for comparisons.

Average case O(n log(n)) for comparisons.

Note that the quick sort is inferior to insertion sort and merge sort if the list is sorted, nearly sorted, or reverse sorted.

©Duane Szafron 1999

29

Space Complexity of Quick SortSpace Complexity of Quick Sort

This sort appears to only need one temporary memory element for the swap.

However, in the worst case, there are n recursive calls, each of which requires three parameters: a reference to the array, and two ints, left and right.

This means that there are 3*n more words of storage necessary.

In the average and best cases there are only log(n) recursive calls, so this extra storage is negligible.

Note that we ignored this extra space in the merge sort since there were only log(n) recursive calls.