Sorting Sorting Algorithms Algorithms 2 2
Jan 02, 2016
Sorting Sorting AlgorithmsAlgorithms
22
QuicksortQuicksort
General Quicksort Algorithm:General Quicksort Algorithm: Select an element from the array to be Select an element from the array to be
the pivotthe pivot Rearrange the elements of the array into Rearrange the elements of the array into
a left and right subarraya left and right subarray All values in the left subarray are < pivotAll values in the left subarray are < pivot All values in the right subarray are > pivotAll values in the right subarray are > pivot
Independently sort the subarraysIndependently sort the subarrays No merging required, as left and right No merging required, as left and right
are independent problems are independent problems [ Parallelism?!? ][ Parallelism?!? ]
QuicksortQuicksort
void quicksort(int* arrayOfInts, int first, int void quicksort(int* arrayOfInts, int first, int last) last)
{ { int pivot; int pivot; if (first < last) if (first < last) { { pivot = partition(arrayOfInts, first, last); pivot = partition(arrayOfInts, first, last); quicksort(arrayOfInts,first,pivot-1); quicksort(arrayOfInts,first,pivot-1); quicksort(arrayOfInts,pivot+1,last); quicksort(arrayOfInts,pivot+1,last); } } } }
QuicksortQuicksortint partition(int* arrayOfInts, int first, int last) int partition(int* arrayOfInts, int first, int last) { { int temp;int temp; int p = first; int p = first; // set pivot = first // set pivot = first
indexindex for (int k = first+1; k <= last; k++) // for every other indxfor (int k = first+1; k <= last; k++) // for every other indx { { if (arrayOfInts[k] <= arrayOfInts[first]) // if data is if (arrayOfInts[k] <= arrayOfInts[first]) // if data is
smallersmaller { {
p = p + 1; p = p + 1; // update final pivot // update final pivot locationlocation swap(arrayOfInts[k], arrayOfInts[p]);swap(arrayOfInts[k], arrayOfInts[p]);
}} } } swap(arrayOfInts[p], arrayOfInts[first]);swap(arrayOfInts[p], arrayOfInts[first]); return p; return p; }}
99 55 33 2020181899 1818 33 20205599 33 1818 202055
Partition Step ThroughPartition Step Through
partition(cards, 0, 4)P = 0 K = 1 P = 1 K = 3
cards[1] < cards[0] ? No cards[3] < cards[0]? YesP = 2
P = 0 K = 2 temp = cards[3]cards[2] < cards[0] ? Yes cards[3] = cards[2]P = 1 cards[2] = cards[3]temp = cards[2] P = 2 K = 4cards[2] = cards[1] cards[4] < cards[0]? Nocards[1] = temp
temp = cards[2], cards[2] = cards[first]cards[first] = temp, return p = 2;
33 99 1818 202055
Complexity of QuicksortComplexity of Quicksort
Worst case is O(nWorst case is O(n22)) What does worst case correspond to?What does worst case correspond to?
Already sorted or near sortedAlready sorted or near sorted Partitioning leaves heavily unbalanced Partitioning leaves heavily unbalanced
subarrayssubarrays
On average is O(n logOn average is O(n log22n), and it is n), and it is average a lot of the time.average a lot of the time.
Complexity of QuicksortComplexity of Quicksort
Recurrence Relation: [Average Case]Recurrence Relation: [Average Case]
2 sub problems2 sub problems
½ size (if good pivot)½ size (if good pivot)
Partition is O(n)Partition is O(n)
a = 2 b = 2 k = 1a = 2 b = 2 k = 1
2 = 22 = 211 Master Theorem: Master Theorem: O(nlogO(nlog22n)n)
Complexity of QuicksortComplexity of Quicksort
Recurrence Relation: [Worst Case]Recurrence Relation: [Worst Case] Partition separates into (n-1) and (1)Partition separates into (n-1) and (1) Can’t use master theorem:Can’t use master theorem:
b (subproblem size) changesb (subproblem size) changes
n-1/n n-2/n-1 n-3/n-2n-1/n n-2/n-1 n-3/n-2 Note that sum of partition work:Note that sum of partition work:
n + (n-1) + (n-2) + (n-3) …n + (n-1) + (n-2) + (n-3) …
Sum(1,N) Sum(1,N) = N(N+1)/2= N(N+1)/2 = = O(NO(N22))
Complexity of QuicksortComplexity of Quicksort
Requires stack space to implement Requires stack space to implement recursionrecursion
Worst case: O(n) stack space Worst case: O(n) stack space If pivot breaks into 1 element and n-1 If pivot breaks into 1 element and n-1
element subarrayselement subarrays Average case: O(log n)Average case: O(log n)
Pivot splits evenlyPivot splits evenly
MergeSortMergeSort General Mergesort Algorithm:General Mergesort Algorithm:
Recursively split subarrays in halfRecursively split subarrays in half Merge sorted subarraysMerge sorted subarrays
Splitting is first in recursive call, so continues Splitting is first in recursive call, so continues until have one item subarraysuntil have one item subarrays
One item subarrays are by definition sortedOne item subarrays are by definition sorted
Merge recombines subarrays so result is Merge recombines subarrays so result is sortedsorted
1+1 item subarrays => 2 item subarrays1+1 item subarrays => 2 item subarrays 2+2 item subarrays => 4 item subarrays2+2 item subarrays => 4 item subarrays Use fact that subarrays are sorted to simplify merge Use fact that subarrays are sorted to simplify merge
algorithmalgorithm
MergeSortMergeSort
void mergesort(int* array, int* tempArray, int low, void mergesort(int* array, int* tempArray, int low, int high, int size)int high, int size)
{{if (low < high)if (low < high)
{{ int middle = (low + high) / 2;int middle = (low + high) / 2; mergesort(array,tempArray,low,middle, size);mergesort(array,tempArray,low,middle, size); mergesort(array,tempArray,middle+1, high, mergesort(array,tempArray,middle+1, high,
size);size); merge(array,tempArray,low,middle,high, size);merge(array,tempArray,low,middle,high, size); }}}}
MergeSortMergeSortvoid merge(int* array, int* tempArray, int low, int middle, int high, int size) void merge(int* array, int* tempArray, int low, int middle, int high, int size) {{ int i, j, k;int i, j, k;
for (i = low; i <= high; i++) { tempArray[i] = array[i]; } // copy into temp for (i = low; i <= high; i++) { tempArray[i] = array[i]; } // copy into temp arrayarray
i = low; j = middle+1; k = low;i = low; j = middle+1; k = low;
while ((i <= middle) && (j <= high)) {while ((i <= middle) && (j <= high)) { // merge// merge if (tempArray[i] <= tempArray[j])if (tempArray[i] <= tempArray[j]) // if lhs item is smaller// if lhs item is smaller
array[k++] = tempArray[i++];array[k++] = tempArray[i++]; // put in final array, increment// put in final array, increment elseelse // final array position, lhs index// final array position, lhs index
array[k++] = tempArray[j++];array[k++] = tempArray[j++]; // else put rhs item in final array// else put rhs item in final array }} // increment final array // increment final array
positionposition// rhs index// rhs index
while (i <= middle)while (i <= middle) // one of the two will run out// one of the two will run out array[k++] = tempArray[i++];array[k++] = tempArray[i++]; // copy the rest of the data// copy the rest of the data}} // only need to copy if in lhs // only need to copy if in lhs
arrayarray // rhs array already in right place// rhs array already in right place
MergeSort ExampleMergeSort Example
Recursively Split
2020 33 1818 99 55
2020 33 1818 99 55
MergeSort ExampleMergeSort Example
Recursively Split
2020 33 1818 99 55
2020 33 181899
55
MergeSort ExampleMergeSort Example
Merge
2020 33 1818 99 55
Merge Sort ExampleMerge Sort Example
Temp Array
i j
Array
Temp[i]< Temp[j]
Yes
2 cardsNot very interestingThink of as swap
2020 33
1818
202033
202033
k
33
MergeSort ExampleMergeSort ExampleTemp Array
i j
Array
Temp[i]< Temp[j]
No
k
Update J, K by 1 => Hit Limit of Internal While Loop, as J > High NowCopy until I > Middle
1818202033 33 1818
k
2020Array
181833
MergeSort ExampleMergeSort Example2 Card Swap
Final aftermergingabove sets
99 55 55 99
2020181833 55 99
33i=1,j=3
55i=1,j=4
i=0,j=3
99i=1,j=5
1818i=2,j=5
2020i=3,j=5
Complexity of MergeSortComplexity of MergeSort
Recurrence relation:Recurrence relation:2 subproblems2 subproblems½ size½ sizeMerging is O(n) for any subproblemMerging is O(n) for any subproblem
Always moving forwards in the arrayAlways moving forwards in the array
a = 2 b = 2 k = 1a = 2 b = 2 k = 12 = 22 = 211 Master Theorem: O(n logMaster Theorem: O(n log22n)n)
Always O(n logAlways O(n log22n) in both average and worst case n) in both average and worst case Doesn’t rely on quality of pivot choiceDoesn’t rely on quality of pivot choice
Space Complexity of Space Complexity of MergesortMergesort
Need an additional O(n) temporary Need an additional O(n) temporary arrayarray
Number of recursive calls:Number of recursive calls: Always O(logAlways O(log22n)n)
TradeoffsTradeoffs
When it is more useful to:When it is more useful to: Just searchJust search Quicksort or Mergesort and searchQuicksort or Mergesort and search
Assume Z searchesAssume Z searchesSearch on random data: Z * O(n)Search on random data: Z * O(n)Fast Sort and binary search: Fast Sort and binary search:
O(nlogO(nlog22n) + Z *logn) + Z *log22nn
TradeoffsTradeoffs
Z * n <= nlogZ * n <= nlog22n + Zlogn + Zlog22nnZ(n - logZ(n - log22n) <= n logn) <= n log22nn
Z <= (n logZ <= (n log22n) / (n-logn) / (n-log22n)n)
Z <= (n logZ <= (n log22n) / n [Approximation]n) / n [Approximation]Z <= logZ <= log22n [Approximation]n [Approximation]
Where as before, had to do N searches to make Where as before, had to do N searches to make up for cost of sorting, now only do logup for cost of sorting, now only do log22NN
1,000,000 items = 19 searches, instead of 1,000,000 items = 19 searches, instead of 1,000,0001,000,000
How Fast?How Fast?
Without specific details of what Without specific details of what sorting, O(n logsorting, O(n log22n) is the maximum n) is the maximum speed sort possible.speed sort possible. Only available operations: Compare, SwapOnly available operations: Compare, Swap
Proof: Decision Tree – describes how Proof: Decision Tree – describes how sort operatessort operates Every vertex represents a comparison, Every vertex represents a comparison,
every branch a resultevery branch a result Moving down tree – Tracing a possible Moving down tree – Tracing a possible
run through the algorithmrun through the algorithm
How Fast?How Fast?
K1 <= K2 [1,2,3]
K2 <= K3 K1 <= K3[1,2,3] [2,1,3]
Yes No
stop K1 <= K3
YesNo
K2 <= K3stop
stop stopstop stop
Yes
Yes
YesNo
No
No
[1,2,3]
[1,3,2]
[2,1,3]
[1,3,2] [3,1,2]
[2,3,1]
[2,3,1] [3,2,1]
How Fast?How Fast? There are n! possible “stop” nodes – There are n! possible “stop” nodes –
effectively all permutations of the n effectively all permutations of the n numbers in the array.numbers in the array.
Thus any decision tree representing a Thus any decision tree representing a sorting algorithm must have n! leavessorting algorithm must have n! leaves
The height of a this type of tree (The height of a this type of tree (a binary a binary treetree) is correlated with number of leaves:) is correlated with number of leaves: Height k = 2^(k-1) leavesHeight k = 2^(k-1) leaves Must be at least logMust be at least log22n! + 1 heightn! + 1 height
How Fast?How Fast? Path from top to bottom of tree – trace of a Path from top to bottom of tree – trace of a
run of the algorithmrun of the algorithm Need to prove that (logNeed to prove that (log22n!) is lower bounded n!) is lower bounded
by (n logby (n log22n)n)
n! = (n)(n-1)(n-2)(n-3) … (3)(2)(1)n! = (n)(n-1)(n-2)(n-3) … (3)(2)(1) > (n)(n-1)(n-2)(n-3) … ceil(n/2) // doing fewer multiplies> (n)(n-1)(n-2)(n-3) … ceil(n/2) // doing fewer multiplies > ceil(n/2) > ceil(n/2) (ciel(n/2))(ciel(n/2)) // doing multiplies of bigger things // doing multiplies of bigger things > approximately (n/2)> approximately (n/2)(n/2)(n/2)
log log 22 n! > log n! > log 2 2 (n/2)(n/2)(n/2) (n/2)
log log 22 n! > (n/2) log n! > (n/2) log 2 2 (n/2)(n/2) //exponentiation in logs = multiplication out //exponentiation in logs = multiplication out frontfront
log log 22 n! > (n/2)(log n! > (n/2)(log22n – logn – log22 2) // division in logs = subtraction 2) // division in logs = subtractionlog log 22 n! > (n/2)(log n! > (n/2)(log22n – 1)n – 1)log log 22 n! > (n/2)(log n! > (n/2)(log22n) – (n/2)n) – (n/2)log log 22 n! > (1/2) [nlog n! > (1/2) [nlog22n – n]n – n]log log 22 n! ~ O(n log n! ~ O(n log22n)n)