2008 Pearson Education, Inc. All rights reserved. 1 19 Searching and Sorting.

1

2008 Pearson Education, Inc. All rights reserved.

1919

Searching andSorting

2


With sobs and tears he sorted out Those of the largest size…

— Lewis CarrollAttempt the end, and never stand to doubt;Nothing’s so hard, but search will find it out.

— Robert Herrick

‘Tis in my memory lock’d, And you yourself shall keep the key of it.

— William Shakespeare

It is an immutable law in business that words are words, explanations are explanations, promises are promises — but only performance is reality

— Harold S. Green

3


OBJECTIVES

In this chapter you will learn: To search for a given value in a vector using

binary search. To sort a vector using the recursive merge sort

algorithm. To determine the efficiency of searching and

sorting algorithms.

4


19.1 Introduction

19.2 Searching Algorithms

19.2.1 Efficiency of Linear Search

19.2.2 Binary Search

19.3 Sorting Algorithms

19.3.1 Efficiency of Selection Sort

19.3.2 Efficiency of Insertion Sort

19.3.3 Merge Sort (A Recursive Implementation)

19.4 Wrap-Up

5


19.1 Introduction

• Searching data– Determine whether a value (the search key) is present in

the data• If so, find its location

– Algorithms• Linear search

– Simple

• Binary search

– Faster but more complex

6


19.1 Introduction (Cont.)

• Sorting data– Place data in order

• Typically ascending or descending

• Based on one or more sort keys

– Algorithms• Insertion sort

• Selection sort

• Merge sort

– More efficient, but more complex

7


19.1 Introduction (Cont.)

• Big O notation– Estimates worst-case runtime for an algorithm

• How hard an algorithm must work to solve a problem

8


Fig. 19.1 | Searching and sorting algorithms in this text. (Part 1 of 2)

Chapter Algorithm Location

Searching Algorithms

7 Linear search Section 7.7

20 Binary search Section 19.2.2

Recursive linear search Exercise 19.8

Recursive binary search Exercise 19.9

21 Binary tree search Section 20.7

Linear search of a linked list Exercise 20.21

23 binary_search standard library function

Section 22.5.6

9


Fig. 19.1 | Searching and sorting algorithms in this text. (Part 2 of 2)

Chapter Algorithm Location

Sorting Algorithms

7 Insertion sort Section 7.8

8 Selection sort Section 8.6

20 Recursive merge sort Section 19.3.3

Bubble sort Exercises 19.5 and 19.6

Bucket sort Exercise 19.7

Recursive quicksort Exercise 19.10

21 Binary tree sort Section 20.7

23 sort standard library function Section 22.5.6

Heap sort Section 22.5.12

10


19.2 Searching Algorithms

• Searching algorithms– Find element that matches a given search key

• If such an element does exist

– Major difference between search algorithms• Amount of effort they require to complete search

– Particularly dependent on number of data elements

– Can be described with Big O notation

11


19.2.1 Efficiency of Linear Search

• Big O notation– Measures runtime growth of an algorithm relative to

number of items processed• Highlights dominant terms

• Ignores terms that become unimportant as n grows

• Ignores constant factors

12


19.2.1 Efficiency of Linear Search (Cont.)

• Big O notation (Cont.)– Constant runtime

• Number of operations performed by algorithm is constant

– Does not grow as number of items increases

• Represented in Big O notation as O(1)

– Pronounced “on the order of 1” or “order 1”

• Example

– Test if the first element of an n-vector is equal to the second element

• Always takes one comparison, no matter how large the vector

13



• Big O notation (Cont.)– Linear runtime

• Number of operations performed by algorithm grows linearly with number of items

• Represented in Big O notation as O(n)

– Pronounced “on the order of n” or “order n”

• Example

– Test if the first element of an n-vector is equal to any other element

• Takes n - 1 comparisons

• n term dominates, -1 is ignored

14



• Big O notation (Cont.)– Quadratic runtime

• Number of operations performed by algorithm grows as the square of the number of items

• Represented in Big O notation as O(n2)

– Pronounced “on the order of n2” or “order n2”

• Example

– Test if any element of an n-vector is equal to any other element

• Takes n2/2 – n/2 comparisons

• n2 term dominates, constant 1/2 is ignored, -n/2 is ignored

15



• Efficiency of linear search– Linear search runs in O(n) time

• Worst case: every element must be checked

• If size of the vector doubles, number of comparisons also doubles

16


Performance Tip 19.1

Sometimes the simplest algorithms perform poorly. Their virtue is that they are easy to program, test and debug. Sometimes more complex algorithms are required to realize maximum performance.

17


19.2.2 Binary Search

• Binary search algorithm– Requires that the vector first be sorted

• Can be performed by Standard Library function sort

– Takes two random-access iterators

– Sorts elements in ascending order

– First iteration (assuming sorted ascending order)• Test the middle element in the vector

– If it matches search key, the algorithm ends

– If it is greater than search key, continue with only the first half of the vector

– If it is less than search key, continue with only the second half of the vector

18


19.2.2 Binary Search (Cont.)

• Binary search algorithm (Cont.)– Subsequent iterations

• Test the middle element in the remaining subvector

– If it matches search key, the algorithm ends

– If not, eliminate half of the subvector and continue

– Terminates when• Element matching search key is found

• Current subvector is reduced to zero size

– Can conclude that search key is not in vector

19


1 // Fig 19.2: BinarySearch.h

2 // Class that contains a vector of random integers and a function

3 // that uses binary search to find an integer.

4 #include <vector>

5 using std::vector;

6

7 class BinarySearch

8 {

9 public:

10 BinarySearch( int ); // constructor initializes vector

11 int binarySearch( int ) const; // perform a binary search on vector

12 void displayElements() const; // display vector elements

13 private:

14 int size; // vector size

15 vector< int > data; // vector of ints

16 void displaySubElements( int, int ) const; // display range of values

17 }; // end class BinarySearch

Outline

BinarySearch.h

(1 of 1)

20


1 // Fig 19.3: BinarySearch.cpp

2 // BinarySearch class member-function definition.

3 #include <iostream>

4 using std::cout;

5 using std::endl;

6

7 #include <cstdlib> // prototypes for functions srand and rand

8 using std::rand;

9 using std::srand;

10

11 #include <ctime> // prototype for function time

12 using std::time;

13

14 #include <algorithm> // prototype for sort function

15 #include "BinarySearch.h" // class BinarySearch definition

16

17 // constructor initializes vector with random ints and sorts the vector

18 BinarySearch::BinarySearch( int vectorSize )

19 {

20 size = ( vectorSize > 0 ? vectorSize : 10 ); // validate vectorSize

21 srand( time( 0 ) ); // seed using current time

22

23 // fill vector with random ints in range 10-99

24 for ( int i = 0; i < size; i++ )

25 data.push_back( 10 + rand() % 90 ); // 10-99

26

27 std::sort( data.begin(), data.end() ); // sort the data

28 } // end BinarySearch constructor

Outline

BinarySearch.cpp

(1 of 3)

Initialize the vector with random ints from 10-99

Sort the elements in vector data in ascending order

21


29

30 // perform a binary search on the data

31 int BinarySearch::binarySearch( int searchElement ) const

32 {

33 int low = 0; // low end of the search area

34 int high = size - 1; // high end of the search area

35 int middle = ( low + high + 1 ) / 2; // middle element

36 int location = -1; // return value; -1 if not found

37

38 do // loop to search for element

39 {

40 // print remaining elements of vector to be searched

41 displaySubElements( low, high );

42

43 // output spaces for alignment

44 for ( int i = 0; I < middle; i++ )

45 cout << " ";

46

47 cout << " * " << endl; // indicate current middle

48

49 // if the element is found at the middle

50 if ( searchElement == data[ middle ] )

51 location = middle; // location is the current middle

52 else if ( searchElement < data[ middle ] ) // middle is too high

53 high = middle - 1; // eliminate the higher half

54 else // middle element is too low

55 low = middle + 1; // eliminate the lower half

56

57 middle = ( low + high + 1 ) / 2; // recalculate the middle

58 } while ( ( low <= high ) && ( location == -1 ) );

Outline

BinarySearch.cpp

(2 of 3)

Calculate the low end index, high end index and middle index of the portion of the vector being searched

Initialize the location of the found element to -1, indicating that the search key has not (yet) been found

Loop until the subvector is of zero size or the search key is located

Test if the middle element is equal to searchElement

Eliminate half of the remaining values in the vector

22


59

60 return location; // return location of search key

61 } // end function binarySearch

62

63 // display values in vector

64 void BinarySearch::displayElements() const

65 {

66 displaySubElements( 0, size - 1 );

67 } // end function displayElements

68

69 // display certain values in vector

70 void BinarySearch::displaySubElements( int low, int high ) const

71 {

72 for ( int i = 0; i < low; i++ ) // output spaces for alignment

73 cout << " ";

74

75 for ( int i = low; i <= high; i++ ) // output elements left in vector

76 cout << data[ i ] << " ";

77

78 cout << endl;

79 } // end function displaySubElements

Outline

BinarySearch.cpp

(3 of 3)

23


1 // Fig 19.4: Fig19_04.cpp

2 // BinarySearch test program.


4 using std::cin;

5 using std::cout;

6 using std::endl;

7

8 #include "BinarySearch.h" // class BinarySearch definition

9

10 int main()

11 {

12 int searchInt; // search key

13 int position; // location of search key in vector

14

15 // create vector and output it

16 BinarySearch searchVector ( 15 );

17 searchVector.displayElements();

18

19 // get input from user

20 cout << "\nPlease enter an integer value (-1 to quit): ";

21 cin >> searchInt; // read an int from user

22 cout << endl;

23

24 // repeatedly input an integer; -1 terminates the program

25 while ( searchInt != -1 )

26 {

27 // use binary search to try to find integer

28 position = searchVector.binarySearch( searchInt );

29

Outline

Fig20_04.cpp

(1 of 3)

Perform a binary search on the data

24


30 // return value of -1 indicates integer was not found

31 if ( position == -1 )

32 cout << "The integer " << searchInt << " was not found.\n";

33 else

34 cout << "The integer " << searchInt

35 << " was found in position " << position << ".\n";

36

37 // get input from user

38 cout << "\n\nPlease enter an integer value (-1 to quit): ";

39 cin >> searchInt; // read an int from user

40 cout << endl;

41 } // end while

42

43 return 0;

44 } // end main 26 31 33 38 47 49 49 67 73 74 82 89 90 91 95 Please enter an integer value (-1 to quit): 38 26 31 33 38 47 49 49 67 73 74 82 89 90 91 95 * 26 31 33 38 47 49 49 * 26 31 33 38 47 49 49 67 73 74 82 89 90 91 95 (Continued at top of next slide... )

Outline

Fig19_04.cpp

(2 of 3)

25


Outline

Fig19_04.cpp

(3 of 3)

(...continued from bottom of previous slide ) Please enter an integer value (-1 to quit): 38 26 31 33 38 47 49 49 67 73 74 82 89 90 91 95 * 26 31 33 38 47 49 49 * The integer 38 was found in position 3. Please enter an integer value (-1 to quit): 91 26 31 33 38 47 49 49 67 73 74 82 89 90 91 95 * 73 74 82 89 90 91 95 * 90 91 95 * The integer 91 was found in position 13. Please enter an integer value (-1 to quit): 25 26 31 33 38 47 49 49 67 73 74 82 89 90 91 95 * 26 31 33 38 47 49 49 * 26 31 33 * 26 * The integer 25 was not found. Please enter an integer value (-1 to quit): -1

26


19.2.2 Binary Search (Cont.)

• Efficiency of binary search– Logarithmic runtime

• Number of operations performed by algorithm grows logarithmically as number of items increases

• Represented in Big O notation as O(log n)

– Pronounced “on the order of log n” or “order log n”

• Example

– Binary searching a sorted vector of 1023 elements takes at most 10 comparisons ( 10 = log 2 ( 1023 + 1 ) )

• Repeatedly dividing 1023 by 2 and rounding down results in 0 after 10 iterations

27


19.3 Sorting Algorithms

• Sorting algorithms– Placing data into some particular order

• Such as ascending or descending

– One of the most important computing applications

– End result, a sorted vector, will be the same no matter which algorithm is used

• Choice of algorithm affects only runtime and memory use

28


19.3.1 Efficiency of Selection Sort

• Selection sort– At ith iteration

• Swaps the ith smallest element with element i

– After ith iteration• Smallest i elements are sorted in increasing order in first i

positions

– Requires a total of (n2 – n)/2 comparisons• Iterates n - 1 times

– In ith iteration, locating ith smallest element requires n – i comparisons

• Has Big O of O(n2)

29


19.3.2 Efficiency of Insertion Sort

• Insertion sort– At ith iteration

• Insert (i + 1)th element into correct position with respect to first i elements

– After ith iteration• First i elements are sorted

– Requires a worst-case of n2 inner-loop iterations• Outer loop iterates n - 1 times

– Inner loop requires n – 1iterations in worst case

• For determining Big O, nested statements mean multiply the number of iterations

• Has Big O of O(n2)

30


19.3.3 Merge Sort (A Recursive Implementation)

• Merge sort– Sorts vector by

• Splitting it into two equal-size subvectors

– If vector size is odd, one subvector will be one element larger than the other

• Sorting each subvector

• Merging them into one larger, sorted vector

– Repeatedly compare smallest elements in the two subvectors

– The smaller element is removed and placed into the larger, combined vector

31


19.3.3 Merge Sort (A Recursive Implementation) (Cont.)

• Merge sort (Cont.)– Our recursive implementation

• Base case

– A vector with one element is already sorted

• Recursion step

– Split the vector (of ≥ 2 elements) into two equal halves

• If vector size is odd, one subvector will be one element larger than the other

– Recursively sort each subvector

– Merge them into one larger, sorted vector

32



• Merge sort (Cont.)– Sample merging step

• Smaller, sorted vectors

– A: 4 10 34 56 77

– B: 5 30 51 52 93• Compare smallest element in A to smallest element in B

– 4 (A) is less than 5 (B)

• 4 becomes first element in merged vector

– 5 (B) is less than 10 (A)

• 5 becomes second element in merged vector

– 10 (A) is less than 30 (B)

• 10 becomes third element in merged vector

– Etc.

33


1 // Fig 19.5: MergeSort.h

2 // Class that creates a vector filled with random integers.

3 // Provides a function to sort the vector with merge sort.

4 #include <vector>


6

7 // MergeSort class definition

8 class MergeSort

9 {

10 public:

11 MergeSort( int ); // constructor initializes vector

12 void sort(); // sort vector using merge sort

13 void displayElements() const; // display vector elements

14 private:

15 int size; // vector size

16 vector< int > data; // vector of ints

17 void sortSubVector( int, int ); // sort subvector

18 void merge( int, int, int, int ); // merge two sorted vectors

19 void displaySubVector( int, int ) const; // display subvector

20 }; // end class SelectionSort

Outline

MergeSort.h

(1 of 1)

34


1 // Fig 19.6: MergeSort.cpp

2 // Class MergeSort member-function definition.


4 using std::cout;

5 using std::endl;

6

7 #include <vector>


9

10 #include <cstdlib> // prototypes for functions srand and rand

11 using std::rand;

12 using std::srand;

13

14 #include <ctime> // prototype for function time

15 using std::time;

16

17 #include "MergeSort.h" // class MergeSort definition

18

19 // constructor fill vector with random integers

20 MergeSort::MergeSort( int vectorSize )

21 {

22 size = ( vectorSize > 0 ? vectorSize : 10 ); // validate vectorSize

23 srand( time( 0 ) ); // seed random number generator using current time

24

25 // fill vector with random ints in range 10-99

26 for ( int i = 0; i < size; i++ )

27 data.push_back( 10 + rand() % 90 );

28 } // end MergeSort constructor

Outline

MergeSort.cpp

(1 of 5)

35


29

30 // split vector, sort subvectors and merge subvectors into sorted vector

31 void MergeSort::sort()

32 {

33 sortSubVector( 0, size - 1 ); // recursively sort entire vector

34 } // end function sort

35

36 // recursive function to sort subvectors

37 void MergeSort::sortSubVector( int low, int high )

38 {

39 // test base case; size of vector equals 1

40 if ( ( high - low ) >= 1 ) // if not base case

41 {

42 int middle1 = ( low + high ) / 2; // calculate middle of vector

43 int middle2 = middle1 + 1; // calculate next element over

44

45 // output split step

46 cout << "split: ";

47 displaySubVector( low, high );

48 cout << endl << " ";

49 displaySubVector( low, middle1 );

50 cout << endl << " ";

51 displaySubVector( middle2, high );

52 cout << endl << endl;

53

54 // split vector in half; sort each half (recursive calls)

55 sortSubVector( low, middle1 ); // first half of vector

56 sortSubVector( middle2, high ); // second half of vector

57

Outline

MergeSort.cpp

(2 of 5)

Call function sortSubVector with 0 and size – 1 as the beginning and ending indices

Test the base case

Split the vector in two

Recursively call function sortSubVector on the two subvectors

36


58 // merge two sorted vectors after split calls return

59 merge( low, middle1, middle2, high );

60 } // end if

61 } // end function sortSubVector

62

63 // merge two sorted subvectors into one sorted subvector

64 void MergeSort::merge( int left, int middle1, int middle2, int right )

65 {

66 int leftIndex = left; // index into left subvector

67 int rightIndex = middle2; // index into right subvector

68 int combinedIndex = left; // index into temporary working vector

69 vector< int > combined( size ); // working vector

70

71 // output two subvectors before merging

72 cout << "merge: ";

73 displaySubVector( left, middle1 );

74 cout << endl << " ";

75 displaySubVector( middle2, right );

76 cout << endl;

77

78 // merge vectors until reaching end of either

79 while ( leftIndex <= middle1 && rightIndex <= right )

80 {

81 // place smaller of two current elements into result

82 // and move to next space in vector

83 if ( data[ leftIndex ] <= data[ rightIndex ] )

84 combined[ combinedIndex++ ] = data[ leftIndex++ ];

85 else

86 combined[ combinedIndex++ ] = data[ rightIndex++ ];

87 } // end while

Outline

MergeSort.cpp

(3 of 5)

Combine the two sorted vectors into one larger, sorted vector

Loop until the end of either subvector is reached

Test which element at the beginning of the vectors is smaller

Place the smaller element in the combined vector

37


88

89 if ( leftIndex == middle2 ) // if at end of left vector

90 {

91 while ( rightIndex <= right ) // copy in rest of right vector

92 combined[ combinedIndex++ ] = data[ rightIndex++ ];

93 } // end if

94 else // at end of right vector

95 {

96 while ( leftIndex <= middle1 ) // copy in rest of left vector

97 combined[ combinedIndex++ ] = data[ leftIndex++ ];

98 } // end else

99

100 // copy values back into original vector

101 for ( int i = left; i <= right; i++ )

102 data[ i ] = combined[ i ];

103

104 // output merged vector

105 cout << " ";

106 displaySubVector( left, right );


108 } // end function merge

109

110 // display elements in vector

111 void MergeSort::displayElements() const

112 {

113 displaySubVector( 0, size - 1 );

114 } // end function displayElements

Outline

MergeSort.cpp

(4 of 5)

Fill the combined vector with the remaining elements of the right vector or…

…else fill the combined vector with the remaining elements of the left vector

Copy the combined vector into the original vector

38


115

116 // display certain values in vector

117 void MergeSort::displaySubVector( int low, int high ) const

118 {

119 // output spaces for alignment

120 for ( int i = 0; i < low; i++ )

121 cout << " ";

122

123 // output elements left in vector

124 for ( int i = low; i <= high; i++ )

125 cout << " " << data[ i ];

126 } // end function displaySubVector

Outline

MergeSort.cpp

(5 of 5)

39


1 // Fig 19.7: Fig19_07.cpp

2 // MergeSort test program.


4 using std::cout;

5 using std::endl;

6

7 #include "MergeSort.h" // class MergeSort definition

8

9 int main()

10 {

11 // create object to perform merge sort

12 MergeSort sortVector( 10 );

13

14 cout << "Unsorted vector:" << endl;

15 sortVector.displayElements(); // print unsorted vector


17

18 sortVector.sort(); // sort vector

19

20 cout << "Sorted vector:" << endl;

21 sortVector.displayElements(); // print sorted vector

22 cout << endl;

23 return 0;

24 } // end main

Outline

Fig19_07.cpp

(1 of 3)

40


Outline

Fig19_07.cpp

(2 of 3)

Unsorted vector:

30 47 22 67 79 18 60 78 26 54 split: 30 47 22 67 79 18 60 78 26 54 30 47 22 67 79 18 60 78 26 54 split: 30 47 22 67 79 30 47 22 67 79 split: 30 47 22 30 47 22 split: 30 47 30 47 merge: 30 47 30 47 merge: 30 47 22 22 30 47 split: 67 79 67 79 merge: 67 79 67 79 merge: 22 30 47 67 79 22 30 47 67 79 (continued at top of next slide... )

41


Outline

Fig19_07.cpp

(3 of 3)

(...continued from bottom of previous slide)

split: 18 60 78 26 54 18 60 78 26 54 split: 18 60 78 18 60 78 split: 18 60 18 60 merge: 18 60 18 60 merge: 18 60 78 18 60 78 split: 26 54 26 54 merge: 26 54 26 54 merge: 18 60 78 26 54 18 26 54 60 78 merge: 22 30 47 67 79 18 26 54 60 78 18 22 26 30 47 54 60 67 78 79 Sorted vector: 18 22 26 30 47 54 60 67 78 79

42



• Efficiency of merge sort– n log n runtime

• Halving vectors means log 2 n levels to reach base case

– Doubling size of vector requires one more level

– Quadrupling size of vector requires two more levels

• O(n) comparisons are required at each level

– Calling sortSubVector with a size-n vector results in

• Two sortSubVector calls with size-n/2 subvectors

• A merge operation with n – 1 (order n) comparisons

– So, always order n total comparisons at each level

• Represented in Big O notation as O(n log n)

– Pronounced “on the order of n log n” or “order n log n”

43


Fig. 19.8 | Searching and sorting algorithms with Big O values.

Algorithm Location Big O

Searching Algorithms

Linear search Section 7.7 O(n)

Binary search Section 20.2.2 O(log n)

Recursive linear search

Exercise 20.8 O(n)

Recursive binary search

Exercise 20.9 O(log n)

Sorting Algorithms

Insertion sort Section 7.8 O(n2)

Selection sort Section 8.6 O(n2)

Merge sort Section 20.3.3 O(n log n)

Bubble sort Exercises 16.3 and 16.4

O(n2)

Quick sort Exercise 20.10 Worst case: O(n2) Average case: O(n log n)

44


Fig. 19.9 | Approximate number of comparisons for common Big O notations.

n Approximate decimal value

O(log n) O(n) O(n log n) O(n2)

210 1000 10 210 10 ·210 220

220 1,000,000 20 220 20 ·220 240

230 1,000,000,000 30 230 30 ·230 260

2008 Pearson Education, Inc. All rights reserved. 1 19 Searching and Sorting.

Documents

pearson education

search algorithms

sorting slide

sorting algorithms

complete search

complex slide

big o notation slide

green slide