DSA 1.pdf

7/27/2019 DSA 1.pdf

1/18

Lecture Notes: 1

Introduction to Algorithms

Consider the following three examples. What do they all have in common?

Chocolate Cream Pie

1. Heat milk, marshmallows and chocolate in 3-quart saucepan over low heat, stirringconstantly, until chocolate and marshmallows are melted and blended. Refrigerate about

20 minutes, stirring occasionally until mixture mounds slightly when dropped from a

spoon.

2. Beat whipping cream in chilled small bowl with electric mixer on high speed until softpeaks form. Fold chocolate mixture into whipped cream. Pour into pie shell. Refrigerate

uncovered about 8 hours or until set. Garnish with milk chocolate curls and whipped

cream.

Directions to John's House

From the Quik Mart, you should follow Saddle road for four miles until you reach a stoplight.

Then make a left-hand turn at the stop light. Now you will be on Hollow street. Continue

driving on Hollow street for one mile. You should drive past four blocks until you reach the

post office. Once you are at the post office, turn right onto Jackson road. Then stay on

Jackson for about 10 miles. Eventually you will pass the Happy Meadow farm on your right.

Just after Happy Meadow, you should turn left onto Brickland drive. My house is the first

house on your left.

How to change your motor oil

1. Place the oil pan underneath the oil plug of your car.2. Unscrew the oil plug.3. Drain oil.4. Replace the oil plug.5. Remove the oil cap from the engine.6. Pour in 4 quarts of oil.7. Replace the oil cap.

Each of these examples are algorithms, a set of instructions for solving a problem. Once we

have created an algorithm, we no longer need to think about the principles on which the

algorithm is based. For example, once you have the directions to John's house, you do not

need to look at a map to decide where to make the next turn. The intelligence needed to find

the correct route is contained in the algorithm. All you have to do is follow the directions.

This means that algorithms are a way of capturing intelligence and sharing it with others.

Once you have encoded the necessary intelligence to solve a problem in an algorithm, many

people can use your algorithm without needing to become experts in a particular field.

Algorithms are especially important to computers because computers are really general

purpose machines for solving problems. But in order for a computer to be useful, we must

give it a problem to solve and a technique for solving the problem. Through the use ofalgorithms, we can make computers "intelligent" by programming them with various

Notes Compiled for III Sem B.Tech CSE C & D batch Page 1

7/27/2019 DSA 1.pdf

2/18

Lecture Notes: 1

algorithms to solve problems. Because of their speed and accuracy, computers are well-suited

for solving tedious problems such as searching for a name in a large telephone directory or

adding a long column of numbers. However, the usefulness of computers as problem solving

machines is limited because the solutions to some problems cannot be stated in an algorithm.

Much of the study of computer science is dedicated to discovering efficient algorithms andrepresenting them so that they can be understood by computers.

Definition of Algorithms

In the introduction, we gave an informal definition of an algorithm as "a set of instructions

for solving a problem" and we illustrated this definition with a recipe, directions to a friend's

house, and instructions for changing the oil in a car engine. You also created your own

algorithm for putting letters and numbers in order. While these simple algorithms are fine for

us, they are much too ambiguous for a computer. In order for an algorithm to be applicable to

a computer, it must have certain characteristics. We will specify these characteristics in our

formal definition of an algorithm.

An algorithm is a well-ordered collection of unambiguous and effectively computable

operations that when executed produces a result and halts in a finite amount of time

With this definition, we can identify five important characteristics of algorithms.

1. Algorithms are well-ordered.2. Algorithms have unambiguous operations.3. Algorithms have effectively computable operations.4. Algorithms produce a result.5. Algorithms halt in a finite amount of time.

These characteristics need a little more explanation, so we will look at each one in detail.

1. Algorithms are well-ordered

Since an algorithm is a collection of operations or instructions, we must know the correct

order in which to execute the instructions. If the order is unclear, we may perform the wrong

instruction or we may be uncertain which instruction should be performed next. This

characteristic is especially important for computers. A computer can only execute an

algorithm if it knows the exact order of steps to perform.

2. Algorithms have unambiguous operations

Each operation in an algorithm must be sufficiently clear so that it does not need to be

simplified. Given a list of numbers, you can easily order them from largest to smallest with

the simple instruction "Sort these numbers." A computer, however, needs more detail to sort

numbers. It must be told to search for the smallest number, how to find the smallest number,

how to compare numbers together, etc. The operation "Sort these numbers" is ambiguous to a

computer because the computer has no basic operations for sorting. Basic operations used for

writing algorithms are known as primitive operations or primitives. When an algorithm is

written in computer primitives, then the algorithm is unambiguous and the computer canexecute it.


7/27/2019 DSA 1.pdf

3/18

Lecture Notes: 1

3. Algorithms have effectively computable operations

Each operation in an algorithm must be doable, that is, the operation must be something that

is possible to do. Suppose you were given an algorithm for planting a garden where the first

step instructed you to remove all large stones from the soil. This instruction may not be

doable if there is a four ton rock buried just below ground level. For computers, manymathematical operations such as division by zero or finding the square root of a negative

number are also impossible. These operations are not effectively computable so they cannot

be used in writing algorithms.

4. Algorithms produce a result

In our simple definition of an algorithm, we stated that an algorithm is a set of instructions for

solving a problem. Unless an algorithm produces some result, we can never be certain

whether our solution is correct. Have you ever given a command to a computer and

discovered that nothing changed? What was your response? You probably thought that the

computer was malfunctioning because your command did not produce any type of result.Without some visible change, you have no way of determining the effect of your command.

The same is true with algorithms. Only algorithms which produce results can be verified as

either right or wrong.

5. Algorithms halt in a finite amount of time

Algorithms should be composed of a finite number of operations and they should complete

their execution in a finite amount of time. Suppose we wanted to write an algorithm to print

all the integers greater than 1. Our steps might look something like this:

1. Print the number 2.2. Print the number 3.3. Print the number 4.

.

.

.

While our algorithm seems to be pretty clear, we have two problems. First, the algorithm

must have an infinite number of steps because there are an infinite number of integers greater

than one. Second, the algorithm will run forever trying to count to infinity. These problems

violate our definition that an algorithm must halt in a finite amount of time. Every algorithmmust reach some operation that tells it to stop.

Specifying Algorithms

When writing algorithms, we have several choices of how we will specify the operations in

our algorithm. One option is to write the algorithm using plain English. An example of this

approach is the directions to John's house given in the introduction lesson. Although plain

English may seem like a good way to write an algorithm, it has some problems that make it a

poor choice. First, plain English is too wordy. When we write in plain English, we must

include many words that contribute to correct grammar or style but do nothing to help

communicate the algorithm. Second, plain English is too ambiguous. Often an English


7/27/2019 DSA 1.pdf

4/18

Lecture Notes: 1

sentence can be interpreted in many different ways. Remember that our definition of an

algorithm requires that each operation be unambiguous.

Another option for writing algorithms is using programming languages. These languages are

collections of primitives (basic operations) that a computer understands. While programming

languages avoid the problems of being wordy and ambiguous, they have some otherdisadvantages that make them undesirable for writing algorithms. Consider the following

lines of code from the programming language C++.

a = 1;b = 0;whi l e ( a

7/27/2019 DSA 1.pdf

5/18

Lecture Notes: 1

For the remainder of this study, we will write our algorithms using the structured English

approach.

Sorting Algorithms

Now that we have a definite way of writing our algorithms, let's look at some algorithms forsolving the problem of sorting. Sorting is a very common problem handled by computers. For

example, most graphical email programs allow users to sort their email messages in several

ways: date received, subject line, sender, priority, etc. Each time you reorder your email

messages, the computer uses a sorting algorithm to sort them. Since computers can compare a

large number of items quickly, they are quite good at sorting.

Here, you will learn three different algorithms for sorting: the Simple Sort, the Insertion Sort,

and the Selection Sort. These algorithms will give us a basis for comparing algorithms and

determining which ones are the best. We will illustrate each algorithm by sorting a hand of

playing cards like the ones below. Traditionally, a group of playing cards is called a "hand"

of cards. In the next few lessons, we will use the term "hand" to refer to a group of seven

playing cards.

Then we will show how the algorithm also applies to the problem of sorting a list of numbers

in a computer. Our list of numbers will be stored in an array of memory cells like the diagram

below.

It is important to note that we will be using the same algorithm to sort both playing cards and

numbers. One important quality of a good algorithm is that it solves a class of problems and

not just one particular problem. A good sorting algorithm should provide a solution to the

problem of sorting for many types of items. Imagine if you were trying to design an email

program that allowed users to sort their messages by date received, subject line, and sender.Would you want to write a new algorithm for each different sort, or would you prefer to write

a single algorithm that handled all three? Of course you would prefer the latter approach, so

when you designed your algorithm, you would design it to solve a class of problems (e.g.

sorting) rather than an individual problem (e.g. sorting emails by date).

Basic Operation

The sorting algorithms you will learn in the next few lessons share two basic operations in

common. These operations are the comparison operation and the swap operation. We will

look at each one in more detail before we examine our sorting algorithms.


7/27/2019 DSA 1.pdf

6/18

Lecture Notes: 1

The Comparison Operation

The comparison operation is simply a way of determining which item in a list should come

first. If we are sorting a list of numbers from smallest to largest, the comparison operation

tells us to place the number with the least value first. If we are sorting a list of letters

alphabetically, the comparison operation tells us to place 'a' before 'b', 'b' before 'c', and so on.Sorting algorithm must usually perform many comparisons in order to correctly sort a list.

The Swap Operation

The swap operation is one way we move items as we are sorting. By swapping small items

with large ones, we can place all the items in the correct order. When we use computers for

sorting, the swap operation can be a little tricky because of the way computers copy data

from one memory location to another. Using the animation below, see if you can correctly

determine the algorithm for the swap operation.

Let's take one more look at the swap algorithm:

1. Copy the contents of cell A to temp.2. Copy the contents of cell B to cell A.3. Copy the contents of temp to cell B.

Notice that this operation requires three copies. It is important to remember that the swap

operation is really a combination of copy operations. In our next lesson, we will learn a

sorting algorithm called the Simple Sort that uses just the copy operation rather than the swap

operation.

Simple card sort

One basic algorithm for sorting is the Simple Sort. We can illustrate this algorithm using a

group of unsorted playing cards. The Simple Sort sort works by selecting the smallest card in

the unsorted hand and moving this card to a second hand. Once all the cards have been

removed from the unsorted hand, the second hand contains the cards in sorted order. The

algorithm for the Simple Sort is given in the box below.

Simple Card Sort Algorithm

1. Get a hand of unsorted cards2. Repeat steps 3 through 5 until the unsorted hand is empty3. Compare unsorted cards4. Select the smallest unsorted card5. Move this card to the sorted hand6. Stop

Most of the steps in this algorithm are simple, but the third step needs a little more

explanation. In order to be certain that we are selecting the smallest card, we need to compare

all the cards in our hand. We can do this by comparing the first two cards and keeping the


7/27/2019 DSA 1.pdf

7/18

Lecture Notes: 1

smaller of the two. Then we can compare this card to the remaining cards until we find a

smaller one or check all the cards.

Of course, we all know that the Simple Card Sort algorithm is not very useful to a computer.

However, we can use the same idea as in our Simple Card Sort to write a Simple Sort that can

be used by a computer. Let's see what this algorithm looks like and how it can be used to sortnumbers in a computer.

Simple Sort Algorithm

1. Get a list of unsorted numbers2. Repeat steps 3 through 6 until the unsorted list is empty3. Compare the unsorted numbers4. Select the smallest unsorted number5. Move this number to the sorted list6. Store a maximum value in the place of the smallest number7. Stop

Notice that most of the steps in our new algorithm are the same as the steps in the Simple

Card Sort. The exception is step 6. We need this extra step because we are no longer moving

cards from hand to hand but copying numbers in computer memory. For our algorithm to

work, we must replace our original number with a special marker so it will not be considered

again. The steps below illustrate how the Simple Sort algorithm works on a computer.

1. First, we give the computer a list of unsorted numbers. These numbers are stored in agroup of contiguous memory cells called an array. Each memory cell of the array holds asingle number.

2. As the computer sorts these numbers, it will repeatedly compare them to find thesmallest number. This is similar to the comparisons we made when sorting our hand of

cards. Each time we compared two cards and kept the smaller of the two. Then we

compared this card to the remaining cards until we found a smaller one or checked all the

cards. The computer uses the same process only with numbers rather than cards.

Once the smallest number is found, the computer will copy this number to a new array of

memory cells and replace the old number with a special number called MAX. MAX is the

largest number a single memory cell can hold. None of the remaining numbers can be

larger than MAX, so this number is a good choice for marking memory cells that have

already been sorted.


7/27/2019 DSA 1.pdf

8/18

Lecture Notes: 1

Unsorted Array

Sorted Array

3. Next, the computer begins searching for the smallest number in the unsorted list.Although it is easy for us to scan the numbers and select the 2 as smallest, the computer

must compare all the memory cells in the unsorted array to be certain which number is

smallest. This means the computer must perform six comparisons: (7 < 8), (7 > 5),

(5 > 2), (2 < 4), (2 < 6), and finally (2 < 3). Once the comparisons are done, the computercopies 2 to the sorted array and replaces the original 2 with MAX.

Unsorted Array

Sorted Array

4. Now the computer begins searching for the smallest number again. Six morecomparisons are required to determine that 3 is smallest: (7 < 8), (7 > 5), (5 < MAX),

(5 > 4), (4 < 6), and finally (4 > 3). Now we can see the importance of replacing 2 with

MAX in our previous step. If we had not made this change, then 2 would have been

selected as the smallest number again. After copying 3 to the sorted array, the computer

also replaces the original with MAX.

Unsorted Array

Sorted Array

5. With six more comparisons, the computer selects 4 as the smallest number, copies itto the sorted array, and replaces the original with MAX.


7/27/2019 DSA 1.pdf

9/18

Lecture Notes: 1

Unsorted Array

Sorted Array

6. The numbers 5, 6, 7, and 8 are also selected in turn by six comparisons, a copy, and areplacement of the original. Once all the memory cells in the unsorted array have been

considered, the sorted array contains our original numbers in sorted order.

Unsorted Array

Sorted Array

Insertion Card Sort

Another basic algorithm for sorting is the Insertion Sort. We can also illustrate this algorithm

with playing cards. The Insertion Sort works by sorting cards within a single hand (i.e. a

group of seven cards) rather than using two hands like the Simple Sort. In order to do this, we

must have a marker to indicate which section of our hand is sorted. The algorithm for this sortis given in the box below.

Insertion Card Sort Algorithm

1. Get a hand of unsorted cards2. Set a marker for the sorted section after the first card of the hand3. Repeat steps 4 through 6 until the unsorted section is empty4. Select the first unsorted card5. Swap this card to the left until it arrives at the correct sorted position.6. Advance the marker to the right one card7. Stop

We begin the sort by placing the marker after the first card in our hand. Then we compare the

second card with the first. If the second card is smaller, we swap it with the first. Then we

advance our marker to indicate that the sorted section now contains two ordered cards. By

repeating this process of swapping an unsorted card into the correct sorted position and

advancing our sort marker, we can order the entire hand of cards.


7/27/2019 DSA 1.pdf

10/18

Lecture Notes: 1

Insertion sort

Now let's look at how the Insertion Sort algorithm would work inside a computer. Below is

our modified algorithm for sorting a list of numbers.

Insertion Sort Algorithm

1. Get a list of unsorted numbers2. Set a marker for the sorted section after the first number in the list3. Repeat steps 4 through 6 until the unsorted section is empty4. Select the first unsorted number5. Swap this number to the left until it arrives at the correct sorted position6. Advance the marker to the right one position7. Stop

This time the steps of our modified algorithm are almost identical to the steps in our card

sorting algorithm. We have simply substituted numbers for playing cards and a list of

numbers for a hand of cards. The steps below illustrate how the Insertion Sort algorithm

works on a computer.

1. First, we give the computer a list of unsorted numbers and store them in an array ofmemory cells.

2. To begin the sort, the computer divides the sorted and unsorted sections of the list byplacing a marker after the first number. To sort the numbers, it will repeatedly

compare the first unsorted number with the numbers in the sorted section. If the

unsorted number is smaller than its sorted neighbor, the computer will swap them.

3. The first number in the unsorted section is 8, so the computer compares it with thenumber to the left. Since 8 is greater than 7, these numbers do not need to swapped

and the computer simply advances the marker one position. Notice that only one

comparison was needed to sort the 8.


7/27/2019 DSA 1.pdf

11/18

Lecture Notes: 1

4. Now the first number in the unsorted section is 5. 5 is less than 8, so the computerswaps these numbers.

5. 5 is also less than 7, so the computer swaps these numbers as well.

6. Now 5 is in the correct order, so the computer advances the marker one position. Thistime two comparisons and two swaps were needed to sort the number.

7. Now the first number in the unsorted section is 2. 2 is less than 8, 7, and 5, so afterthree comparisons and three swaps, 2 arrives at the correct sorted position, and the

computer advances the sort marker.

8. Now the first number in the unsorted section is 4. 4 is less than 8, 7, and 5 but it is notless than 2. This time the computer performs four comparisons and three swaps to put

the 4 in the correct order. Only three swaps were needed since the 2 and the 4 did not

need to be switched. After these comparisons and swaps, the computer advances thesort marker.

9. Now 6 is the first number in the unsorted section. After three comparisons and twoswaps, the computer places the 6 in the correct position between 5 and 7. Notice that

the computer did not need to compare the 6 with the 2 or the 4 since it already knows

these numbers are less than 5. Once the computer finds a number in the sorted section


7/27/2019 DSA 1.pdf

12/18

Lecture Notes: 1

less than 6, it knows it has found the correct position for 6 and it can advance the sort

marker.

10.The final unsorted number is 3. To find the correct position for 3, the computer mustcompare it with every number in the unsorted section. However, only five swaps are

required since the first number (2) is less than 3. After moving 3 to the correct

position and advancing the sort marker, the Insertion Sort is complete since the

unsorted section is empty.

Selection card sort algorithm

The final sorting algorithm we will learn is the Selection Sort. We will begin by using our

playing cards to illustrate the algorithm. Similar to the Simple Sort, the Selection Sort

searches through the hand of cards for the smallest card. But unlike the Simple Sort, this sort

does not use two hands for sorting. Instead, the Selection Sort uses a marker to divide the

sorted and unsorted section of the hand just like the Insertion Sort. The algorithm for the

Selection Sort is given in the box below.

Selection Card Sort Algorithm

1. Get a hand of unsorted cards2. Set a marker for the unsorted section at the front of the hand3. Repeat steps 4 through 7 until one card remains in the unsorted section4. Compare all unsorted cards5. Select the smallest unsorted card6. Swap this number with the first card in the unsorted section7. Advance the marker to the right one card8. Stop

We begin this sort by setting our marker at the front of the hand. Then we search the hand for

the smallest card, swap this card with the first card in the unsorted section, and update our

marker. By repeating this process of searching and then swapping, we can sort the entire hand

of cards.


7/27/2019 DSA 1.pdf

13/18

Lecture Notes: 1

Selection sort

Now that you have a general idea of how the Selection Sort works, let's see how the computer

would perform this sort with numbers. Below is our modified algorithm for sorting a list of

numbers.

Selection Sort Algorithm

1. Get a list of unsorted numbers2. Set a marker for the unsorted section at the front of the list3. Repeat steps 4 - 6 until one number remains in the unsorted section4. Compare all unsorted numbers in order to select the smallest one5. Swap this number with the first number in the unsorted section6. Advance the marker to the right one position7. Stop

Again our modified algorithm is almost identical to our card sorting algorithm. We only

needed to substitute numbers for playing cards and a list of numbers for a hand of cards. One

other slight change is that steps 4 and 5 of the Selection Card Sort have been combined. This

is typical since a computer will usually keep track of the smallest number while it compares

all the numbers. The steps below illustrate how the Selection Sort algorithm works on a

computer.

1. First, we give the computer a list of unsorted numbers and store them in an array ofmemory cells.

2. To begin the sort, the computer divides the sorted and unsorted sections of the list byplacing a marker before the first number. To sort the numbers, the computer will

repeatedly search the unsorted section for the smallest number, swap this number with

the first number in the unsorted section, and update the sort marker.

3. To find the smallest number in the unsorted section, the computer must make sixcomparisons: (7 < 8), (7 > 5), (5 > 2), (2 < 4), (2 < 6), and (2 > 3). After these

comparisons, the computer knows that 2 is the smallest number, so it swaps this

number with 7, the first number in the unsorted section, and advances the sort marker.


7/27/2019 DSA 1.pdf

14/18

Lecture Notes: 1

4. Now five more comparisons are needed to find the smallest number in the unsortedsection: (8 > 5), (5 < 7), (5 > 4), (4 < 6), and (4 > 3). After these comparisons, the

computer swaps 3, the smallest number in the unsorted section, with 8, the first

number in the unsorted section, and advances the sort marker.

5. This time four comparisons are needed to determine that 4 is the smallest number inthe unsorted section: (5 < 7), (5 > 4), (4 < 6), and (4 < 8). After these comparisons, the

computer swaps 4 with 5 and then advances the sort marker.

6. After three more comparisons, the computer identifies 5 as the smallest unsortednumber: (7 > 5), (5 < 6), and (5 < 8). Then the computer swaps 5 with 7 and advancesthe sort marker.

7. This time only two comparisons are needed to determine that 6 is the smallestnumber: (7 > 6) and (6 < 8). After these two comparisons, the computer swaps 6 with

7 and then advances the sort marker.

8. Now we only need a single comparison to find the right position for 7: (7 < 8). Since7 is the smallest number and it is also the first number in the unsorted section, the

computer does not need to swap this number. It only needs to advance the sort

marker. Now there is only one number in the unsorted section, so the list of numbersis sorted and the Selection Sort algorithm is complete.


7/27/2019 DSA 1.pdf

15/18

Lecture Notes: 1

Algorithm Analysis

For any given problem, it is quite possible that there is more than one algorithm that

represents a correct solution. A good example of this is the problem of sorting. Dozens of

different algorithms have been written to solve this problem. Given such a wide range of

solutions, how can we determine which algorithm is the best one to use? To do this, we must

analyize our algorithms is such a way that we can gauge the efficiency of the algorithm. Once

we have calculated the efficiency of an algorithm, we can compare our measurements and

select the best solution.

Let's analyze the three sorting algorithms in this section and determine which one was themost efficient solution for sorting the list of seven numbers in our examples. To do this we

need a way to measure the efficiency of our algorithms. We can actually measure the

efficiency in two different ways: space efficiency and time efficiency. An algorithm that is

space-efficient uses the least amount of computer memory to solve the problem of sorting.

An algorithm that is time-efficient uses the least amount of time to solve the problem of

sorting. Since most of the sorting operations are comparisons, copies, and swaps, we can

count these operations and use our results as a measure of time efficiency. The table below

summarizes our measures of time and space efficiency in sorting.

Type of Efficiency Measures

Space Amount of computer memory

Time

# of items copied

# of items swapped

# of items compared

Space Efficiency

Let's begin our analysis by determining which sort was the most space-efficient. We

discussed in our previous lesson that space efficiency can be measured by calculating the

number of memory cells a particular sort requires. For simplicity, we will assume that a

memory cell can hold one number. This means that a sort with five numbers would require at

least five memory cells just to store the numbers in the computer. The total number of

memory cells needed for sorting will depend on how many additional cells the algorithm

requires to order the numbers.

Type of Efficiency Measure

Space Amount of computer memory

Of the three sorts, we see that the Insertion Sort and the Selection Sort were the most space-

efficient. These sorts order the items within the list using the swap operation rather than

copying items to a new list. Since the Simple Sort copies every item to a new list, it requirestwice as much computer memory. Remember, however, that the swap operation requires a


7/27/2019 DSA 1.pdf

16/18

Lecture Notes: 1

temporary memory cell. That is why your answers for the Selection Sort and Insertion Sort

may have been a little surprising.

Time Efficiency

Now let's determine which sort was the most time-efficient. To do this, we will count thenumber of operations each sort performed. Most of the operations that are done by the

computer during sorting fall into two groups: copying numbers or comparing numbers. The

algorithm which requires the least copying and comparing is the one that will execute the

fastest.

Type of Efficiency Measures

Time

# of items copied

# of items swapped

# of items compared

For the Insertion Sort and the Selection Sort, it will be easier to count the number of swaps

that are done rather than the number of copies. Remember that the swap operation requires

three copies. We can find the total number of copies that the algorithms perform by counting

the number of swaps and multiplying by three. The Simple Sort does not use the swap

operation, so you can count the number of copies directly.

Comparison of sorts

Now that you have completed your calculations, let's summarize the results. We already

know that the Insertion Sort and the Selection Sort were the most space-efficient, but we have

yet to determine which sort is the most time-efficient. We will see that this answer is a littlemore difficult to determine.

Space Efficiency

Algorithm # of memory cells

Simple Sort

Insertion Sort

Selection Sort

14

8

8

Time Efficiency

Algorithm # of copies # of comparisons

Simple Sort

Insertion Sort

Selection Sort

7

45

15

42

19

21

Notice that the Simple Sort required the least amount of copies. We would expect this to be

true since it does not swap numbers while sorting. Instead the numbers are copied to a new

list in the computer. This is a common tradeoff between time and space. Although the Simple

Sort loses space efficiency by using two lists, it gains time efficiency because less copies are

required. Of course this does not mean that it is always best to use the Simple Sort to gainmore speed. If we are trying to sort a list of 5 million names the Simple Sort would use too


7/27/2019 DSA 1.pdf

17/18

Lecture Notes: 1

much space in the computer's memory. It would be much better to swap items within the list

rather than create two lists.

For number of comparisons, the Selection Sort and Insertion Sort were nearly the same. The

Simple Sort, however, required twice as many comparisons. We can see the reason for this

difference by thinking about how the algorithms work. Each algorithm repeatedly searchesfor the smallest number and then places this number in the correct position. For the Insertion

Sort and the Selection Sort, each iteration of this process reduces the unsorted section by one

number. During the next search, the computer does not need to make as many comparisons to

find the smallest number. The Simple Sort, however, replaces sorted numbers with a marker

called MAX. Each time the computer searches for the smallest number, it must compare all

seven memory cells. This approach is much less efficient.

Given the particular set of seven numbers we sorted, the Selection Sort was the most time-

efficient. However, it is important to understand that this may not be true for every set of

seven numbers. Consider the following example.

If we use the Insertion Sort on these numbers only 8 comparisons and 1 swap would be

needed to sort them. However, if we use the Selection Sort, 21 comparisons and 1 swap

would be needed. In this case, the Insertion sort is more efficient.

Worst case comparison

Now that we have formulas to describe the worst case time efficiency of our sorting

algorithms, we can see how the algorithms will perform as we increase the number of items

to be sorted. First, let's review our formulas. Notice that the formulas for swaps have been

converted to copies by multiplying by 3. This allows us to correctly compare Simple Sort

with the other two sorts.

Algorithm Comparisons Copies

Simple Sort n * (n-1) n

InsertionSort

(n-1) + (n-2) + ... + 1 3 * [(n-1) + (n-2) + ... + 1]

Selection

Sort(n-1) + (n-2) + ... + 1 3 * (n-1)

Using our formulas, we will compute the number of comparisons and copies needed to sort

several lists of items. The number of items in each list increases by a power of ten.


7/27/2019 DSA 1.pdf

18/18

Lecture Notes: 1

Comparisons Required

# of Items Simple Sort Insertion Sort Selection Sort

10 90 45 45

100 9,900 4,950 4,950

1,000 999,000 499,500 499,500

10,000 99,990,000 49,995,000 49,995,000

From our table of comparisons, we can see two important facts. First, the Simple Sort always

requires twice as many comparisons as the Insertion and Selection Sorts. This means we can

rewrite the formula for these sorts as follows: (n * (n-1)) / 2. This expression is much easier

to use than the summation (n-1) + (n-2) + ... + 1, especially when n is large.

Second, all three sorts require a tremendous number of comparisons as the number of items

increases. In fact, we can see with the Simple Sort that the number of comparisons is very

close to n2. Take another look at the formula you derived for the number of comparison theSimple Sort requires in the worst case. If we multiply the n into the parentheses, the formula

becomes n2 - n. Now you see why our total comparisons is very near n

2. You can also see

why the number of comparisons needed is growing so quickly.

Copies Required

# of Items Simple Sort Insertion Sort Selection Sort

10 10 135 27

100 100 14,850 297

1,000 1,000 1,498,500 2,99710,000 10,000 149,985,000 29,997

Now let's compare the number of copies each algorithm requires. Notice how quickly the

number of copies required by the Insertion Sort is growing. This makes us suspect that the

formula for the Insertion Sort contains an n2 term. In fact, the formula does contain such a

term, but we must rewrite the formula to see it. Using the relationship we discovered earlier,

we can replace the summation (n-1) + (n-2) + ... + 1 with (n * (n-1)) / 2 and simplify.

# of copies = 3 * [(n-1) + (n-2) + ... + 1]

# of copies = 3 * [(n * (n-1)) / 2]

# of copies = (3 / 2) * [n * (n-1)]

# of copies = (3 / 2) * [n2 - n]

The simplified formula shows the n2 term that we suspected. Notice that the other two

formulas only have an n term rather than a n2 term. This is the reason that they grow slowly

as the number of items increases.

N t C il d f III S B T h CSE C & D b t h Page 18

DSA 1.pdf

Documents