Page 1
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
COEN 352 Data Structures and Algorithms:Mathematical Preliminaries
Stuart Thiel
Concordia UniversityDepartment of Electrical Computer Engineering
Summer, 2015
1/41
Page 2
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Outline
Notation
Sets
Sets and Relations
Identities
Recursion
Proof Techniques
2/41
Page 3
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Size vs. Time
I Size is often a power of 2
I Time is usually decimal
I Shaffer talks about his preferred notation in 2.2, I willattempt to avoid ambiguity
3/41
Page 4
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Stirling’s Approximation of n!
I n! ≈√
2πn(ne )n
I We often write n! but are using the approximation forresulting analysis
I proving this not for the faint of heart, but fascinating!
I This and other stuff Stirling did is critical to theunderstanding of advanced algorithm analysis.
4/41
Page 5
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Combinatorics
I Permutation S is the members of S arranged in someorder
I n members leads to n! possible permutations
I Often need to generate random permutations foranalysis
I How many do you need to test?
I What does “random” mean?
I How real is “random”?
5/41
Page 6
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Notations
I A boolean value is true or false
I A =⇒ B means if A then B
I A ∧ B means A and B
I A ∨ B means A or B
I A means ∼ A means not A, or the negation of AI a%b is the modulus operator, gives remainder of integer
divisionI different languages can do modulus differently, be
careful
6/41
Page 7
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
More Notations
I dxe ceiling rounds up to the nearest larger integer
I bxc floor rounds down to the nearest smaller integer
7/41
Page 8
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Sets
I useful abstraction for algorithm design
I distinguishable elements of a base type.
I elements are primitive elements of base type
I elements may also be sets
I no duplication
I no order
8/41
Page 9
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Set Example
I P = {7, 11, 42}I elements are 7, 11 and 42
I P is the set
I Integers are the base type
I Sets can be empty: ∅
9/41
Page 10
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Set Definitions
I enumerating each element can be long
I formal definition can communicate well
I “All positive integers”
I “All even integers”
I “All prime numbers”
10/41
Page 11
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Set Definition Example
I P = {x |x is a positive integer}I P = {x |x > 0}I more specific: P = {x |x > 0 ∧ x ∈ Z}
11/41
Page 12
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Set Operations with Elements
I in a set
I not in a set
I cardinality of a set
12/41
Page 13
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Set Operations with Elements Example
I x ∈ P
I x /∈ P
I Q = {7, 11, 42}I |Q| = 3
13/41
Page 14
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Set Operations with Other Sets
I subset
I supersets
I union
I intersection
I difference
14/41
Page 15
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Set Operations with Other Sets Example 1
I P = {x |x > 0 ∧ x ∈ Z}I Q = {x |x ∈ Z}I P ⊆ Q
I Q ⊇ P
15/41
Page 16
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Set Operations with Other Sets Example 2
I P = {4, 12, 18}I Q = {10, 12, 22}I P ∩ Q = {12}I P ∪ Q = {4, 10, 12, 18, 22}I P \ Q = {4, 18}
16/41
Page 17
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Powerset
I All possible subsets of a set
I Includes empty set!
17/41
Page 18
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Bags
I Bags are Sets that allow duplication
I using square brackets is a common distinction
I P = [4, 12, 12, 18]
I also called multisets
I useful in advanced analysis!
18/41
Page 19
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Sequence
I A bag with an order
I using angle brackets is a common distinction
I P = 〈4, 12, 12, 18〉I They need not be ordered a particular way
I also called tuple or vector
I like our implemented data structures
I 〈4, 12, 12, 18〉 6= 〈12, 4, 12, 18〉
19/41
Page 20
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Set Context
I Set? Bag? Sequence?
I The context is often the guide
I If lucky, brackets distinguish
I Sorting problem? Looks like sets, probably sequences.
20/41
Page 21
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Relations
I A relation over a set is a set of order pairs
I A set of sequences of cardinality 2
I elements in the ordered pairs are elements from the set
21/41
Page 22
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Relations example
I Set P = {a, b, c}I Relation R1 = {〈a, c〉, 〈b, c〉, 〈c , b〉}I Relation R2 = {〈a, a〉, 〈a, c〉, 〈b, b〉, 〈b, c〉, 〈c , c〉}I How the left item relates to the right item
I lRr
I How many different relations can there be on a set?
22/41
Page 23
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Properties of Relations
I Useful to classify relations
I Good for describing, identifying types of problem
I reflexive, ∀a ∈ S : aRa
I symmetric, ∀a, b ∈ S : aRb =⇒ bRa
I antisymmetric, ∀a, b ∈ S : aRb ∧ bRa =⇒ a = b
I transitive, ∀a, b, c ∈ S : aRb ∧ bRc =⇒ aRc
I useful to group of these properties
23/41
Page 24
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Properties of Relations Example
I ordinal relation of less than or equal is reflexive
I being a partner is symmetric
I natural numbers are antisymmetric
I being a relative is transitive
24/41
Page 25
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Equivalence Relation
I reflexive, symmetric and transitiveI breaks a set into equivalence classesI These partitions are disjoint from all othersI Elements in these partitions are “equivalent”
I “less than or equal” is an ok example
I modulus is a better example
25/41
Page 26
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Partial Order Relation
I Partial Ordering Definitions Differ. We use Shaffer.
I antisymmetric and transitive
I “older than” is a good example
26/41
Page 27
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Advantage of Identities
I You can replace expressions with other equivalentexpressions
I You can use identities to change the tools you can useI summation may be tricky in math, but it has an identity
that is regular arithmeticI Identities that lead to arithmetic expressions are called
closed-form solutions
I You can make patterns more visible
27/41
Page 28
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Outline I
Notation
Sets
Sets and Relations
IdentitiesLogarithmsSummation
Recursion
Proof Techniques
28/41
Page 29
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Logarithms
I logarithm operation finds the exponent that a givenbase is raised to in order to equal the parameter
I log usually implies base 2
I ln usually implies base e
I log 2b = b
I ln eb = b
I log3 3b = b
29/41
Page 30
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Logarithmic Identities
I log ab = log a + log b
I log a/b = log a− log b
I log xb = b log x
I log b√x = 1
b log x
I eg: log3 81 = log3 92 = 2 log3 9 = 2 log3 32 = 4
30/41
Page 31
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Outline I
Notation
Sets
Sets and Relations
IdentitiesLogarithmsSummation
Recursion
Proof Techniques
31/41
Page 32
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Analysing Loops and Recurrences
I We want to count costs
I We are looking at algorithms
I algorithms have repeating structures
I count these structures
I evaluate the cost of each
I summation gives us a tool to help
I a powerful tool for analysis
In∑
i=1f (i)
32/41
Page 33
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Summation identities
In∑
i=1i = n(n+1)
2
In∑
i=1i2 = n(n+1)(2n+1)
6
33/41
Page 34
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Identifying Recurrence Relations
I Looking for closed-form solutions can be done
I Simply, one can try to unfold a summation thatdescribes an algorithm
I You look for a pattern
I Using identities can swap out unfamiliar expressions andhelp you see patterns
I An advanced topic, but we will look at one importantexample analysing a sorting algorithm later this semester
34/41
Page 35
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Loops or Recursion
I an algorithm may call itself on a smaller sub-problem
I this makes it a recursive algorithm
I such algorithms have one or more base cases (orterminating cases) that can give a direct solutionwithout recursion
I such algorithms otherwise break the problem down,including the recursive call that will eventually lead to abase case
35/41
Page 36
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Recursion Example
I Factorials
I Towers of Hanoi
36/41
Page 37
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Proofs
I Do not be sloppy
I Understanding proofs is key
I As Engineers, proofs are the basis for justifyingeverything you do
I You should always be comfortable justifying everythingyou do
I People question you, but you should always be able toprove what you did is correct
37/41
Page 38
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Direct Proof
I The easy one. Logical Explanation.
I Logic or arithmetic leads directly there.
I I also include the use of accepted identities here.
I If the exam is tomorrow, it is not today. Today is nottomorrow.
38/41
Page 39
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Proof by Contradiction
I Can disprove with one counter-example.
I No number of examples can prove something true.
I Not true, ALL examples are a proof, but that is usuallyintractable.
I We can do something similar:
I Assume what you want to prove is false.
I Find a counter-example for this.
I eg: There is no largest integer!
39/41
Page 40
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Proof by Induction
I Looks like Recursion, really
I Not a coincidence!
I Base case(s)
I Induction Step
40/41
Page 41
COEN 352 DataStructures andAlgorithms:MathematicalPreliminaries
Stuart Thiel
Notation
Sets
Sets and Relations
Identities
Logarithms
Summation
Recursion
Proof Techniques
Proof by Induction Examples
I Summation
I Coin Example
41/41