Elements of Programming Alexander Stepanov Paul McJones March 14, 2008 Stepanov, McJones Elements of Programming March 14, 2008 1 / 880
Elements of Programming
Alexander Stepanov Paul McJones
March 14, 2008
Stepanov, McJones Elements of Programming March 14, 2008 1 / 880
Contents I
1 Introduction
2 Foundations
3 Transformations and their orbits
4 Algorithms on algebraic structures
5 Orderings
6 Combining concepts
7 Refining concepts of iterators
Stepanov, McJones Elements of Programming March 14, 2008 2 / 880
Contents II
8 Permutations and rearrangements
9 Rotations
10 Algorithms on increasing ranges
11 Coordinate structures
12 Composite objects
13 Iterative algorithms for divide-and-conquer
14 Mathematical notation
Stepanov, McJones Elements of Programming March 14, 2008 3 / 880
Contents III
15 C++ machinery
16 Acknowledgments
17 Index
Stepanov, McJones Elements of Programming March 14, 2008 4 / 880
Approach
Programs are mathematical objectsMathematics provides tools for
reasoningderivingcomposingoptimizing
Mathematics applied to programming addressessecurityreliabilityperformanceproductivity
Stepanov, McJones Elements of Programming March 14, 2008 5 / 880
Blending programming and mathematics
1 Start with a practical algorithm2 Define mathematical requirements on which it depends3 Implement it abstractly and efficiently4 Place it in the general taxonomy of requirements and algorithms
Stepanov, McJones Elements of Programming March 14, 2008 6 / 880
Intended audience
This book is for those who seek deeper understanding ofprogramming and are willing to invest substantial effort workingthrough the materialThe book is not encyclopedic and is designed to be readsequentially, in its entirety1
While this book does not have formal prerequisites, programmingmaturity and a readiness to deal with serious mathematics areassumedWe present a unified approach to programming spanning from thehighest level of abstraction down to the machine
1Perhaps more than onceStepanov, McJones Elements of Programming March 14, 2008 7 / 880
The choice of C++
We use a small subset of C++ as a good approximation to the idealteaching languagePrograms in this book deal with both memory addresses andabstract mathematical theories
C defines an abstract machine that serves as a foundation formodern processor architectures2
C++ extends Cwith mechanisms for writing abstract versions ofalgorithms and data structures without sacrificing efficiency
2C does not address the issues of memory hierarchies, instruction-level parallelism,and thread-level parallelism; research is needed to incorporate these features into thesystem programming language of the future
Stepanov, McJones Elements of Programming March 14, 2008 8 / 880
The relationship to C++ STL
The book and C++ STL are results of the same researchThe book departs from it significantly by providing improvedinterfaces in a number of placesThe book does not attempt to provide a full set of libraryinterfaces or deal with all the issues a real library must address
Stepanov, McJones Elements of Programming March 14, 2008 9 / 880
The machine model
Our machine model is abstracted from C/C++Sequential (single-threaded) stacked-based executionA linear address space, not necessarily all accessibleMemory allocation and deallocation under programmer control
Our programs and techniques can be used as building blocks forsystems with concurrent execution and automatic storagedeallocation
Stepanov, McJones Elements of Programming March 14, 2008 10 / 880
C++ usage conventions
We avoid implicit conversions to make our code strongly typedwith respect to concepts 3
We do not use postincrement ++ nor do we depend on the returnvalue of preincrement ++We omit inline since its use depends on compiler, targetplatform, and the code
3Therefore all the constructors defined in the book should be considered to beexplicit
Stepanov, McJones Elements of Programming March 14, 2008 11 / 880
Our use of mathematics
We present only enough mathematics to provide the appropriatesettings for the programs in the bookWe strive for the level of rigor typical in modern mathematicaltextbooks, while attempting to preserve mathematical andalgorithmic intuitionsAll the lemmas in this book can be proved via straightforwardderivations from the provided definitionsWe urge the reader to prove every lemma to ensure truecomprehension
Stepanov, McJones Elements of Programming March 14, 2008 12 / 880
Outline of the book
Part I: Algorithms on mathematical abstractionsChapter 1 - FoundationsChapter 2 - Transformations and their orbitsChapter 3 - Algorithms on algebraic structuresChapter 4 - OrderingsChapter 5 - Combining concepts
Part II: Algorithms on abstractions of memoryChapter 6 - Refining concepts of iteratorsChapter 7 - Permutations and rearrangementsChapter 8 - RotationsChapter 9 - Algorithms on increasing rangesChapter 10 - Coordinate structuresChapter 11 - Composite objectsChapter 12 - Iterative algorithms for divide-and-conquer
Appendix 1 - Mathematical notationAppendix 2 - C++ machinery
Stepanov, McJones Elements of Programming March 14, 2008 13 / 880
Contents I
1 Introduction
2 FoundationsMeaningObjectsRelations on objectsProceduresAssignment, destruction, and constructionRelationships between typesConcept RegularProcedures as objectsGeneralized proceduresParameter passingRegular functions
Stepanov, McJones Elements of Programming March 14, 2008 14 / 880
Contents II
Complexity
3 Transformations and their orbits
4 Algorithms on algebraic structures
5 Orderings
6 Combining concepts
7 Refining concepts of iterators
8 Permutations and rearrangements
Stepanov, McJones Elements of Programming March 14, 2008 15 / 880
Contents III
9 Rotations
10 Algorithms on increasing ranges
11 Coordinate structures
12 Composite objects
13 Iterative algorithms for divide-and-conquer
14 Mathematical notation
15 C++ machinery
Stepanov, McJones Elements of Programming March 14, 2008 16 / 880
Contents IV
16 Acknowledgments
17 Index
Stepanov, McJones Elements of Programming March 14, 2008 17 / 880
Things, ideas, and descriptions
Humans observe and interact with real thingsWe observe things: our minds form ideas (mental images) ofthem; we act: our ideas affect things or create new thingsWe create descriptions of our ideas to preserve them,communicate them, or allow them to participate in manual andautomatic processesNote that a description is itself a thing
Stepanov, McJones Elements of Programming March 14, 2008 18 / 880
Universe of ideas
A value is an abstract idea (logical or mathematical) such as falseor 2 or a geometrical pointAn entity is an idea of a thing such as a car or an email message
An entity has attributes with values, such as job level or salary of anemployeeAn entity can be composed of other entities called parts, such as theengine of a car or the body of an email messageAn entity can be in relationships with other entities, such as thesupervisor of an employee
An activity is an idea of changes performed on entities thatchanges their attributes or creates new entities, such as promotinga given employeeAn abstraction is what is common to a class of entities
An abstraction of values, such as IntegerAn abstraction of entities, such as EmployeeAn abstraction of activities, such as Promotion
Stepanov, McJones Elements of Programming March 14, 2008 19 / 880
Entities
When we form an idea of an entity, we grasp:Its existenceIts natureActivities affecting itIts particulars: attributes, parts, and relationships
In computer programsExistence corresponds to object identityNature corresponds to object typeActivities correspond to proceduresParticulars correspond to object state, object components, andobject relationships
Stepanov, McJones Elements of Programming March 14, 2008 20 / 880
Values
Similar values (natural numbers, real numbers, polygons, etc.)belong to different sorts
Stepanov, McJones Elements of Programming March 14, 2008 21 / 880
The universality of mathematics
“Mathematics, in general, is the science of quantity; or, the sciencewhich investigates the means of measuring quantity” 4
Mathematical reasoning, including discrete arithmetic reasoningand continuous spatial reasoning, is a fundamental, innate abilityof the human mind
4Leonard Euler. Elements of Algebra. Third Edition, Longman, London, 1822.Stepanov, McJones Elements of Programming March 14, 2008 22 / 880
Continuity and discreteness
Ancient mathematicians (the early Pythagoreans) discoveredspace can’t be represented using discrete quantitiesThe arithmetization of geometry required the introduction of thereal numbersThe continuity of physical processes allows us to effectivelyapproximate real numbers with discrete, finite-precision quantitiesScientists and engineers have encountered no problem wherecareful approximation has failed to workThis is the key to the applicability of digital computers
Stepanov, McJones Elements of Programming March 14, 2008 23 / 880
Computer descriptions of entities
As we said earlier, we understand the world in terms of entitiesComputer programmers map these entities into objects containingdiscrete, bounded quantities approximating the values of theattributesThe meaning of a program is in the mind, not in the computer
Stepanov, McJones Elements of Programming March 14, 2008 24 / 880
Objects
DefinitionAn object is a set of memory locations and other resources togetherwith primitive procedures defined on it that provide itscomputational basisThe state of an object is the addresses of its memory locationstogether with the content of those locations at a particular point intimeThe interpretation of an object is a mapping from its state to anentity
An interpretation maps states in such a way that procedures on theobject are consistent with the corresponding activities on entities
Stepanov, McJones Elements of Programming March 14, 2008 25 / 880
Well-formed object states
DefinitionAn object is in a well-formed state if there is an entity corresponding to it
ExamplesEvery state of a computer word is well-formed when interpretedas an integerA pointer containing an illegal address is not well-formedAn IEEE 754 floating point NaN (Not a Number) is notwell-formed when interpreted as a real number
Stepanov, McJones Elements of Programming March 14, 2008 26 / 880
Partial and total representations
DefinitionA representation for an entity of a given nature is an object statecorresponding to it; the entity is representable if such a state exists
DefinitionA representation is partial if not every entity is representable by anobject state; a representation is total if it is not partial
ExampleThe type int is a partial representation of integers
Stepanov, McJones Elements of Programming March 14, 2008 27 / 880
Unique representations
DefinitionA representation is unique if there is at most one object statecorresponding to every set of particulars of the entity
ExamplesA representation of a truth value as a byte that interprets zero asfalse and nonzero as true is not uniqueSigned-magnitude representation does not provide a uniquerepresentation of zero
Stepanov, McJones Elements of Programming March 14, 2008 28 / 880
Identity
In the real world, identity is conceptually straightforward, butoften slippery to deal with in specific situations
It was a major astronomical discovery that the Morning Star andEvening Star are identicalThe question of whether the author of Hamlet and WilliamShakespeare identical is still debatedWhen does replacing parts change the identity?
A computer programmer must determine how to map the identityof real things to objectsMultiple techniques are possible for representing object identity
A memory addressAn index into a data structureA unique value of an attribute (e.g., Social Security Number)
Stepanov, McJones Elements of Programming March 14, 2008 29 / 880
Equality
Leibniz’s lawx = y if, and only if, everything that may be said about any one of thethings x or ymay also be said about the other a
aAlfred Tarski, Introduction to Logic and to the Methodology of Science, Oxford, 1965
Stepanov, McJones Elements of Programming March 14, 2008 30 / 880
Properties of equality
Formalization of Leibniz’s Law:
x = y⇔ ∀P(P(x)⇔ P(y))
The following properties follow:reflexivity x = x
symmetry (x = y)⇔ (y = x)
transitivity ((x = y) ∧ (y = z))⇒ (x = z)
Stepanov, McJones Elements of Programming March 14, 2008 31 / 880
Equality in mathematics
Equational reasoning is the basis of mathematics
Arithmetic: 24 = 1
2Geometry: 4ABC = 4ACB
Algebra: (a+ b)2 = a2 + 2ab+ b2
Stepanov, McJones Elements of Programming March 14, 2008 32 / 880
Equality in the real world
Equality generally implies equality of essential attributesNot all attributes are necessarily essentialEssentialness of attributes often depends on context
All US quarters are equal to a fare collector, but not to a coincollector
Equality is not the same as identityA thing’s attributes can change without affecting its identityDistinct things can have equal attributes and, therefore, be equalthemselves
Stepanov, McJones Elements of Programming March 14, 2008 33 / 880
Equality in programming
This book extends Leibniz’s law from unchanging values tomutable software objectsLeibniz’s law is the fundamental property of equality, but is not aconstructive definition
It establishes the consequences of the equality of two integersIt does not describe how to compare them for equality
Developing constructive implementations of equality is one of thetasks of this book
Stepanov, McJones Elements of Programming March 14, 2008 34 / 880
Genus
DefinitionTwo objects are of the same genus if they represent entities with thesame nature under their corresponding interpretations
ExamplesPolar and Cartesian representations of a complex number are ofthe same genusTwo’s-complement and signed-magnitude representations ofintegers are of the same genus16-bit and 32-bit unsigned representations of natural numbers areof the same genus
Stepanov, McJones Elements of Programming March 14, 2008 35 / 880
Type
DefinitionTwo objects have the same type if they share the same procedures andthe same interpretation
Stepanov, McJones Elements of Programming March 14, 2008 36 / 880
Intuition for type
The English word type derives from the Greek word tupoc (tupos):“the impress made by the blow, what is formed, what leaves itsimpress, the form-giving form, hence form generally as outline”5
The technical sense of tupos was “. . . the pattern in conformity towhich a thing must be made” 6
In programming, type is a fundamental idea preceding any notionof type in programming languagesA type is a design for objects, as an engineering drawing is adesign for manufactured objects
5Kittel and Friedrich. Theological Dictionary of the New Testament, Volume 8,Eerdmans, 1972, page 246
6Thayer and Smith. “Greek Lexicon entry for Tupos”. “The New Testament GreekLexicon”
Stepanov, McJones Elements of Programming March 14, 2008 37 / 880
Equality of objects
DefinitionTwo objects of the same genus are equal if their states represent entitieswith equal essential particulars
Like Leibniz’s Law, this is not a constructive definition; it,however, allows us to apply the mathematical understanding ofequality on abstract values to the world of programming
Stepanov, McJones Elements of Programming March 14, 2008 38 / 880
Nonessential objects
DefinitionA nonessential object is one whose value does not affect thecomputation
ExampleAn object used only to instrument or tune the performance isnonessential
Stepanov, McJones Elements of Programming March 14, 2008 39 / 880
How procedures interact with objects
A procedure interacts with the state of the computation throughfour kinds of objects:
Arguments are objects given to a procedure at its point of invocationLocal state is objects created and accessed during a single invocationof the procedureGlobal state is objects accessible to this and other procedures acrossmultiple invocationsOwn state is objects accessible only to this procedure but sharedacross multiple invocations
Stepanov, McJones Elements of Programming March 14, 2008 40 / 880
Domain, argument types, and definition space
In mathematics, a function typically takes a single argument, or anoperation such as addition takes two argumentsIn programming, it is common for a procedure to take manyargumentsRather than defining the domain of a procedure as the directproduct of the types of its arguments, we refer to the argumenttypes of a procedureIn the next chapter we extend the definition of domain tohomogeneous functions (all of whose arguments are of the sametypes)
DefinitionThe definition space for a procedure is that subset of the direct productof its argument types to which the procedure is intended to be applied
Stepanov, McJones Elements of Programming March 14, 2008 41 / 880
Codomain and result space
DefinitionThe codomain for a procedure is the type it returns
DefinitionThe result space for a procedure is the set of values from its codomainreturned by the procedure for some inputs from its definition space
Stepanov, McJones Elements of Programming March 14, 2008 42 / 880
Total and partial procedures
DefinitionA procedure is partial if its definition space is a subset of the directproduct of the types of its arguments; it is total if they are equala
aIn mathematical usage, “partial” includes “total”; for example, total orderingimplies partial ordering
There is a (potentially implicit) predicate, is_defined, that is truewhen applied to a procedure and set of arguments lying withinthe definition space of that procedurePreconditions are used to specify the definition spaces forproceduresBefore a partial procedure is called, either its precondition must besatisfied, or the call must be guarded by a call on is_defined7
7For example of the former, seeweak_remainder in Chapter 5, and for an exampleof the latter, see collision_point in Chapter 2
Stepanov, McJones Elements of Programming March 14, 2008 43 / 880
Terminating and non-terminating procedures
DefinitionA procedure is terminating, semi-terminating, or non-terminatingdepending on whether it always terminates, sometimes terminates, ornever terminates on its definition space
A partial procedure may terminate even for arguments outside itsdefinition spaceAll the procedures in this book are terminating unless explicitlystated otherwiseA terminating procedure is known as an algorithm
Stepanov, McJones Elements of Programming March 14, 2008 44 / 880
Stateful and indexed procedures
DefinitionA procedure is stateful, indexed, or stateless depending on whether itsown state is mutable, immutable, or empty
Stepanov, McJones Elements of Programming March 14, 2008 45 / 880
Proper and improper procedures
DefinitionA procedure is proper if it does not modify any essential objects in itsglobal state; otherwise it is improper
All the procedures in this book are proper
Stepanov, McJones Elements of Programming March 14, 2008 46 / 880
Functions and mutators
DefinitionA function is a procedure that does not modify its arguments andreturns a newly constructed object called its result; a mutator is aprocedure whose main purpose is to modify its arguments
Stepanov, McJones Elements of Programming March 14, 2008 47 / 880
Procedure examples
Addition is a total function on integers, but is partial on intDivision is a partial function on integers since its definition spaceomits zero as a value for its second argumentA daemon is a non-terminating procedureThe Knuth-Bendix procedure in automatic theorem proving issemi-terminatingSorting is a terminating proper mutatorA pseudo-random number generator that keeps its state in an ownvariable is a stateful proper functionMultiplication mod k for a fixed k is an indexed proper functionwith k being its indexSetting a global graphics state variable is an improper mutator
Stepanov, McJones Elements of Programming March 14, 2008 48 / 880
Assignment
DefinitionAn assignment is a mutator taking two objects of the same genus whoseeffect is to make the first object equal to the second without modifyingthe second
The final interpretation of the first object must equal theinterpretation of the second object
Stepanov, McJones Elements of Programming March 14, 2008 49 / 880
Destruction
DefinitionA destructor is a mutator causing the cessation of an object’s existence
After a destructor has been invoked on an object, no procedurecan be applied to it and its former memory locations andresources may be reused for other purposes
Stepanov, McJones Elements of Programming March 14, 2008 50 / 880
Partially-formed state
DefinitionAn object is in a partially-formed state if it can be assigned to ordestroyed
For an object that is partially-formed but not well-formed, theeffect of any procedure other than assignment (only on the leftside) and destruction is not defined
Stepanov, McJones Elements of Programming March 14, 2008 51 / 880
Construction
DefinitionA constructor is a mutator transforming memory locations into anobject
The possible behaviors range from doing nothing to establishingcomplex object state
Stepanov, McJones Elements of Programming March 14, 2008 52 / 880
Type constructors
DefinitionA type constructor is a mechanism for creating a new type from one ormore existing types
ExamplesIn C++, T* is the type “pointer to T”, for any type TIn C++, struct{T0, ..., Tn−1} is an n-ary type constructorIn Chapter 10 we show how to construct types corresponding todata structures such as lists and arrays
Stepanov, McJones Elements of Programming March 14, 2008 53 / 880
Type attributes
DefinitionA type attribute is a mapping from a type to a value describing somecharacteristic of the type
ExamplesThe size of an object in bytesThe alignment of an objectThe number of members in a struct
DefinitionIf F is a procedure type, Arity(F) returns its number of arguments
Stepanov, McJones Elements of Programming March 14, 2008 54 / 880
Type functions
DefinitionA type function is a mapping from a type to an affiliated type
ExamplesGiven “pointer to T”, the value type TThe result type of the difference of two pointers of a given typeThe type of the ith member of a struct type (counting from 0)
DefinitionIf F is a procedure type and i < Arity(F), ArgumentType(F, i)returns the type of the ith argument (counting from 0)If F is a procedure type, Codomain(F) returns the type of theresult
In Appendix 2 we show how to define some type functions in C++
Stepanov, McJones Elements of Programming March 14, 2008 55 / 880
Concepts
DefinitionA concept is a predicate on types stated in terms of properties ofprocedures, type attributes, and type functions defined on the types
DefinitionA concept is said to be modeled by specific types, or the types modelthe concept, if the properties of the concept are satisfied for thosetypesTo assert that a concept C is modeled by types t0, . . . , tn−1, wewrite C(t0, . . . , tn−1)
Stepanov, McJones Elements of Programming March 14, 2008 56 / 880
Refinement and weakening of concepts
DefinitionConcept C ′ refines concept C if whenever C ′ is satisfied for a set oftypes, C is also satisfied for those types a
We say C weakens C ′ if C ′ refines C
aTechnically, it is sufficient for C to hold for a subset of the types, possibly reordered
Stepanov, McJones Elements of Programming March 14, 2008 57 / 880
Type concepts
DefinitionA type concept is a concept defined on one type
Stepanov, McJones Elements of Programming March 14, 2008 58 / 880
Examples of type concepts in C++
ExamplesIntegral type
Unsigned integral typeModeled by: unsigned, unsigned long, unsigned char
Signed integral typeModeled by: int, long, char
Pointer typeModeled by: int*, char*
SequenceModeled by: std::vector<int>, std::list<int>
Bidirectional iteratorModeled by: std::list<int>::iterator, int*
Stepanov, McJones Elements of Programming March 14, 2008 59 / 880
Primitive type concepts
ProcedureProperProcedureFunction
Stepanov, McJones Elements of Programming March 14, 2008 60 / 880
Concept HomogeneousFunction
DefinitionHomogeneousFunction(F)⇒
Function(F)
Arity(F) > 0For all i, j such that 0 6 i, j < Arity(F),ArgumentType(F, i) = ArgumentType(F, j)
DefinitionThe type function Domain(F) is defined on any function type Fsatisfying HomogeneousFunction(F):
Domain(F) ≡ ArgumentType(F, 0)
Stepanov, McJones Elements of Programming March 14, 2008 61 / 880
Requirements for interoperability of components
We want a sort procedure that works on an array of elementsThe requirements on the element type allowing objects to besorted should be satisfied by arrays themselves, so we can sort anarray of arraysAll built-in types (e.g., int, float) should satisfy these commonrequirementsThe requirements are based on equational reasoningSince the requirements assure regularity of behavior andinteroperability, we call the type concept corresponding to themregular and the types modeling it regular types 8
8Regular types were first introduced in:
James C. Dehnert and Alexander A. Stepanov.Fundamentals of Generic Programming.Report of the Dagstuhl Seminar on Generic Programming, Schloss Dagstuhl,Germany, April 1998; also appears in Lecture Notes in Computer Science (LNCS)volume 1766, pages 1-11.
Stepanov, McJones Elements of Programming March 14, 2008 62 / 880
Concept Regular
A Regular type has9:EqualityAssignmentDestructorDefault constructorCopy constructorDefault total ordering (defined in Chapter 4)
9In Chapter 12 we extend Regular with the notion of underlying type and itsaffiliated types and functions
Stepanov, McJones Elements of Programming March 14, 2008 63 / 880
Equality for Regular
A regular type must implement equalityEquality should be defined on well-formed object states; twoobjects are equal if and only if they represent the same abstractvalueEquality is written as a = b and inequality is written as a , b
Stepanov, McJones Elements of Programming March 14, 2008 64 / 880
Assignment for Regular
A regular type must implement assignmentAssignment is written as a← b;
Stepanov, McJones Elements of Programming March 14, 2008 65 / 880
Destructor for Regular
A regular type must implement a destructor
Stepanov, McJones Elements of Programming March 14, 2008 66 / 880
Default constructor for Regular
A regular type must implement a default constructorThis is a constructor that takes no arguments and leaves the objectin a partially-formed stateThe default construction for a local object a of type T is written T a;
Stepanov, McJones Elements of Programming March 14, 2008 67 / 880
Copy constructor for Regular
A regular type must implement a copy constructorThis is a constructor that takes an additional argument of the sametype and constructs a new object equal to itThe effect of the copy constructor is equivalent to defaultconstruction followed by assignmentThe copy constructor for a local object a of type T with initialvalue b is written T a← b;
Stepanov, McJones Elements of Programming March 14, 2008 68 / 880
Additional functions and affiliated types for Regular
In later chapters we introduce additional functions and affiliatedtypes required for a Regular type
Stepanov, McJones Elements of Programming March 14, 2008 69 / 880
Procedures as objects
A procedure is itself an object, and thus can be constructed,destructed, assigned, copied, and tested for equalityThese operations on a procedure are all operations on its own stateTwo procedures of different type that perform the samecomputation belong to the same genus
For example, heapsort and quicksort
Stepanov, McJones Elements of Programming March 14, 2008 70 / 880
Generalized procedures
DefinitionA generalized procedure is a procedure defined in terms of typerequirements
Each type requirement has a formal parameter name, introducedvia a typename clause following the template keywordRequirements on the types are specified via the requires clause,whose argument is an expression built up from actual types, formaltypes, applications of type functions, type equality, concepts, andtheir logical connectivesa
aIn this book we use only conjunction
Stepanov, McJones Elements of Programming March 14, 2008 71 / 880
Syntax of type expressions and requires clauses
<type expression> ::= <type> | <formal type> |<type function> ( <type expression> )
<type predicate> ::= <type expression> == <type expression> |<type concept> ( <type expression )
<requires expression> ::= <type predicate> |<requires expression> && <type predicate>
<requires clause> ::= requires( <requires expression> )
A <type expression> can be used to declare procedurearguments and local variablesOur requires clause is implemented in Appendix 2 with a simplemacro10
10In the future C++ is expected to include support for concepts with different syntaxStepanov, McJones Elements of Programming March 14, 2008 72 / 880
Example of a generalized procedure
template <typename Op>requires(SemigroupOperation(Op))
Domain(Op) square(const Domain(Op)& x){
return op(x, x);}
Stepanov, McJones Elements of Programming March 14, 2008 73 / 880
Passing parameters to procedures
We distinguish three cases of how a procedure uses an objectpassed to it as a parameter:For reading The procedure depends only on the initial value of
the object, not on its identityFor writing The procedure mutates the object
For selecting The procedure selects one of several objects(without mutating any of them) and returns theselected object (not a copy)
Stepanov, McJones Elements of Programming March 14, 2008 74 / 880
Passing parameters for reading
To pass a parameter for reading in C++, we use one of twotechniques
If the size of the parameter is small, we pass it by value (whichmakes a local copy)Otherwise, we pass it by const referenceIf the procedure needs a local copy it can mutate, we pass it byvalue even when it is large
template <typename I>requires(Readable(I) && Iterator(I))
I find(I f, I l, const ValueType(I)& x){
while (f != l && source(f) != x)++f;
return f;}
Regular types allow us to use call by value: there is a copyconstructor, and it creates a copy equal to the original
Stepanov, McJones Elements of Programming March 14, 2008 75 / 880
Passing parameters for writing
To pass a parameter for writing in C++, we pass it by reference
template <typename T>requires(Regular(T))
void swap(T& x, T& y){
T tmp = x;x = y;y = tmp;
}
(In a later chapter we show a more efficient way to write swap)
Stepanov, McJones Elements of Programming March 14, 2008 76 / 880
Passing parameters for selecting
To pass a parameter for selecting in C++, we must provide bothby reference and by const reference versions of the procedure:
template <typename T>requires(StrictTotallyOrdered(T))
T& min(T& a, T& b){
if (b < a) return b;else return a;
}
template <typename T>requires(StrictTotallyOrdered(T))
const T& min(const T& a, const T& b){
if (b < a) return b;else return a;
}
Stepanov, McJones Elements of Programming March 14, 2008 77 / 880
More on selecting
A function such as min can be used on the left side of anassignment when all its parameters are writable
Increase the smaller of two variables by 100: min(a, b) += 100;
Note the return type of the two versions of the procedure mustagree with the selecting parameters with respect to constnessThe reasons why C++ requires both versions lie outside the scopeof this book(In this book we often show only the non-const version)
Stepanov, McJones Elements of Programming March 14, 2008 78 / 880
Concept RegularFunction
DefinitionRegularFunction(F)⇒
Function(F)
Regular(F)For all f,g ∈ F such that f = g and for all 0 6 i < n:
Regular(ArgumentType(F, i))For all xi,yi ∈ ArgumentType(F, i) such that xi = yi,
f(x0, . . . , xn−1) = g(y0, . . . ,yn−1)
where n = Arity(F)
By extension, we call a generalized function regular if all of itsinstantiations are regular
Stepanov, McJones Elements of Programming March 14, 2008 79 / 880
Reasons for non-regular functions
In mathematics, all functions are regular as a consequence ofLeibniz’s LawIn programming, there are situations when it’s not so
When a function returns the address of a memory locationThe “address of” operator (& in C++)
When the function returns a value determined by the state of thereal world
Input from a device or another processWhen a stateful function returns a value depending on its own state
A pseudo-random number generatorWhen a function returns a nonessential attribute of an object
Amount of preallocated memory for a data structure
Stepanov, McJones Elements of Programming March 14, 2008 80 / 880
Program optimization
Regular types and regular functions allow us to many standardtransformations, such as:
Common subexpression eliminationLoop hoistingCopy propagation
ExampleIf f and g are regular, we can optimize f(g(x),g(x)) by evaluating g(x)only once
Stepanov, McJones Elements of Programming March 14, 2008 81 / 880
Area of an object
The area of an object is the total number of bytes it occupies11
A type is of constant area if every object of that type has the sameareaA type is of fixed area if every object of that type has an an area thatdoes not change during its lifetimeA type is of dynamic area if the area of an object of the type canvary during the lifetime of the objectFor every type, there is a (potentially implicit) function, area, thatreturns the area of an object of that type
11In a binary computer we should in principle measure areas in bits, but unless weused packed representations we deal in bytes
Stepanov, McJones Elements of Programming March 14, 2008 82 / 880
Storage efficiency
Definition
The byte storage efficiency of a fixed-area object x is dlog256s(x)earea(x) , where
s(x) is the number of distinct states of x
ExampleThe byte storage efficiency of struct{bool} is 1The byte storage efficiency of struct{bool,bool} is 1/2The byte storage efficiency of struct{bool,bool,bool} is 1/3
Stepanov, McJones Elements of Programming March 14, 2008 83 / 880
Cost of a procedure call
The cost of a procedure call is the time required for its executionThe worst-case cost of a procedure call for arguments of given areasis the maximum cost across all procedure calls with arguments ofthese areasThe average cost of a procedure call for arguments of given areas isthe average cost across all procedure calls with arguments of theseareasA procedure has uniform cost if the cost of a procedure calldepends only on the areas of its argumentsA procedure has fixed cost if the cost of a procedure call does notdepend on its argumentsThere is a (possibly implicit) function, cost, that returns the costwhen applied to a procedure and set of arguments lying withinthe definition space of that procedure
Stepanov, McJones Elements of Programming March 14, 2008 84 / 880
Contents I
1 Introduction
2 Foundations
3 Transformations and their orbitsTransformationsOrbitsApplicationsExamplesConclusionsProject
4 Algorithms on algebraic structures
Stepanov, McJones Elements of Programming March 14, 2008 85 / 880
Contents II
5 Orderings
6 Combining concepts
7 Refining concepts of iterators
8 Permutations and rearrangements
9 Rotations
10 Algorithms on increasing ranges
11 Coordinate structures
Stepanov, McJones Elements of Programming March 14, 2008 86 / 880
Contents III
12 Composite objects
13 Iterative algorithms for divide-and-conquer
14 Mathematical notation
15 C++ machinery
16 Acknowledgments
17 Index
Stepanov, McJones Elements of Programming March 14, 2008 87 / 880
Concept Operation
DefinitionOperation(Op)⇒
RegularFunction(Op)
HomogeneousFunction(Op)
Codomain(Op) = Domain(Op)
ExampleUnary abs : double→ double
Binary + : double× double→ double
Ternary multiply_add : double× double× double→ double
Stepanov, McJones Elements of Programming March 14, 2008 88 / 880
Concept Transformation
DefinitionTransformation(F)⇒
Operation(F)
Arity(F) = 1The predicate is_defined : F×Domain(F)→ bool is defined
ExampleNumbers Square root
Geometry Plane rotations; space translationsCombinatorics Reversal of a sequence
Stepanov, McJones Elements of Programming March 14, 2008 89 / 880
Composition and reachability
Transformations are self-composablef(x), f(f(x)), . . .
Self-composability allows us to define iterative algorithms
Definitionf0(x) = x
fn+1(x) = f(fn(x))
Definitiony is reachable from x under a transformation f if y = x or there is a zreachable from x and f(z) = y
Stepanov, McJones Elements of Programming March 14, 2008 90 / 880
Cyclic and terminal elements
Definitionx is cyclic under f if f(x) is defined and x is reachable from f(x)
Definitionx is terminal under f if and only if f is not defined at x
Stepanov, McJones Elements of Programming March 14, 2008 91 / 880
Orbits
DefinitionsAn orbit of x under a transformation f is the set of all elementsreachable from x under f
LemmaAn orbit does not contain both a cyclic and a terminal element
LemmaAn orbit contains at most one terminal element
Stepanov, McJones Elements of Programming March 14, 2008 92 / 880
Classification of orbits
DefinitionsAn orbit of x under f is:
infinite if it has no cyclic or terminal elementsterminating if it has a terminal element
circular if x is cyclicρ-shaped if x is not cyclic and its orbit contains a cyclic element
An orbit of x is finite if it is not infinite
Stepanov, McJones Elements of Programming March 14, 2008 93 / 880
Finite orbits
terminating
circular
ρ-shaped
Stepanov, McJones Elements of Programming March 14, 2008 94 / 880
Structure of orbits
DefinitionThe orbit cycle is the set of cyclic elements in the orbit
DefinitionThe orbit handle is the complement of the orbit cycle with respect to theorbit
DefinitionThe connection point is the first cyclic element
Stepanov, McJones Elements of Programming March 14, 2008 95 / 880
Sizes
DefinitionsThe orbit size o of an orbit is the number of distinct elements in itThe handle size h of an orbit is the number of elements in the orbithandleThe cycle size c of an orbit is the number of elements in the orbitcycle
Lemma
o = h+ c
Stepanov, McJones Elements of Programming March 14, 2008 96 / 880
Finite orbit assumption
Finiteness of a transformation is equivalent to the halting problemThus no algorithm can determine whether an orbit is finite orinfinite for an arbitrary transformationFor many transformations, finiteness is easily provableThere is an implicit precondition of orbit finiteness for all thealgorithms in this chapter
Stepanov, McJones Elements of Programming March 14, 2008 97 / 880
Algorithmic intuition for cycle detection
If two cars, a fast car and a slow car, start along a path, the fast carwill catch up with the slow car if and only if there is a cycle
If there is no cycle, the fast car will reach the end of the path beforethe slow carIf there is a cycle, by the time the slow car enters the cycle, the fastcar will already be there, and will catch up eventually
Stepanov, McJones Elements of Programming March 14, 2008 98 / 880
Collision point
DefinitionThe collision point of a transformation f and a starting point x is theunique y such that
y = fn(x) = f2n+1(x)
and n > 0 is the smallest integer satisfying this condition
Stepanov, McJones Elements of Programming March 14, 2008 99 / 880
collision_point
template <typename F>requires(Transformation(F))
Domain(F) collision_point(const Domain(F)& x, F f){
Domain(F) fast = x;Domain(F) slow = x; // n← 0 (completed iterations)while (true) { // slow = fn(x) ∧ fast = f2n(x)
if (!is_defined(f, fast)) break;
fast = f(fast); // slow = fn(x) ∧ fast = f2n+1(x)A: if (fast == slow) break;
if (!is_defined(f, fast)) break;
fast = f(fast); // slow = fn(x) ∧ fast = f2n+2(x)
slow = f(slow); // slow = fn+1(x) ∧ fast = f2n+2(x)// n← n+ 1
}return fast;// Postcondition: fast is either the terminal point or the collision point of f and x
}
Stepanov, McJones Elements of Programming March 14, 2008 100 / 880
Proof of termination of collision_point
The movement of fast is guarded by a call of is_definedThe movement of slow is unguarded, because by the regularity off, slowwill be traversing the same orbit as fastIf there is no cycle, is_definedwill eventually return false becauseof finitenessIf there is a cycle, slow will eventually reach the connection point(the first element in the cycle):
Consider the distance d from fast to slow at the point labeled Aonce slow enters the cycle
0 6 d < c
If d = 0 the procedure terminatesOtherwise the distance decreases by one on each iteration
By induction, the procedure always terminatesWhen it terminates, slow has moved a total of h+ d steps
Stepanov, McJones Elements of Programming March 14, 2008 101 / 880
Position of collision_pointwhen a cycle exists
The annotations show that when there is a cycle and n > 0 is thenumber of completed iterations, then fn(x) = f2n+1(x)
n is the smallest such integer since we checked the condition forevery i < nWe express n = h+ dwhere d is the distance slowmoves afterreaching the connection pointTherefore h+ d+ qc = 2h+ 2d+ 1 for some q > 0 that counts thenumber of cycles completed by fast when slow enters the cycleSimplifying gives qc = h+ d+ 1Represent h in terms of its quotient and remainder when dividedby c: h = mc+ r for 0 6 r < c
Substitution gives qc = mc+ r+ d+ 1 or d = (q−m)c− r− 10 6 d < c implies q−m = 1, so d = c− r− 1The distance from the collision point to the connection point ise = c− d = r+ 1Stepanov, McJones Elements of Programming March 14, 2008 102 / 880
Distinguishing circular from ρ-shaped case
As a corollary of e = r+ 1, it follows that in case of a circular orbith = 0, which implies r = 0 and so e = 1So the distance from the collision point to the beginning of theorbit is equal to 1Circularity therefore can be checked with the following simpleprocedure (we leave implementation of similar checks forterminating and ρ-shaped to the reader)
Stepanov, McJones Elements of Programming March 14, 2008 103 / 880
is_circular
template <typename F>requires(Transformation(F))
bool is_circular(const Domain(F)& x, F f){
Domain(F) y = collision_point(x, f);return is_defined(f, y) && x == f(y);
}
Stepanov, McJones Elements of Programming March 14, 2008 104 / 880
Finding connection point
Let xw = xh+c−r−1+1 = xh+c−r be the element one past thecollision pointThe element h steps beyond xw is xw+h = xh+c−r+h
Substituting h = mc+ r gives xh+c−r+mc+r = xh+(m+1)c = xhsince it ism+ 1 times around the cycle from xh
This suggests an algorithm for determining xh, the connectionpoint
Stepanov, McJones Elements of Programming March 14, 2008 105 / 880
connection_point
template <typename F>requires(Transformation(F))
Domain(F) convergent_point(Domain(F) x, Domain(F) y, F f){
// Precondition: the orbits of x and y convergewhile (x != y) {
x = f(x);y = f(y);
}return x;
}
template <typename F>requires(Transformation(F))
Domain(F) connection_point(const Domain(F)& x, F f){
Domain(F) y = collision_point(x, f);if (!is_defined(f, y)) return y;return convergent_point(x, f(y), f);
}
Stepanov, McJones Elements of Programming March 14, 2008 106 / 880
Complexity
The complexity analysis of these algorithms trivially follows fromthe termination proofs and is left as an exercise for the reader
Stepanov, McJones Elements of Programming March 14, 2008 107 / 880
Applications
Determining whether a linked structure contains a cycleSee for example the Common Lisp function list-length
Determining the length and the period of a random numbergenerator – see for example:
Donald Knuth.Exercise 3.1.6.The Art of Computer Programming, Volume 2, 3rd edition, 1998,page 7.Credits two-pointer orbit detection to R.W. Floyd.
Our code works for both cases
Stepanov, McJones Elements of Programming March 14, 2008 108 / 880
Problem: representing sizes
What type to use for o, h, and c?We need to define a type function to obtain an integer type bigenough to encode the orbit size for a given type T
Stepanov, McJones Elements of Programming March 14, 2008 109 / 880
Definition of distance type
DefinitionThe distance type for a type T is an integer type a that allows us toencode the maximum number of transformations from oneelement of T into anotherIf a type occupies k bits, its distance type can be represented withan unsigned integer type occupying k bitsThis avoids an infinite tower of types(It is difficult to have a count type that could count the number ofelements in any collection of type T , because that would requirean extra value)The type function DistanceType(T) returns the distance type ofT ; for the implementation in C++, see Appendix 2
aWe will discuss integer types briefly in the next chapter and in depth in Chapter 4
Stepanov, McJones Elements of Programming March 14, 2008 110 / 880
distance
template <typename F>requires(Transformation(F))
DistanceType(Domain(F)) distance(const Domain(F)& x, const Domain(F)& y, F f){
// Precondition: y is reachable from x under f
typedef DistanceType(Domain(F)) D;Domain(F) z = x;D n = D(0);while (z != y) {
z = f(z);n = n + D(1);
}return n;
}
Stepanov, McJones Elements of Programming March 14, 2008 111 / 880
Representing sizes
Note that o, h, and cmay not fit in the distance type
But each of these does fit:
o− 1h− 1c− 1
For non-terminating cases, h fits also
Stepanov, McJones Elements of Programming March 14, 2008 112 / 880
orbit_structure
template <typename F>requires(Transformation(F))
triple<DistanceType(Domain(F)), DistanceType(Domain(F)), Domain(F)>orbit_structure(const Domain(F)& x, F f){
typedef DistanceType(Domain(F)) D;Domain(F) y = connection_point(x, f);D m = distance(x, y, f);D n(0);if (is_defined(f, y))
n = distance(f(y), y, f);// Terminating: m = h− 1 ∧n = 0// Otherwise: m = h∧n = c− 1return triple<D, D, Domain(F)>(m, n, y);
}
Stepanov, McJones Elements of Programming March 14, 2008 113 / 880
Postcondition of orbit_structure
Case first second third
terminating h− 1 0 terminalcircular 0 c− 1 x
ρ-shaped h c− 1 connection
Stepanov, McJones Elements of Programming March 14, 2008 114 / 880
Testing random number generators
ExerciseGiven the C++ function object
struct random_function{
int operator()(int x){
srand(x);return rand();
}};
use orbit_structure on enough different initial elements to decidewhether the cycle length generated by std::rand on your platform issatisfactory
Stepanov, McJones Elements of Programming March 14, 2008 115 / 880
Conclusions
Practical algorithms can be described using basic mathematicaltheoriesNomenclature helps
e.g., orbit kinds and sizes
Equational reasoning on types and functions is essentialAbstract code facilitates reasoning and complexity analysis
Stepanov, McJones Elements of Programming March 14, 2008 116 / 880
Project
There are other algorithms for orbit analysis:
R.T. Sedgwick, T.G. Szymanski and A.C. Yao.The complexity of finding cycles in periodic functions.Proc. 11th SIGACT Meeting, 1979, pages 376-390.
Richard P. Brent.An improved Monte Carlo factorization algorithm.BIT, Volume 20, 1980, pages 176-184.
Leon S. Levy.An improved list-searching algorithm.Information Processing Letters, Volume 15, Issue 1, August 1982,pages 43-45.
Write generic versions of these algorithms and compare theirefficiencies
Stepanov, McJones Elements of Programming March 14, 2008 117 / 880
Contents I
1 Introduction
2 Foundations
3 Transformations and their orbits
4 Algorithms on algebraic structuresAlgebra and abstractionPower on semigroups, monoids, and groupsSlow power algorithmStepwise derivation of fast power algorithmConcept for the exponentComplexityApplications
Stepanov, McJones Elements of Programming March 14, 2008 118 / 880
Contents II
OverloadingConclusionsReferenceProjects
5 Orderings
6 Combining concepts
7 Refining concepts of iterators
8 Permutations and rearrangements
9 Rotations
Stepanov, McJones Elements of Programming March 14, 2008 119 / 880
Contents III
10 Algorithms on increasing ranges
11 Coordinate structures
12 Composite objects
13 Iterative algorithms for divide-and-conquer
14 Mathematical notation
15 C++ machinery
16 Acknowledgments
Stepanov, McJones Elements of Programming March 14, 2008 120 / 880
Contents IV
17 Index
Stepanov, McJones Elements of Programming March 14, 2008 121 / 880
Elementary algebra
Since its inception by the Arabs in the ninth century, algebra hasbeen viewed as the theory of operations on numbers irrespectiveof their particular kinds (such as integers or rationals) or valuesThe common algebraic operations are addition, subtraction,multiplication, and divisionThese operations possess fundamental properties such asassociativity of addition or distributivity of multiplication overaddition
Stepanov, McJones Elements of Programming March 14, 2008 122 / 880
Higher algebra
Through the eighteenth and nineteenth century algebra wasextended to include aggregates of numbers (such as matrices andpolynomials) together with the common algebraic operations onthemThe operations on aggregates turned out to satisfy the sameproperties as the operations on numbers
Stepanov, McJones Elements of Programming March 14, 2008 123 / 880
Precursors of abstract algebra
By the late nineteenth century mathematicians (Jordan, Klein, Lie,etc.) observed that a cluster of properties called a group (namelyan associative, invertible binary operation) appears in severaldifferent domains (such as theory of equations and geometry)At a slightly later time, mathematicians (Dedekind, Weber andHilbert) observed that a cluster of properties called a ringconnecting addition and multiplication also appears in multiplecontextsBy the early twentieth century, people learned to recognizegroups, rings, vector spaces, and other similar clusters ofproperties
Stepanov, McJones Elements of Programming March 14, 2008 124 / 880
Abstract algebra
In the 1920s a fundamentally new point of view on algebra wasdeveloped by several German mathematicians (Noether, Artin,etc.)The new approach was to take a well-understood cluster ofproperties (such as a group) as axioms, and explore theirconsequences independently of any underlying settingAs a result, the same algebraic techniques can be used in anysetting where these axioms are satisfied, leading to effectiveapplications in domains ranging from quantum mechanics tocrystallographyComputer science has found specialized use of these techniques inareas such as automata theory and coding theory
Stepanov, McJones Elements of Programming March 14, 2008 125 / 880
Abstract mathematics
As with abstract algebra, from 1930 until about 1970 a largeportion of mathematics was reinterpreted putting classical resultsin their most abstract settingsThis process led to
an increase in soundnessreuse of techniques across different domains
Many of the known theories from abstract mathematics(especially algebra) provide a natural setting for practicalalgorithms, with a corresponding increase in soundness and reuse
Stepanov, McJones Elements of Programming March 14, 2008 126 / 880
Abstraction
By the time of Aristotle—if not even earlier—it had been recognizedthat every formal discipline rested on what might be called“the principle of voluntary denial of complete knowledge”:abstraction or generalization literally indicates a systematicdiscarding of certain aspects of the studied objects. 12
12Jean Dieudonné. Linear Algebra and Geometry. Herman, Paris, 1969, page 17Stepanov, McJones Elements of Programming March 14, 2008 127 / 880
Grounding abstraction
Concentrating only on abstractions could lead to forgetting theunderlying realities and the problems to be solvedAbstraction in programming should always start with the bestexisting and practically useful algorithms and data structures
Stepanov, McJones Elements of Programming March 14, 2008 128 / 880
Concept BinaryOperation
DefinitionBinaryOperation(Op)⇒
Operation(Op)
Arity(Op) = 2
Stepanov, McJones Elements of Programming March 14, 2008 129 / 880
Concept SemigroupOperation
DefinitionSemigroupOperation(Op)⇒
BinaryOperation(Op)
For all op ∈ Op and for all a,b, c ∈ Domain(Op):
op(a,op(b, c)) = op(op(a,b), c)
The property is associativityA common convention is to represent a semigroup operation asthe infix operator ◦; for example: a ◦ (b ◦ c) = (a ◦ b) ◦ c
Stepanov, McJones Elements of Programming March 14, 2008 130 / 880
Powers
a2 = a ◦ aa3 = a ◦ a ◦ a
a1 = a
an = a ◦ a ◦ · · · ◦ a︸ ︷︷ ︸n
an ◦ am = an+m
Stepanov, McJones Elements of Programming March 14, 2008 131 / 880
Properties of powers
Lemmaan ◦ am = am ◦ an = an+m (powers of the same element commute)
Lemma(an)m = anm
Stepanov, McJones Elements of Programming March 14, 2008 132 / 880
Frobenius’s theorem
DefinitionAn element x has finite order under a semigroup operation if there existintegers 0 < n < m such that xn = xm
DefinitionAn element x is an idempotent element under a semigroup operation ifx = x2
TheoremAn element of finite order has an idempotent power a
aGeorg Ferdinand Frobenius. Über endliche Gruppen. Berlin 1895. In:Sitzungesberichte der Königlich Preussischen Akademie der Wissenschaften zu Berlin.Phys.-math. Classe 1895, pages 163-194
Stepanov, McJones Elements of Programming March 14, 2008 133 / 880
Proof of Frobenius’s theorem
template <typename Op>requires(SemigroupOperation(Op))
struct multiply_transformation{
Domain(Op) x;Op op;multiply_transformation(Domain(Op) x, Op op) : x(x), op(op) {}Domain(Op) operator()(const Domain(Op)& y){
return op(y, x);}
};
template <typename Op>requires(SemigroupOperation(Op))
Domain(Op) idempotent_power(Domain(Op) x, Op op){
return collision_point(x, multiply_transformation<Op>(x, op));}
Stepanov, McJones Elements of Programming March 14, 2008 134 / 880
Explanation of proof of Frobenius’s theorem
Assume x is an element of finite order under a semigroupoperation opLet
g(z) = multiply_transformation(op, x) = op(z, x) = z ◦ x
Let y = idempotent_power(x,op)Since x is an element of finite order, its orbit under g has a cycleBy the postcondition of collision_point, y = gn(x) for thesmallest n > 0 such that gn(x) = g2n+1(x), whereg1(x) = g(x) = x ◦ x = x2
Thus gn(x) = xn+1 and g2n+1(x) = x2n+2 = x2(n+1) = (xn+1)2
Therefore y = gn(x) = xn+1 is the idempotent power of x
Stepanov, McJones Elements of Programming March 14, 2008 135 / 880
Concept MonoidOperation
DefinitionMonoidOperation(Op)⇒
SemigroupOperation(Op)
identity_element : Op→ Domain(Op) is definedFor all op ∈ Op, for all a ∈ Domain(Op), andfor e = identity_element(op):
op(a, e) = a = op(e,a)
Stepanov, McJones Elements of Programming March 14, 2008 136 / 880
Concept GroupOperation
DefinitionGroupOperation(Op)⇒
MonoidOperation(Op)
inverse_operation : Op→ (Domain(Op)→ Domain(Op)) isdefinedFor all op ∈ Op, for all a ∈ Domain(Op), and forg = inverse_operation(op):
op(a,g(a)) = identity_element(op) = op(g(a),a)
A common convention is to represent the inverse of a as a−1
Stepanov, McJones Elements of Programming March 14, 2008 137 / 880
Extending powers to non-positive exponents
a0 = e
an ◦ a0 = an+0 = an
a−n = (an)−1
an ◦ a−n = an−n = a0
Lemma(a−1)n = a−n
Stepanov, McJones Elements of Programming March 14, 2008 138 / 880
Concept Integer
We will study various concepts describing integers in Chapter 4In the meantime we will rely on the intuitive understanding ofwhat integers areModels include:
all C++ integral types, signed and unsignedbignums (type allowing arbitrary-precision integers)
Operations are + - * / %with their standard semanticsAll integer types contain constants 0 and 1
Initially we use the constant 2, but later in the chapter we introducespecial-case functions that allow us to avoid its use
In C++ the constants 0, 1, and 2 of an integer type I are representedas I(0), I(1), and I(2)
Stepanov, McJones Elements of Programming March 14, 2008 139 / 880
slow_power_positive
template <typename I, typename Op>requires(Integer(I) && SemigroupOperation(Op))
Domain(Op) slow_power_positive(Domain(Op) a, I n, Op op){
// Precondition: n > I(0)if (n == I(1))
return a;else
return op(a, slow_power_positive(a, n - I(1), op));}
Stepanov, McJones Elements of Programming March 14, 2008 140 / 880
slow_power_nonnegative
template <typename I, typename Op>requires(Integer(I) && MonoidOperation(Op))
Domain(Op) slow_power_nonnegative(Domain(Op) a, I n, Op op){
// Precondition: n > I(0)if (n == I(0))
return identity_element(op);else
return slow_power_positive(a, n, op);}
Stepanov, McJones Elements of Programming March 14, 2008 141 / 880
Observations
When computing an, we perform n− 1 operationsWe never perform the operation with the identity element
Except when a is the identity elementIt is not worth adding an extra check for the case of a being theidentity element
This would slow down the average caseThe caller can do this if it is important
Stepanov, McJones Elements of Programming March 14, 2008 142 / 880
slow_power
template <typename I, typename Op>requires(Integer(I) && GroupOperation(Op))
Domain(Op) slow_power(Domain(Op) a, I n, Op op){
if (n < I(0)){
a = inverse_operation(op)(a);n = -n;
}return slow_power_nonnegative(a, n, op);
}
Stepanov, McJones Elements of Programming March 14, 2008 143 / 880
Ancient Egyptian discovery
Egyptian scribe Ahmes described (circa 1650 BC) a differentpower algorithm that gives an efficient way to multiply 13
It is sometimes called the Russian peasant algorithm 14
13Gay Robins and Charles Shute. The Rhind Mathematical Papyrus, British MuseumPress, 1987
14The oldest reference to Russian origin we have found appears in: Sir ThomasHeath. A History of Greek Mathematics, Volume I, Clarendon Press, 1921, page 53.Reprint: Dover, 1981. Heath writes: “I have been told that there is a method in useto-day (some say in Russia, but I have not been able to verify this) . . . ” We have notbeen able to verify this either.
Stepanov, McJones Elements of Programming March 14, 2008 144 / 880
Exploiting associativity
For example: aaaaaaa = ((aa)a)((aa)a)a
In general: an = (a2)n/2an mod 2
This translates into the following procedure
Stepanov, McJones Elements of Programming March 14, 2008 145 / 880
Initial (recursive) version
template <typename I, typename Op>requires(Integer(I) && MonoidOperation(Op))
Domain(Op) power_nonnegative_0(Domain(Op) a, I n, Op op){
// Precondition: n > I(0)if (n == I(0))
return identity_element(op);else if (n == I(1))
return a;else
return op(power_nonnegative_0(op(a, a), n / I(2), op),power_nonnegative_0(a, n % I(2), op));
}
Stepanov, McJones Elements of Programming March 14, 2008 146 / 880
Transforming a program
We will now demonstrate a sequence of transformations thatimprove the performance of this algorithm without changing itsasymptotic complexityFor the rest of the book we will typically only show final oralmost-final versions
Stepanov, McJones Elements of Programming March 14, 2008 147 / 880
Eliminating unnecessary multiplication by identity
template <typename I, typename Op>requires(Integer(I) && SemigroupOperation(Op))
Domain(Op) power_positive_0(Domain(Op) a, I n, Op op){
// Precondition: n > I(0)if (n == I(1))
return a;else if (n % I(2) == I(0))
return power_positive_0(op(a, a), n / I(2), op);else
return op(power_positive_0(op(a, a), n / I(2), op), a);}
Stepanov, McJones Elements of Programming March 14, 2008 148 / 880
Eliminating common subexpression
template <typename I, typename Op>requires(Integer(I) && SemigroupOperation(Op))
Domain(Op) power_positive_1(Domain(Op) a, I n, Op op){
// Precondition: n > I(0)if (n == I(1))
return a;Domain(Op) result = power_positive_1(op(a, a), n / I(2), op);if (n % I(2) != I(0))
result = op(result, a);return result;
}
Stepanov, McJones Elements of Programming March 14, 2008 149 / 880
Introducing accumulation variable
template <typename I, typename Op>requires(Integer(I) && SemigroupOperation(Op))
Domain(Op) power_accumulate_nonnegative_0(Domain(Op) r, Domain(Op) a, I n, Op op){
// Precondition: n > I(0)if (n == I(0))
return r;if (n % I(2) != I(0)) r = op(r, a);return power_accumulate_nonnegative_0(r, op(a, a), n / I(2), op);
}
Recursion invariant: ran
Stepanov, McJones Elements of Programming March 14, 2008 150 / 880
Almost iterative
template <typename I, typename Op>requires(Integer(I) && SemigroupOperation(Op))
Domain(Op) power_accumulate_nonnegative_1(Domain(Op) r, Domain(Op) a, I n, Op op){
// Precondition: n > I(0)if (n == I(0))
return r;if (n % I(2) != I(0)) r = op(r, a);a = op(a, a);n = n / I(2);return power_accumulate_nonnegative_1(r, a, n, op);
}
Recursion invariant: ran
Stepanov, McJones Elements of Programming March 14, 2008 151 / 880
Iterative
template <typename I, typename Op>requires(Integer(I) && SemigroupOperation(Op))
Domain(Op) power_accumulate_nonnegative_2(Domain(Op) r, Domain(Op) a, I n, Op op){
// Precondition: n > I(0)while (true) {
if (n == I(0))return r;
if (n % I(2) != I(0)) r = op(r, a);a = op(a, a); // wasted on last iterationn = n / I(2);
}}
Loop invariant: ran
Stepanov, McJones Elements of Programming March 14, 2008 152 / 880
Rotating the loop: reordering
template <typename I, typename Op>requires(Integer(I) && SemigroupOperation(Op))
Domain(Op) power_accumulate_nonnegative_3(Domain(Op) r, Domain(Op) a, I n, Op op){
// Precondition: n > I(0)while (true) {
if (n == I(0))return r;
if (n % I(2) != I(0)) r = op(r, a);n = n / I(2); // reordera = op(a, a); // independent statements
}}
Loop invariant: ran
Stepanov, McJones Elements of Programming March 14, 2008 153 / 880
Rotating the loop: duplicating exit
template <typename I, typename Op>requires(Integer(I) && SemigroupOperation(Op))
Domain(Op) power_accumulate_nonnegative_4(Domain(Op) r, Domain(Op) a, I n, Op op){
// Precondition: n > I(0)while (true) {
if (n == I(0))return r;
if (n % I(2) != I(0)) r = op(r, a);n = n / I(2);if (n == I(0))
return r; // early exita = op(a, a);
}}
Loop invariant: ran
Stepanov, McJones Elements of Programming March 14, 2008 154 / 880
Rotating the loop: hoisting exit
template <typename I, typename Op>requires(Integer(I) && SemigroupOperation(Op))
Domain(Op) power_accumulate_nonnegative_5(Domain(Op) r, Domain(Op) a, I n, Op op){
// Precondition: n > I(0)if (n == I(0))
return r; // moved first test out of loopwhile (true) {
if (n % I(2) != I(0)) r = op(r, a);n = n / I(2);if (n == I(0))
return r;a = op(a, a);
}}
Loop invariant: ran
Stepanov, McJones Elements of Programming March 14, 2008 155 / 880
Dependency between conditions
nwill reach 0 only from 11 is oddThe second condition will only be true when the first condition istrue:
if (n % I(2) != I(0)) r = op(r, a);n = n / I(2);if (n == I(0))
return r;
Stepanov, McJones Elements of Programming March 14, 2008 156 / 880
Utilizing dependency
template <typename I, typename Op>requires(Integer(I) && SemigroupOperation(Op))
Domain(Op) power_accumulate_nonnegative_6(Domain(Op) r, Domain(Op) a, I n, Op op){
// Precondition: n > I(0)if (n == I(0))
return r;while (true) {
bool odd = n % I(2) != I(0);n = n / I(2);if (odd) {
r = op(r, a);if (n == I(0))
return r;}a = op(a, a);
}}
Stepanov, McJones Elements of Programming March 14, 2008 157 / 880
Specializing for positive n
template <typename I, typename Op>requires(Integer(I) && SemigroupOperation(Op))
Domain(Op) power_accumulate_positive_0(Domain(Op) r, Domain(Op) a, I n, Op op){
// Precondition: n > I(0)while (true) {
bool odd = n % I(2) != I(0);n = n / I(2);if (odd) {
r = op(r, a);if (n == I(0))
return r;}a = op(a, a);
}}
Stepanov, McJones Elements of Programming March 14, 2008 158 / 880
(Nearly final) power_accumulate_nonnegative
template <typename I, typename Op>requires(Integer(I) && SemigroupOperation(Op))
Domain(Op) power_accumulate_nonnegative_7(Domain(Op) r, Domain(Op) a, I n, Op op){
// Precondition: n > I(0)if (n == I(0))
return r;else
return power_accumulate_positive_0(r, a, n, op);}
Stepanov, McJones Elements of Programming March 14, 2008 159 / 880
Eliminating accumulation variable
template <typename I, typename Op>requires(Integer(I) && SemigroupOperation(Op))
Domain(Op) power_positive_2(Domain(Op) a, I n, Op op){
// Precondition: n > I(0)n = n - I(1);return power_accumulate_nonnegative_7(a, a, n, op);
}
Stepanov, McJones Elements of Programming March 14, 2008 160 / 880
Observation
When n is 16, we do 7 operations where only 4 are neededWhen n is odd, this code is fine
Stepanov, McJones Elements of Programming March 14, 2008 161 / 880
Factoring out powers of 2
template <typename I, typename Op>requires(Integer(I) && SemigroupOperation(Op))
Domain(Op) power_positive_3(Domain(Op) a, I n, Op op){
// Precondition: n > I(0)while (n % I(2) == I(0)) {
a = op(a, a);n = n / I(2);
}n = n - I(1);if (n == I(0))
return a;else
return power_accumulate_positive_0(a, a, n, op);}
Stepanov, McJones Elements of Programming March 14, 2008 162 / 880
Observation
power_accumulate_positive_0 checks if n is odd on the first passWe know n is even at that pointOne extra operation!
Stepanov, McJones Elements of Programming March 14, 2008 163 / 880
Unrolling one iteration ofpower_accumulate_positive_0
template <typename I, typename Op>requires(Integer(I) && SemigroupOperation(Op))
Domain(Op) power_positive_4(Domain(Op) a, I n, Op op){
// Precondition: n > I(0)while (n % I(2) == I(0)) {
a = op(a, a);n = n / I(2);
}n = n / I(2);if (n == I(0))
return a;else
return power_accumulate_positive_0(a, op(a, a), n, op);}
Stepanov, McJones Elements of Programming March 14, 2008 164 / 880
Operations on exponent
In the final version we used these operations:n - I(1)n = n / T(2);n % T(2) == T(0)n == T(0)
General / and % are very expensiveFor all integral types, if we know n is non-negative, we can useshifts and masks
Stepanov, McJones Elements of Programming March 14, 2008 165 / 880
Special cases of procedures
It is frequently useful to identify commonly-occuring expressionsinvolving procedures and constants of a type by definingspecial-case proceduresOften these special cases can be implemented more efficientlythan the general case
For built-in types there may exist machine instructions for thespecial cases (e.g., shifting instead of multiplication by a power oftwo)
For complex user-defined types there are often even moresignificant opportunities for optimizing special cases
Division of two arbitrary polynomials is more difficult thandivision of a polynomial by xDivision of two Gaussian integers (numbers of the form a+ biwhere a and b are integers and i =
√−1) is more difficult than
division of a Gaussian integer by 1 + i
Stepanov, McJones Elements of Programming March 14, 2008 166 / 880
Special cases of Integer procedures
Integer(I) ∧ i ∈ I⇒
successor(i) ≡ i+ 1predecessor(i) ≡ i− 1
half_nonnegative(i) ≡ bi/2cbinary_scale_down_nonnegative(i,k) ≡ i/2k
binary_scale_up_nonnegative(i,k) ≡ 2kiis_positive(i) ≡ i > 0is_negative(i) ≡ i < 0
is_zero(i) ≡ i = 0is_even(i) ≡ (i mod 2) = 0is_odd(i) ≡ (i mod 2) , 0
See default C++ implementations in Appendix 2Stepanov, McJones Elements of Programming March 14, 2008 167 / 880
power_accumulate_positive
template <typename I, typename Op>requires(Integer(I) && SemigroupOperation(Op))
Domain(Op) power_accumulate_positive(Domain(Op) r, Domain(Op) a, I n, Op op){
// Precondition: is_positive(n)while (true) {
bool odd = is_odd(n);halve_nonnegative(n);if (odd) {
r = op(r, a);if (is_zero(n))
return r;}a = op(a, a);
}}
Stepanov, McJones Elements of Programming March 14, 2008 168 / 880
power_accumulate_nonnegative
template <typename I, typename Op>requires(Integer(I) && MonoidOperation(Op))
Domain(Op) power_accumulate_nonnegative(Domain(Op) r, Domain(Op) a, I n, Op op){
// Precondition: ¬is_negative(n)if (is_zero(n))
return r;else
return power_accumulate_positive(r, a, n, op);}
Stepanov, McJones Elements of Programming March 14, 2008 169 / 880
power_positive
template <typename I, typename Op>requires(Integer(I) && SemigroupOperation(Op))
Domain(Op) power_positive(Domain(Op) a, I n, Op op){
// Precondition: is_positive(n)while (is_even(n)) {
a = op(a, a);halve_nonnegative(n);
}halve_nonnegative(n);if (is_zero(n))
return a;else
return power_accumulate_positive(a, op(a, a), n, op);}
Stepanov, McJones Elements of Programming March 14, 2008 170 / 880
power_nonnegative
template <typename I, typename Op>requires(Integer(I) && MonoidOperation(Op))
Domain(Op) power_nonnegative(Domain(Op) a, I n, Op op){
// Precondition: ¬is_negative(n)if (is_zero(n))
return identity_element(op);else
return power_positive(a, n, op);}
Stepanov, McJones Elements of Programming March 14, 2008 171 / 880
power
template <typename I, typename Op>requires(Integer(I) && GroupOperation(Op))
Domain(Op) power(Domain(Op) a, I n, Op op){
if (is_negative(n)) {a = inverse_operation(op)(a);n = -n;
}return power_nonnegative(a, n, op);
}
Stepanov, McJones Elements of Programming March 14, 2008 172 / 880
The number of operations performed by power
Let u = number of significant bits in n > 0Let v = number of one-bits in nLet p = number of operations performedThen p = (u− 1) + (v− 1) = u+ v− 2 6 2blog2 nc
ExerciseProve that power achieves this complexity
Stepanov, McJones Elements of Programming March 14, 2008 173 / 880
The operation count of power is not optimal
For n = 15, u = 4 and v = 4, so u+ v− 2 = 6But 15 = 3× 5, so a15 = (a3)5
a3 takes 2 operationsa5 takes 3 operations(a3)5 takes 5 total
There are faster ways for 23, 27, 39, 43, . . .See Project 3
Stepanov, McJones Elements of Programming March 14, 2008 174 / 880
Choosing algorithms based on complexity
Complexity is an important part of specificationsFor example, a stack is not a stack if the cost of push grows linearlywith the size of the stack
When only one algorithm for solving a problem is available, themain issue is whether the algorithm is feasible for a particularsituation (set of arguments determining problem size, functionalparameters, etc.)When there are more than one algorithm for solving a problem, weneed to analyze which algorithm to use for a particular situationThe analysis depends on the complexity properties of thealternative algorithms combined with the complexity propertiesof the types to which these algorithms are applied
Stepanov, McJones Elements of Programming March 14, 2008 175 / 880
Specifying complexity
Knuth pioneered the analysis of algorithms based on counting themachine instructions performed by a MIX programSince we write generalized procedures defined in terms of typerequirements and sometimes taking function objects asarguments, we need to count applications of the function objectsand the operations on the types
Stepanov, McJones Elements of Programming March 14, 2008 176 / 880
Complexity of slow_power versus power
The total cost is the sum of the cost of the operations on theexponent, the cost of the semigroup operation, and the cost of thecontrol structureFor all the models of Integer we are familiar with, the logarithmicnumber of exponent operations performed by powerwill be fasterthan the linear number of operations performed by slow_power,except for very small nFor most models of the semigroup operation, power will be faster,but there are exceptions
Stepanov, McJones Elements of Programming March 14, 2008 177 / 880
When is power faster than slow_power?
Suppose cost(op, xi, xj) = k(x) · (i · j)α
If α = 0, we have a fixed-cost operation and power is alwaysmuch faster than slow_power
Examples: fixed-precision multiplication; matrix multiplicationwith fixed-precision elements
If α 6 1, power is always faster15
Examples: multiplication of univariate polynomials withfixed-precision coefficients; multiplication of bignums
If α > 2, slow_power is always faster16
Examples: multiplication of bivariate polynomials withfixed-precision coefficients; multiplication of univariatepolynomials with bignum coefficients
15R.L. Graham, A.C.-C. Yao, and F.-F. Yao. Addition chains with multiplicativecosts. Discrete Mathematics, Volume 23, Number 2, 1978, pages 115-119.
16D.P. McCarthy. Effect of Improved Multiplication Efficiency on ExponentationAlgorithms Derived from Addition Chains. Mathematics of Computation, Volume 46,Number 174, April 1986, pages 603-608.
Stepanov, McJones Elements of Programming March 14, 2008 178 / 880
Applications
The main application of fast exponentiation is cryptography
Donald Knuth.Secret factors.The Art of Computer Programming, Vol. 2, pages 403-407.RSA.
Donald Knuth.Improved primality tests.The Art of Computer Programming, Vol. 2, pages 394-396.Primality testing.
Manindra Agrawal, Neeraj Kayal, and Nitin Saxena.PRIMES is in P.Annals of Mathematics, 160 (2004), pages 781-793.Primality testing.
Stepanov, McJones Elements of Programming March 14, 2008 179 / 880
Reflections on deriving useful interfaces
power_accumulate_nonnegative is useful whenever one wantsto compute axn
Even power_accumulate_positive is useful in a situation when itis known that n > 0
ReflectionWhile developing a component, it is often possible to discoveradditional useful interfaces
Stepanov, McJones Elements of Programming March 14, 2008 180 / 880
Reflection on code transformations
ReflectionCompilers can perform code transformations when the semanticsof the operations are knownCurrently this is only for built-in typesSomeday we will be able to tell the compiler the semantics of ouroperations
Stepanov, McJones Elements of Programming March 14, 2008 181 / 880
Fibonacci numbers
Definition
f0 = 0f1 = 1
fn+2 = fn+1 + fn
Example0, 1, 1, 2, 3, 5, 8, 13, . . .
Stepanov, McJones Elements of Programming March 14, 2008 182 / 880
Recursive way to calculate fn
template <typename I>requires(Integer(I))
I fibonacci_recursive(I n){
if (n == I(0))return I(0);
else if (n == I(1))return I(1);
elsereturn fibonacci_recursive(n - I(1))
+ fibonacci_recursive(n - I(2));}
Stepanov, McJones Elements of Programming March 14, 2008 183 / 880
Iterative way to calculate fn
template <typename I>requires(Integer(I))I fibonacci_iterative(I n){
if (n == I(0))return I(0);
I fib_i = I(0);I fib_j = I(1);while (n != I(1)) {
I next = fib_i + fib_j;fib_i = fib_j;fib_j = next;n = n - I(1);
}return fib_j;
}
Stepanov, McJones Elements of Programming March 14, 2008 184 / 880
Fibonacci matrices
F1 =
(1 11 0
)Fn =
(fn+1 fnfn fn−1
)F1Fn =
(fn+1 + fn fn + fn−1fn+1 fn
)=
(fn+2 fn+1fn+1 fn
)= Fn+1
Fn = F1F1 . . . F1︸ ︷︷ ︸n
= Fn1
FmFn = Fm1 Fn1 = Fm+n
1 = Fm+n
Stepanov, McJones Elements of Programming March 14, 2008 185 / 880
Fibonacci matrices
Fm =
(fm+1 fmfm fm−1
)Fn =
(fn+1 fnfn fn−1
)FmFn =
(fm+1fn+1 + fmfn fm+1fn + fmfn−1fmfn+1 + fm−1fn fmfn + fm−1fn−1
)
Stepanov, McJones Elements of Programming March 14, 2008 186 / 880
Observations
We can represent the matrix with a pair corresponding to thebottom row
pair::first is fn and pair::second is fn−1
The identity fn+1 = fn−1 + fn allows us to compute the missingterm from the top row of Fn needed to compute the bottom row ofFmFn
Stepanov, McJones Elements of Programming March 14, 2008 187 / 880
fibonacci_multiplies
template <typename I>requires(Integer(I))
struct fibonacci_multiplies{
typedef pair<I, I> D;D operator()(const D& x, const D& y) const{
return D(x.first * (y.first + y.second) + x.second * y.first,x.first * y.first + x.second * y.second);
}};
Stepanov, McJones Elements of Programming March 14, 2008 188 / 880
fibonacci
template <typename I>requires(Integer(I))
I fibonacci(I n){
// Precondition: n > I(0)if (n == I(0))
return I(0);fibonacci_multiplies<I> op;return power_positive(pair<I, I>(I(1), I(0)), n, op).first;
}
Stepanov, McJones Elements of Programming March 14, 2008 189 / 880
Overloading
DefinitionOverloading an operator symbol or function name means using it foroperations or functions on several types
ReflectionProgress in mathematics often involves extending an operator to a newdomain
Example+ on
natural numbersintegersrationalspolynomialsmatricesStepanov, McJones Elements of Programming March 14, 2008 190 / 880
Overloading should preserve semantics
In all of these examples+ is associative+ is commutative+ obeys the cancellation law
a+ c = b+ c⇒ a = b
× distributes over +
Stepanov, McJones Elements of Programming March 14, 2008 191 / 880
Type parameterization results in overloading
When we define template functions such as collision_point andpower, we are overloading the function nameIf the actual type parameters satisfy the requirements thesemantics is preserved since the same algorithm is used
Stepanov, McJones Elements of Programming March 14, 2008 192 / 880
Two ways operations are provided to power
The group operation is passed as a parameterThis allows power to be used with different operations on the sametype, and the operation can have a state such as a modulus for usein encryption
The integer operations are provided through overloading of -, /,%, and ==
Using explicit operators is convenient whenever there are severalrelated operations on the type and no other correct interpretation ofthe operations on the type
Stepanov, McJones Elements of Programming March 14, 2008 193 / 880
Concept CommutativeOperation
DefinitionCommutativeOperation(Op)⇒
BinaryOperation(Op)
For all op ∈ Op and for all a,b ∈ Domain(Op),
op(a,b) = op(b,a)
Stepanov, McJones Elements of Programming March 14, 2008 194 / 880
Concept CancellableOperation
DefinitionCancellableOperation(Op)⇒
BinaryOperation(Op)
For all op ∈ Op and for all a,b, c ∈ Domain(Op),
op(a, c) = op(b, c)⇒ a = b
op(c,a) = op(c,b)⇒ a = b
Stepanov, McJones Elements of Programming March 14, 2008 195 / 880
Concept AdditiveSemigroup
DefinitionAdditiveSemigroup(T)⇒
The binary operation infix + is defined on TSemigroupOperation(+)
CommutativeOperation(+) a
aIt would have been nice to require the cancellation law, but digital logic designersuse + for an operation that is not cancellable
Stepanov, McJones Elements of Programming March 14, 2008 196 / 880
Concept MultiplicativeSemigroup
DefinitionMultiplicativeSemigroup(T)⇒
The binary operation infix × is defined on TSemigroupOperation(×)
Stepanov, McJones Elements of Programming March 14, 2008 197 / 880
Concept AdditiveMonoid
DefinitionAdditiveMonoid(T)⇒
AdditiveSemigroup(T)
MonoidOperation(+)
The constant T(0) is definedidentity_element(+) = T(0)
Stepanov, McJones Elements of Programming March 14, 2008 198 / 880
Concept MultiplicativeMonoid
DefinitionMultiplicativeMonoid(T)⇒
MultiplicativeSemigroup(T)
MonoidOperation(×)
The constant T(1) is definedidentity_element(×) = T(1)
Stepanov, McJones Elements of Programming March 14, 2008 199 / 880
Concept AdditiveGroup
DefinitionAdditiveGroup(T)⇒
AdditiveMonoid(T)
GroupOperation(+)
The unary operation negation (prefix −) is defined on Tinverse_operation(+) = negation
DefinitionFor an additive group, subtraction is defined as a− b = a+ (−b)
Stepanov, McJones Elements of Programming March 14, 2008 200 / 880
Concept MultiplicativeGroup
DefinitionMultiplicativeGroup(T)⇒
MultiplicativeMonoid(T)
GroupOperation(×)
The unary operation reciprocal is defined on Tinverse_operation(×) = reciprocal
DefinitionFor a multiplicative group, division is defined asa/b = a× reciprocal(b)
Stepanov, McJones Elements of Programming March 14, 2008 201 / 880
Power for multiplicative types
template <typename G, typename I>requires(MultiplicativeSemigroup(G) && Integer(I))
G power_positive(G a, I n){
return power_positive(a, n, multiplies<G>());}
template <typename G, typename I>requires(MultiplicativeMonoid(G) && Integer(I))
G power_nonnegative(G a, I n){
return power_nonnegative(a, n, multiplies<G>());}
template <typename G, typename I>requires(MultiplicativeGroup(G) && Integer(I))
G power(G a, I n){
return power(a, n, multiplies<G>());}
See Appendix 2 for the enabling C++ definitions
Stepanov, McJones Elements of Programming March 14, 2008 202 / 880
Conclusions
Algorithms are generic by natureThey can be used with different models satisfying the samerequirements
Algorithms are affiliated with algebraic structures: semigroup,monoid, . . .Stepwise refinement leads from mathematical definitions toefficient codeSpecial-case procedures can make code more efficient and evenmore genericMathematics leads to surprising algorithms: fibonacci
Stepanov, McJones Elements of Programming March 14, 2008 203 / 880
Reference
D. Kapur, D.R. Musser, and A.A. Stepanov.Operators and Algebraic Structures.Proceedings of the 1981 conference on Functional programminglanguages and computer architecture, pages 59-63.One of the first presentations of algorithms on algebraic structures.
Stepanov, McJones Elements of Programming March 14, 2008 204 / 880
Project 1
ProjectOur definition of the Fibonacci sequence starts from zero and goes up;extend the definition and our code to work for negative indices a
aSuggested by Oleg Zabluda
Stepanov, McJones Elements of Programming March 14, 2008 205 / 880
Project 2
Charles M. Fiduccia.An efficient formula for linear recurrences.SIAM Journal of Computing, Volume 14, Number 1, February 1985,page 106-112.Generalizes the fibonacci algorithm to arbitrary linear recurrences.
ProjectCreate a library implementing Fiduccia’s algorithm
Stepanov, McJones Elements of Programming March 14, 2008 206 / 880
Project 3
Donald E. Knuth.Addition chains.The Art of Computer Programming, Volume 2: SeminumericalAlgorithms, 3rd edition, Addison-Wesley, San Francisco, 1998,pages 465-481.Describes a way to do minimal-operation exponentiation usingaddition chains.
ProjectImplement a useful library doing power optimally for exponentsknown at compile time
Stepanov, McJones Elements of Programming March 14, 2008 207 / 880
Project 4
ProjectFloating-point multiplication and addition are not associative, somay give different results when they are used as the operation forslow_power and power; establish whether slow_power or powergives a more accurate result for:
1 Raising a floating-point number to an integral power2 Multiplying a floating-point number by an integral factor
Stepanov, McJones Elements of Programming March 14, 2008 208 / 880
Contents I
1 Introduction
2 Foundations
3 Transformations and their orbits
4 Algorithms on algebraic structures
5 OrderingsMotivationRelation; transitivity; orderingStrict versus reflexiveSymmetric versus asymmetric
Stepanov, McJones Elements of Programming March 14, 2008 209 / 880
Contents II
Equivalence; equalitySymmetric complement; total ordering; weak ordering; partialorderingMultiple orderings; natural total ordering; default ordering;overloadingIntervalsOrder selection algorithmsConclusionsReferenceProject
6 Combining concepts
7 Refining concepts of iterators
Stepanov, McJones Elements of Programming March 14, 2008 210 / 880
Contents III
8 Permutations and rearrangements
9 Rotations
10 Algorithms on increasing ranges
11 Coordinate structures
12 Composite objects
13 Iterative algorithms for divide-and-conquer
14 Mathematical notation
Stepanov, McJones Elements of Programming March 14, 2008 211 / 880
Contents IV
15 C++ machinery
16 Acknowledgments
17 Index
Stepanov, McJones Elements of Programming March 14, 2008 212 / 880
The importance of linear ordering
Equality allows comparing individual elements easilySearching a collection using only equality requires linear timeIntersecting two collections using only equality requires timeproportional to the product of the sizes of the collectionsA linear ordering allows organizing elements so that theircollections can be
matched (intersection, subset, union) in linear timesearched in logarithmic time
Stepanov, McJones Elements of Programming March 14, 2008 213 / 880
Concepts Predicate and Relation
DefinitionPredicate(Op)⇒
RegularFunction(Op)
Codomain(Op) = bool
DefinitionRelation(Op)⇒
Predicate(Op)HomogeneousFunction(Op)
Arity(Op) = 2(Other books use this term for more general non-unary predicates)
Stepanov, McJones Elements of Programming March 14, 2008 214 / 880
Concept Ordering
DefinitionOrdering(Op)⇒
Relation(Op)
For all op ∈ Op and for all a,b, c ∈ Domain(Op)
op(a,b) ∧ op(b, c)⇒ op(a, c)
This property is called transitivity
Stepanov, McJones Elements of Programming March 14, 2008 215 / 880
Examples of ordering
ExamplesEqualityEquality of the first characterReachability in an orbitDivisibility
Stepanov, McJones Elements of Programming March 14, 2008 216 / 880
Strict versus reflexive
DefinitionAn ordering r is strict if ¬r(a,a)
An ordering r is reflexive if r(a,a)
An ordering r is weakly reflexive if r(a,b)⇒ r(a,a) ∧ r(b,b)
ExamplesReflexive: all examples on the previous slideStrict: proper factor
Stepanov, McJones Elements of Programming March 14, 2008 217 / 880
Symmetric versus asymmetric
DefinitionA relation r is symmetric if r(a,b)⇒ r(b,a)
A relation r is asymmetric if r(a,b)⇒ ¬r(b,a)
LemmaA strict ordering is asymmetric
LemmaA symmetric ordering is weakly reflexive
ExamplesSymmetric: siblingAsymmetric: parent
Stepanov, McJones Elements of Programming March 14, 2008 218 / 880
Concept EquivalenceRelation
DefinitionEquivalenceRelation(R)⇒
Ordering(R)
Reflexive(R)
Symmetric(R)
ExamplesEqualityGeometric congruencea ≡ b (mod n)
Stepanov, McJones Elements of Programming March 14, 2008 219 / 880
Equivalence and equality
LemmaIf r is an equivalence relation, a = b⇒ r(a,b)
Stepanov, McJones Elements of Programming March 14, 2008 220 / 880
Key function
DefinitionIf T is a regular type and r ∈ R is an equivalence relation on T , afunction f ∈ F : T → T ′ is a key function for r ifr(x, x ′)⇔ f(x) = f(x ′)
Given the choice of a particular key function f, we say f(x) is acanonical representation of the elements equivalent to x
One way to implement an equivalence relation is to define a keyfunction for it and then apply equality to the results
Stepanov, McJones Elements of Programming March 14, 2008 221 / 880
symmetric_complement
template <typename R>requires(Relation(R))
struct symmetric_complement{
R r;symmetric_complement(R r) : r(r) {}
bool operator()(const Domain(R)& a, const Domain(R)& b){
return !r(a, b) && !r(b, a);}
};
Stepanov, McJones Elements of Programming March 14, 2008 222 / 880
Concept StrictTotalOrdering
DefinitionStrictTotalOrdering(R)⇒
Ordering(R)
For all r ∈ R and for all a,b ∈ Domain(R)
symmetric_complementr(a,b)⇔ a = b
Stepanov, McJones Elements of Programming March 14, 2008 223 / 880
Properties of total ordering
LemmaA total ordering is strict
LemmaA total ordering is asymmetric
LemmaThe trichotomy law holds: r(a,b) ∨ r(b,a) ∨ (a = b)
LemmaExactly one clause of the trichotomy law holds
Stepanov, McJones Elements of Programming March 14, 2008 224 / 880
Extending strict total orderings to a direct product
Strict total orderings r0 ∈ R0, . . . , rn ∈ Rn can be extended to astrict total ordering on the direct productDomain(R0)× . . .×Domain(Rn) via the lexicographic ordering rdefined by:
r((x0, . . . , xn), (y0, . . . ,yn)) ≡(∃k ∈ [0,n])(x0 = y0 ∧ . . . ∧ xk−1 = yk−1 ∧ rk(xk,yk))
Lemmar is a strict total ordering
Stepanov, McJones Elements of Programming March 14, 2008 225 / 880
Concept StrictWeakOrdering
DefinitionStrictWeakOrdering(R)⇒
Ordering(R)
EquivalenceRelation(symmetric_complementR)
Stepanov, McJones Elements of Programming March 14, 2008 226 / 880
Weak ordering is a weakening of total ordering
LemmaStrictTotalOrdering(R)⇒ StrictWeakOrdering(R)
Stepanov, McJones Elements of Programming March 14, 2008 227 / 880
Properties of weak ordering
LemmaA weak ordering is strict
LemmaA weak ordering is asymmetric
LemmaThe trichotomy law holds:r(a,b) ∨ r(b,a) ∨ symmetric_complementr(a,b)
LemmaExactly one clause of the trichotomy law holds
Stepanov, McJones Elements of Programming March 14, 2008 228 / 880
Extending strict weak orderings to a direct product
Strict weak orderings can be extended to a direct product vialexicographic ordering, just as with strict total orderings
Stepanov, McJones Elements of Programming March 14, 2008 229 / 880
Defining a strict weak ordering using a key function
A key function f on a set T and a strict total ordering r on thecodomain of f define a weak ordering r(x,y)⇔ r(f(x), f(y))
Example
Weak ordering on ith component of a direct product
template <typename F, typename R>requires(Function(F) && Arity(F) == 1 &&StrictTotalOrdering(R) && Codomain(F) == Domain(R))
struct key_ordering models(StrictWeakOrdering){
F f;R r;key_ordering(F f, R r) : f(f), r(r) { }bool operator()(const Domain(F)& x, const Domain(F)& y) const{
return r(f(x), f(y));}
};
Stepanov, McJones Elements of Programming March 14, 2008 230 / 880
Not every strict ordering is weak
Suppose a relation ∝ on {a,b, c,d, e} is a ∝ b ∝ c ∝ d,a ∝ e ∝ d:
a
b c
d
e
The symmetric complement � of ∝ is b � e, e � b, e � c, c � eIf �were an equivalence relation it would have to include b � cand c � b by transitivity
Stepanov, McJones Elements of Programming March 14, 2008 231 / 880
Mathematical conventions for usage of weak and semi
Weak refers to weakening (which includes dropping) an axiomA strict weak ordering replaces equality with equivalence
Semi refers to dropping an operationA semigroup lacks the inverse operation
Stepanov, McJones Elements of Programming March 14, 2008 232 / 880
Partial ordering
There are algorithms that deal with orderings that are not weak:algorithms on partially-ordered setsThe most important is topological sort:
Donald Knuth.Topological sorting.The Art of Computer Programming, Volume 1, Addison-Wesley,1997, pages 261-268.
Operations as simple asmax andmin do not make sense withouta weak order
Stepanov, McJones Elements of Programming March 14, 2008 233 / 880
Multiple orderings on a type
There is one equality relation on a type TThere can be many equivalence relationsSimilarly there can be many orderings
Stepanov, McJones Elements of Programming March 14, 2008 234 / 880
Natural total ordering
DefinitionThe total ordering on a type that is consistent with algebraic operationson it is called the natural total ordering
ExampleExamples of consistent axioms for natural ordering:
Incrementable a < successor(a)
Incrementable a < b⇒ successor(a) < successor(b)Ordered group a < b⇒ a+ c < b+ c
Ordered ring (a < b) ∧ (0 < c)⇒ ca < cb
Stepanov, McJones Elements of Programming March 14, 2008 235 / 880
Default ordering
Sometimes a type does not have a natural total orderingComplex numbersIterators on linked lists
We still want a total ordering to enable logarithmic searching, sowe always define the default ordering for regular types
Lexicographic ordering for complex numbersAddress-based ordering for iterators on linked lists
When the natural order exists, it coincides with the defaultordering
Definitionless<T> defines the default ordering for T
Stepanov, McJones Elements of Programming March 14, 2008 236 / 880
Concept StrictTotallyOrdered
DefinitionStrictTotallyOrdered(T)⇒
Regular(T)The relation < is defined on TStrictTotalOrdering(<)
< is reserved for the natural total ordering
Stepanov, McJones Elements of Programming March 14, 2008 237 / 880
Clusters of derived procedures
There are procedures that naturally come in clustersIf some of the cluster are defined, the definitions of the othersnaturally follow
ExampleIn an additive group, negation and subtraction constitute such a cluster
Stepanov, McJones Elements of Programming March 14, 2008 238 / 880
Derived relations
Given any relation, say, r(a,b), there arecomplement ¬r(a,b)
converse r(b,a)
complement of converse ¬r(b,a)
Given a symmetric relation, say r(a,b), since the converse is equalto the original relation, the only derivable relation is thecomplement, ¬r(a,b)
Stepanov, McJones Elements of Programming March 14, 2008 239 / 880
Properties of derived relations
LemmaGiven an ordering, its complement, its converse, and thecomplement of its converse are orderingsIf an ordering is strict, its converse is also strict and itscomplement and complement of converse are reflexiveIf an ordering is reflexive, its converse is also reflexive and itscomplement and complement of converse are strictThere exist C++ template function object classes complement,converse, and complement_of_converse which, given a relation,construct the corresponding derived relations
Stepanov, McJones Elements of Programming March 14, 2008 240 / 880
Equality and inequality
For every regular type, we have = and ,We assure consistency by always defining equality and relying onthe following template for inequality
template <typename T>requires(Regular(T))
bool operator!=(const T& a, const T& b){
return !(a == b);}
Stepanov, McJones Elements of Programming March 14, 2008 241 / 880
<, >, 6, and >
For every totally ordered type, we have <, >, 6, and >
We assure consistency by always defining < and relying on thefollowing templates for the others
template <typename T>requires(StrictTotallyOrdered(T))
bool operator>(const T& a, const T& b) { return b < a; }
template <typename T>requires(StrictTotallyOrdered(T))
bool operator<=(const T& a, const T& b) { return !(b < a); }
template <typename T>requires(StrictTotallyOrdered(T))
bool operator>=(const T& a, const T& b) { return !(a < b); }
Stepanov, McJones Elements of Programming March 14, 2008 242 / 880
Intervals
DefinitionA closed interval [a,b] is the set of all elements x such that a 6 x 6 b
An open interval (a,b) is the set of all elements x such thata < x < b
A half-open on right interval [a,b) is the set of all elements x suchthat a 6 x < b
A half-open on left interval (a,b] is the set of all elements x such thata < x 6 b
A half-open interval is our short-hand for half-open on rightThese definitions generalize to weak orderings
Stepanov, McJones Elements of Programming March 14, 2008 243 / 880
min andmax
template <typename R>requires(StrictWeakOrdering(R))
Domain(R)& min(Domain(R)& a, Domain(R)& b, R r){
if (r(b, a)) return b;else return a;
}
template <typename R>requires(StrictWeakOrdering(R))
Domain(R)& max(Domain(R)& a, Domain(R)& b, R r){
if (r(b, a)) return a;else return b;
}
Stepanov, McJones Elements of Programming March 14, 2008 244 / 880
Othermin andmax
template <typename R>requires(StrictWeakOrdering(R))
Domain(R)& other_min(Domain(R)& a, Domain(R)& b, R r){
if (r(a, b)) return a;else return b;
}
template <typename R>requires(StrictWeakOrdering(R))
Domain(R)& other_max(Domain(R)& a, Domain(R)& b, R r){
if (r(a, b)) return b;else return a;
}
We will see shortly whymin andmax are preferable toother_min and other_max 17
17The C++ standard library usesmin and other_maxStepanov, McJones Elements of Programming March 14, 2008 245 / 880
A convention for order selection functions
For the order selection procedures in this section, there are fouruseful versions
Since the parameters are being passed for selecting, versions takingboth references and const references are neededFor convenience, we want versions for totally ordered types (with<) as well as for an explicitly-supplied ordering
From now on, we show only the non-const reference version withan explict ordering; see Appendix 2 for the other versionsThe code below shows how to obtain a version for a totallyordered type given a version for an explicitly supplied ordering
Stepanov, McJones Elements of Programming March 14, 2008 246 / 880
Stability
DefinitionAn algorithm is stable if it respects the original order of equivalentelements
Stability is not relevant with a total ordering since equivalentelements are equal and, therefore, indistinguishable
ExampleStable sort
Multiple passes compose naturallySorting by first name and then by last name results in expected order
Stable partitionEven/odd partition of {1, 2, 3, 4, 5} results in {1, 3, 5, 2, 4}, not{1, 5, 3, 4, 2} as produced by a fast partitioning algorithm
Stability sometimes increases the complexity
Stepanov, McJones Elements of Programming March 14, 2008 247 / 880
Increasing order
Definition
A sequence . . . , x, . . . , x ′, . . . is in{
increasingstrictly increasing
}order with
respect to a strict weak ordering r if{
¬r(x ′, x)r(x, x ′)
}a
aWhile some would use the terms non-decreasing and increasing, we follow NicolasBourbaki, Theory of Sets, III.1.5, page 138.
In this book we always assume increasing order for sortedsequences
Stepanov, McJones Elements of Programming March 14, 2008 248 / 880
sort_2
template <typename R>requires(StrictWeakOrdering(R))
void sort_2(Domain(R)& a, Domain(R)& b, R r){
if (r(b, a)) swap(a, b);}
sort_2 is stableIt does the minimal amount of work since it does not swapequivalent elements
Stepanov, McJones Elements of Programming March 14, 2008 249 / 880
Stability ofmin andmax
Natural postcondition for sort_2:a0 = a; b0 = b;sort_2(a, b, r);assert(a == min(a0, b0, r) && b == max(a0, b0, r));
min andmax satisfy it, while other_min and other_max do not
Stepanov, McJones Elements of Programming March 14, 2008 250 / 880
Properties ofmin andmax for StrictTotalOrdering r
Lemmaassociativity minr(minr(a,b), c) = minr(a,minr(b, c))associativity maxr(maxr(a,b), c) = maxr(a,maxr(b, c))
commutativity minr(a,b) = minr(b,a)
commutativity maxr(a,b) = maxr(b,a)
absorption minr(a,maxr(a,b)) = a
absorption maxr(minr(a,b),b) = b
idempotency minr(a,a) = a
idempotency maxr(a,a) = a
Stepanov, McJones Elements of Programming March 14, 2008 251 / 880
Properties ofmin andmax for StrictWeakOrdering r
Lemmaassociativity minr(minr(a,b), c) = minr(a,minr(b, c))associativity maxr(maxr(a,b), c) = maxr(a,maxr(b, c))
commutativity symmetric_complementr(minr(a,b),minr(b,a))
commutativity symmetric_complementr(maxr(a,b),maxr(b,a))
ExerciseWhat are the absorption and idempotency laws forminr andmaxr?
Stepanov, McJones Elements of Programming March 14, 2008 252 / 880
min_3 andmax_3
template <typename R>requires(StrictWeakOrdering(R))
Domain(R)& min_3(Domain(R)& a, Domain(R)& b, Domain(R)& c, R r){
return min(min(a, b, r), c, r);}
template <typename R>requires(StrictWeakOrdering(R))
Domain(R)& max_3(Domain(R)& a, Domain(R)& b, Domain(R)& c, R r){
return max(max(a, b, r), c, r);}
Stepanov, McJones Elements of Programming March 14, 2008 253 / 880
Implementing order selection
There are more complicated cases thanmin andmax such asmedian of 3 and select 2nd of 4Writing order selections is somewhat complicated and can behelped by decomposition into simpler subproblems
Stepanov, McJones Elements of Programming March 14, 2008 254 / 880
median_3
template <typename R>requires(StrictWeakOrdering(R))
Domain(R)& median_3(Domain(R)& a, Domain(R)& b, Domain(R)& c, R r){
if (r(b, a)) return median_3_stage1(b, a, c, r);else return median_3_stage1(a, b, c, r);
}
template <typename R>requires(StrictWeakOrdering(R))
Domain(R)& median_3_stage1(Domain(R)& a, Domain(R)& b, Domain(R)& c, R r){
// Precondition: a, b are sortedif (!r(c, b)) return b; // a, b, c are sortedelse return max(a, c, r); // b is not the median
}
Stepanov, McJones Elements of Programming March 14, 2008 255 / 880
Stability ofmedian_3
ExerciseDefine the appropriate stability property formedian_3
Lemmamedian_3 is stable
Stepanov, McJones Elements of Programming March 14, 2008 256 / 880
Complexity ofmedian_3
median_3 does 3 comparisons in the worst caseThe function does 2 comparisons only when c ismax_3(a,b, c)and that happens in one-third of the casesThe average number of comparison is 2 2
3
Stepanov, McJones Elements of Programming March 14, 2008 257 / 880
Selecting second smallest
Finding second smallest implies finding smallestIf after finding second smallest, two candidates for smallest remain,then the second smallest is not second smallest!
Finding second smallest from n elements requires at leastn+ logn− 2 comparisons
Finding second out of four requires 4 comparisons
Stepanov, McJones Elements of Programming March 14, 2008 258 / 880
select_2nd_4
template <typename R> requires(StrictWeakOrdering(R))Domain(R)&select_2nd_4(Domain(R)& a, Domain(R)& b, Domain(R)& c, Domain(R)& d, R r) {
if (r(b, a)) return select_2nd_4_stage1(b, a, c, d, r);else return select_2nd_4_stage1(a, b, c, d, r);
}
template <typename R> requires(StrictWeakOrdering(R))Domain(R)&select_2nd_4_stage1(Domain(R)& a, Domain(R)& b, Domain(R)& c, Domain(R)& d, R r) {
// Precondition: a 6 bif (r(d, c)) return select_2nd_4_stage2(a, b, d, c, r);else return select_2nd_4_stage2(a, b, c, d, r);
}
template <typename R> requires(StrictWeakOrdering(R))Domain(R)&select_2nd_4_stage2(Domain(R)& a, Domain(R)& b, Domain(R)& c, Domain(R)& d, R r) {
// Precondition: a 6 b∧ c 6 dif (r(c, a)) return min(a, d, r);else return min(b, c, r);
}
Stepanov, McJones Elements of Programming March 14, 2008 259 / 880
Exercises
ExerciseIs select_2nd_4 stable?
ExerciseImplement select_3rd_4
Stepanov, McJones Elements of Programming March 14, 2008 260 / 880
median_5
template <typename R> requires(StrictWeakOrdering(R))Domain(R)& median_5(
Domain(R)& a, Domain(R)& b, Domain(R)& c, Domain(R)& d, Domain(R)& e, R r) {if (r(b, a)) return median_5_stage1(b, a, c, d, e, r);else return median_5_stage1(a, b, c, d, e, r);
}
template <typename R> requires(StrictWeakOrdering(R))Domain(R)& median_5_stage1(
Domain(R)& a, Domain(R)& b, Domain(R)& c, Domain(R)& d, Domain(R)& e, R r) {// Precondition: a 6 bif (r(d, c)) return median_5_stage2(a, b, d, c, e, r);else return median_5_stage2(a, b, c, d, e, r);
}
template <typename R> requires(StrictWeakOrdering(R))Domain(R)& median_5_stage2(
Domain(R)& a, Domain(R)& b, Domain(R)& c, Domain(R)& d, Domain(R)& e, R r) {// Precondition: a 6 b∧ c 6 dif (r(c, a)) return select_2nd_4_stage1(a, b, d, e, r);else return select_2nd_4_stage1(c, d, b, e, r);
}
Stepanov, McJones Elements of Programming March 14, 2008 261 / 880
Exercises
ExerciseShow thatmedian_5 is not stable
ExerciseDesign a stable version ofmedian_5
ExerciseFind an algorithm for median of 5 that does slightly fewercomparisons on average
Stepanov, McJones Elements of Programming March 14, 2008 262 / 880
Conclusions
Weak orderings allow efficient algorithms on collections ofelementsThe axioms of ordering provide the interface to connect specificorderings with general purpose algorithmsOverloaded operators should preserve their semantics inmathematics
Stepanov, McJones Elements of Programming March 14, 2008 263 / 880
Reference
Our treatment of ordering is based on:
N. Bourbaki.Chapter 3, Section 1: Order relations. Ordered Sets.Theory of Sets, Springer, 2004, pages 131-148.
Stepanov, McJones Elements of Programming March 14, 2008 264 / 880
Project
Using material from
Donald E. Knuth.Section 5.3: Optimum Sorting.The Art of Computer Programming, Volume 3: Sorting andSearching, 2nd edition, Addison-Wesley, 1998.
create a library for generic minimum-comparison networks forsorting, merging, and selectionFor sorting and merging networks, minimize not only the numberof comparisons, but the number of data movementsAssure stability of these networks
Stepanov, McJones Elements of Programming March 14, 2008 265 / 880
Contents I
1 Introduction
2 Foundations
3 Transformations and their orbits
4 Algorithms on algebraic structures
5 Orderings
6 Combining conceptsCombining algebraic structures and orderingRemainder
Stepanov, McJones Elements of Programming March 14, 2008 266 / 880
Contents II
Greatest common divisorExtending the domain of greatest common divisorQuotientQuotient and remainder for negative quantitiesIntegersConclusionsReferencesProject
7 Refining concepts of iterators
8 Permutations and rearrangements
9 Rotations
Stepanov, McJones Elements of Programming March 14, 2008 267 / 880
Contents III
10 Algorithms on increasing ranges
11 Coordinate structures
12 Composite objects
13 Iterative algorithms for divide-and-conquer
14 Mathematical notation
15 C++ machinery
16 Acknowledgments
Stepanov, McJones Elements of Programming March 14, 2008 268 / 880
Contents IV
17 Index
Stepanov, McJones Elements of Programming March 14, 2008 269 / 880
Axioms combine concepts
Combining different structures on the same typeCombining different types
Stepanov, McJones Elements of Programming March 14, 2008 270 / 880
Ordered structures
DefinitionOrderedAdditiveSemigroup(T)⇒
AdditiveSemigroup(T)
StrictTotallyOrdered(T)
For all x,y, z ∈ T ,x < y⇒ x+ z < y+ z
DefinitionOrderedAdditiveMonoid(T)⇒Monoid(T) ∧ OrderedAdditiveSemigroup(T)
DefinitionOrderedAdditiveGroup(T)⇒ Group(T) ∧ OrderedAdditiveMonoid(T)
Stepanov, McJones Elements of Programming March 14, 2008 271 / 880
Absolute value
template <typename T>requires(OrderedAdditiveGroup(T))
T abs(const T& x){
if (x < T(0)) return -x;else return x;
}
The correctness of abs depends on
x < T(0)⇒ x+ (−x) < T(0) + (−x)⇒ T(0) < −x
We use the notation |x| for the absolute value of x
Stepanov, McJones Elements of Programming March 14, 2008 272 / 880
Concept OrderedCancellableMonoid
DefinitionOrderedCancellableMonoid(T)⇒
OrderedAdditiveMonoid(T)
The partial binary infix operator − is defined on TFor all a,b ∈ T
b 6 a⇒ is_defined(−,a,b) ∧ (a− b) + b = a
Stepanov, McJones Elements of Programming March 14, 2008 273 / 880
Division with remainder
As repeated addition induces multiplication,repeated subtraction induces division with remainderInterestingly, it is easier to introduce the remainder computation,leaving the quotient for later
Stepanov, McJones Elements of Programming March 14, 2008 274 / 880
weak_remainder
template <typename T>requires(OrderedCancellableMonoid(T))
T weak_remainder(T a, T b){
// Precondition: a > T(0) ∧ b > T(0)while (b <= a) a = a - b;return a;
}
The concept OrderedCancellableMonoid is not strong enough toprove termination of weak_remainder; we need anotherproperty to ensure there are no unreachable (infinite) elements
Stepanov, McJones Elements of Programming March 14, 2008 275 / 880
Concept ArchimedeanMonoid
DefinitionArchimedeanMonoid(T)⇒
OrderedCancellableMonoid(T)
For all a and all b > 0 ∈ T , weak_remainder(a,b) terminates
The property is called the Axiom of Archimedes
ReflectionA concept can be defined with an axiom stating that a particularprocedure terminates
Stepanov, McJones Elements of Programming March 14, 2008 276 / 880
The Axiom of Archimedes
The Axiom of Archimedes is usually written as “there exists anintegerm such that a < m · b”Archimedes originally wrote: “... that the excess by which thegreater of (two) unequal areas exceeds the less can, by beingadded to itself, be made to exceed any given finite area.” 18
Archimedes attributes this lemma to Eudoxus 19
18T.L. Heath, translator. The Works of Archimedes, Preface to Quadrature of theParabola. Dover, 2002, page 234
19Sir Thomas Heath. A History of Greek Mathematics, Volume I. Dover, 1981, page 327Stepanov, McJones Elements of Programming March 14, 2008 277 / 880
Examples of Archimedean monoids
Line segments in Euclidean geometryRational numbersBinary fractions { n2k }
Ternary fractions { n3k }
Stepanov, McJones Elements of Programming March 14, 2008 278 / 880
Intuition for a fast algorithm for remainder
Repeated doubling leads to fast (logarithmic complexity) powerA related algorithm is possible for remainderAs with power, the Egyptians used this algorithm to do divisionwith remainder
Stepanov, McJones Elements of Programming March 14, 2008 279 / 880
Deriving a fast algorithm for remainder
If c is a monoid element andm is a nonnegative integer,m · cstands for c+ · · ·+ c︸ ︷︷ ︸
m times
Let a = n · b+ r, where r = weak_remainder(a,b)Let n = 2q+ p, where q = bn/2c and p = n mod 2a = q · (2 · b) + (p · b+ r) where (p · b+ r) < 2 · bremainder(a,b) = r =a if a < ba− b if a− b < b
remainder(a, 2 · b)) if remainder(a, 2 · b) < bremainder(a, 2 · b)) − b otherwise
Stepanov, McJones Elements of Programming March 14, 2008 280 / 880
remainder_nonnegative
template <typename T>requires(ArchimedeanMonoid(T))
T remainder_nonnegative(T a, T b){
// Precondition: a > T(0) ∧ b > T(0)if (a < b) return a;if (a - b < b) return a - b;a = remainder_nonnegative(a, b + b);if (a < b) return a;else return a - b;
}
The first comparison is not needed in the recursive calls and couldbe eliminated by the introduction of a helper functiona− b < b serves as a guard to ensure b+ b will not overflow
Stepanov, McJones Elements of Programming March 14, 2008 281 / 880
Complexity of remainder_nonnegative
It is trivial to see that remainder_nonnegative is logarithmic
ExerciseDetermine the exact counts of different operations
Stepanov, McJones Elements of Programming March 14, 2008 282 / 880
Concept WeaklyHalvableMonoid
In many situations while the monoid does not provide generaldivision by an integer, it has a division by 2
While general k-section of an angle by ruler and compass can notbe done, bisection is trivial
Divisibility by 2 gives an iterative version of remainder
DefinitionWeaklyHalvableMonoid(T)⇒
ArchimedeanMonoid(T)
The partial unary operation half_nonnegative is defined on T+
where T+ = {x ∈ T |x > T(0)}
For all a,b ∈ T+
a = b+ b⇒ half_nonnegative(a) = b
Stepanov, McJones Elements of Programming March 14, 2008 283 / 880
remainder_nonnegative_iterative 20
template <typename T>requires(WeaklyHalvableMonoid(T))
T remainder_nonnegative_iterative(T a, const T& b){
// Precondition: a > T(0) ∧ b > T(0)if (a < b) return a;T c = b;while (a - c >= c) c = c + c;a = a - c;while (c != b) {
c = half_nonnegative(c);if (c <= a) a = a - c;
}return a;
}
20Edsger W. Dijkstra attributes this algorithm to N.G. de Bruijn on page 13 of Noteson Structured Programming, in: O.-J. Dahl, E. W. Dijkstra and C.A.R. Hoare.Structured Programming. Academic Press, London and New York, 1972. The algorithmalso appears in: Niklaus Wirth. Systematic Programming: An Introduction.Prentice-Hall, 1973, program 7.22, page 38.
Stepanov, McJones Elements of Programming March 14, 2008 284 / 880
Divisibility on an Archimedean monoid T
DefinitionFor a > T(0) and b > T(0), b divides a⇔ remainder(a,b) = T(0)
Stepanov, McJones Elements of Programming March 14, 2008 285 / 880
Properties of divisibility on an Archimedean monoid
LemmaIn an Archimedean monoid T with positive x,a,b
b divides a⇒ b 6 a
b > a∧ x divides a∧ x divides b⇒ x divides (b− a)
x divides a∧ x divides b⇒ x divides remainder(a,b)
Stepanov, McJones Elements of Programming March 14, 2008 286 / 880
Concept DiscreteArchimedeanMonoid
DefinitionDiscreteArchimedeanMonoid(T)⇒
ArchimedeanMonoid(T)
(∃u ∈ T)(∀x ∈ T)x < u⇒ ¬(T(0) < x)
Such an element u is called a unit
LemmaEvery element of a discrete Archimedean monoid is divisible by theunit
Stepanov, McJones Elements of Programming March 14, 2008 287 / 880
Greatest common divisor
DefinitionThe greatest common divisor of a and b, denoted by gcd(a,b), is adivisor of a and b that is divisible by any other divisor of a and b
This definition works for Archimedean monoid, and, as we shallsee later, for other structuresNote the definition does not depend on ordering but is expressedstrictly in terms of divisibility
Stepanov, McJones Elements of Programming March 14, 2008 288 / 880
Properties of greatest common divisor
LemmaIn an Archimedean monoid with positive x,a,b, the following hold
x divides a∧ x divides b⇒ x 6 gcd(a,b)gcd(a,b) is uniquegcd(a,a) = a
Stepanov, McJones Elements of Programming March 14, 2008 289 / 880
subtractive_gcd
template <typename T>requires(ArchimedeanMonoid(T))
T subtractive_gcd(T a, T b){
// Precondition: a > T(0) ∧ b > T(0)
while (true) {if (b < a)
a = a - b;else if (a < b)
b = b - a;else
return a;}
}
This is known as Euclid’s algorithm 21
It is fairly certain that the algorithm had been known at least acentury before Euclid
21Sir Thomas Heath, translator. Euclid’s Elements. Dover, 1956: Volume 2, Book VII,Propositions 1 and 2, pages 296-300; Volume 3, Book X, Propositions 1-3, pages 14-22
Stepanov, McJones Elements of Programming March 14, 2008 290 / 880
Correctness of subtractive_gcd
Proof.1 Each subtraction preserves the property of being divisible by any
divisor of the original a and b2 If the variables become equal, they divide the original a and b3 Therefore it returns a divisor of a and b that is divisible by any
other divisor4 In other words, it returns the greatest common divisor
�
Stepanov, McJones Elements of Programming March 14, 2008 291 / 880
Termination of subtractive_gcd
LemmaIt always terminates for integers and rationals
LemmaIt does not always terminate for reals
Stepanov, McJones Elements of Programming March 14, 2008 292 / 880
Concept EuclideanMonoid
DefinitionEuclideanMonoid(T)⇒
ArchimedeanMonoid(T)
For all a > T(0),b > T(0) ∈ T , subtractive_gcd(a,b) terminates
LemmaEvery discrete Archimedean monoid is Euclidean
Stepanov, McJones Elements of Programming March 14, 2008 293 / 880
Extending subtractive_gcd to zero
It is straightforward to extend subtractive_gcd to the case whereone of its arguments is zero
template <typename T>requires(EuclideanMonoid(T))
T subtractive_gcd_with_zero(T a, T b){
// Precondition: a > T(0) ∧ b > T(0) ∧ ¬(a = T(0) ∧ b = T(0))
while (true) {if (b == T(0)) return a;while (b < a) a = a - b;if (a == T(0)) return b;while (a < b) b = b - a;
}}
Stepanov, McJones Elements of Programming March 14, 2008 294 / 880
Speeding up subtractive gcd
Each of the inner while statements in subtractive_gcd_with_zerois equivalent to a call of slow_remainderBy using our (logarithmic) remainder algorithm, we can speed upthe case where a and b are very different in magnitude
Stepanov, McJones Elements of Programming March 14, 2008 295 / 880
fast_subtractive_gcd
template <typename T>requires(EuclideanMonoid(T))
T fast_subtractive_gcd(T a, T b){
// Precondition: a > T(0) ∧ b > T(0) ∧ ¬(a = T(0) ∧ b = T(0))
while (true) {if (b == T(0)) return a;a = remainder_nonnegative(a, b);if (a == T(0)) return b;b = remainder_nonnegative(b, a);
}}
Stepanov, McJones Elements of Programming March 14, 2008 296 / 880
Generalizing the greatest common divisor algorithm
On a Euclidean monoid, the computation of greatest commondivisor is based on a particular remainder algorithmIf a structure has a properly-defined remainder, it does not have tobe a Euclidean monoid
PolynomialsGaussian integers
Mathematicians refer to these other types as Euclidean domainsOur goal is to define a concept unifying Euclidean monoids andEuclidean domains, so we can use the same gcd algorithm for both
Stepanov, McJones Elements of Programming March 14, 2008 297 / 880
Concept CommutativeSemiring
DefinitionCommutativeSemiring(T)⇒
AdditiveMonoid(T)
MultiplicativeSemigroup(T) ∧ CommutativeOperation(×)
For any a, T(0)× a = T(0)
The constant T(1) is definedFor any a , T(0), T(1)× a = a
For all a,b, c ∈ T , a× (b+ c) = a× b+ a× c
The last property is distributivity
Stepanov, McJones Elements of Programming March 14, 2008 298 / 880
Examples of CommutativeSemiring
ExampleNonnegative integers constitute a commutative semiring
Stepanov, McJones Elements of Programming March 14, 2008 299 / 880
Concept Semimodule
DefinitionSemimodule(T)⇒
AdditiveMonoid(T)
The type function Scalar(T) is definedCommutativeSemiring(Scalar(T))
The binary infix function · : Scalar(T)× T → T is definedFor all α,β ∈ Scalar(T) and for all x,y ∈ T ,
α · (β · x) = (α× β) · x(α+ β) · x = α · x+ β · xα · (x+ y) = α · x+ α · y
Scalar(T)(1) · x = x
Stepanov, McJones Elements of Programming March 14, 2008 300 / 880
Examples of Semimodule
ExampleThe set {(a,b)} of two-dimensional vectors with nonnegative realcoefficients constitutes a semimodule over nonnegative integersPolynomials with nonnegative integer coefficients constitute asemimodule over nonnegative integers
Stepanov, McJones Elements of Programming March 14, 2008 301 / 880
Concept QRSemimodule
DefinitionQRSemimodule(T)⇒
Semimodule(T)
The partial binary operation remainder is defined on TThe partial binary infix functionquotient : T × T → Scalar(T) is definedFor all a ∈ T ,b , T(0) ∈ T ,
a = quotient(a,b) · b+ remainder(a,b)
Stepanov, McJones Elements of Programming March 14, 2008 302 / 880
gcd for QRSemimodule
template <typename T>requires(QRSemimodule(T))
T gcd(T a, T b){
// Precondition: ¬(a = T(0) ∧ b = T(0))
while (true) {if (b == T(0)) return a;a = remainder(a, b);if (a == T(0)) return b;b = remainder(b, a);
}}
This procedure does not always terminate
Stepanov, McJones Elements of Programming March 14, 2008 303 / 880
Concept EuclideanSemimodule
DefinitionEuclideanSemimodule(T)⇒
QRSemimodule(T)
For all a,b ∈ T such that ¬(a = T(0) ∧ b = T(0)), gcd(a,b)terminates
Stepanov, McJones Elements of Programming March 14, 2008 304 / 880
gcd
template <typename T>requires(EuclideanSemimodule(T))
T gcd(T a, T b){
// Precondition: ¬(a = T(0) ∧ b = T(0))
while (true) {if (b == T(0)) return a;a = remainder(a, b);if (a == T(0)) return b;b = remainder(b, a);
}}
Stepanov, McJones Elements of Programming March 14, 2008 305 / 880
Concept EuclideanSemiring
DefinitionEuclideanSemiring(T)⇒
CommutativeSemiring(T)
For all a,b ∈ T , a× b = T(0)⇒ a = T(0) ∨ b = T(0)
A Euclidean function w : T →N is definedFor all a,b ∈ T , w(a× b) > w(a)
Partial binary operations remainder and quotientare defined on TFor all a ∈ T ,b , T(0) ∈ T ,
remainder(a,b) , T(0)⇒ w(remainder(a,b)) < w(b)
The fact that w decreases with application of remainder assuresthat gcd terminatesStepanov, McJones Elements of Programming March 14, 2008 306 / 880
Euclidean semiring is Euclidean semimodule
Every commutative semiring is a semimodule over itselfThis implies every Euclidean semiring is a Euclidean semimoduleThis implies gcd can be used both for Euclidean monoids and thetraditional concept of Euclidean semirings 22, such as polynomialsover reals
22Technically, the traditional concept introduced by Noether and van der Waerdenis the Euclidean ring, but semirings suffice
Stepanov, McJones Elements of Programming March 14, 2008 307 / 880
Deriving a fast algorithm for quotient and remainder
Let a = n · b+ r, where r = weak_remainder(a,b)Let n = 2q+ p, where q = bn/2c and p = n mod 2a = q · (2 · b) + (p · b+ r) where (p · b+ r) < 2 · bremainder(a,b) = r =a if a < ba− b if a− b < b
remainder(a, 2 · b)) if remainder(a, 2 · b) < bremainder(a, 2 · b)) − b otherwise
quotient(a,b) =0 if a < b1 if a− b < b
2 · quotient(a, 2 · b)) if remainder(a, 2 · b) < b2 · quotient(a, 2 · b)) + 1 otherwise
Stepanov, McJones Elements of Programming March 14, 2008 308 / 880
quotient_remainder_nonnegative
template <typename I, typename T>requires(Integer(I) && ArchimedeanMonoid(T))
pair<I, T> quotient_remainder_nonnegative(T a, T b){
// Precondition: a > T(0) ∧ b > T(0)if (a < b)
return pair<I, T>(I(0), a);if (a - b < b)
return pair<I, T>(I(1), a - b);pair<I, T> q = quotient_remainder_nonnegative(a, b + b);I n = q.first + q.first;a = q.second;if (a < b)
return pair<I, T>(n, a);else
return pair<I, T>(n + I(1), a - b);}
Stepanov, McJones Elements of Programming March 14, 2008 309 / 880
quotient_nonnegative
While we can compute remainder without quotient, we don’tknow how to compute quotient without remainderMost computer instruction sets include an instruction thatcomputes both quotient and remainderIt would be nice if programming languages provided a bindingfor this instruction
template <typename I, typename T>requires(Integer(I) && ArchimedeanMonoid(T))
I quotient_nonnegative(T a, T b){
return quotient_remainder_nonnegative<I>(a, b).first;}
Stepanov, McJones Elements of Programming March 14, 2008 310 / 880
quotient_remainder_nonnegative_iterative
template <typename I, typename T>requires(Integer(I) && WeaklyHalvableMonoid(T))
pair<I, T> quotient_remainder_nonnegative_iterative(T a, const T& b){
// Precondition: a > T(0) ∧ b > T(0)if (a < b) return pair<I, T>(I(0), a);T c = b;while (a - c >= c) c = c + c;a = a - c;I n = I(1);while (c != b) {
n = n + n;c = half_nonnegative(c);if (c <= a) {
a = a - c;n = n + I(1);
}}return pair<I, T>(n, a);
}
Stepanov, McJones Elements of Programming March 14, 2008 311 / 880
Requirements for extending quotient and remainderto negative quantities
Quotient and remainder on signed quantities must satisfy theseproperties:
quotient(a,b) is an integera = b · quotient(a,b) + remainder(a,b)|remainder(a,b)| < |b|
For any integer n, remainder(a,b) = remainder(a+ n · b,b)
Stepanov, McJones Elements of Programming March 14, 2008 312 / 880
Problems with current implementations of quotientand remainder
Our requirements are not satisfied by many programminglanguages, in which quotient truncates toward zero 23
Truncation violates our fourth requirement 24
In addition, truncation is an inferior way of rounding because itsends twice as many values to zero as to any other integer, thusleading to a nonuniform distribution
23For an excellent discussion of this and other problems, see: Raymond T. Boute.The Euclidean Definition of the Functions div and mod. ACM Transactions onProgramming Languages and Systems, Volume 14, Number 2, April 1992, pages 127-144
24This requirement is equivalent to the classical mathematical definition ofcongruence: “If two numbers a and b have the same remainder r relative to the samemodulus k they will be called congruent relative to the modulus k (following Gauss).”—P. G. L. Dirichlet, Lectures on Number Theory, American Mathematical Society, 1999
Stepanov, McJones Elements of Programming March 14, 2008 313 / 880
Adapting nonnegative remainder and quotient
The next two functions are adaptors that convert remainder andquotient_remainder correctly defined for nonnegative inputs toproduce “correct” results for positive or negative inputs
Stepanov, McJones Elements of Programming March 14, 2008 314 / 880
remainder
template <typename T, typename Op>requires(OrderedAdditiveGroup(T) && BinaryOperation(Op) && Domain(Op) == T)
T remainder(T a, T b, Op rem){
// Precondition: b , 0T r;if (a < T(0))
if (b < T(0)) {r = -rem(-a, -b);
} else {r = rem(-a, b); if (r != T(0)) r = b - r;
}else
if (b < T(0)) {r = rem(a, -b); if (r != T(0)) r = b + r;
} else {r = rem(a, b);
}return r;
}
Stepanov, McJones Elements of Programming March 14, 2008 315 / 880
quotient_remainder
template <typename I, typename T, typename Op>requires(Integer(I) && OrderedAdditiveGroup(T) &&
HomogeneousFunction(Op) && Arity(Op) == 2 && Domain(Op) == T && Codomain(Op) == pair<I, T>)pair<I, T> quotient_remainder(T a, T b, Op quo_rem){
// Precondition: b , 0pair<I, T> q_r;if (a < 0) {
if (b < 0) { q_r = quo_rem(-a, -b); q_r.second = -q_r.second; }else {
q_r = quo_rem(-a, b);if (q_r.second != 0) { q_r.second = b - q_r.second; q_r.first = q_r.first + 1; }q_r.first = -q_r.first;
}} else {
if (b < 0) {q_r = quo_rem( a, -b);if (q_r.second != 0) { q_r.second = b + q_r.second; q_r.first = q_r.first + 1; }q_r.first = -q_r.first;
}else
q_r = quo_rem( a, b);}return q_r;
}
Stepanov, McJones Elements of Programming March 14, 2008 316 / 880
Concept DiscreteArchimedeanSemiring
DefinitionDiscreteArchimedeanSemiring(T)⇒
CommutativeSemiring(T)
ArchimedeanMonoid(T)
For all a,b, c ∈ T , a < b∧ 0 < c⇒ a× c < b× cFor all a ∈ T , a < T(1)⇒ ¬(T(0) < a)
The last property is discreteness
Stepanov, McJones Elements of Programming March 14, 2008 317 / 880
Concept NonnegativeDiscreteArchimedeanSemiring
DefinitionNonnegativeDiscreteArchimedeanSemiring(T)⇒
NonnegativeDiscreteArchimedeanSemiring(T)
For all a ∈ T , T(0) 6 a
Stepanov, McJones Elements of Programming March 14, 2008 318 / 880
Concept DiscreteArchimedeanRing
DefinitionDiscreteArchimedeanRing(T)⇒
DiscreteArchimedeanSemiring(T)
AdditiveGroup(T)
Stepanov, McJones Elements of Programming March 14, 2008 319 / 880
Univalent concepts
DefinitionTwo types T1 and T2 are isomorphic if it is possible to write conversionfunctions from T1 to T2 and from T2 to T1 that preserve the proceduresand their semantics
DefinitionA concept is univalent if any types satisfying it are isomorphic
NonnegativeDiscreteArchimedeanSemiring is univalent; typessatisfying it are isomorphic toN, the natural numbers 25
DiscreteArchimedeanRing is univalent; types satisfying it areisomorphic to Z, the integers
25We follow Peano and include 0 in the natural numbers: Giuseppe Peano.Formulario Mathematico, Edizioni Cremonese, Roma, 1960, page 27
Stepanov, McJones Elements of Programming March 14, 2008 320 / 880
Computer integer types
Computer instruction sets and programming languages providepartial implementations of natural numbers and integersWhile providing a detailed formal specification for computerintegers lies outside the scope of this book, there are someimportant points we must makeFew if any programming languages provide access to the fullpower of hardware operations for addition, subtraction,multiplication, quotient, and remainderWe sketch appropriate interfaces
Stepanov, McJones Elements of Programming March 14, 2008 321 / 880
Bounded unsigned and signed integer types
DefinitionA bounded unsigned integer type, Un, where n = 8, 16, 32, 64, . . ., is anunsigned integer type capable of representing a value in the interval[0, 2n)
DefinitionA bounded signed integer type, Sn, where n = 8, 16, 32, 64, . . ., is asigned integer type capable of representing a value in the interval[−2n−1, 2n−1)
Stepanov, McJones Elements of Programming March 14, 2008 322 / 880
Operations for bounded unsigned and signed types
Programming languages typically provide operations restrictedwithin a single typeArithmetic operations such as multiplication and addition returnresults that are larger than the type of their operandsWhile computer instructions return full results, a programmer in ahigher-level language has no access to themExtended operations such as the ones below should be provided
sum_extended : Un ×Un ×U1 → U1 ×Undifference_extended : Un ×Un ×U1 → U1 ×Unproduct_extended : Un ×Un → U2n
quotient_remainder_extended : U2n ×Un → Un ×Un
Stepanov, McJones Elements of Programming March 14, 2008 323 / 880
Arithmetic in a programming language
A study of the instruction sets for modern computer architecturesshows the functionality that should be encompassedA good abstraction of these instruction sets is provided byKnuth’s MMIX 26
26Donald E. Knuth. The Art of Computer Programming, Volume 1, Fascicle 1, MMIX : ARISC Computer for the New Millenium, Addison-Wesley, 2005, pages 1-28
Stepanov, McJones Elements of Programming March 14, 2008 324 / 880
Conclusions
Our task is to combine algorithms and mathematical structuresinto a seamless whole by describing algorithms in abstract termsand adjusting theories to fit algorithmic requirementsIf this chapter seems too mathematical, it should be noted that thealgorithms and mathematics in it are modern restatements ofresults that are thousands of years old
Stepanov, McJones Elements of Programming March 14, 2008 325 / 880
References
Donald Knuth.The Greatest Common Divisor, and Division of Polynomials.The Art of Computer Programming, Volume 2, 3rd edition, 1998,Sections 4.5.2 and 4.6.1, pages 333-356 and 420-439.Exhaustive coverage of both Euclidean and Stein (binary) gcd.
Pierre Samuel.About Euclidean Rings.Journal of Algebra, Volume 19, 1971, pages 282-301.Exhaustive and relatively elementary treatment of Euclidean rings.
Stepanov, McJones Elements of Programming March 14, 2008 326 / 880
binary_gcd_nonnegative
In 1961, Josef Stein discovered the following gcd algorithm 27:
template <typename T>requires(Integer(T))
T binary_gcd_nonnegative(T a, T b){
if (is_zero(a)) return b;if (is_zero(b)) return a;int d = 0;while (is_even(a) && is_even(b)) {
halve_nonnegative(a); halve_nonnegative(b); d = d + 1;}while (is_even(a)) halve_nonnegative(a);while (is_even(b)) halve_nonnegative(b);while (true)
if (a < b) {b = b - a; do { halve_nonnegative(b); } while (is_even(b));
} else if (b < a) {a = a - b; do { halve_nonnegative(a); } while (is_even(a));
} else return binary_scale_up_nonnegative(a, d);}
27Josef Stein. Computational problems associated with Racah algebra. J. Comput.Phys., Volume 1, 1967, pages 397-405
Stepanov, McJones Elements of Programming March 14, 2008 327 / 880
Generalizations of binary_gcd_nonnegative
While Stein’s algorithm might at first appear to be a “hack”dependent on binary integers, it is actually much deeperThe key observation is that 2 is the smallest integer prime, and theonly nonzero remainder mod2 is 1, a unit or invertible ringelementIt is possible to generalize it to other domains by using smallestprimes in those domains, such as the monomial x for polynomialsover reals or 1 + i for Gaussian integers
Polynomials See Knuth (Exercise 4.6.1.6, page 435 andSolution, page 673)
Gaussian integers See WeilertOther algebraic integer rings See Damgård and Frandsen,
and Agarwal and Frandsen
Stepanov, McJones Elements of Programming March 14, 2008 328 / 880
Project
ProjectFind the correct abstract setting for binary binary_gcd_nonnegative(Stein domain)
Stepanov, McJones Elements of Programming March 14, 2008 329 / 880
Additional references for binary_gcd_nonnegative
Andre Weilert.(1+ i)-ary GCD Computation in Z[i] as an Analogue of the BinaryGCD Algorithm.J. Symbolic Computation (2000) 30, pages 605-617.
Ivan Bjerre Damgård and Gudmund Skovbjerg Frandsen.Efficient algorithms for GCD and cubic residuosity in the ring ofEisenstein integers.Proceedings of the 14th International Symposium on Fundamentals ofComputation Theory, Lecture Notes in Computer Science 2751,Springer-Verlag (2003), pages 109-117.
Saurabh Agarwal and Gudmund Skovbjerg Frandsen.Binary GCD Like Algorithms for Some Complex Quadratic Rings.ANTS 2004, pages 57-71.
Stepanov, McJones Elements of Programming March 14, 2008 330 / 880
Contents I
1 Introduction
2 Foundations
3 Transformations and their orbits
4 Algorithms on algebraic structures
5 Orderings
6 Combining concepts
7 Refining concepts of iterators
Stepanov, McJones Elements of Programming March 14, 2008 331 / 880
Contents II
Memory and dereferencingActionsIteratorsRangesForward iteratorsBidirectional iteratorsIndexed iteratorsRandom access iteratorsConclusionsReferenceProject
8 Permutations and rearrangements
9 Rotations
Stepanov, McJones Elements of Programming March 14, 2008 332 / 880
Contents III
10 Algorithms on increasing ranges
11 Coordinate structures
12 Composite objects
13 Iterative algorithms for divide-and-conquer
14 Mathematical notation
15 C++ machinery
16 Acknowledgments
Stepanov, McJones Elements of Programming March 14, 2008 333 / 880
Contents IV
17 Index
Stepanov, McJones Elements of Programming March 14, 2008 334 / 880
Memory
IntuitionMemory is a set of locations, each with an address and a valueGetting or setting the value given an address is “fast”The association of a value with a location is changeable
ExamplesThe physical and virtual address spaces of a computerThe locations visited by an algorithmThe locations owned by a data structure
Stepanov, McJones Elements of Programming March 14, 2008 335 / 880
Concept Readable
DefinitionReadable(T)⇒
The type function ValueType(T) is definedRegular(ValueType(T))source : T → ValueType(T) is defined a
asource returns its result by value, by reference, or by constant reference
Stepanov, McJones Elements of Programming March 14, 2008 336 / 880
Concept Writable
DefinitionWritable(T)⇒
The type function ValueType(T) is definedRegular(ValueType(T))If x ∈ T and v ∈ ValueType(T), then sink(x)← v is defined
No other use of sink(x) can be justified by the concept Writable,although other uses may be supported by a specific type modelingWritableFor a particular state of an object x, only a single assignment tosink(x) can be justified by the concept Writable; a specific typemight provide a protocol allowing subsequent assignments tosink(x)
Stepanov, McJones Elements of Programming March 14, 2008 337 / 880
is_aliased
template <typename T, typename U>requires(Writable(T) && Readable(U) && ValueType(T) == ValueType(U))
bool is_aliased(const T& x, const U& y){
// return true⇔ immediately after executing sink(x)← v, source(y) = v// For many types, this works:typedef const ValueType(T)* P;return P(&sink(x)) == P(&source(y));
}
Stepanov, McJones Elements of Programming March 14, 2008 338 / 880
Concept Mutable
DefinitionMutable(T)⇒
Readable(T) ∧ Writable(T)For all x ∈ T , is_aliased(x, x)
Stepanov, McJones Elements of Programming March 14, 2008 339 / 880
Concept Dereferenceable
DefinitionDereferenceable(T)⇒
Readable(T) ∨ Writable(T) ∨ Mutable(T)
Dereferenceable is a useful concept for describing types eventhough it is not used in programs (since it’s not clear which ofsource or sink could be used)
Stepanov, McJones Elements of Programming March 14, 2008 340 / 880
Extending Dereferenceable functions to all regular types
ReflectionIt is useful if dereferencing an object x that doesn’t refer to anotherobject returns x itselfTherefore we assume that unless otherwise defined,ValueType(T) = T and source and sink return the object towhich they are applied
Stepanov, McJones Elements of Programming March 14, 2008 341 / 880
Concept Action
DefinitionAction(T)⇒
ProperProcedure(T)Arity(T) = 1T takes its argument by reference
Stepanov, McJones Elements of Programming March 14, 2008 342 / 880
Duality of actions and transformations
For every action there is a corresponding transformation, and viceversaThe following two functions demonstrate this equivalenceThe complexity of an action when implemented independently ofthe corresponding transformation could be smaller
Example: interchange the first and last elements of a sequence
Stepanov, McJones Elements of Programming March 14, 2008 343 / 880
transformation_from_action
template <typename A>requires(Action(A))
struct transformation_from_action{
A a;transformation_from_action() {}transformation_from_action(const A& a) : a(a) {}Domain(A) operator()(Domain(A) x) { a(x); return x; }
};
Stepanov, McJones Elements of Programming March 14, 2008 344 / 880
action_from_transformation
template <typename F>requires(Transformation(F))
struct action_from_transformation{
F f;action_from_transformation() {}action_from_transformation(const F& f) : f(f) {}void operator()(Domain(F)& x) { x = f(x); }
};
Stepanov, McJones Elements of Programming March 14, 2008 345 / 880
Concept RegularAction
DefinitionRegularAction(T)⇒
Action(T)
RegularFunction(transformation_from_actionT )
Stepanov, McJones Elements of Programming March 14, 2008 346 / 880
Concept AmortizedConstantTime
DefinitionThe amortized complexity of an operation is the complexity averagedover a worst-case sequence of operations a
aFor an extensive treatment of amortized complexity, see: Robert Endre Tarjan.Amortized Computational Complexity. SIAM Journal on Algebraic and DiscreteMethods, Volume 6, Number 2, April 1985, pages 306-318
DefinitionAmortizedConstantTime(T)⇒
ProperProcedure(T)The amortized time complexity of it is constant and “small”
Formalizing complexity specifications of abstract operations is asubject for future research
Stepanov, McJones Elements of Programming March 14, 2008 347 / 880
Concept Iterator 28
DefinitionIterator(T)⇒
Regular(T)The type function DistanceType(T) is definedInteger(DistanceType(T))The prefix operator ++ is defined on TAction(++)
AmortizedConstantTime(++)
28Our treatment of iterators departs significantly from that in the STLStepanov, McJones Elements of Programming March 14, 2008 348 / 880
Iterator protocols
++ is not necessarily regularIf there are two copies of an iterator and one is incremented, theother may become invalidThus an iterator is too weak to be used in a multipass algorithm
Assignment to a sink is not idempotent: a call to ++must separatetwo assignments to an iterator
The asymmetry between readable and writable iterators isintentionalThe ability to read from sourcemore than once allows us to writesimple, useful functions like find_ifNo corresponding benefits seem to accrue in the case of sink
Stepanov, McJones Elements of Programming March 14, 2008 349 / 880
Examples of Iterator
An iterator where ++ advances an input streamAn iterator where ++ advances an output streamAn iterator on a singly-linked listAn iterator on a doubly-linked listAn iterator on a one-dimensional arrayint∗int
Stepanov, McJones Elements of Programming March 14, 2008 350 / 880
successor and - for Iterator
template <typename I>requires(Iterator(I))
I successor(I f){
++f;return f;
}
template <typename I>requires(Iterator(I))
DistanceType(I) operator-(I l, I f){
return distance(f, l, successor<I>);}
Stepanov, McJones Elements of Programming March 14, 2008 351 / 880
+= and + for Iterator
template <typename I>requires(Iterator(I))
void operator+=(I& f, DistanceType(I) n){
typedef DistanceType(I) N;while (n != N(0)) {
++f;n = n - N(1);
}}
template <typename I>requires(Iterator(I))
I operator+(I f, DistanceType(I) n){
f += n;return f;
}
Stepanov, McJones Elements of Programming March 14, 2008 352 / 880
Orbit of successor
Since successor is a transformation, it defines an orbit starting atany iteratorIn this book, we only deal with terminating iterator orbitsTherefore, if, for i > 0, successori(x) is defined, it is not equal to x
Stepanov, McJones Elements of Programming March 14, 2008 353 / 880
The natural ordering on an orbit under successor
There exists a natural total ordering ≺ on the iterators in the orbitof x under successor defined by
x ≺ y⇔ (∃i > 0)y = successori(x)
≺ and the corresponding nonstrict � are used in specificationssuch as preconditions and postconditions of algorithmsFor many pairs of values of an iterator type, ≺ is not defined, sothere is often no effective way to write code implementing ≺
There is no efficient way to determine if one node precedes anotherin a linked structure (the nodes might not even be linked together)
Stepanov, McJones Elements of Programming March 14, 2008 354 / 880
Sequences of iterators
It is useful to have standard terminology and notation forsequences and subsequences of iterators
To describe the input and/or output of an algorithmTo describe ranges in a data structure
It is useful to describe a (possibly empty) sequence of iteratorsstarting at a particular iterator
ExampleBinary search looks for the sequence of iterators whose values areequal to a given valueThis sequence is empty if there are no such values
The location of this empty sequence carries information: the valuesin the sequence approximating the given value
Stepanov, McJones Elements of Programming March 14, 2008 355 / 880
Ranges
Such a located sequence of iterators is fully specified by a startingiterator and a nonnegative integerIt is often convenient to specify a sequence by specifying a startingiterator together with another one that bounds the sequenceA specification of a such a located sequence is called a rangeRanges can be either counted or bounded, and either semi-open orclosed
Stepanov, McJones Elements of Programming March 14, 2008 356 / 880
Different kinds of ranges
DefinitionA semi-open counted range Ji,nM, where n > 0 is an integer, denotes thesequence of iterators {j|i � j ≺ successorn(i)}
DefinitionA closed counted range Ji,nK, where n > 0 is an integer, denotes thesequence of iterators {k|i � k � successorn−1(i)}
DefinitionA semi-open bounded range [i, j) denotes the sequence of iterators{k|i � k ≺ j}
DefinitionA closed bounded range [i, j] denotes the sequence of iterators{k|i � k � j}
Stepanov, McJones Elements of Programming March 14, 2008 357 / 880
Terminology for iterators in a range
DefinitionAn iterator f in a range [i, j) is called the first if f = i∧ f , j
An iterator l in a range [i, j) is called the limit if l = j
An iterator k in a range [i, j) is called the lasta if successor(k) = j
Otherwise an iterator in the range is called not last
aThe terminology distinguishing limit and last was suggested by John Banning
Stepanov, McJones Elements of Programming March 14, 2008 358 / 880
Size of a range
DefinitionThe size of a counted range Ji,nM or Ji,nK is nThe size of a semi-open bounded range [i, j) isdistance(i, j, successor)The size of a closed bounded range [i, j] isdistance(i, j, successor) + 1
The size of a range is used in preconditions and postconditions ofalgorithms even when it cannot be effectively computed becauseof an iterator too weak for a multipass algorithm
Stepanov, McJones Elements of Programming March 14, 2008 359 / 880
Empty ranges
DefinitionAn empty semi-open range is specified by Ji, 0M or [i, i) for someiterator iThere are no empty closed ranges
Stepanov, McJones Elements of Programming March 14, 2008 360 / 880
Definition space of ++ on ranges
Lemma++ and successor are defined for every iterator in a semi-open range,and for every iterator except the last in a closed range
Stepanov, McJones Elements of Programming March 14, 2008 361 / 880
Readable, writable, and mutable ranges
DefinitionA range r is readable, writable, or mutable if, correspondingly, source,sink, or both of them are defined on all the iterators in the range
Stepanov, McJones Elements of Programming March 14, 2008 362 / 880
Type requirements of copy
template <typename I, typename O>requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&
ValueType(I) == ValueType(O))O copy(I f, I l, O o);
Readable(I) ensures source is definedIterator(I) ensures ++ is definedWritable(O) ensures sink is definedIterator(O) ensures ++ is definedValueType(I) = ValueType(O) ensures assignment is defined
Stepanov, McJones Elements of Programming March 14, 2008 363 / 880
Precondition and postcondition of copy
template <typename I, typename O>requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&
ValueType(I) == ValueType(O))O copy(I f, I l, O o);
Precondition The input range must be readableThe output range must be writable and of at leastthe size of the input rangeIf the ith iterator in the input range is aliased to thejth iterator of the output, then i 6 j (informally,every value is used before it is overwritten)
Postcondition The sequence of values in the output range is equalto the sequence of original values in the input range
Stepanov, McJones Elements of Programming March 14, 2008 364 / 880
Implementation of copy
template <typename I, typename O>requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&
ValueType(I) == ValueType(O))O copy(I f, I l, O o){
while (f != l) {sink(o) = source(f);++f;++o;
}return o;
}
We return the end of the output range because it might not beknown to the caller, who might find it useful
It is worth the small constant time to return it
Stepanov, McJones Elements of Programming March 14, 2008 365 / 880
Example using copy
template <typename O>requires(Writable(O) && Iterator(O) && Integer(ValueType(O)))
O iota(ValueType(O) n, O o) // like APL ι{
return copy(ValueType(O)(0), n, o);}
Stepanov, McJones Elements of Programming March 14, 2008 366 / 880
copy_bounded
template <typename I, typename O>requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&
ValueType(I) == ValueType(O))pair<I, O> copy_bounded(I f, I l, O o_f, O o_l){
while (f != l && o_f != o_l) {sink(o_f) = source(f);++f;++o_f;
}return pair<I, O>(f, o_f);
}
While the ends of both ranges are known to the caller, returningthe pair allows determining which range is smaller and where inthe larger range copying stopped
Stepanov, McJones Elements of Programming March 14, 2008 367 / 880
copy_n
template <typename I, typename O>requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&
ValueType(I) == ValueType(O))pair<I, O> copy_n(I f, DistanceType(I) n, O o){
typedef DistanceType(I) N;while (n != N(0)) {
sink(o) = source(f);++f;++o;n = n - N(1);
}return pair<I, O>(f, o);
}
Stepanov, McJones Elements of Programming March 14, 2008 368 / 880
copy_k
template <int k, typename I, typename O>requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&
ValueType(I) == ValueType(O))struct copy_k{
void operator()(I& f, O& r){
copy_k<k - 1, I, O>(f, r);sink(r) = source(f);++f; ++r;
}};
template <typename I, typename O>struct copy_k<0, I, O> {
void operator()(I&, O&) { }};
Stepanov, McJones Elements of Programming March 14, 2008 369 / 880
copy_n_unrolled
template <typename I, typename O>requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&
ValueType(I) == ValueType(O))pair<I, O> copy_n_unrolled(I f, DistanceType(I) n, I r){
typedef DistanceType(I) N;const int k = 4; // unroll factorwhile (n >= N(k)) {
copy_k<k, I, O>()(f, r);n = n - N(k);
}return copy_n(f, n, r);
}
Stepanov, McJones Elements of Programming March 14, 2008 370 / 880
Concept ForwardIterator
DefinitionForwardIterator(T)⇒
Iterator(T)RegularAction(++)
Iterator and ForwardIterator differ only by an axiom; there are nonew operations
Stepanov, McJones Elements of Programming March 14, 2008 371 / 880
Examples of ForwardIterator
An iterator on a singly-linked listAn iterator on a doubly-linked listAn iterator on a one-dimensional arrayint∗int
Stepanov, McJones Elements of Programming March 14, 2008 372 / 880
Advantages of regularity of ++
ReflectionForwardIterator allows
algorithms that maintain more than one iterator into a rangemultipass algorithms
Stepanov, McJones Elements of Programming March 14, 2008 373 / 880
Concept BidirectionalIterator
DefinitionBidirectionalIterator(T)⇒
ForwardIterator(T)The prefix operator -- and the corresponding transformationpredecessor are definedRegularAction(--)
AmortizedConstantTime(--)For any iterator k in a semi-open range [i, j):
k , j⇒ predecessor(successor(k)) = k
k , i⇒ successor(predecessor(k)) = k
Stepanov, McJones Elements of Programming March 14, 2008 374 / 880
Examples of BidirectionalIterator
An iterator on a doubly-linked listAn iterator on a one-dimensional arrayint∗int
Stepanov, McJones Elements of Programming March 14, 2008 375 / 880
-= and - for BidirectionalIterator
template <typename I>requires(BidirectionalIterator(I))
void operator-=(I& f, DistanceType(I) n){
typedef DistanceType(I) N;while (n != N(0)) {
--f;n = n - N(1);
}}
template <typename I>requires(BidirectionalIterator(I))
I operator-(I f, DistanceType(I) n){
f -= n;return f;
}
Stepanov, McJones Elements of Programming March 14, 2008 376 / 880
Using copy with overlapping ranges
If we need to copy a range to a destination that begins within thesource range, copy cannot be used because its aliasingprecondition is violated
Stepanov, McJones Elements of Programming March 14, 2008 377 / 880
copy_backward
The main difference from copy is the aliasing precondition: if theith iterator in the input range is aliased to the jth iterator of theoutput, then j 6 i
template <typename I, typename O>requires(Readable(I) && BidirectionalIterator(I) &&
Writable(O) && BidirectionalIterator(O) &&ValueType(I) == ValueType(O))
O copy_backward(I f, I l, O o){
while (f != l) {--l;--o;sink(o) = source(l);
}return o;
}
Stepanov, McJones Elements of Programming March 14, 2008 378 / 880
copy_backward_n
template <typename I, typename O>requires(Readable(I) && BidirectionalIterator(I) &&
Writable(O) && BidirectionalIterator(O) &&ValueType(I) == ValueType(O))
pair<I, O> copy_backward_n(I l, DistanceType(I) n, O o){
typedef DistanceType(I) N;while (n != N(0)) {
--l;--o;sink(o) = source(l);n = n - N(1);
}return pair<I, O>(l, o);
}
Stepanov, McJones Elements of Programming March 14, 2008 379 / 880
Concept IndexedIterator
DefinitionIndexedIterator(T)⇒
ForwardIterator(T)The mutating assignment += is defined on T ×DistanceType(T)- : T × T → DistanceType(T) is definedAmortizedConstantTime(+=)AmortizedConstantTime(-)
+= and -, which were defined for Iterator, are now required to beprimitive
Stepanov, McJones Elements of Programming March 14, 2008 380 / 880
+ for IndexedIterator
The default implementation of + : I×DistanceType(I)→ I interms of += for Iterator becomes constant-time for IndexedIterator
Stepanov, McJones Elements of Programming March 14, 2008 381 / 880
Examples of IndexedIterator
An iterator on a one-dimensional arrayint∗int
Stepanov, McJones Elements of Programming March 14, 2008 382 / 880
copy_k_indexed
template <int k, typename I, typename O>requires(Readable(I) && IndexedIterator(I) &&
Writable(O) && IndexedIterator(O) &&ValueType(I) == ValueType(O))
struct basic_copy_k_indexed {void operator()(I f, O o){
basic_copy_k_indexed<k - 1, I, O>()(f, o);sink(o + (k - 1)) = source(f + (k - 1));
}};
template <typename I, typename O>struct basic_copy_k_indexed<0, I, O>{ void operator()(I, O) { } };
template <typename I, typename O, int k>requires(Readable(I) && IndexedIterator(I) &&
Writable(O) && IndexedIterator(O) &&ValueType(I) == ValueType(O))
void copy_k_indexed(I& f, O& o) {basic_copy_k_indexed<k, I, O>()(f, o);f += k; o += k;
}Stepanov, McJones Elements of Programming March 14, 2008 383 / 880
copy_n_unrolled_indexed
template <typename I, typename O>requires(Readable(I) && IndexedIterator(I) &&
Writable(O) && IndexedIterator(O) &&ValueType(I) == ValueType(O))
pair<I, O> copy_n_unrolled_indexed(I f, DistanceType(I) n, O o){
typedef DistanceType(I) N;const int k = 4; // unroll factorwhile (n >= N(k)) {
copy_k_indexed<k, I, O>(f, o);n = n - N(k);
}return copy_n(f, n, o);
}
Stepanov, McJones Elements of Programming March 14, 2008 384 / 880
Out-of-order execution
Out-of-order execution of adjacent iterations speeds up theexecution of copyA compiler cannot achieve this because it cannot determine thatthe source and destination ranges do not aliasProgrammers know when this is the case, and should be able tospecify when such interleaved execution is legitimate
Stepanov, McJones Elements of Programming March 14, 2008 385 / 880
copy_parallel for disjoint ranges
The main difference from copy is the aliasing precondition: theinput and output ranges do not aliasA special forall executes its body for all values of its iterationvariable in arbitrary order, possibly concurrently
template <typename I, typename O>requires(Readable(I) && IndexedIterator(I) &&
Writable(O) && IndexedIterator(O) &&ValueType(I) == ValueType(O))
void copy_parallel(I f, DistanceType(I) n, O o){
typedef DistanceType(I) N;forall(N i, N(0), n) {
sink(o + i) = source(f + i);}
}
Stepanov, McJones Elements of Programming March 14, 2008 386 / 880
Concept RandomAccessIterator (1 of 2)
Definition (requirements, signatures)RandomAccessIterator(T)⇒
BidirectionalIterator(T) ∧ IndexedIterator(T)StrictTotallyOrdered(T)
The type function DifferenceType(T) is definedInteger(DifferenceType(T))DifferenceType(T) is large enough to contain distances and theirnegationsThe mutating assignments += and -= are defined onT ×DifferenceType(T)+ : T ×DifferenceType(T)→ T is defined- : T ×DifferenceType(T)→ T is defined- : T × T → DifferenceType(T) is defined
Stepanov, McJones Elements of Programming March 14, 2008 387 / 880
Concept RandomAccessIterator (2 of 2)
Definition (complexity, definition spaces)RandomAccessIterator(T)⇒
AmortizedConstantTime(<)AmortizedConstantTime(-=)AmortizedConstantTime(- : T ×DifferenceType(T)→ T)
+=, -=, +, and - : T ×DifferenceType(T)→ T accept negativevalues for their second argumentAmortizedConstantTime(- : T × T → DifferenceType(T))
- : T × T → DifferenceType(T)) accepts iterators in either order
Stepanov, McJones Elements of Programming March 14, 2008 388 / 880
Axioms for RandomAccessIterator
ExerciseWrite an appropriate set of axioms relating the operations to each other
Stepanov, McJones Elements of Programming March 14, 2008 389 / 880
Examples of RandomAccessIterator
int∗int
Stepanov, McJones Elements of Programming March 14, 2008 390 / 880
Equivalence ofRandomAccessIterator and IndexedIterator
TheoremFor any function defined on a range of random access iterators, there isanother function defined on indexed iterators with the sameasymptotic complexity
Stepanov, McJones Elements of Programming March 14, 2008 391 / 880
Proof of the equivalence
Assuming these definitions:
U ≡ DistanceType(I)W ≡ sign_extendedU
where sign_extended is a templated struct adding a sign bit andappropriate integer operations, we rewrite the function with thesesubstitutions:
Replace WithDifferenceType(I) W
i < j i - f < j - fi += nwhen n < 0 i = f + ((i - f) - U(-n))
i -= n i += -ni - j W(i - f) - W(j - f)
Stepanov, McJones Elements of Programming March 14, 2008 392 / 880
Reflection on RandomAccessIterator and IndexedIterator
The theorem shows the theoretical equivalence of these conceptsin any context in which the beginning of ranges are known
copy_backward does not satisfy this requirement!
In practice we have found there is no performance penalty forusing the weaker concept
Stepanov, McJones Elements of Programming March 14, 2008 393 / 880
copy_backward_n_indexed
copy_backward is not realizable for indexed iteratorscopy_backward_n_indexed, which is often just as useful, isrealizable
template <typename I, typename O>requires(Readable(I) && IndexedIterator(I) &&
Writable(O) && IndexedIterator(O) &&ValueType(I) == ValueType(O))
void copy_backward_n_indexed(I f, DistanceType(I) n, O o){
typedef DistanceType(I) N;while (n != N(0)) {
n = n - N(1);sink(o + n) = source(f + n);
}}
There is no useful information to return
Stepanov, McJones Elements of Programming March 14, 2008 394 / 880
Relationships between iterator concepts
It FI
BI
II
RI
Stepanov, McJones Elements of Programming March 14, 2008 395 / 880
Conclusions
Refinement generates mathematical structures such as groups,Abelian groups, totally-ordered groups, and Archimedean groupsIt also generates concepts describing the fundamental notion ofcomputer science, iterating through a data structureWe have used three types of refinement, by adding
an operationan axioma tighter complexity requirement
Stepanov, McJones Elements of Programming March 14, 2008 396 / 880
Reference
Alexander Stepanov and Meng Lee.The Standard Template Library.HP Laboratories Technical Report 95-11(R.1), November 14, 1995.Introduced different categories of iterators and their axioms.
Stepanov, McJones Elements of Programming March 14, 2008 397 / 880
Intuition for SegmentedIterator
There are data structures where ++ can be implemented fasterwithin certain ranges or segments
hash tablesadjacency list representation of graphsSTL-like deques
It is possible to optimize many algorithms for such structures bytransforming inner loops into nested loops:
a loop over segmentsa loop within a segment
This results in a new dimension of the classification of iterators:homogeneoussegmented
Stepanov, McJones Elements of Programming March 14, 2008 398 / 880
Concept SegmentedIterator (1 of 2)
Definition (requirements, signatures, definition spaces)SegmentedIterator(T)⇒
ForwardIterator(T)The type function SegmentIterator(T) is definedForwardIterator(SegmentIterator(T))begin : T → SegmentIterator(T) is definedend : T → SegmentIterator(T) is defined+ : T × SegmentIterator(T)⇒ T is definedAmortizedConstantTime(begin)
AmortizedConstantTime(end)AmortizedConstantTime(+)
For any i ∈ [f, l], begin, end, and + are definedFor any w ∈ [begin(i), end(i)], i+w is defined
Stepanov, McJones Elements of Programming March 14, 2008 399 / 880
Concept SegmentedIterator (2 of 2)
Definition (axioms)SegmentedIterator(T)⇒If
[f, l) is a range of segmented iteratorsi ∈ [f, l)w is a segment iterator in [begin(i), end(i))
then the following hold:1 is_aliased(i,begin(i))
2 begin(i+w) = w
3 i+ begin(i) = i
4 i+ (begin(i) + 1) = i+ 15 begin(i) + 1 , end(i)⇒ begin(i) + 1 = begin(i+ 1)
6 begin(i) , end(i)
Stepanov, McJones Elements of Programming March 14, 2008 400 / 880
copy_from_segmented
template <typename I, typename O>requires(Readable(I) && SegmentedIterator(I) &&
Writable(O) && Iterator(O) &&ValueType(I) == ValueType(O))
O copy_from_segmented(I f, I l, O o){
while (f + end(f) != l + end(l)) {// f and l are in different segmentso = copy(begin(f), end(f), o);f = f + end(f);
}// f and l are in the same segmentreturn copy(begin(f), begin(l), o);
}
Stepanov, McJones Elements of Programming March 14, 2008 401 / 880
Project
Implement segmented iterators for one or more data structuresSTL-like dequeSGI STL-like hashed containers
Produce segmented iterator versions of suitable (STL) algorithmsAnalyze if the axioms are independent, consistent, and complete
Stepanov, McJones Elements of Programming March 14, 2008 402 / 880
Contents I
1 Introduction
2 Foundations
3 Transformations and their orbits
4 Algorithms on algebraic structures
5 Orderings
6 Combining concepts
7 Refining concepts of iterators
Stepanov, McJones Elements of Programming March 14, 2008 403 / 880
Contents II
8 Permutations and rearrangementsPermutationsPermutation groupsCycle decomposition of a permutationRearrangementsUnderlying typeRearranging arbitrary cyclesReverse permutationReverse algorithmsCategory dispatchConclusionsReading
9 Rotations
Stepanov, McJones Elements of Programming March 14, 2008 404 / 880
Contents III
10 Algorithms on increasing ranges
11 Coordinate structures
12 Composite objects
13 Iterative algorithms for divide-and-conquer
14 Mathematical notation
15 C++ machinery
16 Acknowledgments
Stepanov, McJones Elements of Programming March 14, 2008 405 / 880
Contents IV
17 Index
Stepanov, McJones Elements of Programming March 14, 2008 406 / 880
Onto functions
DefinitionA (regular) function f is onto if for all y ∈ Codomain(F), there existsx ∈ Domain(R) such that y = f(x)
Stepanov, McJones Elements of Programming March 14, 2008 407 / 880
One-to-one functions
DefinitionA (regular) function f is one-to-one if for all x, x ′ ∈ Domain(F),f(x) = f(x ′)⇒ x = x ′
Stepanov, McJones Elements of Programming March 14, 2008 408 / 880
Permutation
DefinitionA permutation is a transformation on a finite domain that is one-to-oneand onto
Stepanov, McJones Elements of Programming March 14, 2008 409 / 880
Example of a permutation on [0, 6)
p(0) = 5p(1) = 2p(2) = 4p(3) = 3p(4) = 1p(5) = 0
Stepanov, McJones Elements of Programming March 14, 2008 410 / 880
Identity permutation
DefinitionA fixed point of a transformation is an element x such that f(x) = x
DefinitionThe identity permutation on a set S, identityS, maps each element of S toitself; every element in S is a fixed point of identityS
Stepanov, McJones Elements of Programming March 14, 2008 411 / 880
Composition of permutations
DefinitionIf p and q are two permutations on a set S, the composition q ◦ p takesx ∈ S to q(p(x))
LemmaThe composition of permutations is a permutation
LemmaComposition of permutations is associative
Stepanov, McJones Elements of Programming March 14, 2008 412 / 880
Inverse of a permutation
LemmaFor every permutation p on a set S, there is an inverse permutation p−1
such that p−1 ◦ p = p ◦ p−1 = identityS
Stepanov, McJones Elements of Programming March 14, 2008 413 / 880
Permutation group
The permutations on a set form a group under composition
LemmaEvery group is a subgroup of a permutation group of its elementswhere every permutation in the subgroup is generated by multiplyingall the elements by an individual element
Example
× 1 2 3 41 1 2 3 42 2 4 1 33 3 1 4 24 4 3 2 1
Multiplication mod 5:Every row and column of the multiplication table is a permutation
Stepanov, McJones Elements of Programming March 14, 2008 414 / 880
Cycles in a permutation
DefinitionA cycle is a circular orbit within a permutation
DefinitionA trivial cycle is one with a cycle size of 1
The element in a trivial cycle is a fixed point
LemmaEvery element in a permutation of a finite set belongs to a unique cycle
Stepanov, McJones Elements of Programming March 14, 2008 415 / 880
Example of cycle decomposition
p(0) = 5p(1) = 2p(2) = 4p(3) = 3p(4) = 1p(5) = 0
p = (0 5)(1 2 4)(3)
Stepanov, McJones Elements of Programming March 14, 2008 416 / 880
Structure of cycle decomposition
LemmaAny permutation of a set with n elements contains k 6 n disjoint cyclesEach cycle is itself a permutation of this set, called a cyclic permutationDisjoint cyclic permutations commuteEvery permutation can be represented as a product of its cyclesThe inverse of a permutation is the product of the inverses of its cycles
Stepanov, McJones Elements of Programming March 14, 2008 417 / 880
Finite sets
DefinitionA finite set S of size n is a set for which there exists a pair of functions
chooseS : [0,n)→ S
indexS : S→ [0,n)
satisfying
chooseS(indexS(x)) = x
indexS(chooseS(i)) = i
Stepanov, McJones Elements of Programming March 14, 2008 418 / 880
Index permutation of a permutation
DefinitionIf p is a permutation on a finite set S of size n, there is a correspondingindex permutation p ′ on [0,n) defined as
p ′(i) = indexS(p(chooseS(i)))
Lemma
p(x) = chooseS(p′(indexS(x)))
We will frequently define permutations by the correspondingindex permutations
Stepanov, McJones Elements of Programming March 14, 2008 419 / 880
Rearrangement
DefinitionA rearrangement is an algorithm that rearranges the elements of amutable range to satisfy a given postcondition
Stepanov, McJones Elements of Programming March 14, 2008 420 / 880
Classifying rearrangements
Rearrangements are classified according to the following fundamentalcharacteristics:
Characterization of the postconditionPostcondition kindStability
Implementation constraintsIterator requirementsMutative versus copying
ComplexityTimeSpace
Stepanov, McJones Elements of Programming March 14, 2008 421 / 880
Postcondition kind
position-based The destination of a value depends only on its originalposition and not on the value itself; for example,“reverse the range”
bin-based The destination of a value depends only on the result ofapplying a k-way bin function to that value, and notdirectly on the value itself; for example, “move badvalues before good ones”
A useful subcase is predicate-based algorithms,where k = 2
ordering-based The destination of a value depends only on the outcomeof applications of an ordering to pairs of values in therange; for example, “put the smallest value first”
Stepanov, McJones Elements of Programming March 14, 2008 422 / 880
Stability
DefinitionA rearrangement is stable if it respects the original order of the range tothe maximal extent possible while satisfying the postcondition
ExamplesStable index partition Move all the elements at even positions in the
sequence before the elements at odd positions,keeping the original order in both groups
Stable partition Move all the elements with even values beforethose with odd values, keeping the originalorder in both groups
Stable sort Sort pairs of integers by their first componentsso pairs with equal first components remain inthe original order
Stepanov, McJones Elements of Programming March 14, 2008 423 / 880
Iterator requirements
For the same problem, there are often different algorithms fordifferent iterator requirements
Examplesreverse_forwardreverse_bidirectionalreverse_indexed
Stepanov, McJones Elements of Programming March 14, 2008 424 / 880
Mutative versus copying
DefinitionRearrangements as we have defined them are mutative
DefinitionA rearrangement is copying if it sets a writable range to a rearrangedcopy of a readable range in way that satisfies a given postcondition
A copying rearrangement could always be obtained by composingcopywith the corresponding mutative rearrangementOften, however, there are faster algorithms that rearrange whilecopying
Stepanov, McJones Elements of Programming March 14, 2008 425 / 880
Space complexity
DefinitionA mutative algorithm is in place (or in situ) if it uses an amount ofadditional space that is (poly-)logarithmic in the size of the input
DefinitionA memory-adaptive algorithm uses as much additional space as it canacquire to maximize performance
A small percentage of additional space, while theoretically“linear,” can lead to a large performance improvement
DefinitionAn algorithm with buffer requires the caller to provide a buffer to beused by the algorithm
Stepanov, McJones Elements of Programming March 14, 2008 426 / 880
Time complexity
Rearranging n elements might take n+ k assignments for someconstant kn logn algorithms arise
Determining the final position may require collecting additionalinformation, as in sortingAbsence of random access on some iterators requires multipassalgorithms, as with in-place random shuffle for forward iterators
Stepanov, McJones Elements of Programming March 14, 2008 427 / 880
cycle_2
template <typename I>requires(Mutable(I) && Iterator(I))
void cycle_2(I x, I y){
ValueType(I) t = source(x);sink(x) = source(y);sink(y) = t;
}
Stepanov, McJones Elements of Programming March 14, 2008 428 / 880
Problems with cycle_2
It could be much slower than necessarystd::vector copies element-by-element instead of interchangingthe headers
It could throw an exception due to unnecessary resource allocationWe should exploit the fact it is needed for implementing mutativerearrangements, which should not need to construct or destroyany objects, but just move them aroundWe will address the issue in Chapters 12 with a notion of anunderlying typeFor the time being, imagine that the value types of iterators arebuilt-in types or C-style structs
Stepanov, McJones Elements of Programming March 14, 2008 429 / 880
cycle_left_3
template <typename I>requires(Mutable(I) && Iterator(I))
void cycle_left_3(I x, I y, I z){
ValueType(I) t = source(x);sink(x) = source(y);sink(y) = source(z);sink(z) = t;
}
Stepanov, McJones Elements of Programming March 14, 2008 430 / 880
To-permutation and from-permutation
DefinitionEvery rearrangement corresponds to two permutations of itsrange
A to-permutation mapping an iterator i to the iterator pointing to thedestination of the element at iA from-permutation mapping an iterator i to the iterator pointing tothe origin of the element moved to i
These two permutations are inverses of each other
Stepanov, McJones Elements of Programming March 14, 2008 431 / 880
do_cycle_from
template <typename I, typename P>requires(Mutable(I) && ForwardIterator(I) &&
Transformation(P) && Domain(P) == I)void do_cycle_from(I i, P p){
// Precondition: p is a from-permutation of the range containing iValueType(I) t = source(i);I f = i;I n = p(i);while (n != i) {
sink(f) = source(n);f = n;n = p(n);
}sink(f) = t;
}
Stepanov, McJones Elements of Programming March 14, 2008 432 / 880
Strict lower bound for rearrangement
TheoremThe minimum number of assignments required for a rearrangement isn+ cN − cT , where n is the number of elements, cN the number ofnontrivial cycles, and cT the number of trivial cycles
Stepanov, McJones Elements of Programming March 14, 2008 433 / 880
Proof of strict lower bound for rearrangement
Consider a rearrangement realizing a permutation p, and let C bea cycle of p of length > 1The rearrangement must include an assignment of a value of C toeach memory location of CSince such an assignment overwrites the value at that location,there must be an additional assignment of some value of C to alocation outside CSince the cycles are disjoint, these additional assignments aredistinct, so the total number of assignments must be at leastn+ cN − cT , where n is the number of elements of p, cN thenumber of nontrivial cycles, and cT the number of trivial cycles29
do_cycle_from (applied to nontrivial cycles) make this boundstrict
29The idea for the proof was provided to us by John WilkinsonStepanov, McJones Elements of Programming March 14, 2008 434 / 880
do_cycle_to
ExerciseImplement do_cycle_to and compare the number of assignments itperforms to do_cycle_from
Stepanov, McJones Elements of Programming March 14, 2008 435 / 880
Reverse permutation
DefinitionThe permutation p on a finite set with n elements defined by an indexpermutation p(i) = (n− 1)− i is called the reverse permutation of the set
The number of nontrivial cycles in a reverse permutation is bn/2c;the number of trivial cycles is n mod 2bn/2c is the largest possible number of nontrivial cycles in apermutation
Stepanov, McJones Elements of Programming March 14, 2008 436 / 880
Reverse rearrangement
DefinitionThe reverse rearrangement of a range is the rearrangement induced bythe reverse permutation
The lower bound formula gives the number of assignments asn+ cN − cT = 3bn/2c
Stepanov, McJones Elements of Programming March 14, 2008 437 / 880
reverse_n_indexed
The definition of reverse directly gives this:
template <typename I>requires(Mutable(I) && IndexedIterator(I))
void reverse_n_indexed(I f, DistanceType(I) n){
typedef DistanceType(I) N;N i(0);while (i < n / N(2)) {
cycle_2(f + i, f + ((n - N(1)) - i));i = i + N(1);
};}
The code does the lower bound number of assignmentsIf the algorithm is used with forward or bidirectional iterators, itperforms a quadratic number of iterator incrementsSince all the cycles are disjoint, this code benefits from forall
Stepanov, McJones Elements of Programming March 14, 2008 438 / 880
Return value of reverse
It is tempting to define the reverse algorithms to return the rangeof elements that were not moved
The middle element when the size of the range is oddThe empty range between the two “middle” elements when thesize of the range is even
However, we do not know of any example when it is useful and,therefore, return void
Stepanov, McJones Elements of Programming March 14, 2008 439 / 880
reverse_bidirectional
template <typename I>requires(Mutable(I) && BidirectionalIterator(I))
void reverse_bidirectional(I f, I l){
while (f != l && f != predecessor(l)) {--l;cycle_2(f, l);++f;
}}
Stepanov, McJones Elements of Programming March 14, 2008 440 / 880
reverse_n_bidirectional
template <typename I>requires(Mutable(I) && BidirectionalIterator(I))
void reverse_n_bidirectional(I f, I l, DistanceType(I) n){
// Precondition: n 6 l− ftypedef DistanceType(I) N;N i(0);while (i < n / N(2)) {
--l;cycle_2(f, l);++f;i = i + N(1);
}}
Passing n < l− f effectively gives reverse_until_n
Stepanov, McJones Elements of Programming March 14, 2008 441 / 880
reverse_indexed
template <typename I>requires(Mutable(I) && IndexedIterator(I))
void reverse_indexed(I f, I l){
reverse_n_indexed(f, l - f);}
Stepanov, McJones Elements of Programming March 14, 2008 442 / 880
Intuition for divide and conquer reverse algorithm
1 Split the range into two parts2 Reverse each part3 Interchange the parts
Stepanov, McJones Elements of Programming March 14, 2008 443 / 880
Illustration of divide and conquer reverse algorithm
[a] b [c] d [e] f [g] [h] i [j] k [l] m [n]
[c b a] d [g f e] [j i h] k [n m l]
[g f e d c b a] [n m l k j i h]
n m l k j i h g f e d c b a
Stepanov, McJones Elements of Programming March 14, 2008 444 / 880
swap_ranges_n
template <typename I1, typename I2, typename N>requires(Mutable(I1) && Iterator(I1) &&
Mutable(I2) && Iterator(I2) &&ValueType(I1) == ValueType(I2) &&Integer(N))
pair<I1, I2> swap_ranges_n(I1 f1, I2 f2, N n){
while (n != N(0)) {cycle_2(f1, f2);++f1;++f2;n = n - N(1);
};return pair<I1, I2>(f1, f2);
}
Stepanov, McJones Elements of Programming March 14, 2008 445 / 880
reverse_n_recursive
template <typename I>requires(Mutable(I) && ForwardIterator(I))
I reverse_n_recursive(I f, DistanceType(I) n){
typedef DistanceType(I) N;const N h = n / N(2);const N r = n - N(2) * h;if (h == N(0))
return f + n;I m = reverse_n_recursive(f, h);m += r;I l = reverse_n_recursive(m, h);swap_ranges_n(f, m, h);return l;
}
Stepanov, McJones Elements of Programming March 14, 2008 446 / 880
Correctness of reverse_n_recursive
LemmaThe reverse permutation on [0,n) is the only permutation satisfyingi < j⇒ p(j) < p(i)
1 This condition obviously holds for ranges of size 12 The recursive calls inductively establish that the condition holds
within each half3 swap_ranges_n reestablishes the condition between the halves
(and the skipped middle element, if any)
Stepanov, McJones Elements of Programming March 14, 2008 447 / 880
Complexity of reverse_n_recursive
Lemma
For a range of length n =∑blognci=0 ai2i, where ai is the ith digit in the
binary representation of n, the number of assignments equals32∑blognci=0 aii2i
Stepanov, McJones Elements of Programming March 14, 2008 448 / 880
reverse_n_forward
template <typename I>requires(Mutable(I) && ForwardIterator(I))
void reverse_n_forward(I f, DistanceType(I) n){
reverse_n_recursive(f, n);}
Stepanov, McJones Elements of Programming March 14, 2008 449 / 880
reverse_forward
template <typename I>requires(Mutable(I) && ForwardIterator(I))
void reverse_forward(I f, I l){
reverse_n_forward(f, l - f);}
Stepanov, McJones Elements of Programming March 14, 2008 450 / 880
reverse_copy_n
template <typename B, typename I>requires(Mutable(B) && BidirectionalIterator(B) &&
Mutable(I) && Iterator(I) &&ValueType(B) == ValueType(I))
I reverse_copy_n(B l, DistanceType(I) n, I r){
typedef DistanceType(I) N;while (n != N(0)) {
--l;sink(r) = source(l);++r;n = n - N(1);
}return r;
}
Stepanov, McJones Elements of Programming March 14, 2008 451 / 880
reverse_n_with_buffer
template <typename I, typename B>requires(Mutable(I) && ForwardIterator(I) &&
Mutable(B) && BidirectionalIterator(B) &&ValueType(I) == ValueType(B))
I reverse_n_with_buffer(I f, DistanceType(I) n, B b){
return reverse_copy_n(copy_n(f, n, b), n, f);}
Stepanov, McJones Elements of Programming March 14, 2008 452 / 880
reverse_n_adaptive
template <typename I, typename B>requires(Mutable(I) && ForwardIterator(I) &&
Mutable(B) && BidirectionalIterator(B) &&ValueType(I) == ValueType(B))
I reverse_n_adaptive(I f, DistanceType(I) n, B b, DistanceType(I) n_b){
typedef DistanceType(I) N;const N h = n / N(2);const N r = n - N(2) * h;if (h == N(0))
return f + n;if (n <= n_b)
return reverse_n_with_buffer(f, n, b);I m = reverse_n_adaptive(f, h, b, n_b);m += r;I l = reverse_n_adaptive(m, h, b, n_b);swap_ranges_n(f, m, h);return l;
}
Stepanov, McJones Elements of Programming March 14, 2008 453 / 880
Complexity of reverse_n_adaptive
ExerciseDerive a formula for the number of assignments performed byreverse_n_adaptive for given range and buffer sizes
Stepanov, McJones Elements of Programming March 14, 2008 454 / 880
Conjecture
ConjectureThere is no algorithm that reverses a forward iterator range withpolylogarithmic additional space and in linear time
Stepanov, McJones Elements of Programming March 14, 2008 455 / 880
Selecting algorithms according to requirements
It is important to be able to automatically select from a family ofalgorithms based on the requirements the types satisfyIn Appendix 2 we show a mechanism called category dispatch fordoing such selection in C++Future versions of C++may provide a more natural way ofexpressing it
Stepanov, McJones Elements of Programming March 14, 2008 456 / 880
Conclusions
Permutations lead us back to algebra, specifically group theoryA permutation of a range induces a rearrangementA taxonomy of rearrangements was presentedWeakening iterator requirements often raises interestingalgorithmic problemsA convention known as iterator category dispatch allowsautomatic selection of the appropriate algorithmWe introduced a new class of algorithms, memory-adaptivealgorithms
Stepanov, McJones Elements of Programming March 14, 2008 457 / 880
Reading
Donald Knuth.Section 1.2.5: Permutations and Factorials.The Art of Computer Programming, Volume 1, Addison-Wesley, 1997,pages 45-50.
Stepanov, McJones Elements of Programming March 14, 2008 458 / 880
Contents I
1 Introduction
2 Foundations
3 Transformations and their orbits
4 Algorithms on algebraic structures
5 Orderings
6 Combining concepts
7 Refining concepts of iterators
Stepanov, McJones Elements of Programming March 14, 2008 459 / 880
Contents II
8 Permutations and rearrangements
9 RotationsRotation permutationsLeast common multipleCycle structure of rotationsInterface for rotaterotate for indexed iteratorsLoop fusionrotate for bidirectional iteratorsrotate for forward iteratorsConclusionsReadingProjects
Stepanov, McJones Elements of Programming March 14, 2008 460 / 880
Contents III
10 Algorithms on increasing ranges
11 Coordinate structures
12 Composite objects
13 Iterative algorithms for divide-and-conquer
14 Mathematical notation
15 C++ machinery
16 Acknowledgments
Stepanov, McJones Elements of Programming March 14, 2008 461 / 880
Contents IV
17 Index
Stepanov, McJones Elements of Programming March 14, 2008 462 / 880
Definition of rotation
DefinitionThe permutation p on a finite set with n elements defined by an indexpermutation p(i) = (i+ k) mod n is called the k-rotation of the set
LemmaThe inverse of a k-rotation on an n-element set is an (n− k)-rotation
Stepanov, McJones Elements of Programming March 14, 2008 463 / 880
Least common multiple
DefinitionThe least common multiple of integers a and b, written lcm(a,b), is thesmallest integerm such that both a and b divide it
Lemmalcm(a,b) is a multiple of a and b that divides any other multiple of aand b
Stepanov, McJones Elements of Programming March 14, 2008 464 / 880
Connecting lcm and gcd
Theorema · b = lcm(a,b) · gcd(a,b)
Proof.If the prime decompositions of a and b are respectively
∏puii and∏
pvii , then a · b =
∏pui+vii , lcm(a,b) =
∏p
max(ui,vi)i , and
gcd(a,b) =∏p
min(ui,vi)i , and the result follows from the useful
identity x+ y = max(x,y) + min(x,y) �
Stepanov, McJones Elements of Programming March 14, 2008 465 / 880
(Divisibility lattice of integers)
DefinitionAn ordered set E is said to be a lattice if every subset consisting of twoelements of E has a least upper bound and a greatest lower bound in E
ExampleThe set of integers > 1, ordered by the relation “m divides n” betweenm and n, is a lattice; the least upper bound of {m,n} is lcm(m,n), andthe greatest lower bound is gcd(m,n)
The definition and example are adapted from:
N. Bourbaki.Chapter 3, Section 1.11.Theory of Sets, Springer, 2004, pages 145-146.
Stepanov, McJones Elements of Programming March 14, 2008 466 / 880
Cycle structure of k-rotation on n elements
An element with index i is in the cycle
{i, (i+ k) mod n, (i+ 2k) mod n, . . .} = {(i+ uk) mod n}
The length of the cycle is the smallest positive integerm such thati = (i+mk) mod nThis is equivalent tomk mod n = 0, which shows the length ofthe cycle to be independent of iSincem is the smallest positive number such thatmk mod n = 0,it is obvious that lcm(k,n) = mk
m =lcm(k,n)k = kn
gcd(k,n)k = ngcd(k,n)
The number of cycles, therefore, is gcd(k,n)
Stepanov, McJones Elements of Programming March 14, 2008 467 / 880
Disjoint cycles of k-rotation on n elements
Consider two different elements in a cycle, (i+ uk) mod n and(i+ vk) mod nThe distance between them is|(i+uk) mod n−(i+vk) mod n| = (u−v)k mod n = (u−v)k−pn
where p = quotient((u− v)k,n)
Since both k and n are divisible by d = gcd(k,n), so is the distanceTherefore the distance between different elements in the samecycle is > d
Elements with indices in [0,d) belong to disjoint cycles
Stepanov, McJones Elements of Programming March 14, 2008 468 / 880
Interface for rotate
k-rotation rearrangement of a range [f, l) is equivalent tointerchanging the relative positions of [f,m) and [m, l), wherem = f+ ((l− f) − k) = l− k
Experimentally,m is a more useful input than k (and whenforward or bidirectional iterators are involved, it avoidsperforming linear-time operations to computem from k)Experimentally, returning the iteratorm ′ = f+ k pointing to thenew position of element at f is useful for many other algorithms30
30Joseph Tighe suggests returning a pair,m andm ′, in the order constituting a validrange; while it is an interesting suggestion and preserves all the information, we donot yet know of a compelling use of such interface
Stepanov, McJones Elements of Programming March 14, 2008 469 / 880
From-permutation for k-rotation in terms of f,m, l
To-permutation: p(i) = (i+ k) mod nFrom-permutation: p−1(i) = (i+ (n− k)) mod nn = l− f
Sincem goes to f,m+ k = f+ n⇒ n− k = m− f
k = n− (m− f) = l−m
i < k⇒ i+ (n− k) < n⇒ p−1(i) = i+ (n− k)
i > k⇒ p−1(i) = i+ (n− k) − n = i− k
p−1(i) =
{i+ (n− k) if i < ki− k if i > k
Stepanov, McJones Elements of Programming March 14, 2008 470 / 880
A from-permutation for RandomAccessIterator
template <typename I>requires(Mutable(I) && RandomAccessIterator(I))
struct k_rotate_from_permutation_random_access{
DistanceType(I) k;DistanceType(I) n_minus_k;I m_prime;k_rotate_from_permutation_random_access(I f, I m, I l) :
k(l - m), n_minus_k(m - f), m_prime(f + (l - m)) {}I operator()(I x) {
if (x < m_prime)return x + n_minus_k;
elsereturn x - k;
}};
Stepanov, McJones Elements of Programming March 14, 2008 471 / 880
A from-permutation for IndexedIterator
The absence of < and − for indexed iterators costs us an extraoperation or two (+ and −)On modern processors this may not cost any time
template <typename I>requires(Mutable(I) && IndexedIterator(I))
struct k_rotate_from_permutation_indexed{
DistanceType(I) k;DistanceType(I) n_minus_k;I f;k_rotate_from_permutation_indexed(I f, I m, I l) :
k(l - m), n_minus_k(m - f), f(f) {}I operator()(I x) {
DistanceType(I) i = x - f;if (i < k)
return x + n_minus_k;else
return f + (i - k);}
};
Stepanov, McJones Elements of Programming March 14, 2008 472 / 880
rotate_indexed_helper
template <typename I, typename P>requires(Mutable(I) && IndexedIterator(I) &&
Transformation(P) && Domain(P) == I)I rotate_indexed_helper(I f, I m, I l, P p){
// Precondition: p is a from-permutation on [f, l)typedef DistanceType(I) N;N d = gcd(m - f, l - m);N i = N(0);while (i < d) {
do_cycle_from(f + i, p);++i;
}return f + (l - m);
};
This algorithm was first published by:
William Fletcher and Roland Silver.Algorithm 284: Interchange of Two Blocks of Data.CACM, Volume 9, Number 5, May 1966, page 326.
Stepanov, McJones Elements of Programming March 14, 2008 473 / 880
rotate_indexed and rotate_random_access
template <typename I> requires(Mutable(I) && IndexedIterator(I))I rotate(I f, I m, I l, indexed_iterator_tag){
k_rotate_from_permutation_indexed<I> p(f, m, l);return rotate_indexed_helper(f, m, l, p);
};
template <typename I> requires(Mutable(I) && RandomAccessIterator(I))I rotate(I f, I m, I l, random_access_iterator_tag){
k_rotate_from_permutation_random_access<I> p(f, m, l);return rotate_indexed_helper(f, m, l, p);
};
template <typename I> requires(Mutable(I) && ForwardIterator(I))I rotate(I f, I m, I l){
if (m == f) return l;if (m == l) return f;return rotate(f, m, l, IteratorCategory(I)());
}
Stepanov, McJones Elements of Programming March 14, 2008 474 / 880
Complexity of rotate for IndexedIterator
The number of assignments is n+ c− t = n+ gcd(n,k)On average, gcd� n
Stepanov, McJones Elements of Programming March 14, 2008 475 / 880
Loop fusion
Assuming uniform distribution of rotation points, the expectedvalue of k = n/2That gives very bad locality of referenceSince the cycles are disjoint, we can improve locality of referenceby operating on adjacent elements in different cyclesThis technique is called loop fusion, and was first discussed in thispaper:
A.P. Yershov.ALPHA–An Automatic Programming System of HighEfficiency.J. ACM, Volume 13, Number 1, January 1966, pages 17-24.
Stepanov, McJones Elements of Programming March 14, 2008 476 / 880
do_fused_cycles_from_with_buffer
template <typename I, typename P, typename B>requires(Mutable(I) && IndexedIterator(I) &&
Transformation(P) && Domain(P) == I &&Mutable(B) && IndexedIterator(B) &&ValueType(I) == ValueType(B))
void do_fused_cycles_from_with_buffer(I i, P p, B b, DistanceType(I) n){
// Precondition: p is a from-permutation on the range containing icopy_n(i, b, n);I f = i;I next = p(i);while (next != i) {
copy_n(next, f, n);f = next;next = p(next);
}copy_n(b, f, n);
}
Stepanov, McJones Elements of Programming March 14, 2008 477 / 880
rotate_indexed_helper_fused
template <typename I, typename P>requires(Mutable(I) && IndexedIterator(I) &&
Transformation(P) && Domain(P) == I)I rotate_indexed_helper_fused(I f, I m, I l, P p){
// Precondition: p is a from-permutation on [f, l)typedef DistanceType(I) N;const N fusion_factor(16);ValueType(I) buffer[fusion_factor];N d = gcd(m - f, l - m);N i(0);while (i + fusion_factor < d) {
do_fused_cycles_from_with_buffer(f + i, p, buffer, fusion_factor);i = i + fusion_factor;
}do_fused_cycles_from_with_buffer(f + i, p, buffer, d - i);return f + (l - m);
};
Stepanov, McJones Elements of Programming March 14, 2008 478 / 880
Issues with fused rotate
Loop fusion is an important optimization technique thatprogrammers need to knowIn this particular case, it is not that beneficial since often there arenot too many loops to fuse
gcd(n,k) = 1 with probability ≈ 60%
There are algorithms that do more more assignments but havegood locality of reference and work for weaker iteratorrequirements
Stepanov, McJones Elements of Programming March 14, 2008 479 / 880
Connection of rotate and reverse
LemmaThe k-rotation on [0,n) is the only permutation p such that
1 i < n− k∧ n− k 6 j < n⇒ p(j) < p(i)
2 i < j < n− k∨ n− k 6 i < j⇒ p(i) < p(j)
The reverse permutation will satisfy condition 1, but not 2Applying reverse to subranges [0,n− k) and [n− k,n) and thenapplying reverse to the entire range will satisfy both conditions
Stepanov, McJones Elements of Programming March 14, 2008 480 / 880
rotate_three_reverses
This insight gives us the following:
template <typename I>requires(Mutable(I) && BidirectionalIterator(I))
void rotate_three_reverses(I f, I m, I l){
reverse(f, m);reverse(m, l);reverse(f, l);
}
The only difficulty is finding the return value,m ′
Stepanov, McJones Elements of Programming March 14, 2008 481 / 880
reverse_until
The following auxiliary function allows us to find the return valuewithout doing any extra work31
template <typename I>requires(Mutable(I) && BidirectionalIterator(I))
pair<I, I> reverse_until(I f, I m, I l){
while (f != m && m != l) {--l;cycle_2(f, l);++f;
}return pair<I, I>(f, l);
}
31It was suggested to us by Raymond Lo and Wilson HoStepanov, McJones Elements of Programming March 14, 2008 482 / 880
rotate for BidirectionalIterator
template <typename I>requires(Mutable(I) && BidirectionalIterator(I))
I rotate(I f, I m, I l, bidirectional_iterator_tag){
reverse(f, m);reverse(m, l);pair<I, I> p = reverse_until(f, m, l);reverse(p.first, p.second);if (m == p.first)
return p.second;else
return p.first;}
Stepanov, McJones Elements of Programming March 14, 2008 483 / 880
Complexity of rotate for BidirectionalIterator
LemmaThe number of assignments is 3(bn/2c+ bk/2c+ b(n− k)/2c), whichgives us 3nwhen n and k are both even and 3(n− 2) in every othercase
Stepanov, McJones Elements of Programming March 14, 2008 484 / 880
swap_ranges
template <typename I1, typename I2>requires(Mutable(I1) && Iterator(I1) &&
Mutable(I2) && Iterator(I2) &&ValueType(I1) == ValueType(I2))
pair<I1, I2> swap_ranges(I1 f1, I1 l1, I2 f2, I2 l2){
while (f1 != l1 && f2 != l2) {cycle_2(f1, f2);++f1;++f2;
};return pair<I1, I2>(f1, f2);
}
Form ∈ [f, l) such that l−m = m− f, swap_ranges(f,m,m, l)rotates [f, l) aboutm
Stepanov, McJones Elements of Programming March 14, 2008 485 / 880
Intuition for ForwardIterator rotate algorithm
m = m ′ swap_ranges is sufficientm ′ < m after swap_ranges, [f,m ′) are in the final position; we
need to rotate [m ′, l) aroundmm < m ′ after swap_ranges, [f,m) are in the final position; we
need to rotate [m, l) aroundm ′
This algorithm was first published by:
David Gries and Harlan Mills.Swapping Sections.Technical Report 81-452, Department of Computer Science, CornellUniversity, January 1981.
Stepanov, McJones Elements of Programming March 14, 2008 486 / 880
rotate_0
template <typename I>requires(Mutable(I) && ForwardIterator(I))
void rotate_0(I f, I m, I l){
I c = m;while (true) {
pair<I, I> p = swap_ranges(f, m, m, l);if (p.first == m && p.second == l)
return;if (p.first == m) {
m = p.second;f = p.first;
} else {assert(p.second == l);f = p.first;
}}
}
Stepanov, McJones Elements of Programming March 14, 2008 487 / 880
Annotated rotate_0
template <typename I>requires(Mutable(I) && ForwardIterator(I))
void rotate_0_annotated(I f, I m, I l){
DistanceType(I) u = m - f;DistanceType(I) v = l - m;
I c = m;while (true) {
pair<I, I> p = swap_ranges(f, m, m, l);if (p.first == m && p.second == l) { assert(u == v);
return;} else if (p.first == m) { assert(v > u);
m = p.second;f = p.first; v = v - u;
} else {assert(p.second == l); assert(u > v);f = p.first; u = u - v;
}}
}
u and v compute subtractive gcd of the initial lengths
Stepanov, McJones Elements of Programming March 14, 2008 488 / 880
Complexity of rotate for ForwardIterator
LemmaThe number of assignments is 3(n− gcd(n,k))
Stepanov, McJones Elements of Programming March 14, 2008 489 / 880
rotate_unguarded
Inlining swap_ranges and small optimizations give:
template <typename I>requires(Mutable(I) && ForwardIterator(I))
void rotate_unguarded(I f, I m, I l){
// Precondition: f ,m∧m , lI c = m;while (c != l) {
cycle_2(f, c);++f;++c;if (f == m)
m = c;if (c == l)
c = m;}
}
The first time the second if clause is satisfied, f equalsm ′
Stepanov, McJones Elements of Programming March 14, 2008 490 / 880
rotate for ForwardIterator
template <typename I>requires(Mutable(I) && ForwardIterator(I))
I rotate(I f, I m, I l, forward_iterator_tag){
// Precondition: f ,m∧m , lI m_prime = l;I c = m;while (c != l) {
cycle_2(f, c);++f;++c;if (f == m)
m = c;if (c == l) {
if (m_prime == l) m_prime = f;c = m;
}}return m_prime;
}
The guards in the category dispatch rotate guarantee the assertion
Stepanov, McJones Elements of Programming March 14, 2008 491 / 880
Conclusions
Rotation rearrangements allow interchanging the order ofadjacent ranges of different sizesApplying elementary number theory allows us to determine thecycle structure of rotationsInteresting algorithms exist for forward, bidirectional, andindexed iterator algorithmsComplexity is often dominated by locality of reference concernsrather than minimizing operation counts
Stepanov, McJones Elements of Programming March 14, 2008 492 / 880
Reading
Donald Knuth.Section 1.3.3: Permutations and Factorials.The Art of Computer Programming, Volume 1, Addison-Wesley, 1997,pages 164-185.In particular, see problems 34 and 35.
Stepanov, McJones Elements of Programming March 14, 2008 493 / 880
Project 1: industrial-strength rotate
Theoretically rotate for indexed iterators is much faster than theother two algorithms because it performs approximately 1
3 asmany assignments, but its locality of reference is poorDesign a benchmark comparing performance of all threealgorithms for different array sizes, element sizes, and rotationamountsBased on the results of the benchmark, design a compositealgorithm that appropriately uses one of the three algorithmsdepending on its inputs
Stepanov, McJones Elements of Programming March 14, 2008 494 / 880
Project 2: Research in position-based rearrangements
We have presented two kinds of position-based rearrangementalgorithms: reverse and rotateThere are, however, many other examples of such algorithms; youcan find some of them in the section of Knuth given in Reading aswell as in the Collected Algorithms of the ACM (see, for example,numbers 302, 380, and 467)Develop a taxonomy of position-based rearrangements, discoveradditional algorithms, and produce a library
Stepanov, McJones Elements of Programming March 14, 2008 495 / 880
Contents I
1 Introduction
2 Foundations
3 Transformations and their orbits
4 Algorithms on algebraic structures
5 Orderings
6 Combining concepts
7 Refining concepts of iterators
Stepanov, McJones Elements of Programming March 14, 2008 496 / 880
Contents II
8 Permutations and rearrangements
9 Rotations
10 Algorithms on increasing rangesIncreasing rangesPartitionSearchingCopying merging and sortingIn-place merging and sortingAdaptive merging and sortingConclusionsProject
11 Coordinate structures
Stepanov, McJones Elements of Programming March 14, 2008 497 / 880
Contents III
12 Composite objects
13 Iterative algorithms for divide-and-conquer
14 Mathematical notation
15 C++ machinery
16 Acknowledgments
17 Index
Stepanov, McJones Elements of Programming March 14, 2008 498 / 880
Increasing ranges
DefinitionA range [f, l) is increasing if for every non-last iteratorm ∈ [f, l),¬(source(successor(m)) < source(m))
A range [f, l) is increasing with respect to a strict weak ordering r iffor every non-last iteratorm ∈ [f, l),¬r(source(successor(m)), source(m))
A range [f, l) is key-increasing a with respect to a regular functionkey and a strict weak ordering r if for every non-last iteratorm ∈ [f, l), ¬r(key(source(successor(m))),key(source(k)))
aIn database parlance, a key is a part of a record on which sorting is done
Stepanov, McJones Elements of Programming March 14, 2008 499 / 880
Defaulting the key function
Note that an increasing range is a key-increasing range with thekey function being the identity function on the value type of theiterator
Stepanov, McJones Elements of Programming March 14, 2008 500 / 880
find_out_of_order
template <typename I, typename F, typename R>requires(Readable(I) && ForwardIterator(I) &&
RegularFunction(F) && Domain(F) == ValueType(I) &&StrictWeakOrdering(R) && Domain(R) == Codomain(F))
I find_out_of_order(I f, I l, F key, R r, forward_iterator_tag){
if (f == l) return l;I n(successor(f));while (n != l) {
if (r(key(source(n)), key(source(f))))return n;
f = n;++n;
}return l;
}
To adapt the algorithm for Iterator, we need to copy the valuebefore we advance
Stepanov, McJones Elements of Programming March 14, 2008 501 / 880
find_out_of_order
template <typename I, typename F, typename R>requires(Readable(I) && Iterator(I) &&
RegularFunction(F) && Domain(F) == ValueType(I) &&StrictWeakOrdering(R) && Domain(R) == Codomain(F))
I find_out_of_order(I f, I l, F key, R r, iterator_tag){
if (f == l) return l;I n(successor(f));Codomain(F) v = key(source(f));while (n != l) {
Codomain(F) u = key(source(n));if (r(u, v))
return n;v = u;++n;
}return l;
}
Stepanov, McJones Elements of Programming March 14, 2008 502 / 880
is_increasing
template <typename I, typename F, typename R>requires(Readable(I) && Iterator(I) &&
RegularFunction(F) && Domain(F) == ValueType(I) &&StrictWeakOrdering(R) && Domain(R) == Codomain(F))
bool is_increasing(I f, I l, F key, R r){
return l == find_out_of_order(f, l, key, r);}
Stepanov, McJones Elements of Programming March 14, 2008 503 / 880
Partition
DefinitionUnaryPredicate(P)⇒
Predicate(P)Arity(P) = 1
DefinitionA range is partitioned if it is key-increasing with respect to a keyfunction that is a unary predicateMore generally, a range is k-partitioned if it is key-increasing withrespect to a key function that returns one of k distinct values forsome (compile-time) constant integer k > 0
There exist faster specialized versions of many algorithms dealingwith key-increasing ranges when the range of the key function is abounded set of integersStepanov, McJones Elements of Programming March 14, 2008 504 / 880
Partition points
DefinitionA partition point in a range [f, l) partitioned by a unary predicate p is aniterator i ∈ [f, l] such that ¬p(source(j)) for all j ∈ [f, i) andp(source(k)) for all k ∈ [i, l)
Informally, a partition point is the first position where thepredicate is satisfied, if such a position exists, or the limit of therange otherwise
Stepanov, McJones Elements of Programming March 14, 2008 505 / 880
Lower bound and upper bound
DefinitionIn an increasing range [f, l), every value a of the value type of therange determines two partition points, lower bound and upper bound,defined by, respectively:
Pa(x) ≡ x < a
P ′a(x) ≡ ¬(a < x)
It is trivial to provide similar definitions for key-increasingsequences and sequences with respect to a strict weak order r
Lemmalower bound � upper bound
Stepanov, McJones Elements of Programming March 14, 2008 506 / 880
Intuition for lower bound and upper bound
Informally, a lower bound is the first position where a value equalto a could occur in the increasing sequenceSimilarly, an upper bound is the successor of the last positionwhere a value equal to a could occurTherefore, elements equal to a appear only in the semi-open rangefrom lower bound to upper bound
A sequence with lower bound i and upper bound j for the value a(note any of the three regions may be empty):
x0, x1, . . . , xi−1︸ ︷︷ ︸<a
, xi, . . . , xj−1︸ ︷︷ ︸=a
, xj, xj+1, . . . , xn−1︸ ︷︷ ︸a<
Stepanov, McJones Elements of Programming March 14, 2008 507 / 880
Partition algorithms
Mathematically a partitioned range is simply a key-increasingrangeAlgorithmically, it is worth providing special-case algorithms fordealing with partitions
Testing whether a range is partitioned dereferences half as manyiterators as testing whether it is increasingPartitioning a range of size n is linear while sorting it is n log2 n
Efficient algorithms for dealing with increasing ranges can becomposed from algorithms dealing with partitioned ranges
Quicksort is implemented using a specialized partition algorithmFinding a lower or upper bound uses the algorithm for finding apartition point
Stepanov, McJones Elements of Programming March 14, 2008 508 / 880
Testing whether a range is partitioned
LemmaGiven any range and a unary predicate, some prefix of the range ispartitioned by the predicate
If we can determine the longest partitioned prefix of a range, thenby testing whether the prefix equals the range we can determinewhether the range is partitioned
Stepanov, McJones Elements of Programming March 14, 2008 509 / 880
find_if and find_if_not
We need to find the first element satisfying the predicate
template <typename I, typename P>requires(Readable(I) && Iterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))I find_if(I f, I l, P p) {
while (f != l && !p(source(f)))++f;
return f;}
Then finding the first element not satisfying the predicate givesthe longest partitioned prefix
template <typename I, typename P>requires(Readable(I) && Iterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))I find_if_not(I f, I l, P p) {
while (f != l && p(source(f)))++f;
return f;}
Stepanov, McJones Elements of Programming March 14, 2008 510 / 880
is_partitioned
template <typename I, typename P>requires(Readable(I) && Iterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))pair<I, I> find_partitioned_prefix(I f, I l, P p){
I i = find_if(f, l, p);I j = find_if_not(i, l, p);return pair<I, I>(i, j);
}
The first value returned is the partition point of the partitionedprefix
template <typename I, typename P>requires(Readable(I) && Iterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))bool is_partitioned(I f, I l, P p){
return l == find_partitioned_prefix(f, l, p).second;}
Stepanov, McJones Elements of Programming March 14, 2008 511 / 880
Quantifier functions
template <typename I, typename P>requires(Readable(I) && Iterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))bool all(I f, I l, P p) { return l == find_if_not(f, l, p); }
template <typename I, typename P>requires(Readable(I) && Iterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))bool none(I f, I l, P p) { return l == find_if(f, l, p); }
template <typename I, typename P>requires(Readable(I) && Iterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))bool not_all(I f, I l, P p) { return !all(f, l, p); }
template <typename I, typename P>requires(Readable(I) && Iterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))bool some(I f, I l, P p) { return !none(f, l, p); }
Stepanov, McJones Elements of Programming March 14, 2008 512 / 880
is_partition_point
template <typename I, typename P>requires(Readable(I) && Iterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))bool is_partition_point(I f, I m, I l, P p){
return none(f, m, p) && all(m, l, p);}
Stepanov, McJones Elements of Programming March 14, 2008 513 / 880
Importance of finding a partition point
Finding the partition point in an already-partitioned range byitself is not that usefulOn an increasing range, it can be combined with an appropriatepredicate to implement all the varieties of binary search –allowing us to find a value without examining every position
Stepanov, McJones Elements of Programming March 14, 2008 514 / 880
Intuition for finding a partition point
Suppose we evaluate a predicate p at some pointm in apartitioned range [f, l)
If p(m) is false, we know the partition point of [f, l) is greater thanm, and in fact is equal to the partition point of [successor(m), l)If p(m) is true, we know the partition point is not greater thanm, soit is equal to the partition point of [f,m)
Note the partition point of [f,m) will equalm if the predicate p isfalse for every iterator in [f,m)
LemmaChoosingm in the middle of the range assures optimal worst-caseperformance
If we have prior knowledge of the distribution, biasing the choiceofmmight give better average performance
Stepanov, McJones Elements of Programming March 14, 2008 515 / 880
partition_point_n
template <typename I, typename P>requires(Readable(I) && ForwardIterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))I partition_point_n(I f, DistanceType(I) n, P p){
typedef DistanceType(I) N;while (n != N(0)) {
N h = half_nonnegative(n);I m = f + h;if (!p(source(m))) {
n = n - successor(h); f = successor(m);} else
n = h;}return f;
}
template <typename I, typename P>requires(Readable(I) && ForwardIterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))I partition_point(I f, I l, P p) { return partition_point_n(f, l - f, p); }
Stepanov, McJones Elements of Programming March 14, 2008 516 / 880
Complexity of partition_point
partition_point_n applies the predicate blog2 nc+ 1 times sincethe length of the range is reduced by a factor of 2 at each stepIt also performs a logarithmic number of iterator/integeradditions, which for an indexed or random access iterator areconstant-time but which for a forward or bidirectional iterator(singly- or doubly-linked list) require n iterator incrementspartition_point adds another n iterator increments for forwardand bidirectional iteratorsIt may still be worthwhile to use it on linked lists when thepredicate application is much more expensive than an iteratorincrement
Stepanov, McJones Elements of Programming March 14, 2008 517 / 880
Origin of the bisection technique
The bisection technique underlying partition point dates back atleast as far as the proof of the Intermediate Value Theoremdiscovered by Bernard Bolzano32 and, independently, byAugustin-Louis Cauchy about 4 years later33
The proof of the Bolzano-Cauchy theorem leads us to thefollowing interpolation algorithm34
32Bernard Bolzano. Rein analytischer Beweis des Lehrsatzes, daß zwischen je zweyWerthen, die ein entgegengesetztes Resultat gewähren, wenigstens eine reelle Wurzel derGleichung liege. 1817
33A.-L. Cauchy. Cours D’Analyse de L’Ecole Royale Polytechnique. 182134This book explores the connections between some simple algebraic concepts and
algorithms; we strongly believe it is also possible to define algorithms on analyticconcepts
Stepanov, McJones Elements of Programming March 14, 2008 518 / 880
bisection_interpolation
template <typename F>requires(Transformation(F) && ArchimedeanOrderedField(Domain(F)))
Domain(F) bisection_interpolation(F f, Domain(F) x0, Domain(F) x1, Domain(F) y0, Domain(F) e)
{// Precondition: f is continuous and increasing on [x0,x1]typedef Domain(F) R;R x;while (true) {
x = (x0 + x1) / R(2);R y = f(x);if (y <= y0) {
x0 = x; if (y0 < y + e) break;} else {
x1 = x; if (y < y0 + e) break;}
}return x;// Postcondition: x is in the original interval [x0,x1] and |f(x) − y0| < ε
}
Stepanov, McJones Elements of Programming March 14, 2008 519 / 880
count_if and count_if_not
template <typename I, typename P>requires(Readable(I) && Iterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))I count_if(I f, I l, P p) {
DistanceType(I) n(0);while (f != l) {
if (p(source(f))) n = successor(n);++f;
}return n;
}
template <typename I, typename P>requires(Readable(I) && Iterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))I count_if_not(I f, I l, P p) {
DistanceType(I) n(0);while (f != l) {
if (!source(f)) n = successor(n);++f;
}return n;
}
Stepanov, McJones Elements of Programming March 14, 2008 520 / 880
potential_partition_point
template <typename I, typename P>requires(Readable(I) && ForwardIterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))I potential_partition_point(I f, I l, P p){
I i = f;while (f != l) {
if (!p(source(f))) ++i;++f;
}return i;// Postcondition: i = f+ count_if_not(f, l,p)
}
Stepanov, McJones Elements of Programming March 14, 2008 521 / 880
Multiplicity of partitioned rearrangements
Since partition requires only that false values of the predicateprecede true values, there are u!v! ways to partition a range with ufalse values and v true valuesA partition algorithm would satisfy all our needs if it had theseproperties:
Requires only forward iteratorsStableIn placeLinear time, with minimal number of assignments and predicateapplications
We conjecture no such algorithm existsWe present some partition algorithms to demonstrate the kind oftradeoffs that must be made
Stepanov, McJones Elements of Programming March 14, 2008 522 / 880
partition_copy35
template <typename I, typename O0, typename O1, typename P>requires(Readable(I) && Iterator(I) &&
Writable(O0) && Iterator(O0) &&Writable(O1) && Iterator(O1) &&UnaryPredicate(P) && Domain(P) == ValueType(I) &&ValueType(I) == ValueType(O0) &&ValueType(I) == ValueType(O1))
pair<O0, O1> partition_copy(I f, I l, O0 o0, O1 o1, P p){
typedef DistanceType(I) N;while (f != l) {
if (p(source(f))) {sink(o1) = source(f); ++o1;
} else {sink(o0) = source(f); ++o0;
}++f;
}return pair<O0, O1>(o0, o1);
}
35T.K. Lakshman suggested the interfaceStepanov, McJones Elements of Programming March 14, 2008 523 / 880
Properties of partition_copy 37
It is stable: the values of each of the two output ranges are in thesame relative order as in the input rangeIt is a copying rearrangement36
It is optimal in terms both of number of predicate applications andassignments (both numbers are n, the size of the range)
36Note that if the input range is defined by mutable forward iterators, it may besupplied as one of the output ranges
37A variation we use later, partition_copy_n, is given in Appendix 2Stepanov, McJones Elements of Programming March 14, 2008 524 / 880
Semistable partition rearrangements
DefinitionA partition rearrangement is semistable if the relative order of elementsnot satisfying the predicate is preserved
Semistable partition can be used for efficient removal from anextent-based sequence (such as the array of Chapter 11 or theC++ std::vector) of elements satisfying a given predicate
First partition the sequence by the predicateErase the elements from the partition point to the end of thesequenceThe sequence now contains the elements not satisfying thepredicate, in their original order 38
38In Chapter 12 we will explain why this is the right thing to doStepanov, McJones Elements of Programming March 14, 2008 525 / 880
semistable_partition39
template <typename I, typename P>requires(Mutable(I) && ForwardIterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))I semistable_partition(I f, I l, P p){
I i = find_if(f, l, p); // none(f, i,p)I j = find_if_not(i, l, p); // none(f, i,p) ∧all(i, j,p) ∧ (j = l∨ ¬p(j))while (j != l) { // none(f, i,p) ∧all(i, j,p) ∧ p(i) ∧ ¬p(j)
cycle_2(i, j); // none(f, i,p) ∧all(i+ 1, j,p) ∧ ¬p(i) ∧ p(j)// (none(f, i,p)∧¬p(i))∧(all(i+1, j,p)∧p(j))
++i; ++j; // none(f, i,p) ∧all(i, j,p)j = find_if_not(j, l, p); // j = l∨ ¬p(j)
}return i;
}
39The algorithm is due to Nico Lomuto, according to: Jon Bentley. ProgrammingPearls. Communications of the ACM, Volume 27, Number 4, April 1984, pages 287-291
Stepanov, McJones Elements of Programming March 14, 2008 526 / 880
Useful lemmas
The annotations on the previous slide follow from some trivial butuseful lemmas
LemmaAfter executing i = find_if(f, l,p), none(f, i,p) ∧ (i = l∨ p(i))
After executing j = find_if_not(f, l,p), all(f, j,p)∧ (j = l∨¬p(j))
If p(source(i)) ∧ q(source(j)) before executing cycle_2(i, j), thenafterward q(source(i)) ∧ p(source(j))
none(f, l,p) ∧ ¬p(l)⇒ none(f, l+ 1,p)all(f+ 1, l,p) ∧ p(f)⇒ all(f, l,p)
Stepanov, McJones Elements of Programming March 14, 2008 527 / 880
Properties of semistable_partition
It is in placeLet n = l− f be the number of elements in the range and let w bethe number of elements not satisfying the predicate following thepartitioned prefix40
The predicate is applied n timescycle_2 is performed w timesThe number of iterator increments is n+w
It is totally inappropriate for use in quicksort since it does notdivide a range of equal values at the midpoint and that leads toquadratic complexity in many practical situations
40w = count_if_not(find_partitioned_prefix(f, l,p), l,p)Stepanov, McJones Elements of Programming March 14, 2008 528 / 880
semistable_partition performs too manyassignments
Consider a range t, f, f, f, f, f, f, fsemistable_partitionwill perform seven calls of cycle_2, whileone suffices
Stepanov, McJones Elements of Programming March 14, 2008 529 / 880
assured_find_if and assured_find_if_not
We can eliminate one of the two tests in the loop for find_if if weare assured there is an element in the range that satisfies thepredicate
template <typename I, typename P>requires(Readable(I) && Iterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))I assured_find_if(I f, P p) {
// Let l be the end of the implied range starting with f// Precondition: some(f, l,p)while (!p(source(f))) ++f;return f;
}
template <typename I, typename P>requires(Readable(I) && Iterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))I assured_find_if_not(I f, P p) {
// Let l be the end of the implied range starting with f// Precondition: not_all(f, l,p)while (p(source(f))) ++f;return f;
}Stepanov, McJones Elements of Programming March 14, 2008 530 / 880
unstable_forward_partition
template <typename I, typename P>requires(Mutable(I) && ForwardIterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))I unstable_forward_partition(I f, I l, P p){
I i = potential_partition_point(f, l, p);I j = find_if_not(i, l, p);while (j != l) {
f = assured_find_if(f, p);cycle_2(f, j); ++f; ++j;j = find_if_not(j, l, p);
}return i;
}
We can use assured_find_if because the number of misplacedelements before and after the partition point are equalThe code does one extra iterator comparison that could beeliminated by inlining both calls to find_if_not (the sameoptimization applies to many following algorithms)
Stepanov, McJones Elements of Programming March 14, 2008 531 / 880
Properties of unstable_forward_partition
It is in placeThe number of times cycle_2 is performed, v, equals the numberof misplaced elements not satisfying the predicate, that is, thenumber of elements not satisfying the predicate in the subrangefrom the potential partition point to the end of the range41
The algorithm applies the predicate 2n times, therefore it is usefulwhen the cost of predicate application is insignificant compared tothe cost of cycle_2This technique of computing the sizes of a set of buckets and thenmoving the elements appears in radix sort42
With a bidirectional iterator, we can avoid precomputing thebucket sizes
41v = count_if_not(potential_partition_point(f, l,p))42See the use of this technique for in-place k-partition Exercise 5.2-13 in Donald
Knuth. The Art of Computer Programming. Volume 3, Addison-Wesley, 1998, page 80;Solution, page 618
Stepanov, McJones Elements of Programming March 14, 2008 532 / 880
find_backward_if and find_backward_if_not
To indicate “not found”, we return f, which forces us to returni+ 1 if we find a satisfying element at position i
template <typename I, typename P>requires(Readable(I) && BidirectionalIterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))I find_backward_if(I f, I l, P p) {
while (true) {if (f == l) return f;--l;if (p(source(l))) return successor(l);
}}template <typename I, typename P>
requires(Readable(I) && BidirectionalIterator(I) &&UnaryPredicate(P) && Domain(P) == ValueType(I))
I find_backward_if_not(I f, I l, P p) {while (true) {
if (f == l) return f;--l;if (!p(source(l))) return successor(l);
}}
Stepanov, McJones Elements of Programming March 14, 2008 533 / 880
unstable_bidirectional_partition
template <typename I, typename P>requires(Mutable(I) && BidirectionalIterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))I unstable_bidirectional_partition(I f, I l, P p){
while (true) {f = find_if(f, l, p);l = find_backward_if_not(f, l, p);if (f == l) return f;--l;cycle_2(f, l);++f;
}return f;
}
Stepanov, McJones Elements of Programming March 14, 2008 534 / 880
Properties of unstable_bidirectional_partition
It is in placeThe predicate is applied n timescycle_2 is called v times (where v is as previously defined forunstable_forward_partition)The algorithm moves only those elements that need to be moved,and puts two elements in their final destination with each call tocycle_2The total number of assignments, therefore, is 3v
Stepanov, McJones Elements of Programming March 14, 2008 535 / 880
Minimizing the number of assignments
It is possible to accomplish partition with a differentrearrangement that has only a single cycle, resulting in 2v+ 1assignmentsThe idea is to:
Save the first misplaced element, creating a “hole”Repeatedly find a misplaced element on the opposite side of thepotential partition point and move it into the hole, creating a newholeMove the saved element into the final hole
Stepanov, McJones Elements of Programming March 14, 2008 536 / 880
assured_find_backward_if andassured_find_backward_if_not
template <typename I, typename P>requires(Readable(I) && BidirectionalIterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))I assured_find_backward_if(I l, P p) {
do --l; while (!p(source(l)));return l;
}
template <typename I, typename P>requires(Readable(I) && BidirectionalIterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))I assured_find_backward_if_not(I l, P p) {
do --l; while (p(source(l)));return l;
}
We chose to return the iterator rather than its successor as infind_backward_if since we do not need to indicate failure
Stepanov, McJones Elements of Programming March 14, 2008 537 / 880
single_cycle_partition
template <typename I, typename P>requires(Readable(I) && BidirectionalIterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))I single_cycle_partition(I f, I l, P p){
f = find_if(f, l, p);l = find_backward_if_not(f, l, p);if (f == l) return f;--l;ValueType(I) tmp = source(f);while (true) {
sink(f) = source(l);f = find_if(successor(f), l, p);if (f == l) {
sink(l) = tmp;return f;
}sink(l) = source(f);l = assured_find_backward_if_not(l, p);
}}
Stepanov, McJones Elements of Programming March 14, 2008 538 / 880
Properties of single_cycle_partition
It remains to be observed whether using one large cycle instead ofmultiple 2-cycles is faster on modern computers withinstruction-level parallelismThe technique, however, is worth careful study
Stepanov, McJones Elements of Programming March 14, 2008 539 / 880
assured_bidirectional_partition
template <typename I, typename P>requires(Readable(I) && BidirectionalIterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))I assured_bidirectional_partition(I f, I l, P p){
// Precondition:// (¬all(f, l,p) ∧ some(f, l,p)) ∨ (¬p(source(f− 1)) ∧ p(source(l)))while (true) {
f = assured_find_if(f, p);l = assured_find_backward_if_not(l, p);if (successor(l) == f) return f;cycle_2(f, l);++f; // ¬p(source(f− 1)) ∧ p(source(l))
}}
Stepanov, McJones Elements of Programming March 14, 2008 540 / 880
Correctness of assured_bidirectional_partition
The precondition provides the guards for assured_find_if andassured_find_backward_if_notAfter these two calls, f , l, since they point to elements withdifferent predicate valuesl− f decreases by at least two each iterationf and l can cross over by only one position
Suppose they had crossed by more than one, and consider anyiteratorm between l and fIf ¬p(source(m)), then fwould have stopped there; otherwise, lwould have stopped thereTherefore, there is no suchm
If the exit is not taken, f ≺ l, and f and l point, correspondingly, tosatisfying and non-satisfying elementsThe cycle_2 call moves these elements to correct positions, andthe increment of f restores the invariant
Stepanov, McJones Elements of Programming March 14, 2008 541 / 880
sentinel_partition
With a little extra work we can establish the appropriateprecondition to use an assured function43
template <typename I, typename P>requires(Readable(I) && BidirectionalIterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))I sentinel_partition(I f, I l, P p){
f = find_if(f, l, p);l = find_backward_if_not(f, l, p);if (f == l) return f;--l;cycle_2(f, l);++f;return assured_bidirectional_partition(f, l, p);
}
43It is possible to combine the sentinel and single-cycle techniques, producingsentinel_single_cycle_partition
Stepanov, McJones Elements of Programming March 14, 2008 542 / 880
The origin of partition algorithms
Three previous algorithms are based on a paper44 by C.A.R.Hoare:
unstable_bidirectional_partitionsingle_cycle_partitionsentinel_partition
44C.A.R. Hoare. Quicksort. The Computer Journal, Volume 5, Number 1, 1962, pages10-16. This paper, which is a serious contender for the best-ever computer sciencepaper, should not be confused with his earlier publication: C.A.R. Hoare. Algorithm63, Partition; Algorithm 64, Quicksort; Algorithm 65, Find. Communications of theACM, Volume 4, Number 7, July 1961, pages 321-322. No one should attempt toimplement or teach quicksortwithout studying this paper.
Stepanov, McJones Elements of Programming March 14, 2008 543 / 880
Regularity of predicate for partition algorithms
The UnaryPredicate concept requires that the predicate is a regularfunction; this assures the same boolean value is returned everytime the predicate is applied to a particular objectThis in turn assures that after the partition algorithm returns, acall to is_partitioned(f, l,p) would return trueHowever the algorithms using assured finds depend on regularityin a more fundamental way: to assure they stay within bounds ofthe rangeConsider, for example, using a nonregular predicate to perform arandom shuffle
Stepanov, McJones Elements of Programming March 14, 2008 544 / 880
Exercise
ExerciseDesign an algorithm implementing in-place uniform randomshuffle of a range of forward iterators (hint: recursively partitionthe range with a non-regular coin-tossing predicate a )Prove that the expected number of coin tosses is n log2 n
Prove that in-place uniform random shuffle of a range of forwarditerators cannot be done in linear time
aThe algorithm was discovered by Raymond Lo and Wilson Ho
Stepanov, McJones Elements of Programming March 14, 2008 545 / 880
Intuition for stable partition
An empty range is trivially partitionedA singleton range can be partitioned with one predicateapplication and possibly one iterator incrementAn arbitrary range can be partitioned with a divide-and-conqueralgorithm:
Divide the range in the middleStably partition each halfCombine the results by rotating the range bounded by the first andsecond partition points around the middle: **** simple picture *****
This easily extends to a memory adaptive version
Stepanov, McJones Elements of Programming March 14, 2008 546 / 880
Base cases for stable partition
template <typename I, typename P>requires(Mutable(I) && ForwardIterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))pair<I, I> stable_partition_0(I f, P){
return pair<I, I>(f, f);}
template <typename I, typename P>requires(Mutable(I) && ForwardIterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))pair<I, I> stable_partition_1(I f, P p){
I l = successor(f);if (p(source(f)))
return pair<I, I>(f, l);else
return pair<I, I>(l, l);}
Stepanov, McJones Elements of Programming March 14, 2008 547 / 880
Combining operation for stable partition
template <typename I>requires(Mutable(I) && ForwardIterator(I))
pair<I, I> stable_partition_combine(pair<I, I> r0, pair<I, I> r1){
I m = rotate(r0.first, r0.second, r1.first);return pair<I, I>(m, r1.second);
}
Stepanov, McJones Elements of Programming March 14, 2008 548 / 880
stable_partition_n
template <typename I, typename P>requires(Mutable(I) && ForwardIterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))pair<I, I> stable_partition_n(I f, DistanceType(I) n, P p){
typedef DistanceType(I) N;if (n == N(0))
return stable_partition_0(f, p);if (n == N(1))
return stable_partition_1(f, p);N h = half_nonnegative(n);pair<I, I> r0 = stable_partition_n(f, h, p);pair<I, I> r1 = stable_partition_n(r0.second, n - h, p);return stable_partition_combine(r0, r1);
}
Stepanov, McJones Elements of Programming March 14, 2008 549 / 880
stable_partition_n_with_buffer
template <typename I, typename B, typename P>requires(Mutable(I) && ForwardIterator(I) &&
Mutable(B) && ForwardIterator(B) &&UnaryPredicate(P) && Domain(P) == ValueType(I) &&ValueType(I) == ValueType(B))
pair<I, I> stable_partition_n_with_buffer(I f, DistanceType(I) n, B b, P p){
pair<I, B> r = partition_copy_n(f, n, f, b, p);return pair<I, I>(r.first, copy(b, r.second, r.first));
}
Stepanov, McJones Elements of Programming March 14, 2008 550 / 880
stable_partition_n_adaptive
template <typename I, typename B, typename P>requires(Mutable(I) && ForwardIterator(I) &&
Mutable(B) && ForwardIterator(B) &&UnaryPredicate(P) && Domain(P) == ValueType(I) &&ValueType(I) == ValueType(B))
pair<I, I> stable_partition_n_adaptive(I f, DistanceType(I) n, B b, DistanceType(I) n_b, P p)
{typedef DistanceType(I) N;if (n == N(0))
return stable_partition_0(f, p);if (n == N(1))
return stable_partition_1(f, p);if (n <= n_b)
return stable_partition_n_with_buffer(f, n, b, p);N h = half_nonnegative(n);pair<I, I> r0 = stable_partition_n_adaptive(f, h, b, n_b, p);pair<I, I> r1 = stable_partition_n_adaptive(r0.second, n - h, b, n_b, p);return stable_partition_combine(r0, r1);
}
Stepanov, McJones Elements of Programming March 14, 2008 551 / 880
Properties of stable partition
Exactly n predicate applications are performed at the bottom levelof recursionThe depth of the recursion for stable_partition_n is dlog2 neAt every recursive level we rotate n/2 elements on the average,requiring between n/2 and 3n/2 assignments depending on theiterator categoryThe total number of assignments is n log2 n/2 for random accessiterators and 3n log2 n/2 for forward and bidirectional iterators
ExerciseDetermine the number of assignments for the adaptive version
Stepanov, McJones Elements of Programming March 14, 2008 552 / 880
Interfaces for binary search
What is commonly referred to as binary search is really threerelated algorithms for determining the lower bound, the upperbound, or bothThe advantage of these as compared to the way binary search isusually specified in textbooks is that the lower and upper boundsare always defined, so there is no issue of what to return when theelement is not presentSeveral algorithms in this chapter will demonstrate the utility ofthese interfaces
Stepanov, McJones Elements of Programming March 14, 2008 553 / 880
Implementing lower bound and upper bound
Earlier in this chapter we defined the lower bound and upperbound for a value a as the partition points of two predicates:
Pa(x) ≡ x < a
P ′a(x) ≡ ¬(a < x)
They can be implemented by passing functions corresponding toPa and P ′a to partition_point_nSee Appendix 2 for the C++ versions
ExerciseImplement a function that returns both lower and upper bounds anddoes fewer comparisons than the sum of the comparisons that wouldbe done by calling both lower_bound_n and upper_bound_na
aA similar STL function is called equal_range
Stepanov, McJones Elements of Programming March 14, 2008 554 / 880
Merging
DefinitionMerge is an operation that combines two increasing ranges into a singleincreasing range that contains all the elements of both ranges and noother elements
DefinitionA merge is stable if the output range preserves the relative order ofequivalent elements both within each input range, and between thefirst and second input range
LemmaStability establishes a unique ordering of the elements in the outputrange
Stepanov, McJones Elements of Programming March 14, 2008 555 / 880
Merge interfaces
A mutating45 merge operation takes two adjacent increasingranges and rearranges the elements in the combined range intoincreasing order; a range [f, l) is mergeable if there is an iteratorm ∈ [f, l) such that the subranges [f,m) and [m, l) are increasingA copying merge operation sets a writable range to the merge oftwo readable increasing rangesA melding merge operation combines two ranges defined by nodeiterators by changing their relative order46
45A mutating merge may or may not be in place depending on how much extramemory it uses
46We define node iterators in the next chapterStepanov, McJones Elements of Programming March 14, 2008 556 / 880
is_mergeable
template <typename I, typename R>requires(Readable(I) && ForwardIterator(I) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I))bool is_mergeable(I f, I m, I l, R r){
return is_increasing(f, m, r) && is_increasing(m, l, r);}
Stepanov, McJones Elements of Programming March 14, 2008 557 / 880
is_merged
ExerciseImplement this function that checks whether the range starting at o isequal to the merge of the ranges [f0, l0) and [f1, l1)
template <typename I, typename R>requires(Readable(I) && ForwardIterator(I) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I))bool is_merged(I f0, I l0, I f1, I l1, I o, R r);
Stepanov, McJones Elements of Programming March 14, 2008 558 / 880
Algebraic properties of merge
Merge is a binary operation on rangesAn empty range is its identity element (both left and right)Merge is associative
Its complexity needs to be taken into consideration whenreassociating
Merge is commutative when stability is not taken into account
Stepanov, McJones Elements of Programming March 14, 2008 559 / 880
Complexity property of copying merge
If ◦ denotes the merge operation and C(x,y) denotes the cost (thenumber of assignments) to merge ranges x and y, then Csatisfies47:
C(x ◦ y, z) =C(x,y) + C(x, z) + C(y, z)
2
A consequence of this is:
C(x,y) < C(y, z)⇒ C(x,y) + C(x ◦ y, z) < C(y, z) + C(x,y ◦ z)
In other words, it is better to do the least-costly merge first
47In the next chapter we will define a Huffman semigroup, where the cost of thesemigroup operation satisfies this axiom
Stepanov, McJones Elements of Programming March 14, 2008 560 / 880
merge_copy_n
template <typename I0, typename I1, typename O, typename R>requires(Readable(I0) && Iterator(I0) && Readable(I1) && Iterator(I1) &&
Writable(O) && Iterator(O) &&StrictWeakOrdering(R) && Domain(R) == ValueType(I0) &&ValueType(I0) == ValueType(O) && ValueType(I1) == ValueType(O))
void merge_copy_n(I0 f0, DistanceType(I0) n0,I1 f1, DistanceType(I1) n1, O o, R r) {
// Precondition: f1 = f0 +n0 ∧ is_mergeable(f0, f1, f1 +n1, r)// Precondition: the input ranges do not overlap the output rangetypedef DistanceType(I0) N0; typedef DistanceType(I1) N1;while (true) {
if (n0 == N0(0)) { copy_n(f1, n1, o); return; }if (n1 == N1(0)) { copy_n(f0, n0, o); return; }if (r(source(f1), source(f0))) {
sink(o) = source(f1); ++f1; n1 = n1 - N1(1);} else {
sink(o) = source(f0); ++f0; n0 = n0 - N0(1);}++o;
}}
The overlap precondition is not the weakest possibleStepanov, McJones Elements of Programming March 14, 2008 561 / 880
merge_copy_backward_n
template <typename I0, typename I1, typename O, typename R>requires(Readable(I0) && BidirectionalIterator(I0) &&
Readable(I1) && BidirectionalIterator(I1) &&Writable(O) && BidirectionalIterator(O) &&StrictWeakOrdering(R) && Domain(R) == ValueType(I0) &&ValueType(I0) == ValueType(O) && ValueType(I1) == ValueType(O))
void merge_copy_backward_n(I0 l0, DistanceType(I0) n0,I1 l1, DistanceType(I1) n1, O o, R r) {
// Let f0 = l0 −n0 and f1 = l1 −n1// Precondition: f1 = f0 +n0 ∧ is_mergeable(f0, f1, f1 +n1, r)typedef DistanceType(I0) N0; typedef DistanceType(I1) N1;while (true) {
if (n0 == N0(0)) { copy_backward_n(l1, n1, o); return; }if (n1 == N1(0)) { copy_backward_n(l0, n0, o); return; }--o;if (r(source(predecessor(l1)), source(predecessor(l0)))) {
--l0; sink(o) = source(l0); n0 = n0 - N0(1);} else {
--l1; sink(o) = source(l1); n1 = n1 - N1(1);}
}}
Stability requires the order of the if clauses is reversed from the one inmerge_copy_nStepanov, McJones Elements of Programming March 14, 2008 562 / 880
Return value for copying merges
ExerciseReimplementmerge_copy_n andmerge_copy_backward_nwith thecorrect return values
Stepanov, McJones Elements of Programming March 14, 2008 563 / 880
Complexity of copying merges
LemmaThe number of assignments is always n0 + n1 and the worst-casenumber of comparisons is n0 + n1 − 1
ExerciseDetermine the average number of comparisons
Stepanov, McJones Elements of Programming March 14, 2008 564 / 880
merge_n_with_buffer for ForwardIterator
template <typename I, typename B, typename R>requires(Mutable(I) && ForwardIterator(I) &&
Mutable(B) && ForwardIterator(B) &&StrictWeakOrdering(R) && Domain(R) == ValueType(I) &&ValueType(I) == ValueType(B))
void merge_n_with_buffer(I f0, DistanceType(I) n0,I f1, DistanceType(I) n1, B b, R r,forward_iterator_tag)
{// Precondition: f1 = f0 +n0 ∧ is_mergeable(f0, f1, f1 +n1, r)// Precondition: the range beginning with b is of size at leastmin(n0,n1)if (n0 <= n1) {
copy_n(f0, n0, b);merge_copy_n(b, n0, f1, n1, f0, r);
} else {f1 = rotate(f0, f1, f1 + n1);copy_n(f0, n1, b);merge_copy_n(f1, n0, b, n1, f0, r);
}}
Stepanov, McJones Elements of Programming March 14, 2008 565 / 880
merge_n_with_buffer for BidirectionalIterator
template <typename I, typename B, typename R>requires(Mutable(I) && BidirectionalIterator(I) &&
Mutable(B) && BidirectionalIterator(B) &&StrictWeakOrdering(R) && Domain(R) == ValueType(I) &&ValueType(I) == ValueType(B))
void merge_n_with_buffer(I f0, DistanceType(I) n0,I f1, DistanceType(I) n1, B b, R r,bidirectional_iterator_tag)
{// Precondition: f1 = f0 +n0 ∧ is_mergeable(f0, f1, f1 +n1, r)// Precondition: the range beginning with b is of size at leastmin(n0,n1)if (n0 <= n1) {
copy_n(f0, n0, b);merge_copy_n(b, n0, f1, n1, f0, r);
} else {pair<I, B> tmp = copy_n(f1, n1, b);merge_copy_backward_n(f1, n0, tmp.second, n1, tmp.first, r);
}}
Stepanov, McJones Elements of Programming March 14, 2008 566 / 880
Concept Merger
DefinitionMerger(M)⇒
The type function IteratorType(M) is definedProcedure(M)
For allm ∈M, I = IteratorType(M), and N = DistanceType(I),m : I×N× I×N→ void
There is an implicit function ordering such that for allm ∈M,ordering(m) is a strict weak ordering on the value type of theiterator type ofMFor allm ∈M, a callm(f0,n0, f1,n1) satisfying the precondition
f1 = f0 + n0 ∧ is_mergeable(f0, f1, f1 + n1,ordering(m))
will establish the postcondition
is_increasing(f0, f0 + n0 + n1,ordering(m))Stepanov, McJones Elements of Programming March 14, 2008 567 / 880
merger_with_buffer
template <typename I, typename B, typename R>requires(Mutable(I) && BidirectionalIterator(I) &&
Mutable(B) && BidirectionalIterator(B) &&StrictWeakOrdering(R) && Domain(R) == ValueType(I) &&ValueType(I) == ValueType(B))
struct merger_with_buffer{
B b;R r;merger_with_buffer(B b, R r) : b(b), r(r) { }void operator()(I f0, DistanceType(I) n0, I f1, DistanceType(I) n1){
// Precondition: f1 = f0 +n0 ∧ is_mergeable(f0, f1, f1 +n1, r)// Precondition: the range beginning with b is of size at leastmin(n0,n1)merge_n_with_buffer(f0, n0, f1, n1, b, r);
}};
Stepanov, McJones Elements of Programming March 14, 2008 568 / 880
Sorting algorithms
DefinitionA key-sorting (or just sorting) algorithm is a rearrangement thosepostcondition is that the range is key-increasing (or justincreasing) with respect to a given strict weak orderA key-sorting (or sorting) algorithm is stable if it does not changethe relative order of values that are equivalent under thesymmetric complement of the given strict weak order
While the key function is a useful feature, including it in thesorting functions in this chapter is not instructive sincetransforming a function dealing with an increasing sequence intoone with a key-increasing sequence is straightforward
Stepanov, McJones Elements of Programming March 14, 2008 569 / 880
merge_sort_n
template <typename I, typename M>requires(Mutable(I) && ForwardIterator(I) &&
Merger(M) && IteratorType(M) == I)I merge_sort_n(I f, DistanceType(I) n, M merger){
typedef DistanceType(I) N;if (n < N(2)) return f + n;N h = half_nonnegative(n);I m = merge_sort_n(f, h, merger);I l = merge_sort_n(m, n - h, merger);merger(f, h, m, n - h);return l;
}
Stepanov, McJones Elements of Programming March 14, 2008 570 / 880
stable_sort_n_with_buffer
template <typename I, typename B, typename R>requires(Mutable(I) && ForwardIterator(I) &&
Mutable(B) && ForwardIterator(B) &&StrictWeakOrdering(R) && Domain(R) == ValueType(I) &&ValueType(I) == ValueType(B))
I stable_sort_n_with_buffer(I f, DistanceType(I) n, B b, R r){
// Precondition: the range beginning with b is of size at least n/2return merge_sort_n(f, n, merger_with_buffer<I, B, R>(b, r));
}
Note that h 6 n− h always holds, so the call on rotate in theForwardIterator version ofmerge_n_with_buffer will never beexecuted
Stepanov, McJones Elements of Programming March 14, 2008 571 / 880
Complexity of stable_sort_n_with_buffer
The algorithm has dlog2 ne recursive levelsEach level performs 3n/2 assignments, for a total number ofassignments of 3
2ndlog2 neAt ith level from the bottom, the worst-case number ofcomparisons is n− n
2i ; summing over all levels gives
ndlog2 ne−dlog2ne∑i=1
n
2i≈ ndlog2 ne− n
Stepanov, McJones Elements of Programming March 14, 2008 572 / 880
Intuition for stable in-place merge
TheoremIf is_mergeable(f,m, l, r), then for every iterator i ∈ [f,m), there is aunique j ∈ [m, l) such that after x← source(i) andm ′ ← rotate(i,m, j),
[f, l) is stably partitioned, with [f,m ′) containing elements lessthan or equal to x and [m ′, l) containing elements greater than orequal to xis_mergeable(f, i,m ′, r) ∧ is_mergeable(m ′, j, l, r)
Proof.Consider j = lower_bound(m, l, source(i), r) �
An analogous result holds for every j ∈ [m, l): leti = upper_bound(f,m, source(j), r), and performm ′ ← rotate(i,m, successor(j))
Stepanov, McJones Elements of Programming March 14, 2008 573 / 880
A sketch of an algorithm for stable in-place merge
This suggests a divide-and-conquer algorithm: choose an i in[f,m) and the corresponding j in [m, l), or a j in [m, l) and acorresponding i in [f,m), rotate appropriately, and then mergeeach of the mergeable subrangesNow since j−m ′ = m− i, choosing i halfway between f andm orj halfway betweenm and lwill cut the minimum ofm− f andl−m in halfThese considerations lead to the following algorithm
Stepanov, McJones Elements of Programming March 14, 2008 574 / 880
merge_n_in_place
template <typename I, typename R>requires(Mutable(I) && ForwardIterator(I) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I))void merge_n_in_place(I f0, DistanceType(I) n0, I f1, DistanceType(I) n1, R r) {
// Precondition: f1 = f0 +n0 ∧ is_mergeable(f0, f1, f1 +n1, r)typedef DistanceType(I) N;if (min(n0, n1) == N(0)) return;I m0, m1;N h0, h1;if (n0 < n1) {
h0 = half_nonnegative(n0); m0 = f0 + h0;m1 = lower_bound_n(f1, n1, source(m0), r); h1 = m1 - f1;I m = rotate(m0, f1, m1);merge_n_in_place(f0, h0, m0, h1, r);merge_n_in_place(successor(m), n0 - successor(h0), m1, n1 - h1, r);
} else {h1 = half_nonnegative(n1); m1 = f1 + h1;m0 = upper_bound_n(f0, n0, source(m1), r); h0 = m0 - f0;I m = rotate(m0, f1, successor(m1));merge_n_in_place(f0, h0, m0, h1, r);merge_n_in_place(m, n0 - h0, successor(m1), n1 - successor(h1), r);
}}
Stepanov, McJones Elements of Programming March 14, 2008 575 / 880
Properties ofmerge_n_in_place
Lemma1 Each call of rotate puts one element into its final position2 The algorithm terminates with a sorted range3 Each level of recursion reducesm = min(n0,n1) to no more thanbm/2c
4 There are at most blog2(min(n0,n1))c+ 1 recursive levels
The algorithm is due to Dudzinski and Dydek, whose paper48
also contains a careful complexity analysis
48Krzysztof Dudzinski and Andrzej Dydek. On a Stable Minimum Storage MergingAlgorithm. Information Processing Letters, Volume 12, Number 1, February 1981, pages5-8.
Stepanov, McJones Elements of Programming March 14, 2008 576 / 880
Complexity ofmerge_n_in_place: number ofassignments
TheoremLet
kn be the maximum number of assignments required to rotate arange of length na(n0,n1) be the maximum number of assignments required for astable in-place merge of two ranges with lengths n0 and n1, wheren0 6 n1
Thena(n0,n1) 6 k(n1 + dn0/2e) log2(2n0) 6 kn log2 n
kmay be taken to be 3/2 for indexed iterators and 3 for forward(and bidirectional) iterators
Stepanov, McJones Elements of Programming March 14, 2008 577 / 880
Number of assignments ofmerge_n_in_place49
Proof.By induction on n0.
n0 = 1 A single rotation of a range of length at most n1 + 1 accomplishes the merge, soa(1,n1) = r(n1 + 1) 6 k(n1 + 1) = k(n1 + d1/2e) log2(2).
n0 > 1 We have a(n0,n1) 6 r(n1 +n0 −h0) +a(h0,h1) +a(n0 −h0 − 1,n1 −h1). Bythe induction hypothesis, a(h0,h1) 6 k(h1 + dh0/2e) log2(2h0) and
a(n0 −h0 − 1,n1 −h1) 6 k(n1 −h1 + d(n0 −h0 − 1)/2e) log2(2(n0 −h0 − 1)).
It is easily verified that dx/2e+ dy/2e 6 d(x+ y+ 1)/2e for any x and y. Thus
dh0/2e+ d(n0 −h0 − 1)/2e 6 dn0/2e
and
a(n0,n1) 6 k(n1 + dn0/2e)(log2(n0) + 1) = k(n1 + dn0/2e)(log2(2n0)).
�
49We are indebted to John Wilkinson for this proofStepanov, McJones Elements of Programming March 14, 2008 578 / 880
Complexity ofmerge_n_in_place: number ofcomparisons
TheoremLet c(n0,n1) be the maximum number of comparisons required for astable in-place merge of a range of length n0 with a range of length n1,where n0 6 n1; then c(n0,n1) 6 kn1
ExerciseDetermine the bound k and prove the theorem
Stepanov, McJones Elements of Programming March 14, 2008 579 / 880
merger_in_place
template <typename I, typename R>requires(Mutable(I) && BidirectionalIterator(I) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I))struct merger_in_place{
R r;merger_in_place(R r) : r(r) { }void operator()(I f0, DistanceType(I) n0, I f1, DistanceType(I) n1){
// Precondition: f1 = f0 +n0 ∧ is_mergeable(f0, f1, f1 +n1, r)merge_n_in_place(f0, n0, f1, n1, r);
}};
Stepanov, McJones Elements of Programming March 14, 2008 580 / 880
stable_sort_n_in_place
template <typename I, typename R>requires(Mutable(I) && ForwardIterator(I) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I))I stable_sort_n_in_place(I f, DistanceType(I) n, R r){
return merge_sort_n(f, n, merger_in_place<I, R>(r));}
Stepanov, McJones Elements of Programming March 14, 2008 581 / 880
merge_n
template <typename I, typename B, typename R>requires(Mutable(I) && ForwardIterator(I) &&
Mutable(B) && ForwardIterator(B) &&StrictWeakOrdering(R) && Domain(R) == ValueType(I) &&ValueType(I) == ValueType(B))
void merge_n(I f0, DistanceType(I) n0, I f1, DistanceType(I) n1,B b, DistanceType(I) n_b, R r) {
// Precondition: f1 = f0 +n0 ∧ is_mergeable(f0, f1, f1 +n1, r)if (min(n0, n1) <= n_b) { merge_n_with_buffer(f0, n0, f1, n1, b, r); return; }I m0, m1; DistanceType(I) h0, h1;if (n0 < n1) {
h0 = half_nonnegative(n0); m0 = f0 + h0;m1 = lower_bound_n(f1, n1, source(m0), r); h1 = m1 - f1;I m = rotate(m0, f1, m1);merge_n(f0, h0, m0, h1, b, n_b, r);merge_n(successor(m), n0 - successor(h0), m1, n1 - h1, b, n_b, r);
} else {h1 = half_nonnegative(n1); m1 = f1 + h1;m0 = upper_bound_n(f0, n0, source(m1), r); h0 = m0 - f0;I m = rotate(m0, f1, successor(m1));merge_n(f0, h0, m0, h1, b, n_b, r);merge_n(m, n0 - h0, successor(m1), n1 - successor(h1), b, n_b, r);
}}
Stepanov, McJones Elements of Programming March 14, 2008 582 / 880
merger
template <typename I, typename B, typename R>requires(Mutable(I) && BidirectionalIterator(I) &&
Mutable(B) && BidirectionalIterator(B) &&StrictWeakOrdering(R) && Domain(R) == ValueType(I) &&ValueType(I) == ValueType(B))
struct merger_adaptive{
typedef DistanceType(B) N;B b;N n_b;R r;merger_adaptive(B b, N n_b, R r) : b(b), n_b(n_b), r(r) { }void operator()(I f0, DistanceType(I) n0, I f1, DistanceType(I) n1){
// Precondition: f1 = f0 +n0 ∧ is_mergeable(f0, f1, f1 +n1, r)// Precondition: the range beginning with b is of size at leastmin(n0,n1)merge_n(f0, n0, f1, n1, b, n_b, r);
}};
Stepanov, McJones Elements of Programming March 14, 2008 583 / 880
stable_sort_n
template <typename I, typename B, typename R>requires(Mutable(I) && ForwardIterator(I) &&
Mutable(B) && ForwardIterator(B) &&StrictWeakOrdering(R) && Domain(R) == ValueType(I) &&ValueType(I) == ValueType(B))
I stable_sort_n(I f, DistanceType(I) n, B b, DistanceType(I) n_b, R r){
return merge_sort_n(f, n, merger_adaptive<I, B, R>(b, n_b, r));}
Stepanov, McJones Elements of Programming March 14, 2008 584 / 880
Importance of stable in-place and adaptive merge andsort
Stable in-place sort is of theoretical interest50; in practice, it shouldnever be usedIts simple extension to the memory-adaptive algorithm leads to agood practical algorithm when a buffer of as low as 1% is availableand becomes extremely competitive with 10-25%
50Donald Knuth ranked this problem as 47 out of 50 – just a notch below the LastFermat Theorem – in the first edition Volume 3 of his Art of Computer Programming
Stepanov, McJones Elements of Programming March 14, 2008 585 / 880
The origin of computer sorting and binary searching
The first description of binary searching, binary insertion sort, andcopying merge sort was John W. Mauchly’s 1946 lecture “Sortingand Collating”51
51Martin Campbell-Kelly and Michael R. Williams, editors. The Moore SchoolLectures: Theory and Techniques for Design of Electronic Digital Computers. TheMIT Press and Tomash Publishers, 1985.
Stepanov, McJones Elements of Programming March 14, 2008 586 / 880
Conclusions
Mathematicians use definitions and theorems as building blocksfor ever more powerful resultsThis chapter shows how a difficult problem of stable sorting inminimal space can be effectively solved using simple buildingblocks
Copying (forward and backward)Rotate (for various iterator categories)Lower and upper bound
It provides specific examples of why our attention to interfaces(arguments and return values) is importantIt shows that the technique of memory adaptive algorithms,which we first introduced in a rather artifical context (reverse forforward iterators), leads to a simple, efficient version of animportant algorithm (stable sort)Lower and upper bound, merge, and sort allow sequences to beused efficiently as representations of sets and multisets
Stepanov, McJones Elements of Programming March 14, 2008 587 / 880
Project
ProjectDesign a benchmark comparing performance ofunstable_bidirectional_partition, single_cycle_partition,sentinel_partition, and sentinel_single_cycle_partition fordifferent array sizes, element sizes, and distributions of elementssatisfying the predicateDesign a benchmark to analyze the performance of stable_sort_nfor different array sizes, element sizes, and buffer sizes
Stepanov, McJones Elements of Programming March 14, 2008 588 / 880
Contents I
1 Introduction
2 Foundations
3 Transformations and their orbits
4 Algorithms on algebraic structures
5 Orderings
6 Combining concepts
7 Refining concepts of iterators
Stepanov, McJones Elements of Programming March 14, 2008 589 / 880
Contents II
8 Permutations and rearrangements
9 Rotations
10 Algorithms on increasing ranges
11 Coordinate structuresCoordinate structuresLinkable iteratorsLinking operationsLink rearrangementReversing a linkable rangeExamples of linkable iteratorsGeneralized rearrangementsPartition
Stepanov, McJones Elements of Programming March 14, 2008 590 / 880
Contents III
MergingSortingTemporarily breaking invariantsBifurcate coordinatesTraversing treesSimilarity, equivalence, and orderingConclusionProjects
12 Composite objects
13 Iterative algorithms for divide-and-conquer
14 Mathematical notation
Stepanov, McJones Elements of Programming March 14, 2008 591 / 880
Contents IV
15 C++ machinery
16 Acknowledgments
17 Index
Stepanov, McJones Elements of Programming March 14, 2008 592 / 880
Intuition for coordinate structures
In Chapter 6 we defined a family of iterator concepts thatgeneralize pointers and describe linear traversals within anytype-homogenous data structureTo allow traversal within arbitrary data structures, we generalizeiterators to a family of concepts called coordinate structuresWhile an iterator type has only linear traversals (e.g., i 7→ i+ 1 ori 7→ i+ n), and a single static value type, a coordinate structuremay have several interrelated coordinate types, each with its ownvalue type and diverse traversal functionsCoordinate structures abstract the navigational aspects of datastructures, while containers (introduced in the next chapter)abstract storage management and ownershipThere could be multiple coordinate structures describing the sameset of objects
Stepanov, McJones Elements of Programming March 14, 2008 593 / 880
Definition of coordinate structure
DefinitionA concept is a coordinate structure if it consists of:
One or more coordinate typesFor each coordinate type Ti, a value type ValueType(Ti)For each coordinate type, dereferencing functions source and/orsink providing “fast” readable, writable, or mutable access
Access via a coordinate value is as fast as via any other mechanism,so there is no need to “cache” a low-level pointer or reference
For each coordinate type Ti, one or more traversal functions:
fi,j : Ti × · · · → Tk
Stepanov, McJones Elements of Programming March 14, 2008 594 / 880
Reflections on coordinate structure
Note that “coordinate structure” is not a concept but a family ofrelated concepts
It is not a concept because it does not have specific coordinate typesor traversal functions
No algorithms can be defined on coordinate structure as such;meta-algorithms are possible but are outside the scope of this book
Stepanov, McJones Elements of Programming March 14, 2008 595 / 880
Taxonomy of coordinate structures
Related types Most coordinate structures have a singlecoordinate type; a few have two or more
Dereference regularity The dereferencing functions on a coordinatetype may be regular (repeatable) or may beweak, meaning a protocol must be followedbetween dereferences
Topology The traversal functions may allow linear,bifurcating, multi-dimensional, or graphtraversal
Traversal regularity The traversal functions may be regular(supporting multi-pass algorithms), or weak(supporting only single-pass algorithms)
Complexity The traversal functions may have linear,logarithmic, or amortized constant-timecomplexity
Changeable topology The topology between coordinate values maybe fixed or mutable
Stepanov, McJones Elements of Programming March 14, 2008 596 / 880
Plan for the chapter
Iterators are the simplest and most useful coordinate structures;they are concepts of linear fixed-topology coordinatesThis chapter introduces coordinate structures illustrating moredimensions in our taxonomy:
Linkable iterators: changeable topology, with different algorithmsfor partition, merging, and sortingBifurcated coordinates: nonlinear topologies, with algorithms onbinary treesLinkable bifurcated coordinates: algorithms for iterative,constant-space binary tree traversal
Stepanov, McJones Elements of Programming March 14, 2008 597 / 880
Linkable iterators
A linkable iterator does not have a fixed successor or predecessor;these relationships between linkable iterators are mutableA linkable range is a range defined by a pair of linkable iterators[f, l)The mutability of the relationships between linkable iteratorsrequires care to preserve the invariants of ranges:
Iterators in a range are totally ordered by ≺Multiple input or output ranges of an algorithm are disjointThe iterators in the input to an algorithm are preserved in its output
Stepanov, McJones Elements of Programming March 14, 2008 598 / 880
Linking operations for linkable iterators
The iterator concepts require the unique traversal operationssuccessor and predecessor for a given typeWe treat the linking operation differently, passing them asparameters, because of different ways linking is performed on thesame typeSimilarly, in chapter 3 the operation was an explicit parameter topower
To define the requirements on the type of objects performinglinking operations, we define related concepts ForwardLinker,BackwardLinker, and BidirectionalLinker
Stepanov, McJones Elements of Programming March 14, 2008 599 / 880
Concept ForwardLinker
DefinitionForwardLinker(S)⇒
A type function IteratorType(S) is definedForwardIterator(IteratorType(S))An object of type S is a proper mutator taking two iterators of theaffiliated typeFor all set_link ∈ S, immediately after set_link(i, j) is performed,successor(i) = j
Stepanov, McJones Elements of Programming March 14, 2008 600 / 880
Concept BackwardLinker
DefinitionBackwardLinker(S)⇒
A type function IteratorType(S) is definedBidirectionalIterator(IteratorType(S))An object of type S is a proper mutator taking two iterators of theaffiliated typeFor all set_link ∈ S, immediately after set_link(i, j) is performed,i = predecessor(j)
Stepanov, McJones Elements of Programming March 14, 2008 601 / 880
Concept BidirectionalLinker
DefinitionBidirectionalLinker(S)⇒
ForwardLinker(S)BackwardLinker(S)
Stepanov, McJones Elements of Programming March 14, 2008 602 / 880
Link rearrangements
A link rearrangement52 is an algorithm taking a linked range andproducing a linkable range, with the properties:Precondition (The input range is well-formed, i.e., doesn’t
contain a cycle)
Postcondition (The output range is well-formed)Every iterator in the input range appears in theoutput rangeEvery iterator in the output range appeared in theinput rangeEvery iterator in the output range designates thesame object as before the rearrangement, and thisobject has the same valuesuccessor and predecessor relationships that heldin the input range may or may not hold in theoutput range
52The rearrangements introduced in Chapter 7 rearrange data; link rearrangementsrearrange links
Stepanov, McJones Elements of Programming March 14, 2008 603 / 880
reverse_linkable
template <typename S>requires(ForwardLinker(S))
IteratorType(S) reverse_linkable(IteratorType(S) f, IteratorType(S) l,S set_link)
{typedef IteratorType(S) I;I r = l;while (f != l) {
I tmp = successor(f);set_link(f, r);r = f;f = tmp;
}return r;
}
Stepanov, McJones Elements of Programming March 14, 2008 604 / 880
Concept LinkableForwardIterator
DefinitionLinkableForwardIterator(I)⇒
ForwardIterator(I)For all i ∈ I,
source(i.p).forward_link = successor(i)For all j ∈ I, immediately after sink(i.p).forward_link← j,successor(i) = j
Stepanov, McJones Elements of Programming March 14, 2008 605 / 880
Concept LinkableBidirectionalIterator
DefinitionLinkableBidirectionalIterator(I)⇒
BidirectionalIterator(I) ∧ LinkableForwardIterator(I)For all j ∈ I,
source(j.p).backward_link = predecessor(j)For all i ∈ I, immediately after sink(j.p).backward_link← i,i = predecessor(j)
Stepanov, McJones Elements of Programming March 14, 2008 606 / 880
forward_linker, backward_linker andbidirectional_linker
template <typename I> requires(LinkableForwardIterator(I))struct forward_linker{
void operator()(I x, I y){ sink(x.p).forward_link = y.p; }
};
template <typename I> requires(LinkableBidirectionalIterator(I))struct backward_linker{
void operator()(I x, I y){ sink(y.p).backward_link = x.p; }
};
template <typename I> requires(LinkableBidirectionalIterator(I))struct bidirectional_linker{
void operator()(I x, I y){ sink(x.p).forward_link = y.p; sink(y.p).backward_link = x.p; }
};
Stepanov, McJones Elements of Programming March 14, 2008 607 / 880
reverse_forward_linkable andreverse_bidirectional_linkable
template <typename I>requires(LinkableForwardIterator(I))
I reverse_forward_linkable(I f, I l){
return reverse_linkable(f, l, l, forward_linker<I>());}
template <typename I>requires(LinkableBidirectionalIterator(I))
I reverse_bidirectional_linkable(I f, I l){
return reverse_linkable(f, l, l, bidirectional_linker<I>());}
Stepanov, McJones Elements of Programming March 14, 2008 608 / 880
Generalized rearrangements
A generalized rearrangement is an algorithm taking k > 1 rangesand producing l > 1 ranges
Copying merge takes two input ranges and produces one outputrangeIn-place partition may be viewed as taking one input range andproducing two output rangesIn-place rotate may be viewed as taking two input ranges ([f,m)and [m, l)) and producing two output ranges ([f,m ′) and [m ′, l))
Stepanov, McJones Elements of Programming March 14, 2008 609 / 880
Generalized link rearrangements
A generalized link rearrangement has additional properties:Precondition The input ranges must be disjoint
Postcondition Every iterator in an input range appears in one ofthe output rangesEvery iterator in an output range appeared in oneof the input rangesEvery iterator in the output range designates thesame object as before the rearrangement, and thisobject has the same valuesuccessor and predecessor relationships that heldin the input range may or may not hold in theoutput rangeThe output ranges are disjoint
Stepanov, McJones Elements of Programming March 14, 2008 610 / 880
partition_linkable_unoptimized
template <typename S, typename P>requires(ForwardLinker(S) && Readable(IteratorType(S)) &&
UnaryPredicate(P) && Domain(P) == ValueType(IteratorType(S)))pair<pair<IteratorType(S), IteratorType(S)>,
pair<IteratorType(S), IteratorType(S)> >partition_linkable_unoptimized(IteratorType(S) f, IteratorType(S) l, P p,
S set_link){
typedef IteratorType(S) I;I h0 = l; I t0 = l;I h1 = l; I t1 = l;while (f != l) {
if (p(source(f))) {if (h1 == l) { h1 = f; t1 = f; } else { set_link(t1, f); t1 = f; }
} else {if (h0 == l) { h0 = f; t1 = f; } else { set_link(t0, f); t0 = f; }
}++f;
}return pair<pair<I, I>, pair<I, I> >
(pair<I, I>(h0, t0), pair<I, I>(h1, t1));}
Compare with partition_copy_n in Chapter 9Stepanov, McJones Elements of Programming March 14, 2008 611 / 880
Improving on partition_linkable_unoptimized
By unrolling the loop, the inner tests on h0 and h1 can be avoidedafter the not-satisfying and satisfying objects are foundBy maintaining a state indicating whether the predicate wassatisfied for the previous element, set_link calls that do notchange the successor can be avoidedThe state can be represented by the location in the code ratherthan by an explicit state variable; using goto avoids setting andtesting a state variable53
53Undisciplined use of goto leads to code that is harder to understand and lesslikely to be correct; disciplined use frequently improves clarity and efficiency.Benchmarking is required to verify that versions with goto are faster on processorswith instruction level parallelism.
Stepanov, McJones Elements of Programming March 14, 2008 612 / 880
partition_linkable signature
template <typename S, typename P>requires(ForwardLinker(S) && Readable(IteratorType(S)) &&
UnaryPredicate(P) && Domain(P) == ValueType(IteratorType(S)))pair<pair<IteratorType(S), IteratorType(S)>,
pair<IteratorType(S), IteratorType(S)> >partition_linkable(IteratorType(S) f, IteratorType(S) l, P p, S set_link){
typedef IteratorType(S) I;
Stepanov, McJones Elements of Programming March 14, 2008 613 / 880
partition_linkable body
entry: I h0 = l; I t0 = l; I h1 = l; I t1 = l;if (f == l) { goto exit; }if (p(source(f))) { h1 = f; goto s1; }else { h0 = f; goto s0; }
s0: t0 = f; ++f;if (f == l) { goto exit; }if (p(source(f))) { h1 = f; goto s3; }else { goto s0; }
s1: t1 = f; ++f;if (f == l) { goto exit; }if (p(source(f))) { goto s1; }else { h0 = f; goto s2; }
s2: t0 = f; ++f;if (f == l) { goto exit; }if (p(source(f))) { set_link(t1, f); goto s3; }else { goto s2; }
s3: t1 = f; ++f;if (f == l) { goto exit; }if (p(source(f))) { goto s3; }else { set_link(t0, f); goto s2; }
exit: return pair<pair<I, I>, pair<I, I> >(pair<I, I>(h0, t0), pair<I, I>(h1, t1));
}
Stepanov, McJones Elements of Programming March 14, 2008 614 / 880
merge_linkable_nonempty
template <typename S, typename R>requires(ForwardLinker(S) && Readable(IteratorType(S)) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(IteratorType(S)))triple<IteratorType(S), IteratorType(S), IteratorType(S)>merge_linkable_nonempty(IteratorType(S) f0, IteratorType(S) l0,
IteratorType(S) f1, IteratorType(S) l1, R r, S set_link){
typedef IteratorType(S) I;typedef triple<I, I, I> Triple;
entry: I h, t;if (r(source(f1), source(f0))) { h = f1; goto s1; }else { h = f0; goto s0; }
s0: t = f0; ++f0;if (f0 == l0) { set_link(t, f1); return Triple(h, t, l1); }if (r(source(f1), source(f0))) { set_link(t, f1); goto s1; }else { goto s0; }
s1: t = f1; ++f1;if (f1 == l1) { set_link(t, f0); return Triple(h, t, l0); }if (r(source(f1), source(f0))) { goto s1; }else { set_link(t, f0); goto s0; }
}
Stepanov, McJones Elements of Programming March 14, 2008 615 / 880
Interface ofmerge_linkable_nonempty
merge_linkable_nonempty takes two linkable ranges andreturns the beginning and end of the merged linkable range as thefirst and third elements of a tripleIt returns the last-visited iterator in the output range as the secondelement of the triple; this iterator could be used with the followingprocedure to link the range to another linkable range
template <typename I>requires(LinkableForwardIterator(I))
void link_at_end(I f0, I l0, I f1){
// Precondition: f0 , l0I next = successor(f0);while (next != l0) {
f0 = next;++next;
}set_successor(f0, f1);
}
Stepanov, McJones Elements of Programming March 14, 2008 616 / 880
Properties of partition_linkable andmerge_linkable_nonempty
partition_linkable andmerge_linkable_nonempty are stable:every set_link call links to an iterator occurring not earlier in theoriginal list(s)
Stepanov, McJones Elements of Programming March 14, 2008 617 / 880
sort_linkable_n
template <typename S, typename R>requires(ForwardLinker(S) && Readable(IteratorType(S)) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(IteratorType(S)))pair<IteratorType(S), IteratorType(S)>sort_linkable_n(IteratorType(S) f, DistanceType(IteratorType(S)) n,
R r, S set_link){
// Precondition: n > 0typedef IteratorType(S) I;typedef triple<I, I, I> Triple;typedef pair<I, I> Pair;if (n == 1) return Pair(f, successor(f));Pair p1 = sort_linkable_n(f, half_nonnegative(n), r, set_link);Pair p2 = sort_linkable_n(p1.second, n - half_nonnegative(n), r, set_link);Triple t = merge_linkable_nonempty(p1.first, p1.second,
p2.first, p2.second, r, set_link);link_at_end(t.second, t.third, p2.second);return Pair(t.first, p2.second);
}
While the number of operations performed is close to optimal,poor locality of reference limits its usefulness if the linkedstructure does not fit into cache memoryStepanov, McJones Elements of Programming March 14, 2008 618 / 880
Exercise
Exercisesort_linkable_n andmerge_sort_n from Chapter 9 are very similar;figure out how to unify them
Stepanov, McJones Elements of Programming March 14, 2008 619 / 880
sort_linkable_recursive
template <typename S, typename R>requires(ForwardLinker(S) && Readable(IteratorType(S)) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(IteratorType(S)))IteratorType(S) sort_linkable_recursive(IteratorType(S) f, IteratorType(S) l,
R r, S set_link){
if (f == l) return f;else return sort_linkable_n(f, l - f, r, set_link).first;
}
Stepanov, McJones Elements of Programming March 14, 2008 620 / 880
sort_forward_linkable_recursive
template <typename I, typename R>requires(Readable(I) && LinkableForwardIterator(I) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I))I sort_forward_linkable_recursive(I f, I l, R r){
return sort_linkable_recursive(f, l, r, forward_linker<I>());}
Stepanov, McJones Elements of Programming March 14, 2008 621 / 880
Temporarily breaking invariants
A nontrivial invariant involving more than one object will betemporarily violated during an update of those variables
For example, there is a point during the execution of swap(x,y)when both x and y are copies of the same value
Most linked rearrangements do not depend on predecessor linksMaintaining the invariant i = predecessor(successor(i)) requiresextra time; for sorting this requires n log2 n extra timeParameterizing the ForwardLinker used by a linked rearrangementallows the caller to control when to restore the predecessorinvariant
Stepanov, McJones Elements of Programming March 14, 2008 622 / 880
restore_backward_links
template <typename S>requires(BackwardLinker(S))
void restore_backward_links(IteratorType(S) f, IteratorType(S) l, S set_link){
while (f != l) {set_link(f, successor(f));++f;
};}
Stepanov, McJones Elements of Programming March 14, 2008 623 / 880
sort_bidirectional_linkable_recursive
template <int k, typename I, typename R>requires(Readable(I) && LinkableBidirectionalIterator(I) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I))I sort_bidirectional_linkable_recursive(I f, I l, R r){
f = sort_linkable_recursive(f, l, r, forward_linker<I>());restore_backward_links(f, l, backward_linker<I>());return f;
}
Stepanov, McJones Elements of Programming March 14, 2008 624 / 880
Forward versus bidirectional linkable structures
We have demonstrated efficient algorithms for reverse, partition,and merge (hence merge sort) of forward linkable structuresAdding backward links to a forward linkable structure does notappear to improve algorithms on the structure as a wholeHowever, backward links allow constant-time removal andinsertion of elements at an arbitrary locationSince it is the efficiency of insertion and deletion that is often thereason for choosing linkable structures in the first place,bidirectional linkage should be seriously considered
Stepanov, McJones Elements of Programming March 14, 2008 625 / 880
Concept WeakBifurcateCoordinate
DefinitionWeakBifurcateCoordinate(T)⇒
Regular(T)The type functionWeightType(T) is definedInteger(WeightType(T))The unary predicates is_empty, has_left_successor andhas_right_successor are defined on TThe actions step_left and step_right, and the correspondingtransformations left_successor and right_successor, are definedon TAmortizedConstantTime(step_left) ∧
AmortizedConstantTime(step_right)
Stepanov, McJones Elements of Programming March 14, 2008 626 / 880
Implication of weakness of WeakBifurcateCoordinate
Because of the lack of regularity of step_left and step_right, afteran algorithm takes one of the branches, the other branch can neverbe taken
Stepanov, McJones Elements of Programming March 14, 2008 627 / 880
Concept BifurcateCoordinate
DefinitionBifurcateCoordinate(T)⇒
WeakBifurcateCoordinate(T)RegularAction(step_left) ∧ RegularAction(step_right)
Stepanov, McJones Elements of Programming March 14, 2008 628 / 880
Concept BidirectionalBifurcateCoordinate (1 of 3)
Definition (requirements, signatures)BidirectionalBifurcateCoordinate(T)⇒
BifurcateCoordinate(T)The unary predicate has_predecessor is defined on TThe action step_up and the corresponding transformationpredecessor are defined on TRegularAction(step_up)AmortizedConstantTime(step_up)
Stepanov, McJones Elements of Programming March 14, 2008 629 / 880
Concept BidirectionalBifurcateCoordinate (2 of 3)
Definition (signatures)
template <typename C>requires(BidirectionalBifurcateCoordinate(C))
bool is_left_successor(C c){
// Precondition: has_predecessor(c)C parent = predecessor(c);return has_left_successor(parent) && left_successor(parent) == c;
}
template <typename C>requires(BidirectionalBifurcateCoordinate(C))
bool is_right_successor(C c){
// Precondition: has_predecessor(c)C parent = predecessor(c);return has_right_successor(parent) && right_successor(parent) == c;
}
Stepanov, McJones Elements of Programming March 14, 2008 630 / 880
Concept BidirectionalBifurcateCoordinate (3 of 3)
Definition (axioms)For any coordinate x ∈ T :
has_left_successor(x)⇒ predecessor(left_successor(x)) = x
has_right_successor(x)⇒ predecessor(right_successor(x)) = x
has_predecessor(x)⇒is_left_successor(x) ∨ is_right_successor(x)
Stepanov, McJones Elements of Programming March 14, 2008 631 / 880
Reachability of bifurcate coordinates
template <typename C>requires(BifurcateCoordinate(C))
bool reachable(C x, C y){
if (x == y) return true;if (has_left_successor(x)) return reachable(left_successor(x), y);if (has_right_successor(x)) return reachable(right_successor(x), y);return false;
}
template <typename C>requires(BifurcateCoordinate(C))
bool left_reachable(C x, C y){
return has_left_successor(x) && reachable(left_successor(x), y);}
template <typename C>requires(BifurcateCoordinate(C))
bool right_reachable(C x, C y){
return has_right_successor(x) && reachable(right_successor(x), y);}
Stepanov, McJones Elements of Programming March 14, 2008 632 / 880
Properties of bifurcate coordinates
DefinitionThe descendants of a bifurcate coordinate x are all the coordinates ysuch that reachable(x,y)
DefinitionThe descendants of x are acyclic if for all y in the descendants of x,¬left_reachable(y,y) ∧ ¬right_reachable(y,y)
DefinitionThe descendants of x are a tree if they are acyclic and for all y, z in thedescendants of x, ¬left_reachable(y, z) ∨ ¬right_reachable(y, z)
Stepanov, McJones Elements of Programming March 14, 2008 633 / 880
weight_recursive
template <typename C>requires(BifurcateCoordinate(C))
WeightType(C) weight_recursive(C f){
WeightType(C) w(1);if (has_left_successor(f)) w += weight_recursive(left_successor(f));if (has_right_successor(f)) w += weight_recursive(right_successor(f));return w;
}
LemmaThe descendants of x are a finite tree if and only if they are a tree andweight_recursive(x) terminates
Stepanov, McJones Elements of Programming March 14, 2008 634 / 880
height_recursive
template <typename C>requires(BifurcateCoordinate(C))
WeightType(C) height_recursive(C f){
WeightType(C) l(0), r(0);if (has_left_successor(f)) l = height_recursive(left_successor(f));if (has_right_successor(f)) r = height_recursive(right_successor(f));return 1 + max(l, r);
}
Lemmaheight(x) 6 weight(x)
Stepanov, McJones Elements of Programming March 14, 2008 635 / 880
Recursive tree traversal
template <typename C, typename Pre, typename In, typename Post>requires(BifurcateCoordinate(I) &&
UnaryProcedure(Pre) && Domain(Pre) == ValueType(C) &&UnaryProcedure(In) && Domain(Pre) == ValueType(C) &&UnaryProcedure(Post) && Domain(Pre) == ValueType(C))
void traverse_nonempty_recursive(C r, Pre f, In g, Post h){
f(source(r));if (has_left_successor(r))
traverse_nonempty_recursive(left_successor(r), v);g(source(r));if (has_right_successor(r))
traverse_nonempty_recursive(right_successor(r), v);h(source(r));
}
Stepanov, McJones Elements of Programming March 14, 2008 636 / 880
Problems with recursive traversal
It requires stack space proportional to the height of the tree, whichcan be as large as the weight, which is often unacceptable forlarge, unbalanced treesThe interface is not flexible, for example it cannot be used tocompare two treesWe will now develop low-cost iterator-like traversal mechanisms
Stepanov, McJones Elements of Programming March 14, 2008 637 / 880
traverse_step
enum visit { pre, in, post };
template <typename C>requires(BidirectionalBifurcateCoordinate(C))
void traverse_step(C& c, visit& v){
// Precondition: has_predecessor(c) ∨ v , postswitch (v) {case pre:
if (has_left_successor(c)) step_left(c);else v = in;break;
case in:if (has_right_successor(c)) { v = pre; step_right(c); }else v = post;break;
case post:if (is_left_successor(c)) v = in;step_up(c);
}}
Stepanov, McJones Elements of Programming March 14, 2008 638 / 880
Iterative tree traversal
template <typename C, typename Pre, typename In, typename Post>requires(BidirectionalBifurcateCoordinate(C) &&
UnaryProcedure(Pre) && Domain(Pre) == ValueType(C) &&UnaryProcedure(In) && Domain(Pre) == ValueType(C) &&UnaryProcedure(Post) && Domain(Pre) == ValueType(C))
void traverse(C c, Pre f_pre, In f_in, Post f_post){
if (is_empty(c)) return;C root = c;visit v = pre;while (true) {
switch (v) {case pre: f_pre (source(c)); break;case in: f_in (source(c)); break;case post: f_post(source(c)); if (c == root) return;}traverse_step(c, v);
}}
Stepanov, McJones Elements of Programming March 14, 2008 639 / 880
Binary trees and iterators
Traversal through binary trees is fundamentally different thaniterating through a range because of the three directions ofmovementtraverse_step and traverse provide full access to the structure ofa binary treeIt is possible to write iterator adaptors to traverse_step thatprovide sequential access to the nodes of a binary tree in preorder,inorder, or postorder, but such an iterator loses structuralinformationAn iterator adapter whose value type pairs a visit with acoordinate preserves the structural information(See the Projects section at the end of this chapter for more details)
Stepanov, McJones Elements of Programming March 14, 2008 640 / 880
Advantages of bidirectional binary trees
We recommended bidirectional linkage for constant-time insertionand deletion of elementsWith binary trees the situation is different: without predecessorlinks even the simplest traversal algorithms require linear spaceAdding predecessor links allows constant-space traversal, andalso makes possible various rebalancing algorithms
The overhead for the extra link, while 50% in the worst case, istypically much less in practice
Stepanov, McJones Elements of Programming March 14, 2008 641 / 880
Linkable bifurcate coordinates
For linkable iterators, we passed the linking operation as aparameter because of the need to use different linking operations,such as restoring back links after sortFor linkable bifurcate coordinates there does not appear to be aneed for alternate versions of the linking operations, so weoverload them on the concept
Stepanov, McJones Elements of Programming March 14, 2008 642 / 880
Concept LinkableBifurcateCoordinate
DefinitionLinkableBifurcateCoordinate(T)⇒
BifurcateCoordinate(T)Proper mutators set_left_successor and set_right_successortaking two iterators of type T are definedImmediately after set_left_successor(i, j) is performed,left_succesor(i) = j
Immediately after set_right_successor(i, j) is performed,right_succesor(i) = j
Stepanov, McJones Elements of Programming March 14, 2008 643 / 880
Concept LinkableBidirectionalBifurcateCoordinate
DefinitionLinkableBidirectionalBifurcateCoordinate(T)⇒
LinkableBifurcateCoordinate(T)Immediately after set_left_successor(i, j) is performed,has_left_successor(i)⇒ predecessor(j) = i
Immediately after set_right_successor(i, j) is performed,has_right_successor(i)⇒ predecessor(j) = i
Stepanov, McJones Elements of Programming March 14, 2008 644 / 880
Reversing links
traverse_step is an efficient way to traverse via bidirectionalbifurcating coordinates, but requires the predecessor functionWhen the predecessor function is not available and recursive(stack-based) traversal is unacceptable because of unbalancedtrees, there are interesting algorithms that depend on temporarilystoring the link to the predecessor in a link normally containing asuccessor; this assures there is a path back to the root 54
54Link reversal was introduced in: H. Schorr and W.M. Waite. An Efficient andMachine-Independent Procedure for Garbage Collection in Various List Structures.Communications of the ACM, Volume 10, Number 8, August 1967, pages 501-506 (it wasindependetly discovered by L.P. Deutsch). A version without tag bits was publishedin: J.M. Robson. An Improved Algorithm for Traversing Binary Trees WithoutAuxiliary Stack. Information Processing Letters, Volume 2, 1973, pages 12-14; and JosephM. Morris. Traversing Binary Trees Simply and Cheaply, Information Processing Letters,Volume 9, Number 5, 1979, pages 197-200. An ingenious technique of “rotating” thelinks was published in: Gary Lindstrom. Scanning List Structures Without Stack orTag Bits. Information Processing Letters, Volume 2, 1973, pages 47-51; andindependently in: Barry Dwyer. Simple Algorithms for Traversing a Tree Without anAuxiliary Stack. Information Processing Letters, Volume 2, 1974, pages 143-145.
Stepanov, McJones Elements of Programming March 14, 2008 645 / 880
Exclusive-or’ing links
An interesting technique for iterative traversal without extrastorage for predecessor links stores in each successor link theexclusive-or of the predecessor and the corresponding successor55
By additionally requiring that the address of a node’s leftsuccessor is less than the address of its right successor, it ispossible to design a data structure that supports a non-recursivetraversal algorithm that does not modify the tree and uses only aconstant amount of additional storage
55The exclusive-or technique was first applied to binary trees in: Laurent Siklóssy.Fast and Read-only Algorithms for Traversing Trees Without an Auxiliary Stack.Information Processing Letters, Volume 1, 1972, pages 149-152.
Stepanov, McJones Elements of Programming March 14, 2008 646 / 880
Similarity
DefinitionTwo sets S and S ′ of coordinates from the same coordinate type T of acoordinate structure are similar under a given one-to-onecorrespondence if any traversal function maps correspondingcoordinates to corresponding coordinates
This definition can be extended in various waysTo coordinate structures with more than one coordinate typeTo sets of coordinates from different but isomorphic coordinatestructures
Similarity does not depend on the values of the objects pointed toby the coordinates
Algorithms for testing similarity use only traversal functions butnot dereferencing functions
Stepanov, McJones Elements of Programming March 14, 2008 647 / 880
Examples of similarity
ExamplesTwo ranges of iterators [f, l) and [f ′, l ′) are similar if l− f = l ′ − f ′
Two trees are similar if both are empty or they have similar leftand right subtrees
Stepanov, McJones Elements of Programming March 14, 2008 648 / 880
Equivalence
DefinitionTwo sets S and S ′ of coordinates from the same coordinate type T of acoordinate structure represent equivalent value sets under a givenone-to-one correspondence and a given equivalence relation if they aresimilar and dereferencing corresponding coordinates gives equivalentobjects
Stepanov, McJones Elements of Programming March 14, 2008 649 / 880
lexicographic_equivalence
template <typename I0, typename I1, typename R>requires(Readable(I0) && Iterator(I0) &&
Readable(I1) && Iterator(I1) &&ValueType(I0) == ValueType(I1) &&EquivalenceRelation(R) && Domain(R) == ValueType(I0))
bool lexicographic_equivalence(I0 f0, I0 l0, I1 f1, I1 l1, R r){
while (true) {if (f0 == l0 && f1 == l1) return true;if (f0 == l0 || f1 == l1) return false;if (!r(source(f0), source(f1))) return false;++f0; ++f1;
}}
Stepanov, McJones Elements of Programming March 14, 2008 650 / 880
lexicographic_ordering
template <typename I0, typename I1, typename R>requires(Readable(I0) && Iterator(I0) &&
Readable(I1) && Iterator(I1) &&ValueType(I0) == ValueType(I1) &&StrictWeakOrdering(R) && Domain(R) == ValueType(I0))
bool lexicographic_ordering(I0 f0, I0 l0, I1 f1, I1 l1, R r){
while (true) {if (f1 == l1) return false;if (f0 == l0) return true;if (r(source(f0), source(f1))) return true;if (r(source(f0), source(f1))) return false;++f0; ++f1;
}}
This would benefit from having a three-way comparison insteadof an ordering
Stepanov, McJones Elements of Programming March 14, 2008 651 / 880
bifurcate_similarity
template <typename C, typename R>requires(BidirectionalBifurcateCoordinate(C))
bool bifurcate_similarity(C c0, C c1){
if (is_empty(c0) || is_empty(c1)) return is_empty(c0) && is_empty(c1);C root0 = c0; C root1 = c1;visit v0 = pre; visit v1 = pre;while (true) {
traverse_step(c0, v0); traverse_step(c1, v1);if (d0 != d1) return false;if (c0 == root0 && v0 == post) return true;
}}
Stepanov, McJones Elements of Programming March 14, 2008 652 / 880
bifurcate_equivalence
template <typename C, typename R>requires(BidirectionalBifurcateCoordinate(C) &&
EquivalenceRelation(R) && Domain(R) == ValueType(C))bool bifurcate_equivalence(C c0, C c1, R r){
if (is_empty(c0) || is_empty(c1)) return is_empty(c0) && is_empty(c1);C root0 = c0; C root1 = c1;visit v0 = pre; visit v1 = pre;while (true) {
if (v0 == pre && !r(source(c0), source(c1))) return false;traverse_step(c0, v0); traverse_step(c1, v1);if (v0 != v1) return false;if (c0 == root0 && v0 == post) return true;
}}
Stepanov, McJones Elements of Programming March 14, 2008 653 / 880
bifurcate_ordering
template <typename C, typename R>requires(BidirectionalBifurcateCoordinate(C) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(C))bool bifurcate_ordering(C c0, C c1, R r){
if (is_empty(c0) || is_empty(c1)) return is_empty(c0) && !is_empty(c1);C root0 = c0; C root1 = c1;visit v0 = pre; visit v1 = pre;while (true) {
if (v0 == pre) {if (r(source(c0), source(c1))) return true;if (r(source(c1), source(c0))) return false;
}traverse_step(c0, v0); traverse_step(c1, v1);if (v0 != v1) return v0 > v1;if (c0 == root0 && v0 == post) return false;
}}
Stepanov, McJones Elements of Programming March 14, 2008 654 / 880
Conclusions
While iterators are very flexible, we introduced severalgeneralizations:
Linkable iterators allow the topology between iterators to bechanged rather than changing the objects to which the iteratorspointBifurcate coordinates extend from linear to binary tree topology
There is, of course, a wide variety of different coordinate structuresNo computational device (e.g., goto) is harmful per se; it could andshould be used when appropriateAn invariant is associated with a scope, such as a loop, analgorithm, or a datatype
Outside the scope, the invariant must holdInside the scope, it is permissible to violate the invariantProper maintenance of the invariant requires attention toconcurrency and exceptions
Stepanov, McJones Elements of Programming March 14, 2008 655 / 880
Projects
Project 1Design an adapter that converts traverse_step into an iterator, takinginto consideration the following issues:
What should the value type be?What iterator concept should be modeled?What should the limit iterator be to allow traversal of a subtreerooted at an arbitrary coordinate?
Project 2Extend ordering to an arbitrary coordinate structure by defining anappropriate meta-algorithm
Stepanov, McJones Elements of Programming March 14, 2008 656 / 880
Contents I
1 Introduction
2 Foundations
3 Transformations and their orbits
4 Algorithms on algebraic structures
5 Orderings
6 Combining concepts
7 Refining concepts of iterators
Stepanov, McJones Elements of Programming March 14, 2008 657 / 880
Contents II
8 Permutations and rearrangements
9 Rotations
10 Algorithms on increasing ranges
11 Coordinate structures
12 Composite objectsComposite objects, regular types, and containersEquality on composite objectsCollocated typesDistributed typesConstruction
Stepanov, McJones Elements of Programming March 14, 2008 658 / 880
Contents III
Memory allocation and deallocationExplicit construction and destructionListsArraysContainers as elementsRelational conceptsCompatibilityGeneralized copy algorithmsUnderlying typeGeneralized permutation algorithmsMore generalized permutation algorithmsGeneralized partition algorithmsUnderlying iteratorMore generalized partition algorithmsunderlying_compare
Stepanov, McJones Elements of Programming March 14, 2008 659 / 880
Contents IV
More generalized merge and sort algorithmsComposite objects
13 Iterative algorithms for divide-and-conquer
14 Mathematical notation
15 C++ machinery
16 Acknowledgments
17 Index
Stepanov, McJones Elements of Programming March 14, 2008 660 / 880
Definition of composite object (1 of 4)
DefinitionA type T belongs to a concept that is a composite object concept if:
Parts Objects of type T are made up of other objects calledparts
Subparts Subsidiary definition: x is a subpart of y if x is a part of yor x is a subpart of a part of y
Stepanov, McJones Elements of Programming March 14, 2008 661 / 880
Definition of composite object (2 of 4)
DefinitionConnected T has an affiliated coordinate structure
Every object of type T has a distinguished startingaddressIt is possible to reach every part of an object of type Tusing traversals in the coordinate structure, beginningat the starting address
Stepanov, McJones Elements of Programming March 14, 2008 662 / 880
Definition of composite object (3 of 4)
DefinitionNoncircular No object is a subpart of itself
Disjoint Subsidiary definition: Two objects are disjoint if theyhave no subpart in commonIf x and y are objects, either one is a subpart of theother or they are disjoint
Stepanov, McJones Elements of Programming March 14, 2008 663 / 880
Definition of composite object (4 of 4)
DefinitionComplete destruction Destroying an object of type T causes the
destruction of every part of the object (and,therefore, every subpart)
Stepanov, McJones Elements of Programming March 14, 2008 664 / 880
Reflections on composite objects
Like coordinate structure, “composite object” is not a concept buta family of related conceptsNo algorithms can be defined on composite objects as such;meta-algorithms are possible but are outside the scope of this book
Stepanov, McJones Elements of Programming March 14, 2008 665 / 880
Examples of composite objects
ExampleA tuple: traversal via member accessAn array: traversal via random access iteratorA binary tree: traversal via bifurcate coordinateA complex number: traversal via member access
Stepanov, McJones Elements of Programming March 14, 2008 666 / 880
Regularity and composite objects
In Chapter 1 we introduced the concept Regular and noted
The requirements on the element type allowing objects to besorted should be satisfied by arrays themselves, so we can sortan array of arrays
In this chapter we show how to implement a variety of compositeobjects in such a way that they satisfy the requirements for regulartypes
EqualityAssignment (respecting equality)Default construction (making an object assignable/destructable)Copy construction (equivalent to default construction followed byassignment)Total ordering (natural or default)Destruction
Stepanov, McJones Elements of Programming March 14, 2008 667 / 880
Container
DefinitionA container is a composite object whose interpretation is a collection ofthe interpretations of the parts of the container
ExampleA tupleAn arrayA binary tree
Stepanov, McJones Elements of Programming March 14, 2008 668 / 880
The role of containers
Mathematicians construct all of mathematics on a foundationconsisting only of setsThe computer science analog of sets is containers, and they comein many varieties
Stepanov, McJones Elements of Programming March 14, 2008 669 / 880
Coordinate structure of a container
***** This is subsumed by our definition of composite object
DefinitionEach container type defines a corresponding coordinate structure andfunctions giving one or more specific coordinates to initiate traversalsor access
ExampleTuple: member accessArray: random-access iterators and begin and endBinary tree: bifurcate coordinates and root
Stepanov, McJones Elements of Programming March 14, 2008 670 / 880
Equality on containers
DefinitionTwo containers are equal if corresponding parts are equal
ExampleArray equality: use lexicographic_equivalence with equality onthe element typeBinary tree equality: use bifircate_equivalence with equality onthe element type
As with attributes, there may be nonessential parts that are nottaken into account by equality; we will see examples later in thischapter
Stepanov, McJones Elements of Programming March 14, 2008 671 / 880
Structural equality
Recall from Chapter 1 that we follow Leibniz in defining equalityin terms of consistent observed behaviorThere is a related notion of structural equality
DefinitionTwo composite objects are structurally equal if their states are equal
For a container, structural equality is the same as Leibniz equalityObjects that are not structurally equal could still be equal in theLeibniz sense (such objects do not have unique representations)
Stepanov, McJones Elements of Programming March 14, 2008 672 / 880
Structural equality versus equality
ExamplesComplex numbers x+ iy and x ′ + iy ′ are equal if x = x ′ ∧ y = y ′
Complex numbers reiϕ and r ′eiϕ′
are equal if
r = r ′ ∧ϕ ≡ ϕ ′ (mod 2π)
Rational numbers a/b and a ′/b ′ are equal if ab ′ = ba ′
For two objects of the same type, these implications hold
They are identical (same object)⇒They are structurally equal (same state)⇒They are equal (same behavior)⇒They are equivalent (same under some equivalence relation)
Stepanov, McJones Elements of Programming March 14, 2008 673 / 880
Leibniz equality is difficult to implement for certaintypes
ExampleImplementing Leibniz equality for priority queues requires comparingsuccessive values from both queues
This consumes the elements compared, requiring copies to bemade of the two queuesIt requires n logn time
Stepanov, McJones Elements of Programming March 14, 2008 674 / 880
Collocated types
DefinitionA collocated type is one all of whose parts are adjacent in memory
ExampleGiven types T0, . . . , Tk−1, we can construct the heterogeneous typestructT0,...,Tk−1 with useful special cases
pairT0,T1
tripleT0,T1,T2
If all the Ti are the same, we can define the homogeneous typearrayk,T
Stepanov, McJones Elements of Programming March 14, 2008 675 / 880
Properties of collocated types
The correct semantics for assignment, default and copyconstruction, and destruction of a collocated type is elementwise;for equality and ordering it is the lexicographic techniquesdefined in the previous chapter56
Collocated types have constant area, so they can be fully allocatedat compile timeThere are several situations when a collocated type is not optimal
When the area of an object is not constant, that is, its area is fixed ordynamicWhen the object might be frequently moved (e.g., an element of anarray being sorted)
56In some situations C++ automatically generates elementwise constructors,destructor, and assignment for struct types; equality and ordering are neverautomatically generated. Constant-size arrays in C++ are not first-class citizens andone must define a type such as the array_k in Appendix 2 to make them be regular.
Stepanov, McJones Elements of Programming March 14, 2008 676 / 880
Distributed types
DefinitionA distributed type has parts that are not adjacent:
Data representation of the externally-visible behavior of thetype
Connectors traversal through all the dataHeader the origin for all traversal
Padding memory locations included to maintain hardwarealignment or optimize cache behavior
Reserve memory locations reserved for future growthOther locks, software caches, handles for system resources, etc.
Stepanov, McJones Elements of Programming March 14, 2008 677 / 880
Ownership: containers versus iterators
We distinguish pointers representing ownership within acontainer (connectors) from pointers representing intermediatepositions within an algorithm (iterators or coordinates)
Example: list
The functions begin and end, when applied to a list, each return a listiterator
The function successor, when applied to a list iterator, returns anotherlist iterator; there is no function like successor defined on list
The destructor of the listT type is responsible for iterating through allthe elements, destroying each object of type T , and freeing the storage ofits containing list node
Destroying a list iterator has no impact on any list object
Stepanov, McJones Elements of Programming March 14, 2008 678 / 880
list: example of container versus iterator
template <typename T> requires(Regular(T))struct list_node{
T value;list_node* forward_link;// Destructor does nothing
};
template <typename T> requires(Regular(T))struct list_iterator{
list_node<T>* p;// Destructor does nothing
};
template <typename T> requires(Regular(T))struct list{
list_iterator<T> first;// Destructor erases all elements
};
Stepanov, McJones Elements of Programming March 14, 2008 679 / 880
Primary objects
DefinitionA primary object is one that is not a subpart of another object; an objectthat is not primary is secondary
A primary object may have static storage durationA primary object may have automatic storage durationA distributed primary object may have parts with dynamicstorage duration
Stepanov, McJones Elements of Programming March 14, 2008 680 / 880
Construction
Construction of a primary object takes place in static or automaticstorageConstruction of a secondary object takes place in a subpart of aprimary object
Example***** Show code with global object, local object, and insert constructor?????
Stepanov, McJones Elements of Programming March 14, 2008 681 / 880
(Languages without ownership)
Many languages follow the example of Lisp57 use the same typefor both a list as a container and a list iteratorThis design decision means it is not possible for the programmerto explicitly free a list since there might be other variablescontaining list objects with overlapping storage
57John McCarthy. Recursive Functions of Symbolic Expressions and TheirComputation by Machine, Part I. Communications of the ACM, Volume 3, Number 4,1960, pages 184-195
Stepanov, McJones Elements of Programming March 14, 2008 682 / 880
remote
template <typename T>requires(Regular(T))
struct remote{
T* p;remote() : p(0) {}remote(const T& x) : p(new T(x)) { }remote(const remote& x) : p(new T(source(x.p))) { }~remote() { delete p; }friend bool operator==(const remote& x, const remote& y){ return source(x.p) == source(y.p); }friend void swap(remote& x, remote& y) { swap(x.p, y.p); }void operator=(remote& x) { swap(sink(this), x); }friend bool operator<(const remote& x, const remote& y){ return source(x.p) < source(y.p); }
};
Stepanov, McJones Elements of Programming March 14, 2008 683 / 880
Allocation and deallocation
void* raw_allocate(size_t n){
return ::operator new(n);}
template <typename T>requires(Regular(T))
T* allocate(size_t n){
return (T*)raw_allocate(n * sizeof(T));}
void deallocate(void* p){
::operator delete(p);}
Stepanov, McJones Elements of Programming March 14, 2008 684 / 880
construct_copy and destroy
template <typename T>requires(Regular(T))
struct construct_copy{
const T* p;construct_copy(const T& x) : p(&x) {}void operator()(T& q){
new (&q) T(source(p));}
};
template <typename T>requires(Regular(T))
void destroy(T& p){
sink(&p).~T();}
Stepanov, McJones Elements of Programming March 14, 2008 685 / 880
list_node and list_iterator
template <typename T>requires(Regular(T))
struct list_node{
T value;list_node* forward_link;list_node(const T& v, list_node* f) : value(v), forward_link(f) {}
};
template <typename T>requires(Regular(T))
struct list_iterator{
list_node<T>* p;list_iterator() : p(0) {}list_iterator(list_node<T>* p) : p(p) {}
};
Stepanov, McJones Elements of Programming March 14, 2008 686 / 880
Functions for list_iterator
template <typename T> requires(Regular(T))void operator++(list_iterator<T>& i) { i.p = source(i.p).forward_link; }
template <typename T> requires(Regular(T))void set_successor(list_iterator<T> i, list_iterator<T> j){ sink(i.p).forward_link = j.p; }
template <typename T> requires(Regular(T))bool operator==(list_iterator<T> i, list_iterator<T> j){ return i.p == j.p; }
template <typename T> requires(Regular(T))const T& source(list_iterator<T> i) { return source(i.p).value; }
template <typename T> requires(Regular(T))T& sink(list_iterator<T> i) { return sink(i.p).value; }
Stepanov, McJones Elements of Programming March 14, 2008 687 / 880
Type functions for list_iterator
template <typename T>requires(Regular(T))
struct value_type< list_iterator<T> >{
typedef T type;};
template <typename T>requires(Regular(T))
struct distance_type< list_iterator<T> >{
typedef DistanceType(list_node<T>*) type;};
template <typename T>requires(Regular(T))
struct iterator_category< list_iterator<T> >{
typedef forward_iterator_tag category;};
Stepanov, McJones Elements of Programming March 14, 2008 688 / 880
Insert functions for list_node
template <typename T, typename F>requires(Regular(T) && ConstructorObject(F) && Domain(F) == T)
list_iterator<T> insert_construct(list_iterator<T> j, F f){
list_iterator<T> i(allocate< list_node<T> >(1));f(sink(i));set_successor(i, j);return i;
}
template <typename T>requires(Regular(T))
list_iterator<T> insert(list_iterator<T> i, const T& x){
return insert_construct(i, construct_copy<T>(x));}
Stepanov, McJones Elements of Programming March 14, 2008 689 / 880
More insert functions for list_node
template <typename T, typename F>requires(Regular(T) && ConstructorObject(F) && Domain(F) == T)
list_iterator<T> insert_after_construct(list_iterator<T> i, F f){
list_iterator<T> j = insert_construct(successor(i), f);set_successor(i, j);return j;
}
template <typename T>requires(Regular(T))
list_iterator<T> insert_after(list_iterator<T> i, const T& x){
return insert_after_construct(i, construct_copy<T>(x));}
Stepanov, McJones Elements of Programming March 14, 2008 690 / 880
list_insert_after_iterator
template <typename T>requires(Regular(T))
struct list_insert_after_iterator{
list_iterator<T> h;list_iterator<T> t;list_insert_after_iterator() { }list_insert_after_iterator(list_iterator<T> h, list_iterator<T> t)
: h(h), t(t) { }void operator=(const T& x){
if (t == list_iterator<T>()) {h = insert(h, x);t = h;
} elset = insert_after(t, x);
}void operator++() { }
};
Stepanov, McJones Elements of Programming March 14, 2008 691 / 880
(Equality for list_insert_after_iterator)
template <typename T>requires(Regular(T))
bool operator==(const list_insert_after_iterator<T>& i,const list_insert_after_iterator<T>& j)
{return i.h == j.h && i.t == j.t;
}
Stepanov, McJones Elements of Programming March 14, 2008 692 / 880
Type functions for list_insert_after_iterator
template <typename T>requires(Regular(T))
struct value_type< list_insert_after_iterator<T> >{
typedef T type;};
template <typename T>requires(Regular(T))
struct distance_type< list_insert_after_iterator<T> >{
typedef DistanceType(list_iterator<T>) type;};
template <typename T>requires(Regular(T))
struct iterator_category< list_insert_after_iterator<T> >{
typedef iterator_tag category;};
Stepanov, McJones Elements of Programming March 14, 2008 693 / 880
Erase functions for list_node
template <typename T>requires(Regular(T))
list_iterator<T> erase_first(list_iterator<T> i){
list_iterator<T> j = successor(i);destroy(sink(i));deallocate(i.p);return j;
}
template <typename T>requires(Regular(T))
void erase_after(list_iterator<T> i){
set_successor(i, erase_first(successor(i)));}
Stepanov, McJones Elements of Programming March 14, 2008 694 / 880
list container
template <typename T>requires(Regular(T))
struct list{
list_iterator<T> first;list() : first(0) {}template <typename I>
requires(Readable(I) && Iterator(I) && ValueType(I) == T)list(I f, I l);list(const list& x);~list();friend void swap(list& x, list& y) { swap(x.first, y.first); }void operator=(list x) { swap(sink(this), x); }
};
Stepanov, McJones Elements of Programming March 14, 2008 695 / 880
Type functions for list
***** Really need a concept Container ?????
DefinitionThe type function IteratorType(C) is defined on any container type C;it returns a type such that Iterator(IteratorType(C))
Stepanov, McJones Elements of Programming March 14, 2008 696 / 880
Type functions for list
template <typename T>requires(Regular(T))
struct iterator_type< list<T> >{
typedef list_iterator<T> type;};
Stepanov, McJones Elements of Programming March 14, 2008 697 / 880
Functions for list
template <typename T> requires(Regular(T))IteratorType(list<T>) begin(const list<T>& x){
return x.first;}
template <typename T> requires(Regular(T))IteratorType(list<T>) end(const list<T>& x){
return list_iterator<T>();}
template <typename C> requires(Container(C))DistanceType(IteratorType(C)) size(const C& x){
return end(x) - begin(x);}
template <typename C>requires(Container(C))
bool is_empty(const C& x){
return begin(x) == end(x);}
Stepanov, McJones Elements of Programming March 14, 2008 698 / 880
Constructors for list
template <typename T, typename I>requires(Regular(T) && Iterator(I) &&
ValueType(I) == T)void insert(list<T>& x, IteratorType(list<T>) i, I f, I l){
x.first = copy(f, l, list_insert_after_iterator<T>(begin(x), i)).h;}
template <typename T>template <typename I>
requires(Regular(T) && Readable(I) && Iterator(I) &&ValueType(I) == T)
list<T>::list(I f, I l) : first(0){
insert(sink(this), end(sink(this)), f, l);}
template <typename T>requires(Regular(T))
list<T>::list(const list& x) : first(0){
insert(sink(this), end(sink(this)), begin(x), end(x));}
Stepanov, McJones Elements of Programming March 14, 2008 699 / 880
Destructor for list
template <typename T>requires(Regular(T))
void pop_all(list<T>& x){
while (!is_empty(x))x.first = erase_first(begin(x));
}
template <typename T>requires(Regular(T))
list<T>::~list(){
pop_all(sink(this));}
Stepanov, McJones Elements of Programming March 14, 2008 700 / 880
Equality and less than for list
template <typename T>requires(Regular(T))
struct equality{
bool operator()(const T& x, const T& y) { return x == y; }};
template <typename T>requires(Regular(T))
bool operator==(const list<T>& x, const list<T>& y){
return lexicographic_equivalence(begin(x), end(x), begin(y), end(y),equality<T>());
}
template <typename T>requires(Regular(T))
bool operator<(const list<T>& x, const list<T>& y){
return lexicographic_ordering(begin(x), end(x), begin(y), end(y), less<T>());}
These definitions work for any container *****Stepanov, McJones Elements of Programming March 14, 2008 701 / 880
partition for list
template <typename T, typename P>requires(Regular(T) && UnaryPredicate(P) && Domain(P) == T)
void partition_list(list<T>& x, list<T>& y, P p){
typedef IteratorType(list<T>) I;pair< pair<I, I>, pair<I, I> > pp =
partition_linkable(begin(x), end(x), p, forward_linker<I>());x.first = pp.m0.m0;if (pp.m1.m0 != end(x)) {
set_successor(pp.m1.m1, begin(y));y.first = pp.m1.first;
}}
Stepanov, McJones Elements of Programming March 14, 2008 702 / 880
merge for list
template <typename T, typename R>requires(Regular(T) && StrictWeakOrdering(R) && Domain(R) == T)
void merge(list<T>& x, list<T>& y, R r){
if (is_empty(y)) return;if (is_empty(x)) x = y;else
x.first = merge_linkable_nonempty(begin(x), end(x), begin(y), end(y),r, forward_linker<IteratorType(list<T>)>()).m0;
y.first = end(y);}
Stepanov, McJones Elements of Programming March 14, 2008 703 / 880
sort for list
template <typename T, typename R>requires(Regular(T) && StrictWeakOrdering(R) && Domain(R) == T)
void sort_recursive(list<T>& x, R r){
x.first = sort_forward_linkable_recursive(begin(x), end(x), r);}
template <typename T, typename R>requires(Regular(T) && StrictWeakOrdering(R) && Domain(R) == T)
void sort(list<T>& x, R r){
x.first = sort_forward_linkable<32>(begin(x), end(x), r);}
Stepanov, McJones Elements of Programming March 14, 2008 704 / 880
array_header
template <typename T>requires(Regular(T))
struct array_header{
T* m;T* l;T a;
};
Invariants, where f = &a:[f,m) are constructed elements[m, l) are unconstructed (reserved) elements
Stepanov, McJones Elements of Programming March 14, 2008 705 / 880
allocate_array_header
template <typename T>requires(Regular(T))
array_header<T>* allocate_array(DistanceType(T*) n){
typedef array_header<T>* P;size_t bsize = size_t(predecessor(n)) * sizeof(T);P p = P(raw_allocate(sizeof(array_header<T>) + bsize));T* f = &sink(p).a;sink(p).m = f;sink(p).l = f + n;return p;
}
Stepanov, McJones Elements of Programming March 14, 2008 706 / 880
deallocate_array_header
template <typename T>requires(Regular(T))
void deallocate_array(array_header<T>* p){
deallocate(p);}
Stepanov, McJones Elements of Programming March 14, 2008 707 / 880
array container
template <typename T>requires(Regular(T))
struct array{
typedef DistanceType(IteratorType(array<T>)) N;array_header<T>* p;array() : p(0) {}array(N c); // size is 0 and capacity is carray(N s, N c, const T& x); // size is s, capacity is c, all elements equal to xtemplate <typename I>
requires(Readable(I) && Iterator(I) && ValueType(I) == T)array(I f, I l);array(const array& x);~array();friend void swap(array& x, array& y) { swap(x.p, y.p); }void operator=(array x) { swap(sink(this), x); }
};
template <typename T>requires(Regular(T))
void reserve(array<T>& x, DistanceType(IteratorType(array<T>)) n);
Stepanov, McJones Elements of Programming March 14, 2008 708 / 880
Type functions for array
template <typename T>requires(Regular(T))
struct iterator_type< array<T> >{
typedef T* type;};
Stepanov, McJones Elements of Programming March 14, 2008 709 / 880
Functions for array
template <typename T>requires(Regular(T))
IteratorType(array<T>) begin(const array<T>& x){
if (x.p == 0) return IteratorType(array<T>)(0);return IteratorType(array<T>)(&source(x.p).a);
}
template <typename T>requires(Regular(T))
IteratorType(array<T>) end(const array<T>& x){
if (x.p == 0) return IteratorType(array<T>)(0);return IteratorType(array<T>)(source(x.p).m);
}
template <typename T>requires(Regular(T))
IteratorType(array<T>) end_of_storage(const array<T>& x){
if (x.p == 0) return IteratorType(array<T>)(0);return IteratorType(array<T>)(source(x.p).l);
}
Stepanov, McJones Elements of Programming March 14, 2008 710 / 880
More functions for array
template <typename T>requires(Regular(T))
DistanceType(IteratorType(array<T>)) capacity(const array<T>& x){
return end_of_storage(x) - begin(x);}
template <typename T>requires(Regular(T))
bool is_full(const array<T>& x){
return end(x) == end_of_storage(x);}
Stepanov, McJones Elements of Programming March 14, 2008 711 / 880
push_construct
template <typename T, typename F>requires(Regular(T) && ConstructorObject(F) && Domain(F) == T)
T& push_construct(array<T>& x, F f){
typedef DistanceType(IteratorType(array<T>)) N;N n = size(x);if (n == capacity(x))
reserve(x, max(N(1), n + n));f(sink(source(x.p).m));++sink(x.p).m;return sink(end(x) - 1);
}
Stepanov, McJones Elements of Programming March 14, 2008 712 / 880
push
template <typename T>requires(Regular(T))
T& push(array<T>& x, const T& y){
return push_construct(x, construct_copy<T>(y));}
Stepanov, McJones Elements of Programming March 14, 2008 713 / 880
array_push_iterator
template <typename T>requires(Regular(T))
struct array_push_iterator{
array<T>* p;array_push_iterator() {}array_push_iterator(array<T>& x) : p(&x) {}friend bool operator==(array_push_iterator x, array_push_iterator y){
return x.p == y.p;}friend void operator++(array_push_iterator& x) {}void operator=(const T& x){
push(sink(p), x);}
};
Stepanov, McJones Elements of Programming March 14, 2008 714 / 880
Type functions for array_push_iterator
template <typename T>requires(Regular(T))
struct value_type< array_push_iterator<T> >{
typedef T type;};
template <typename T>requires(Regular(T))
struct distance_type< array_push_iterator<T> >{
typedef DistanceType(IteratorType(array<T>)) type;};
template <typename T>requires(Regular(T))
struct iterator_category< array_push_iterator<T> >{
typedef iterator_tag category;};
Stepanov, McJones Elements of Programming March 14, 2008 715 / 880
push for a range
template <typename T, typename I>requires(Regular(T) &&
Readable(I) && Iterator(I) && ValueType(I) == T)void push(array<T>& x, I f, I l){
copy(f, l, array_push_iterator<T>(x));}
Stepanov, McJones Elements of Programming March 14, 2008 716 / 880
Pop functions for array
template <typename T>requires(Regular(T))
void pop(array<T>& x){
--sink(x.p).m;destroy(sink(source(x.p).m));if (is_empty(x)) {
deallocate_array(x.p);x.p = 0;
}}
template <typename T>requires(Regular(T))
void pop_all(array<T>& x){
while (!is_empty(x)) pop(x);}
Stepanov, McJones Elements of Programming March 14, 2008 717 / 880
Constructors for array
template <typename T>requires(Regular(T))
array<T>::array(DistanceType(IteratorType(array<T>)) c) :p(allocate_array<T>(c))
{}
template <typename T>requires(Regular(T))
array<T>::array(DistanceType(IteratorType(array<T>)) s,DistanceType(IteratorType(array<T>)) c,const T& x) :
p(allocate_array<T>(c)){
while (s != N(0)) { push(sink(this), x); --s; }}
Stepanov, McJones Elements of Programming March 14, 2008 718 / 880
More constructors for array
template <typename T>template <typename I>
requires(Regular(T) && Readable(I) && Iterator(I) && ValueType(I) == T)array<T>::array(I f, I l) : p(0){
push<T, I>(sink(this), f, l);}
template <typename T>requires(Regular(T))
array<T>::array(const array& x) : p(allocate_array<T>(size(x))){
push<T>(sink(this), begin(x), end(x));}
Stepanov, McJones Elements of Programming March 14, 2008 719 / 880
Destructor for array
template <typename T>requires(Regular(T))
array<T>::~array(){
pop_all(sink(this));}
Stepanov, McJones Elements of Programming March 14, 2008 720 / 880
reserve
template <typename T>requires(Regular(T))
void reserve(array<T>& x, DistanceType(IteratorType(array<T>)) n){
if (n < size(x) || n == capacity(x)) return;array<T> tmp(n);copy(begin(x), end(x), array_push_iterator<T>(tmp));swap(tmp, x);
}
Stepanov, McJones Elements of Programming March 14, 2008 721 / 880
Equality and less than for array
template <typename T>requires(Regular(T))
bool operator==(const array<T>& x, const array<T>& y){
return lexicographic_equivalence(begin(x), end(x), begin(y), end(y),equality<T>());
}
template <typename T>requires(Regular(T))
bool operator<(const array<T>& x, const array<T>& y){
return lexicographic_ordering(begin(x), end(x), begin(y), end(y), less<T>());}
Stepanov, McJones Elements of Programming March 14, 2008 722 / 880
insert for array
template <typename T, typename I>requires(Regular(T) &&
Readable(I) && Iterator(I) && ValueType(I) == T)void insert(array<T>& x, IteratorType(array<T>) i, I f, I l){
DistanceType(IteratorType(array<T>)) o_f = i - begin(x);DistanceType(IteratorType(array<T>)) o_m = end(x) - begin(x);push(x, f, l);rotate(begin(x) + o_f, begin(x) + o_m, end(x));
}
Stepanov, McJones Elements of Programming March 14, 2008 723 / 880
sort for array
template <typename T, typename R>requires(Regular(T) && StrictWeakOrdering(R) && Domain(R) == T)
void sort(array<T>& x, R r){
typedef DistanceType(IteratorType(array<T>)) N;N n = size(x) / N(10);array<T> buffer(n, n, T());stable_sort_n(begin(x), size(x), begin(buffer), n, r);
}
Stepanov, McJones Elements of Programming March 14, 2008 724 / 880
Introduction
We have presented algorithms and containers that work welltogether when the elements of the containers are built-in types orC-style structsHowever performance suffers when the elements are themselveslarger containers
Sorting an array of arrays
In this section we present a solution to this problem using thenotions of underlying type and type compatibilityWe rewrite versions of various algorithms from chapters 6-10using this approach and define the underlying type for thecontainers in chapter 11
Stepanov, McJones Elements of Programming March 14, 2008 725 / 880
Relational concepts (from Chapter 1)
DefinitionA relational concept is a concept defined on two types
Stepanov, McJones Elements of Programming March 14, 2008 726 / 880
Concept SameGenus (from Chapter 1)
DefinitionSameGenus(T , T ′)⇒
The genus of T and the genus of T ′ are equal
ExampleSameGenus(int, long)
Stepanov, McJones Elements of Programming March 14, 2008 727 / 880
Concept PartiallyCompatible (1 of 3) (from Chapter 6)
Definition (requirements, signatures)PartiallyCompatible(T ,U)⇒
SameGenus(T ,U)
The partial equality predicates (=) on T ×U and U× T are definedThe (potentially implicit) unary predicates is_representableU(T)
and is_representableT (U) are definedThe partial assignments (←) on T ×U and U× T are definedThe morphisms U(T) and T(U) are defined
Stepanov, McJones Elements of Programming March 14, 2008 728 / 880
Concept PartiallyCompatible (2 of 3) (from Chapter 6)
Definition (definition spaces)PartiallyCompatible(T ,U)⇒
For all t ∈ T and for all u ∈ U,is_representableU(t) ∧ is_representableT (u)⇒is_defined(=, t,u) ∧ is_defined(=,u, t)For all t ∈ T and for all u ∈ U,is_representableT (u)⇒ is_defined(←, t,u)
For all t ∈ T and for all u ∈ U,is_representableU(t) ∧ is_representableT (u)⇒is_defined(U(·), t) ∧ is_defined(T(·),u)
Stepanov, McJones Elements of Programming March 14, 2008 729 / 880
Concept PartiallyCompatible (3 of 3) (from Chapter 6)
Definition (axioms)PartiallyCompatible(T ,U)⇒
Where = is defined, it is true if and only if its arguments have thesame interpretation← depends on the type but not the value of its first argument;where it is defined, it makes the first argument equal to the secondwithout modifying the secondWhere the morphisms U(T) and T(U) are defined, each constructsa new element of the target type that has the same interpretationas its argument
Stepanov, McJones Elements of Programming March 14, 2008 730 / 880
Properties of PartiallyCompatible types (from Chapter 6)
LemmaPartiallyCompatible(T ,U)⇒ PartiallyCompatible(U, T)
LemmaIf PartiallyCompatible(T ,U), PartiallyCompatible(U,V),PartiallyCompatible(T ,V), then (∀t ∈ T)(∀u ∈ U)(∀v ∈ V):
t = u⇒ u = t
(t = u∧ u = v)⇒ t = v
t← u⇒ t = u
T t(u)⇒ t = u
(Implication between a statement and a predicate means thepredicate holds immediately after execution of the statement)
Stepanov, McJones Elements of Programming March 14, 2008 731 / 880
Concept Compatible (from Chapter 6)
DefinitionCompatible(T ,U)⇒
PartiallyCompatible(T ,U)
For all t ∈ T and for all u ∈ U,is_representableU(t) ∧ is_representableT (u)
Stepanov, McJones Elements of Programming March 14, 2008 732 / 880
Type requirements of copy (from Chapter 6)
#if 0 // *****
template <typename I, typename O>requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&
PartiallyCompatible(ValueType(I), ValueType(O)))O copy(I f, I l, O r);
Readable(I) ensures source is definedIterator(I) ensures ++ is definedWritable(O) ensures sink is definedIterator(O) ensures ++ is definedPartiallyCompatible(ValueType(I),ValueType(O)) ensuresassignment is defined
Stepanov, McJones Elements of Programming March 14, 2008 733 / 880
Preconditions and postconditions of copy (fromChapter 6)
template <typename I, typename O>requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&
PartiallyCompatible(ValueType(I), ValueType(O)))O copy(I f, I l, O r);
Preconditions The input range must be readableThe output range must be writable and of at leastthe size of the input rangeEach value of the input range must berepresentable in the output value typeIf the ith iterator in the input range is aliased to thejth iterator of the output, then i 6 j (informally,every value is used before it is overwritten)
Postconditions The sequence of values in the output range isequal to the sequence of original values in theinput range
Stepanov, McJones Elements of Programming March 14, 2008 734 / 880
Implementation of copy (from Chapter 6)
template <typename I, typename O>requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&
PartiallyCompatible(ValueType(I), ValueType(O)))O copy(I f, I l, O r){
while (f != l) {sink(r) = source(f);++f;++r;
}return r;
}
We return the end of the output range because it might not beknown to the caller, who might find it useful
It is worth the small constant time to return it
Stepanov, McJones Elements of Programming March 14, 2008 735 / 880
Example using copy (from Chapter 6)
template <typename N, typename O>requires(Integer(N) && Writable(O) && Iterator(O) &&
PartiallyCompatible(N, ValueType(O)))O iota(N n, O r) // like APL ι{
return copy(N(0), n, r);}
Stepanov, McJones Elements of Programming March 14, 2008 736 / 880
copy_bounded (from Chapter 6)
template <typename I, typename O>requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&
PartiallyCompatible(ValueType(I), ValueType(O)))pair<I, O> copy_bounded(I f, I l, O r_f, O r_l){
while (f != l && r_f != r_l) {sink(r_f) = source(f);++f;++r_f;
}return pair<I, O>(f, r_f);
}
While the ends of both ranges are known to the caller, returningthe pair allows determining which range is smaller and where inthe larger range copying stopped
Stepanov, McJones Elements of Programming March 14, 2008 737 / 880
copy_n (from Chapter 6)
template <typename I, typename O>requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&
PartiallyCompatible(ValueType(I), ValueType(O)))pair<I, O> copy_n(I f, DistanceType(I) n, O r){
typedef DistanceType(I) N;while (n != N(0)) {
sink(r) = source(f);++f;++r;n = n - N(1);
}return pair<I, O>(f, r);
}
Stepanov, McJones Elements of Programming March 14, 2008 738 / 880
copy_k (from Chapter 6)
template <typename I, typename O, int k>requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&
PartiallyCompatible(ValueType(I), ValueType(O)))void copy_k(I& f, O& r){
copy_k<k - 1>(f, r);sink(r) = source(f);++f; ++r;
}
template <typename I, typename O>void copy_k<0>(I&, O&) { }
Stepanov, McJones Elements of Programming March 14, 2008 739 / 880
copy_n_unrolled (from Chapter 6)
template <typename I, typename O>requires(Readable(I) && Iterator(I) && Writable(O) && Iterator(O) &&
PartiallyCompatible(ValueType(I), ValueType(O)))pair<I, O> copy_n_unrolled(I f, DistanceType(I) n, I r){
typedef DistanceType(I) N;const int k = 4; // unroll factorwhile (n >= N(k)) {
copy_k<k>(f, r);n = n - N(k);
}return copy_n(f, n, r);
}
Stepanov, McJones Elements of Programming March 14, 2008 740 / 880
copy_backward (from Chapter 6)
The main difference from copy is the aliasing precondition: if theith iterator in the input range is aliased to the jth iterator of theoutput, then j 6 i
template <typename I, typename O>requires(Readable(I) && BidirectionalIterator(I) &&
Writable(O) && BidirectionalIterator(O) &&PartiallyCompatible(ValueType(I), ValueType(O)))
O copy_backward(I f, I l, O r){
while (f != l) {--l;--r;sink(r) = source(l);
}return r;
}
Stepanov, McJones Elements of Programming March 14, 2008 741 / 880
copy_k_indexed (from Chapter 6)
template <typename I, typename O, int k>requires(Readable(I) && IndexedIterator(I) &&
Writable(O) && IndexedIterator(O) &&PartiallyCompatible(ValueType(I), ValueType(O)))
void basic_copy_k_indexed(I f, O r){
basic_copy_k<k - 1>()(f, r);sink(r + (k - 1)) = source(f + (k - 1));
}
template <typename I, typename O>void basic_copy_k_indexed<0>(I, O) { }
template <typename I, typename O, int k>requires(Readable(I) && IndexedIterator(I) &&Writable(O) && IndexedIterator(O) && ValueType(I) == ValueType(O))
void copy_k_indexed(I& f, O& r){
basic_copy_k<k>(f, r);f += k; r += k;
}
Stepanov, McJones Elements of Programming March 14, 2008 742 / 880
copy_n_unrolled_indexed (from Chapter 6)
template <typename I, typename O>requires(Readable(I) && IndexedIterator(I) &&
Writable(O) && IndexedIterator(O) &&PartiallyCompatible(ValueType(I), ValueType(O)))
pair<I, O> copy_n_unrolled_indexed(I f, DistanceType(I) n, O r){
typedef DistanceType(I) N;const int k = 4; // unroll factorwhile (n >= N(k)) {
copy_k_indexed<k>(f, r);n = n - N(k);
}return copy_n(f, n, r);
}
Stepanov, McJones Elements of Programming March 14, 2008 743 / 880
copy_parallel for disjoint ranges (from Chapter 6)
The main difference from copy is the aliasing precondition: theinput and output ranges do not aliasA special forall executes its body for all values of its iterationvariable in arbitrary order, possibly concurrently
template <typename I, typename O>requires(Readable(I) && IndexedIterator(I) &&
Writable(O) && IndexedIterator(O) &&PartiallyCompatible(ValueType(I), ValueType(O)))
void copy_parallel(I f, DistanceType(I) n, O r){
typedef DistanceType(I) N;forall(N i, N(0), n) {
sink(r + i) = source(f + i);}
}
Stepanov, McJones Elements of Programming March 14, 2008 744 / 880
copy_backward_n (from Chapter 6)
copy_backward is not realizable for indexed iteratorscopy_backward_n, which is often just as useful, is realizable
template <typename I, typename O>requires(Readable(I) && IndexedIterator(I) &&
Writable(O) && IndexedIterator(O) &&PartiallyCompatible(ValueType(I), ValueType(O)))
O copy_backward_n(I f, DistanceType(I) n, O r){
typedef DistanceType(I) N;while (n != N(0)) {
n = n - N(1);sink(r + n) = source(f + n);
}return r;
}
Stepanov, McJones Elements of Programming March 14, 2008 745 / 880
copy_from_segmented (from Chapter 6)
template <typename I, typename O>requires(Readable(I) && SegmentedIterator(I) &&
Writable(O) && Iterator(O) &&PartiallyCompatible(ValueType(I), ValueType(O)))
O copy_from_segmented(I f, I l, O r){
while (f + end(f) != l + end(l)) {// f and l are in different segmentsr = copy(begin(f), end(f), r);f = f + end(f);
}// f and l are in the same segmentreturn copy(begin(f), begin(l), r);
}
Stepanov, McJones Elements of Programming March 14, 2008 746 / 880
Underlying type (from Chapter 7)
DefinitionFor any type T , the underlying type U is an affiliated type satisfying:
T and U are compatibleReferences with value types T and Umay be reinterpretively castinto each otherConstruction of type U and assignment to type U never throw anexceptionAn object of type Umay only be used to hold temporary valueswhile implementing a rearrangement of a range of T objectsA reference to an object of type Umay be reinterpretively cast to areference to T and passed as a const T& parameter
We denote the underlying type of T as UnderlyingType(T)
Stepanov, McJones Elements of Programming March 14, 2008 747 / 880
Motivation for underlying type (from Chapter 7)
The implementation of types T and U could exploit this restrictionto save timeAs we shall see when implementing containers, underlying typeallows multiple headers to point to the same data whileperforming rearrangements
Stepanov, McJones Elements of Programming March 14, 2008 748 / 880
UnderlyingType type function (from Chapter 7)
template <typename T> requires(Regular(T))struct underlying_type{
typedef T type; // default};
#define UnderlyingType(T) typename underlying_type<T>::type
template <typename I> requires(Readable(I) && Iterator(I))const UnderlyingType(ValueType(I))&underlying_source(I x){
return reinterpret_cast<const UnderlyingType(ValueType(I))&>(source(x));
}
template <typename I> requires(Writable(I) && Iterator(I))UnderlyingType(ValueType(I))&underlying_sink(I x){
return reinterpret_cast<UnderlyingType(ValueType(I))&>(sink(x));
}
Stepanov, McJones Elements of Programming March 14, 2008 749 / 880
cycle_2 (from Chapter 7)
template <typename I>requires(Mutable(I) && Iterator(I))
void cycle_2(I x, I y){
UnderlyingType(ValueType(I)) t = underlying_source(x);underlying_sink(x) = underlying_source(y);underlying_sink(y) = t;
}
Stepanov, McJones Elements of Programming March 14, 2008 750 / 880
Underlying type and invariants (from Chapter 7)
To be correct, it is sufficient for a sequence of code usingUnderlyingType(T) to:
Be within a critical section: that is, avoid concurrent accessAvoid any exceptions being thrownRestore the class invariants of T by the end of the sequence
ReflectionDisciplined violation of invariants to enhance performance is perfectlylegitimate
Stepanov, McJones Elements of Programming March 14, 2008 751 / 880
cycle_left_3 (from Chapter 7)
template <typename I>requires(Mutable(I) && Iterator(I))
void cycle_left_3(I x, I y, I z){
UnderlyingType(ValueType(I)) t = underlying_source(x);underlying_sink(x) = underlying_source(y);underlying_sink(y) = underlying_source(z);underlying_sink(z) = t;
}
Stepanov, McJones Elements of Programming March 14, 2008 752 / 880
swap (from Chapter 7)
We can interchange the contents of two variables usingunderlying type
#define UnderlyingRef(T) reinterpret_cast<UnderlyingType(T)&>
template <typename T>requires(Regular(T))
void swap(T& x, T& y){
UnderlyingType(T) tmp;tmp = UnderlyingRef(T)(x);UnderlyingRef(T)(x) = UnderlyingRef(T)(y);UnderlyingRef(T)(y) = tmp;
}
Stepanov, McJones Elements of Programming March 14, 2008 753 / 880
do_cycle_from (from Chapter 7)
template <typename I, typename P>requires(Mutable(I) && ForwardIterator(I) &&
Transformation(P) && Domain(P) == I)void do_cycle_from(I i, P p){
// Precondition: p is a from-permutation of the range containing iUnderlyingType(ValueType(I)) t = underlying_source(i);I f = i;I n = p(i);while (n != i) {
underlying_sink(f) = underlying_source(n);f = n;n = p(n);
}underlying_sink(f) = t;
}
Stepanov, McJones Elements of Programming March 14, 2008 754 / 880
swap_ranges_n (from Chapter 7)
template <typename I1, typename I2, typename N>requires(Mutable(I1) && Iterator(I1) &&
Mutable(I2) && Iterator(I2) &&PartiallyCompatible(ValueType(I1), ValueType(I2)) &&Integer(N))
pair<I1, I2> swap_ranges_n(I1 f1, I2 f2, N n){
while (n != N(0)) {cycle_2(f1, f2);++f1;++f2;n = n - N(1);
};return pair<I1, I2>(f1, f2);
}
Stepanov, McJones Elements of Programming March 14, 2008 755 / 880
reverse_n_with_buffer (from Chapter 7)
template <typename I, typename B>requires(Mutable(I) && ForwardIterator(I) &&
Mutable(B) && BidirectionalIterator(B) &&UnderlyingType(ValueType(I)) == ValueType(B))
I reverse_n_with_buffer(I f, DistanceType(I) n, B b){
return reverse_copy_n(copy_n(f, n, b), n, f);}
Stepanov, McJones Elements of Programming March 14, 2008 756 / 880
reverse_n_adaptive (from Chapter 7)
template <typename I, typename B>requires(Mutable(I) && ForwardIterator(I) &&
Mutable(B) && BidirectionalIterator(B) &&UnderlyingType(ValueType(I)) == ValueType(B))
I reverse_n_adaptive(I f, DistanceType(I) n, B b, DistanceType(I) n_b){
typedef DistanceType(I) N;const N h = n / N(2);const N r = n - N(2) * h;if (h == N(0))
return f + n;if (n <= n_b)
return reverse_n_with_buffer(f, n, b);I m = reverse_n_adaptive(f, h, b, n_b);m += r;I l = reverse_n_adaptive(m, h, b, n_b);swap_ranges_n(f, m, h);return l;
}
Stepanov, McJones Elements of Programming March 14, 2008 757 / 880
do_fused_cycles_from_with_buffer (from Chapter8)
template <typename I, typename P, typename B>requires(Mutable(I) && ForwardIterator(I) &&
Transformation(P) && Domain(P) == I &&Mutable(B) && ForwardIterator(B) &&UnderlyingType(ValueType(I)) == ValueType(B))
void do_fused_cycles_from_with_buffer(I i, P p, B b, DistanceType(I) n){
// Precondition: p is a from-permutation on the range containing icopy_n(i, b, n);I f = i;I next = p(i);while (next != i) {
copy_n(next, f, n);f = next;next = p(next);
}copy_n(b, f, n);
}
Stepanov, McJones Elements of Programming March 14, 2008 758 / 880
rotate_indexed_helper_fused (from Chapter 8)
template <typename I, typename P>requires(Mutable(I) && IndexedIterator(I) &&
Transformation(P) && Domain(P) == I)I rotate_indexed_helper_fused(I f, I m, I l, P p){
// Precondition: p is a from-permutation on [f, l)typedef DistanceType(I) N;const N fusion_factor(16);UnderlyingType(ValueType(I)) buffer[fusion_factor];N d = gcd(m - f, l - m);N i(0);while (i + fusion_factor < d) {
do_fused_cycles_from_with_buffer(f + i, p, buffer, fusion_factor);i = i + fusion_factor;
}do_fused_cycles_from_with_buffer(f + i, p, buffer, d - i);return f + (l - m);
};
Stepanov, McJones Elements of Programming March 14, 2008 759 / 880
swap_ranges (from Chapter 8)
template <typename I1, typename I2>requires(Mutable(I1) && Iterator(I1) &&
Mutable(I2) && Iterator(I2) &&PartiallyCompatible(ValueType(I1), ValueType(I2)))
pair<I1, I2> swap_ranges(I1 f1, I1 l1, I2 f2, I2 l2){
while (f1 != l1 && f2 != l2) {cycle_2(f1, f2);++f1;++f2;
};return pair<I1, I2>(f1, f2);
}
Form ∈ [f, l) such that l−m = m− f, swap_ranges(f,m,m, l)rotates [f, l) aboutm
Stepanov, McJones Elements of Programming March 14, 2008 760 / 880
partition_copy_n (from Chapter 9)
template <typename I, typename O0, typename O1, typename P>requires(Readable(I) && Iterator(I) &&
Writable(O0) && Iterator(O0) &&Writable(O1) && Iterator(O1) &&UnaryPredicate(P) && Domain(P) == ValueType(I) &&PartiallyCompatible(ValueType(I), ValueType(O0)) &&PartiallyCompatible(ValueType(I), ValueType(O1)))
pair<O0, O1> partition_copy_n(I f, DistanceType(I) n, O0 r0, O1 r1, P p){
typedef DistanceType(I) N;while (n != N(0)) {
if (p(source(f))) {sink(r1) = source(f); ++r1;
} else {sink(r0) = source(f); ++r0;
}++f;n = n - N(1);
}return pair<O0, O1>(r0, r1);
}
Stepanov, McJones Elements of Programming March 14, 2008 761 / 880
single_cycle_partition (from Chapter 9)
template <typename I, typename P>requires(Readable(I) && BidirectionalIterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))I single_cycle_partition(I f, I l, P p){
f = find_if(f, l, p);l = find_backward_if_not(f, l, p);if (f == l) return f;UnderlyingType(ValueType(I)) tmp = source(f);while (true) {
--l;sink(f) = source(l);f = find_if(successor(f), l, p);if (f == l) {
sink(l) = tmp;return f;
}sink(l) = source(f);l = assured_find_backward_if_not(l, p);
}}
Stepanov, McJones Elements of Programming March 14, 2008 762 / 880
underlying_forward_iterator (from Chapter 9)
template <typename I>requires(ForwardIterator(I))
struct underlying_forward_iterator{
typedef UnderlyingType(ValueType(I)) U;typedef DistanceType(I) N;typedef underlying_forward_iterator UFI;I i;underlying_forward_iterator() {}underlying_forward_iterator(const I& x) : i(x) {}operator I() { return i; }void operator++() { ++i; }UFI operator+(N n) { return i + n; }friend N operator-(UFI x, UFI y) { return x.i - y.i; }friend bool operator==(const UFI& x, const UFI& y) { return x.i == y.i; }friend const U& source(const UFI& x) { return underlying_source(x.i); }friend U& sink(UFI& x) { return underlying_sink(x.i); }
};#define UFI(I) underlying_forward_iterator<I>
Stepanov, McJones Elements of Programming March 14, 2008 763 / 880
underlying_bidirectional_iterator (from Chapter 9)
template <typename I>requires(BidirectionalIterator(I))
struct underlying_bidirectional_iterator{
typedef UnderlyingType(ValueType(I)) U;typedef DistanceType(I) N;typedef underlying_bidirectional_iterator UBI;I i;underlying_bidirectional_iterator() {}underlying_bidirectional_iterator(const I& x) : i(x) {}operator I() { return i; }void operator++() { ++i; }void operator--() { --i; }UBI operator+(N n) { return i + n; }friend N operator-(UBI x, UBI y) { return x.i - y.i; }friend bool operator==(const UBI& x, const UBI& y) { return x.i == y.i; }friend const U& source(const UBI& x) { return underlying_source(x.i); }friend U& sink(UBI& x) { return underlying_sink(x.i); }
};#define UBI(I) underlying_bidirectional_iterator<I>
Stepanov, McJones Elements of Programming March 14, 2008 764 / 880
Type functions for underlying_forward_iterator(from Chapter 9)
template <typename I>requires(ForwardIterator(I))
struct value_type<underlying_forward_iterator<I> >{
typedef UnderlyingType(ValueType(I)) type;};
template <typename I>requires(ForwardIterator(I))
struct distance_type<underlying_forward_iterator<I> >{
typedef DistanceType(I) type;};
template <typename I>requires(ForwardIterator(I))
struct iterator_category<underlying_forward_iterator<I> >{
typedef forward_iterator_tag category;};
Stepanov, McJones Elements of Programming March 14, 2008 765 / 880
stable_partition_n_with_buffer (from Chapter 9)
template <typename I, typename B, typename P>requires(Mutable(I) && ForwardIterator(I) &&
Mutable(B) && ForwardIterator(B) &&UnaryPredicate(P) && Domain(P) == ValueType(I) &&UnderlyingType(ValueType(I)) == ValueType(B))
pair<I, I> stable_partition_n_with_buffer(I f, DistanceType(I) n, B b, P p){
pair<UFI(I), B> r = partition_copy_n(UFI(I)(f), n, UFI(I)(f), b, p);return pair<I, I>(I(r.first), I(copy(b, r.m1, r.first)));
}
Stepanov, McJones Elements of Programming March 14, 2008 766 / 880
stable_partition_n_adaptive (from Chapter 9)
template <typename I, typename B, typename P>requires(Mutable(I) && ForwardIterator(I) &&
Mutable(B) && ForwardIterator(B) &&UnaryPredicate(P) && Domain(P) == ValueType(I) &&UnderlyingType(ValueType(I)) == ValueType(B))
pair<I, I> stable_partition_n_adaptive(I f, DistanceType(I) n, B b, DistanceType(I) n_b, P p)
{typedef DistanceType(I) N;if (n == N(0))
return stable_partition_0(f, p);if (n == N(1))
return stable_partition_1(f, p);if (n <= n_b)
return stable_partition_n_with_buffer(f, n, b, p);N h = half_nonnegative(n);pair<I, I> r0 = stable_partition_n_adaptive(f, h, b, n_b, p);pair<I, I> r1 = stable_partition_n_adaptive(r0.m1, n - h, b, n_b, p);return stable_partition_combine(r0, r1);
}
Stepanov, McJones Elements of Programming March 14, 2008 767 / 880
merge_copy_n (from Chapter 9)
template <typename I0, typename I1, typename O, typename R>requires(Readable(I0) && Iterator(I0) && Readable(I1) && Iterator(I1) &&
Writable(O) && Iterator(O) &&StrictWeakOrdering(R) && Domain(R) == ValueType(I) &&PartiallyCompatible(ValueType(I0), ValueType(O)) &&PartiallyCompatible(ValueType(I1), ValueType(O)))
void merge_copy_n(I0 f0, DistanceType(I0) n0,I1 f1, DistanceType(I1) n1, O o, R r)
{typedef DistanceType(I0) N0; typedef DistanceType(I1) N1;while (true) {
if (n0 == N0(0)) { copy_n(f1, n1, o); return; }if (n1 == N1(0)) { copy_n(f0, n0, o); return; }if (r(source(f1), source(f0))) {
sink(o) = source(f1); ++f1; n1 = n1 - N1(1);} else {
sink(o) = source(f0); ++f0; n0 = n0 - N0(1);}++o;
}}
Stepanov, McJones Elements of Programming March 14, 2008 768 / 880
underlying_compare (from Chapter 9)
template <typename T, typename R>requires(Regular(T) && Relation(R) && Domain(R) == T)
struct underlying_compare{
typedef UnderlyingType(T) U;R r;underlying_compare(R r) : r(r) {}bool operator()(const U& x, const U& y){
return r(reinterpret_cast<const T&>(x), reinterpret_cast<const T&>(y));}
};
Stepanov, McJones Elements of Programming March 14, 2008 769 / 880
merge_n_with_buffer (from Chapter 9)
template <typename I, typename B, typename R>requires(Mutable(I) && ForwardIterator(I) &&
Mutable(B) && ForwardIterator(B) &&StrictWeakOrdering(R) && Domain(R) == ValueType(I) &&UnderlyingType(ValueType(I)) == ValueType(B))
void merge_n_with_buffer(I f0, DistanceType(I) n0, I f1, DistanceType(I) n1, B b, R r)
{typedef underlying_compare<ValueType(I), R> UR;copy_n(UFI(I)(f0), n0, b);merge_copy_n(b, n0, UFI(I)(f1), n1, UFI(I)(f0), UR(r));
}
Stepanov, McJones Elements of Programming March 14, 2008 770 / 880
merge_n_adaptive (from Chapter 9)
template <typename I, typename B, typename R>requires(Mutable(I) && ForwardIterator(I) &&
Mutable(B) && ForwardIterator(B) &&StrictWeakOrdering(R) && Domain(R) == ValueType(I) &&UnderlyingType(ValueType(I)) == ValueType(B))
void merge_n_adaptive(I f0, DistanceType(I) n0, I f1, DistanceType(I) n1,B b, DistanceType(I) n_b, R r)
{typedef DistanceType(I) N;if (n0 + n1 < N(8)) // ***** MEASURE AND TUNE
return insertion_merge_n(f0, n0, f1, n1, r);if (n0 <= n_b)
return merge_n_with_buffer(f0, n0, f1, n1, b, r);I i, j;if (n0 > n1) {
i = f0 + half_nonnegative(n0);j = lower_bound_n(f1, n1, source(i), r);
} else {j = f1 + half_nonnegative(n1);i = upper_bound_n(f0, n0, source(j), r);
}I m = rotate(i, f1, j);merge_n_adaptive(f0, i - f0, i, m - i, b, n_b, r);merge_n_adaptive(m, j - m, j, n1 - (j - f1), b, n_b, r);
} Stepanov, McJones Elements of Programming March 14, 2008 771 / 880
stable_sort_n_adaptive (from Chapter 9)
template <typename I, typename B, typename R>requires(Mutable(I) && ForwardIterator(I) &&
Mutable(B) && ForwardIterator(B) &&StrictWeakOrdering(R) && Domain(R) == ValueType(I) &&UnderlyingType(ValueType(I)) == ValueType(B))
I stable_sort_n_adaptive(I f, DistanceType(I) n, B b, DistanceType(I) n_b, R r)
{typedef DistanceType(I) N;if (n < N(16))
return binary_insertion_sort_n(f, n, r);N h = half_nonnegative(n);I m = stable_sort_n_adaptive(f, h, b, n_b, r);I l = stable_sort_n_adaptive(m, n - h, b, n_b, r);merge_n_adaptive(f, n / 2, m, n - h, b, n_b, r);return l;
}
Stepanov, McJones Elements of Programming March 14, 2008 772 / 880
Type functions for list (from Chapter 11)
template <typename T> requires(Regular(T))struct underlying_type< list<T> >{
typedef list_iterator<T> type; // or IteratorType(list<T>)};
template <typename C>requires(Container(C))
struct iterator_type{
typedef C type;};#define IteratorType(C) typename iterator_type<C>::type
template <typename T> requires(Regular(T))struct iterator_type< list<T> >{
typedef list_iterator<T> type;};
Stepanov, McJones Elements of Programming March 14, 2008 773 / 880
Type functions for array (from Chapter 11)
template <typename T> requires(Regular(T))struct underlying_type< array<T> >{
typedef struct { array_header<T>* p; } type;};
template <typename T> requires(Regular(T))struct iterator_type< array<T> >{
typedef T* type;};
Stepanov, McJones Elements of Programming March 14, 2008 774 / 880
reserve (from Chapter 11)
template <typename T>requires(Regular(T))
void reserve(array<T>& a, DistanceType(IteratorType(array<T>)) n){
if (n < size(a) || n == capacity(a)) return;typedef UnderlyingType<T> U;typedef UFI(IteratorType(array<T>)) UI;array<U> tmp(n)copy(UI(begin(a)), UI(end(a)), array_push_iterator<U>); // never throwsswap(tmp, a);
}
Stepanov, McJones Elements of Programming March 14, 2008 775 / 880
sort for array (from Chapter 11)
template <typename T, typename R>requires(Regular(T) && StrictWeakOrdering(R) && Domain(R) == T)
void sort(array<T>& x, R r){
typedef DistanceType(IteratorType(array<T>)) N;typedef UnderlyingType(T) U;N n = size(x) / N(10);array<U> buffer(n, n, U());stable_sort_n_adaptive(begin(x), size(x), begin(buffer), n, r);
}
#endif // *****
Stepanov, McJones Elements of Programming March 14, 2008 776 / 880
Conclusions
. . .
Stepanov, McJones Elements of Programming March 14, 2008 777 / 880
Contents I
1 Introduction
2 Foundations
3 Transformations and their orbits
4 Algorithms on algebraic structures
5 Orderings
6 Combining concepts
7 Refining concepts of iterators
Stepanov, McJones Elements of Programming March 14, 2008 778 / 880
Contents II
8 Permutations and rearrangements
9 Rotations
10 Algorithms on increasing ranges
11 Coordinate structures
12 Composite objects
13 Iterative algorithms for divide-and-conquerReductionSortingStable partition redux
Stepanov, McJones Elements of Programming March 14, 2008 779 / 880
Contents III
ConclusionsReadingProject
14 Mathematical notation
15 C++ machinery
16 Acknowledgments
17 Index
Stepanov, McJones Elements of Programming March 14, 2008 780 / 880
Concept PartialSemigroupOperation
DefinitionPartialSemigroupOperation(Op)⇒
BinaryOperation(Op)
For all op ∈ Op and for all a,b, c,∈ Domain(Op), if
is_defined(op,a,b) ∧ is_defined(op,b, c)
then
is_defined(op,op(a,b), c) ∧ is_defined(op,a,op(b, c)) ∧
op(op(a,b), c) = op(a,op(b, c))
Stepanov, McJones Elements of Programming March 14, 2008 781 / 880
reduce_nonempty
template <typename I, typename Op>requires(Readable(I) && Iterator(I) &&
SemigroupOperation(Op) && Domain(Op) == ValueType(I))Domain(Op) reduce_nonempty(I f, I l, Op op){
// Precondition: f , lDomain(Op) r = source(f);while (true) {
++f;if (f == l) return r;r = op(r, source(f));
}}
Stepanov, McJones Elements of Programming March 14, 2008 782 / 880
reduce
template <typename I, typename Op>requires(Readable(I) && Iterator(I) &&
MonoidOperation(Op) && Domain(Op) == ValueType(I))Domain(Op) reduce(I f, I l, Op op, const Domain(Op)& z){
if (f == l) return z;return reduce_nonempty(f, l, op);
}
In some situations the caller may need to supply a value for zother than identity_element(op)
***** Explain that there can be many monoids within Domain(Op)?????
Stepanov, McJones Elements of Programming March 14, 2008 783 / 880
find_not
template <typename I>requires(Readable(I) && Iterator(I))
I find_not(I f, I l, const ValueType(I)& x){
while (f != l && source(f) == x) ++f;return f;
}
Stepanov, McJones Elements of Programming March 14, 2008 784 / 880
reduce_nonzeroes
template <typename I, typename Op>requires(Readable(I) && Iterator(I) &&
MonoidOperation(Op) && Domain(Op) == ValueType(I))Domain(Op) reduce_nonzeroes(I f, I l, Op op, const Domain(Op)& z){
f = find_not(f, l, z);if (f == l) return z;Domain(Op) r = source(f);while (true) {
f = find_not(successor(f), l, z);if (f == l) return r;r = op(r, source(f));
}}
Stepanov, McJones Elements of Programming March 14, 2008 785 / 880
add_to_counter
template <typename I, typename Op>requires(Mutable(I) && ForwardIterator(I) &&
BinaryOperation(Op) && Domain(Op) == ValueType(I))Domain(Op) add_to_counter(I f, I l, Op op, Domain(Op) x, const Domain(Op)& z){
if (x == z) return z;while (f != l) {
if (source(f) != z) {x = op(source(f), x);sink(f) = z;
} else {sink(f) = x;return z;
}++f;
}return x;
}
Formally, this procedure requires an Iterator typeThe intent is to call it many times with the same mutable counterThus we specify the requirement as ForwardIteratorStepanov, McJones Elements of Programming March 14, 2008 786 / 880
tranpose_operation
template <typename Op>requires(BinaryOperation(Op))
struct transpose_operation{
Op op;transpose_operation(Op op) : op(op) { }typedef Domain(Op) T;T operator()(const T& x, const T& y){
return op(y, x);}
};
A more general version would allow functions with differentcodomain and domain
Stepanov, McJones Elements of Programming March 14, 2008 787 / 880
reduce_balanced
template <int k, typename I, typename Op>requires(Readable(I) && ForwardIterator(I) &&
MonoidOperation(Op) && Domain(Op) == ValueType(I))Domain(Op) reduce_balanced(I f, I l, Op op, const Domain(Op)& z){
// Precondition: 2k > l− ftypedef array_k<k, Domain(Op)> C;C counter;IteratorType(C) c_f = begin(counter), c_l = c_f;while (f != l) {
Domain(Op) carry = add_to_counter(c_f, c_l, op, source(f), z);if (carry != z) {
sink(c_l) = carry;++c_l;
};++f;
}return reduce_nonzeroes(c_f, c_l, transpose_operation<Op>(op), z);
}
Stepanov, McJones Elements of Programming March 14, 2008 788 / 880
merger_linkable
template <typename S, typename R>requires(ForwardLinker(S) && Readable(IteratorType(S)) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(IteratorType(S)))struct merger_linkable{
typedef IteratorType(S) I;I l;R r;S set_link;merger_linkable(I l, R r, S set_link) : l(l), r(r), set_link(set_link) {}I operator()(I x, I y){
return merge_linkable_nonempty(x, l, y, l, r, set_link).first;}
};
Stepanov, McJones Elements of Programming March 14, 2008 789 / 880
sort_linkable
template <int k, typename S, typename R>requires(ForwardLinker(S) && Readable(IteratorType(S)) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(IteratorType(S)))IteratorType(S) sort_linkable(IteratorType(S) f, IteratorType(S) l, R r, S set_link){
// Precondition: 2k > l− ftypedef merger_linkable<S, R> Op;Op op(l, r, set_link);typedef IteratorType(S) I;typedef array_k<k, I> C;C counter;IteratorType(C) c_f = begin(counter), c_l = c_f;while (f != l) {
I old_f = f;++f;set_link(old_f, l);I carry = add_to_counter(c_f, c_l, op, old_f, l);if (carry != l) {
sink(c_l) = carry;++c_l;
}}return reduce_nonzeroes(c_f, c_l, transpose_operation<Op>(op), l);
}Stepanov, McJones Elements of Programming March 14, 2008 790 / 880
sort_forward_linkable
template <int k, typename I, typename R>requires(Readable(I) && LinkableForwardIterator(I) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I))I sort_forward_linkable(I f, I l, R r){
return sort_linkable<32>(f, l, r, forward_linker<I>());}
Stepanov, McJones Elements of Programming March 14, 2008 791 / 880
sort_bidirectional_linkable
template <int k, typename I, typename R>requires(Readable(I) && LinkableBidirectionalIterator(I) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I))I sort_bidirectional_linkable(I f, I l, R r){
f = sort_linkable<32>(f, l, r, forward_linker<I>());restore_backward_links(f, l, backward_linker<I>());return f;
}
Stepanov, McJones Elements of Programming March 14, 2008 792 / 880
combine_ranges
template <typename I>requires(ForwardIterator(I)) // but not Readable(I)
struct combine_ranges{
typedef pair<I, I> Pair;Pair operator()(const Pair& x, const Pair& y) const{
return Pair(rotate(x.first, x.second, y.first),y.second);
}};
Stepanov, McJones Elements of Programming March 14, 2008 793 / 880
partition_trivial
template <typename I, typename P>requires(ForwardIterator(I) &&UnaryPredicate(P) && Domain(P) == ValueType(I))
struct partition_trivial{
P p;partition_trivial(const P & p) : p(p) {}pair<I, I> operator()(I i) const {
// return stable_partition_1<I, P>(i, p);if (p(source(i)))
return pair<I, I>(i, i);else
return pair<I, I>(i, successor(i));}
};
Stepanov, McJones Elements of Programming March 14, 2008 794 / 880
value_iterator
template <typename I, typename F>requires(Incrementable(I) && Transformation(F) && Domain(F) == I)
struct value_iterator{
I i;F f;value_iterator() {}value_iterator(const I& i, const F& f) : i(i), f(f) {}void operator++() {
++i;}friend Codomain(F) source(const value_iterator& x) {
return x.f(x.i);}friend bool operator==(const value_iterator& x, const value_iterator& y) {
// Precondition: x.f = y.freturn x.i == y.i;
}friend bool operator!=(const value_iterator& x, const value_iterator& y) {
return !(x == y);}
};
Stepanov, McJones Elements of Programming March 14, 2008 795 / 880
stable_partition_iterative
template <typename I, typename P>requires(ForwardIterator(I) && UnaryPredicate(P) && ValueType(I) == Domain(P))
I stable_partition_inplace_iterative(I f, I l, P p){
typedef partition_trivial<I, P> Fun;typedef value_iterator<I, Fun> RangeIterator;typedef combine_ranges<I> Op;Fun fun(p);RangeIterator f1(f, fun);RangeIterator l1(l, fun);combine_ranges<I> op;pair<I, I> z(l, l);return reduce_balanced<32, RangeIterator, Op>(f1, l1, op, z).first;
}
***** There is a bug in this code
Stepanov, McJones Elements of Programming March 14, 2008 796 / 880
Conclusions
***** reduce_balanced and Huffman monoid *****
Stepanov, McJones Elements of Programming March 14, 2008 797 / 880
Reading
Stepanov, McJones Elements of Programming March 14, 2008 798 / 880
Project
Project***** Write an iterative version of stable_partition_n andstable_partition_n_adaptive from Chapter 9 usingreduce_balanced
***** Give hint that many z’s per op will be required ?????
***** Compare the performance of the corresponding recursiveand iterative versions
Stepanov, McJones Elements of Programming March 14, 2008 799 / 880
Contents I
1 Introduction
2 Foundations
3 Transformations and their orbits
4 Algorithms on algebraic structures
5 Orderings
6 Combining concepts
7 Refining concepts of iterators
Stepanov, McJones Elements of Programming March 14, 2008 800 / 880
Contents II
8 Permutations and rearrangements
9 Rotations
10 Algorithms on increasing ranges
11 Coordinate structures
12 Composite objects
13 Iterative algorithms for divide-and-conquer
14 Mathematical notation
Stepanov, McJones Elements of Programming March 14, 2008 801 / 880
Contents III
15 C++ machinery
16 Acknowledgments
17 Index
Stepanov, McJones Elements of Programming March 14, 2008 802 / 880
Connectives
DefinitionIf P and Q are propositions, then so are ¬P (read as “not P”), P ∨Q (“Por Q”), P ∧Q (“P and Q”), P ⇒ Q (“P implies Q”), and P ⇔ Q (“P isequivalent to Q”), with these truth tables:
P Q ¬P P ∨Q P ∧Q P ⇒ Q P ⇔ Q
F F T F F T TF T T T F T FT F F T F F FT T F T T T T
For equivalence, we often write P if and only if Q, abbreviatedP iff Q
Stepanov, McJones Elements of Programming March 14, 2008 803 / 880
Existential and universal quantifiers
DefinitionIf P is a proposition and x is a variable, then (∃x)P is a proposition(read as "there exists x such that P")
DefinitionIf P is a proposition and x is a variable, then (∀x)P is a proposition(read as “for all x, P”); (∀x)P ⇔ (¬(∃x)¬P)
Stepanov, McJones Elements of Programming March 14, 2008 804 / 880
Sets and functions
We use the common vocabulary from naive set theory:a ∈ X (“a is an element of X”)X ⊂ Y (“X is a subset of Y”){a0, . . . ,an} (“the finite set with elements a0, . . . , and an”){a ∈ X|P(a)} (“the subset of X for which the predicate P holds”)X ∪ Y (“the union of X and Y”)X ∩ Y (“the intersection of X and Y”)X− Y (“the complement of Y in X”)X× Y (“the direct product of X and Y”)f : X→ Y (“f is a function from X to Y”)
X is the domain of fY is the codomain of f
Stepanov, McJones Elements of Programming March 14, 2008 805 / 880
Contents I
1 Introduction
2 Foundations
3 Transformations and their orbits
4 Algorithms on algebraic structures
5 Orderings
6 Combining concepts
7 Refining concepts of iterators
Stepanov, McJones Elements of Programming March 14, 2008 806 / 880
Contents II
8 Permutations and rearrangements
9 Rotations
10 Algorithms on increasing ranges
11 Coordinate structures
12 Composite objects
13 Iterative algorithms for divide-and-conquer
14 Mathematical notation
Stepanov, McJones Elements of Programming March 14, 2008 807 / 880
Contents III
15 C++ machineryFoundationsDistanceType type functionSpecial cases of Integer proceduresDefault implementations for additive and multiplicative typesOrderingsIteratorsCategory dispatch, with reverse and reverse_n examplesIncreasing sequencesLinkable rangesBifurcate coordinatesCollocated typesfill_iterator
16 Acknowledgments
Stepanov, McJones Elements of Programming March 14, 2008 808 / 880
Contents IV
17 Index
Stepanov, McJones Elements of Programming March 14, 2008 809 / 880
Syntax for requires and models
#define requires(...)
#define models(...)
Stepanov, McJones Elements of Programming March 14, 2008 810 / 880
ArgumentType, Domain, and Codomain typefunctions
A C++ technique called a trait class can be used to simulate a typefunction:
template <typename T, int i>requires(Procedure(T))
struct argument_type;
#define ArgumentType(T, i) typename argument_type< T, i >::type#define Domain(T) ArgumentType(T, 0)
template <typename T>requires(Procedure(T))
struct codomain_type;
#define Codomain(T) typename codomain_type< T >::type
// The macros work only inside a template definition// because of the use of the keyword typename
Stepanov, McJones Elements of Programming March 14, 2008 811 / 880
DistanceType type function
template <typename T>requires(Countable(T)) /∗ ????? ∗/
struct distance_type{
typedef size_t type;};
template <typename T>requires(Regular(T))
struct distance_type<T*>{
typedef ptrdiff_t type;};#define DistanceType(T) typename distance_type< T >::type
template <>struct distance_type<short>{
typedef unsigned short type;};
Stepanov, McJones Elements of Programming March 14, 2008 812 / 880
Default implementations of special cases of Integerprocedures
// The successor defined in Chapter 6 for Iterator sufficestemplate <typename I> requires(Integer(I))I predecessor(I a) { --a; return a; }template <typename I> requires(Integer(I))I half_nonnegative(const I& a) { return a >> I(1); }template <typename I> requires(Integer(I))void halve_nonnegative(I& a) { a >>= I(1); }template <typename I> requires(Integer(I))I binary_scale_down_nonnegative(const I& a, const I& k) { return a >> k; }template <typename I> requires(Integer(I))I binary_scale_up_nonnegative(const I& a, const I& k) { return a << k; }template <typename I> requires(Integer(I))bool is_positive(const I& a) { return I(0) < a; }template <typename I> requires(Integer(I))bool is_negative(const I& a) { return a < I(0); }template <typename I> requires(Integer(I))bool is_zero(const I& a) { return a == I(0); }template <typename I> requires(Integer(I))bool is_even(const I& a) { return (a & I(1)) == I(0); }template <typename I> requires(Integer(I))bool is_odd(const I& a) { return (a & I(1)) != I(0); }
Stepanov, McJones Elements of Programming March 14, 2008 813 / 880
Default identity and inverse for additive types
template <typename T>requires(AdditiveSemigroup(T))
struct plus {T operator()(const T& x, const T& y) const { return x + y; }
};
template <typename T>requires(AdditiveMonoid(T))
T identity_element(const plus<T>&) { return T(0); }
template <typename T>requires(AdditiveGroup(T))
struct negate {T operator()(const T& x) const { return -x; }
};
template <typename T>requires(AdditiveGroup(T))
negate<T> inverse_operation(const plus<T>&) {return negate<T>();
}
Stepanov, McJones Elements of Programming March 14, 2008 814 / 880
Define ArgumentType(plusT , 0)
template <typename T>requires(AdditiveSemigroup(T))
struct argument_type<plus<T>, 0>{
typedef T type;};
Stepanov, McJones Elements of Programming March 14, 2008 815 / 880
Default identity and inverse for multiplicative types
template <typename T>requires(MultiplicativeSemigroup(T))
struct multiplies {T operator()(const T& x, const T& y) const { return x * y; }
};
template <typename T>requires(MultiplicativeMonoid(T))
T identity_element(const multiplies<T>&) { return T(1); }
template <typename T>requires(MultiplicativeGroup(T))
struct reciprocal {T operator()(const T& x) const { return T(1) / x; }
};
template <typename T>requires(MultiplicativeGroup(T))
reciprocal<T> inverse_operation(const multiplies<T>&) {return reciprocal<T>();
}
Stepanov, McJones Elements of Programming March 14, 2008 816 / 880
Default ordering for StrictTotallyOrdered
template <typename T>requires(StrictTotallyOrdered(T))
struct less{
bool operator()(const T& x, const T& y) const { return x < y; }};
template <typename T>requires(StrictTotallyOrdered(T))
struct argument_type<less<T>, 0>{
typedef T type;};
Stepanov, McJones Elements of Programming March 14, 2008 817 / 880
Default order selection for StrictTotallyOrdered
template <typename T>requires(StrictTotallyOrdered(T))
T& min(T& a, T& b){
return min(a, b, less<T>());}
Stepanov, McJones Elements of Programming March 14, 2008 818 / 880
Dereferenceable functions for T∗
// We will define this latertemplate <typename T>
requires(Regular(T))struct value_type;
template <typename T>requires(Regular(T))
struct value_type<T*> { typedef T type; };
#define ValueType(T) typename value_type< T >::type
template <typename T>requires(Regular(T))
const T& source(T* x) { return *x; }
template <typename T>requires(Regular(T))
T& sink(T* x) { return *x; }
Stepanov, McJones Elements of Programming March 14, 2008 819 / 880
More aliasing tests
template <typename T0, typename T1>requires(Writable(T0) && Writable(T1) && ValueType(T0) == ValueType(T1))
bool is_aliased_sinks(const T0& x0, const T1& x1){
// Return true⇔// For allU with Readable(U) and for all y ∈U,// is_aliased(x0, y) == is_aliased(x1, y)// For many types, this works:typedef const ValueType(T0)* P;return P(&sink(x0)) == P(&sink(x1));
}
template <typename T0, typename T1>requires(Readable(T0) && Readable(T1) && ValueType(T0) == ValueType(T1))
bool is_aliased_sources(const T0& x0, const T1& x1){
// Return true⇔// For allU with Writable(U) and for all y ∈U,// is_aliased(y, x0) == is_aliased(y, x1)// For many types, this works:typedef const ValueType(T0)* P;return P(&source(x0)) == P(&source(x1));
}
Stepanov, McJones Elements of Programming March 14, 2008 820 / 880
Default Dereferenceable functions for T
template <typename T>requires(Regular(T))
struct value_type{
typedef T type;};
template <typename T>requires(Regular(T))
const T& source(const T& x){
return x;}
template <typename T>requires(Regular(T))
T& sink(T& x){
return x;}
Stepanov, McJones Elements of Programming March 14, 2008 821 / 880
Mechanics of category dispatch
A tag type (containing no data) is defined for each categoryA type function is defined to obtain a type tag from a typeOverloading is used to select from multiple signatures differingonly by a tag type
Stepanov, McJones Elements of Programming March 14, 2008 822 / 880
Iterator tag types
It FI
BI
II
RI
struct iterator_tag {};struct forward_iterator_tag {};struct bidirectional_iterator_tag {};struct indexed_iterator_tag {};struct random_access_iterator_tag {};
Stepanov, McJones Elements of Programming March 14, 2008 823 / 880
IteratorCategory type function
template <typename T>requires(Iterator(T))
struct iterator_category{
typedef iterator_tag category;};
#define IteratorCategory(T) typename iterator_category< T >::category
template <typename T>requires(Regular(T))
struct iterator_category<T*>{
typedef random_access_iterator_tag category;};
Stepanov, McJones Elements of Programming March 14, 2008 824 / 880
Category dispatch for reverse_n and reverse
template <typename I>requires(Mutable(I) && ForwardIterator(I))
void reverse_n(I f, DistanceType(I) n){
reverse_n(f, n, IteratorCategory(I)());}
template <typename I>requires(Mutable(I) && ForwardIterator(I))
void reverse(I f, I l){
reverse(f, l, IteratorCategory(I)());}
Stepanov, McJones Elements of Programming March 14, 2008 825 / 880
Category dispatch cases for reverse_n
template <typename I> requires(Mutable(I) && ForwardIterator(I))void reverse_n(I f, DistanceType(I) n, forward_iterator_tag){
reverse_n_forward(f, n);}
template <typename I> requires(Mutable(I) && BidirectionalIterator(I))void reverse_n(I f, DistanceType(I) n, bidirectional_iterator_tag){
reverse_n_bidirectional(f, n);}
template <typename I> requires(Mutable(I) && IndexedIterator(I))void reverse_n(I f, DistanceType(I) n, indexed_iterator_tag){
reverse_n_indexed(f, n);}
template <typename I> requires(Mutable(I) && RandomAccessIterator(I))void reverse_n(I f, DistanceType(I) n, random_access_iterator_tag){
reverse_n_indexed(f, n);}
Stepanov, McJones Elements of Programming March 14, 2008 826 / 880
Category dispatch cases for reverse
template <typename I> requires(Mutable(I) && ForwardIterator(I))void reverse(I f, I l, forward_iterator_tag){
reverse_forward(f, l);}
template <typename I> requires(Mutable(I) && BidirectionalIterator(I))void reverse(I f, I l, bidirectional_iterator_tag){
reverse_bidirectional(f, l);}
template <typename I> requires(Mutable(I) && IndexedIterator(I))void reverse(I f, I l, indexed_iterator_tag){
reverse_indexed(f, l);}
template <typename I> requires(Mutable(I) && RandomAccessIterator(I))void reverse(I f, I l, random_access_iterator_tag){
reverse_indexed(f, l);}
Stepanov, McJones Elements of Programming March 14, 2008 827 / 880
find_out_of_order
template <typename I, typename R>requires(Readable(I) && ForwardIterator(I) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I))I find_out_of_order(I f, I l, R r, forward_iterator_tag){
if (f == l) return l;I n(successor(f));while (n != l) {
if (r(source(n), source(f)))return n;
f = n;++n;
}return l;
}
Stepanov, McJones Elements of Programming March 14, 2008 828 / 880
find_out_of_order
template <typename I, typename R>requires(Readable(I) && Iterator(I) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I))I find_out_of_order(I f, I l, R r, iterator_tag){
if (f == l) return l;I n(successor(f));ValueType(I) v = source(f);while (n != l) {
if (r(source(n), v))return n;
v = source(n);++n;
}return l;
}
Stepanov, McJones Elements of Programming March 14, 2008 829 / 880
Category dispatch for find_out_of_order
template <typename I, typename R>requires(Readable(I) && RandomAccessIterator(I) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I))I find_out_of_order(I f, I l, R r, random_access_iterator_tag){
return find_out_of_order(f, l, r, forward_iterator_tag());}
template <typename I, typename R>requires(Readable(I) && BidirectionalIterator(I) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I))I find_out_of_order(I f, I l, R r, bidirectional_iterator_tag){
return find_out_of_order(f, l, r, forward_iterator_tag());}
template <typename I, typename R>requires(Readable(I) && Iterator(I) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I))I find_out_of_order(I f, I l, R r){
return find_out_of_order(f, l, r, IteratorCategory(I)());}
Stepanov, McJones Elements of Programming March 14, 2008 830 / 880
is_increasing
template <typename I, typename R>requires(Readable(I) && Iterator(I) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I))bool is_increasing(I f, I l, R r){
return l == find_out_of_order(f, l, r);}
Stepanov, McJones Elements of Programming March 14, 2008 831 / 880
partition_copy_n
template <typename I, typename O0, typename O1, typename P>requires(Readable(I) && Iterator(I) &&
Writable(O0) && Iterator(O0) &&Writable(O1) && Iterator(O1) &&UnaryPredicate(P) && Domain(P) == ValueType(I) &&ValueType(I) == ValueType(O0) &&ValueType(I) == ValueType(O1))
pair<O0, O1> partition_copy_n(I f, DistanceType(I) n, O0 o0, O1 o1, P p){
typedef DistanceType(I) N;while (n != N(0)) {
if (p(source(f))) {sink(o1) = source(f); ++o1;
} else {sink(o0) = source(f); ++o0;
}++f;n = n - N(1);
}return pair<O0, O1>(o0, o1);
}
Stepanov, McJones Elements of Programming March 14, 2008 832 / 880
unstable_indexed_partition
template <typename I, typename P>requires(Mutable(I) && IndexedIterator(I) &&
UnaryPredicate(P) && Domain(P) == ValueType(I))I unstable_indexed_partition(I f, I l, P p){
DistanceType(I) i = 0;DistanceType(I) j = l - f;while (true) {
while (true) {if (i == j) return f + i;if (p(source(f + i))) break;i = i + 1;
}while (true) {
j = j - 1;if (i == j) return f + j + 1;if (!p(source(f + j))) break;
}cycle_2(f + i, f + j);i = i + 1;
}}
Stepanov, McJones Elements of Programming March 14, 2008 833 / 880
lower_bound_n
template <typename T, typename R>requires(Regular(T) && StrictWeakOrdering(R) && Domain(R) == T)
struct greater_than_or_equal_to_a{
const T& a;R r;greater_than_or_equal_to_a(const T& a, R r) : a(a), r(r) { }bool operator()(const T& x) const { return !r(x, a); }
};
template <typename I, typename R>requires(Mutable(I) && ForwardIterator(I) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I))I lower_bound_n(I f, DistanceType(I) n, const ValueType(I)& a, R r){
greater_than_or_equal_to_a<ValueType(I), R> predicate(a, r);return partition_point_n(f, n, predicate);
}
Stepanov, McJones Elements of Programming March 14, 2008 834 / 880
upper_bound_n
template <typename T, typename R>requires(Regular(T) && StrictWeakOrdering(R) && Domain(R) == T)
struct greater_than_a{
const T& a;R r;greater_than_a(const T& a, R r) : a(a), r(r) { }bool operator()(const T& x) const { return r(a, x); }
};
template <typename I, typename R>requires(Mutable(I) && ForwardIterator(I) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(I))I upper_bound_n(I f, DistanceType(I) n, const ValueType(I)& a, R r){
greater_than_a<ValueType(I), R> predicate(a, r);return partition_point_n(f, n, predicate);
}
Stepanov, McJones Elements of Programming March 14, 2008 835 / 880
Category dispatch formerge_n_with_buffer
template <typename I, typename B, typename R>requires(Mutable(I) && RandomAccessIterator(I) &&
Mutable(B) && RandomAccessIterator(B) &&StrictWeakOrdering(R) && Domain(R) == ValueType(I) &&ValueType(I) == ValueType(B))
void merge_n_with_buffer(I f0, DistanceType(I) n0,I f1, DistanceType(I) n1, B b, R r,random_access_iterator_tag)
{merge_n_with_buffer(f0, n0, f1, n1, b, r, bidirectional_iterator_tag());
}
template <typename I, typename B, typename R>requires(Mutable(I) && ForwardIterator(I) &&
Mutable(B) && ForwardIterator(B) &&StrictWeakOrdering(R) && Domain(R) == ValueType(I) &&ValueType(I) == ValueType(B))
void merge_n_with_buffer(I f0, DistanceType(I) n0,I f1, DistanceType(I) n1, B b, R r)
{merge_n_with_buffer(f0, n0, f1, n1, b, r, IteratorCategory(I)());
}
Stepanov, McJones Elements of Programming March 14, 2008 836 / 880
IteratorType type function
template <typename T>requires(ImplementsIteratorType(T))
struct iterator_type;
#define IteratorType(T) typename iterator_type< T >::type
Stepanov, McJones Elements of Programming March 14, 2008 837 / 880
IteratorType for linkers
template <typename I> requires(LinkableForwardIterator(I))struct forward_linker;
template <typename I> requires(LinkableForwardIterator(I))struct iterator_type< forward_linker<I> > { typedef I type; };
template <typename I> requires(LinkableBidirectionalIterator(I))struct backward_linker;
template <typename I> requires(LinkableBidirectionalIterator(I))struct iterator_type< backward_linker<I> > { typedef I type; };
template <typename I> requires(LinkableBidirectionalIterator(I))struct bidirectional_linker;
template <typename I> requires(LinkableBidirectionalIterator(I))struct iterator_type< bidirectional_linker<I> > { typedef I type; };
Stepanov, McJones Elements of Programming March 14, 2008 838 / 880
Domain for transpose_operation
template <typename Op>requires(BinaryOperation(Op))
struct transpose_operation;
template <typename Op>requires(BinaryOperation(Op))
struct argument_type< transpose_operation<Op>, 0 >{
typedef Domain(Op) type;};
Stepanov, McJones Elements of Programming March 14, 2008 839 / 880
Domain formerger_linkable
template <typename S, typename R>requires(ForwardLinker(S) && Readable(IteratorType(S)) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(IteratorType(S)))struct merger_linkable;
template <typename S, typename R>requires(ForwardLinker(S) && Readable(IteratorType(S)) &&
StrictWeakOrdering(R) && Domain(R) == ValueType(IteratorType(S)))struct argument_type<merger_linkable<S, R>, 0>{
typedef IteratorType(S) type;};
Stepanov, McJones Elements of Programming March 14, 2008 840 / 880
Domain for combine_ranges
template <typename I>requires(ForwardIterator(I)) // but not Readable(I)
struct combine_ranges;
template <typename I>requires(ForwardIterator(I)) // but not Readable(I)
struct argument_type<combine_ranges<I>, 0>{
typedef pair<I, I> type;};
Stepanov, McJones Elements of Programming March 14, 2008 841 / 880
Codomain for partition_trivial
template <typename I, typename P>
requires(ForwardIterator(I) && UnaryPredicate(P) && ValueType(I) == Domain(P))struct partition_trivial;
template <typename I, typename P>
requires(ForwardIterator(I) && UnaryPredicate(P) && ValueType(I) == Domain(P))struct codomain_type< partition_trivial<I, P> >{
typedef pair<I, I> type;};
Stepanov, McJones Elements of Programming March 14, 2008 842 / 880
WeightType type function
template <typename T>requires(WeakForwardBifurcateCoordinate(T))
struct weight_type{
typedef size_t type; // ***** What should the default be ?????};
template <typename T>requires(WeakForwardBifurcateCoordinate(T))
struct weight_type<T*>{
typedef ptrdiff_t type;};#define WeightType(T) typename weight_type< T >::type // ***** ?????
Stepanov, McJones Elements of Programming March 14, 2008 843 / 880
left_successor and right_successor forWeakForwardBifurcateCoordinate
template <typename C>requires(WeakForwardBifurcateCoordinate(C))
C left_successor(C f){
move_left(f);return f;
}
template <typename C>requires(WeakForwardBifurcateCoordinate(C))
C right_successor(C f){
move_right(f);return f;
}
Stepanov, McJones Elements of Programming March 14, 2008 844 / 880
pair
template <typename T0, typename T1>requires(Regular(T0) && Regular(T1))
struct pair{
T0 m0;T1 m1;pair() {}pair(T0 m0, T1 m1) : m0(m0), m1(m1) { }pair(const pair& x) : m0(x.m0), m1(x.m1) { }void operator=(const pair& x) { m0 = x.m0; m1 = x.m1; }friend void bool operator==(const pair& x, const pair& y){ return x.m0 == y.m0 && x.m1 == y.m1; }friend void bool operator<(const pair& x, const pair& y){ return x.m0 < y.m0 || (!y.m0 < x.m0 && x.m1 < y.m1); }
};
Stepanov, McJones Elements of Programming March 14, 2008 845 / 880
triple
template <typename T0, typename T1, typename T2>requires(Regular(T0) && Regular(T1) && Regular(T2))
struct triple{
T0 m0;T1 m1;T2 m2;triple() {}triple(T0 m0, T1 m1, T2 m2) :
m0(m0), m1(m1), m2(m2) {}triple(const triple& x) : m0(x.m0), m1(x.m1), m2(x.m2) { }void operator=(const triple& x){ m0 = x.m0; m1 = x.m1; m2 = x.m2}friend void bool operator==(const triple& x, const triple& y){ return x.m0 == y.m0 && x.m1 == y.m1 && x.m2 == y.m2; }friend void bool operator<(const triple& x, const triple<T0, T1, T2>& y){ return x.m0 < y.m0 || (!y.m0 < x.m0 && x.m1 < y.m1 ||
(!y.m1 < x.m1 && x.m2 == y.m2)); }};
Stepanov, McJones Elements of Programming March 14, 2008 846 / 880
lexicographic_equalityk
template <int k, typename I0, typename I1>requires(Readable(I0) && ForwardIterator(I0) &&
Readable(I1) && ForwardIterator(I1) &&ValueType(I0) == ValueType(I1))
struct lexicographic_equality_k{
bool operator()(I0 f0, I1 f1){
if (source(f0) != source(f1)) return false;return lexicographic_equality_k<k - 1, I0, I1>()(f0 + 1, f1 + 1);
}};
template <typename I0, typename I1>struct lexicographic_equality_k<0, I0, I1>{
bool operator()(I0, I1){
return true;}
};
Stepanov, McJones Elements of Programming March 14, 2008 847 / 880
lexicographic_orderingk
template <int k, typename I0, typename I1>requires(Readable(I0) && ForwardIterator(I0) &&
Readable(I1) && ForwardIterator(I1) &&ValueType(I0) == ValueType(I1))
struct lexicographic_ordering_k{
bool operator()(I0 f0, I1 f1){
if (source(f0) < source(f1)) return true;if (source(f0) > source(f1)) return false;return lexicographic_equality_k<k - 1, I0, I1>()(f0 + 1, f1 + 1);
}};
template <typename I0, typename I1>struct lexicographic_ordering_k<0, I0, I1>{
bool operator()(I0, I1){
return false;}
};
Stepanov, McJones Elements of Programming March 14, 2008 848 / 880
arrayk
***** Need const and non-const versions of begin and end ?????***** What about subscript: a[i] ?????template <int k, typename T>
requires(Regular(T))struct array_k{
typedef T* I; // IteratorType(arrayk,T )T a[k];array_k() { }array_k(const array_k& x) { basic_copy_k_indexed<k, I, O>(&x.a, &a); }friend void operator=(const array_k& x){ basic_copy_k_indexed<k, I, O>(&x.a, &a); }friend size_t size(const array_k& x) { return size_t(k); }friend I begin(array_k& x) { return &x.a[0]; }friend I end(array_k& x) { return &x.a[k]; }friend bool operator==(const array_k& x, const array_k& y){ return lexicographic_equality_k<k, I, I>(begin(x), begin(y)); }friend bool operator<(const array_k& x, const array_k& y){ return lexicographic_ordering_k<k, I, I>(begin(x), begin(y)); }
};
Stepanov, McJones Elements of Programming March 14, 2008 849 / 880
Forward declaration for array_k data structure
template <int k, typename T>requires(Regular(T))
struct array_k;
array_k is defined in Chapter 11
Stepanov, McJones Elements of Programming March 14, 2008 850 / 880
IteratorType for array_k
template <int k, typename T>requires(Regular(T))
struct iterator_type< array_k<k, T> >{
typedef T* type;};
Stepanov, McJones Elements of Programming March 14, 2008 851 / 880
fill_iterator
template <typename T, typename N>requires(Regular(T) && Integer(N))
struct fill_iterator{
const T* v;N n;fill_iterator() {}fill_iterator(const T& v, N n) : v(&v), n(n) {}friend void operator++(fill_iterator& x) { ++x.n; }friend const T& source(fill_iterator& x) { return source(x.v); }friend void operator+=(fill_iterator& x, N n) { x.n += n; }friend N operator-(fill_iterator x, fill_iterator y) { return x.n - y.n; }friend bool operator==(fill_iterator x, fill_iterator y) {
return x.n == y.n;}
};
Stepanov, McJones Elements of Programming March 14, 2008 852 / 880
Type functions for fill_iterator
template <typename T, typename N>requires(Regular(T) && Integer(N))
struct value_type< fill_iterator<T, N> >{
typedef T type;};
template <typename T, typename N>requires(Regular(T) && Integer(N))
struct distance_type< fill_iterator<T, N> >{
typedef N type;};
template <typename T, typename N>requires(Regular(T) && Integer(N))
struct iterator_category< fill_iterator<T, N> >{
typedef indexed_iterator_tag category;};
Stepanov, McJones Elements of Programming March 14, 2008 853 / 880
Acknowledgments
We are grateful for comments and suggestions fromAndrei Alexandrescu,Matt Austern, Dave Abrahams, Jon Brandt,Boris Fomitchev, Kevlin Henney, Jussi Ketonen, Karl Malbrain,Larry Masinter, David Musser, Dave Parent, Sean Parent,Dmitry Polukhin, Jon Reid, Mark Ruzon, Geoff Scott,David Simons, Tony Van Eerd, Walter Vannini, John Wilkinson,and Oleg Zabluda
Stepanov, McJones Elements of Programming March 14, 2008 854 / 880
Index I
++definition space on range, 361
6standard definition, 242
>standard definition, 242
>standard definition, 242
absolute value, 272absorption
formin,max, 251abstraction, 19action
duality with transformation, 343Action concept, 342activity on entities, 19acyclic
descendants of bifurcate coordinate, 633AdditiveGroup concept, 200AdditiveMonoid concept, 198AdditiveSemigroup concept, 196algorithm
memory-adaptive, 426with buffer, 426
algorithmically-induced structure, 276aliasing, 338, 385amortized complexity, 347AmortizedConstantTime concept, 347
Stepanov, McJones Elements of Programming March 14, 2008 855 / 880
Index II
and (∧), 803Archimedean monoid
discrete, 287ArchimedeanMonoid concept, 276ArchimedeanOrderedField concept, 519area
constant, 82fixed, 82of an object, 82variable, 82
area function, 82argument, 40ArgumentType type function, 55Arity type attribute, 54assignment, 49
for Regular, 65associativity, 130
exploited by power, 145ofmin,max, 251of permutation composition, 412
asymmetric relation, 218attribute of entity, 19average cost, 84Axiom of Archimedes, 276, 277
backward movement in range, 533BackwardLinker concept, 601Banning, John, 358bidirectional binary trees, advantages, 641
Stepanov, McJones Elements of Programming March 14, 2008 856 / 880
Index III
BidirectionalBifurcateCoordinate concept, 629BidirectionalIterator concept, 374BidirectionalLinker concept, 602BifurcateCoordinate concept, 628bignum, 139bin-based rearrangement, 422binary gcd, 327binary search, 553BinaryOperation concept, 129bisection technique, 518Bolzano, Bernard, 518bounded integer type, 322buffer
with buffer algorithm, 426
CancellableOperation concept, 195category dispatch
mechanism, 822Cauchy, Augustin Louis, 518closed bounded range, 357closed counted range, 357closed interval, 243clusters
of derived procedures, 238code transformations
enabled by regular types, 81, 181codomain of function, 805codomain of procedure, 42Codomain type function, 55
Stepanov, McJones Elements of Programming March 14, 2008 857 / 880
Index IV
CommutativeOperation concept, 194CommutativeSemiringconcept, 298commutativity
ofmin,max, 251Compatible concept, 732complement (−), 805complement of converse of relation, 239complement of relation, 239complexity
algorithm selection, 175amortized, 347counting operations, 176in specifications, 175of generalized algorithm, 175slow_power versus power, 177, 178to determine feasibility, 175
compositionof permutations, 412
concept, 56ArchimedeanMonoid, 276Action, 342AdditiveGroup, 200AdditiveMonoid, 198AdditiveSemigroup, 196AmortizedConstantTime, 347ArchimedeanOrderedField, 519BackwardLinker, 601BidirectionalBifurcateCoordinate, 629BidirectionalIterator, 374
Stepanov, McJones Elements of Programming March 14, 2008 858 / 880
Index V
BidirectionalLinker, 602BifurcateCoordinate, 628BinaryOperation, 129CancellableOperation, 195CommutativeOperation, 194CommutativeSemiring, 298Compatible, 732Dereferenceable, 340DiscreteArchimedeanMonoid, 287DiscreteArchimedeanRing, 319DiscreteArchimedeanSemiring, 317EquivalenceRelation, 219EuclideanMonoid, 293EuclideanSemimodule, 304EuclideanSemiring, 306examples from C++, 59ForwardIterator, 371ForwardLinker, 600Function, 60GroupOperation, 137HomogeneousFunction, 61IndexedIterator, 380Integer, 139Iterator, 348LinkableBidirectionalBifurcateCoordinate, 644LinkableBidirectionalIterator, 606LinkableBifurcateCoordinate, 643LinkableForwardIterator, 605Merger, 567
Stepanov, McJones Elements of Programming March 14, 2008 859 / 880
Index VI
MonoidOperation, 136MultiplicativeGroup, 201MultiplicativeMonoid, 199MultiplicativeSemigroup, 197Mutable, 339NonnegativeDiscreteArchimedeanSemiring, 318Operation, 88OrderedAdditiveGroup, 271OrderedAdditiveMonoid, 271OrderedAdditiveSemigroup, 271OrderedCancellableMonoid, 273Ordering, 215PartiallyCompatible, 728PartialSemigroupOperation, 781Predicate, 214Procedure, 60ProperProcedure, 60QRSemimodule, 302RandomAccessIterator, 387Readable, 336refinement, 57Regular, 63RegularAction, 346RegularFunction, 79Relation, 214relational concept, 726SameGenus, 727SegmentedIterator, 399SemigroupOperation, 130
Stepanov, McJones Elements of Programming March 14, 2008 860 / 880
Index VII
Semimodule, 300StrictTotallyOrdered, 237StrictTotalOrdering, 223StrictWeakOrdering, 226Transformation, 89type concept, 58UnaryPredicate, 504univalent, 320WeakBifurcateCoordinate, 626weakening, 57WeaklyHalvableMonoid, 283Writable, 337
constant area, 82constructor, 52
type constructor, 53container, 668converse of relation, 239coordinate structure, 594copy constructor
for Regular, 68copying rearrangement, 425cost
average, 84fixed, 84uniform, 84worst-case, 84
cost function, 84cost of a procedure call, 84cycle in a permutation, 415
Stepanov, McJones Elements of Programming March 14, 2008 861 / 880
Index VIII
cyclic permutation, 417
default constructorfor Regular, 67
default ordering, 236definition space of procedure, 41Dereferenceable concept, 340derived relations, 239descendants
of bifurcate coordinate, 633description, 18destructor, 50
for Regular, 66DifferenceType type function, 387direct product (×), 805discrete Archimedean monoid, 287DiscreteArchimedeanMonoid concept, 287DiscreteArchimedeanRing concept, 319DiscreteArchimedeanSemiring concept, 317discreteness, 317distance type, 110DistanceType type function, 348distributivity
of semiring, 298divisibility lattice of integers, 466divisibility on an Archimedean monoid, 285division
Egyptian, 279division on multiplicative group, 201
Stepanov, McJones Elements of Programming March 14, 2008 862 / 880
Index IX
domain of function, 805Domain type function, 61Dudzinski, Krzysztof, 576Dydek, Andrzej, 576
Egyptian division, 279element (∈), 805empty range, 360entity
activity on, 19attribute, 19part, 19particulars, 20relationship, 19
equalityfor Regular, 64in the real world, 33of objects, 38on containers, 670, 671properties of, 31unique on a type, 234
equality (=), 30equational reasoning, 32
in mathematics, 32EquivalenceRelation concept, 219equivalent (⇔), 803essential object, 39Euclidean domain, 297Euclidean function, 306
Stepanov, McJones Elements of Programming March 14, 2008 863 / 880
Index X
EuclideanMonoid concept, 293EuclideanSemimodule concept, 304EuclideanSemiring concept, 306exclusive-or of links, 646exists (∃), 804
Fibonaccimatrices, 185numbers, 182
Fiduccia, Charles M., 206finite set, 418finite tree
descendents of bifurcate coordinate, 634first in a range, 358fixed area, 82fixed cost, 84fixed point, 411for all (∀), 804ForwardIterator concept, 371ForwardLinker concept, 600from-permutation, 431function
one-to-one, 408onto, 407
function (→), 805Function concept, 60function procedure, 47
Stepanov, McJones Elements of Programming March 14, 2008 864 / 880
Index XI
Gaussian integersdefined, 166example of Euclidean domain, 297Stein’s algorithm, 328
gcdand lcm, 465binary, 327greatest common divisor, 288subtractive, 290
generalized link rearrangement, 610generalized procedure, 71generalized rearrangement, 609genus, 35global state, 40goto
considered not harmful, 655greatest common divisor, 288group
of permutations, 414GroupOperation concept, 137
half-open interval, 243Ho, Wilson, 482, 545Hoare, C.A.R., 543HomogeneousFunction concept, 61
idea, 18idempotency
Stepanov, McJones Elements of Programming March 14, 2008 865 / 880
Index XII
ofmin,max, 251identity
in the real world, 29of objects, 29
identity permutation, 411implies (⇒), 803improper procedure, 46in place, 426in situ, 426increasing order, 248increasing range, 499index permutation, 419indexed iterator
equivalent to random access iterator, 391IndexedIterator concept, 380inequality
standard definition, 241Integer concept, 139interface
for rotate, 469useful variations, 180
interoperability, 62interpretatation of object, 25intersection (∩), 805interval, 243invariant, 655invariants
breaking, 622inverse
Stepanov, McJones Elements of Programming March 14, 2008 866 / 880
Index XIII
of permutation, 413, 417is_defined predicate, 43isomorphic types, 320iterator
linkable, 598Iterator concept, 348
k-partition, 504key function, 499, 569key-increasing range, 499key-sorting, 569
Lakshman, T.K., 523last in a range, 358lattice, 466lcm
and gcd, 465least common multiple, 464
least common multiple, 464Leibniz’s Law, 30lexicographic ordering, 225, 229limit in a range, 358linear ordering
importance of, 213linear recurrences, 206link rearrangement, 603linkable iterator, 598linkable range, 598
Stepanov, McJones Elements of Programming March 14, 2008 867 / 880
Index XIV
linkable structuresforward versus bidirectional, 625
LinkableBidirectionalBifurcateCoordinate concept, 644LinkableBidirectionalIterator concept, 606LinkableBifurcateCoordinate concept, 643LinkableForwardIterator concept, 605linking operation, 599links
exclusive-or of, 646reversing, 645
Lo, Raymond, 482, 545local state, 40locality of reference, 476, 618located sequence, 356Lomuto, Nico, 526loop fusion, 476lower bound, 506
implementation, 554
Mauchly, John W., 586memory, 335memory-adaptive algorithm, 426merge, 555
algebraic properties, 559complexity property, 560copying, 556melding, 556mutating, 556stability, 555
Stepanov, McJones Elements of Programming March 14, 2008 868 / 880
Index XV
mergeable range, 556Merger concept, 567models
models (|=), 56type models concept, 56
MonoidOperation concept, 136MultiplicativeGroup concept, 201MultiplicativeMonoid concept, 199MultiplicativeSemigroup concept, 197Mutable concept, 339mutable range, 362mutative rearrangement, 425mutator procedure, 47
natural total ordering, 235< reserved for, 237
Noether, Emmy, 307non-terminating procedure, 44nonessential object, 39NonnegativeDiscreteArchimedeanSemiring concept, 318not (¬), 803not last in a range, 358
object, 24, 25interpretation, 25state, 25
one-to-one function, 408onto function, 407
Stepanov, McJones Elements of Programming March 14, 2008 869 / 880
Index XVI
open interval, 243Operation concept, 88optimization
enabled by regular types, 81loop fusion, 476using segmented iterator, 398
or (∨), 803orbit, 92OrderedAdditiveGroup concept, 271OrderedAdditiveMonoid concept, 271OrderedAdditiveSemigroup concept, 271OrderedCancellableMonoid concept, 273ordering
reflexive, 217strict, 217weakly reflexive, 217
Ordering concept, 215ordering-based rearrangement, 422out-of-order execution, 385overlapping ranges, 377overloading, 190, 599, 642own state, 40
partialprocedure, 43representation, 27
partial orderingexample, 231topological sort defined on, 233
Stepanov, McJones Elements of Programming March 14, 2008 870 / 880
Index XVII
partially-formed, 51PartiallyCompatible concept, 728PartialSemigroupOperation concept, 781particulars of entity, 20partition, 504
k-partition, 504partition algorithm, 508partition algorithms, origin of, 543partition point, 505
finding, 515lower bound, 506upper bound, 506
partition rearrangementsemistable, 525
partitioned prefix of a range, 509parts of an entity, 19permutation, 409
composition, 412cycle, 415cyclic, 417from, 431identity, 411index, 419inverse, 413, 417product of its cycles, 417reverse, 436rotation, 463to, 431
permutation group, 414
Stepanov, McJones Elements of Programming March 14, 2008 871 / 880
Index XVIII
pointersexclusive-or of, 646
polynomialsexample of Euclidean domain, 297
position-based rearrangement, 422powers
non-positive exponent, 138of same element commute, 132positive exponent, 131
Predicate concept, 214procedure, 40
function, 47generalized, 71improper, 46mutator, 47non-terminating, 44partial, 43proper, 46semi-terminating, 44terminating, 44total, 43
Procedure concept, 60proper procedure, 46ProperProcedure concept, 60
QRSemimodule concept, 302quicksort, 528, 543
Stepanov, McJones Elements of Programming March 14, 2008 872 / 880
Index XIX
random access iteratorequivalent to indexed iterator, 391
RandomAccessIterator concept, 387range, 356
backward movement, 533closed bounded, 357closed counted, 357empty, 360first, 358increasing, 499k-partition, 504key-increasing, 499last, 358limit, 358linkable, 598lower bound, 506mergeable, 556mutable, 362not last, 358overlapping, 377partition, 504partition point, 505partitioned prefix, 509readable, 362semi-open bounded, 357semi-open counted, 357size, 359uniform random shuffle, 545upper bound, 506
Stepanov, McJones Elements of Programming March 14, 2008 873 / 880
Index XX
writable, 362reachability of bifurcate coordinates, 632reachable, 90Readable concept, 336readable range, 362rearrangement, 420
bin-based, 422copying, 425generalized, 609generalized link, 610key-sorting, 569link, 603mutative, 425ordering-based, 422partition, 508position-based, 422reverse, 437rotation, 469sorting, 569time complexity, 427, 434
refinement of concept, 57reflexive ordering, 217reflexivity of equality, 31regular
allows code transformations, 81, 181required for predicate with assured find, 544
Regular concept, 63RegularAction concept, 346RegularFunction concept, 79
Stepanov, McJones Elements of Programming March 14, 2008 874 / 880
Index XXI
relationcomplement, 239complement of converse, 239converse, 239symmetric, 218symmetric complement of, 222
Relation concept, 214relational concept, 726relations
derived, 239relationship between entities, 19representation, 27
partial, 27total, 27unique, 28
requires clause, 71, 72result space of procedure, 42returning useful information, 365, 367, 394, 439, 469, 482, 523, 533, 537, 553, 563, 735, 737reverse
permutation, 436rearrangement, 437
reversing links, 645rotation
permutation, 463rearrangement, 469
Russian peasant algorithm, 144
same genus, 35SameGenus concept, 727
Stepanov, McJones Elements of Programming March 14, 2008 875 / 880
Index XXII
Scalar type function, 300segmented iterator
intuition, 398SegmentedIterator concept, 399SegmentIterator type function, 399semi
usage convention, 232semi-open bounded range, 357semi-open counted range, 357semi-terminating procedure, 44SemigroupOperation concept, 130Semimodule concept, 300semistable partition rearrangement, 525sequence
located, 356sequence of iterators, 355sets, 805sink, 337size
of range, 359sort (type) of value, 21sorting
stability, 569sorting algorithm, 569space complexity
forward iterator reverse conjecture, 455in place, 426memory adaptive, 426with buffer, 426
Stepanov, McJones Elements of Programming March 14, 2008 876 / 880
Index XXIII
stability, 247ofmin,max, 250of rearrangement, 423of sorting algorithm, 569
stable adaptive merge and sort, importance of, 585stable in-place merge and sort, importance of, 585stable merge, 555state of object, 25Stein, Josef, 327storage efficiency, 83strict lower bound, 434strict ordering, 217strictly increasing order, 248StrictTotallyOrdered concept, 237StrictTotalOrdering concept, 223StrictWeakOrdering concept, 226subset (⊂), 805subtraction on additive group, 200subtractive gcd, 290symmetric complement of a relation, 222symmetric relation, 218symmetry of equality, 31
terminating procedure, 44thing, 18Tighe, Joseph, 469time complexity
of rearrangement, 427, 434to-permutation, 431
Stepanov, McJones Elements of Programming March 14, 2008 877 / 880
Index XXIV
totalprocedure, 43representation, 27
trait class, 811transformation
duality with action, 343fixed point of, 411self-composable, 90
Transformation concept, 89transitivity, 215transitivity of equality, 31traversal
of tree, recursive, 637tree
descendants of bifurcate coordinate, 633trichotomy law
for total ordering, 224for weak ordering, 228
trivial cycle, 415type, 36
constant area, 82fixed area, 82intuition, 37isomorphism, 320variable area, 82
type attribute, 54Arity, 54
type concept, 58type constructor, 53
Stepanov, McJones Elements of Programming March 14, 2008 878 / 880
Index XXV
type expression, 72type function, 55
ArgumentType, 55Codomain, 55DifferenceType, 387DistanceType, 348Domain, 61example, 109Scalar, 300SegmentIterator, 399simulated in C++ via trait class, 811UBI, 764UFI, 763UnderlyingType, 747ValueType, 336, 337WeightType, 626
UBI type function, 764UFI type function, 763UnaryPredicate concept, 504underlying type, 429, 747UnderlyingType type function, 747uniform cost, 84uniform random shuffle, 545union (∪), 805unique representation, 28univalent concept, 320upper bound, 506
implementation, 554
Stepanov, McJones Elements of Programming March 14, 2008 879 / 880
Index XXVI
value, 19sort (type), 21
value type, 336ValueType type function, 336, 337van der Waerden, Bartel Leendert, 307variable area, 82
weakusage convention, 232
WeakBifurcateCoordinate concept, 626weakening of concept, 57weakly reflexive ordering, 217WeaklyHalvableMonoid concept, 283WeightType type function, 626well-formed object state, 26with buffer algorithm, 426worst-case cost, 84Writable concept, 337writable range, 362
Zabluda, Oleg, 205
Stepanov, McJones Elements of Programming March 14, 2008 880 / 880