Intro to Python Programming

8/12/2019 Intro to Python Programming

http://slidepdf.com/reader/full/intro-to-python-programming 1/28

1 Python Programming

The previous Chapter provided an overview of modeling in optimization. Theproblems considered were intentionally simple, and could therefore be solved

analytically. In practice, most optimization problems in engineering entailnumerical analysis and software programming.

In this Chapter, we discuss a particular software language, namely Python, fordeveloping various optimization algorithms. The reasons for choosing Python,over, say, Matlab or C++, are:

(1) Python is free(2) It is object-oriented(3) It is interpreted(4) It is operating-system independent(5) It has an excellent optimization module(6) It offers modern COM modules for interfacing with SolidsWorks.

In short, it is an ideal programming language for students and practicingengineers to learn optimization. In this Chapter, we highlight some of the salientfeatures of Python. A good grasp of these fundamentals is sufficient for theremainder of this text. However, the reader is encouraged to consult www.python.org, or other Python texts [References], as needed.

1.1 Getting Started with Python

The easiest way to get started with Python is to download the Python(x,y)package from http://code.google.com/p/pythonxy/. The Python(x,y) packagecomes with all numerical and scientific Python modules that we will need.

Further, included in Python(x,y) is Spyder, an excellent integrated developmentenvironment (IDE).

Once Python(x,y) has been installed, fire-up Spyder, and the following set of windows should show up. Then, in the Console window, type print ‘HelloWorld!’ and hit enter. If everything goes well, the response should be as shown.(Note that your particular configuration may be slightly different). For those of you familiar with Matlab, you will find it easy to use Spyder IDE.

You can configure the Spyder IDE through Tools-Preferences Menu. Forexample, you can add/remove line numbers on the editor window, increase fontsize, change colors etc. You can also move the file-explorer and console windows, by directly dragging/dropping them to the desired location.

The reader should be aware that Spyder can run under two Console modes,namely, the standard Python mode, and the more powerful Interactive Python(IPython) mode. We will use the IPython mode for the remainder of this text. Youcan choose the appropriate mode under the Run menu.



Figure 1-1: The Spyder IDE.

1.2 Basic Objects

Python is an object-oriented language. Indeed, everything one creates inPython is an object, including integers, float, strings, arrays, etc. Below are a few basic objects, assigned them to variables ‘i', ‘x’ and ‘a’.

Associated with objects are methods that act on these objects. By typing a ‘dot’after the object variable name, you can access a list of methods associated with it.For example, with the string object ‘a’ are methods to capitalize the first letter,find a substring, etc. This is illustrated below; the reader is encouraged tofamiliarize themselves with various methods associated with integer, float andstring objects.

One can manipulate, combine and display objects as illustrated below. Observethat the meaning of the ‘+’ operator is context dependent. For integers and floats,it is interpreted as the usual addition; for strings, it is interpreted in Python as aconcatenation.



Also, observe that, one can assign new variables to existing objects (as in b = a, below). Reassigning the variable b to a different object (in this case) does notalter the contents of a (as in b = ‘world!’).

1.3 Lists

A particularly useful object in Python is that of a list that is simply a collectionof other Python objects. Below are a few examples of list objects. Observe that: (1)lists can contain a variety of objects (integers, strings, etc), and (2) they can even

contain other list objects as in b = [34, a], (3) addition of lists leads to aconcatenation as in c = a + a, and (4) access to individual elements of a list isthrough the [] operator (as in a[2]); the indexing of individual elements of a liststarts from 0, rather than 1.

List objects are however different from integers, floats and strings in thefollowing sense. In the example below, observe that ‘a’ and ‘b’ variables areinitially ‘pointing’ to the same list object. However, modifying b, by accessing itsindividual elements, modifies the underlying list, and is therefore reflected in the variable a. This fundamental difference is to be noted.



A particularly useful method of creating a list of integers is the ‘range’ function.

Lists play an important role in this text since arrays (to be discussed later on) area special incarnation of lists.

1.4 Python Scripts

The Console window, which we used in the previous examples, plays animportant role for interactive computing. However, to develop larger programs, we rely on the Editor within Spyder. For example, we have created a simplePython program in the Editor, and saved the file as ‘2.1-PythonObjects.py’, where‘.py’ extension refers to a Python file.

You can execute the above program through the Run command (or via the F5short-cut). When the Python program is executed for the first time in Spyder, thefollowing configuration window opens up. In the window, choose the Interactive

Python (IPython) mode, and hit Run.



Then, the following output appears in the Console window.

The reader is encouraged to study the Python script and the output.In the above script file, we have included two types of Python comments: (1)

the hash (‘#’) symbol used for single-line comments, i.e., the remainder of theline following the ‘#’ is treated as a comment, and (2) the triple-quotes for multi-line comments, i.e., any text between two triple-quotes is treated as a comment;the triple-quotes also serves as a means of documenting, also referred to as adocstring, the script file (we shall revisit this topic later on).

1.5 Flow Control

As in any programming language, Python offers a variety of options for ‘flow

control’. The following example illustrates the use of ‘for’, ‘if’ and ‘while’commands in Python



Observe that there is no ‘end’ statement that identifies the end of the for-block.Indeed, in Python, the ‘end’ is implied by the consistent use of indentation. Thereader is encouraged to ‘mess-around’ with the indentation and study the impact

on the output. The output of the above script is shown below.

1.6 User Input

Python provides two commands, namely ‘raw_input’ and ‘input’ for userinteraction. The first command returns the user input as a string, while thesecond command will interpret and evaluate the input, and return the interpreted value (say a float or a list), if the evaluation is meaningful. The difference betweenthe two commands is illustrated below.



1.7 Numerical Python

Thus far, we have focused on the core Python language. However, there areother numerical objects (such as arrays) and methods (such as the dot product)that are not part of the core Python language, but are part of the numpy and scipy

libraries/modules. These libraries have been installed on your machine when youinstalled Python(x,y). However, in order to access them in a script file, you mustimport them (these libraries are automatically imported in the Console window.)

We can import each of these modules individually via ‘import numpy’, ‘importscipy’, etc. Alternately, all ‘popular’ Python modules may be imported via thecommand ‘import pylab’ as in line-2 below. Note that we have used theabbreviation ‘py’ as a synonym for pylab.

Various constants (such as π ), objects (such as an array), and methods (suchas sin, cosine) can now be accessed via Pylab, as illustrated below. The reader isencouraged to experiment with these and other methods within Pylab.

The resulting output in the Console window is shown below.

1.8 Linear Algebra

Pylab also supports numerous matrix operations; below are a few examples.



Observe, in the above script, that:• The solve function solves Ax = b

• The eigen values of a matrix are computed via the ‘eig’ function

• The ‘dot’ function performs the operation A*x

• “norm” is the usual 2-norm of a vector, defined as

( ) ( ) ( )2 2 2

0 1 1... N x x x −= + +x

The output is shown below.

1.9 Complex Numbers

Python also supports the use of complex numbers through the use of symbol

‘j’ that represents 1− . Below are a few examples to illustrate the use of complexnumbers.



1.10 Plots

The ability to plot and visualize functions is extremely valuable inoptimization. Pylab supports 2-D and 3-D plotting via matlibplot(http://matplotlib.sourceforge.net) package that can be accessed through pylab.

Below is a simple example to illustrate 2-D plotting of functions (numerous otherexamples can be found at the matplotlib web-site).

This results in the figure below; the reader is encouraged to experiment with thecode.

Figure 1-2: Plot of sin( )x and cos( )x .

Matlibplot offers various 3-D plotting routines as well; below is an example toplot the function:

2 4( , ) ( 1) ( 4) f α β α β = − + − (1.1)



This results in the plot shown below.

Figure 1-3: Surface plot of 2 4( , ) ( 1) ( 4) f α β α β = − + − An alternate means of visualizing 2-dimensional functions is through a contourplot. A contour is the set of (x, y) points for which the function ( , ) f x y takes a

constant value, i.e.,

{ }20 0( , ) ( , ) | ( , )C f c x y R f x y c = ∈ = (1.2)

We will study the importance of contours in later Chapters. Continuing with theprevious script, the following script shows two ways to plot contours usingmatlibplot.

This results in 2 different contour plots shown below.



Figure 1-4: 3-D and 2-D contour plot of 2 4( , ) ( 1) ( 4) f α β α β = − + − .

Note that:1. There are infinite contours for a function, but no two contours may cross

each other2. For every point 0 0( , )x y there exists a unique contour passing through that

point, however, for a given value of the right hand side value 0c in

Equation, a contour may not exist, or there may be multiple contours.3. At every point 0 0( , )x y , the gradient of the function, i.e., f ∇ (a 2-

dimensional vector), is perpendicular to the contour passing through thatpoint, and its direction is along increasing contour values of f .



1.11 User-Defined Functions

Thus far, we have considered ‘built-in’ Python functions from existingmodules. One can, of course, create custom functions. For example, consider thefunction

2

( ) 1 x f x xe −= − (1.3)

One can create the above function via the keyword ‘def’ in Python as shown below. The script is saved in a file called ‘myFunction.py’.

Once the above file has been created, you can load and use the function as follows(assuming that the above file is in the current directory of Spyder).

One can now also use the plot function as follows (observe the vectorizedoperation):

This results in the figure below.

Figure 1-5: Plot of2

1 x xe −− .



1.12 Modules

One can include multiple functions within a single Python file, and accesseach one of them individually (a distinct advantage over Matlab). For example, below is a file ‘SimpleFunctions.py’ containing multiple functions. For example,

the function ‘f3’ implements2 4( , ) ( 1) ( 4) f α β α β = − + − (1.4)

Each of these functions is accessible once the SimpleFunctions.py has been

imported, as illustrated below.

Further, one can query the above module via the ‘help’ command in Python:



Observe that Python has intelligently interpreted your code via the triple-quotes,and provided a useful synopsis.

1.13 Module Testing

One can test individual modules (such as the SimpleFunctions.py) byincluding a ‘main’ function as part of the file. For example, continuing with theabove script, we add the following lines of Python code (observe the specialsyntax of line-34).

One can now run the SimpleFunctions.py module (through F5 shortcut), and theresult is shown below.



Figure 1-6: Plot of

2

1 x xe −− , and its gradient.

1.14 Function Arguments

Python offers a rich set of language features for passing arguments intofunctions. For example, consider the function ‘f1’ below (together with a testingscript). The function takes up to 3 arguments: the argument ‘x’ does not have adefault value, while the argument ‘a’ takes the default value of 4, and ‘s’ takes thedefault value of ‘hello’. Any variable that does not have a default value must bespecified when calling functions.

In any call to a function, arguments can be passed either by position, f1(0.3),or keyword, f1(x=0.3). This allows for a highly flexible and natural way for

passing data to a function. When passing by position, one must respect theposition defined in the function description, but when passing by keyword anyorder is acceptable. In addition, when mixing position and keyword styles, allpositional arguments must come before keyword arguments.

The result of running the above script is displayed below.



1.15 Python Quirks

There are a few Python ‘quirks’ that one must keep in mind. Consider thefollowing division of two real numbers in Python; the result is as expected

Now consider the following.

Python returns the integer component of the result. This implicit integerinterpretation is common to many languages, and can cause headaches innumerical computation; the reader should therefore be very watchful of implicitinteger conversion. One can override this behavior as follows

In Python modules, the import statement must appear at the first line.Henceforth, we shall include the above statement in all Python modules.

A second ‘feature’ unique to Python is the strict adherence to indentation atthe beginning of an expression. Consider the following code:

The only difference between this and the previous code was the inclusion of anextra ‘space’ before the number of ‘5.0’. Python does not allow extraneous use ofspace or tabs at the beginning of expressions. The underlying reason is that ituses spaces and tabs to identify blocks of code associated. We shall revisit thisconcept again later on.

The third aspect is particularly relevant to readers familiar with Matlab™.Consider the multplication of a 2x2 matrix and a 2x1 vector. In Matlab, one would construct a matrix and vector as follows:



Then, the multiplication in Matlab is as follows:

Mathematically, in Matlab, the '*' operator is interpreted as:

i ij j

j

b A x = ∑ (1.5)

In Python, the syntax for creating matrices and vectors is a bit more elaborate:

Now consider the following multiplication in Python:

Unfortunately, this creates a matrix b:

In Python, the '*' operator is interpreted as:

ij ij j b A x = (1.6)

The desired matrix-vector multiplication of Equation (1.5) is achieved in Pythonthrough the 'dot' method:

The reader has been warned!

1.16 Sampling Algorithm

As an illustrative example of a Python program, consider an algorithm thatfinds the global minimum of a one-dimensional function ( ) f x within a given

interval [ , ]a b through uniform sampling.

The Python script below contains the function ‘Sampling’ that takes asarguments 3 parameters: (1) a function name that must be minimized, (2) aninterval to search within, and (3) number of sampling points.



Its usage is as follows.

We shall compare the performance of this algorithm against more sophisticatedoptimization algorithms later on.

1.17 Newton-Raphson AlgorithmThe Newton-Raphson algorithm is used extensively to solve non-linear

equations of the form:( ) 0=g x (1.7)

The theory behind Newton-Raphson algorithm is discussed later in the text, but

briefly, the algorithm is iterative in that given an initial guess 0x , it typically finds

increasingly better solutions via:11 ( ) ( )i i i i −+ = − x x J x g x (1.8)

where ( )J x is the Jacobian matrix, associated with Equation (1.7), defined as:

i ij

j

g x

∂=∂

J (1.9)

Example 1-1: Find a solution to the set of nonlinear equations:

2 2

2 2 4 0

2 2 0

4 3 0

x x

y y

x x y

µ µ

µ

+ − =

+ =

− + + =

(1.10)



Solution: Note that the 3 unknowns are { , , } x y µ =x . As the reader can verify,

the Jacobian is given by:

2 2 0 2 4

0 2 2 2

2 4 2 0

x

y

x y

µ

µ

+ −

= +

−

J (1.11)

Let 0 {0,0,0}T =x ; thus

0

0

( ) 0

3

=

g x and 0

2 0 4

( ) 0 2 0

4 0 0

−

= −

J x (1.12)

From Equation (1.8) { }1 0.75 0 0.375T

=x , and

1

0.5625( ) 0

0.5625

=

g x and 1

2.75 0 2.5( ) 0 2.75 0

2.5 0 0

−

= −

J x (1.13)

leading to { }1 0.975 0 0.8475T

=x . After few more iterations, the solution

converges towards { }* 1 0 1T

=x , the exact solution.

♦ In practice, it is challenging to compute the Jacobian analytically via Equation

(1.9). Instead, a finite difference approximation may be used:( ) ( )i j i

ij

g he g

h

+ −≈

x xJ (1.14)

where 1 2{1, 0,.., 0} , {0,1,.., 0}T T e e = = , etc.

We implement the Newton-Raphson method via a Python method 'fsolve'(mimicking the Matlab function with the same name). The first few lines of thefsolve module is as follows:



Observe that the function definition has two essential arguments: (1) the function

( )g x and (2) the initial guess 0x . Optional keyword arguments include the

Jacobian function, tolerances for termination and the maximum allowediterations.

If the Jacobian function is not provided, we rely on equation (1.14) to compute

an approximate Jacobian; its implementation is as follows:

Observe that the above function is defined inside the fsolve function. Then, themain Newton-Raphson iteration is implemented as follows:

The reader is encouraged to study the above Python implementation.

The 'fsolve' function may be used as follows . First the g function is definedper Equation (1.10):

One can now solve the nonlinear equations via:

Optionally, the Jacobian of Equation (1.11) may be defined via:



With this additional function, one can solve the problem via:

In both cases, the solution converges to { }* 1 0 1T

=x .

1.18 Dynamic Function Synthesis

Sometimes there is a need to dynamically synthesize or create a function. Inother words, the nature of function is determined during run-time. We will comeacross such a scenario in Chapter 4. Here, we provide an illustrative example ofdynamic function synthesis.

Consider a scenario where a function ( ) f x is defined as follows:

sin( ); 0( ; )

cos( ); 0

x f x

x

β β

β

≥ = <

(1.15)

In other words, depending on the value of β , the function is defined dynamically.

In Python one would implement and test this as follows.

Observe that the function ‘fCreate’ returns a function object. The functionreturned by ‘fCreate’ depends on the auxiliary variable β . Once the function is

created, it can be used like any function in Python.



1.19 Logging in Python

Python also provides excellent tools for debugging large software projects.One such tool is ‘logging’. Below is an example to illustrate the use of logging inPython. Observe that we have imported the logging module. There are various

methods for configuring logging; a few different options are illustrated below. Inparticular, the ‘level’ argument can be used to determine the behavior of thelogging utility. We shall make extensive use of logging later in the text.

When the above is executed, we see the following output on the console.

Further a file ‘example.log’ is also created, with the following contents:



1.20 Python Class

Finally, we discuss the important concept of a ‘class’. A class, in objectoriented languages such as Python, is a collection of objects and methods that areclosely related. We illustrate the notion of a class through a simple example of a

‘polynomial class’. The objective is to treat polynomials of the form:2 10 1 2 1( ) ... N

N p x a a x a x a x −−= + + + + (1.16)

as objects with intrinsic methods such as evaluation of polynomials at a point,addition of polynomials, etc. The polynomial in Equation (1.16) is fully defined bythe array of coefficients:

{ }0 1 2 1... N a a a a −=a (1.17)

For example, the following polynomial:2( ) 1 3.2p x x x = − + (1.18)

is captured by the coefficients:

{ }1, 1, 3.2= −a (1.19)

In Python, a Polynomial class may be defined via the following syntax:

Observe the use of the keyword ‘class’, and the keyword ‘self’ that allows access toquantities associated with a polynomial object. The ‘py.array’ is used to copy ‘aIn’so we do not experience the quirk we saw when dealing with lists. Once the aboveclass is defined in a file ‘PolynomialClass.py’, we can create a polynomial object asfollows:

The above code, by convention of the class, creates the polynomial object

representing: 2( ) 1 3.2p x x x = − + (1.20)

The class does nothing more at this point, since we have not associated any behavior with this object. Let us now overload the ‘__str__’ method so that‘print’ provides a human readable form:



The above code essentially retrieves the coefficients and creates a string fordisplay; the reader is encouraged to understand the logic behind the code. Wecan now display the polynomial object created earlier as follows:

We can also add a method to evaluate polynomial objects at a particular point:

Its usage is:

The reader can verify that the reported value is indeed correct.

Next, we consider the addition of polynomials of the same order. Recall thatthe addition operator takes on different interpretation depending on the objectassociated with it. For integers and floats, it is interpreted as the usual addition, while for strings and lists, it is interpreted as concatenation. We now overload theaddition operator for the polynomial class, with the natural interpretation:

( ) ( ) ( ) ( )

2 10 1 2 1

2 10 1 2 1

2 10 0 1 1 2 2 1 1

( ) ...

( ) ...

( ) ( ) ...

N

N

N

N

N

N N

p x a a x a x a x

q x b b x b x b x

p x q x a b a b x a b x a b x

−−

−−

−− −

= + + + +

= + + + +

⇒

+ = + + + + + + + +

(1.21)

In the Polynomial class, we create a new method with the keyword ‘__add__’that overloads the ‘+’ operator:



Its usage is:

We can continue to add other methods (such as subtraction of polynomials,multiplication of a polynomial by a scalar, etc).

1.21 Exercises

Exercise 1-1: Under Python command prompt, evaluate the following

expressions: (a) 3 310 1 10+ − , (b) 16 1610 1 10+ − , (c) 51 10 1−+ − , and (d)161 10 1−+ − . Explain the apparent discrepancies.

Exercise 1-2: Explain the results of the following Python script

Exercise 1-3: Solve, using Python the 3 linear algebra problems:

2 3 4 5 2 3 4 5 2 3 4 5

6 2 2 6 2 2 6 2 2

5 2 6 1 5 15 2 11 5 15 2 15

x y z x y z x y z

x y z x y z x y z

x y z x y z x y z

+ + = + + = + + = − − = − − = − − = − + = − − − = − − =

(1.22)

Explain the results.



Exercise 1-4: Find the eigen-values and eigen-vectors of the above matrices.Now explain the conclusions of Equation via eigen-values and eigen-vectors.

Exercise 1-5: Consider the differential equation with boundary conditions

2

2 1; (0) (1) 0d f

f f dx = − = = (1.23)

Pose and solve the above differential equation via central finite difference. Thetheory to convert differential equation problem into a linear algebra problem viafinite difference. Also see the diag command in Python to capture the resultingfinite difference matrix.

Exercise 1-6: Using sampling, find the minimum location (to within 0.001) of

the function2

( ) 1 x f x xe −= − . Verify that the gradient of the function is nearly

zero at that location.

Exercise 1-7: Using sampling, find the location where the derivative of2

( ) 1 x f x xe −= − is almost zero (to within 0.001). Compare the result with the

previous exercise.

Exercise 1-8: Write a Python function ‘binSearch.py’ that asks the reader tothink of an integer between 0 & 1000. Then guess this number using no morethan 10 questions for which the user must reply with a y/n (yes or no).

Exercise 1-9: Write a Python function ‘coins.py’ that asks the reader to input aninteger between 1 and 100 (cents). Then the program should return the leastnumber of coins to reconstruct the input integer. Allowed coins are: a penny (1cent), nickel (5 cents), dime (10 cents) and quarter (25 cents). For examples,suppose the user specifies 38 cents, the desired solution is 1 quarter, 1 dime, 3pennies.

Exercise 1-10: Use graphical means to find the minimum location (to within

0.001) of the function2

2( ) 1 x f x xe −= − . Verify that the gradient of the function is

nearly zero at that location.

Exercise 1-11: Using contour plots, find the minimum location (to within aradius of 0.001) of the function

( )( ) ( )

22 2

22 2

1

100 ( 1) 12( , )1

500 ( 1) 1 10 82

u v f u v

u v u v

+ + − + = + − − − +

Verify that the gradient of the function is nearly zero at that location.



Exercise 1-12: Modify the Polynomial class so that one can add Polynomials ofdifferent orders. Include the following test case:

2( ) 1 3.2

( ) 1

( ) ( ) ( ) ?

( ) ( ) ( ) ?

p x x x

q x x

r x p x q x

r x q x p x

= − +

= +

= + =

= + =

(1.24)

Exercise 1-13: Expand the Polynomial class to include subtraction ofpolynomials of different orders. Include the following test case:

2( ) 1 3.2

( ) 1

( ) ( ) ( ) ?

( ) ( ) ( ) ?

p x x x

q x x

r x p x q x

r x q x p x

= − +

= +

= − =

= − =

(1.25)

Exercise 1-14: Expand the Polynomial class to include multiplication ofpolynomials of different orders. Include the following test case:

2( ) 1 3.2

( ) 1

( ) ( ) ( ) ?

( ) ( ) ( ) ?

p x x x

q x x

r x p x q x

r x q x p x

= − +

= +

= =

= =

(1.26)



Intro to Python Programming

Documents