Python Final

CYBER SECURITY INFORMATION TECHNOLOGY CRITICAL INFRASTRUCTURE HOMELAND SECURITY MULTI-USER NETWORK CYBER SECURITY INFORMATION TECHNOLOGY CRITICAL INFRASTRUCTURE

State of Colorado Office of Cyber Security

AppSec (By Rich Helton)

Python for

Security Professionals (September 21, 2011)

(with some Droid and Django)




Class Introduction

  My information, [email protected]

  A good reference for an online Python class is http://code.google.com/edu/languages/google-python-class/introduction.html , PyQuick Python Class from Google University.   Another place for reference Python Documentation, http://www.python.org/doc/



Python History

  From http://en.wikipedia.org/wiki/Python_(programming_language) ,   Python is an interpreted, general-purpose high-level programming language whose design philosophy emphasizes code readability.   The reference implementation of Python (CPython) is free and open source software and has a community-based development model, as do nearly all of its alternative implementations.   Python interpreters are available for many operating systems, and Python programs can be packaged into stand-alone executable code for many systems using various tools.   Python’s implementation was started in December 1989 by Guido van Rossum in the Netherlands as a successor to the ABD programming language.



Installing Python

  Python can be downloaded and installed from http://www.python.org/getit/



Just my 2 Cents,

  Referencing others, Python does not define types ahead of time, but uses declaration at runtime. Many say the variable names in Python have to be watched more closely due to this interpreted runtime.   You need to be prepared to create modules and break apart lines more often to keep readability more apparent, while a statement in Java code could easily occupy three lines of text, Python works with line breaks and indentation and cannot easily occupy multiple lines for the one statement. This concept ensures Python as a more compact language and the need to occupy less space.   This in turn makes Python able to get a lot more done with fewer lines of code and to be great at utilities and tools. Python is concise.   Keep it simple,



Other Advantages

  Python does not require as much typing as Java and integrates more into the Operating System as a scripting language.   While Java has many more frameworks than Python, Python does have a Spring Framework (Middleware) and many Web Frameworks.   I like to work with Python for admin and 3D graphics integration:

  If you do graphics programming, Python is a must have. There are packages like Maya, that uses Python plugins, Blender 3D, and Disney’s original Panda3D application is based on Python.   If you manage Linux systems and do network and system security, Python is a language that many scanning tools are based on.   Python is a great language for incorporating modules from other languages and applications.   For a list Python software, see http://en.wikipedia.org/wiki/List_of_Python_software



Python Software



Java vs. Python

  A simple “Hello World” (Python is concise, see left pane):



Panda3D (Free Python 3D Engine)

  Python can be powerful, see Disney’s game engine, http://www.panda3d.org/



Panda3D (Free Python 3D Engine)

  Panda3D Boxing Program Sample, nothing but Python.



Pygame

  One of the most well known frameworks regarding python is pygames, found at http://www.pygame.org/ and information found at http://en.wikipedia.org/wiki/Pygame   Pygame has been around 2004.   It has many tutorials and editors like a Tile and Level Editor, and even a Map Maker:



Pygame Aliens game



Pygame

  Pygame has a subset for the Android, see http://pygame.renpy.org/ .   To learn the code for Sprite animation for Aliens, simply go through the code at http://svn.pythonmac.org/py2app/py2app/trunk/examples/pygame/aliens.py



My First Python Program

  Many of programs will be done in Windows and by installing python. Linux can be used as well and python normally comes standard in Linux.



My First Python Program (with Error)



Python, not just a Calculator

  Python can perform complex calculations with little effort. Don’t forget to use numbers instead of literals, and note the # (comment symbols):



Creating a py file

  Python files normally end with a “.py” extension.   Using a Open Source editor, you can write a Python program, like http://geanyportable.org/ or http://www.pspad.com/ .   We create prog1.py:



Running a py file

  We can run the “py” file from the IDLE GUI shown as the interpreter to interactively run it.   Or at the command line:



Running a py file

  Other editors, like http://netbeans.org , a Java editor, have plugins to run Python files. See also http://wiki.netbeans.org/Python .   A plugin for Python will have to be installed:



Running a py file

  Running in NetBeans:



Debugging a py file

  Debugging in NetBeans, here we are line by line debugging and printing out the www.google.com webpage in the output window :



Debugging a py file

  Let’s break down the code:



Debugging a py file (Reading a Google URL)

  We can set the request name as a variable. The variable does not require a type.



Using it across the WWW

  In this example, we grab the page to www.google.com and read the output.   We can now start to use this as a web crawler. A Web crawler will grab the URL from the site and walk down it and then grab the next URL, and walk down it, in many cases copying the pages into a local directory to capture the web site structure.   Some even come with Linux operating systems, like wget.   Building your own crawler allows you to test other items as well, such as putting a ‘ or “ in the forms to test for SQL Injection.



• Wget • (Open Source Web downloaders)

  Open Source utilities, like wget, or VisualWget http://www.bebits.com/app/299 , provide a means to download all available URL’s on a Web Site:



• Wget • (Open Source Web downloaders)

  Results:



Python Web Crawlers

  I will discuss the Python implementations more at a later time, but here are some links to some Python Web Crawlers.   There are many sites that discuss Python Web Crawlers, the benefit is that you can add to look for specific patterns:

 Simple Web Crawler http://code.activestate.com/recipes/576551-simple-web-crawler/   http://ms4py.org/2010/04/27/python-search-engine-crawler-part-1/   http://blog.theanti9.com/2009/02/14/python-web-crawler-in-less-than-50-lines/

 There are many sites that discuss Python ping.py implementation that I will discuss in the network section, that may help in network mapping:

  http://stackoverflow.com/questions/316866/ping-a-site-in-python   http://www.g-loaded.eu/2009/10/30/python-ping/



Some Python Tools



PyLint

  PyLint is a static source analysis tool, available by command line or integrated into tools like PyDev and frameworks like Django   See http://en.wikipedia.org/wiki/Pylint   After downloading the PyLint package and unzipping it from http://pypi.python.org/pypi/pylint#downloads , run the “python setup.py install” in its root package to install it to the local Python application.   It will install it under the local Python\Scripts directory, so make sure that this directory is in the executable Path environment.



Running PyLint

  For our “test.py” file, we simply run “pylint test.py”:



PyDoc

  The pydoc module automatically generates documentation from the Python modules. It can be presented as text in the console, server to a Web browser, or saved to HTML files. http://docs.python.org/library/pydoc.html   Running “python pydoc.py sys” from the Python\Lib directory gives us info on the sys module:



PyDoc

  Running “python pydoc.py –g” will bring up the Tkinter GUI interface to search and a button to load a browser with all the modules:



PyUnit

  This is the Python version of Junit, http://pyunit.sourceforge.net/ , which is the standard unit testing framework.   Download it from this website and install it with the “python setup.py install” cmd.



PyUnit

  PyUnit makes use of the “unittest” module from Python, so it must be installed. See http://pyunit.sourceforge.net/unittest.html   The Python file will need an “import unittest” self using functions to design a unit test. See http://docs.python.org/library/unittest.html



PyUnit

  Here we have “ArithTest” that will use “self.fail” functions to test the cases:



Creating a Python Executable



Download the py2exe.org

  Download the correct version from http://www.py2exe.org to match the Python interpreter installed, i.e. 2.7.   In this case it will be installed in the 2.7 directory….



Quick test

  Let’s cd to the download sample directory and run the simple sample. The directory can be viewed from the installation dialog.   The test will show an error. This is because there are multiple Python versions and that’s why is didn’t compile. Panda3D was also installed and installed 2.6.5, OOps:



To Compensate (Multiple Pythons)

  When applications are installed with Python support, it may install another copy of Python.   You can de-install down to one copy, and use a copy of Python that is supported by all the local Python applications, likely not the most current, or start compiling directly from the local directory.   In this case, we will call the correct python directly and copy the simple samples over to the python directory.   Then we will run the local “python setup.py py2exe”. This setup.py came with the sample and defines to compile a hello.py.   There will be several pages of compile information ending with:



Compiled

  A “dist” directory will be created from the samples with the compiled python scripts.   The hello.py is now hello.exe:



Non-Windows Multiple Pythons

  You may notice that many Python files may have the first line with something like “#!/usr/bin/python”. This means to use the python under “/usr/bin” in Linux or MAC OSX.   You may even see “#!/usr/bin/env python2.6” that simply means to use the first Python 2.6 binary found in the user’s environment.   Depending on what you put in this first line will control the Python binary to use, if not used, it will find the default one in the current environment. See http://stackoverflow.com/questions/2429511/why-do-people-write-usr-bin-env-python-on-the-first-line-of-a-python-script



LET’S BREAK DOWN THE PROGRAM



Components of a program…

  Comments   Reserved words   Statements   Blocks   Classes   Methods   The main method



Comments

  Comments are used to communicate the purpose of parts of the program in the code.

  Comments are not executed as part of the code and are ignored by the compiler.

  Using the (#) double slashes will tell the compiler to ignore the entire line.

  Using the (“ “ “) triple quote multiline comment to comment code it meets the ending ending triple quote (“ “ “).



Reserved Words

  Reserved words, or keywords, are words with special meaning that define how the program should behave in the compiler.

  These are words will explain to the compiler how to construct the program.

  Being reserved to the compiler, these words cannot be duplicated anywhere else in the code as class names, variable names, or method names.

  Words in the program, such as “assert” tells the compiler to asset that a statement is true or it will return an exception.



Reserved Words

  Python reserved words:



Conditionals

  Conditionals, if/else clauses, are probably the most important construct in programming.

  It checks to see if a condition is true and executes its block, otherwise it may execute a “else” if set. http://docs.python.org/tutorial/controlflow.html



For Loop

  A “For” loop will increment through a specified length. Length can be used instead of the integer.

  Remember to indent the code within conditions, loops, and definitions, because the indentation defines the code within the block.



For Loop 2

  Let’s create something more useful, like printing out the OS environment



Operating System Commands

  Python has many Operating System (os) commands that are part of the Python os module, see http://effbot.org/librarybook/os.htm and http://docs.python.org/library/os.html

  These commands can create and remove files, directories, get current environment variables, get information on files and directories, change permissions on files and directories, start programs, end programs, start services and daemons and mush more.

  We will look at it more in a bit, back to loops.



While Loop

  A While loop will continue until a condition is met, this is example is in cygwin, like Linux, so it has a header “#!/usr/bin/python” :



Statements

  A statement is line for a singe unit of work.   A line in Python is ended just at the end of line in the file.



Blocks

  A block is a group of actions that is normally defined in a method, condition, exception, loop or other area construct.

  A block is started with the “:” in the section, defined by indentation until another block is encountered.



Main method

  Python with multiple functions need to define a “main” method, it defines the method in the “if __name__ == "__main__":” block.



Object Oriented Development

  Python supports Object Oriented Programming (OOP), here is an example, see http://www.zugiart.com/notes/programming/python/ :



What is a class?

  A class is a definition of a group of methods, variables, state and behavior.

  In the sample, the class is MyClass.   As MyClass it can contain variables, methods, state and behavior

that are defined. In this case it simply prints “helloWorld” with an argument.

  The idea of a class is that is can define a car, a catalog, and another group of a construct.

  The class can create subgroups, like a car class can have sub classes of itself like “blue car” or “compact car” to define the construct in more detail.

  When a class is created and started, called instantiation, it becomes an object. A class can create many objects.



Creating a module

  Python statements are not very long compared to other languages, and by that I simply mean that while I could easily write a statement that takes 3 lines of code, Python delaminates on indentation and end of line.

  To compact the code, modules are used to break the code into different libraries, just like functions and classes are used to break the code up into different blocks in the file.

  See http://docs.python.org/tutorial/modules.html



Creating a module

  Let’s create a Fibonacci numbers module with a “fib” function:



Creating a module

  Let’s call the fib.py module and “fib” function from a myFibo.py file using the “import” keyword:



OS Module



OS Module

  We briefly touched the os module in the last chapter. We will walk through a few programs (Getting File info, this was Windows):



OS Module

  Here we copy a file and give it the same info as the original:



OS Module

  Changing to a subdirectory called samples and back again:



Lists, Sets and Tuples



Lists

  A list in Python is an ordered group of items or elements. http://en.wikibooks.org/wiki/Python_Programming/Lists

  They use “ ( … )” parenthesis and their data can be changed.



Tuples

  Tuples in Python are much like lists, except they are immutable, meaning once it is created, it is never changed. See http://en.wikibooks.org/wiki/Python_Programming/Tuples

  Notice parenthesis are used instead of brackets.



Sets

  Sets in Python are a type of mathematical collection that supports set operations like unions, intersections, differences and more. See http://en.wikibooks.org/wiki/Python_Programming/Sets



Types of Web Hacking



• Web Attacks are the most common • (from the 2010 ArcSight survey)



• There are many hacks….

  And more are discovered every day. For 2010, the Open Web Application Security Project (OWASP) published the top ten web hacks, http://www.owasp.org/index.php/Category:OWASP_Top_Ten_Project :

1)  Injection Flaws 2)  Cross Site Scripting (XSS) 3)  Broken Authentication and Session Management 4)  Insecure Direct Object Reference 5)  Cross Site Request Forgery (CSRF) 6)  Security Misconfiguration 7)  Insecure Cryptographic Storage 8)  Failure to Restrict URL Access 9)  Insufficient Transport Layer Protection 10)  Unvalidated Redirects and Forwards



• SANs 2010 Top Cyber Security Risks



There are many Checklists….

  Here are some checklists to help with the principles:   The ASP.NET Security Checklist

http://msdn.microsoft.com/en-us/library/ff648269.aspx   WebAppSec Excel checklist

http://img.a4apphack.com/dl/appsecchck-checklist.zip   SANs reading Web Security Checklist

http://www.sans.org/reading_room/whitepapers/securecode/security-checklist-web-application-design_1389

  The Open Web Application Security Project Application checklist is http://www.sans.org/reading_room/whitepapers/securecode/security-checklist-web-application-design_1389



Checklists Principles seem the same…

  All the checklists, regardless of programming languages, have many of the same principles:

  Find and validate all input. This includes URL’s, JavaScript's, links, username and passwords, and especially any field calling a database.

  Never trust data in files, the network or database to be secure. Encrypt anything important, passwords, SSN’s, configurations.

  Never trust the source, be it customer or a service. Authenticate, Authorize and validate.

  Whenever a abnormal behavior occurs, error check and log.

  Keep testing, as people from all skills will be testing anything on line and may try common threats.



SQL Injection (Most common Injection

Flaw)



Intro to SQL Injection…

  Many web pages communicate directly to a backend database for processing.   For example, a username and password is asked for on the Web page and the web page will pass it to the database to validate the information.   Some applications will not validate the field adequately before passing it to the database, and the database will process whatever it will receive.   Hackers will pass SQL commands directly to the database, and in some cases tables like “passwords” are returned because the SQL commands are not being filtered adequately.   SQL may return errors in the web page that even lists the correct tables to query so that the hacker may make more accurate attempts to get data.



SQL Injection

  SQL Injection is the ability to inject malicious SQL commands into the backend code.

  For example: SELECT * FROM users WHERE username = ‘USRTEXT '

AND password = ‘PASSTEXT’   Passing ' OR 1=1-- in the USRTEXT field generates: SELECT * FROM users WHERE username = ‘’ OR 1=1 -- '

AND password = ‘PASSTEXT’   The OR 1=1 returns true and the rest is commented out



• Live sites…

  There are live web sites that simulate banks and other businesses to practice web hacking and test hacking tools against. They include:   Spi Dynamics - http://zero.webappsecurity.com/   Cenzic - http://crackme.cenzic.com/Kelev/view/home.php   WatchFire - http://demo.testfire.net/   HackThisSite - http://www.hackthissite.org/   NTO - http://hackme.ntobjectives.com/   Accunetix - http://testaspnet.acunetix.com/login.aspx



SQL Inj with Python

  We are going to http://demo.testfire.net/bank/login.aspx , and looking at the site, we will see that Username is the field “uid” and Password is the field “passw”:



SQL Inj with Python   Running the following script will put a ‘ (tick) in uid and ‘ (tick) in

passw:



SQL Inj with Python   The results, OleDB errors



SQL Inj with Python   If we were to load the results in the browser:



SQL Inj with Python   The OleDB starts to give us enough information to start looking at

the database to see if we can get through. We can see that the field in the database is “username”.

  While Python simulates what we can do manually, automating the process allows us to start looping though the website to find more information faster.



Jython



Jython

  If you installed NetBeans correctly, you would notice that Jython was installed as well.   Jython, http://en.wikipedia.org/wiki/Jython , is Python for the Java platform.   The language can be found at http://www.jython.org/ .   Python can run in a Java Virtual Machine (JVM) and call Java Frameworks that will extend Python. Therefore, Python can run from any machine that will run Java.   This also means that Java can be run from the Python Language.   Jython can be downloaded and installed from http://www.jython.org/downloads.html   Installation Instructions are found at http://wiki.python.org/jython/InstallationInstructions



Jython

  After installing Jython by running the installation JAR, we run the jython.bat (in XP) under the jython\bin directory.   It gives us a command prompt similar to Python, and now we can run Java frameworks from the Python language, here we will call the Java Util package’s Date class:



Strings



Strings

  A String is just a form of representing text.   A link for various functions in types, including strings is http://docs.python.org/library/stdtypes.html .   We can use the python command line interpreter to fill in a string variable “fred”:



Cases

  We can check and set fred to a lower case:

 Now upper:



Regex



Regex

  Regex is short for Regular Expression, see http://en.wikipedia.org/wiki/Regular_expression   Regex is simply a form pattern matching to strings or text, and a method of strings for finding these patterns.   A large library of various Regex expressions can be found at http://www.regxlib.com/   A pattern for finding SSNs can be found at Regex Lib for the following:



Regex

  Let’s test the previous SSN Regex. It will show a “matched” string if it matches (It matches):

  Also see http://www.tutorialspoint.com/python/python_reg_expressions.htm



Regex Cheatsheet

  There are many Cheat Sheets to help people learning Regex, one of my favorites is at http://regexlib.com/CheatSheet.aspx :



Secret Writing (Encryption)



Who’s seeing your data?

  Data at rest and in transient can be at risk to prying eyes.   When a system is in production, and especially on the Internet, there is no guarantee that you know who is watching the data transmitted between the user and the server. This may also apply to the Local Area Network as well.   Never take it for granted that access cannot be broken.   Always, use common algorithms that come with Python, Java or C#.   Common algorithms are tested well and are vetted by millions.   Keep the keys as secure as the data, because they can unlock the data.   Homemade encryptions algorithms may end up costing more than standard encryptions because the algorithm may be broken.



One-way Hash Algorithms

  There are two common types of one-way hash algorithms, Message Digest 5 (md5), and Secure Hash Algorithm 1 (sha1).

  The one-way hash generates a fixed size hash from some given data of any size.

  The data cannot be reversed engineered from the hash, hence one-way.

  The same data generates the same hash sum.  Different data generates different hash sums. (Note: In rare cases, collisions, different data generates the

same sum).



Md5

  Message Digest 5 (md5) will take data input and generate 128 bit hash sum.

  The 128 bit hash sum can be used to ensure if there has been tampering of data or a file.

  A common comparison is to store hashes in a table instead of the original password. Instead of checking the password, compare the hash of the password, so that the password does not have to be stored, and only a hash is used to check the original password. Therefore the original password is never stored or seen, only the hash.



Md5

  Notice that the same values being hashed produce the same hash:



Sha1

  Secure Hashing Algorithm 1 (Sha1) will take any size byte array and produce a 160 bit hash sum, sometimes called a message digest.

  Other SHA’s are SHA224 (224 bits), SHA256 (256 bits), SHA384 (384 bits), and SHA512 (512 bits), each one denoting the size in bits of the message digest.



Sha1

  Only changed 2 lines of code from Md5:



AES

  The National Security Agency (NSA) updated their algorithm in 2001 to the Advanced Encryption Algorithm (AES) for Top Secret information from the Data Encryption Standard in 1975.   The Rijndael algorithm was selected, developed by two Belgian cryptographers, Joan Daemen and Vincent Rijmen.   The NIST adapted the variable key space into 128, 192, or 256 bits as FIPS 197 and called it AES.   AES is a symmetric key algorithm, meaning that the same key is used to both encrypt and decrypt.

 To use AES algorithm in Python, a library like the PyCrypto module has to be installed, one can be found at http://www.voidspace.org.uk/python/modules.shtml#pycrypto



Python Crypto Modules

  There are several Crypto modules for Python, PyCrypto seems to be very popular and supports many algorithms, but for a 2010 comparison of many of the popular modules, please see http://mikeivanov.com/pc/python-crypto.pdf



AES Python Code

 Initialing an Initialization Vector for AES, and the password , Cipher Mode, and Salt. They must match on both sides. :



RSA

  The most important Asymmetric algorithm to understand is the Rivest-Shamir-Adleman (RSA). So named after the MIT mathematician inventors in 1978.   The Asymmetric algorithm can generate key pairs, one private key for encrypting, and its pair is handed out for decryption to more people, the public key.   The key pair are formulated from a pair of prime numbers using a modulus equation that become linked to each other, therefore only pieces of the formula need to be exchanged and not the complete keys, and only their side that they use is calculated.



RSA Keys, a simple example

1) Choose two prime numbers p and q. P = 61 and q = 53. 2) Compute n = pq, n = 61 * 53 = 3233. 3) Compute the totient φ(n) = (p – 1)(q – 1) =

(61 -1) * (53 – 1) = 3120. 4) Choose a coprime e (like 17) that is not a divisor of the totient. 5) Compute d such that e*d mod φ(n) = 1.

17 * 2753 (d) = 46801, 46801 mod 3120 = 1. public key = (e, n) = (17, 3233) private key = (d, n) = ( 2753, 3233)



RSA Keys, a simple encrypt/decrypt

public key = (e, n) = (17, 3233) private key = (d, n) = ( 2753, 3233)

To compute the ciphertext we use C = Pe (mod n). For example, P = 65 and is the letter ‘H’. C = 2790 = 6517 mod 3233.

Back to Plaintext, P = Cd mod n. P = 65 = 27902753 mod 3233. Which returns 65 for ‘H’.



RSA

  Using a RSA Python program from http://www.prasannatech.net/2008/09/rsa-encryption-algorithm-large-integers.html, we can perform these complex calculations. The program selected e = 7 as the quotient (Python helps with complex calculations):



RSA

  Here’s a RSA example with PyCrypto, similar to the AES sample:



Files



Files

  Reading and writing files is a basic fundamental of a programming language.   Many of the functions for File I/O will simply be a read or write function using a File class.   Some of this can be referenced from the Python documentation at http://docs.python.org/tutorial/inputoutput.html



Writing a File



Reading a File



Directory

  Python is really handy for doing directory structure traversal.   Some of the available file and directory routines can be seen at http://docs.python.org/release/2.5.2/lib/os-file-dir.html   Here’s a quick program to list directories with an argument of the directory, I used “.” for the current one, (used Notepad++) :



XML



Extensible Markup Language (XML)

  XML is a specification for creating custom markup languages.   An example of a language that XML can create is Hypertext Markup

Language (HTML). HTML is the markup language for displaying web pages.

  XML is very flexible because it uses tags to describe data elements: <book>This is a book... </book>

  An XML file is commonly use to transfer language elements over HTTP in the form of Web Services or AJAX.

  The XML file must also contain the proper header information to state that the XML version and encoding scheme: <?xml version="1.0" encoding="UTF-8"?>

There are several types of parsers for XML but the basic types are Document Object Model (DOM) and Simple API for XML (SAX).



HTMLParser

  HTML is one of the languages generated from XML. In other words, HTML is a meta-language and a subset of XML.

  XML creates the HTML grammar from its specific Document Type Definition (DTD)

  Python includes a HTMLParser, see http://docs.python.org/library/htmlparser.html

  Let’s parse (Looks like XML):



HTMLParser

  This is why Python is powerful, not a lot of code:



Running HTMLParser

  Running it, the program prints the start and end tags:



SAX

  SAX, Simple API for XML, see http://docs.python.org/library/xml.sax.html, python will use the xml.sax module.

  It basically opens the XML file with an XML reader, and passes the file to a handler.

  The handler will traverse the XML file, parsing the tags for specific input until it finishes with the file.

  SAX operates on the individual tags where DOM operates on the document as a whole, http://en.wikipedia.org/wiki/Simple_API_for_XML



SAX

  Looking at a library.xml:



SAX

  The file for parsing out the book and author tags:



SAX

  Running the file:



DOM

  Document Object Model (DOM) is the XML parsing technique used by most browsers, see http://en.wikipedia.org/wiki/Document_Object_Model

  The DOM parser uses a Document Type Definition file that defines the elements in the XML file that makes up the markup language.

  The program loads the entire file into a structured hierarchy, usually storing elements in tree collection.

  DOM requires more RAM to load the files in the memory all at once.

  Python makes use of minidom, a lightweight DOM implementation using the “xml.dom.minidom” module, see http://docs.python.org/library/xml.dom.minidom.html



DOM

  Let’s parse out “books.xml”:



DOM

  We will use “pprint”, Data pretty printer to print the data, see http://docs.python.org/library/pprint.html .

  The code to parse it may look something like:



DOM

  Running the program will produce the following, which is a printout of the different elements:



Databases



Databases   Writing and reading from a database is a rudimentary skill of any programming language, and Python works with most common databases, see http://wiki.python.org/moin/DatabaseProgramming/ and http://wiki.python.org/moin/DatabaseInterfaces



SQLite3

  Whenever possible, I like to use SQLite3, http://en.wikipedia.org/wiki/SQLite, simply because there is a lot of iPhone and Android support if I want to make my App mobile.   The Python documentation for Sqlite3 can be found at http://docs.python.org/py3k/library/sqlite3.html



SQLite3

  Here, we create a sample.db, add text, like one, and interger, like 1, and read the rows:



SQLite3

  It did create a sample.db file that is the database:



SQLite3

  Just reading the rows:



MySQL

  We are going to install MySQL and the Python module for MySQL, http://www.codegood.com/archives/4 .

  A free Open Source database that can run as a service and is more relational for production use is the MySQL database. See http://en.wikipedia.org/wiki/MySQL .

  Install MySQL from http://www.mysql.com/downloads/mysql/   Here is an example to create the myTestAppDB:

  mysql> create database myTestAppDB character set utf8;   Here is an example to create user “myUser” password “XXXXXXXX”

•  mysql> CREATE USER ‘myUser’@’localhost’ IDENTIFIED BY ‘XXXXXXXX’;

•  mysql> GRANT ALL PRIVILEGES ON *.* TO ‘jirauser’@’localhost’ WITH GRANT OPTION;



MySQL

  To test to see if the MySQLdb module is installed, we will check the version of MySQL. An error will return if the library is not set up correctly.



MySQL

  Now we are going to create an animal tabl and 4 rows of animals, see http://www.kitebird.com/articles/pydbapi.html .



Network Programming



Network Programming

  A good Tutorial on Network programming in Python can be found http://heather.cs.ucdavis.edu/~matloff/Python/PyNet.pdf .   Python uses sockets, very similar in convention to Windows and Java socket programming.   Socket programming involves first creating a Socket server, that will have a base port number to listen on and accept incoming connections. This is a connection oriented socket, i.e. TCP.   A incoming connection is generated from the client that will create a socket, connect to the Server’s host port number and address, and send information to the server.   Sockets were originally derived from Berkeley sockets, http://en.wikipedia.org/wiki/Berkeley_sockets .



Network Programming

  Here’s a client communicating with a localhost port 5000 server. The client is in the IDLE Shell, and the Server is in NetBeans:



Network Programming

  Client code:



Network Programming

  Server code:



GUI Programming



Gui Toolkits

  There are many Gui toolkits for python, see http://wiki.python.org/moin/GuiProgramming



Gui Toolkits

  Looking at Tkinter “Python’s standard GUI Library, see http://wiki.python.org/moin/TkInter



Tkinter

  First, we need to import “from Tkinter import *”, to get an example button click:



Tkinter line

  We can create a line from end to end on a canvas:



Tkinter rectangle



Error Handling



Has my system been compromised?

  Logging and Error handling is one of the most important concept in Security.

  When an incident happens, the first questions are always “How did they get in?” and “What data was compromised?”.

  The least favorite answer is usually “No one knows.”   With efficient logging of authorization, access to secure

information, and any anomalous interaction with the system, a proper recovery of the system is usually insured.

  The logs should be store into a different system in case the Web system is ever compromised, one where the Web system sends them but never asks for them back.

  Logging is a fundamental API that comes with any language.



Has my system been compromised?

  Some references for Python Logging, http://docs.python.org/library/logging.html and http://docs.python.org/howto/logging-cookbook.html#logging-cookbook

  Logging is imported using the “import logging” construct.



Logging the Python way….



Exception Handling

  Exception handling has helped debugging immensely. It allows a programmer to code for anomalies and handle a bizarre behavior.

  There are 3 components of handling an exception, and they are the “try”, “catch” and “finally” blocks.

  The “try” block will throw an exception from normal code, the “catch” block will catch the exception and handle it, and the “finally” block will process the cleanup afterwards.

  The “catch” block can log the anomaly, stop the program, or process it in a hundred different ways.

  You can write your own custom exception classes to trace specific pieces of code.



Python Exception Handling code….



Built-in Exceptions



Assertions

  Assertions are another way to handle Exceptions.   Assertions will check to see if a statement is true, and if the

statement is false, then it will generate an “AssertionError”.   See

http://docs.python.org/reference/simple_stmts.html#the-assert-statement .

  The assert statement are used extensively in debugging to check if the statement is always true.

  What makes Assertions useful is that they can check for any statement and does not have to be Exception specific.



Assertions example



Android Python



Android Python

  Android is basically a Micro Edition of Linux, and because Linux normally runs Python, Android can add a package to run Python.   See http://hameedullah.com/develop-your-first-android-application-in-python.html



Set up the Android SDK

  First the Android environment, Software Development Kit (SDK), needs to be set up on the local Desktop, http://developer.android.com/sdk/installing.html :



Android Virtual Device (AVD)

 An AVD will have to be created for debugging and testing, this is your target device,



Android Python

  Create an AVD from your now installed Android SDK, by creating a new Virtual Device, http://developer.android.com/guide/developing/devices/index.html



Android Python

  The “New” Virtual Device for Android,



AVD Created

  The new AVD called “PythonDev”,



AVD Start

  Start “PythonDev”,



Use the Browser in the AVD

  Open the Browser in “PythonDev”, and go to http://code.google.com/p/android-scripting so install Android Scripting



Download SL4A

  After tapping on the QR code icon (center box) to download,



Install SL4A

  Double click the download to install,



SL4A Installed

  Installed,



Now Python has to be installed

  Go back in the browser and to the SL4A website, hit the download URL for Python for Android and you will get,



Install

  After downloading, you need to install Python,



Run a script

  Go to applications and select SL4A,



Run HelloWorld.py

  Select the Run icon on script,



Says “Hello, Android!”

  You get the “Hello, Android!”



See the scripts

  Android uses a bridge, called “adb” to talk from the desktop to the emulator. Under the android SDK/platform-tools, run “adb shell ls /sdcard/sl4a/scripts”,



Write a small script

  We are going to write a small script that prints the inputted name with a greeting, Notepad will work,



Set the scripts

  Run “adb push myGreet.py /sdcard/sl4a/scripts”, to copy this python file to the emulator,



Execute the script

  Execute the “myGreet.py” script,



Greeting

  After typing in your name, you get a greeting,



Django – Web Development



Web Frameworks

  A list of the most popular Web Frameworks for Python can be found at http://wiki.python.org/moin/WebFrameworks, Django is the most popular.   Django uses a Model-View-Controller framework for Python, http://stackoverflow.com/questions/2699988/best-environment-to-port-c-c-code-from-linux-to-windows   Django can be used on top of Apache using a mod_wsgi plugin. https://docs.djangoproject.com/en/dev/topics/install/?from=olddocs   Django can be downloaded from https://www.djangoproject.com/download/



Install Django

  Install Django from download site, https://www.djangoproject.com/download/ by downloading, untaring it, and running “python setup.py install” :



Check the Django install

  Check the Django install by checking the version :



Create mysite

  Setup a blank file structure, like mysite, from https://docs.djangoproject.com/en/dev/intro/tutorial01/   The django-admin.pyc must be in the path, and can be found in the Python site-packahes, in my example, it will be in C:\Python27\Lib\site-packages\django\bin.   Run the “django-admin.pyc startproject mysite” to create the blank site map:



Start the Server

  Start the manage script to start the server with “python manage.py runserver 8080”….



Did it Start?

  Go to the localhost website:



MVC

 The Model-View-Controller is the most common design pattern in Software Architecture. Here are the pieces:



MVC

  Django follows the Model-View-Controller design pattern but prefers to call it Model-Template-View (MTV). It prefers to use Templates instead of Views.



Generated files

  Django generated several files.   The manage.py starts the server.   The settings.py will define connection strings to databases, default language, userid and password, timsezones, apps, plugins, logging and more.   The urls.py will load the list of urls that the website will use.



Adding Views

  To show how the urls.py works, we are going to create a views.py that just sends a response that says “Hello World”.



Putting in the path…

  There is a lot of commented code in urls.py to show how it works.   We need to import the views.py and add the url pattern to be called.   The pattern to be called is used by regex conventions. Here we use ‘^$’ which is a root wildcard, meaning if no url, call the hello method.



Starting the server…

  We start the server, it is best to delete the previous urls.pyc so that the urls.py recompiles, here is a lot of commented code in urls.py to show how it works.   Here we show the console which shows the urls being called and the result in the browser:



Resetting to empty mysite…

  If we want to reset to the empty default “mysite” project, just delete the directory “mysite” and re-execute “django-admin.pyc startproject mysite”:



Creating an Django Application

  Creating an empty application for “books” can be done by executing “python manage.py startapp books”.   This will create a blank views.py, for views, models.py, for the models to call the database, and tests.py, to create the tests for the websites.



Django Models

  Django support Object to Relational Modeling (ORM), which means it handles the SQL calls, see https://docs.djangoproject.com/en/dev/topics/db/models/   First, setup the database, MySQL in this case.   Install MySQL from http://dev.mysql.com/downloads/   Install Python Setup tools for mysql-python library http://pypi.python.org/pypi/setuptools   Install mysql-python libraries, for Linux and others, http://sourceforge.net/projects/mysql-python/ , and there are Windows binaries http://www.codegood.com/archives/129   These python libraries are needed for Python integration with MySQL.



Connection String

  The settings.py has to be modified with the MySQL connection string, https://docs.djangoproject.com/en/dev/ref/settings , for example,



Default tables

  Now that there is a connection, the tables have not been created for Django. The same script will create the tables defined in the models.py.   Run “python manage.py syncdb”



Default tables



Admin site

  Once the default tables are set, the Admin site can be setup. The login for the Admin site is the administrator login set in the previous Database syncdb.   Uncomment the urls.py information for the admin:



Admin site

  Now we can go to http://127.0.0.1:8080/admin/ , then Login, and get the Admin configuration utility.



Creating Models

  Now that we have the database started, lets start the books models.py.



Update the settings

  Add the application to the settings.py:



Validate the settings

  We can validate the settings at any point by running “python manage.py validate, 0 errors found is a good thing:



The new tables

  We are going to rely on Django to manage the and create the tables, but we can run “python manage.py sqlall books” to see what Django wants to use to create the tables:



The new tables

  We are going to rely on Django to manage the and create the tables, but we can run “python manage.py sqlall books” to see what Django wants to use to create the tables:



Lets create them

  Running “python manage.py synchdb” will get the tables created :



Lets manage the tables

  Now let’s fill in some tables, see https://docs.djangoproject.com/en/dev/topics/db/queries/ and https://docs.djangoproject.com/en/dev/ref/models/instances/ .   We are going to run “python manage.py shell” to load the environment.



Lets manage the tables

  Now let’s add a Publisher record, we have to import the module, add to the Publisher fields, and save the object.   Django will do the SQL, we finish by printing the record.



Lookup

  After on of the values are entered in the table, let’s look it up by the state field.   This is not SQL, we are looking up by objects and their fields, updating, saving and deleting in like manner.   We will also delete the object.



Models on Admin site

  Now that we have the models defined and the admin site started, lets add the models to the admin site.   This is done by creating an admin.py that registers the models:




  Starting the server, we now can see the models and can populate and change them through the admin site under books:




  Adding a publisher:



The Forms (Views)

  A very helpful site regarding forms is http://www.djangobook.com/en/1.0/chapter07/.   We are going to create a simple search form.   This is MVC, what does this mean, the Controller gets called first, and we will define the controller in the /books/views.py, but first set the object to be called in the /urls.py:



Urls.py

  When “/search” is referenced as a URL, the “mysite/books/view.py” will call the “search” function or object.   The “search” code will be load a template with a query from HTTP Request:



Search.html

  Search.html will be the template that the Controller will load. The template needs to be stored in the template path since we didn’t set an absolute path.   I just put it in the Django Python Template path for now, but for production, we will set it to an Apache path:



Search.html

  Search.html will be the template that interacts with its controller.   It basically is passing in a query and getting the results from the search function in the views.py.



Search.html

  Let’s validate the program and start the server, and see what happens when search gets called:



Search.html

  Let’s call http://127.0.0.1:8080/search and do a search:



Search.html

  When we put in test, we can see that the “q=test”, the query was passed to the controller, but the results were empty.



Search.html

  Now we added a “test” book, notice the different return:



Python Secure FTP



Setting up

  The first thing to do is set up a Secure FTP Server, we will pull down a Java Server, runs anywhere, freeFTPd from http://www.freesshd.com/   Installing it, we add a “rhelton” user ahttp://www.freesshd.com/nd just turn on Secure FTP.



Setting up

  We test the server by ensuring a Java SFTP client works, I usually go to http://j-ftp.sourceforge.net/ and start J-FTP fro the Java Web Start, and Login to the local server:



Setting up

  After a successful connection, I know if my Python code doesn’t work, then it’s the code.   I will install the Pysftp, aPython Secure FTP program from http://code.google.com/p/pysftp/ and install:



The Python Client Program

  After installing Pysftp, I will import it, get a connection, list the directory and pull down a file (It works, the file was received locally):



Questions? Have a Good Day

Python Final

Documents

cyber security

cyber security

cyber security

cyber security

cyber security

cyber security

py sqlall

cyber security