Server-Side Web Programming: Python (Part 2) · Objectives • You will learn about: – Python WSGI programming – Web app frameworks in general (briefly) – The Bottle web app

Server-Side Web Programming: Python (Part 2)

Copyright © 2017 by Robert M. Dondero, Ph.D

Princeton University

1

Objectives

•  You will learn about: – Python WSGI programming – Web app frameworks in general (briefly) – The Bottle web app framework

•  The MVC architecture

2

Agenda

1. WSGI Programming 2. Web Application Frameworks 3.  The Bottle Web Application Framework

3

CGI Problem

•  Problem: CGI apps are slow – For each request, HTTP server forks/execs a

new process – OK for low-volume website – Maybe not OK for high-volume website

4

CGI Example

5

Browser Process

HTTP Server Process

CGI Example

6

Browser Process

HTTP Server Process

request for searchform page

Browser process asks HTTP server process for searchform page

CGI Example

7

Browser Process

HTTP Server Process

searchform Process

request for searchform page

HTTP server process forks child process, execs searchform program

CGI Example

8

Browser Process

HTTP Server Process

searchform page

Searchform process passes searchform page to HTTP server process Searchform process exits

searchform Process

CGI Example

9

Browser Process

HTTP Server Process

searchform page

HTTP server process passes searchform page to browser

CGI Example

10

Browser Process

HTTP Server Process

request for searchresults page

Browser process asks HTTP server process for searchresults page

CGI Example

11

Browser Process

HTTP Server Process

searchresultsProcess

request for searchresults page

HTTP server process forks child process, execs searchresults program

CGI Example

12

Browser Process

HTTP Server Process

searchresults page

Searchresults process passes searchresults page to HTTP server process Searchresults process exits

searchresults Process

CGI Example

13

Browser Process

HTTP Server Process

searchresults page

HTTP server process passes searchresults page to browser

Embedding

•  Solution 1: Embed apps in HTTP server process – HTTP server does not fork processes – HTTP server spawns threads in its process

•  (See Threads lecture later in course)

14

Embedding Example

15

Browser Process

HTTP Server Process

Browser process asks HTTP server process for searchform page

request for searchform page

Embedding Example

16

Browser Process

HTTP Server Process

HTTP server process spawns new thread (See upcoming Threads lecture) Thread generates searchform page

searchform Thread

Embedding Example

17

Browser Process

HTTP Server Process

Searchform thread exits HTTP server process sends searchform page to browser process

searchform Thread

searchform page

Embedding Example

18

Browser Process

HTTP Server Process

Browser process asks HTTP server process for searchresults page

request for searchresults page

Embedding Example

19

Browser Process

HTTP Server Process

HTTP server process spawns new thread Thread generates searchresults page

searchresultsThread

Embedding Example

20

Browser Process

HTTP Server Process

Searchresults thread exits HTTP server process sends searchresults page to browser process

searchresultsThread

searchresults page

Embedding Assessment •  Embedding assessment (simplified):

–  (pro) Faster than CGI •  Spawning a tread is faster than forking a process

–  (con) Inflexible •  Poor load balancing; better for HTTP server to

delegate work to pool of app servers, maybe running on distinct computers

–  (con) Insecure •  HTTP server typically runs as root •  Apps run with same permissions as HTTP server

21

Fast CGI

•  Solution 2: Fast CGI – HTTP server forks processes

•  As with CGI •  Typically one for each app

– Processes persist •  Unlike CGI

22

Fast CGI Example

23

Browser Process

HTTP Server Process

Fast CGI Example

24

Browser Process

HTTP Server Process

request for searchform page

Browser process asks HTTP server process for searchform page

Fast CGI Example

25

Browser Process

HTTP Server Process

Penny Process

request for searchform page

HTTP server process forks child process, execs Penny program

Fast CGI Example

26

Browser Process

HTTP Server Process

searchform page

Penny process passes searchform page to HTTP server process Penny process persists

Penny Process

Fast CGI Example

27

Browser Process

HTTP Server Process

searchform page

HTTP server process passes searchform page to browser

Penny Process

Fast CGI Example

28

Browser Process

HTTP Server Process

request for searchresults page

Browser process asks HTTP server process for searchresults page

Penny Process

Fast CGI Example

29

Browser Process

HTTP Server Process

Penny Process

request for searchresults page

HTTP server process sends request to existing Penny process

Fast CGI Example

30

Browser Process

HTTP Server Process

searchresults page

Penny process passes searchresults page to HTTP server process Penny process persists

Penny Process

Fast CGI Example

31

Browser Process

HTTP Server Process

searchresults page

HTTP server process passes searchresults page to browser

Penny Process

Fast CGI Assessment

•  Fast CGI assessment (simplified): –  (pro) Faster than CGI –  (pro) Flexible

•  Good load balancing; HTTP server can delegate work to pool of app servers, maybe running on different computers

•  Etc. –  (pro) Secure

•  Apps can run with more limited permissions than HTTP server has

32

Python Solution: WSGI

•  Python solution:

•  Web Server Gateway Interface (WSGI) – A specification of two interfaces…

33

Python Solution: WSGI

Browser HTTP Server

Your App

WSGI server-side Defines messages your app Can send to HTTP server

WSGI client-side Defines messages HTTP server can send to your app

Environment info

Callback function

Response via callback function

34

Python Solution: WSGI

Browser

HTTP Server

Your App

Optional middleware provides: App persistence (as with fast CGI) Load balancing among multiple app processes Maybe on multiple computers Authentication …

WSGI Middleware

WSGI Middleware

35

Recall Decorator pattern

Python Solution: WSGI

•  Web Server Gateway Interface (WSGI) – Allows architectural flexibility

•  Unknown to web app developer

– Facilitates separation of concerns: •  Programmers (and others) compose web apps •  System administrators deploy web apps

•  The way to structure Python web apps

36

PennyPythonWsgi App

•  See PennyPythonWsgi app – penny.sql, penny.sqlite, book.py, database.py – runserver – runserver.bat – common.py – penny.py

37

PennyPythonWsgi App

•  Observation: – Programmers don’t use WSGI directly –  Instead:

•  Programmers use web app frameworks •  Many Python web app frameworks use WSGI

•  Suggestion: – Don’t be concerned about WSGI details – …Unless you intend to compose a Python

web app framework!

38

Agenda

1. WSGI Programming 2. Web Application Frameworks 3.  The Bottle Web Application Framework

39

Motivation for Frameworks

•  Ancient approach to building a website: – Write ad hoc client-side code

•  In HTML, CSS, JavaScript, etc.

– Write ad hoc server-side code •  In Python, Java, PHP, etc. •  Write ad hoc code to access DB

40

Motivation for Frameworks

•  Problem: – Common patterns within and among web apps – Replicated code within and among web apps – Web app development involves lots of tedious/

replicated effort •  Solution:

– Web app frameworks •  Web app frameworks mechanize (parts of) the

development process

41

Popular Frameworks Stack Github Overflow Overall Score Score Score

ASP.NET (C#) 100 100 AngularJS (Javascript) 97 97 97 Ruby on Rails (Ruby) 93 98 95 ASP.NET MVC (C#) 94 94 React (JavaScript) 100 84 92 Django (Python) 90 93 91 Laravel (PHP) 92 87 89 Angular (JavaScript) 89 86 87 Spring (Java) 84 91 87 Express (JavaScript) 92 82 87 Meteor (JavaScript) 93 80 86 CodeIgniter (PHP) 84 86 85 Symfony (PHP) 84 85 84 Vue.js (JavaScript) 98 69 83 Flask (Python) 90 76 83

According  to  h,ps://ho1rameworks.com/  on  6/21/17  

42

Popular Frameworks Stack Github Overflow Overall Score Score Score

Ember.js (JavaScript) 86 78 82 JSF (Java) 82 82 Flex (ActionScript) 79 79 CakePHP (PHP) 76 80 78 Google Web Toolkit (Java) 78 78 Play (Scala) 79 75 77 Zend (PHP) 74 78 76 Sails.js (JavaScript) 85 67 76 Yii (PHP) 72 76 74 Tornado (Python) 83 62 72 Sinatra (Ruby) 79 66 72 Grails (Groovy) 64 80 72 Aurelia (JavaScript) 79 61 70 Phoenix (Elixir) 79 60 69 Phalcon (PHP) 78 58 68

According  to  h,ps://ho1rameworks.com/  on  6/21/17  

43

Popular Frameworks Stack Github Overflow Overall Score Score Score

Koa (JavaScript) 84 49 66 Dropwizard (Java) 74 56 65 Struts (Java) 64 64 Bottle (Python) 74 55 63 Wicket (Java) 63 63 Dojo (JavaScript) 56 71 63 Nancy (C#) 73 54 63 Vert.x (Java) 75 51 63 web.py (Python) 71 53 62 Elm (Elm) 70 54 62 Pyramid (Python) 65 59 62 Vapor (Swift) 80 40 60 beego (Go) 81 40 60 Gin (Go) 80 41 60 Kohana (PHP) 61 60 60 …

According  to  h,ps://ho1rameworks.com/  on  6/21/17  

44

Popular Python Frameworks

•  According to https://hotframeworks.com/ on 6/21/17…

•  Some popular Python frameworks: – Django, Flask*, Tornado, Bottle*, web.py,

Pyramid, web2py, CherryPy*, Grok, Zope

– * micro-framework

45

Popular Java Frameworks

•  According to https://hotframeworks.com/ on 6/21/17…

•  Some popular Java frameworks: – Spring MVC, JSF, Google Web Toolkit,

Dropwizard, Struts, Wicket, Vert.x, Vaadin

46

Popular PHP Frameworks

•  According to https://hotframeworks.com/ on 6/21/17…

•  Some popular PHP frameworks: – Laravel, CodeIgniter, Symfony, CakePHP,

Zend, Yii

47

Framework Assessment

•  Positives – Yield reliable code

•  Framework code has been thoroughly tested

– Yield consistent code •  Code has uniform structure across apps

– Make efficient use of programmer time •  Framework handles repetitive parts •  “DRY” (don’t repeat yourself) is encouraged

48

Framework Assessment •  Negatives

– Can yield systems that are: •  Large •  Slow

•  See Joel Spolsky’s Why I Hate Frameworks – http://discuss.joelonsoftware.com/default.asp?

joel.3.219431.12

49

Agenda

1. WSGI Programming 2. Web Application Frameworks 3.  The Bottle Web Application Framework

50

The Bottle Web App Framework

•  Who: Marcel Hellkamp •  Descrip: “Bottle is a fast,

simple and lightweight WSGI micro web-framework for Python. It is distributed as a single file module and has no dependencies other than the Python Standard Library.” -- Wikipedia

51

Why Bottle? •  Why study Bottle?

–  (Instead of some other Python framework) – Easy to learn

•  Simple (“micro-framework”) •  Good documentation and tutorial

– Reasonable to use for Assignments 3 & 4 •  Installed on CourseLab •  Easy to install on your personal computer •  Has test web server •  Allows flat directory structure; easy to submit

•  P.S. I have Penny in Django; ask if you want it

52

PennyBottle1Fund App

•  See PennyBottle1Fund App – penny.sql, penny.sqlite, book.py, database.py – runserver – runserver.bat – common.py – penny.py

•  Generalizing...

53

PennyBottle1Fund App

•  Bottle separates URL from file name –  (Unlike CGI) – Can use “pretty” URLs

•  Bottle separates URL from function name – Can map multiple URLs to same function – Can change function name without changing

URL, or vice versa

54

PennyBottle1Fund App

•  Bottle provides utility functions – Functions to fetch “name=value” pairs – Functions to fetch/create cookies – Function to implement HTTP redirect

55

Aside: Python Decorators

56

def sqr(i): return i * i def main(): print sqr(5) if __name__ == '__main__': main()

Wanted: sqr() prints “sqr was called” each time it is called

Aside: Python Decorators

57

def sqr(i): print 'sqr was called' return i * i def main(): print sqr(5) if __name__ == '__main__': main()

OK, but… Doesn’t generalize Requires edit of def of sqr()

Aside: Python Decorators

58

def printNameDecorator(f): def fwrapper(i): print f.__name__, 'was called' return f(i) return fwrapper def sqr(i): return i * i sqr = printNameDecorator(sqr) def main(): print sqr(5) if __name__ == '__main__': main()

Prints: sqr was called 25

One approach:

Aside: Python Decorators

59

def printNameDecorator(f): def fwrapper(i): print f.__name__, 'was called' return f(i) return fwrapper @printNameDecorator def sqr(i): return i * i def main(): print sqr(5) if __name__ == '__main__': main()

Prints: sqr was called 25

Using a decorator

PennyBottle1Fund App

60

Bottle uses Python decorators

... @route('/searchform') def searchForm(): ... ...

... def searchForm(): ... searchForm = default_app().route( \ path='/searchform', callback=searchForm) ...

Using a decorator

Same code without using a decorator

Toward PennyBottle2Templates

•  Problem: – Code contains many assignment statements

to compose HTML code – Bulky; awkward; error prone

•  Solution: – Templates

61

PennyBottle2Templates App •  See PennyBottle2Templates App

–  runserver, runserver.bat, penny.sql, penny.sqlite, book.py, database.py

– header.tpl –  footer.tpl –  index.tpl – searchform.tpl – searchresults.tpl – penny.py

•  Generalizing...

62

PennyBottle2Templates App

•  Template (informally) – HTML doc with placeholders – Each placeholder is identified by a key

63

Hello <strong>{{username}}</strong> and welcome

hello.tpl

PennyBottle2Templates App

•  To instantiate a template: – template('somefile.tpl', dict)

•  dict contains key/value pairs •  For each placeholder identified by key in somefile.tpl, replaces the placeholder with the value associated with key in dict

•  Returns the resulting str

64

PennyBottle2Templates App

•  To instantiate a template: – Example:

65

dict = {'username': 'Bob'} s = template('hello.tpl', dict)

s = 'Hello <strong>Bob</strong> and welcome'

Hello <strong>{{username}}</strong> and welcome hello.tpl

equivalent to

PennyBottle2Templates App

•  Template can contain: –  Inline expressions

•  …{{expr}}… •  expr usually is a key in the provided dict

•  expr can be any expression that evaluates to a str or has a str representation

66

... {{book.getAuthor()}} ...

... {{prevAuthor}} ...

PennyBottle2Templates App – Embedded pseudo-Python code

•  % line of Python code •  Indentation is ignored •  Blocks that are normally indented must be closed

with end keyword

67

% for book in books: {{book.getAuthor()}}, {{book.getTitle()}}, ${{book.getPrice()}}<br> % end

PennyBottle3Templates App –  Includes of other templates

•  % include('other.tpl')

68

... % include('header.tpl') ...

Toward PennyBottle3Templates

•  Problem??? – Computation of morning/afternoon is in

penny.py; better to move it to header.tpl? – Computation of current time is in penny.py;

better to move it to footer.tpl? •  Solution???

– Move more Python code to templates…

69

PennyBottle3Templates App

•  See PennyBottle3Templates App –  runserver, runserver.bat, penny.sql,

penny.sqlite, book.py, database.py, common.py, index.tpl, searchform.tpl, searchresults.tpl

– header.tpl –  footer.tpl – penny.py

•  Generalizing...

70

MVC Architecture

•  Bottle encourages... •  The Model-View-Controller (MVC)

architecture – Model: The system’s data access code – View: The system’s data presentation code – Controller: (vague) The system’s business

code; the logic that connects the model with its view(s)

71

MVC Architecture

•  In a Bottle app – Views: template files

•  In Penny apps: *.tpl files

– Controller: “main” Python file •  In Penny apps: penny.py

– Model: DB-related Python files •  In Penny apps: database.py, book.py •  Many “non-micro” frameworks facilitate model dev

72

MVC Assessment

•  Positives – Allows separation of concerns:

•  Model: DB administrator (SQL, normalization, ...) •  Views: Web designer (HTML, CSS, ...) •  Controller: Programmer (Python, Java, …)

73

MVC Assessment

•  Positives (cont.) – Facilitates parallel development

•  Can develop model, views, controller concurrently

– Facilitates maintenance •  Framework provides loose coupling •  (To some extent) can change model, views,

controller independently

74

MVC Suggestions

•  Suggestions: – Use MVC!!! – But keep views simple

•  Avoid embedded code in views/templates when you reasonably can

•  Is PennyBottle3Templates app better or worse than PennyBottle2Templates app?

75

Aside: ORMs

•  Problem: Impedence mismatch – Relational DB data model: tables, rows,

columns – OO data model: objects, object composition,

classes, class inheritance – Very different!

76

Aside: ORMs

•  Solution 1: Cursor – Relational DB driver fetches (sub)table from

DB, presents it to OO code as a cursor – OO code uses cursor to traverse table

77

Aside: ORMs •  Solution 2: Object-relational mappers

(ORMs) – Provided stand-alone (see Wikipedia) – Provided by many full-featured frameworks – Not provided by micro-frameworks (e.g., Bottle) – Through config files, map relational DB tables to

classes/objects – OO code fetches/stores data by sending

messages to objects, not by executing SQL statements

– Facilitates model development

78

Aside: Django and COS 333 •  Django

– Use for COS 333 assignments? –  (pro) The most popular Python framework –  (pro) Has ORM –  (con) More complex/powerful than necessary –  (con) Requires non-flat directory structure –  (con) ORM requires control of DB

•  Django’s ORM requires control fields in tables •  Could not use Django’s ORM to access Penny DB •  Limitation beyond COS 333???

79

More Bottle!

•  There is much more to Bottle •  Bottle user’s guide:

– http://bottlepy.org/docs/dev/ •  Bottle tutorial:

– http://bottlepy.org/docs/dev/tutorial.html

80

Summary

•  We have covered: – Python WSGI programming – Web app frameworks in general (briefly) – The Bottle web app framework

•  The MVC architecture

81