Using Google App Engine Python

Akshay Mathur@akshaymathu

2

Ground Rules

• Disturb Everyone– Not by phone rings– Not by local talks– By more information

and questions

@akshaymathu

3

Let’s Know Each Other

• Do you code?• OS?• Language?• Used GAE?• Why are you attending?

@akshaymathu

@akshaymathu 4

Akshay Mathur

• Founding Team Member of– ShopSocially (Enabling “social” for retailers)– AirTight Neworks (Global leader of WIPS)

• 15+ years in IT industry– Currently Principal Architect at ShopSocially– Mostly worked with Startups• From Conceptualization to Stabilization• At different functions i.e. development, testing, release• With multiple technologies

@akshaymathu 5

What shall we learn

• Why GAE (Google App Engine)?• General advantages of – IaaS (Infrastructure as a Service)– PaaS (Platform as a Service)

• Using GAE SDK– Available Services from Google

• Filling the Gap– Including third party libraries– Creating MVC structure

• Available boilerplate code for quick start

@akshaymathu 6

Why GAE

• Application PaaS maintained by Google– Same platform that powers Google Applications

• Familiar technologies– Python, PHP, Java, Go

• Readymade additional services– Storage, Email, Chat, Task queue etc.

• No (or very low) initial cost– Free quotas

@akshaymathu 7

@akshaymathu 8

IaaS

• Move your local/data center servers to cloud– Don’t worry about hardware– Scale up/down as needed

• Simplified logistics for IT admins• Pay only for used infrastructure

• Custom hardware is not available

@akshaymathu 9

PaaS: Advantages

• Move your application environment on cloud– Focus only on your application– Don’t worry for upgrades– Easy to scale– Easy to administer

• Get all benefits of IaaS• Control in developers’ hand

10@akshaymathu

@akshaymathu 11

PaaS: Limitations

• No control over machines– No access to file system– No ssh

• No native software installation– Use only the provided software– Upgrade with the provider

• Limitation on usage of network stack– No sniffing of traffic– Limited use of ports

Getting Started

Hello world!

@akshaymathu 13

Why Python

• Because we love it • GAE says:

@akshaymathu 14

Getting Started

• Install Python • Download and install GAE SDK• Configure app.yaml• Write code• Test locally as GAE app• Deploy

@akshaymathu 15

Runtime configuration: app.yaml

@akshaymathu 16

Code: main.py

@akshaymathu 17

Deploy

18@akshaymathu

Gearing up for Bigger App

Creating the basic Structure

@akshaymathu 20

Included Libraries

Webapp2, Django, Ssl, PIL, Pycrypto, Setuptools, Webob, Yaml, MySQLdb

Endpoints: Libraries for building APIs in an App Engine application.

Jinja2: A modern and designer friendly templating language for Python.

Lxml: A Pythonic binding for the C libraries libxml2 and libxslt.

Markupsafe: A XML/HTML/XHTML markup safe string for Python.

Matplotlib: A 2D plotting library which produces publication-quality figures.

Numpy: A general-purpose library for array-processing.

Protorpc: A framework for implementing HTTP-based remote procedure call (RPC) services.

PyAMF: A library that provides (AMF) Action Message Format functionality.

@akshaymathu 21

Available Services

@akshaymathu 22

Data Storage• Datastore

– A schemaless object datastore providing robust, scalable storage for your web application, a rich data modeling API, and a SQL-like query language called GQL.

• Blobstore– Allows your application to serve large data objects, such as video or

image files, that are too large for storage in the Datastore service.• Memcache

– A distributed, in-memory data cache that can be used to greatly improve application performance.

• Logs– Provides programmatic access to application and request logs from

within your application.

@akshaymathu 23

Communications

• Channel– Creates a persistent connection between your application and Google

servers, so you can send messages to JavaScript clients in real time without "polling."

• Mail– Sends email messages on behalf of administrators and users with

Google Accounts, and receives mail at various addresses.• URL Fetch

– Uses Google's networking infrastructure to efficiently issue HTTP and HTTPS requests to URLs on the web.

• XMPP– Enables an application to send and receive chat messages to and from

any XMPP-compatible chat messaging service.

@akshaymathu 24

Process Management

• Task Queue– Allows applications to perform work outside of a

user request, and organize that work into small, discrete units, called "tasks," to be executed later.

• Scheduled Tasks– Allows applications to configure regularly

scheduled tasks that operate at defined times or regular intervals.

@akshaymathu 25

Computation

• Backends– Instances of your application that are exempt from

request deadlines and have access to more memory and CPU resources.

• Images– Manipulates, combines, and enhances images,

converts images between formats, and queries image metadata such as height and frequency of colors.

@akshaymathu 26

App configuration and management

• App Identity– Gives code access to the application identity; provides framework to assert this identity over OAuth.

• Capabilities– Provides detection of outages and scheduled maintenance for specific APIs and services, so that your

application may bypass them or inform your users.• SSL for Custom Domains

– Allows applications to be served via both HTTPS and HTTP via a custom domain instead of an appspot.com address.

• Remote– Lets external applications transparently access App Engine services. For example, you can use Remote API to

access a production datastore from an app running on your local machine.• Multitenancy

– Makes it easy to compartmentalize your data to serve many client organizations from a single instance of your application.

• Traffic Splitting– Allows you to roll out features for your app slowly over a period of time, and do A/B Testing. Traffic Splitting

works by splitting incoming requests to different versions of your app.• Users

– Allows applications to sign in users with Google Accounts or OpenID, and address these users with unique identifiers.

@akshaymathu 27

Third-party Services

• SendGrid (Email)– Use SendGrid's library to send emails from your

app and you can see statistics on opens, clicks, unsubscribes, spam reports and more.

• Twilio (SMS/Voice)– Enables your application to make and receive

phone calls, send and receive text messages, and make VoIP calls from any phone, tablet, or browser.

@akshaymathu 28

Preview Features• Google Cloud Endpoints

– Enables automatic generation of APIs, making it easier to create a web backend for web clients and mobile clients such as Android or Apple's iOS.

• Google Cloud SQL– A fully-managed web service that allows you to create, configure, and use

relational databases that live in Google's cloud.• Google Cloud Storage Client Library

– Lets your application read files from and write files to buckets in Google Cloud Storage, with with internal error handling and retry logic.

• Modules– Lets developers factor large applications into logical components that can share

stateful services and communicate in a secure fashion.• Sockets

– Enables support for outbound sockets using the language-specific, built-in libraries.

@akshaymathu 29

Experimental Features• MapReduce

– An optimized adaptation of the MapReduce computing model for efficient distributed computing over large data sets.

• OAuth– Using Google Accounts and the OAuth API, any App Engine application can be an OAuth

consumer.• OpenID

– An open technology used for authenticating users across various web services.• PageSpeed

– A family of tools that automatically optimizes the performance of your application.• Task Queue REST API

– Enables the use of an App Engine task queue over REST.• Task Queue Tagging

– Leases up to a specified number of tasks with the same tag from the queue for a specified period of time.

@akshaymathu 30

Search

• Search– Allows your application to perform Google-like

searches over structured data such as: plain text, HTML, atom, numbers, dates, and geographic locations.

• Prospective Search– A querying service that allows your application to

match search queries against real-time data streams.

@akshaymathu 31

Filling the Gap

• Including other libraries– Place them within your app– Only pure Python libs can be used

• Creating MVC structure– Create your own directory structure– Adjust path for supporting it

@akshaymathu 32

MVC Structure

@akshaymathu 33

New Code: main.py

@akshaymathu 34

Global Configuration: config.py

@akshaymathu 35

Route List

@akshaymathu 36

Controller

@akshaymathu 37

Base Controller

@akshaymathu 38

Third Party Libs

@akshaymathu 39

Static Files

40@akshaymathu

Dive Deeper

Get the app working

@akshaymathu 42

Scheduled Tasks: cron.yaml

@akshaymathu 43

Datastore

• NoSQL database is available in free quota– SQL database (Google Cloud SQL) is also available

@akshaymathu 44

Data Manipulation

• Functions– .get_by_id()– .all()

– .put()– db.delete()

• GQL– .gql()

• People.gql("where email_addr = :1 and passwd = :2", username, get_password_hash(passwd))

– gql_query()• gql_query(”select * from people where email_addr = :1 and passwd = :2", username, get_password_hash(passwd))

@akshaymathu 45

Task Queue

@akshaymathu 46

Sending Emails

• Gmail is available as mail service by default– Sendgrid can also be used

@akshaymathu 47

Using Others’ Web API

• Urlfetch can be used for accessing external web urls

@akshaymathu 48

Sockets

• Only for paid apps• Works exactly same as standard socket library• Inbound sockets are not allowed• Outbound sockets can be used with

restrictions

@akshaymathu 49

Google’s Login

from google.appengine.api import users

user = users.get_current_user()if user:   user.nickname() users.create_logout_url('/')))else: users.create_login_url('/'))

@akshaymathu 50

Working with Images

• Images can be stored in database in blob fields• Available transforms– Resize, Crop– Rotate, Flip horizontal /vertical– Enhance (Im feeling lucky)

• Formats Conversions– from: JPEG, PNG, WEBP, GIF, BMP, TIFF and ICO

– to: JPEG, WEBP and PNG

@akshaymathu 51

Image Manipulation

from google.appengine.api import images

img = images.Image(blob_key=blob_key)

img.resize(width=80, height=100)

img.im_feeling_lucky()

thumbnail = img.execute_transforms(

output_encoding=images.JPEG)

@akshaymathu 52

Caching

53@akshaymathu

@akshaymathu 54

Available Code on GitHub

• Implementation of login system– mathurakshay/gae-social-login• https://github.com/mathurakshay/gae-social-

• Just the MVC structure– droot/gae-boilerplate• https://github.com/droot/gae-boilerplate

@akshaymathu 55

Summary

• GAE is good readymade platform– For trying out things– For your side project

• Many services are readily available• Starts with no (or very low) upfront cost• Think thrice before you start big business on

GAE

56

Thanks

@akshaymathu@akshaymathu