Hacking the Python AST

@jargnar

github.com/jargnar

Hacking the Python AST

01010101010101010101010101...

Aprimeroncompilingcomputerlanguages

Front end Middle end Back endC

JAVAARMx86

IR IR

Intel’s developer manual

Aprimeroncompilingcomputerlanguages

Front end Middle end Back endC

JAVAARMx86

IR IR

@jargnar

github.com/jargnar

The Dragon Book

Grammars

S –> ABA –> aA | εB –> b | bB

SAB S–>ABaAB A–>aAaaAB A–>aAaaaAB A–>aAaaaεB A–>εaaab B–>b

A parse of aaabDoes aaab belong to this Grammar?

@jargnar

github.com/jargnar

LL(1) GrammarLeft-to-right scanLeftmost derivation

@jargnar

github.com/jargnar

Let’stakeaquicklookatPython’sGrammar(Grammar/Grammar)

DesignoftheCPython Compiler

Parser/pgen.c

Python/ast.c

Python/compile.c

Python/compile.c

Parse Tree

.pyfile

Byte code

AST CFG

@jargnar

github.com/jargnar

dis

DesignoftheCPython Compiler

Parser/pgen.c

Python/ast.c

Python/compile.c

Python/compile.c

Parse Tree

.pyfile

Byte code

AST CFG

Pythontox86andthenthe01010101s

Parser/pgen.c

Python/ast.c

Python/compile.c

Python/compile.c

Parse Tree

.pyfile

Byte code

AST CFG

Python/ceval.c

@jargnar

github.com/jargnar

AccessingtheAST

a = 26b = 0print(a + b)

example.pyIn [1]: import ast

In [2]: source = open('example.py').read()

In [3]: tree = ast.parse(source)

AccessingtheAST

a = 26b = 0print(a + b)

example.py

Module(body=[Expr(value=Str(s='Docstring')),Assign(

targets=[Name(id='a',ctx=Store())],

value=Num(n=26)),Assign(

targets=[Name(id='b',ctx=Store())],

value=Num(n=0)),Expr(value=Call(

func=Name(id='print',ctx=Load()),

args=[BinOp(left=Name(

id='a',ctx=Load()),

op=Add(),right=Name(

id='b',ctx=Load()))],

keywords=[]))])

Havingalittlefun,now

In [3]: tree = ast.parse('x = 5')

In [4]:print(astunparse.dump(tree))Module(body=[Assign(

targets=[Name(id='x',ctx=Store())],

value=Num(n=5))])

In [5]: tree.body[0].targets[0].idOut[5]: 'x'

In [6]: astunparse.unparse(tree)

x = 5

In [7]:tree.body[0].targets[0].id = 'y'

In [8]:print(astunparse.unparse(tree))

y = 5

@jargnar

github.com/jargnar

Let’sdoabsolutelysillythingswith

ast.NodeVisitorast.NodeTransformer

@jargnar

github.com/jargnar

Evenmoresillythingswith

Flake8

@jargnar

github.com/jargnar

Hacking is a mindset, not a skill.

suhas.org