-
Towards Automatic Resource Bound Analysis for OCaml
Jan Hoffmann Shu-Chun WengYale University
{jan.hoffmann, shu-chun.weng}@yale.edu
AbstractUnderstanding the resource usage of programs is crucial
for devel-oping software that is safe, secure, and efficient.
Consequently, thereis ongoing interest in the development of
techniques that providesoftware developers with support for
inferring resource bounds atcompile time. This article introduces a
new resource analysis systemfor OCaml programs. The system
automatically derives worst-caseresource bounds for higher-order,
polymorphic programs with side-effects and user-defined inductive
types. The technique is parametricin the resource and can derive
bounds for time, memory, and energyusage. The derived bounds are
multivariate resource polynomialsthat are functions of different
size parameters that depend on thestandard OCaml types. Bound
inference is fully automatic and re-duced to a standard linear
optimization problem that is passed to anoff-the-shelf LP solver.
Technically, the analysis system is based ona novel multivariate
automatic amortized resource analysis (AARA).It builds on existing
work on linear AARA for higher-order pro-grams with user-defined
inductive types and on multivariate AARAfor first-order programs
with built-in lists and binary trees. For thefirst time, it is
possible to automatically derive polynomial boundsfor higher-order
functions and polynomial bounds that depend onuser-defined
inductive types. Moreover, the analysis handles sideeffects and
even outperforms the linear bound inference of previoussystems. At
the same time, it preserves the expressivity and effi-ciency of
existing AARA techniques. The practicality of the analysissystem is
demonstrated with an implementation and the integrationwith Inria’s
OCaml compiler. In a case study, the system infersbounds on the
number of queries that are sent by OCaml programsto DynamoDB, a
commercial NoSQL cloud database service.
1. IntroductionThe quality of software crucially depends on the
amount of resources—such as time, memory, and energy—that are
required for its exe-cution. Statically understanding and
controlling the resource usageof software continues to be a
pressing issue in software develop-ment. Performance bugs are very
common and among the bugsthat are most difficult to detect [40, 46]
and large software systemsare plagued by performance problems.
Moreover, many securityvulnerabilities exploit the space and time
usage of software [21, 42].
Developers would greatly profit from high-level
resource-usageinformation in the specifications of software
libraries and otherinterfaces, and from automatic warnings about
potentially high-resource usage during code review. Such
information is particularlyrelevant in contexts of mobile
applications and cloud services, whereresources are limited or
resource usage is a major cost factor.
Recent years have seen fast progress in developing frameworksfor
statically reasoning about the resource usage of programs.
Manyadvanced techniques for imperative integers programs apply
abstractinterpretation to generate numerical invariants. The
obtained size-change information forms the basis for the
computation of actualbounds on loop iterations and recursion
depths; using counter
instrumentation [27], ranking functions [2, 6, 15, 48],
recurrencerelations [3, 4], and abstract interpretation itself [18,
54]. Automaticresource analysis techniques for functional programs
are based onsized types [50], recurrence relations [23],
term-rewriting [10], andamortized resource analysis [32, 34, 41,
47].
Despite major steps forward, there are still many obstaclesto
overcome to make resource analysis technologies availableto
developers. On the one hand, typed functional programs
areparticularly well-suited for automatic resource-bound analysis
sincethe use of pattern matching and recursion often results in a
relativelyregular code structure. Moreover, types provide detailed
informationabout the shape of data structures. On the other hand,
existingautomatic techniques for higher-order programs can only
inferlinear bounds [41, 50]. Furthermore, techniques that can
derivepolynomial bounds are limited to bounds that depend on
predefinedlists and binary trees [29, 32] or integers [15, 48].
Finally, resourceanalyses for functional programs have been
implemented for customlanguages that are not supported by mature
tools for compilationand development [32, 34, 41, 47, 50].
The goal of a long term research effort is to overcome
theseobstacles by developing Resource Aware ML (RAML), a
resource-aware version of the functional programming language
OCaml.RAML is based on an automatic amortized resource
analysis(AARA) that derives multivariate polynomials that are
functionsof the sizes of the inputs. In this paper, we report on
three maincontributions that are part of this effort.
1. We present the first implementation of an AARA that is
inte-grated with an industrial-strength compiler.
2. We develop the first automatic resource analysis system
thatinfers multivariate polynomial bounds that depend on
sizeparameters of complex user-defined data structures.
3. We present the first AARA that infers polynomial bounds
forhigher-order functions.
The techniques we develop are not tied to a particular resource
butare parametric in the resource of interest. RAML infers tight
boundsfor many complex example programs such as sorting algorithms
withcomplex comparison functions, Dijkstra’s single-source
shortest-path algorithm, and the most common higher-order functions
suchas (sequences) of nested maps, and folds. The technique is
naturallycompositional, tracks size changes of data across function
bound-aries, and can deal with amortization effects that arise, for
instance,from the use of a functional queue. Local inference rules
generatelinear constraints and reduce bound inference to
off-the-shelf LPsolving, despite deriving polynomial bounds.
To ensure compatibility with OCaml’s syntax, we reuse the
parserand type inference engine from Inria’s OCaml compiler. We
extracta type-annotated syntax tree to perform (resource
preserving) codetransformations and the actual resource-bound
analysis. To preciselymodel the evaluation of OCaml, we introduce a
novel operationalsemantics that makes the efficient handling of
function closures in
Draft 1 2015/7/27
-
Inria’s compiler explicit. The semantics is complemented by a
newtype system that refines function types.
To express a wide range of bounds, we introduce a novel classof
multivariate resource polynomials that map data of a given typeto a
non-negative number. These novel multivariate resource poly-nomials
are a substantial generalization of the resource polynomialsthat
have been previously defined for lists and binary trees [32].
Todeal with realistic OCaml code, we develop a novel
multivariateAARA that handles higher-order functions. To this end,
we drawinspirations from multivariate AARA for first-order programs
[32]and linear AARA for higher-order programs [41]. However, our
newsolution is more than the combination of existing techniques.
Forinstance, we infer linear bounds for the curried append function
forlists, which has not been possible previously [41].
We performed experiments on more then 3000 lines of OCamlcode.
While it is still not straightforward to automatically
analyzecomplete existing applications, it is easy to develop and
analyzereal OCaml applications if we keep the current capabilities
of thesystem in mind. In Section 8, we present a case study in
which weautomatically bound the number of queries that an OCaml
programissues to Amazon’s DynamoDB NoSQL cloud database
service.Such bounds are interesting since Amazon charges
DynamoDBusers based on the number of queries made to a
database.
2. OverviewBefore we describe the technical development, we give
a shortoverview of the challenges and achievements of our
work.Currying and Function Closures. Currying and function
closurespose a challenge to automatic resource analysis systems
that hasnot been addressed in the past. To see why, assume that we
wantto design a type system to verify resource usage. Now consider
forexample the curried append function which has the type append :α
list Ñ α list Ñ α list in OCaml. At first glance, we might saythat
the time complexity of append is Opnq if n is the length of
thefirst argument. But a closer inspection of the definition of
appendreveals that this is a gross simplification. In fact, the
complexity ofthe partial function call app par = append ` is
constant. Moreover,the complexity of the function app par is
linear—not in the lengthof the argument but in the length of the
list ` that is captured in thefunction closure. We are not aware of
any existing approach that canautomatically derive a worst-case
time bound for the curried appendfunction. For example, previous
AARA systems would fail withoutderiving a bound [32, 41].
In Inria’s OCaml implementation, the situation is even
morecomplex since the resource usage (time and space) depends on
howa function is used at its call sites. If append is partially
applied to oneargument then a function closure is created as
expected. However—and this is one of the reasons of OCaml’s great
performance—ifappend is applied to both of its arguments at the
same time thenthe intermediate closure is not created and the
performance of thefunction is even better than that of the curried
version since we donot have to create a pair before the
application.
To model the resource usage of curried functions accurately
werefine function types to capture how functions are used at their
callsites. For example, append can have both of the following
types
α list Ñ α list Ñ α list and rα list, α lists Ñ α list .The
first type implies that the function is partially applied and
thesecond type implies that the function is applied to both
argumentsat the same time. Of course, it is possible that the
function has bothtypes (technically we achieve this using let
polymorphism). For thesecond type, our system automatically derives
tight time and spacebounds that are linear in the first argument.
However, our systemfails to derive a bound for the first type. The
reason is that we madethe design decision to not derive bounds that
asymptotically depend
on data captured in function closures to keep the complexity of
thesystem at a manageable level.
Fortunately, append belongs to a large set of OCaml functions
inthe standard library that is defined in the from let rec f x y z
= e. Ifsuch a function is partially applied, the only computation
that hap-pens is the creation of a closure. As a result, eta
expansion does notchange the resource behavior of programs. This
means for examplethat we can safely replace the expression let app
par = append ` in ewith the expression let app par x = append ` x
in e prior the analy-sis. Consequently, we can always use the type
rα list, α lists Ñα list of append that we can successfully
analyze.
The conditions under which functions can be analyzed mightlook
complex at first but they can be boiled down to simple
principle:
The worst-case resource usage of a function must be express-ible
as a function of the sizes of its arguments.
Higher-Order Arguments. The other main challenge with
higher-order resource analysis is functions with higher-order
arguments. Toa large extend, this problem has been successfully
solved for linearresource bounds in previous work [41]. Basically,
the higher-ordercase is reduced to the first-order case if the
higher-order argumentsare available. It is not necessary to
reanalyze such higher-orderfunctions for every call site since we
can abstract the resourceusage with a constraint system that has
holes for the constraintsof the function arguments. However, a
presentation of the systemin such a way mixes type checking with
the constraint-based typeinference. Therefore, we chose to present
the analysis system in amore declarative way in which the bound of
a function with higher-order arguments is derived with respect to a
given set of resourcebehaviors of the argument functions.
A concrete advantage of our declarative view is that we
canderive a meaningful type for a function like map for lists even
whenthe higher-order argument is not available. The function map
canhave the following types.
pαÑ βq Ñ α list Ñ β list rαÑ β, α lists Ñ β listUnlike append,
the resource usage of map does not depend on thesize of the first
argument. So both types are equivalent in our systemexcept for the
cost of creating an intermediate closure. If the higher-order
argument is not available then previous systems [41] producea
constraint system that is not meaningful to a user. An innovationin
this work is that we are also able to report a meaningful
resourcebound for map if the arguments are not available. To this
end, weassume that the argument function does not consume
resources. Forexample, we report in the case of map that the number
of evaluationsteps needed is 11n ` 3 and the number of heap cells
needed is4n`2 where n is the length of the input list. Such bounds
are usefulfor two purposes. First, a developer can see the cost
that map itselfcontributes to the total cost of a program. Second,
the time bound formap proves that map is guaranteed to terminate if
the higher-orderargument terminates for every input.
In contrast, consider the function rec scheme : pα list Ñα listq
Ñ α list Ñ β list that is defined as follows.
let rec rec_scheme f l =match l with | [] Ñ []
| x::xs Ñ rec_scheme f (f l);;let g = rec_scheme tail;;
Here, RAML is not able to derive an evaluation-step bound forrec
scheme since the number of evaluation steps (and even termina-tion)
depends on the argument f . However, RAML derives the
tightevaluation-step bound 12n` 7 for the function g.Polynomial
Bounds and Inductive Types. Existing AARA sys-tems are either
limited to linear bounds [34, 41] or to polynomialbounds that are
functions of the sizes of simple predefined list and
Draft 2 2015/7/27
-
let comp f x g = fun z Ñ f x (g z)
let rec walk f xs =match xs with | [] Ñ (fun z Ñ z)
| x::ys Ñ match x with | Left _ Ñfun y Ñ comp (walk f) ys (fun z
Ñ x::z) y
| Right l Ñlet x’ = Right (quicksort f l) infun y Ñ comp (walk
f) ys (fun z Ñ x’::z) y
let rev_sort f l = walk f l []
RAML output for rev sort (after 0.68s run time):
10 + 23*K*M + 32*L*M + 20*L*M*Y + 13*L*M*Y^2where
M is the num. of ::-nodes of the 2nd comp. of the arg.L is the
fraction of Right-nodes in the ::-nodes of
the 2nd component of the argumentY is the maximal number of
::-nodes in the Right-nodes
in the ::-nodes of the 2nd component of the arg.K is the
fraction of Left-nodes in the ::-nodes of the
2nd component of the argument
Figure 1. Modified challenge example from Avanzini et al.
[10]and shortened output of the automatic bound analysis performed
byRAML for the function rev sort. The derived bound is a tight
boundon the number of evaluation steps in the big-step semantics if
we donot take into account the cost of the higher-order argument
f.
binary-tree data structures [32]. In contrast, this work
presents thefirst analysis that can derive polynomial bounds that
depend on sizeparameters of complex user-defined data
structures.
The bounds we derive are multivariate resource polynomials
thatcan take into account individual sizes of inner data
structures. Whileit is possible to simplify the resource
polynomials in the user output,it is essential to have this more
precise information for intermediateresults to derive tight
whole-program bounds.
In general, the resource bounds are built of functions that
countthe number of specific tuples that one can form from the
nodesin a tree-like data structure. In their simplest form (i.e.,
withoutconsidering the data stored inside the nodes), they have the
form
λa.|t~a | ai is an Aki -node in a and if i ă j then ai ăapre
aju|
where a is an inductive data structure with constructorsA1, . .
. , Am,~a “ pa1, . . . , anq, and ăapre denotes the pre-order on
the tree a.We are able to keep track of changes of these quantities
in patternmatches and data construction fully automatically by
generatinglinear constraints. At the same time, they allow us to
accuratelydescribe the resource usage of many common functions in
thesame way it has been done previously for simple types [28]. Asan
interesting special case, we can also derive conditional boundsthat
describe the resource usage as a conditional statement.
Forinstance, for an expression such as
match x with | True Ñ quicksort y | False Ñ y
we derive a bound that is quadratic in the length of y if and
only if xis True.
Effects. Our analysis handles references and arrays by
ensuringthat resource cost does not asymptotically depend on values
thathave been stored in mutable cells. While it has been shown
thatit is possible to extend AARA to handle mutable state [17],
wedecided not to add the feature in the current system to focus
onthe presentation of the main contributions. There are still a lot
ofpossible interactions with mutable state, such as storing
functions inreferences.
Example Bound Analysis. To demonstrate some of the capabili-ties
of the new analysis system, Figure 1 shows the output of RAMLfor a
concrete example. The code is an adoption of a challengingexample
that has been recently presented by Avanzini et al. [10] asa
function that can not be handled by existing tools. To illustrate
thechallenges of resource analysis for higher-order programs,
Avanziniet al. implemented a (somewhat contrived) reverse function
rev forlists using higher-order functions. RAML can automatically
derivea tight linear bound on the number of evaluation steps used
by rev.
To show more features of our analysis, we modified Avanzini
etal.’s rev in Figure 1 by adding an additional argument f and a
patternmatch to the definition of the function walk. The resulting
type ofwalk ispαÑ αÑ boolq Ñ rpβ ˚ α listq either list; pβ ˚ α
listq either lists
Ñ pβ ˚ α listq either listLike before the modification, walk is
essentially the append reversefunction for lists. However, we
assume that the input lists containnodes of the form Left a or
Right b so that b is a list. During thereverse process of the first
list in the argument, we sort each list thatis contained in a
Right-node using the standard implementation ofquick sort (not
given here). RAML derives the tight evaluation-stepbound that is
shown in Figure 1. Since the comparison function forquicksort
(argument f) is not available, RAML assumes that it doesnot consume
any resources during the analysis. If rev sort is appliedto a
concrete argument f then the analysis is repeated to derive abound
for this instance.
3. Setting the StageWe describe and formalize the new resource
analysis using CoreRAML, a subset of the intermediate language that
we use to performthe analysis. Expressions in Core RAML are in
share-let-normalform, which means that syntactic forms allow only
variables insteadof arbitrary terms whenever possible without
restricting expressivity.We automatically transform user-level
OCaml programs to CoreRAML without changing their resource behavior
before the analysis.Syntax. For the purpose of this article, the
syntax of Core RAMLexpressions is defined by the following grammar.
The actual coreexpressions also contain constants and operators for
primitivedata types such as integer, float, and boolean; arrays and
built-inoperations for arrays; conditionals; and free versions of
syntacticforms. These free versions are semantically identical to
the standardversions but do not contribute to the resource cost.
This is neededfor the resource preserving translation of user-level
code to share-let-normal form.
e ::“ x | x x1 ¨ ¨ ¨xn | C x | λx.e | ref x | !x | x1 :“ x2|
matchxwithC y Ñ e1 | e2| px1, . . . , xnq | matchxwith px1, . . . ,
xnq Ñ e| sharex as px1, x2q in e | letx “ e1 in e2 | let rec F in
e
F ::“ f “ λx.e | F1 andF2The syntax contains forms for
variables, function application, dataconstructors, lambda
abstraction, references, tuples, pattern match-ing, and (recursive)
binding. For simplicity, we only allow recursivedefinitions of
functions. In the function application we allow theapplication of
several arguments at once. This is useful to staticallydetermine
the cost of closure creation but also introduces ambigu-ity. The
type system will determine if an expression like f x1 x2is parsed
as pf x1 x2q or pf x1q x2. The sharing expressionssharex as px1,
x2q in e is not standard and used to explicitly intro-duce multiple
occurrences of a variable. It binds the free variablesx1 and x2 in
e.
We focus on this set of language features since it is sufficient
topresent the main contributions of our work. We sometimes take
the
Draft 3 2015/7/27
-
V pxq “ `¨, V,H M$x ó p`,Hq |M var
(E:VAR)S ‰ ¨ HpV pxqq “ pλx.e, V 1q S, V 1, H M$λx.e ó w | pq,
q1q
S, V,H M$x ó w |M var¨pq, q1q(E:VARAPP)
S, V,H M$ e ó ˝ | 0(E:ABORT)
V px1q:: ¨ ¨ ¨ ::V pxnq, V,H M$x ó w | pq, q1q S “ ¨ _ w P tK,
˝uS, V,H M$x x1 ¨ ¨ ¨xn ó w |M appn ¨pq, q1q
(E:APP)
S ‰ ¨ V px1q:: ¨ ¨ ¨ ::V pxnq, V,H M$x ó pλx.e, V 1q | pq, q1q
S, V 1, H M$λx.e ó w | pp, p1qS, V,H M$x x1 ¨ ¨ ¨xn ó w |M appn
¨pq, q1q¨pp, p1q
(E:APPAPP)
S, V rx ÞÑ `s, H M$ e ó w | pq, q1q`::S, V,H M$λx.e ó w
|Mbind¨pq, q1q
(E:ABSBIND)¨, V,H M$ e1 ó w | pq, q1q w P tK, ˝uS, V,H M$ letx “
e1 in e2 ó w |M let1 ¨pq, q1q
(E:LET1)
H 1 “ H, ` ÞÑ pλx.e, V q¨, V,H M$λx.e ó p`,H 1q |M abs
(E:ABSCLOS)¨, V,H M$ e1 ó p`,H 1q | pq, q1q S, V rx ÞÑ `s, H 1M$
e2 ó w | pp, p1q
S, V,H M$ letx “ e1 in e2 ó w |M let1 ¨pq, q1q¨M let2 ¨pp,
p1q¨w(E:LET2)
F fi f1 “ λx1.e1 and ¨ ¨ ¨ and fn “ λxn.en V 1 “ V rf1 ÞÑ `1, .
. . , fn ÞÑ `nsH 1 “ H, `1 ÞÑ pλx1.e1, V 1q, . . . , `n ÞÑ pλxn.en,
V 1q S, V 1, H 1M$ e0 ó w | pq, q1q
S, V,H M$ let recF in e0 ó w |M rec¨pq, q1q¨w(E:LETREC)
Figure 2. Selected rules of the operational big-step
semantics.
liberty to describe examples in user level syntax and to use
featuressuch as built-in data types that are not described in this
article.
Big-Step Operational Cost Semantics. The resource usage ofRAML
programs is defined by a big-step operational cost semantics.The
semantics has three interesting non-standard features. First,
itmeasures (or defines) the resource consumption of the
evaluationof an RAML expression by using a resource metric that
defines aconstant cost for each evaluation step. If this cost is
negative thenresources are returned. Second, it models terminating
and divergingexecutions by inductively describing finite subtrees
of infiniteexecution trees. Third, it models OCaml’s stack-based
mechanismfor function application, which avoids creation of
intermediatefunction closures.
The semantics of Core RAML is formulated with respect to astack
(to store arguments for function application), an environment,and a
heap. Let Loc be an infinite set of locations modeling
memoryaddresses. A heap is a finite partial mapping H : Loc á Val
thatmaps locations to values. An environment is a finite partial
mappingV : Var á Loc from variable identifiers to locations. An
argumentstack S ::“ ¨ | `::S is a finite list of locations. The set
of RAMLvalues Val is given by
v ::“ ` | p`1, . . . , `kq | pλx.e, V q | pC, `q
A value v P Val is either a location ` P Loc, a tuple of
locationsp`1, . . . , `kq, a function closure pλx.e, V q, or a node
of a datastructure pC, `q where C is a constructor and ` is a
location. In afunction closure pλx.e, V q, V is an environment, e
is an expression,and x is a variable.
Since we also consider resources like memory that can
becomeavailable during an evaluation, we have to track the
watermarkof the resource usage, that is, the maximal number of
resourceunits that are simultaneously used during an evaluation. To
derivea watermark of a sequence of evaluations from the watermarks
ofthe sub evaluations one has to also take into account the number
ofresource units that are available after each sub evaluation.
The big-step operational evaluation rules in Figure 2 are
formu-lated with respect to a resource metric M . They define an
evaluation
judgment of the form
S, V,H M$ e ó p`,H 1q | pq, q1q .It expresses the following. If
the argument stack S, the environmentV , and the initial heap H are
given then the expression e evaluatesto the location ` and the new
heap H 1. The evaluation of e needsq P Q`0 resource units
(watermark) and after the evaluation there areq1 P Q`0 resource
units available. The actual resource consumptionis then δ “ q ´ q1.
The quantity δ is negative if resources becomeavailable during the
execution of e.
There are two other behaviors that we have to express in
thesemantics: failure (i.e., array access outside array bounds)
anddivergence. To this end, our semantic judgement not only
evaluatesexpressions to values but also to an error K and to
incompletecomputations expressed by ˝. The judgement has the
general formS, V,H M$ e ó w | pq, q1q where w ::“ p`,Hq | K | ˝
.
Intuitively, this evaluation statement expresses that the
watermark ofthe resource consumption after some number of
evaluation steps is qand there are currently q1 resource units
left. A resource metricM : K ˆ N Ñ Q defines the resource
consumption in eachevaluation step of the big-step semantics where
K is a set ofconstants. We write Mkn for Mpk, nq and Mk for Mpk,
0q.
It is handy to view the pairs pq, q1q in the evaluation
judgmentsas elements of a monoid Q “ pQ`0 ˆQ`0 , ¨q. The neutral
elementis p0, 0q, which means that resources are neither needed
beforethe evaluation nor returned after the evaluation. The
operationpq, q1q ¨ pp, p1q defines how to account for an evaluation
consistingof evaluations whose resource consumptions are defined by
pq, q1qand pp, p1q, respectively. We define
pq, q1q ¨ pp, p1q “"
pq ` p´ q1, p1q if q1 ď ppq, p1 ` q1 ´ pq if q1 ą p
If resources are never returned (as with time) then we only
haveelements of the form pq, 0q and pq, 0q ¨ pp, 0q is just pq ` p,
0q.We identify a rational number q with an element of Q as
follows:q ě 0 denotes pq, 0q and q ă 0 denotes p0,´qq. This
notationavoids case distinctions in the evaluation rules since the
constantsK that appear in the rules can be negative. In the
semantic rules
Draft 4 2015/7/27
-
we use the notation H 1 “ H, ` ÞÑ v to indicate that ` R
dompHq,dompH 1q “ dompHq Y t`u, H 1p`q “ v, and H 1pxq “ Hpxq
forall x ‰ `.
To model the treatment of function application in Inria’s
OCamlcompiler, we use a stack S on which we store the locations
offunction arguments. The only rules that push locations to S
areE:APP and E:APPAPP. To pop locations from the stack we modifythe
leaf rules that can return a function closure, namely, the
rulesE:VAR and E:ABS for variables and lambda abstractions:
Wheneverwe would return a function closure pλx.e, V q we inspect
theargument stack S. If S contains a location ` then we pop it
formthe stack S, bind it to the argument x, and evaluate the
functionbody e in the new environment V rx ÞÑ `s. This is defined
by therule E:ABSBIND and indirectly by the rule E:VARAPP.
Anotherrule that modifies the argument stack is E:LET2. Here, we
evaluatethe subexpression e1 with an empty argument stack because
thearguments on the stack when evaluating the let expressions
areconsumed by the result of the evaluation of e2.
The argument stack accurately captures Inria’s OCaml
compiler’sbehavior to avoid the creation of intermediate function
closures. Italso extends naturally to the evaluation of expressions
that are notin share-let-normal form. As we will see in Section 6,
the argumentstack is also necessary to prove the soundness of the
multivariateresource bound analysis.
Another important feature of the big-step semantics, is that
itcan model failing and diverging evaluations by allowing
partialderivation judgments that can be used to derive the resource
usageafter n steps. Technically, this is realized by the rule
E:ABORTwhich can be applied at any point to abort the current
evaluationwithout additional resource cost. The mechanism of
abording anevaluation is most visible in the rules E:LET1 and
E:LET2: Duringthe evaluation of a let expression we have two
possibilties. The firstpossibility is that the evaluation of the
subexpression e1 is abortedusing E:ABORT at some point. We can then
apply the rule E:LET1to pass on the resource usage before the
abort. The second possibilityis that e1 evaluates to a location `.
We can then apply the E:LET2to bind ` to the variable x and
evaluate the expression e2.
4. Simple Type SystemIn this section, we introduce a type system
that is a refinement ofOCaml’s type system. In this type system, we
mirror the resource-aware type system and introduce some
particularities that explainfeatures of the resource-aware types.
For the purpose of this article,we define simple types as
follows.
T ::“ X | T ref | T1 ˚ ¨ ¨ ¨ ˚ Tn | rT1, . . . , Tns Ñ T| µX.
〈C1 : T1˚Xn1 , . . . , Ck : Tk˚Xnk 〉
A (simple) type T is an uninterpreted type variable X P X ,
atype T ref of references of type T , a tuple type T1 ˚ ¨ ¨ ¨ ˚
Tn,a function type rT1, . . . , Tns Ñ T , or an inductive data
typeµX. 〈C1 : T1˚Xn1 , . . . , Ck : Tk˚Xnk 〉.
Two parts of this definition are non-standard and deserve
furtherexplanation. First, bracket function types rT1, . . . , Tns
Ñ T cor-respond to the standard function type T1 Ñ ¨ ¨ ¨ Ñ Tn Ñ T .
Themeaning of rT1, . . . , Tns Ñ T is that the function is applied
to itsfirst n arguments at the same time. The type T1 Ñ ¨ ¨ ¨ Ñ Tn
Ñ Tindicates that the function is applied to its first n arguments
one afteranother. These two uses of a function can result in a very
differentresource behavior. For instance, in the latter case we
have to createn´ 1 function closures. Also we have n different
costs to accountfor: the evaluation cost after the first argument
is present, the cost ofthe closure when the second argument is
present, etc. Of course, it ispossible that a function is used in
different ways in program. We ac-count for that with let
polymorphism (see the following subsection).
Also note that rT1, . . . , Tns Ñ T still describes a
higher-order func-tion while T1 ˚ ¨ ¨ ¨ ˚ Tn Ñ T describes a
first-order function withn arguments.
Second, inductive types are required to have a particular
form.This makes it possible to track cost that depends on size
parametersof values of such types. It is of course possible to
allow arbritaryinductive types and not to track such cost. Such an
extension isstraighforward and we do not present it in this
article.
We assume that each constructor C P C is part of at mostone
recursive type. Furthermore we assume that each recursivetype has
at least one constructor. For an inductive type T “µX. 〈C1 : T1˚Xn1
, . . . , Ck : Tk˚Xnk 〉 we sometimes write T “〈C1 : pT1, n1q, . . .
, Ck : pTk, nkq〉. We say that Ti is the node typeand ni is the
branching number of the constructor Ci. The maximalbranching number
n “ maxtn1, . . . , nku of the constructors is thebranching number
of T .
Let Polymorphism and Sharing. Modelling the design of
theresource-aware type system, our simple type system is affine.
Thatmeans that a variable in a context can be used at most once in
anexpression. However, we enable multiple uses of a variable with
thesharing expression sharex as px1, x2q in e that denotes that x
canbe used twice in e using the (different) names x1 and x2. For
inputprograms we allow multiple uses of a variable x an expression
ein RAML. We then introduce sharing constructs, and replace
theoccurrences of x in e with the new names before the
analysis.
Interestingly, this mechanism is closely related to let
polymor-phism. To see this relation, first note that our type
system is poly-morphic but that a value can only be used with a
single type in anexpression. In practice, that would mean for
instance that we have todefine a different map function for every
list type. A simple and well-known solution to this problem that is
often applied in practice islet polymorphism. In principle, let
polymorphism replaces variableswith their definitions before type
checking. For our map function itwould mean to type the expression
rmap ÞÑ emapse instead of typingthe expression let map “ emap in
e.
In principle, it would be possible to treat sharing of variables
in asimilar way as let polymorphism. But if we start form an
expressionletx “ e1 in e2 and replace the occurrences of x in the
expressione2 with e1 then we also change the resource consumption
of theevaluation of e2 because we evaluate e1 multiple times.
Interestingly,this problem coincides with the treatment of let
polymorphism forexpressions with side effects (the so called value
restriction).
In RAML, we support let polymorphism for function closuresonly.
Assume we have a function definition let f “ λx.ef in e thatis used
twice in e. Then the usual approach to enable the analysis inour
system would be to use sharing
let f “ λx.ef in share f as pf1, f2q in e1 .
To enable let polymorphism, we will however define f twice
andensure that we only pay once for the creation of the closure and
thelet binding:
let f1 “ λx.ef in let f2 “ λx.ef in e1
The functions f1 and f2 can now have different types. This
methodcan cause an exponential blow up of the size of the
expression. Itis nevertheless appealing because it enables us to
treat resourcepolymorphism in the same way as let polymorphism.
Type Judgements. Type judgements have the form
Σ; Γ $ e : T
where Σ “ T1, . . . , Tn is a list of types, Γ : Var á T is a
typecontext that maps variables to types, e is a core expression,
and Tis a (simple) type. The intuitive meaning (which is formalized
laterin this section) is as follows. Given an evaluation
environment that
Draft 5 2015/7/27
-
¨;x : T $ x : T (T:VAR) Σ;x : Σ Ñ T $ x : T (T:VARPUSH) ¨;x :
rT1, . . . , TnsÑT, x1:T1, . . . , xn:Tn $ xx1 ¨ ¨ ¨xn :
T(T:APP)
Σ; Γ, x:T1 $ e : T2T1::Σ; Γ $ λx.e : T2
(T:ABSPUSH)Σ;x : rT1, . . . , TnsÑΣÑT, x1 : T1, . . . , xn : Tn
$ xx1 ¨ ¨ ¨xn : T
(T:APPPUSH)
Σ; Γ $ λx.e : T¨; Γ $ λx.e : Σ Ñ T (T:ABSPOP)
T “ µX. 〈. . . C : U˚Xn . . .〉¨;x : U˚Tn $ C x : T (T:CONS)
¨; Γ1 $ e1 : T1 Σ; Γ2, x : T1 $ e2 : T2Σ; Γ1,Γ2 $ letx “ e1 in
e2 : T2
(T:LET)
Σ; Γ $ e : BΣ; Γ, x:A $ e : B (T:WEAK)
F fi f1 “ λx1.e1 and ¨ ¨ ¨ and fn “ λxn.en∆ “ f1 : T1, . . . ,
fn : Tn @i : ¨; Γi,∆ $ λxi.ei : Ti Σ; Γ0,∆ $ e : T
Σ; Γ0, . . . ,Γn $ let recF in e : T(T:LETREC)
Figure 3. Selected rules of the simple affine type system.
matches the type context Γ and an argument stack that matches
thetype stack Σ then e evaluates to a value of type T .
The most interesting feature of the type judgements is
thehandling of bracket function types rT1, . . . , Tns Ñ T . Even
thoughfunction types can have multiple forms, a well-typed
expressionhas often a unique type (in a given type context). This
type isderived from the way a function is used. For instance, we
haveλf.λx.λy.f x y : prT1, T2s Ñ T q Ñ T1 Ñ T2 Ñ T andλf.λx.λy.pf
xq y : pT1 Ñ T2 Ñ T q Ñ T1 Ñ T2 Ñ T , andthe two function types are
unique.
A type T of an expression e has a unique type derivation
thatproduces a type judgement ¨,Γ $ e : T with an empty typestack.
We call this canonical type derivation for e and a closedtype
judgement. If T is a function type Σ Ñ T 1 then there is asecond
type derivation for e that we call an open type derivation.
Itderives the open type judgement Σ; Γ $ e : T 1 where |Σ| ą 0.
Thefollowing lemma can be proved by induction on the type
derivations.
Lemma 1. ¨; Γ $ e : Σ Ñ T if and only if Σ; Γ $ e : T .
Open and canonical type judgements are not interchangeable.An
open type judgement Σ; Γ $ e : T can only appear in aderivation
with an open root of the form Σ1,Σ; Γ $ e : T , orin a subtree of a
derivation whose root is a closed judgement of theform ¨; Γ $ e :
Σ2,Σ Ñ T where |Σ2| ą 0. In other words, in anopen derivation Σ; Γ
$ e : T , the expression e is a function thathas to be applied to n
ą |Σ| arguments at the same time. In a giventype context and for a
fixed function type, a well-typed expressionhas as most one open
type derivation.
Type Rules. Figure 3 presents selected type rules of the
typesystem. As usual Γ1,Γ2 denotes the union of the type contexts
Γ1and Γ2 provided that dompΓ1q X dompΓ2q “ H. We thus have
theimplicit side condition dompΓ1qXdompΓ2q “ H whenever Γ1,Γ2occurs
in a typing rule. Especially, writing Γ “ x1:T1, . . . , xk:Tkmeans
that the variables xi are pairwise distinct.
There is a close correspondence between the evaluation rules
andthe type rules in the sense that every evaluation rule
corresponds toexactly one type rule. (We view the two rules for
pattern match andlet binding as one rule, respectively.) The type
stack is modified bythe rules T:VARPUSH, T:APPPUSH, T:ABSPUSH, and
T:ABSPOP.For every leaf rule that can return a function type, such
as T:VAR,T:APP, and T:APPPOP, we add a second rule that derives
theequivalent open type. The reason becomes clear in the
resource-aware type system in Section 6. The rules that directly
control theshape of the function types are T:ABSPUSH and T:ABSPOP
forlambda abstraction. While the other rules are
(deterministically)syntax driven, the rules for lambda abstraction
introduce a non-
deterministic choice. However, there is often only one
possiblechoice depending on how the abstracted function is
used.
As mentioned, the type system is affine and every variable in
acontext can at most be used once in the typed expression.
Multipleuses have to be introduced explicitly using the rule
T:SHARE. Theonly exception is the rule T:LETREC. Here we allow the
use of thecontext ∆ in the body of all defined functions. The
reason for this isapparent in the resource aware version: sharing
of function types isalways possible without any restrictions.
Well-Formed Environments. For each simple type T we induc-tively
define a set JT K of values of type T . Our goal here is not
toadvance the state of the art in denotational semantics but rather
tocapture the tree structure of data structures stored on the heap.
Tothis end, we distinguish mainly inductive types (possible inner
nodesof the trees) and other types (leaves). For the formulation of
typesoundness, we also require that function closures are
well-formed.We simply interpret polymorphic data with the set of
locations Loc.
JXK “ LocJT refK “ tRpaq | a P JT Ku
JΣ Ñ T K “ tpλx.e, V q | DΓ : H ( V :Γ^ ¨; Γ $ λx.e : ΣÑT uJT1
˚¨ ¨ ¨˚ TnK “ JT1Kˆ ¨ ¨ ¨ ˆ JTnK
JBK “ TrpBq if B “ 〈C1:pT1, n1q, . . . , Cn:pTk, nkq〉
Here, T “ Trp〈C1:pT1, n1q, . . . , Cn:pTk, nkq〉q is the set of
treesτ with node labels C1, . . . , Ck which are inductively
defined asfollows. If i P t1, . . . , ku, ai P JTiK, and τj P T for
all 1 ď j ď nithen Cipai, τ1, . . . , τniq P T .
If H is a heap, ` is a location, A is a type, and a P JAK thenwe
write H ( ` ÞÑ a :A to mean that ` defines the semanticvalue a P
JAK when pointers are followed in H in the obvious way.The judgment
is formally defined in Figure 4. For a heap H theremay exist
different semantic values a and simple types A such thatH ( ` ÞÑ a
:A . However, if we fix a simple type A and a heap Hthen there
exists at most one value a such that H ( ` ÞÑ a :A .
We write H ( ` :A to indicate that there exists a,
necessarilyunique, semantic value a P JAK so that H ( v ÞÑ a :A .
Anenvironment V and a heap H are well-formed with respect to
acontext Γ if H ( V pxq : Γpxq holds for every x P dompΓq. Wethen
writeH ( V : Γ. Similarly, an argument stack S “ `1, . . . , `nis
well-formed with respect to a type stack Σ “ T1, . . . , Tn in
heapH , written H ( S : Σ, if H ( `i :Ti for all 1 ď i ď n.
Note that the rules in Figure 4 are interpreted coinductively.
Thereason is that in the rule V:FUN, the location ` can be part of
theclosure environment V if the closure has been created with the
ruleE:LETREC. The influence of the coinductive definition on the
proofsis minimal since all proofs in this article are by
induction.
Draft 6 2015/7/27
-
X P X ` P dompHqH ( ` ÞÑ ` : X
(V:TVAR)
Hp`q “ `1 H ( `1 ÞÑ a : TH ( ` ÞÑ Rpaq : T ref
(V:REF)
Hp`q “ pλx.e, V qDΓ : H ( V : Γ ^ ¨; Γ $ λx.e : Σ Ñ T
H ( ` ÞÑ pλx.e, V q : Σ Ñ T(V:FUN)
Hp`q “ p`1, . . . , `nq @i : H ( `i ÞÑ ai : TiH ( ` ÞÑ pa1, . .
. , anq : T1 ˚ ¨ ¨ ¨ ˚ Tn
(V:TUPLE)
B “ µX. 〈. . . , C : T˚Xn, . . .〉Hp`q “ pC, `1q H ( `1 ÞÑ pa,
b1, . . . , bnq : T˚Bn
H ( ` ÞÑ Cpa, b1, . . . , bnq : B(V:CONS)
Figure 4. Coinductively relating heap cells to semantic
values.
Type Preservation. Theorem 1 shows that the evaluation of a
well-typed expression in a well-formed environment results in a
well-formed environment.
Theorem 1. If Σ; Γ $ e : T , H ( V : Γ, H ( S : Σ, andS, V,H M$
e ó p`,H 1q | pq, q1q then H 1 ( V : Γ, H 1 ( S : Σ,and H 1 ( ` : T
.
Theorem 1 is proved by induction on the evaluation
judgement.
5. Multivariate Resource PolynomialsIn this section we define
the set of resource polynomials which is asearch space of our
automatic resource bound analysis. A resourcepolynomial p : JT K Ñ
Q`0 maps a semantic value of some simpletype T to a non-negative
rational number.
An analysis of typical polynomial computations operating ona
list ra1, . . . , ans shows that it consists of operations that
areexecuted for every k-tuple pai1 , . . . , aik q with 1 ď i1 ă ¨
¨ ¨ ăik ď n. The simplest examples are linear map operations
thatperform some operation for every ai. Other common examples
aresorting algorithms that perform comparisons for every pair pai,
ajqwith 1 ď i ă j ď n in the worst case.
In this article, we generalize this observation to user-defined
tree-like data structures. In lists of different node types with
constructorsC1, C2 and C3, a linear computation is for instance
often carriedout for all C1-nodes, all C2-nodes, or all C1 and C3
nodes. Ingeneral, a typical polynomial computation is carried out
for alltuples pa1, . . . , akq such that ai is a list element with
constructorCj for some j and ai appears in the list before ai`1 for
all i.
As in previous work, which considered binary trees, we
willessentially interpret all tree-like data structures as lists
with differentnodes by flattening them in pre-order. As a result,
our resourcepolynomials only depend on the number of nodes of a
certain kindin tree but not on structural measures like the height
of the tree.To include the height into resource polynomials in a
general way,we would need a way to express a maximum (or a choice)
in theresource polynomials. We leave this for future research in
favorof compositionality and modularity. In practice, it is useful
thatthe potential of a data structure is invariant under changes in
thestructure of the tree.Base Polynomials and Indices. In Figure 5,
we define for eachsimple type T a set PpT q of functions p : JT K Ñ
N that map valuesof type T to natural numbers. The resource
polynomials for type Tare then given as non-negative rational
linear combinations of thesebase polynomials.
λa. 1 P PpT q@i : pi P PpTiq
λ~a.ź
i“1,...,kpipaiq P PpT1 ˚ ¨ ¨ ¨ ˚ Tkq
B “ 〈C1 : pT1, n1q, . . . , Cm : pTk, nmq〉C “ rCj1 , . . . , Cjk
s @i : pi P PpTjiqλ b.
ÿ
~aPτBpC,bq
ź
i“1,...,kpipaiq P PpBq
Figure 5. Defining the set PpT q of base polynomials for type T
.
˚ P IpT q@j : Ij P IpTjq
pI1, . . . , Ikq P IpT1 ˚ ¨ ¨ ¨ ˚ Tkq
B “ 〈C1 : pT1, n1q, . . . , Ck : pTm, nmq〉 @i : Iji P
IpTjiqr〈I1, Cj1〉 , . . . , 〈Ik, Cjk 〉s P IpBq
Figure 6. Defining the set IpT q of indices for type T .
Let B “ 〈C1 : pT1, n1q, . . . , Ck : pTk, nmq〉 be an
inductivetype. Let C “ rCj1 , . . . , Cjk s be a list of
B-constructors andb P JBK. We inductively define a set τBpC, bq of
k-tuples asfollow: τBpC, bq is the set of k-tuples pa1, . . . , akq
such thatCj1pa1,~b1q, . . . , Cjk pak,~bkq are nodes in the tree b
P JBK andCj1pa1,~b1q ăpre ¨ ¨ ¨ ăpre Cjk pak,~bkq for the pre-order
ăpre on b.
Like in the lambda calculus, we use the notation λa. epaq for
theanonymous function that maps an argument a to the natural
numberthat is defined by the expression epaq. Every set PpT q
contains theconstant function λa. 1. In the case of an inductive
data type B thisarises also for C “ rs (one element sum, empty
product).
In Figure 6, we indicatively define for each simple type T a
setof indices IpT q. For tuple types T1 ˚ ¨ ¨ ¨ ˚ Tk we identify
the index‹ with the index p‹, . . . , ‹q. Similarly, we identify
the index ‹ withthe index rs for inductive types.
Let T be a base type. For each index i P IpT q, we define a
basepolynomial pi : JT K Ñ N as follows.
p‹paq “ 1ppI1,...,Ikqpa1, . . . , akq “
ź
j“1,...,kpIj pajq
pr〈I1,C1〉,...,〈Ik,Ck〉spbq “ÿ
~aPτBprC1,...,Cks,bq
ź
j“1,...,kpIj pajq
Examples. To illustrate the definitions, we construct the set
ofbase polynomials for different data types. It is handy to use the
unittype that is treated like a type variable X in the previous
definitionsbut only has a single semantic value, that is, JunitK “
tpqu.
We first consider the inductive type singleton that has only
oneconstructor without arguments.
singleton “ µX 〈Nil : unit〉
Then we have JsingletonK “ tNilppqqu and Ppsingletonq “tλa. 1, λ
a. 0u. To see why, we first examine the set of tuplesT pCq “
τsingletonpC,Nilppqqq for different list of constructors C. If|C| ą
1 then T pCq “ H because the tree Nilppqq does not containany
tuples of size 2. Thus we have pr〈I1,C1〉,...,〈Ik,Ck〉spNilppqqq “0
in this case (empty sum). The only list of remaining construc-tor
lists C are rs and r〈‹,Nil〉s. As always prspNilppqqq “ 1
Draft 7 2015/7/27
-
(singleton sum). Furthermore pr〈‹,Nil〉spNilppqqq “ 1
becauseτsingletonpr〈‹,Nil〉s,Nilppqqq “ tNilppqqu and P (unit) =
{λa. 1}.
Let us now consider the usual sum type
sumpT1, T2q “ µX 〈Left : T1,Right : T2〉 ; .Then JsumpT1, T2qK “
tLeftpaq | a P JT1Ku Y tRightpbq | b PJT2Ku. If we define
σCppqpC 1paqq"
ppaq if C “ C 10 otherwise
then PpsumpT1, T2qq “ tx ÞÑ 1, x ÞÑ 0uYtσLeftppq | p P PpT1quY
tσRightppq | p P PpT2qu.
The next example is the list type
listpT q “ µX 〈Cons : T˚X,Nil : unit〉 .Then JlistpT qK “
tNilppqq,Conspa1,Nilppqqq, . . .u and we writeJlistpT qK “ trs,
ra1s, ra1, a2s, . . . | ai P JT Ku. It holds thatτlistpr〈‹,Cons〉s,
ra1, . . . , ansq “ ta1, . . . , anu and moreoverτlistpr〈‹,Cons〉 ,
〈‹,Cons〉s, ra1, . . . , ansq “ tpai, ajq | 1 ď i ăj ď nu. More
general, τlistpC, ra1, . . . , ansq “ tpai1 , . . . , aik q |1 ď i1
ă ¨ ¨ ¨ ă ik ď nu if C “ r〈‹,Cons〉 , . . . , 〈‹,Cons〉s orC “
r〈‹,Cons〉 , . . . , 〈‹,Cons〉 , 〈‹,Nil〉s for lists of length k
andk`1, respectively. On the other hand, τlistpD, ra1, . . . , ansq
“ H ifD “ 〈‹,Nil〉 ::D1 for someD1 ‰ rs. Since
ř
~aPτlistpC,ra1,...,ansq 1 “`
nk
˘
and λa. 1 P PpT q we have tλ b.`|b|n
˘
| n P Nu Ă PplistpT qq.Finally consider a list type with two
different Cons-nodes
list2pT1, T2q “ µX 〈C1 : T1˚X,C2 : T2 ˚X,Nil : unit〉 .Then
Jlist2pT qK “ trs, ra1s, ra1, a2s, . . . | ai P tC1, C2u ˆ JT Ku.We
furthermore have τlist2pr〈‹,C1〉s, rb1, . . . , bnsq “ tb1, . . . ,
bn |@iDa : bi “ pC1, aqu and τlist2pr〈‹,C1〉 , 〈‹,C2〉s, rb1, . . . ,
bnsq “tpbi, bjq | @i, jDa, a1 : bi “ pC1, aq ^ bj “ pC2, a1q ^ 1 ď
i ăj ď nu. Let C “ r〈‹,Cons〉 , . . . , 〈‹,Cons〉s and |C| “ k.
Itfurthermore holds that
ř
~aPτlist2pC,bq 1 “`|b|C1k
˘
where |b|C1 de-notes the number of C1-nodes in the list b.
Therefore we havetλ b.
`|b|C1n
˘
| n P Nu Ă Pplist2pT qq.Coinductive types like streampT q “ µX
〈St : T˚X〉 are not
inhabited in our language since we interpret them inductively.
Adata structure of such a type cannot be created since we
allowrecursive definitions only for functions.
Spurious Indices. The previous examples illustrate that for
someinductive data structures, different indices encode the same
re-source polynomial. For example, for the type listpT q we
havepr〈‹,Nil〉spaq “ prspaq “ 1 for all lists a. Additionally, some
in-dices encode a polynomial that is constantly zero. For the
typelistpT q this is for example the case for p〈‹,Nil〉 ::C if |C| ą
0. Wecall such indices spurious.
In practice, it is not beneficial to have spurious indices in
the in-dex sets since they slow down the analysis without being
useful com-ponents of bounds. It is straightforward to identify
spurious indicesfrom the data type definition. The index r〈I1, C1〉
, . . . , 〈Ik, Ck〉s isfor example spurious if k ą 1 and the
branching number of Ci is 0for an i P t1, . . . , k ´ 1u.Resource
Polynomials. A resource polynomial p : JT K Ñ Q`0for a simple type
T is a non-negative linear combination of basepolynomials,
i.e.,
p “ÿ
i“1,...,mqi ¨ pi
for m P N, qi P Q`0 and pi P PpT q. We write RpT q for the set
ofresource polynomials for the base type T .
Selecting a Finite Index Set. Every resource polynomial is
de-fined by a finite number of base polynomials. In an
implementation,
we also have to fix a finite set of indices to make possible an
effec-tive analysis. The selection of the indices to track can be
customizedfor each inductive data type and for every program.
However, wecurrently allow the user only to select a maximal degree
of thebounds and then track all indices that correspond to
polynomials ofthe same or a smaller degree.
6. Resource-Aware Type SystemIn this section, we describe the
resource aware type system. Es-sentially, we annotate the simple
type system from Section 4 withresource annotations so that type
derivations correspond to proofsof resource bounds.
Type Annotations. We use the indexes and base polynomials
todefine type annotations and resource polynomials.
A type annotation for a simple type T is defined to be a
family
QT “ pqIqIPIpT q with qI P Q`0We write QpT q for the set of type
annotations for the type T .
An annotated type is a pair pA,Qq where Q is a type
annotationfor the simple type |A| where A and |A| are defined as
follows.
A ::“ X | A ref | A1 ˚ ¨ ¨ ¨ ˚An | 〈rA1, . . . , Ans Ñ B,F〉| µX.
〈C1 : A1˚Xn1 , . . . , Ck : Ak˚Xnk 〉
We define |A| to be the simple type T that can be obtained from
Aby removing all type annotations from function types.
A function type 〈rA1, . . . , Ans Ñ B,F〉 is annotated with a
setF Ď tpQA, QBq | QA P Qp|A1 ˚ ¨ ¨ ¨ ˚ An|q ^ QB P Qp|B|qu.The set
F potentially contains multiple valid resource annotationsfor
arguments and the result of the function.
Potential of Annotated Types and Contexts. Let pA,Qq be
anannotated type. LetH be a heap and let v be a value withH ( v ÞÑa
: |A|. Then the type annotation Q defines the potential
ΦHpv : pA,Qqq “ÿ
IPIpT qqI ¨ pIpaq
Usually, we define type annotations Q by only stating the values
ofthe non-zero coefficients qI .
If a P J|A|K and Q P Qp|A|q is a type annotation then we
alsowrite Φpa : pA,Qqq for
ř
I qI ¨ pIpaq.For use in the type system we need to extend the
definition of
resource polynomials to type contexts and stacks. We treat
themlike tuple types. Let Γ “ x1:A1, . . . , xn:An be a type
context andlet Σ “ B1, . . . , Bm be a list of types. The index set
IpΣ; Γq isdefined through
IpΣ; Γq“tpI1, . . . , Im, J1, . . . , Jnq | IjPIp|Bj |q,
JiPIp|Ai|u .
A type annotation Q for Σ; Γ is a family
Q “ pqIqIPIpΣ;Γq with qI P Q`0 .
We denote a resource-annotated context with Σ; Γ;Q. Let H bea
heap and V be an environment with H ( V : Γ whereH ( V pxjq ÞÑ axj
: |Γpxjq| . Let furthermore S “ `1, . . . , `m bean argument stack
with H ( S : Σ where H ( `i ÞÑ bi : |Bi| forall i. The potential of
Σ; Γ;Q with respect to H and V is
ΦS,V,HpΣ; Γ;Qq “ÿ
~IPIpΣ;Γq
q~I
mź
j“1pIj pbjq
m`nź
j“m`1pIj paxj q
Here, ~I “ pI1, ¨ ¨ ¨ , Im`nq. In particular, if Σ “ Γ “ ¨
thenIpΣ; Γq “ tpqu and ΦV,HpΣ; Γ; qpqq “ qpq. We sometimes
alsowrite q‹ for qpq.
Draft 8 2015/7/27
-
Q “ Q1 `M var
¨;x:B;QM$x : pB,Q1q(A:VAR)
pP, P 1q P F πΣ;¨‹ pQq “ P `M var P 1 “ Q1
Σ;x: 〈Σ Ñ B,F〉 ;QM$x : pB,Q1q(A:VARPUSH)
Γ “ x1:A1, . . . , xn:An pP, P 1q P F πΓ‹ pQq “ P `M appn Q1 “ P
1
¨;x: 〈rA1, . . . , AnsÑB,F〉 ,Γ;QM$xx1 ¨ ¨ ¨xn :
pB,Q1q(A:APP)
Γ “ x1:A1, . . . , xn:An pP, P 1q P F pR,R1q P F 1 πΓ‹ pQq “ P
`M appn πΣ‹ pQq´q‹`p1‹ “ R R1 “ Q1
Σ;x:〈rA1, . . . , AnsÑ
〈ΣÑB,F 1
〉,F
〉,Γ;QM$xx1 ¨ ¨ ¨xn : pB,Q1q
(A:APPPUSH)
Σ; Γ, x:A;P M$ e : pB,Q1q Q “ R`Mbind @ I, ~J : rpI, ~Jq “
pp~J,IqA::Σ; Γ;QM$λx.e : pB,Q1q
(A:ABSPUSH)
Q “ Q1 `M abs @pP, P 1q P F : Σ; Γ;RM$λx.e : pB,P 1q ^ rp~I, ~Jq
“"
p~I if ~J “ ~‹0 otherwise
¨; Γ;QM$λx.e : p〈Σ Ñ B,F〉 , Q1q(A:ABSPOP)
B “ µX. 〈. . . C : A˚Xn . . .〉 Σ; Γ, y:A˚Bn;P M$ e1 : pA1, P
1qΣ; Γ, x:B;RM$ e2 : pA1, R1q CCBpQq “ P`Mmat1 P 1 “ Q1 Q “ R`Mmat2
R1 “ Q1
Σ; Γ, x:B;QM$match x with C y Ñ e1 | e2 : pA1, Q1q(A:MAT)
B “ µX. 〈. . . C : A˚Xn . . .〉 Q “ CCBpQ1q`M cons
¨;x:A˚Bn;QM$C x : pB,Q1q(A:CONS)
Σ; Γ, x1:A, x2:A;P M$ e : pB,Q1q Q “M share` .pP qΣ; Γ, x:A;QM$
sharex as px1, x2q in e : pB,Q1q
(A:SHARE)
Σ; Γ2,Γ1;P M$ e1 Σ; Γ2, x:A;P 1 Σ; Γ2, x:A;RM$ e2 : pB,Q1q Q “ P
`M let1 P 1 “ R`M let2Σ; Γ2,Γ1;QM$ letx “ e1 in e2 : pB,Q1q
(A:LET)
F fi f1 “ λx1.e1 and ¨ ¨ ¨ and fn “ λxn.en ∆ “ f1:A1, . . . ,
fn:An@i : ¨; Γi,∆;Pi M$λxi.ei : pAi, P 1i q πΣ;Γ0~‹ pQq “ π
Σ;Γ0~‹ pP q `M
rec ` n¨M abs Σ; Γ0,∆;P M$ e : pB,Q1qΣ; Γ0, . . . ,Γn;QM$ let
recF in e : pB,Q1q
(A:LETREC)
Σ; Γ;P $ e : pB,P 1q Q ě P ` c Q1 ď P 1 ` cΣ; Γ;Q $ e :
pB,Q1q
(A:WEAK-A)Σ; Γ;πΓ‹ pQqM$ e : pB,Q1qΣ; Γ, x:A;QM$ e : pB,Q1q
(A:WEAK-C)
Σ; Γ;QM$ e : pB1, Q1q B1 ă: BΣ; Γ;QM$ e : pB,Q1q
(A:SUBTYPE-R)Σ; Γ, x:A1;QM$ e : pB,Q1q A ă: A1
Σ; Γ, x:A;QM$ e : pB,Q1q(A:SUBTYPE-C)
˛ ˛ ˛
@j P IpΣ; ∆q: j“~‹ ùñ ¨; Γ;πΓj pQqM$ e : pA, πx:Aj pQ1qq j‰~‹ ùñ
¨; Γ;πΓj pQq cf$ e : pA, πx:Aj pQ1qqΣ; ∆,Γ;QM$ e Σ; ∆, x:A;Q1
(B:BIND)
Figure 7. Selected type rules for annotated types.
Folding of Potential Annotations. A key notion in the type
systemis the folding for potential annotations that is used to
assign potentialto typing contexts that result from a pattern match
(unfolding) orfrom the application of a constructor of an inductive
data type(folding). Folding of potential annotations is
conceptually similar tofolding and unfolding of inductive data
types in type theory.
Let B “ µX. 〈. . . , C : A˚Xn, . . .〉 be an inductive datatype.
Let Σ be a type stack, Γ, b:B be a context and let Q
“pqIqIPIpΣ;Γ,y:Bq be a context annotation. The C-unfolding CCBpQqof
Q with respect to B is an annotation CCBpQq “ pq1IqIPIpΣ;Γ1qfor a
context Γ1 “ Γ, x:A˚Bn that is defined by
q1pI,pJ,L1,...,Lnqq “"
qpI,〈J,C〉 ::L1¨¨¨Lnq ` qpI,L1¨¨¨Lnq j “ 0qpI,〈J,C〉 ::L1¨¨¨Lnq j
‰ 0
Here, L1 ¨ ¨ ¨Ln is the concatenation of the lists L1, . . . ,
Ln.Lemma 2. Let B “ µX. 〈. . . , C : A˚Xn, . . .〉 be an
inductivedata type. Let Σ; Γ, x:B;Q be an annotated context, H ( V
:Γ, x:B, H ( S : Σ, HpV pxqq “ pC, `q, and V 1 “ V ry ÞÑ`s. Then H
( V 1 : Γ, y:A˚Bn and ΦS,V,HpΣ; Γ, x:B;Qq “ΦS,V 1,HpΣ; Γ,
y:A˚Bn;CCBpQqq.
Sharing. Let Σ; Γ, x1:A, x2:A;Q be an annotated context.
Thesharing operation . Q defines an annotation for a context ofthe
form Σ; Γ, x:A. It is used when the potential is split
betweenmultiple occurrences of a variable. Lemma 3 shows that
sharing is alinear operation that does not lead to any loss of
potential.
Lemma 3. Let A be a data type. Then there are natural
numberscpi,jqk for i, j, k P Ip|A|q such that the following holds.
For ev-
Draft 9 2015/7/27
-
ery context Σ; Γ, x1:A, x2:A;Q and every H,V with H ( V :Γ, x:A
and H ( S : Σ it holds that ΦS,V,HpΣ,Γ, x:A;Q1q “ΦS,V 1,HpΣ; Γ,
x1:A, x2:A;Qq where V 1 “ V rx1, x2 ÞÑ V pxqsand q1p`,kq “
ř
i,jPIpAq cpi,jqk qp`,i,jq.
The coefficients cpi,jqk can be computed effectively. We
werehowever not able to derive a closed formula for the
coefficients.The proof is similar as in previous work [33]. For a
contextΣ; Γ, x1:A, x2:A;Q we define .Q to be Q1 from Lemma 3.Type
Judgements. A resource-aware type judgement has the form
Σ; Γ;QM$ e : pA,Q1qwhere Σ; Γ;Q is an annotated context, M is a
resource metric,A is an annotated type and Q1 is a type annotation
for |A|. Theintended meaning of this judgment is that if there are
more thanΦpΣ; Γ;Qq resource units available then this is sufficient
to coverthe evaluation cost of e under metric M . In addition,
there are atleast Φpv:pA,Q1qq resource units left if e evaluates to
a value v.Notations. Families that describe type and context
annotations aredenoted with upper case letters Q,P,R, . . . with
optional super-scripts. We use the convention that the elements of
the families arethe corresponding lower case letters with
corresponding superscripts,i.e., Q “ pqIqIPI and Q1 “ pq1IqIPI
.
If Q,P and R are annotations with the same index set I thenwe
extend operations on Q pointwise to Q,P and R. For example,we write
Q ď P `R if qI ď pI ` rI for every I P I. For K P Qwe write Q “ Q1
`K to state that q‹ “ q1‹ `K ě 0 and qI “ q1Ifor I ‰ ‹ P I. Let Q
be an annotation for a context Σ; Γ1,Γ2. ForJ P IpΓ2q we define the
projection πΓ1pJ,J1qpQq of Q to Γ1 to be theannotation Q1 for ¨; Γ1
with q1I “ qpJ,I,J1q. In the same way, wedefine the annotations πΣJ
pQq for Σ; ¨ and πΣ;Γ1J pQq for Σ; Γ1.Cost Free Types. We write Σ;
Γ;Q cf$ e : pA,Q1q to refer to cost-free type judgments where cf is
the cost-free metric with cfpKq “ 0for constants K. We use it to
assign potential to an extended contextin the let rule. More info
is available in previous work [30].Subtyping. As usual, subtyping
is defined inductively so that typeshave to be structurally
identical. The most interesting rule is the onefor function
types:
F 1 Ď F @i : A1i ă: Ai B ă: B1
〈rA1, . . . , Ans Ñ B,F〉 ă:〈rA11, . . . , A1ns Ñ B1,F 1
〉 (S:FUN)A function type is a subtype of another function type
if it allows moreresource behaviors (F 1 Ď F ). Result types are
treated covariant andarguments are treated contravariant.
Unsurprisingly, our type system does not have principle
types.This is to allow the typing of examples such as rec scheme
fromSection 2. In a principle type, we would have to assume the
weakesttype for the arguments, that is, function types that are
annotatedwith empty sets of type annotations. This would mean that
wecannot use functions in the arguments. However, it is possible
toderive a principle type 〈Σ Ñ B,F〉 for fixed argument types
Σ.Here, we would derive all possible annotations pQ,Q1q P F in
thefunction annotation and all possible annotations pQ,Q1q that
appearin function annotations of the result type.
If we take the more algorithmic view of previous work [41]then
we can express a principle type for a function with a set
ofconstraints that has holes for the constraint sets of the
higher-orderarguments. It is however unclear what such a type means
for a userand we prefer a more declarative view that clearly
separates typechecking and type inference. An open problem with
constraint basedprinciple types is polymorphism.Type Rules. Figure
7 contains selected type rules for annotatedtypes. Many of the
rules are similar to the rules in previous pa-pers [31, 33, 41] and
detailed explanations can be found there.
Soundness. Our goal is to prove the following soundness
state-ment for type judgements. Intuitively, it says that the
initial potentialis an upper bound on the watermark resource usage,
no matter howlong we execute the program.
If Σ; Γ;QM$ e : pA,Q1q and S, V,H M$ e ó ˝ | pp, p1qthen p ď
ΦS,V,HpΣ; Γ;Qq.
To prove this statement by induction, we need to prove a
strongerstatement that takes into account the return value and the
annotatedtype pA,Q1q of e. Moreover, the previous statement is only
trueif the values in S, V and H respect the types required by Σ
andΓ. Therefore, we adapt our definition of well-formed
environmentsto annotated types. We simply replace the rule V:FUN in
Figure 4with the following rule. Of course, H ( V : Γ refers to the
newlydefined judgment.
Hp`q “ pλx.e, V q DΓ, Q,Q1 : H ( V : Γ^¨; Γ;QM$λx.e : p〈Σ Ñ B,F〉
, Q1qH ( ` ÞÑ pλx.e, V q : 〈Σ Ñ B,F〉 (V:FUN)
In addition to the aforementioned soundness, the Theorem 2
statesa stronger property for terminating evaluations. If an
expressione evaluates to a value v in a well-formed environment
then thedifference between initial and final potential is an upper
bound onthe resource usage of the evaluation.
Theorem 2 (Soundness). Let H ( V : Γ, H ( S : Σ, andΣ; Γ;QM$ e :
pB,Q1q.1. If S, V,H M$ e ó p`,H 1q | pp, p1q then p ď ΦS,V,HpΣ;
Γ;Qq,p´ p1 ď ΦS,V,HpΣ; Γ;Qq ´ΦH1p`:pB,Q1qq, and H ( ` : B.
2. If S, V,H M$ e ó ˝ | pp, p1q then p ď ΦS,V,HpΣ; Γ;Qq.Theorem
2 is proved by a nested induction on the derivation
of the evaluation judgment and the type judgment Σ; Γ;Q
$e:pB,Q1q. The inner induction on the type judgment is
neededbecause of the structural rules. There is one proof for all
possibleinstantiations of the resource constants. An sole induction
on thetype judgement fails because the size of the type derivation
canincrease in the case of the function application in which we
retrievea type derivation for the function body from the
well-formedjudgement as defined by the (updated) rule V:FUN.
The structure of the proof matches the structure of the
previoussoundness proofs for type systems based on AARA [31, 33,
34, 41].The induction case of many rules is similar to the
induction casesof the corresponding rules for multivariate AARA for
first-orderprograms [33] and linear AARA for higher-order programs
[41].For one thing, additional complexity is introduced by the
newresource polynomials for user-defined data types. We designedthe
system so that this additional complexity is dealt with locallyin
the rules A:MAT, A:CONS, and A:SHARE. The soundness ofthese rules
follows directly from an application of Lemma 2 andLemma 3,
respectively. As in previous work [34] the well-formedjudgement
that captures type derivations enables us to treat
functionabstraction and application in a very similar fashion as in
the first-order case [33]. The coinductive definition of the
well-formednessjudgement does not cause any difficulties. A major
novel aspectin the proof is the typed argument stack S : Σ that
also carriespotential. Surprisingly, this typed stack is simply
treated like a typedenvironment V : Γ in the proof. It is already
incorporated in theshift and share operations (Lemma 2 and Lemma
3).
We deal with the mutable heap by requiring that array elementsdo
not influence the potential of an array. As a result, we can
provethe following lemma, which is used in the proof of Theorem
2.
Lemma 4. If H ( V :Γ, H ( S : Σ, Σ; Γ;QM$ e : pB,Q1qand stack,
V,H M$ e ó p`,H 1q | pp, p1q then ΦS,V,HpΓ;Qq “ΦS,V,H1pΓ;Qq.
Draft 10 2015/7/27
-
Parser
Type Inference
Input Program
Typed OCamlSyntax Tree
OCamlBytecode
RAML Compiler
Bracket-Type Inference
Share-Let Normal Form
Stack-Based Type Checking
Explicit Let Polymorphism
Typed RAML Syntax Tree
OCaml - C Bindings
RAML Analyzer
Resource Type Interpretation
LP Solver Frontend
Multivariate AARA
CLP
Resource Metrics
Resource Bounds
Figure 8. Implementation of RAML.
7. Implementation and Bound InferenceFigure 8 shows an overview
of the implementation of RAML. Itconsists of about 12000 lines of
OCaml code, excluding the parts thatwe reused from Inria’s OCaml
implementation. The developmenttook around 8 person months. We
found it very helpful to developthe implementation and the theory
in parallel, and many theoreticalideas have been inspired by
implementation challenges.
We reuse the parser and type inference algorithm from OCaml4.01
to derive a typed OCaml syntax tree from the source program.We then
analyze the function applications to introduce bracketfunction
types. To this end, we copy a lambda abstraction for everycall
site. We still have to implement a unification algorithm
sincefunctions, such as let g = f x, that are defined by partial
applicationmay be used at different call sites. Moreover, we have
to deal withfunctions that are stored in references.
In the next step, we convert the typed OCaml syntax tree into
atyped RAML syntax tree. Furthermore, we transform the programinto
share-let-normal form without changing the resource behavior.For
this purpose, each syntactic form has a free flag that
specifieswhether it contributes to the cost of the original
program. Forexample, all share forms that are introduced are free.
We also inserteta expansions whenever they do not influence
resource usage.
After this compilation phase, we perform the actual
multivariateAARA on the program in share-let-normal form. Resource
metricscan be easily specified by a user. We include a metric for
heap cells,evaluation steps, and ticks. The letter allows the user
to flexiblyspecify the resource cost of programs by inserting tick
commandsRaml.tick(q) where q is a (possibly negative)
floating-point number.
In principle, the actual bound inference works similarly as
inprevious AARA systems [32, 34]: First, we fix a maximal degreeof
the bounds and annotate all types in the derivation of the
simpletypes with variables that correspond to type annotations for
resourcepolynomials of that degree. Second, we generate a set of
linearinequalities, which express the relationships between the
addedannotation variables as specified by the type rules. Third, we
solvethe inequalities with Coin-Or’s fantastic LP solver CLP. A
solutionof the linear program corresponds to a type derivation in
whichthe variables in the type annotations are instantiated
accordingto the solution. The objective function contains the
coefficientsof the resource annotation of the program inputs to
minimize theinitial potential. Modern LP solvers provide support
for iterativesolving that allows us to express that minimization of
higher-degreeannotations should take priority.
The type system we use in the implementation
significantlydiffers from the declarative version we describe in
this article. Forone thing, we have to use algorithmic versions of
the type rules inthe inference in which the non-syntax-directed
rules are integratedinto the syntax-directed ones [33]. For another
thing, we annotatefunction types not with a set of type annotations
but with a functionthat returns an annotation for the result type
if presented with anannotation of the return type. The annotations
here are symbolicand the actual number are yet to be determined by
the LP solver.Function annotations have the side effect of sending
constraintsto the LP solver. It would be possible to keep a
constraint set forthe respective function in memory and to send a
copy with fresh
variables to the LP solver at every call. However, it is more
efficientto lazily trigger the constraint generation from the
function body atevery call site when the function is provided with
a return annotation.
To make the resource analysis more expressive, we also
allowresource-polymorphic recursion. This means that we need a
typeannotation in the recursive call that differs from the
annotation inthe argument and result types of the function. To
infer such typeswe successively infer type annotations of higher
and higher degree.Details can be found in previous work [30].
For the most part, our constraints have the form of a
so-callednetwork (or network-flow) problem [49]. LP solvers can
handlenetwork problems very efficiently and in practice CLP solves
theconstraints RAML generates in linear time. Because our
problemsizes are large, we can save memory and time by reducing the
num-ber of constraints that are generated during typing. A
representativeexample of an optimization is that we try to reuse
constraint namesinstead of producing constraints like p “ q.
RAML provides two ways of analyzing a program. In main modeRAML
derives a bound for evaluation cost of the main expressionof the
program, that is, the last expression in the top-level list oflet
bindings. In module mode, RAML derives a bound for everytop-level
let binding that has a function type.
Apart from the analysis itself, we also implemented the
con-version of the derived resource polynomials into
easily-understoodpolynomial bounds and a pretty printer for RAML
types and expres-sions. Additionally, we implemented an efficient
RAML interpreterthat we use for debugging and to determine the
quality of the bounds.
8. Case Study: Bounds for DynamoDB QueriesHaving integrated the
analysis with Inria’s OCaml compiler enablesus to analyze and
compile real programs. An interesting use caseof our resource bound
analysis is to infer worst-case bounds onDynamoDB queries. DynamoDB
is a commercial NoSQL clouddatabase service, which is part of
Amazon Web Services (AWS).Amazon charges DynamoDB users on a
combination of numberof queries, transmitted fields, and
throughput. Since DynamoDBis a NoSQL service, it is often only
possible to retrieve the wholetable—which can be expensive for
large data sets—or single entriesthat are identified by a key
value. The DynamoDB API is availablethrough the Opam package aws.
We make the API available to theanalysis by using tick functions
that specify resource usage. Sincethe query cost for different
tables can be different, we provide onefunction per action and
table.
let db_query student_id course_id =Raml.tick(1.0);
Awslib.get_item ...
In the following, we describe the analysis of a specific
OCamlapplication that uses a database that contains a large table
that storesgrades of students for different courses. Our first
function computesthe average grade of a student for a given list of
courses.
let avge_grade student_id course_ids =let f acc cid =
let (length,sum) = acc inlet grade = match db_query student_id
cid with
| Some q Ñ q| None Ñ raise (Not_found (student_id,cid))
in(length +. 1.0, sum +. grade)
inlet (length,sum) = foldl f (0.0,0.0) course_ids insum /.
length
In 0.03s RAML computes the tight bound 1 ¨ m where m is
thelength of the argument course ids. We omit the standard
definitionsof functions like foldl and map. However, they are not
built-in intoour systems but the bounds are derived form first
principles.
Draft 11 2015/7/27
-
Next, we sort a given list of students based on the average
gradesin a given list of classes using quick sort. As a first
approximationwe use a comparison function that is based on average
grade.
let geq sid1 sid2 cour_ids =avge_grade sid1 cour_ids >=
avge_grade sid2 cour_ids
This results in Opn2mq database queries where n is the number
ofstudents andm is the number of courses. The reason is that there
areOpn2q comparisons during a run of quick sort. Since the
resourceusage of quick sort depends on the number of courses, we
have tomake the list of courses an explicit argument and cannot
store it inthe closure of the comparison function.
let rec partition gt acc l =match l with
| [] Ñ let (cs,bs,_) = acc in (cs,bs)| x::xs Ñ let (cs,bs,aux) =
acc in
let acc’ = if gt x aux then (cs,x::bs,aux)else
(x::cs,bs,aux)
in partition gt acc’ xs
let rec qsort gt aux l = match l with | [] Ñ []| x::xs Ñ
let ys,zs = partition (gt x) ([],[],aux) xs inappend (qsort gt
aux ys) (x::(qsort gt aux zs))
let sort_students s_ids c_ids = qsort geq c_ids s_ids
In 0.31s RAML computes the tight bound n2m ´ nm forsort students
where n is the length of the argument s ids and m isthe length of
the argument c ids. The negative factor arises from thetranslation
of the resource polynomials to the standard basis.
Given the alarming cubic bound, we reimplement our
sortingfunction using memoization. To this end we create a table
thatlooks up and stores for each student and course the grade inthe
DynamoDB. We then replace the function db query with thefunction
lookup.
let lookup sid cid table =let cid_map = find (fun id Ñ id = sid)
table infind (fun id Ñ id = cid) cid_map
For the resulting sorting function, RAML computes the tight
boundnm in 0.87s.
9. Related WorkOur work builds on past research on automatic
amortized resourceanalysis (AARA). AARA has been introduced by
Hofmann andJost for a strict first-order functional language with
built-in datatypes [34]. The technique has been applied to
higher-order func-tional programs and user defined types [41], to
derive stack-spacebounds [16], to programs with lazy evaluation
[47, 52], to object-oriented programs [35, 38], and to low-level
code by integratingit with separation logic [8]. All the
aforementioned amortized-analysis–based systems are limited to
linear bounds. Hoffmann etal. [29, 32, 33] presented a multivariate
AARA for a first-order lan-guage with built-in lists and binary
trees. Hofmann and Moser [37]have proposed a generalization of this
system in the context of(first-order) term rewrite systems.
However, it is unclear how toautomate this system. In this article,
we introduce the first AARAthat is able to automatically derive
(multivariate) polynomial boundsthat depend on user-defined
inductive data structures. Our systemis the only one that can
derive polynomial bounds for higher-orderfunctions. Even for linear
bounds, our analysis is more expressivethan existing systems for
strict languages [41]. For instance, we canfor the first time
derive an evaluation-step bound for the curriedappend function for
lists. Moreover, we integrated AARA for thefirst time with an
existing industrial-strength compiler.
Type systems for inferring and verifying resource bounds
havebeen extensively studied. Vasconcelos et al. [50, 51] described
anautomatic analysis system that is based on sized-types [39]
andderives linear bounds for higher-order functional programs.
Herewe derive polynomial bounds.
Dal Lago et al. [43, 44] introduced linear dependent typesto
obtain a complete analysis system for the time complexity ofthe
call-by-name and call-by-value lambda calculus. Crary andWeirich
[20] presented a type system for specifying and certifyingresource
consumption. Danielsson [22] developed a library, basedon dependent
types and manual cost annotations, that can be usedfor complexity
analyses of functional programs. The advantage ofour technique is
that it is fully automatic.
Classically, cost analyses are often based on deriving and
solvingrecurrence relations. This approach was pioneered by
Wegbreit [53]and is actively studied for imperative languages [1,
5, 7, 25]. Theseworks are not concerned with higher-order functions
and bounds donot depend on user-defined data structures.
Benzinger [11] has applied Wegbreit’s method in an
automaticcomplexity analysis for Nuprl terms. However, complexity
infor-mation for higher-order functions has to be provided
explicitly.Grobauer [26] reported a mechanism to automatically
derive costrecurrences from DML programs using dependent types.
Danner etal. [23, 24] propose an interesting technique to derive
higher-orderrecurrence relations from higher-order functional
programs. Solvingthe recurrences is not discussed in these works
and in contrast to ourwork they are not able to automatically infer
closed-form bounds.
Abstract interpretation based approaches to resource analysis
[12,18, 27, 48, 54] focus on first-order integer programs with
loops.Cicek et al. [19] study a type system for incremental
complexity.
In an active area of research, techniques from term rewritingare
applied to complexity analysis [9, 15, 45]; sometimes in
combi-nation with amortized analysis [36]. These techniques are
usuallyrestricted to first-order programs and time complexity.
Recently,Avanzini et al. [10] proposed a complexity preserving
defunctional-iztion to deal with higher-order programs. While the
transformationis asymptotically complexity preserving, it is
unclear whether thistechnique can derive bounds with precise
constant factors.
Finally, there exists research that studies cost models to
formallyanalyze parallel programs. Blelloch and Greiner [13]
pioneeredthe cost measures work and depth. There are more advanced
costmodels that take into account caches and IO (see, e.g.,
Blelloch andHarper [14]), However, these works do not provide
machine supportfor deriving static cost bounds.
10. ConclusionWe have presented important first steps towards a
practical automaticresource bound analysis system for OCaml. Our
three main contribu-tions are (1) the integration of automatic
amortized resource analysiswith the OCaml compiler, (2) a novel
automatic resource analysissystem that infers multivariate
polynomial bounds that depend onsize parameters of user-defined
data structures, and (3) the firstAARA that infers polynomial
bounds for higher-order functions.
As the title of this article indicates, there are many open
problemsleft on the way to a usable resource analysis system for
OCaml. Inthe future, we plan to improve the bound analysis for
programswith side-effects and exceptions. We will also work on
mechanismsthat allow user interaction for manually deriving bounds
if theautomation fails. Furthermore, we will work on taking into
accountgarbage collection and the runtime system when deriving time
andspace bounds. Finally, we will investigate techniques to link
the high-level bounds with hardware and the low-level code that is
producedby the compiler. These open questions are certainly
challenging butwe now have the tools to further push the boundaries
of practicalquantitative software verification.
Draft 12 2015/7/27
-
References[1] E. Albert, P. Arenas, S. Genaim, G. Puebla, and D.
Zanardini. Cost
Analysis of Java Bytecode. In 16th Euro. Symp. on Prog.
(ESOP’07),pages 157–172, 2007.
[2] E. Albert, P. Arenas, S. Genaim, and G. Puebla. Closed-Form
UpperBounds in Static Cost Analysis. Journal of Automated
Reasoning,pages 161–203, 2011.
[3] E. Albert, P. Arenas, S. Genaim, M. Gómez-Zamalloa, and G.
Puebla.Automatic Inference of Resource Consumption Bounds. In Logic
forProgramming, Artificial Intelligence, and Reasoning, 18th
Conference(LPAR’12), pages 1–11, 2012.
[4] E. Albert, P. Arenas, S. Genaim, G. Puebla, and D.
Zanardini. CostAnalysis of Object-Oriented Bytecode Programs.
Theor. Comput. Sci.,413(1):142 – 159, 2012.
[5] E. Albert, J. C. Fernández, and G. Román-Dı́ez.
Non-cumulativeResource Analysis. In Tools and Algorithms for the
Construction andAnalysis of Systems - 21st International
Conference, (TACAS’15), pages85–100, 2015.
[6] C. Alias, A. Darte, P. Feautrier, and L. Gonnord.
Multi-dimensionalRankings, Program Termination, and Complexity
Bounds of FlowchartPrograms. In 17th Int. Static Analysis Symposium
(SAS’10), pages117–133, 2010.
[7] D. E. Alonso-Blas and S. Genaim. On the limits of the
classicalapproach to cost analysis. In 19th Int. Static Analysis
Symp. (SAS’12),pages 405–421, 2012.
[8] R. Atkey. Amortised Resource Analysis with Separation Logic.
In 19thEuro. Symp. on Prog. (ESOP’10), pages 85–103, 2010.
[9] M. Avanzini and G. Moser. A Combination Framework for
Complex-ity. In 24th International Conference on Rewriting
Techniques andApplications (RTA’13), pages 55–70, 2013.
[10] M. Avanzini, U. D. Lago, and G. Moser. Analysing the
Complexityof Functional Programs: Higher-Order Meets First-Order.
In 29th Int.Conf. on Functional Programming (ICFP’15), 2012.
[11] R. Benzinger. Automated Higher-Order Complexity Analysis.
Theor.Comput. Sci., 318(1-2):79–103, 2004.
[12] R. Blanc, T. A. Henzinger, T. Hottelier, and L. Kovács.
ABC: AlgebraicBound Computation for Loops. In Logic for Prog., AI.,
and Reasoning- 16th Int. Conf. (LPAR’10), pages 103–118, 2010.
[13] G. E. Blelloch and J. Greiner. A Provable Time and Space
EfficientImplementation of NESL. In 1st Int. Conf. on Funct. Prog.
(ICFP’96),pages 213–225, 1996.
[14] G. E. Blelloch and R. Harper. Cache and I/O Efficent
FunctionalAlgorithms. In 40th ACM Symp. on Principles Prog. Langs.
(POPL’13),pages 39–50, 2013.
[15] M. Brockschmidt, F. Emmes, S. Falke, C. Fuhs, and J. Giesl.
Alternat-ing Runtime and Size Complexity Analysis of Integer
Programs. InTools and Alg. for the Constr. and Anal. of Systems -
20th Int. Conf.(TACAS’14), pages 140–155, 2014.
[16] B. Campbell. Amortised Memory Analysis using the Depth of
DataStructures. In 18th Euro. Symp. on Prog. (ESOP’09), pages
190–204,2009.
[17] Q. Carbonneaux, J. Hoffmann, and Z. Shao. Compositional
CertifiedResource Bounds. In 36th Conf. on Prog. Lang. Design and
Impl.(PLDI’15), 2015.
[18] P. Cerný, T. A. Henzinger, L. Kovács, A. Radhakrishna,
and J. Zwirch-mayr. Segment Abstraction for Worst-Case Execution
Time Analysis.In 24th European Symposium on Programming (ESOP’15),
pages 105–131, 2015.
[19] E. Çiçek, D. Garg, and U. A. Acar. Refinement Types for
IncrementalComputational Complexity. In 24th European Symposium on
Program-ming (ESOP’15), pages 406–431, 2015.
[20] K. Crary and S. Weirich. Resource Bound Certification. In
27th ACMSymp. on Principles of Prog. Langs. (POPL’00), pages
184–198, 2000.
[21] S. A. Crosby and D. S. Wallach. Denial of service via
algorithmiccomplexity attacks. In 12th USENIX Security Symposium
(USENIX’12),2003.
[22] N. A. Danielsson. Lightweight Semiformal Time Complexity
Analysisfor Purely Functional Data Structures. In 35th ACM Symp. on
PrinciplesProg. Langs. (POPL’08), pages 133–144, 2008.
[23] N. Danner, D. R. Licata, and R. Ramyaa. Denotational Cost
Semanticsfor Functional Languages with Inductive Types. In 29th
Int. Conf. onFunctional Programming (ICFP’15), 2012.
[24] N. Danner, J. Paykin, and J. S. Royer. A Static Cost
Analysis for aHigher-Order Language. In 7th Workshop on Prog.
Languages MeetsProg. Verification (PLPV’13), pages 25–34, 2013.
[25] A. Flores-Montoya and R. Hähnle. Resource Analysis of
ComplexPrograms with Cost Equations. In Programming Languages
andSystems - 12th Asian Symposiu (APLAS’14), pages 275–295,
2014.
[26] B. Grobauer. Cost Recurrences for DML Programs. In 6th Int.
Conf.on Funct. Prog. (ICFP’01), pages 253–264, 2001.
[27] S. Gulwani, K. K. Mehra, and T. M. Chilimbi. SPEED: Precise
andEfficient Static Estimation of Program Computational Complexity.
In36th ACM Symp. on Principles of Prog. Langs. (POPL’09),
pages127–139, 2009.
[28] J. Hoffmann. Types with Potential: Polynomial Resource
Boundsvia Automatic Amortized Analysis. PhD thesis,
Ludwig-Maximilians-Universität München, 2011.
[29] J. Hoffmann and M. Hofmann. Amortized Resource Analysis
withPolynomial Potential. In 19th Euro. Symp. on Prog. (ESOP’10),
2010.
[30] J. Hoffmann and M. Hofmann. Amortized Resource Analysis
withPolymorphic Recursion and Partial Big-Step Operational
Semantics. InProg. Langs. and Systems - 8th Asian Symposium
(APLAS’10), 2010.
[31] J. Hoffmann and Z. Shao. Type-Based Amortized Resource
Analy-sis with Integers and Arrays. In 12th International Symposium
onFunctional and Logic Programming (FLOPS’14), 2014.
[32] J. Hoffmann, K. Aehlig, and M. Hofmann. Multivariate
AmortizedResource Analysis. In 38th ACM Symp. on Principles of
Prog. Langs.(POPL’11), 2011.
[33] J. Hoffmann, K. Aehlig, and M. Hofmann. Multivariate
AmortizedResource Analysis. ACM Trans. Program. Lang. Syst.,
2012.
[34] M. Hofmann and S. Jost. Static Prediction of Heap Space
Usage forFirst-Order Functional Programs. In 30th ACM Symp. on
Principles ofProg. Langs. (POPL’03), pages 185–197, 2003.
[35] M. Hofmann and S. Jost. Type-Based Amortised Heap-Space
Analysis.In 15th Euro. Symp. on Prog. (ESOP’06), pages 22–37,
2006.
[36] M. Hofmann and G. Moser. Amortised Resource Analysis and
TypedPolynomial Interpretations. In Rewriting and Typed Lambda
Calculi(RTA-TLCA;14), pages 272–286, 2014.
[37] M. Hofmann and G. Moser. Multivariate Amortised Resource
Analysisfor Term Rewrite Systems. In 13th International Conference
on TypedLambda Calculi and Applications (TLCA’15), pages 241–256,
2015.
[38] M. Hofmann and D. Rodriguez. Automatic Type Inference for
Amor-tised Heap-Space Analysis. In 22nd Euro. Symp. on Prog.
(ESOP’13),pages 593–613, 2013.
[39] J. Hughes, L. Pareto, and A. Sabry. Proving the Correctness
of ReactiveSystems Using Sized Types. In 23th ACM Symp. on
Principles of Prog.Langs. (POPL’96), pages 410–423, 1996.
[40] G. Jin, L. Song, X. Shi, J. Scherpelz, and S. Lu.
Understandingand Detecting Real-World Performance Bugs. In
Conference onProgramming Language Design and Implementation
PLDI’12, pages77–88, 2012.
[41] S. Jost, K. Hammond, H.-W. Loidl, and M. Hofmann. Static
Deter-mination of Quantitative Resource Usage for Higher-Order
Programs.In 37th ACM Symp. on Principles of Prog. Langs. (POPL’10),
pages223–236, 2010.
[42] P. C. Kocher. Timing Attacks on Implementations of
Diffie-Hellman,RSA, DSS, and Other Systems. In Advances in
Cryptology - 16thAnnual International Cryptology Conference
(CRYPTO’96), pages 104–113, 1996.
[43] U. D. Lago and M. Gaboardi. Linear Dependent Types and
RelativeCompleteness. In 26th IEEE Symp. on Logic in Computer
Science(LICS’11), pages 133–142, 2011.
Draft 13 2015/7/27
-
[44] U. D. Lago and B. Petit. The Geometry of Types. In 40th ACM
Symp.on Principles Prog. Langs. (POPL’13), pages 167–178, 2013.
[45] L. Noschinski, F. Emmes, and J. Giesl. Analyzing Innermost
RuntimeComplexity of Term Rewriting by Dependency Pairs. J.
Autom.Reasoning, 51(1):27–56, 2013.
[46] O. Olivo, I. Dillig, and C. Lin. Static Detection of
Asymptotic Per-formance Bugs in Collection Traversals. In
Conference on Program-ming Language Design and Implementation
(PLDI’15), pages 369–378,2015.
[47] H. R. Simões, P. B. Vasconcelos, M. Florido, S. Jost, and
K. Ham-mond. Automatic Amortised Analysis of Dynamic Memory
Allocationfor Lazy Functional Programs. In 17th Int. Conf. on
Funct. Prog.(ICFP’12), pages 165–176, 2012.
[48] M. Sinn, F. Zuleger, and H. Veith. A Simple and Scalable
Approachto Bound Analysis and Amortized Complexity Analysis. In
ComputerAided Verification - 26th Int. Conf. (CAV’14), page
743–759, 2014.
[49] R. J. Vanderbei. Linear Programming: Foundations and
Extensions.Springer US, 2001.
[50] P. Vasconcelos. Space Cost Analysis Using Sized Types. PhD
thesis,School of Computer Science, University of St Andrews,
2008.
[51] P. B. Vasconcelos and K. Hammond. Inferring Costs for
Recursive,Polymorphic and Higher-Order Functi