A Type System forA Type System forHigher-Order ModulesHigher-Order Modules

Derek Dreyer, Karl Crary, and Robert Harper

Carnegie Mellon University

POPL 2003

Type Theory for Module SystemsType Theory for Module Systems

• Lots of work on module system design

• Theory has had impact on real language design:– Harper-Lillibridge 94, Leroy 94 ) SML ’97

– Leroy 95 ) Objective Caml

– Russo 00 ) Moscow ML

• No general semantic framework for understanding relationships between designs

A Unifying Type TheoryA Unifying Type Theory

• High-level semantic analysis ) Unifying type theory

• Previous designs can be seen as subsystems

• Key idea: Explain semantics of abstract data types in terms of purity and effects

ProjectibilityProjectibility

• When is a module expression M projectible?– When can we project out M’s type components?

• “Non-projectible” module expression:

if buttonIsPressed() then struct type t = int ... end else struct type t = bool ... end

• “Projectible” module expression: struct type t = int; val x = 3 end

ProjectibilityProjectibility

• When is a module expression M projectible?– When can we project out M’s type components?

• “Non-projectible” module expression:

if buttonIsPressed() then struct type t = int ... end else struct type t = bool ... end

• “Projectible” module expression: struct type t = int; val x = 3 end

ProjectibilityProjectibility

• When is a module expression M projectible?– When can we project out M’s type components?

• “Non-projectible” module expression:

if buttonIsPressed() then struct type t = int ... end else struct type t = bool ... end

• “Projectible” module expression: struct type t = int; val x = 3 end

ProjectibilityProjectibility

• When is a module expression M projectible?– When can we project out M’s type components?

• “Impure” module expression:

if buttonIsPressed() then struct type t = int ... end else struct type t = bool ... end

• “Pure” module expression: struct type t = int; val x = 3 end

ProjectibilityProjectibility

• When is a module expression M projectible?– When can we project out M’s type components?

• “Impure” module expression:

if buttonIsPressed() then struct type t = int ... end else struct type t = bool ... end

• “Pure” module expression: struct type t = int; val x = ref 3 end

PurityPurity

• M is soundly projectible , M is pure (w.r.t. type components) , Type components of M are the same every time M is evaluated

• M is impure ) Meaning of M.t not statically well-determined

Second-Class ModulesSecond-Class Modules

• Second-class modules admit “phase separation”– Type components can’t depend on run-time conditions

• SML and O’Caml modules are second-class because of syntactic restrictions

• All second-class modules are pure– But should they all be projectible?

SealingSealing

• Principal means of creating abstract data types– M :> , aka “opaque signature ascription”

• Treating sealed modules as projectible violates abstraction:– A = (M :> ) and B = (M :> )

– If (M :> ) is projectible, then A.t = (M :> ).t = B.t

– But “A.t = B.t” not observable in

You Can’t Handle the Truth!You Can’t Handle the Truth!

• In truth, sealing doesn’t affect a module’s purity

• But sealing obstructs our knowledge about a module’s purity

• Projectibility is a judgment of knowledge, not truth:– Sealed modules treated as impure/non-projectible

Total and Partial FunctorsTotal and Partial Functors

• To track purity in the presence of functors:– Need to know whether applying a functor will

unleash an effect or not

• Distinguish types of total and partial functors:

– F : tot s:1.2 , body of F is known to be pure

– F : par s:1.2 , body of F could be impure

Total Total ,, Applicative Applicative

• F : tot s:1.2, M : 1, F and M are pure

• structure Res1 = F(M)

• structure Res2 = F(M)

• F(M) known to be pure ) projectible

Res1.t = F(M).t = Res2.t

Partial Partial ,, Generative Generative

• F : par s:1.2, M : 1, F and M are pure

• structure Res1 = F(M)

• structure Res2 = F(M)

• F(M) possibly impure ) non-projectible

Res1.t Res2.t

Functors with SealingFunctors with Sealing

• If body of a functor contains sealing, then:– Body is impure

– Functor is generative

• Can be both a good and bad thing:– Gives correct semantics of abstraction for functors that use

imperative features

– Overly restrictive for purely functional functors

Importance of GenerativityImportance of Generativity

functor MakeSymbolTable() =

(struct

... (* creates new mutable hash table *)

end :>

sig

type symbol

val string_to_symbol : string -> symbol

val symbol_to_string : symbol -> string

end)

)

• Generativity ties the symbol type to the run-time state of the module defining it

Purely Functional AbstractionPurely Functional Abstraction

functor MakeSet (Elem : COMPARABLE) =

(struct ... end :>

sig

type elem = Elem.elem

type set

val insert : elem * set -> set

end)

• What if a sealed module is purely functional?– Abstract types not tied to any run-time state– Only care about hiding type identity

Hoisting the SealingHoisting the Sealing

• Instead of sealing the body of the functor:

s:1. (M :> 2)

• Seal the functor itself (with a total signature):

( s:1. M) :> tot s:1.2

• Problem: Only works if M is pure– Not true if M contains sealed substructures,

such as datatype definitions

Module ClassificationsModule Classifications

Impure, non-projectible

Pure, projectible

Sealing M :>

Static and Dynamic EffectsStatic and Dynamic Effects

• Split effects into two kinds: static and dynamic

• Module with any kind of effects is impure

• Dynamic effects occur “during execution”

• Static effects occur “during typechecking”

Weak and Strong SealingWeak and Strong Sealing

Pure, projectible

Statically impure, dynamically pure

Statically and dynamically impure

Weak Sealing M :: Strong Sealing M :>

Non-projectibleImpure,

Set Functor RevisitedSet Functor Revisited

functor MakeSet (Elem : COMPARABLE) = (struct ... end :: sig

type elem = Elem.elem

type set

val insert : elem * set -> set

end)

• We expand totality to allow body of a total functor to contain static (but not dynamic) effects

A Unifying FrameworkA Unifying Framework

• Standard ML– Only has strong sealing, all functors are partial/generative

• Objective Caml / Leroy (1995)– Only has weak sealing, all functors are total/applicative

• Shao (1999)– Distinguishes two kinds of functor signatures

– Only tracks dynamic effects and strong sealing

• Russo (2000)– Two languages, one like SML and one like O’Caml

– Moscow ML combines them, but language is unsound

Modules as First-Class ValuesModules as First-Class Values

• Packaging modules as first-class values:– Add a new “package type” <>

• Coercions between modules and terms:– If M : , then pack M as <> : <>– If e : <>, then unpack e as :

Modules as First-Class ValuesModules as First-Class Values

structure X = struct type t = int ... endstructure Y = struct type t = bool ... end

M = (if buttonIsPressed() then X else Y)

• Type components of M actually depend onrun-time conditions

• Unpacking induces a truly dynamic effect

Modules as First-Class ValuesModules as First-Class Values

signature S = sig type t ... endstructure X = struct type t = int ... endstructure Y = struct type t = bool ... end

M = unpack (if buttonIsPressed() then pack X as <S> else pack Y as <S>) as S

• Type components of M actually depend onrun-time conditions

• Unpacking induces a truly dynamic effect

The Rest of the PaperThe Rest of the Paper

• Formalism:– Synthesis of previous work on dependent types and

singleton kinds

• Fully expressive higher-order functors– Via “static” module equivalence

• Decidable typechecking algorithm• Avoidance problem

– Restrict type theory (as little as possible) to avoid it

– Unrestricted language definable by elaboration

ConclusionConclusion

• Future Work:– Recursive modules– Using monads instead of total/partial

• We’ve provided a framework for understanding:– Alternative module system designs– Semantics of abstraction, via a framework of

module-level effects