Rust All Hands Winter 2011

The Rust Programming LanguagePatrick WaltonAll Hands Winter 2011

Overview

• What is Rust?

• How has the Rust design evolved over the past year?

• How far have we come in implementation?

• What are we going to use Rust for?

What is Rust?use std;import std::{ int, vec };fn main(grades : [str]) { let results = vec::map({ |s| if int::parse(s) >= 70 { “Pass” } else { “Fail” } }, grades); log results;}

What is Rust?

• Rust is a new systems programming language designed for safety, concurrency, and speed.

• It was originally conceived by Graydon Hoare and is now developed by a team in Mozilla Research and the community.

What is Rust? — Safety

• Rust has no null pointers. The typical replacement for a nullable pointer is std::option<T>.

• Rust is memory-safe, outside of designated unsafe blocks, which can be easily audited.

• Rust features typestate, an analysis that allows the compiler to track predicates that you define.

What is Rust?—Typestate

• Typestate isn’t new—it’s in Java!

public static void main() {String s;if (new Date().getHours() < 12)

s = “Good morning!”System.out.println(s); // Error: s may be uninitialized

}


• The same initialization checking takes place in Rust through the typestate system and the so-called init predicate. Each value must hold the init predicate (i.e. be initialized) before use.

fn main() {let s : str;if date::localtime(date::now()).hours < 12u {

s = “Good morning!”}log s; // Error: predicate ‘init’ does not hold

}


• Beyond this, Rust allows you to define your own predicates.

• Functions can demand that certain predicates be true before calling them is allowed. You can also use values with predicates as first-class types, such as int|prime.

• You use the check statement to make the Rust compiler aware of assertions that you’re performing. The assertion is checked at runtime, and the program aborts if the assertion fails.


• Suppose we have these declarations:

pred in_range(array : [int], start : uint, end : uint) {start < end && end < vec::len(array)

}fn slice(array : [int], start : uint, end : uint) -> [int] : in_range(array, start, end) { ... }


• Then we can’t call slice in the normal way:

fn main() {let v = [ 1, 2, 3, 4, 5 ];log v.slice(2, 3); // error: predicate //‘in_range(v, 2, 3)’ does // not hold

}


• But we can if we use a check statement:

fn main() {let v = [ 1, 2, 3, 4, 5 ];check in_range(v, 2, 3);log v.slice(2, 3); // OK

}


• Why is this useful?

• The Rust compiler remembers the assertions that you’ve performed, so that you don’t have to perform them repeatedly.

• With typestate, assertions are in the function signature, which helps make functions self-documenting.

What is Rust?—Concurrency

• Rust is designed for concurrency on a large scale, following the principles of Erlang.

• You can spawn thousands of tasks in the same process.

• Under the hood, the Rust scheduler spawns one or two OS threads per CPU and schedules your tasks appropriately.


fn do_something_expensive() -> bigint {ret factorial(99999999);

}fn main() { let task = spawn(do_something_expensive); log join(task);}


• While they’re running, tasks can communicate via channels and ports.

• Ports and channels are typed; e.g. port<int>, chan<dom_node>.

• Ports are the receiving endpoint; channels are the sending endpoint. Mnemonic: bodies of water.

• Multiple channels can go to the same port.


fn something_expensive(ch : chan<bignum>) {send(ch, factorial(99999999));

}fn main() {

let p = port();spawn(bind something_expensive(chan(p)));log recv(p);

}


• Rust ensures that there is no shared state.

• There is no need for mutex locks, condition variables, atomic operations, or concurrent garbage collection.

• All unique pointers (~) are allocated in a shared heap and move from task to task by simply copying a pointer.

• All boxed pointers (@) are allocated in task-local heaps and never move.


• We enforce these concurrency-related invariants through our type system.

• Unique pointers (denoted by ~) are the only pointers that can be sent.

• Once you send a unique pointer, you lose access to it.

• This is enforced through typestate; the unique pointer, once moved, loses its init predicate.

What is Rust?—Speed

• We use the LLVM compiler infrastructure, which is used by the Clang C++ compiler, for the self-hosted compiler.

• We benefit from all of LLVM’s optimization passes.


• We strive for predictable runtime performance with a minimum of overhead.

• Performance should be comparable to idiomatic C++; that is, C++ with use of the STL and avoiding highly unsafe constructs.

• We also have an unsafe sublanguage that allows direct pointer manipulation. It’s restricted to blocks and functions marked unsafe.


• We support three type layers to make memory management simple and fast.

• Interior types (denoted with no sigil) are stored on the stack.

• Unique types (denoted with ~) have ownership semantics. When the pointer goes out of scope, the object it’s pointing to is destroyed.

• Boxed types (denoted with @) are garbage-collected. Multiple boxes can point to the same data.


• All together:

fn main() { let a = ~3; // Unique pointer to number let b = a; // Copies a let c = @3; // Box of number let d = c; // c & d point to the same value} // a and b destroyed; c and d will be GC’d

Language changes

• A year ago, we announced Rust at the Mozilla Summit.

• Since then, we’ve made a large number of changes to the language syntax and semantics.

• Writing the Rust compiler in Rust helped us to quickly find and fix the painful areas of the language.

Language changes—Syntax

• One year ago:

fn f(&option.t[str] s) { auto r; alt (s) { case (some(?ss)) { r = rec(x=123, y=ss); } case (none) { fail; } }}


• Today:

fn f(s : option::t<str>) { let r = alt s { some(ss) { { x: 123, y: ss } } none. { fail } };}


• Small changes:

• We’ve removed parentheses around if, alt, while, etc.

• Pattern matching syntax has been tweaked. Switch statements (alt) no longer require case.

• Types now go after variable bindings: let int x is now written let x : int.

Language changes—Closures

• Instead of simply supporting a bind form, we now essentially have full-featured closures.

• Closures are written as Ruby-style blocks:

vec::eachi([ 1, 2, 3 ], { |x, i| // Parameters log “Element #“, i, “ is “, x;});

Language changes—Sharing

• When Rust was initially designed, only immutable data structures could be sent over channels, like Erlang.

• But we still needed copying to avoid global garbage collection.

• What we really need is to avoid sharing, not immutability.

• The immutability restriction is now the ownership restriction—you can only send data you own, and after you send it you lose that ownership.

Language changes—Macros

• One year ago, macros were more like compiler plugins—they were DLLs loaded by the rustc compiler that could manipulate its internal code representation.

• These kinds of macros are still supported, but we wanted macros to be usable by people other than compiler hackers.

• So now we have macro-by-example, which are pattern-matching macros like those of Scheme.

Current statusHow far are we from achieving our goals of safety, concurrency,

and speed?

Current status—Safety

• We have a type-safe, memory-safe language with a self-hosted compiler.

• We have hundreds of unit tests and a testing framework.

• We’re missing garbage collection and stack unwinding.

• The compiler isn’t yet production-quality. Crashes are uncommon but still happen.

Current status—Concurrency

• We have a working concurrency system that operates similarly to Erlang and Grand Central Dispatch.

• Thousands of simultaneous tasks are supported.

• We’re currently missing growable stacks (to prevent address space exhaustion), unique pointers and closures (for fast data sharing), and OS process isolation.

Current status—Speed

• Our compiler benefits from LLVM’s optimizations and highly-tuned code generation.

• On simple numeric kernels, we generally perform well.

• We’re currently slower than we’d like to be on generic code (type-passing), memory management (too much copying), and cross-crate calls (no inlining).

Roadmap

• A tentative roadmap:

• Rust 0.1 (soon!): Garbage collection/cycle collection; stack unwinding; unique pointers; unique closures; stack growth.

• Future: Rewritten backend; cross-crate function inlining; improved LLVM garbage collection; iterators; fleshed-out standard libraries; package management.

Project ServoOr: “So why are we doing this?”

Servo Project

• Rust is a critical piece of the Servo project, which is a project to develop a parallel browser engine.

• We don’t know how, or whether, we will ship this engine to end users at the moment; it’s very much a research project. It might be a new product, become a part of Gecko/Firefox, or remain purely a research platform.

• Nevertheless, we want our code to be production quality.

Servo Project—Architecture

Platform APIsC++

Servo engineRust

dom.jsJavaScript

Web contentHTML + JavaScript

Servo Project—Architecture

• Servo is not planned to feature a cross-language component model.

• Instead, all JavaScript–Rust communication is performed via the Rust message passing system.

• The single-threaded parts of the browser will run in JavaScript and communicate asynchronously with the highly parallel Servo engine written in Rust.

Servo Project—Components

• These components are planned to be written in Rust:

• A parallel HTML/JavaScript/SVG/CSS parser.

• A layout engine that performs parallel layout, including CSS selector matching, box reflows, etc.


• These components are planned to remain in C++ for the first few iteration:

• The SpiderMonkey JavaScript engine.

• The graphics rendering engine “Azure”.


• These components will be written in JavaScript and HTML:

• The DOM will be dom.js, in order to avoid the problems of XPCOM (cross-language cycles, SpiderMonkey optimization hazards, etc.)

• The UI will be all HTML and JavaScript.

Servo Project—Getting Involved

• We’re still in the early stages of development, but you can help!

• dom.js is actively being worked on and we’d love to have your help: http://github.com/andreasgal/dom.js

• If you have ideas along the lines of “what I would do if I could start over from scratch”, particularly if you’re a platform hacker, we’d like to know!

http://github.com/andreasgal/dom.js

http://github.com/andreasgal/dom.js

Thanks

• The Rust team

• Brian Anderson (brson)

• Tim Chevalier (tjc)

• Marijn Haverbeke (marijn)

• Dave Herman (dherman, @littlecalculist)

• Graydon Hoare (graydon, @graydon_moz)

Thanks

• And the Rust interns:

• Roy Frostig (froystig)

• Eric Holk (eholk, @theinedibleholk)

• Lindsey Kuper (lkuper, @lindsey)

• Paul Stansifer (pauls, @paulstansifer)

• Michael Sullivan (sully)

Questions?

@rustlangirc.mozilla.org #rust

http://github.com/graydon/rust



Rust All Hands Winter 2011

Technology

concurrency rust

rust design

safety rust

rust features typestate

rust scheduler spawns

unique pointers

uint int

fn slicearray