Simple Build Tool

(Simple ?) Build ToolDavid Galichet (@Xebia)

Jonathan Winandy

mercredi 23 novembre 2011

Schedule

• SBT basics• installation and project setup,• SBT usages,• dependency management• defining and using scopes, settings and tasks,• cross building• SBT demo• using SBT• using plugins• writing plugin


A build system for Scala & Java Applications

• compile Scala and Java code source• create Artifacts• manage dependencies (ivy)• run tests• extensible architecture (with plugins)• integrated with Eclipse & Intellij• plugin with Hudson/Jenkins• ...


More than a build system

• run your applications,• launch scala REPL,• triggered execution,• ...


SBT History

• Created by Mark Harrah• First popular branch until 0.7.7• A new popular (and incompatible) branch from 0.9 →

actually 0.11.1 (aka. XSBT)


SBT Setup

• Download the launch-sbt.jar (rename it xsbt-launch.jar if version >= 0.9.x)

• Create a launch script (xsbt) available in your PATH :java -Dfile.encoding=UTF8 -Xmx1536M -Xss1M -XX:+CMSClassUnloadingEnabled -XX:MaxPermSize=256m -jar `dirname $0`/xsbt-launch.jar "$@"


SBT project anatomy/build.sbtsrc/ main/ scala/ java/ resources/ test/ scala/ java/ resources/project/ Build.scala plugins.sbt project/ target/ ... target/ ...target/ ...

Add to your .gitignore !


Using SBT

% xsbt[info] Loading project definition from ...test/project[info] Updating {file:/...test/project/}default-a285df...[info] Done updating.[info] Set current project to Test (in build file:...test/)>


Creating a simple project

• create project directory,• create the src/ directory hierarchy (optional),• create a build.sbt in project root.

• Or use the interactive mode !> set name := "test"> session save

This will automatically create the build.sbt.


First build definition

name := "test"

version := "0.1-SNAPSHOT"

scalaVersion := "2.9.1"

libraryDependencies += "org.specs2" %% "specs2" % "1.6.1" % "test"


SBT basics

• name, version ... are Keys defining settings,• settings are typed (String, Seq[String], Int, ModuleId ...)• := is an assignation operator (override previous value)• += is a modification operator (add a value to a sequence)


ModuleID

"org.specs2" %% "specs2" % "1.6.1" % "test" ============ ======== ======= ====== groupId artifact version configuration

String is implicitly converted to finally create a ModuleID.


Common commands

• reload• clean• compile• test• console• console-project• publish• show• set• inspect• project• ...


Triggered execution

• use ~ to trigger task execution when code change (compile or test for example),• SBT uses incremental compilation → recompile only what is

needed.


Manual dependency management

All jar files in lib directory will be added to the classpath so they will be available when using compile, test, run, console ...


Automatic dependency management

Dependencies are added to settings :libraryDependencies += groupID % artifactID % revision % configuration

where configuration (compile, test, run ...) is optional.

We can also encounter : libraryDependencies += groupID %% artifactID % revision

%% implies that SBT will use the right version according to project scalaVersion (for example specs2_2.9.1)


Dependency management - Resolvers

add a dependency resolver :resolvers += "Repository name" at "http://the-repository/releases"

add local maven repository to resolvers :resolvers += "Local Mvn Repository" at "file://"+Path.userHome.absolutePath+"/.m2/repository"

dependency explicit resolver :libraryDependencies += "slinky" % "slinky" % "2.1" from "http://slinky2.googlecode.com/svn/artifacts/2.1/slinky.jar"

/!\ →use with caution, the explicit resolver doesn't appear in the pom.xml when the artifact is published.


http://the-repository/releases




http://slinky2.googlecode.com/svn/artifacts/2.1/slinky.jar




Dependency management - extra configuration

extra configuration : • intransitive() → disable transitivity for this

dependency,• classifier(..) → add a classifier (ex : "jdk5"), • exclude(groupId,artifactName) → exclude specified

artefact (since 0.11.1),• excludeAll(..) → exclude based on exclusion rules

(since 0.11.1),• ...

It's also possible to add Ivy configuration directly : ivyXML := "<ivysettings>...</ivysettings>


Publish artifacts

To publish artifact locally (in ~/.ivy local repository) :> publish-local

To define a nexus repository (and publish with publish) :publishTo := Some("Scala Tools Nexus" at "http://mydomain.org/content/repositories/releases/")

or an arbitrary location :publishTo := Some(Resolver.file("file", new File( "path/to/my/maven-repo/releases" )) )

To define nexus credentials :credentials += Credentials(Path.userHome / ".ivy2" / ".credentials")


http://nexus.scala-tools.org/content/repositories/releases/




Cross building

To define all scala versions that we want to build for :crossScalaVersions := Seq("2.8.0", "2.8.1", "2.9.1")

Then prefix the action we want to run with + :> + package> + publish

If some dependencies versions depends on scala version :libraryDependencies <+= (scalaVersion) { sv =>

val vMap = Map("2.8.1" -> "0.5.2", "2.9.1" -> "0.6.3")val v = vMap.getOrElse(sv, error("Unsupported ..."))"org.scala" %% "mylib" % v

}

We can also use ++ <version> to temporarily switch version.


Full configuration

Defined in project/Build.scala :

import sbt._import Keys._

object Test extends Build { lazy val root = Project("root", file(".")) .settings( name := "Test", version := "0.1-SNAPSHOT", ... )}


Multi-projects build

• We can define a multi-projects in a full build description :object Test extends Build { lazy val root = Project(id = "root", base = file(".")) aggregate(foo, bar)

lazy val foo = Project(id = "test-foo", base = file("foo")) dependsOn(bar)

lazy val bar = Project(id = "test-bar", base = file("bar"))}

• Settings in all .sbt project description (i.e. foo/build.sbt) will form the project definition and be scoped to the project,• project/*.scala files in sub-project will be ignored,• projects list projects and project <name> change project.


Scopes

We can define settings and use tasks on multiple axis :• on full build,• by project,• by configuration,• by task.


Define scope

Setting defined globally :name := "test"

Setting restricted on specified configuration :name in (Compile) := "test compile"

Inspect :> show name[info] test

> show compile:name[info] test compile


Inspect scope

> inspect name[info] Setting: java.lang.String = Test1[info] Description:[info] Project name.[info] Provided by:[info] {file:/...test/}default-914d18/*:name...

{<build-uri>}<project-id>/config:key(for task-key)


Projects scope

• On a multi-project definition, some Settings are defined in each project definition and assigned to project Scope. For example :

> show version[info] test-foo/*:version[info] 0.7[info] test-bar/*:version[info] 0.9[info] root/*:version[info] 0.5


Build scope

To add a setting on build scope in build.sbt :

myKey in ThisBuild := value

and in Build.scala (out of project settings definition) :

override val settings += ( myKey := value )

then inspect : {file:/home/hp/checkout/hello/}/*:myKey


Custom configuration

lazy val RunDebug = config("debug") extend(Runtime)

lazy val root = Project("root", file(".")) .configs( RunDebug ) .settings( inConfig(RunDebug)(Defaults.configTasks):_* ) .settings( ... javaOptions in RunDebug ++= Seq("-Xdebug", "-Xrunjdwp:...") ... )

then use this configuration : debug:run


SBT settings

• defined by typed keys (SettingKey[T] ...),• keys are defined in sbt.Keys (or in plugin, project, build

definition...),• Keys have assignation methods that returns a Setting[T],• each Setting[T] defines a transformation of SBT internal

build definition Map.

For example : name := "test"

defines a transformation that returns the previous settings Map with a new entry.


Kinds of Settings

The three kinds of Keys :• SettingKey[T] → the Setting is evaluated once,

• TaskKey[T] → the Task is evaluated on each use; Can create side effects,

• InputKey[T] → similar to Tasks but evaluation depends on command line arguments.

When assignation method (:=, ~=, <<= ...) are used on a :• SettingKey[T], it returns a Setting[T],• TaskKey[T], it returns a Setting[Task[T]],• InputKey[T], it returns a Setting[InputTask[T]].


Modify settings

• := is used to replace the setting value :name := "test"

• += is used to add a value to a setting of type Seq[T] :libraryDependencies += "org.specs2" %% "specs2" % "1.6.1" % "test"

• ++= is used to add some values to a setting of type Seq[T] :libraryDependencies ++= Seq("se.scalablesolutions.akka" % "akka-actor" % "1.2", "se.scalablesolutions.akka" % "akka-remote" % "1.2")


Modify settings - transform a value

Sometimes we want to modify the value of an existing.

There's an operator for that : name ~= { name => name.toUpperCase }

or more succinctly : name ~= { _.toUpperCase }


Modify settings - use dependency

We want to compute a value based on other value(s) : organization <<= name(_.toUpperCase)

that is equivalent to : organization <<= name.apply { n => n.toUpperCase }

where SettingKey[T] <<= method is defined as : <<=(app:Initialize[T]):Setting[T]

Setting[T] defines the apply method : apply[U](f: T => U):Initialize[U]

apply transforms a Setting[T] to a Initialize[U].


Modify settings - use dependencies

In case we want to rely on many dependencies :name <<= (name, version)( _ + "-" + _ )

that is equivalent to :name <<= (name, version).apply { (n, v) =>

n + "-" + v }

Tuples (Initialize[T1],..., Initialize[T9]) are implicitly converted to obtain the apply method.


Modify settings - use dependencies

Add a value with dependencies to a Seq[File] :cleanFiles <+= (name) { n => file(.) / (n + ".log") }

Add some values with dependencies to a Seq[File] :unmanagedJars in Compile <++= baseDirectory map {

base => ((base / "myLibs") ** "*.jar").classpath}


Modify settings - tasks with dependencies

Setting[S] apply method returns a Initialize[T] but for a TaskKey[T], <<= method expects a Initialize[Task[T]]

The Setting[S] method map comes to the rescue : map[T](f: S => T):Initialize[Task[T]]

We can set a SettingKey to a TaskKey : taskKey <<= settingKey map identity

For multiple dependencies : watchSources <+= (baseDirectory, name) map{(dir, n) => dir / "conf" / (n + ".properties") }


Settings and tasks definition

A setting key definition sample: val scalaVersion = SettingKey[String]("scala-version", "The version of Scala used for building.")

A task key definition sample: val clean = TaskKey[Unit]("clean", "Deletes files produced by the build, such as generated sources, compiled classes, and task caches.")

Here the clean task returns Unit when executed but can have side effects (produced artefacts are deleted).

Most SBT tasks are defined in Default.scala.


Define your own tasks

Define a task that print and returns the current time : val time = TaskKey[Date]("time", "returns current time")

lazy val root = Project("test", file(".")).settings( time := { val now = new Date() println("%s".format(now)) now })

Usage : > time Wed Nov 16 13:55:38 CET 2011

Tasks unlike Settings are evaluated each time they are called.


Input tasks

• Similar to a Task but can take user input as parameter,• SBT provides a powerful input parsing system (based on scala

parser combinators) and easy tab completion feature,• Key defined in a way similar to SettingKey or TaskKey :

val release = InputKey[Unit]("release", "release version")

• Defining it in settings : release <<= InputTask(releaseParser)(releaseDef)

• Similar to a Command (a kind of tasks that is not defined in Settings and with no return value).


Input tasks - input parser

• Input parser sample :

val releaseParser:Initialize[State => Parser[String]] = (version) { (v:String) => { val ReleaseExtractor(vMaj, vMin, vFix) = v val major = token("major" ^^^ "%s.%s.%s".format(vMaj.toInt + 1, vMin.toInt, vFix.toInt)) val minor = token("minor" ^^^ "%s.%s.%s".format(vMaj.toInt, vMin.toInt + 1, vFix.toInt)) val fix = token("fix" ^^^ "%s.%s.%s".format(vMaj.toInt, vMin.toInt, vFix.toInt + 1)) (state:State) => Space ~> (major | minor | fix) }}


Input tasks - task implementation

• Task input implementation :

val releaseDef = (nextVersion:TaskKey[String]) => { (version, nextVersion) map { case (currentV, nextV) => println("next version : " + nextV) val result = ("git tag " + currentV).lines_!.collect { case s:String if s.contains("fatal") => s } if (result.mkString.isEmpty) println(result.mkString) else { println("Release tagged ! Next one is " + nextV.mkString) // ... } }


Settings prevalence rules

• Build and Project settings in .scala files,• User global settings in ~/.sbt/*.sbt,• Settings injected by plugins,• Settings from .sbt files in the project,• Settings from build definition project (i.e. project/

plugins.sbt)

Lowest

Highestprevalence


Inspect Settings - general informations

> inspect compile[info] Task: sbt.inc.Analysis[info] Description:[info] Compiles sources.[info] Provided by:[info] {file:/Users/.../test/}test/compile:compile...


Inspect Settings - dependencies

...[info] Dependencies:[info] compile:compile-inputs[info] compile:streams(for compile)[info] Reverse dependencies:[info] compile:products[info] compile:defined-sbt-plugins[info] compile:exported-products[info] compile:discovered-main-classes...


Inspect Settings - delegates

...[info] Delegates:[info] compile:compile[info] *:compile[info] {.}/compile:compile[info] {.}/*:compile[info] */compile:compile[info] */*:compile[info] Related:[info] test:compile[info] debug:compile


Extending SBT

• SBT can be extended using plugins,• Plugins are new Settings/Tasks added to SBT,• To add a plugin in the project or globally, add :

resolvers += Classpaths.typesafeResolveraddSbtPlugin("com.typesafe.sbteclipse" % "sbteclipse" % "1.4.0")

in your project/plugins.sbt or in ~/.sbt/plugins/build.sbt


What is a plugin ?

• A plugin is an SBT project added as a dependency to the build definition !• Recursive nature of SBT :

/build.sbtproject/ Build.scala plugins.sbt project/ Build.scala ...

• We can load build definition project with reload plugins and go back to project with reload return.

Project definition

Build definition


Enhance build definition project

• To use a specific library in your project/Build.scala, you can add the following in project/plugins.sbt (or project/project/Build.scala) :

libraryDependencies += "net.databinder" %% "dispatch-http" % "0.8.5"

• To test some build code snippets in a scala REPL :> console-project

this will load all build dependencies.


Some powerful APIs

• IO operations with Path API,• Invoking external process with process API,• Input parsers and tab-completion for Tasks and Commands,• Launcher to launch application without a local Scala

installation,• All the power of Scala API ...


Finally...

• Limited key concepts to understand,• A powerful API,• Easy access to scala ecosystem power,• Increasing number of plugins ...

Is Simple Build Tool Simple ?


Simple Build Tool

Technology

dependency

project denition

triggered

librarydependencies

resolvers

base file

project

scala