Update 20.04.2018: added Polyglot Maven

Scala has a rich ecosystem and active community producing a lot of useful libraries. So much so that, sometimes it is not easy as a newcomer to decide which library to pick for a given task (this is the case for example when it comes to database access and JSON handling). In this article we are going to cover another domain in which there is an increasing number of alternatives: build tools. At the time of writing this article in April 2018 we have at our disposal:

sbt
cbt
mill
fury
maven
polyglot maven
gradle
ant (to be written)
bazel (to be written)
pants (to be written)
make (to be written)

Let’s take a bit of time and walk through each of these options and see how they work

sbt

sbt, shorthand for sbt, is Scala’s first (or so I think) and most well-known build tool. It is, by now, solid, reliable and has a large set of plugins and integrations.

Installation

sbt is available via Homebrew on on Mac, MSI on Windows and packages on linux (Debian or RedHat). On Mac, you’d install it using

brew install sbt@1

Creating a new project

New projects can be created using Giter8 templates. For a simple project you can use the scala-seed template:

~/workspace > sbt new scala/scala-seed.g8
[info] Loading settings from idea.sbt ...
[info] Loading global plugins from /Users/manu/.sbt/1.0/plugins
[info] Updating {file:/Users/manu/.sbt/1.0/plugins/}global-plugins...
[info] Done updating.
[info] Set current project to workspace (in build file:/Users/manu/workspace/)

A minimal Scala project.

name [Scala Seed Project]: sbt-test

Template applied in ./sbt-test

At this point, the project structure is:

.
|-- build.sbt
|-- project
|   |-- Dependencies.scala
|&nbsp;&nbsp; |-- build.properties
|-- src
    |-- main
    ?&nbsp;&nbsp; |-- scala
    |&nbsp;&nbsp;     |-- example
    |&nbsp;&nbsp;         |-- Hello.scala
    |-- test
        |-- scala
            |-- example
                |-- HelloSpec.scala

8 directories, 5 files

Defining the build

sbt requires a build.sbt file located at the root of the project to work. Additional build definitions can be defined in Scala files in the project directory. This is what we get from the template:

~/workspace/sbt-test > cat build.sbt
import Dependencies._

lazy val root = (project in file(".")).
  settings(
    inThisBuild(List(
      organization := "com.example",
      scalaVersion := "2.12.4",
      version      := "0.1.0-SNAPSHOT"
    )),
    name := "sbt-test",
    libraryDependencies += scalaTest % Test
  )

Running the project

There is two ways to working with sbt. You can run a task directly on the command-line shell, or enter the sbt shell. Typically you’ll want to run the tasks from within the sbt shell, because that’s the fastest (starting the shell takes a while). You can start the shell simply by running sbt:

~/workspace/sbt-test > sbt
[info] Loading settings from idea.sbt ...
[info] Loading global plugins from /Users/manu/.sbt/1.0/plugins
[info] Loading project definition from /Users/manu/workspace/sbt-test/project
[info] Loading settings from build.sbt ...
[info] Set current project to sbt-test (in build file:/Users/manu/workspace/sbt-test/)
[info] sbt server started at local:///Users/manu/.sbt/1.0/server/0591eb097678bdf5725f/sock
sbt:sbt-test>

At this point you can execute tasks (press tab for auto-completion):

sbt:sbt-test> run
[info] Packaging /Users/manu/workspace/sbt-test/target/scala-2.12/sbt-test_2.12-0.1.0-SNAPSHOT.jar ...
[info] Done packaging.
[info] Running example.Hello
hello
[success] Total time: 0 s, completed Apr 19, 2018 9:14:28 AM

Adding dependencies

Dependencies are expressed using sbt’s DSL. As projects grow, it is a good idea to keep the dependencies structured, as the template hints at:

~/workspace/sbt-test > cat project/Dependencies.scala
import sbt._

object Dependencies {
  lazy val scalaTest = "org.scalatest" %% "scalatest" % "3.0.5"
}

The %% notation is used for Scala dependencies and makes sure to select the correct Scala version of a dependency. For normal maven / ivy dependencies, a single % is used.

You can then add the dependency using the libraryDependencies setting:

libraryDependencies += scalaTest % Test

The last % in this example defines the dependency scope - in the case of ScalaTest, we have a test dependency. For most dependencies you’ll want to leave this blank.

Creating custom tasks

sbt defines its own task definition, resolution and execution layer. Rather than attempting at explaining this I’ll redirect you to James Roper’s excellent article on sbt.

To define a task in sbt, we first need to define a task key:

1

val sampleStringTask = taskKey[String]("A sample string task.")

This is a bit like an interface - it tells us the name of the task, its type (this task will return a String) and also lets us define a description.

We then need to implement this task:

1
2
3
4
5


lazy val root = (project in file(".")).
  settings(
    // ...
    sampleStringTask := "Hello, world"
  )

This is a short example and only gives you a short glimpse at how things work. For more background I invite you to check out the documentation.

Documentation, plugins, community

sbt has a fairly complete documentation, a strong community and a myriad of plugins (the previous link only shows the plugins that are hosted in the general sbt Github organization, but there are many more out there).

cbt

cbt, shorthand for Chris' Build Tool, is the result of Christopher Vogt having had enough of using sbt, but not quite enough so for choosing an entirely new name. It is a build orchestration tool aiming at using the Scala language and only a few concepts in order to create builds. Unlike sbt, it maps task execution to JVM invocations instead of adding its own layer in between.

Installation

To install cbt you have to clone the cbt repository and can then add the directory to your path:

git clone https://github.com/cvogt/cbt.git

At the first execution, it compiles itself and also nags about being faster when installing nailgun. To install nailgun, run:

brew install nailgun

Creating a new project

Now that we’re all set, we can let cbt create a new project for us:

~/workspace > mkdir cbt-test
~/workspace > cd cbt-test
~/cbt-test > cbt tools createMain 
Created Main.scala
()

cbt-test > cat Main.scala 
package cbt_test
object Main{
  def main( args: Array[String] ): Unit = {
    println( Console.GREEN ++ "Hello World" ++ Console.RESET )
  }
}

Defining the build

Without a build file, cbt will use default build settings. It is possible to let it create a new build file:

~/cbt-test > cbt tools createBuild
Created build/build.scala
()

~/cbt-test > cat build/build.scala
package cbt_test_build
import cbt._
class Build(val context: Context) extends BaseBuild{
  override def dependencies = (
    super.dependencies ++ // don't forget super.dependencies here for scala-library, etc.
    Seq(
      // source dependency
      // DirectoryDependency( projectDirectory ++ "/subProject" )
    ) ++
    // pick resolvers explicitly for individual dependencies (and their transitive dependencies)
    Resolver( mavenCentral, sonatypeReleases ).bind(
      // CBT-style Scala dependencies
      // ScalaDependency( "com.lihaoyi", "ammonite-ops", "0.5.5" )
      // MavenDependency( "com.lihaoyi", "ammonite-ops_2.11", "0.5.5" )

      // SBT-style dependencies
      // "com.lihaoyi" %% "ammonite-ops" % "0.5.5"
      // "com.lihaoyi" % "ammonite-ops_2.11" % "0.5.5"
    )
  )
}

At this point, this is the structure of our project:

.
|-- Main.scala
|-- build
    |-- build.scala

1 directory, 2 files

Running the project

Unlike sbt, cbt doesn’t have a console. You simply run the tasks from the common shell:

~/cbt-test > cbt run
Compiling to /home/manu/workspace/cbt-test/target/scala-2.11/classes
[warn] Pruning sources from previous analysis, due to incompatible CompileSetup.
[info] Compiling 1 Scala source to /home/manu/workspace/cbt-test/target/scala-2.11/classes...
[info] Compile success at Apr 19, 2018 9:13:42 AM [1.186s]
Hello World

Adding dependencies

Dependencies are expressed using the ScalaDependency or MavenDependency constructs like shown in the generated build file above. Those dependencies need to be nested under a Resolver:

1
2
3
4
5
6
7
8
9


Resolver( mavenCentral, sonatypeReleases ).bind(
  // CBT-style Scala dependencies
  // ScalaDependency( "com.lihaoyi", "ammonite-ops", "0.5.5" )
  // MavenDependency( "com.lihaoyi", "ammonite-ops_2.11", "0.5.5" )

  // SBT-style dependencies
  // "com.lihaoyi" %% "ammonite-ops" % "0.5.5"
  // "com.lihaoyi" % "ammonite-ops_2.11" % "0.5.5"
)

What’s interesting is that the sbt DSL is also supported which makes for easy copy-pasting of dependencies from many README files out there.

Creating custom tasks

Creating new taks is rather straight-forward, you only need to create a new function in the build file, like for example:

1

def theAnswer = 42

And then call your task with cbt:

cbt-test cbt theAnswer
Compiling to /Users/manu/workspace/cbt-test/build/target/scala-2.11/classes
[warn] Pruning sources from previous analysis, due to incompatible CompileSetup.
[info] Compiling 1 Scala source to /Users/manu/workspace/cbt-test/build/target/scala-2.11/classes...
[info] Compile success at Apr 19, 2018 9:36:00 AM [0.745s]
42

Documentation, plugins, community

cbt has a documentation that helps you get started with it, a few plugins and a small community (check out the Gitter channel).

Mill

Mill is the only Scala build tool written from the ground up in x86 assembly. It’s fast. You can read Li Haoyi’s introductory post about Mill here.

Edit 23.04.2018: okay, I’m getting too many comments on this. So just to be clear: Mill is not written in x86 assembly. The comment came to me after trying it out, given it has a fast feel to it.

Installation

Mill has a brew package on OS X, an AUR package for arch linux and can be downloaded directly as bat file for Windows. For OS X we’ll just run:

brew install mill

Creating a new project

Mill doesn’t have (yet) a mechanism to generate a new project, instead you can download a sample project.

Defining the build

The build is defined in a file at the root of the
This gives you the following build definition:

1
2
3
4
5
6
7


// build.sc
import mill._
import mill.scalalib._

object foo extends ScalaModule {
  def scalaVersion = "2.12.4"
}

The project structure is:

.
|-- build.sc
|-- foo
    |-- src
        |-- foo
            |-- Example.scala

3 directories, 2 files

What you might notice right away is that this structure is not following the now almost-standard maven project structure. Mill doesn’t impose a particular structure on the projects, instead it allows for quite some flexibility in regards to how modules are layed out. There’s a few common project layouts documented, amongst which an sbt-compatible layout.

Running the project

The common syntax for running project tasks folllows the pattern module.task:

~/workspace/mill-test > mill foo.run
Compiling (synthetic)/ammonite/predef/interpBridge.sc
Compiling /Users/manu/workspace/mill-test/build.sc
[36/36] foo.run
Hello World

Adding dependencies

Mill uses a DSL for adding dependencies to a project:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


import mill._
import mill.scalalib._
object foo extends ScalaModule {
  def scalaVersion = "2.12.4"
  def ivyDeps = Agg(
    ivy"com.lihaoyi::upickle:0.5.1",
    ivy"com.lihaoyi::pprint:0.5.2",
    ivy"com.lihaoyi::fansi:0.2.4",
    ivy"org.scala-lang:scala-reflect:${scalaVersion()}"
  )
}

Similar to sbt’s %% notation, Scala dependencies are expressed using the :: notation. It’d be nice, that is, if like cbt, Mill did also have a compatible way of accepting the sbt format for the sake of copy-paste.

One thing I’d like to point out here is that mill uses coursier to fetch dependencies. Coursier is quite a bit faster than sbt’s dependency resolution which has personally been somwhat painful to use over the years (it has gotten much better, but used to drive me insane). For example, it supports downloading multiple artifacts in parallel and has no global lock (the infamous Waiting for ~/.ivy2/.sbt.ivy.lock to be available which you are guaranteed to get if you are working with both sbt and IntelliJ at the same time.

Creating custom tasks

Mill defines its own task graph abstraction to handle task definition, ordering and cache. Calling one task from another establishes a dependency, which makes it quite natural to understand. There are 3 types of tasks:

targets: to define where outputs get generated / compiled / assembled
sources: to define where code comes from
commands: to define things to do

Let’s define a task:

1
2
3


def theAnswer() = T.command {
  println(42)
}

Which yields:

~/workspace/mill-test > mill foo.theAnswer
Compiling /Users/manu/workspace/mill-test/build.sc
[1/1] foo.theAnswer
42

Documentation, plugins, community

Mill has a complete documentation site and an active community.

Plugins aren’t called plugins, but modules. There doesn’t seem to be a central module repository yet at the time of writing this article or maybe I missed it.

Fury

There is a new build tool in the making:

doing the unthinkable: building a new Scala build tool! @propensive announcing ‘fury’ at #ScalaSphere pic.twitter.com/ZojHTdRUIR— Iulian Dragos (@jaguarul) April 17, 2018

Not having attended ScalaSphere I can only speculate where the name comes from, so I’ll be so free as to hypothetize that it has to something to do with Marvel’s Nick Fury.

This seems to be further supported by the following tweet:

Here's the freshest of Scala memes, straight from @ScalaSphere!

(I'm sorry Jon) pic.twitter.com/RFUKgIIOUv— Jakub Kozłowski λ❄ (@kubukoz) April 17, 2018

I’m looking forward to adding the review of the tool here once it is available!

Maven

Maven is… well, what do you want me to say. It’s maven.

If you have to, you can use maven to build scala projects using the scala-maven-plugin (previously maven-scala-plugin):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20


<project>
    ...
    <build>
        <sourceDirectory>src/main/scala</sourceDirectory>
        <testSourceDirectory>src/test/scala</testSourceDirectory>
        ...
        <plugins>
            ...
            <plugin>
                <groupId>net.alchim31.maven</groupId>
                <artifactId>scala-maven-plugin</artifactId>
                <version>3.3.2</version>
                ... (see other usage or goals for details) ...
            </plugin>
            ...
        </plugins>
        ...
    </build>
    ...
</project>

From there on, it is plain maven, therefore I won’t go into any details as to how to work with the project.

                Polyglot Maven

                If you really need to be using maven but would like to use Scala for your build definition, Polyglot for Maven allows you to do so. Here's an example of how it looks like:

                import org.sonatype.maven.polyglot.scala.model._ import scala.collection.immutable.Seq

              Model(
  "org.exampledriven" % "maven-polyglot-scala-example" % "1.0-SNAPSHOT",
  dependencies = Seq(
    "io.takari.polyglot" % "polyglot-scala" % "0.1.10" % "test"
  ),
  build = Build(
    tasks = Seq(Task("someTaskId", "verify") { ec =>
      println(s"\nbaseDir: \n${ec.basedir}")
    })
  ),
  modelVersion = "4.0.0"
)

```

Gradle

Just to mention this possibility as well, there’s a Gradle Scala plugin if you want to use Gradle for your Scala project build.

Great, now which one to pick?

I find this one more difficult to answer than for the database and JSON topics.

Clearly, sbt is well-established and has a large amount of plugins to integrate with many, many things out there. But if you are a newcomer and need to do something for which there isn’t a well-documented plugin or just a documented way of doing things, things can become quite frustrating. And this isn’t only true for newcomers - you’ll find quite a few experienced Scala developers that are frustrated with sbt. See, to some extent it is possible to just get by using sbt without really understanding it and relying on copy-pasting things from Stack Overflow or from other build definitions. It’s when you need to do this one thing that’s a bit different and not so well documented that it gets frustrating. Or maybe things are well-documented, but you don’t have the conceptual model of sbt in your mind (and don’t find a good way to acquire it), so you don’t know how to proceed. Now, I’m not saying “don’t use sbt”. What I’m trying to say is that sbt is hard and that, as a newcomer, you should approach it as something that is hard, which will help a lot with regards to your personal frustration and willingness to learn (because, if it is simple, you shouldn’t need to have to invest time in learning it and it should just work, right?).

And therefore I completely understand that there are a few new alternatives coming up, which is something I feel happens in the Scala community a bit too easily and often - rather than trying to improve existing tools, people go ahead and create new ones. Except that, well, in this case and with its current design, I don’t see any easy way of making sbt radically easier to grasp.

Now, all that rambling doesn’t help. What to pick? Well, here’s what I do:

for projects at clients, unless there’s a strong drive from the client, I’d stick to sbt, because that’s a safe choice

for my own projects, should I one day have time for them, I think I’ll opt for Mill at the moment. It uses Coursier (remember, I wear the scars of waiting about 4 minutes simply for dependencies to resolve), is well-documented, under active development and even if at the moment there isn’t a clear plugin repository I do get the impression that this is just a matter of time

Happy building!

A quick tour of build tools in Scala

Contents

sbt

Installation

Creating a new project

Defining the build

Running the project

Adding dependencies

Creating custom tasks

Documentation, plugins, community

cbt

Installation

Creating a new project

Defining the build

Running the project

Adding dependencies

Creating custom tasks

Documentation, plugins, community

Mill

Installation

Creating a new project

Defining the build

Running the project

Adding dependencies

Creating custom tasks

Documentation, plugins, community

Fury

Maven

Gradle

Great, now which one to pick?