Version 0.2 has just landed!

First things first …

First, some bad news. Due to limitations of the current macros implementation in Scala (for more details see this discussion) in order to avoid occasional compilation errors it is necessary to add type ascriptions to the dependencies. This is a way of helping the type-checker that is invoked by the macro to figure out the types of the values which can be wired.

In practice it is good to always write the type ascription. For example:

1
2
3
4
5

trait MyModule {
   ...
   lazy val theUserFinder: UserFinder = wire[UserFinder]
   ...
}

This is a major inconvenience, but hopefully will get resolved some day. See the “Limitations” section in the README for more details.

Update 28/04/2013 As noted in the comments below, the recommendation for adding type ascriptions may be in fact relaxed by adding special-handling for declarations that are simple wire[X] expressions. So stay tuned for future releases for improvements :)

Also, it is worth noting that the type ascription may be a subtype of the wired type. This can be useful if you want to expose e.g. a trait that the wired class extends, instead of the full implementation.

… to the point

Now, to the good news. Especially when developing web applications, it is very useful to scope some of the dependencies, so that there’s a new instance for each request (but for each dependency usage, it is the same instance in one request), or that there’s a single instance per http session.

So far MacWire had two “built-in” scopes: singleton (declare the dependency as lazy val) and dependent (def). Now it is possible to create custom scopes, by implementing the Scope trait.

Scopes often seem “magical”; in traditional frameworks it is usually necessary to add an annotation to the class and then the dependency is automagically turned into a scoped one by the container.

In fact implementing a scoped bean is quite simple. We need two things. Firstly, instead of using instances directly, we need to use a proxy which delegates method calls to the “current instance”. What “current instance” means is scope-specific.

Secondly, we need a way to read a value from the scope, and store a value in the scope, if there’s no value yet. The two methods in the Scope trait correspond directly to the two cases outlined above.

Using a scope is quite straightforward; the scope of a dependency is declared in the wiring code, in the module. For example:

1
2
3
4
5

trait WebModule {
   lazy val loggedInUser: LoggedInUser = session(wire[LoggedInUser])
 
   def session: Scope
}

Note that the scope here is abstract (only the name suggests that it is a session scope). This has a couple of advantages. For testing, we can substitute a NoOpScope. Also, we can move the framework-integration code to the integration layer, or even provide a couple of implementations depending on the framework/container used.

MacWire comes with a Scope implementation targeted at synchronous web frameworks, ThreadLocalScope. The scope needs to be associated and disassociated with a scope storage. To implement a:

• request scope, we need a new empty storage for every request (associateWithEmptyStorage method)
• session scope, the storage (a Map) should be stored in the HttpSession (associate(Map) method)

This association can be done, for example, in a servlet filter:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

class ScopeFilter(sessionScope: ThreadLocalScope) extends Filter {
  def init(filterConfig: FilterConfig) {}
 
  def doFilter(request: ServletRequest, response: ServletResponse, chain: FilterChain) {
    val httpRequest = request.asInstanceOf[HttpServletRequest]
 
    val sessionScopeStorage = new ScopeStorage {
      def get(key: String) = Option(httpRequest.getSession.getAttribute(key))
      def set(key: String, value: Any) {
        httpRequest.getSession.setAttribute(key, value)
      }
    }
 
    sessionScope.withStorage(sessionScopeStorage) {
      chain.doFilter(request, response)
    }
  }
 
  def destroy() {}
}

Note also that it is trivial to use a scoped value e.g. in an asynchronous process, where no web request is available: simply associate the scope with any storage, can be a temporary map. In traditional web frameworks this usually requires some amount of hacking.

To sum up, to use a scope in MacWire you need to:

• declare a Scope in your module, use if for scoped dependencies
• provide an implementation of the scope, e.g. ThreadLocalScope
• associate the scope with a storage, e.g. in a servlet filter

You can see how this works in practice by browsing & running a Scalatra+MacWire example application. To run, clone MacWire and use this command: sbt examples-scalatra/run. To browse the code, simply head to GitHub.

The example contains a request-scoped dependency (SubmittedData), session-scoped dependency (LoggedInUser) and three main services (Service1, Service2, Service3). Each class has some dependencies (declared in the constructor), and is wired using wire[]. The code is organised in two modules: logic and servlet. The services have methods to return a string-status, containing the current request and session-scoped values, so that it is possible to verify that indeed the values are shared properly. The servlet module contains a servlet which shows the service status, plus the scopes implementations.

Note that the scopes subproject is completely independent, and can be used stand-alone with manual wiring or any other library/framework. Its only dependency is javassist.

Have fun!
Adam

So while waiting for CodeBrag beta, which will bring a much better code-review experience, here’s how I get an e-mail notification on each push to the repositories I’m interested in.

The general idea is to get a RSS feed, and then use a simple IFTTT recipe to receive an e-mail whenever there’s a new item containing the words “pushed to” in the feed. The recipe looks like this:

There are two ways of getting an RSS feed with all the pushes. Firstly, you can get an RSS feed which aggregates pushes to all of the repositories that you are watching. Simply go to GitHub, log in, and in the upper-right corner you should see a “News Feed” link. Copy that to IFTTT and you’re done!

Secondly, you can get a feed for a single branch of a repository. Go to the repository you wish to get notifications for, and click on the “commits tab” (you should get a page like this one). Next to “Commit History”, there’s a feed symbol, which is a link to the branch’s feed.

Hope this will be useful!
Adam

To implement DI all you really need is to remove the news from your code, and move the dependencies to the constructor. There must be of course some place where the objects are created; for that you need a top-level (“main” class), where all the wiring is done. That’s where DI containers really help: they remove the tedious task of passing the right parameters to the constructors. Usually that’s done at run-time using reflection.

MacWire takes a different approach. Basing on declarations specifying which classes should be instantiated, it generates the code needed to create a new class instance, with the correct parameters taken from the enclosing type. This is done at compile-time using a Scala Macro. The code is then type-checked by the Scala compiler, so the whole process is type-safe, and if a dependency is missing, you’ll know that immediately (unlike with traditional DI containers).

For example, given:

1
2
3
4
5
6
7
8
9
10
11
12
13

class DatabaseAccess()
class SecurityFilter()
class UserFinder(databaseAccess: DatabaseAccess, securityFilter: SecurityFilter)
class UserStatusReader(userFinder: UserFinder)
 
trait UserModule {
    import com.softwaremill.macwire.MacwireMacros._
 
    lazy val theDatabaseAccess   = wire[DatabaseAccess]
    lazy val theSecurityFilter   = wire[SecurityFilter]
    lazy val theUserFinder       = wire[UserFinder]
    lazy val theUserStatusReader = wire[UserStatusReader]
}

The generated code will be:

1
2
3
4
5
6

trait UserModule {
    lazy val theDatabaseAccess   = new DatabaseAccess()
    lazy val theSecurityFilter   = new SecurityFilter()
    lazy val theUserFinder       = new UserFinder(theDatabaseAccess, theSecurityFilter)
    lazy val theUserStatusReader = new UserStatusReader(theUserFinder)
}

The classes that should be wired should be contained in a Scala trait, class or object (the container forms a “module”). MacWire looks up values from the enclosing type (trait/class/object), and from any super-traits/classes. Hence it is possible to combine several modules using inheritance.

Currently two scopes are supported; the dependency can be a singleton (declared as a val/lazy val) or a new instance can be created for each usage (declared as a def).

Note that this approach is very flexible; all that we are dealing with here is regular Scala code, so if a class needs to be created in some special way, there’s nothing stopping us from simply writing it down as code. Also, wire[T] can be nested inside a method’s body, and it will be expanded to new instance creation as well.

For integration testing a module, if for some classes we’d like to use a mock, a simple override suffices, e.g.:

1
2
3
4

trait UserModuleForTests extends UserModule {
    override lazy val theDatabaseAccess = mockDatabaseAccess
    override lazy val theSecurityFilter = mockSecurityFilter
}

The project is a follow up of my earlier blog post. There is also a similar project for Java, which uses annotation processors: Dagger.

MacWire is available in Maven Central. To use, simply add this line to your SBT build:

1

libraryDependencies += "com.softwaremill.macwire" %% "core" % "0.1"

The code is on GitHub, licensed under the Apache2 license, so feel free to use, fork & explore. Take a look at the README which contains some more information.

Future plans include support for factories and by-name parameter lookup, support for configuration values and more scopes, like request or session scopes, which are very useful for web projects.

Adam

On the left, there’s a traditional solution using Maven build modules. Each build module has a pretty elaborate pom.xml, e.g.:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <artifactId>parent</artifactId>
        <groupId>org.veripacks.battle</groupId>
        <version>1.0.0-SNAPSHOT</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>
    <name>Veripacks vs Build Modules: Frontend</name>
 
    <artifactId>frontend</artifactId>
 
    <dependencies>
        <dependency>
            <groupId>org.veripacks.battle</groupId>
            <artifactId>domain</artifactId>
            <version>1.0.0-SNAPSHOT</version>
        </dependency>
    </dependencies>
</project>

On the right, on the other hand, we have a much simpler structure with only one build module. Each application module now corresponds to one top-level project package (see also this blog on package naming conventions).

Notice the package-info.java files. There, using Veripacks, we can specify which packages are visible where. First of all, we specify that the code from top-level packages (frontend, reporting and domain) should be only accessible if explicitly imported, using @RequiresImport. Secondly, we specify that we want to access the domain package in frontend and reporting using @Import; e.g.:

1
2
3
4
5
6

@RequiresImport
@Import("org.veripacks.battle.domain")
package org.veripacks.battle.frontend;
 
import org.veripacks.Import;
import org.veripacks.RequiresImport;

Now, isn’t the Veripacks approach simpler? :) There is still build-time checking, which is possible by running a simple test (see the README for details). Plus, you can also use other Veripacks features, like @Export annotations, which is a generalized version of package-private scope, taking into account package hierarchies. There are also other benefits, like trivial sharing of test code (which is kind of hard with Maven), or much easier refactoring (introducing a new application module is a matter of adding a top-level package).

The immediate question that arises is – what about 3rd party libraries? Most probably, we’d like frontend-specific libraries to be accessible only in the frontend module, and reporting-specific ones in the reporting module. Well, not supported yet, but good news – that will be the scope of the next Veripacks release :).

You can view the example projects on GitHub.

Adam

[rant]
Doing the wiring at runtime with reflection has its downsides though. Firstly, there’s no compile-time checking that each dependency is satisfied. Secondly, we loose some of the flexibility we would have when doing things by hand, as we have to obey the rules by which the objects are created “automatically”. For example, if for some reason an object needs to be created manually, this requires a level of indirection (boilerplate), namely a factory. Finally, often the dependency injection is “global”, that is there is a single container with all the objects, it’s hard to create local/parametrized “universes” (Guice is an exception here). Finally-finally some frameworks do classpath scanning, which is slow, and sometimes can give unexpected results.
[/rant]

Way too magical for such a simple thing.

But isn’t what we really want just a way to have all the news with correct parameters generated for us? If you’re using Scala, and want code generation, the obvious answer are macros!

To finally show some code, given:

1
2
3
4

class A
class B
class C(a: A, b: B)
class D(b: B, c: C)

it would be nice to have:

1
2
3
4

val a    = wire[A]
val theB = wire[B] // "theB", not "b", just to show that we can use any name
val theC = wire[C]
val d    = wire[D]

transformed to:

1
2
3
4

val a    = new A()
val theB = new B()
val theC = new C(a, theB)
val d    = new D(theB, c)

Turns out it’s possible, and even not very complicated.

A proof-of-concept is available on GitHub. It’s very primitive and currently supports only one specific way of defining classes/wirings, but works :). If a dependency is missing, there’s a compile error. To check it out, simply clone the repo, run sbt and then invoke the task: run-main com.softwaremill.di.DiExampleRunner (implementation). During compilation, you should see some info messages regarding the generated code, e.g.:

1
2
3

[info] /Users/adamw/(...)/DiExample.scala:13: Generated code: new C(a, theB)
[info]   val c = wire[C]
[info]               ^

and then a proof that indeed the code was generated correctly: when the code is executed, the instances are printed to stdout so that you can see the arguments.

The macro here is of course the wire method (implementation). What it does is it first checks what are the parameters of the constructor of the class, and then for each parameter, tries to find a val defined in the enclosing class of the desired type (findWiredOfType method; see also this StackOverflow question why the search is limited to the enclosing class). Finally, it assembles a tree corresponding to invoking the constructor with the right arguments:

1
2
3

Apply(
   Select(New(Ident([class's type])), nme.CONSTRUCTOR), 
   List(Ident([arg1]), Ident([arg2]), ...))

This concept can be extended in many ways. Firstly, by adding support for sub-typing (now only exact type matches will work). Then, there’s the ability to define the wirings not only in a class, but also in methods; or extending the search to mixed-in traits, so that you could split the wire definitions among multiple traits (“modules”?). Notice that we could also have full flexibility in how we access the wired valued; it could be a val, lazy val or a def. There’s also support for scoping, factories, singletons, configurations values, …; for example:

(Dependency Injection of the future!)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

// "scopes"
val a = wire[X]
lazy val b = wire[Y]
def c = wire[Z] 
val d = provided(manuallyCreatedInstance)
 
// override a single dependency
val a = wire[X].with(anotherYInstance)
 
// factories: p1, p2, ... are used in the constructor where needed
def e(p1: T, p2: U, ...) = wire[X] 
 
// by-name binding for configuration parameters; whenever a class has a
// "maxConnections" constructor argument, this value is used.
val maxConnections = conf(10)

A recent project by Guice’s creator, Bob Lee, goes in the same direction. Dagger (mainly targeted at Android as far as I know) uses an annotation processor to generate the wiring code; at runtime, it’s just plain constructor invocations, no reflection. Similarly here, with the difference that we use Scala’s macros.

What do you think of such an approach to DI?

Adam

By default, no imports are required. As long as a class is visible (exported), it can be used. This can be now constrained using the @RequiresImport and @Import annotations.

If a package is annotated with @RequiresImport, exported classes from this package (and any child packages) can only be used if the package is imported using @Import. It is illegal to import a package, which doesn’t require importing. Note that only the package with the annotation can be imported (as a whole), not individual child packages.

As all annotations, the effects of @Import are transitive. That is, if a package (A) is imported in a package (B), its classes are also visible in all child packages (of B).

A simple example, combining the export and import features (which can be used separately as well):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

@RequiresImport
package foo.bar.p1 {
  @Export
  class A { ... }
  class B { ... } 
}
 
@Import("foo.bar.p1")
package foo.bar.p2 {
  class Test1 {
    new A(); // OK, p1 is imported and A is exported
    new B(); // illegal, B is not exported
  }
}
 
package foo.bar.p3 {
  class Test2 {
    new A(); // illegal, p1 is not imported
  }
}

Moreover, Veripacks is now available in Maven Central, so using Veripacks just got easier: no need to add a repository to your build definition. See the README for details.

As always the whole code is available on GitHub: https://github.com/adamw/veripacks.

Have fun!

Adam

Some of the most important changes:

• the group id of the plugin is now org.scala-sbt; the plugin is hosted on sbt’s Ivy repository
• proguarding projects with multiple subprojects (modules) now works as expected
• supporting sbt 0.12 (only)
• updating Proguard to 4.8

To use the plugin, add to project/plugins.sbt:

1
2
3

resolvers += Resolver.url("sbt-plugin-snapshots", url("http://repo.scala-sbt.org/scalasbt/sbt-plugin-snapshots/"))(Resolver.ivyStylePatterns)
 
addSbtPlugin("org.scala-sbt" % "xsbt-proguard-plugin" % "0.1.3-SNAPSHOT")

So far I only deployed a snapshot version (0.1.3-SNAPSHOT). If there won’t be any bug reports, or when the bug reports get fixed, I’ll do a release of a normal version soon.

Please test!

Adam

When exporting a subpackage, only the classes exported by the subpackage will be re-exported (same holds for sub-subpackages). This is in line with Veripacks respecting the hierarchical structure of packages.

(Previously, either all subpackages and all classes where exported – when no @Export annotations were present, or no subpackages were exported, if at least one @Export annotation was present.)

There are also two new convenience package annotations, @ExportAllClasses and @ExportAllSubpackages, which have pretty self-explaining names.

How does this work? An example will illustrate it best, using some imaginary syntax:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

package foo.bar.p1.sub_p1 {
   @Export
   class Sub1 { ... }
}
 
package foo.bar.p1.sub_p2 {
   @Export
   class Sub2 { ... }
 
   class Sub3 { ... }
}
 
// Normally the annotation would go to package-info.java
@ExportSubpackages("sub_p1") 
package foo.bar.p1 {
   @Export
   class A { ... 
      // Legal, Sub1 is exported by sub_p1
      new sub_p1.Sub1() 
      // Legal, Sub2 is exported by sub_p2
      new sub_p2.Sub2() 
 
      // Illegal, Sub3 is not exported by sub_p2
      new sub_p2.Sub3() 
   }
}
 
package foo.bar {
   class B {
      // Legal, A is exported by p1
      new p1.A() 
 
      // Legal, sub_p1 is exported by p1, and Sub1 is exported by sub_p1 
      new p1.sub_p1.Sub1() 
 
      // Illegal, sub_p2 is not exported by p1 
      new p1.sub_p2.Sub2() 
   }
}

How can this be useful? Veripacks goes beyond what can be achieved with normal Java access modifiers (especially public and package-private), by extending the meaning of packages to a hierarchical structure. It also aims to replace, in some cases, the need for separate build modules.

What’s next? As mentioned in the README, the plans are to add a possibility to require an explicit import of a package, to be able to use its classes. And of course, some IDE support to be able to quickly see what is exported and where would be great.

The release is available in SoftwareMill’s maven repository, and the code is available under the Apache2 license on GitHub.

Have fun!
Adam

1
2
3
4
5

class FriendsTimelineReader(userFinder: UserFinder,
   userStatusReader: UserStatusReader,
   statusSecurityFilter: StatusSecurityFilter) {
   ...
}

Note that the parameter names are exact copies of the class name. That’s certainly not DRY!

In Java writing such code is a bit easier than in Scala, since you write the type first, and then the variable name can be auto-completed (the fact that there’s an auto-complete in IDEs indicates that it’s a common pattern). In Scala there’s more typing, but then, less boilerplate around declaring fields/defining a constructor/assigning to fields.

How to avoid that? We can always use cryptic variable names, like “ssf” instead of “statusSecurityFilter“. But then the whole effort of having nicely named classes to make the code readable, which is quite a hard task, goes to waste.

Of course, variable names are very useful, for example when we have to distinguish between the 10 String parameters that our method takes, create an Int counter, etc. But the more we use specialised wrapper-types (and now possibly even more, with Scala 2.10 introducing value classes) to make our code more type-safe, the more fine-grained our services – the more often the variable/parameter names will mirror the class name.

Even when there are a couple of instances of a class, often the parameter name contains some qualifier + the class name (e.g. given a Person class, personAssociator.createFriends(firstPerson, secondPerson)). It would be interesting to see some statistics on how often variable name is a mirror of the class name or a suffix – I suspect it can be a large percentage.

Maybe the next step then in type safety is limiting the complete freedom when naming parameters? Or even, getting rid of the parameter names at all in some cases.

For example. In the snippet from the beginning, we just want to get “an instance” of a UserFinder. Most probably there will ever be only one instance of this class in the system, and certainly one will be used at a time by other classes. So why not let the class express that it wants an instance of a class, without having to name it? Let’s use the indefinite article “a” as a keyword (we don’t really care which instance is passed):

1
2
3

class FriendsTimelineReader(a UserFinder,
  a UserStatusReader,
  a StatusSecurityFilter)

Now, how would you use such a variable inside a class? Let’s use the definite article “the” – the instance that was given to the class, for example:

1

val user10 = (the UserFinder).findById(10)

Maybe this looks a bit like science-fiction, but wouldn’t it be convenient? Or maybe this problem is already solved by some mechanisms in other languages?

Adam

EDIT 27/01/2013: Extending the above to handle qualifiers:

1
2
3
4
5

class Friends(a 'first Person, a 'second Person) {
   ...
   (the 'first Person).getName()
   ...
}

ElasticMQ

Version 0.6.3 of ElasticMQ has only one major change – it is built with Scala 2.10. That meant two code changes:

• Using Typesafe’s Scala Logging instead of slf4s
• Using Typesafe’s Config instead of Ostrich for stand-alone server

The second change is the only one visible to the user. The type-safe configuration that was made possible by Ostrich is great, but it doesn’t look as if a 2.10-compatible version is coming soon. Typesafe Config uses more traditional configuration files, with reasonable amount of verification (though it’s a bit funny that migrating to Typesafe’s library causes the config to stop being type-safe ;) ). Also, a good side of the change is that the server always starts up quickly – no need to compile the config file. Locally on my laptop start up takes about 0.5 seconds.

As always, the new release should be available in Maven Central soon, and links to download the standalone server packages can be found on the main page.

scala-macro-debug

I also updated the example Scala macros project, and deployed it to our repositories. If you don’t want to copy the code, you can now use the debug macro by adding:

1
2
3

resolvers += "SoftwareMill Snapshots" at "https://nexus.softwaremill.com/content/repositories/snapshots/"
 
libraryDependencies += "com.softwaremill.scalamacrodebug" %% "macros" % "0.0.1-SNAPSHOT"

Adam