Modules, modules, modules …

I think everybody will agree that writing modular applications and modularity in general is a good thing.

But how does support for modularity look like, both from the Java and Scala languages and various Java/Scala frameworks? There’s a lot of different approaches! Let’s look at some of them. Below “protection” means how well modules are separated either at compile-time or run-time.

Packages

First of all we’ve got Java (or Scala) packages. The concept is very useful when it comes to organizing the source code. However it doesn’t offer compile-time or run-time (except for the package-protected visibility, which isn’t used too widely) protection, so it’s rather hard to say that packages are any help in modularizing the code. Also, there’s a lot of naming problems and hence conventions: if for example we have two implementations of a data reader, one using a database, the second using the filesystem, should the associated classes go to database and filesystem subpackages, or stay in the same top-level package? Should the class names be prefixed with Database and Filesystem, even if they are inside the dedicated packages?

Build subprojects

Secondly, there’s Maven/SBT modules/subprojects. They offer compile-time protection, which is a very nice thing: we can now statically make sure that our code only references classes from the explicitly specified dependencies. However, then comes the obvious question: when is a functionality “big enough” to make it a separate Maven module? Is it ok to have many Maven modules, each containing only several classes (which can be a PITA, having to define a separate pom, directory structure for each), or should they be bigger? What about naming conventions, should the package name correspond to the module name? If so, we’re clearly violating DRY here ;)!

Imagine we decide to put our database and filesystem data readers into separate packages, maven modules and name them appropriately. So we have a DatabaseReader (plus 10 helper classes) in a foo.bar.database package in the myapp-database Maven module. Now a simple refactoring: renaming “database” to “db” becomes a really complex task, most certainly ending up in using various terms in various places.

OSGi

The next step of the evolution would be OSGi bundles (or, equivalently, a module in JBoss Modules or in Project Jigsaw). One such bundle is typically a product of one (or more) Maven module, but it also offers run-time protection by appropriately scoping class-loaders. All of the problems from Maven modules are inherited … with the additional need to name the bundle!

Nested classes/cake pattern

Another possibility of scoping your code is using nested classes, or even to go further and use the Cake pattern in Scala. Apart from the old problems: what package to use and how to name the module class, we’ve got a couple new ones. Firstly, the whole source code of the module is now in one file. This can mean really long files! However maybe that’s more of an IDE problem: nobody said that a file needs to be edited all at once; the editor could show only one module class/method (like in the Smalltalk IDE). Cake pattern is also not free of problems. And, what about nested modules?

DI frameworks

Finally, we’ve got various DI frameworks (like Guice, Spring or CDI), where, if we look from an appropriate perspective, we define small modules (classes) which can depend on other modules (injected by the container). Separating the module interface (java interface) from the module implementation (java class), and only injecting the interface is also very common. But again, the only way to partially statically prevent injecting the implementation we have to resort to creating separate -api and -impl Maven modules.

In fact, if we take the approach of nested classes, a DI framework may be very well suited for resolving inter-module dependencies and doing all of the wiring for us (which can be very useful if we want to be protected from situations where a module has a new dependency, and each use-site must be now amended).

Next?

To sum up, I think that the various approaches to supporting modularity could use some unification. And, unfortunately, this would probably mean totally departing from what we know today from Java/Scala: a new package system, a new build system, a new runtime system. Unfortunately the outcrop of new programming languages doesn’t offer much in that area.

Do you think there should be one module system, usable both in the small (single classes) and in the large (sets of classes)? After all, what a DI container does to resolve the dependencies and instantiate classes isn’t so much different from what Maven or an OSGi runtime does when booting!

Or maybe modules inherently have several types, small ones (class-level) and big ones (maven module/osgi bundle-level)?

Lastly, maybe there are some other non-JVM based languages, which solve the modularity problem (still being usable) in a nicer way?

Adam

Introducing ElasticMQ: Scala SQS alternative

Wanting to explore Scala and some new Scala/Java libraries I started writing ElasticMQ. It is a simple message queueing system, following closely Amazon SQS semantics and exposing a REST SQS-like interface. Currently only the basic operations are implemented. ElasticMQ can be useful as an SQS replacement, e.g. for for testing SQS applications.

My aim was to make the usage as simple as possible, that’s why to create a new node and expose the REST interface all you need to do is:

val node = NodeBuilder.createNode
val server = SQSRestServerFactory.start(node.nativeClient, 8888,
                 "http://localhost:8888")

Now simply point your SQS client to http://localhost:8888 and you should be able to create/delete queues, change the visibility timeout, send and receive messages.
When you are done using ElasticMQ, you can shutdown the node and the server:

server.stop()
node.shutdown()

All of the source code is available on GitHub. See the readme for instructions on how to add ElasticMQ as a dependency to your Maven or SBT project.

On the technical side, ElasticMQ uses a number of interesting things.

Firstly, there’s Netty, an asynchronous, event-driven java NIO framework. I used Netty to implement the REST server. All of the requests are processed in a non-blocking way.

The messages are currently stored in an in-memory H2 database (but this can be easily changed to point e.g. to a MySQL or PostgreSQL DB), and managed using the Squeryl Scala library. Squeryl lets you write SQL queries as typesafe Scala code, e.g.:

update(messages)(m =>
  where(m.id === message.id and m.lastDelivered === message.lastDelivered)
    set(m.lastDelivered := lastDelivered))

For integration testing I am using the Typica library.

On the Scala side, I created a simple internal DSL for defining rest handlers. The server (which uses Netty) is generic and can be provided with any number of “rest handlers”, to which requests are dispatched. Each rest handler specifies what is the path it will respond to, what are the required parameters and what should be run in response to a request. For example:

val handler1 = (createHandler
   forMethod GET
   forPath (root / "products" / %("name") / "price")
   requiringParameters List("currency", "quantity")
   requiringParameterValues Map("department"->"electronics")
   running electronicsPriceRequestLogic)

val server = RestServer.start(handler1 :: handler2 :: … :: Nil, 8888)

See the RequestHandlerLogic file for the implementation and RequestHandlerBuilderTestSuite for example usage.

In fact there’s another small DSL for defining paths. For example (root / "a" / "b" / "c") will only match a/b/c, but (root / "a" / %("p1") / "c") will match a/anything/c, with the middle component being assigned to the p1 parameter. You can also use regex, e.g. (root / "a" / """[a-z0-9]+""".r / "c") for matching path components. See the RestPath class and test for details.

Not sure where ElasticMQ will go in the future, but I can image lots of interesting possibilities. So stay tuned :).

Comments about the code, Scala style, Netty/Squeryl usage are very welcome! As well as any other ideas, improvement suggestions etc.

Adam

Using Envers with AS7

Recently I tried deploying a web application which uses Envers into AS7, but unfortunately I encountered some problems (see the forum discussion). Luckily thanks to the helpful JBoss guys I’ve got it working now. Moreover, thanks to the work done by Strong Liu, you should see full Envers integration (included OOTB as a module) in AS7.1!

Until then, here’s how to use Envers in a webapp in AS7.

First, we need to create an Envers module. Create a directory: jboss-as-7.0.0.Final/modules/org/hibernate/envers/main and inside it, create a module.xml with the following content:

<?xml version="1.0" encoding="UTF-8"?>
<module xmlns="urn:jboss:module:1.0" name="org.hibernate.envers">
  <resources>
    <resource-root path="hibernate-envers-4.0.0.Beta1.jar"/>
  </resources>

  <dependencies>
    <module name="org.hibernate"/>
    <module name="org.jboss.logging"/>
    <module name="org.dom4j"/>
    <module name="javax.api"/>
    <module name="javax.persistence.api"/>
    <module name="javax.transaction.api"/>
    <module name="org.javassist"/>
  </dependencies>
</module>

Moreover, you’ll need to put the Envers jar inside that directory. You can download it straight from the JBoss Maven repository.

I’ve prepared a ready zip of the module, which can be downloaded here. Just unpack its contents to jboss-as-7.0.0.Final/modules/org/hibernate.

Secondly, you must add an entry in your manifest file (META-INF/MANIFEST.MF) to make the new module available to your application:

Dependencies: org.hibernate.envers services

And you’re done. Enjoy! :)

Adam