Modules, modules, modules …

15 Flares 15 Flares ×

I think everybody will agree that writing modular applications and modularity in general is a good thing.

But how does support for modularity look like, both from the Java and Scala languages and various Java/Scala frameworks? There’s a lot of different approaches! Let’s look at some of them. Below “protection” means how well modules are separated either at compile-time or run-time.

Packages

First of all we’ve got Java (or Scala) packages. The concept is very useful when it comes to organizing the source code. However it doesn’t offer compile-time or run-time (except for the package-protected visibility, which isn’t used too widely) protection, so it’s rather hard to say that packages are any help in modularizing the code. Also, there’s a lot of naming problems and hence conventions: if for example we have two implementations of a data reader, one using a database, the second using the filesystem, should the associated classes go to database and filesystem subpackages, or stay in the same top-level package? Should the class names be prefixed with Database and Filesystem, even if they are inside the dedicated packages?

Build subprojects

Secondly, there’s Maven/SBT modules/subprojects. They offer compile-time protection, which is a very nice thing: we can now statically make sure that our code only references classes from the explicitly specified dependencies. However, then comes the obvious question: when is a functionality “big enough” to make it a separate Maven module? Is it ok to have many Maven modules, each containing only several classes (which can be a PITA, having to define a separate pom, directory structure for each), or should they be bigger? What about naming conventions, should the package name correspond to the module name? If so, we’re clearly violating DRY here ;)!

Imagine we decide to put our database and filesystem data readers into separate packages, maven modules and name them appropriately. So we have a DatabaseReader (plus 10 helper classes) in a foo.bar.database package in the myapp-database Maven module. Now a simple refactoring: renaming “database” to “db” becomes a really complex task, most certainly ending up in using various terms in various places.

OSGi

The next step of the evolution would be OSGi bundles (or, equivalently, a module in JBoss Modules or in Project Jigsaw). One such bundle is typically a product of one (or more) Maven module, but it also offers run-time protection by appropriately scoping class-loaders. All of the problems from Maven modules are inherited … with the additional need to name the bundle!

Nested classes/cake pattern

Another possibility of scoping your code is using nested classes, or even to go further and use the Cake pattern in Scala. Apart from the old problems: what package to use and how to name the module class, we’ve got a couple new ones. Firstly, the whole source code of the module is now in one file. This can mean really long files! However maybe that’s more of an IDE problem: nobody said that a file needs to be edited all at once; the editor could show only one module class/method (like in the Smalltalk IDE). Cake pattern is also not free of problems. And, what about nested modules?

DI frameworks

Finally, we’ve got various DI frameworks (like Guice, Spring or CDI), where, if we look from an appropriate perspective, we define small modules (classes) which can depend on other modules (injected by the container). Separating the module interface (java interface) from the module implementation (java class), and only injecting the interface is also very common. But again, the only way to partially statically prevent injecting the implementation we have to resort to creating separate -api and -impl Maven modules.

In fact, if we take the approach of nested classes, a DI framework may be very well suited for resolving inter-module dependencies and doing all of the wiring for us (which can be very useful if we want to be protected from situations where a module has a new dependency, and each use-site must be now amended).

Next?

To sum up, I think that the various approaches to supporting modularity could use some unification. And, unfortunately, this would probably mean totally departing from what we know today from Java/Scala: a new package system, a new build system, a new runtime system. Unfortunately the outcrop of new programming languages doesn’t offer much in that area.

Do you think there should be one module system, usable both in the small (single classes) and in the large (sets of classes)? After all, what a DI container does to resolve the dependencies and instantiate classes isn’t so much different from what Maven or an OSGi runtime does when booting!

Or maybe modules inherently have several types, small ones (class-level) and big ones (maven module/osgi bundle-level)?

Lastly, maybe there are some other non-JVM based languages, which solve the modularity problem (still being usable) in a nicer way?

Adam

  • RR

    package objects as module interface definition.

    http://www.scala-lang.org/docu/files/packageobjects/packageobjects_3.html

  • http://www.warski.org Adam Warski

    Interesting, though right now package objects don’t have any substantial differences to classes nested inside scala objects, if I’m not mistaken.
    But the comments about how the package objects may evolve are certainly promising :)

    Thanks,
    Adam

  • http://onlysoftware.wordpress.com/ Papapetrou Patroklos

    Nice article althought I dont’a agree with everything.

    I have posted a similar article wondering why there is not standard for developing modular web applications
    http://onlysoftware.wordpress.com/2011/07/19/why-there-is-no-standard-for-developing-real-modular-web-applications/

  • Stephen Colebourne

    See pods in the Fantom language for the JVM.
    http://fantom.org/doc/docLang/Structure.html
    http://fantom.org/doc/docLang/Deployment.html

    Each pod is a module. You can only compile modules (you can’t compile classes). Pods store full dependency information and can provide compile and runtime protection. Plus a full database of all the types they hold (easy lookup by class/mixin name or annotation/facet). And soon to be released is the shared repo http://fantom.org/doc/docFanr/index.html

    Overall, a top quality module system!

  • Yardena

    Re your question in the end of the post – have you looked at Newspeak? http://gbracha.blogspot.com/search/label/Modularity

    There’s also a paper http://www.bracha.org/newspeak-modules.pdf

  • Caspar MacRae

    Question: “All of the problems from Maven modules are inherited”?

    First maven & OSGi have a certain synergy, but OSGi “inherits” nothing from maven.

    If you only mean managing dependencies and their versions? Then I see your problems as actually the solutions – you really do have to manage these in any modular environment, they cannot be ignored.

    But if not, please clarify eaxctly what these “problems” are …

  • http://www.warski.org Adam Warski

    Of course OSGi doesn’t “inherit” from Maven anything technologically, but the conceptual problems stay the same; for example – when to factor out a functionality to a separate bundle, when is it big enough? How about naming? You still have to name the bundle, the packages inside and the classes.

    Adam

  • http://www.warski.org Adam Warski

    Looks nice! So if I understand correctly:
    * there are no java-style (nested) packages, only top-level pods (though they can be named com.company.anything, but there’s no subpod-pod relation concept)
    * pods integrate with the build & deployment system
    * compile-time protection, no run-time isolation (?)

    What about “DI”? Can I say that I depend on some pod which implements a given interface?

    This, in fact, brings us to the crucial concept of module interface and module implementation, which I didn’t cover in the blog post (see OCaml of example). So here we would need interface and implementation pods.

    Adam

  • candide

    Sorry but I could not understand the refactoring problem in Maven module example.
    Moreover I could not understand ” All of the problems from Maven modules are inherited … with the additional need to name the bundle!” in the OSGi part.

    Could you clarify a bit more ? I’m not expert in the area but I think you can create a module/bundle in OSGi by using one class but of course you’ll need an interface also since without an interface there can not be a module/bundle.

  • http://www.warski.org Adam Warski

    Sure

    As for the refactoring. If you have a database-reader module, com.company.reader.database package, and insde a DatabaseReader, DatabaseConnector, DatabaseReaderHelper etc classes, refactoring “database” to “db” would involve:
    * changing the module name (optionally: the module indentifier in the bundle definition)
    * changing the package name
    * changing the class name
    It’s quite easy to miss one spot and be left with a db-reader module, com.company.reader.database package and DBReader, DatabaseReaderHelper classes.

    Of course you can create an OSGi bundle/Maven module with a single class, but that’s highly impractical. Imagine defining a pom.xml for every one! That would be a PITA ;). So modularity using OSGi/Maven is naturally limited by practical reasons.

    Adam

  • candide

    Yes I see the problem. I think it is related with the lack of support of IDEs/Tools since modularity efforts came a bit lately…

    I should have written that before to be more specific but… I would rather consider if it is wise to have a module based on only one class (in terms of design) rather than impracticality of doing that with available tools…

    I totally understand the pain of selecting and using tools and their shortcomings however I’m really focused at how to break dependencies and I think at that point OSGi specification (I don’t have much experience about available tools) is quite good IMHO.

    Finally it looks like there is a big divergence in Java community for bringing (natural) modularity to Java hence I guess we will have some pain for some time…

  • http://www.warski.org Adam Warski

    Thanks for the link, interesting. I especially like the arguments against the “import” statement.
    One thing in Newspeak is missing though, namely auto-wiring of dependencies.
    It’s a very nice feature or e.g. Guice that I only need to say “XImpl is the implementation that should be used for classes which require X”, without the need to pass the argument explicitly to each use-site.

    Adam

  • http://www.warski.org Adam Warski

    I would rather consider if it is wise to have a module based on only one class

    Sure, that’s where the “module sizes” come to play: you can keep the class as a DI-bean (Spring bean, CDI bean, Guice bean, whatever), or factor is out as a module. Or even better, we could maybe have one mechanism for both! :)

    Adam

  • Tomasz Nazar

    “Every” other major lang besides Java supports multiple inheritance or dynamic modules (mixins) that can be added to objects runtime… no big deal.. ;) Ruby, Python, Javascript

    Sorry Java folks :>

    One should use at least AOP for mixing separate concerns into a class/object in Java..
    No pb with AspectJ or JbossAOP, or whatever aop came to Java in this area within last 6 years..

    There is really no way you can modularize code well without having possibility to split a class (code) and then merge it runtime/statically being an object..

  • http://www.warski.org Adam Warski

    AOP is good for cross-cutting concernes. I don’t think anybody uses it for mixins.

    Speaking of mixins, how do they solve the above mentioned problems? In fact the Cake Pattern uses mixins extensively (it’s based on some properties of traits), and still it’s very far from perfect.

    Adam

  • Tomasz Nazar

    Looks like I assumed some context of my conclusions, and mistaken modularity you’re talking about..

    I thought you’re writing on primary focus of OO: modularity of classes (encapsulation, DRY, small fns, little classes)..

    .. and in fact you’re speaking on packages, modules.. which is .. ehmm.. secondary.

    So excuse my noise here..
    We could talk about OO though :-)

  • http://www.warski.org Adam Warski

    DRY – yes, but in another sense (name repetition).
    Encapsulation – yes, but on a higher level (hiding code in one module from another). But is it a higher level? The mechanism is the same, but we don’t hide code in a class, we hide whole classes.
    Litte classes -> little modules?

    I guess modularity in the sense I meant isn’t secondary at all. Both are important in fact.

    Adam

  • Pingback: Let’s turn packages into a module system! @ Blog of Adam Warski

15 Flares Twitter 8 Facebook 0 Google+ 5 LinkedIn 2 Email -- 15 Flares ×