Compile Time vs Runtime OSGi Dependencies

Just a quick blog post to talk about compile time vs runtime dependencies in the OSGi container, inspired by this thread: https://web.liferay.com/community/forums/-/message_boards/view_message/105911739#_19_message_106181351.

Basically a developer was able to get Apache POI pulled into a module, but they did so by replicating all of the "optional" directives into the bnd.bnd file and eventually putting it into the bundle's manifest.

So here's the thing - dependencies come in two forms. There are compile time dependencies and there are runtime dependencies.

Compile time dependencies are those direct dependencies that we developers are always familiar with. Oh, you want to create a servlet filter? Fine, you just have a compile time dependency as expressed in a build.gradle file like:

compileOnly group: "javax.servlet", name: "javax.servlet-api", version: "3.0.1"

Basically a compile time dependency is a dependency needed by the compiler to compile your java files into class files. That's really it; you need a class from, say, the XStream XOM package, well then you declare your dependency on in it your build.gradle file and your code compiles just fine.

Runtime dependencies are not as straight forward. What you find, especially when you deploy your OSGi bundles, is that you not only have your direct dependencies like the servlet spec jar or the XStream jar, but there are also indirect dependencies to deal with.

Let's take a look at XStream. If we check mvnrepository.com, these are the listed dependencies for XStream:

Group Name Version
cglib cglib-nodep (optional) 2.2
dom4j dom4j (optional) 1.6.1
joda-time joda-time (optional) 1.6
net.sf.kxml kxml2-min (optional) 2.3.0
net.sf.kxml kxml2 (optional) 2.3.0
org.codehaus.jettison jettison (optional) 1.2
org.codehaus.woodstox wstx-asl (optional) 3.2.7
org.jdom jdom (optional) 1.1.3
org.jdom jdom2 (optional) 2.0.5
stax stax (optional) 1.2.0
stax stax-api (optional) 1.0.1
xmlpull xmlpull 1.1.3.1
xom xom (optional) 1.1
xpp3 xpp3_min 1.1.4c

Note that there are only two dependencies here that are not listed as optional, xmlpull and xpp3_min. These are libraries that XStream uses for lower-level XML stuff.

But what are all of these optional dependencies?

Let's pick the well-known one, Joda Time.  Joda is a date/time library that supports parsing date/times from strings and formatting strings from date/times, amongst other things. The library is marked as "optional" because you don't have to have Joda in order to use XStream.

For example, if you are using XStream to do XOM marshaling on XML that does not have dates/times, well then the code that uses Joda will never be reached. So Joda is absolutely optional, from a library perspective, but as the implementing developer only you know if you need it or not. If you have XML that does have dates/times but you don't have Joda, you'll get ClassNotFoundException errors when you hit the XStream code that leverages Joda.

When the libraries are being built, the scope used for the declared dependencies, i.e. runtime vs compile in Gradle and <optional /> tag in Maven, will translate into the "resolution:= optional" stanza in the jar's MANIFEST.MF. Depending upon how the jar is used, this extra designation can be used or it can be ignored. For example, if use the "java" command with a classpath that includes the XStream jar and your classes, Java will happily run the code whether or not Joda is required. However, if you were to try to process an XML file with a date/time stamp, you may encounter a ClassNotFoundException or the like if Joda is not available.

The OSGi container is stricter about these optional dependencies. OSGi sees that XStream may need Joda, but it cannot determine whether or not it will be needed when the bundle is resolving. This is one reason why you get the "Unresolved Requirement" error when OSGi attempts to start the bundle.

It is up to the bundle developer to know what is required and what isn't, and OSGi forces you to either satisfy the dependency (ala something like this) or excluding the package dependency by masking them out using the Import-Package declaration. If you, the developer, are using XStream, OSGi expects that you should know if you are going to need an optional dependency like Joda or not.

Now I hate picking out one example like this, but I think this is really important to point out. Yes, you can tell OSGi to also treat the dependency as optional. It will get you by the Unresolved Requirement bundle start error. The problem, however, is it leaves you open to a later ClassNotFoundException because you have a dependency on a package which is marked as optional. The last thing you want to have happen is to find your module deployed to production and fail on a sporadic basis because sometimes an XML file has a date/time to parse.

Recommendations

So, now for some recommendations...

If you have a dependency, you have to include it. I tend to use Option 4 from my blog, but I'm using the compileInclude Gradle dependecy directive to handle the grunt work for me. If your dependency has required dependencies, well you have to include them also and compileInclude should cover that for you too.

For the optional dependencies, you have to determine if you need them or not. Yes, this is some analysis on your part, but it will be the only way to ensure that your module is correctly packaged.

If you need the optional dependency, you have to include it. I will typically use the compileInclude directive explictly to ensure the dependency gets pulled into the module correctly.

If you don't need the optional dependency, then exclude it entirely from the bundle. You do this in the bnd.bnd file using the Import-Package directive like:

Import-Package: !org.codehaus.jackson, \
  !org.codehaus.jackson.annotate, \
  !org.codehaus.jackson.format, \
  !org.codehaus.jackson.impl, \
  !org.codehaus.jackson.io, \
  !org.codehaus.jackson.type, \
  !org.codehaus.jackson.util, \
  !org.joda.time, \
  !org.joda.time.format, \
  *

The syntax above tells OSGi that the listed packages are not imported (because of the leading exclamation point). This is how you exclude a package from being imported.

NOTE: When using this syntax, always include that last line, the wildcard. This tells OSGi that any other non-listed packages should be imported and it will require all remaining packages be resolved before starting the bundle.

And the final recommendation is:

Do not pass the optional resolution directive on to OSGi as it may lead to runtime CNFEs due to unsatisfied dependencies.

Got questions about your dependency inclusion? Comment below or post to the forums!