Dynamic Application Modules - Part 7 - Loading At Runtime

A year ago, I wrote a series of posts called "Dynamic Application Modules" that showed how to construct a JavaFX application from loosely-coupled modules.  These modules were so loosely-coupled that the main program didn't need to know about them ahead of time.  This update to that series takes the concept a step further by loading the modules at runtime.

All of the code in this example is available on GitHub.

This video demonstrates the runtime loading of new classes into a JavaFX application.


The video shows an empty screen.  Navigating to a Preferences screen, I browse for JARs.  I select the JARs and import them.  Then, I restart the app.  When the app restarts, it finds the imported JARs and loads them into the app.  The JARs have specially-annotated classes to enable this mechanism which I'll discuss later.

Purpose

If you have a small app, you're best off maintaining a single build project with a few packages and deploying a single JAR (plus dependencies).  However for larger projects, say 1,000 or more files, this is too constraining.  You'll want to move out incremental functionality without touching other parts for improved quality, better deployments, and more fault tolerance.  You'll also want to develop independent pieces broken away from the larger program for agility.

This demonstration is a project with a core (the initial empty screen) that supports the non-functional requirements of things like security, top-level error handling, global functions, and services.  The core can ship with extensions (I'm calling them SubApps) that make up the functional requirements.

Here are some business reasons why you might want to ship different combinations of artifacts.

1. Customer-specific extensions.  A customer paid for a specific extension and it's not appropriate for another customer to see the code.  For example, you're retrieving data from a customer-specific WSDL that's confidential.
2. Different implementations. Two JARs implement the same functionality and it it's only valid to run one of them.
3. Licensing. You're holding back functionality based on what is being offered in a demo or you're charging a customer.
4. Beta customers.  Some customers may be willing to accept a lesser-tested product for a compelling new feature.

ClassLoader

To load new classes into a Java application, you need to use a ClassLoader either directly or indirectly through a Class object.  Running as a desktop app -- without a hierarchical set of ClassLoaders like an app server -- means that the set of classes available to me is defined by the classpath.  However, my requirement is to add to the set of classes.

One way to do this is to keep growing the classpath for deployments involving a startup script.  At the command-level, you can simply appending JAR paths for the imported JARs.  This seems a little bit awkward and risky.  It's awkward because you have to deal with multiple scripts for each target platform.  It's risky because if something goes wrong with the parsing to add and remove scripting code, your users won't be able to bring up the whole app.

The way that I'm extending the pool of classes is to use a custom ClassLoader, specifically a URLClassLoader.  When the app starts up, I get a list of JAR files from a known location $HOME/.examples-javafx-dynamic/subapps.  This list may be empty.

String APP_FOLDER_NAME = ".examples-javafx-dynamic";
String SUBAPP_FOLDER_NAME = "subapps";

String userDir = System.getProperty("user.home");

File appFolder = new File(userDir, APP_FOLDER_NAME);
if( !appFolder.exists() ) {
 appFolder.mkdir();
}

File subappFolder = new File( appFolder, SUBAPP_FOLDER_NAME);
if( !subappFolder.exists() ) {
 subappFolder.mkdir();
}

List jarURLs = new ArrayList<>();
File[] subappJARs = subappFolder.listFiles();
for( File sa : subappJARs ) {
 String urlPath = "jar:file:" + sa.getPath() + "!/";
 jarURLs.add( new URL(urlPath) );
 subappJars.add( sa.toPath() );
}

if( jarURLs.size() > 0 ) {
 urls = jarURLs.toArray( new URL[0] ); 
 urlClassLoader = new URLClassLoader( urls, this.getClass().getClassLoader() );
} else {
 urlClassLoader = this.getClass().getClassLoader();
}

URLClassLoader contains URLs for the newly-added JARs plus the parent ClassLoader.

Thread.setContextClassLoader

You can make direct calls to the URLClassLoader to get classes including those that were found at startup.  However, my app makes use of Google Guice and FXMLoader which are also involved in loading classes.

FXMLLoader scans .fxml files and finds Controllers.  When it finds a Controller String, it loads the class.  FXMLLoader consults the Thread.currentThread().getContextClassLoader() to determine the ClassLoader to use when loading this class.  This is in advance of object creation (Controller factory).

So, after the URLClassLoader object is created, I make a call like this on the FX Thread to allow FXMLLoader to find the classes.  Note that these classes can be found in the Core JAR as well as the imported JARs because it inherits from the parent ClassLoader.

Thread.currentThread().setContextClassLoader( cl );  // used by FXMLLoader

Reflections

Reflections is a project that I'm using to find classes I'm interested in.  It searches class files for classes matching an annotated type.  This is the second part of the discovery phase.  After loading the available classes, I look for classes flagged as @SubApp and grab the metadata I keep in the annotation.

Reflections also needs to know about this new ClassLoader.

Reflections reflections = new Reflections(DEFAULT_PACKAGE_TO_SCAN, cl, urls);

Set<Class<?>> subapps = reflections.getTypesAnnotatedWith(SubApp.class);

// process each subapp...

Uninstall

To implement the uninstall, I had to defer the removal of the imported JARs from the known location ($HOME/.examples-javafx-dynamic/subapps).  The JAR files are in use during the whole time the program is running.  Neither a deleteOnExit nor a delete command in a shutdown hook would allow me to delete the loaded JAR file.

So, I saved off some commands to delete the files at startup.  My uninstall method writes lines of text, one per uninstall operation, of the form ("D /home/carl/.examples-javafx-dynamic/subapps/myjar.jar").  At the next restart, I parse through the file and clean up the list of available JARs before creating the URLClassLoader.

Design Decision

Because this is a desktop app and not an app server, I'm not worried about putting the user through a restart.  The restart is the best way to make sure your program state is in the best shape in terms of fending off memory leaks.  Because you can't remove classes from a ClassLoader, hot deployers like app servers will use many ClassLoaders, creating new ones for new deployments. This often leads to serious memory leak issues.  One bad class can prevent a lot of memory from being reclaimed.

See this video from Zeroturnaround before you attempt to convert this to a hot deploy.

If you're interested in what I've put in the JAR files, start here with the original series of posts.  The GitHub repository for this project is the best resource.  You'll find the following projects to allow you to recreate the demo.

1. examples-javafx-dynamic-parent - Parent project that builds everything
2. examples-javafx-dynamic-framework - Code shared among core and sub apps
3. examples-javafx-dynamic - The main app and core components
4. examples-javafx-dynamic-subapp_a - Example SubApp A
5. examples-javafx-dynamic-subapp_b - Example SubApp B
6. examples-javafx-dynamic-subapp_c - Example SubApp C

mvn clean install the parent.  Run MultiStageMain from examples-javafx-dynamic.  Select File > Preferences and browse to the subapp JAR locations in your Maven repository.  Import each item and restart.