Visualizer, Part 3: Poor Man's PDE Build

This is the third in my series (Part 1, Part 2) of posts about Visualizer. In this post I'll be talking about how to create a simplified PDE build.

As I mentioned in the previous post, Visualizer is built on OSGi. My preferred development environment for doing any Java development, but especially OSGi development, is Eclipse because of its wonderful JDT and PDE tooling. The PDE team has created an awesome environment for developing and managing OSGi bundles. However, one of the requirements that I had for Visualizer was that anyone could download the source code and build it, regardless of their IDE or environment preferences. PDE includes the ability to perform a headless build, but I didn't really want to expect the user to download Eclipse or to include a stripped down version of Eclipse in the Visualizer distribution just so the user could build it from the commandline. So I set out to create a "Poor Man's PDE Build" using just Ant.

Actually building the plugins with Ant is relatively simple. This short Ant file will build the plugin:

<?xml version="1.0" encoding="UTF-8"?>
<project default="build-plugin">
    <property name="src.dir" value="src"/>
    <property name="classes.dir" value="bin"/>
    <property file="META-INF/MANIFEST.MF"/>
    <property file="build.properties"/>
    
    <path id="classpath">
        <fileset dir="${dist.dir}">
            <include name="*.jar"/>
        </fileset>
        <fileset file="${osgi.framework}"/>
        <pathelement path="${java.class.path}"/>
    </path>
    
    <target name="init">
        <mkdir dir="${classes.dir}"/>
    </target>
    
    <target name="compile" depends="init">
        <echo message="Compiling the ${Bundle-SymbolicName} plugin"/>
        <javac srcdir="${src.dir}" destdir="${classes.dir}" classpathref="classpath" debug="true"/>
    </target>
    
    <target name="copy-resources">
        <echo message="Copying resources"/>
        <copy todir="${classes.dir}">
            <fileset dir="." includes="${bin.includes}"/>
        </copy>
    </target>
    
    <target name="build-plugin" depends="compile, copy-resources">
        <jar jarfile="${dist.dir}/${Bundle-SymbolicName}_${Bundle-Version}.jar" basedir="${classes.dir}" manifest="META-INF/MANIFEST.MF"/>
    </target>
    
    <target name="clean">
        <delete includeemptydirs="true">
            <fileset dir="${classes.dir}" includes="**/*"/>
        </delete>
    </target>
</project>

As you can see here, there really isn't much to the actual build. The best part is we can use the META-INF/MANIFEST.MF and build.properties created when we're working in PDE to control the build.

For a single bundle with no dependencies, this effectively duplicates the PDE build process. The difficulties comes in when you start having dependencies. If you use the headless PDE build, it will sort out all the dependencies for you and build your bundles in the proper order.

Implementing proper dependency resolution seemed awfully complicated, especially since PDE build already implements it. Fortunately, Visualizer doesn't require complicated dependency resolution because I've structured the bundles in a logical order. There are three levels of bundles: "core" which implement the main functionality, "ui" which implement the user interface to the core bundles, and "application" bundles that build on both the core and ui bundles.

Armed with this knowledge, we can structure a three stage build process where we first build all of the core bundles then all of the ui bundles and then all of the application bundles. To accomplish this, we have a master build.xml that calls out to the template build-plugin.xml file listed above using a subant task.

<target name="build-framework" depends="init">
    <!-- build org.andrill.visualizer, org.andrill.visualizer.services* -->
    <subant target="build-plugin" genericantfile="build-plugin.xml" failonerror="false">
        <property name="dist.dir" value="../${build.dir}"/>
        <property name="osgi.framework" value="../framework.jar"/>
        <dirset dir=".">
            <include name="org.andrill.visualizer"/>
            <include name="org.andrill.visualizer.services*"/>
        </dirset>
    </subant>
</target>

Here you can see we build first the org.andrill.visualizer bundle and then all of the org.andrill.visualizer.services bundles. As we build each bundle, we copy the bundled JAR file to our dist.dir. Each time a bundle is built, it creates its classpath from all of the JARs in dist.dar. So even though there are dependencies among bundles, we are progressively fulfilling those dependencies by collecting the built bundles in dist.dir.

Once all of the "core" bundles are built, we can kick off the build of the ui bundles:

<target name="build-ui" depends="build-framework">
    <!-- build org.andrill.visualizer.ui* -->
    <subant target="build-plugin" genericantfile="build-plugin.xml" failonerror="false">
        <property name="dist.dir" value="../${build.dir}"/>
        <property name="osgi.framework" value="../framework.jar"/>
        <dirset dir=".">
            <include name="org.andrill.visualizer.ui*"/>
        </dirset>
    </subant>
</target>

Finally we can build all of the "application" bundles by excluding everything we've already built:

<target name="build-apps" depends="build-framework, build-ui">
    <subant target="build-plugin" genericantfile="build-plugin.xml" failonerror="false">
        <property name="dist.dir" value="../${build.dir}"/>
        <property name="osgi.framework" value="../framework.jar"/>
        <dirset dir=".">
            <include name="*.*"/>
            <exclude name="org.andrill.visualizer"/>
            <exclude name="org.andrill.visualizer.services*"/>
            <exclude name="org.andrill.visualizer.ui*"/>
            <exclude name="${build.dir}"/>
            <exclude name="${dist.dir}"/>
        </dirset>
    </subant>
</target>

You can check out the full build file at: build.xml and build-plugin.xml.

It's not nearly as neat as just kicking off a PDE build and letting it do all of the hard work of figuring out the dependencies for you. However, I'm rather fond of my approach because it keeps me honest. If I create a new bundle and it starts breaking the build, then I know I need to go back and make sure I've thought through the dependencies and am not trying to mix "core" code with "ui" code and such. And there's no need to bundle Eclipse with the source to build the thing.

Visualizer, Part 1: Introduction

On and off over the past couple of months, I've been working on an OpenGL visualization application, called Visualizer (original, I know), for viewing high resolution core imagery and data. Visualizer is built on and with numerous technologies: Java, OSGi, JOGL, Eclipse, Ant, etc. Along the way, I've collected a fair number of tips, tricks, and tidbits that I'm going to share here. I hope to make Visualizer posts a regular feature of the blog, at least until the tidbits run out. If you like it, please let me know by commenting.

A quick aside here: Those of you in geosciences might recognize this concept as Corelyzer. My goal was to create a simplified, streamlined version of Corelyzer that would be easy to deploy on laptops/desktops; a sort of Corelyzer "lite" edition if you will. Visualizer doesn't have nearly as many features as the current version of Corelyzer, but it also doesn't suffer some of Corelyzer's warts.

I thought I'd start off with quick introduction of what Visualizer is for folks who aren't familiar with Corelyzer. During scientific geological drilling, hundreds or thousands of core samples are taken, imaged, and analyzed. This results in several (tens of) gigabytes of data and imagery. Collecting all of this data is of little use unless you have a way to view it. This is where tools like Corelyzer and Visualizer come in. They provide a way to view the images and data in context:


This picture illustrates Corelyzer in action. Stretched across the two 30" cinema screens is a high resolution image of core drilled in Antarctica. The user can zoom in or out and pan left or right to view the all of the imagery collected on the expedition.

So now that you know a bit of the background, let's look at the requirements (in no particular order) I had for Visualizer:

• Cross platform - I have users on Mac, Windows, and Linux so I need a solution that will work for all of them.
• Easy to deploy - Visualizer should work out of the box with no complicated setup for most users.
• Easy to update - Needs a fairly automated mechanism to distribute updates and new features.
• Open, extensible platform - Building on an open, extensible platform will allow me or others to quickly add new features or customize to a specific groups needs.
• Reasonable performance - Visualizer should be able to handle 100m or so of imagery and data with reasonable performance on commodity hardware.

In the next blog post, I'll talk about the overall design of Visualizer and specific technologies employed. Future blog entries will include: "OSGi & Native Libraries", "Poor Man's Eclipse PDE Build", "Nifty Ant Tricks", "Automated Update Checks", and anything else that I can think of.

Cheers.