Josh (Formerly) In Antarctica

Griffon Plugins and Addons

2009-11-05T16:15:00.001-06:00

In this post, I'm going to talk a bit about Griffon plugins and addons--what they are, why they're useful, and how you can create your own. As an example, I'll walk through the steps used to create the recently released Griffon Mail Plugin.

What are Plugins and Addons?

Plugins provide a mechanism to extend your Griffon application with new functionality at build-time. They are commonly used to integrate external libraries, add new builders, and support testing tools. The full list of Griffon plugins is available here.
Addons are a new feature in Griffon 0.2 that provide mechanism to extend your Griffon application with new functionality at run-time. Addons can add items to builders, add MVC groups, and add/respond to runtime events. An example of this is the Guice plugin, which also bundles an addon that starts up the Guice framework and injects services into your controllers as they are created. Another example is the REST plugin which uses and addon to add new methods to controllers as they are created.

Plugins and Addons are conceptually very similar. Both are about adding new functionality and you will often see plugins and addons on working hand in hand.

Creating your own Plugins and Addons

Creating your own Griffon plugins and addons is not too difficult. We'll walk through the process by re-creating the Griffon mail plugin. So without further ado, lets get started:

Step 1: Create the plugin

griffon create-plugin mail

Step 2: Edit the plugin metadata

cd mail

edit MailGriffonPlugin.groovy which Griffon created for you:

class MailGriffonPlugin {
    def version = 0.1
    def canBeGlobal = false
    def dependsOn = [:]

    // TODO Fill in your own info
    def author = "Josh Reed"
    def authorEmail = "jareed@andrill.org"
    def title = "Send email from your Griffon app"
    def description = 'Send email from your Griffon app'

    // URL to the plugin's documentation
    def documentation = "http://griffon.codehaus.org/Mail+Plugin"
}

Step 3: Add our plugin functionality
What you do here will depend on your plugin, but in this case we need to add the mail.jar file from the JavaMail API site to the plugins lib/ directory. When the user installs our plugin, this jar file will be added to the application's classpath.

The JavaMail API requires a bit of work to send emails, so we'll also add a helper class that hides this code behind a single static method call.

Step 4: Package your plugin so you can test it
When you're done with your plugin, you will want to test it out. To do this:
griffon package-plugin
This will compile your plugin and package it as a zip file called griffon-mail-0.1.zip. You can then install this plugin in a regular Griffon application with a:
griffon install-plugin griffon-mail-0.1.zip

For the mail plugin, we can send an email by calling our helper class:
MailService.sendMail(mailhost: 'smtp.server.com', to: 'jareed@andrill.org'...)
from our controller or wherever. This is OK but we can do one better by creating an Addon that injects the sendMail method into every controller.

Step 5: Creating an Addon
Head back to our plugin directory and issue a:
griffon create-addon mail
this will create a MailGriffonAddon.groovy file.
(Note: if you try this with 0.2 you may run into an error about 'createMVC' This is a bug that has been fixed, but you can fix it yourself by editing your $GRIFFON_HOME/scripts/CreateAddon.groovy file. The last line should say 'createAddon' instead of 'createMVC')

This is where we do our runtime magic:

import griffon.util.IGriffonApplication

class MailGriffonAddon {
    private IGriffonApplication application
 
    // add ourselves as a listener so we can respond to events
    def addonInit = { app ->
        application = app
        app.addApplicationEventListener(this)
    }
 
    // inject the 'sendMail' into our 
    def onNewInstance = { klass, type, instance ->
        def types = application.config.griffon?.mail?.injectInto ?: ["controller"]
        if (!types.contains(type)) return
        instance.metaClass.sendMail = { Map args -> MailService.sendMail(args)}
    }
}

With this done, we can: griffon package-plugin and install the plugin into our test app: griffon install-plugin griffon-mail-0.1.zip
With the addon in place, we can reference the sendMail() method directly on our controller.

Step 6: Release the plugin
If you have write access to the Griffon Subversion, you can release your plugin with a:
griffon release-plugin
If you don't have write access, ask on the griffon-dev list and someone can help you out.

I hope this demystifies the plugin and addon building process.

Multi-Document Griffon Applications

2009-10-14T23:14:00.001-05:00

In this post we're going to build the skeleton of a multi-document (tabbed) Griffon application. Tabbed interfaces are extremely common. You're probably even viewing this in a web browser with a few open tabs as we speak. We'll be implementing the document-handling logic, so all you will need to do is add your own application-specific logic.

Let's dive in by creating a new Griffon application and creating a Document MVC group:

griffon create-app MultiDocumentExample
cd MultiDocumentExample
griffon create-mvc Document

In our application model, MultiDocumentExampleModel.groovy, we need to add some properties to keep track of open and active documents. We also need a separate 'state' class (more on that later):

import groovy.beans.Bindable

class MultiDocumentExampleModel {
    @Bindable Map activeDocument = null
    List openDocuments = []
    DocumentState state = new DocumentState()
}

@Bindable class DocumentState {
    boolean isDirty = false
}

Next up is the view: MultiDocumentExampleView.groovy. For this minimal example, we'll define some common actions and a tabbed pane for displaying the documents, but you can customize it to suit your application:

actions {
    action(
        id: 'newAction', 
        name:'New', 
        accelerator: shortcut('N'), 
        closure: controller.newAction
    )
    action(
        id: 'openAction', 
        name:'Open', 
        accelerator: shortcut('O'), 
        closure: controller.openAction
    )
    action(
        id: 'closeAction', 
        name:'Close', 
        accelerator: shortcut('W'), 
        closure: controller.closeAction, 
        enabled: bind { model.activeDocument != null }
    )
    action(
        id: 'saveAction', 
        name:'Save', 
        accelerator: shortcut('S'), 
        closure: controller.saveAction, 
        enabled: bind { model.state.isDirty }
    )
}

application(title: 'Multiple Document Example',
    size:[800,600], 
    //pack:true,
    //location:[50,50],
    locationByPlatform:true,
    iconImage: imageIcon('/griffon-icon-48x48.png').image,
    iconImages: [imageIcon('/griffon-icon-48x48.png').image,
        imageIcon('/griffon-icon-32x32.png').image,
        imageIcon('/griffon-icon-16x16.png').image],
    defaultCloseOperation: 0,
    windowClosing: { evt -> if (controller.canClose()) { app.shutdown() } }
) {
    menuBar(id: 'menuBar') {
        menu(text: 'File', mnemonic: 'F') {
            menuItem(newAction)
            menuItem(openAction)
            menuItem(closeAction)
            menuItem(saveAction)
        }
    }
 
    tabbedPane(id: 'documents', stateChanged: { evt -> 
        controller.activeDocumentChanged() 
    })
}

If you look close, you'll see I'm using the window closing trick I blogged awhile ago. The last piece of the puzzle is the controller: MultiDocumentExampleController.groovy. This is where we do the heavy lifting.

class MultiDocumentExampleController {
    int count = 0  // not strictly required
    def model
    def view

    void mvcGroupInit(Map args) {}

    void activeDocumentChanged() {
        // de-activate the existing active document
        if (model.activeDocument) { 
            model.activeDocument.controller.deactivate() 
        }

        // figure out the new active document
        int index = view.documents.selectedIndex
        model.activeDocument = index == -1 ? null : model.openDocuments[index]
        if (model.activeDocument) { 
            model.activeDocument.controller.activate(model.state) 
        }
    }
 
    boolean canClose() {
        return model.openDocuments.inject(true) { flag, doc -> 
            if (flag) flag &= doc.controller.close() }
    }

    def newAction = { evt = null ->
        // TODO: use an id that makes sense for your application, like a file name
        String id = "Document " + (count++) 
  
        def document = buildMVCGroup('Document', id, 
            tabs: view.documents, id: id, name: id /* TODO pass other args */)
        if (document.controller.open()) {
            model.openDocuments << document
            view.documents.addTab(document.model.name, document.view.root)
            view.documents.selectedIndex = view.documents.tabCount - 1
        } else {
            destroyMVCGroup(id)
        }
    }
 
    def openAction = { evt = null ->
        // TODO: pop up a open file dialog or whatever makes sense

        // TODO: use an id that makes sense for your application, like a file name
        String id = "Document" + (count++)

        // check to see if the document is already open
        def open = model.openDocuments.find { it.model.id == id }
        if (open) {
            view.documents.selectedIndex = model.openDocuments.indexOf(open)
        } else {
            def document = buildMVCGroup('Document', id, 
                tabs: view.documents, id: id, name: id /* TODO pass other args */)
            if (document.controller.open()) {
                model.openDocuments << document
                view.documents.addTab(document.model.name, document.view.root)
                view.documents.selectedIndex = view.documents.tabCount - 1
            } else {
                destroyMVCGroup(id)
            }
        }
    }
 
    def saveAction = { evt = null ->
        model.activeDocument.controller.save()
    }
 
    def closeAction = { evt = null ->
        if (model.activeDocument.controller.close()) {
            int index = model.openDocuments.indexOf(model.activeDocument)
            model.openDocuments.remove(index)
            view.documents.removeTabAt(index)
        }
    }
}

There's a fair amount of code but it's all pretty straightforward. The only thing that warrants further discussion is activeDocumentChanged(). This method is called whenever the active tab in the tabbed pane is changed. In it, we deactivate() the previously active document and then activate() the new document. When activating, we pass in the state object I mentioned earlier. The purpose of the state object is to allow us to use bindings that depend on the active document. For example, we really only want the save action to be enabled when we have a document open and the document is dirty. We can't bind directly to the active document's model because as soon as we open or switch to a new document, the binding is no longer correct. We can, however, bind to the state object and then sync the document's state in the activate() method (or other methods).

The implementation of the Document MVC group is straightforward:
DocumentModel.groovy

import groovy.beans.Bindable

@Bindable class DocumentModel {
    String id
    String name = "Untitled"
    boolean isDirty = false
    DocumentState state
}

DocumentView.groovy

panel(id: 'root') {
    // TODO: put your view implementation here
    button(text: 'Mark Dirty', enabled: bind { !model.isDirty }, 
        actionPerformed: { controller.markDirty() })
    button(text: 'Mark Clean', enabled: bind { model.isDirty }, 
        actionPerformed: { controller.markClean() })
}

DocumentController.groovy

import javax.swing.JOptionPane

/**
 * A skeleton controller for a 'document'.
 */
class DocumentController {
    def model
    def view
    def tabs

    void mvcGroupInit(Map args) {
        tabs = args.tabs
    }

    /**
     * Called when the tab with this document gains focus.
     */
    void activate(state) {
        // save the state object so we can signal the 
        model.state = state
  
        // TODO: sync the model and document state
        state.isDirty = model.isDirty
    }
 
    /**
     * Called when the tab with this document loses focus.
     */
    void deactivate() {
        // forget the state object
        model.state = null
    }
 
    /**
     * Called when the document is initially opened.
     */
    boolean open() {
        // TODO: perform any opening tasks
        return true 
    }
 
    /**
     * Called when the document is closed.
     */
    boolean close() {
        if (model.isDirty) {
            switch (JOptionPane.showConfirmDialog(app.appFrames[0], 
                    "Save changes to '${model.name}'?", "Example", 
                    JOptionPane.YES_NO_CANCEL_OPTION)){
                case JOptionPane.YES_OPTION: return save()
                case JOptionPane.NO_OPTION: return true
                case JOptionPane.CANCEL_OPTION: return false
            }
        }
        return true
    }
 
    /**
     * Called when the document is saved.
     */
    boolean save() {
        // TODO: perform any saving tasks
        markClean()
        return true
    }
 
    /**
     * Marks this document as 'dirty'
     */
    void markDirty() {
        model.isDirty = true
        if (model.state) { model.state.isDirty = true }
        // TODO: update any other model/state properties
        setTitle(model.name + "*")
    }
 
    /**
     * Marks this document as 'clean'
     */
    void markClean() {
        model.isDirty = false
        if (model.state) { model.state.isDirty = false }
        // TODO: update any other model/state properties
        setTitle(model.name)
    }
 
    /**
     * Sets this document's tab title.
     */
    void setTitle(title) {
        int index = tabs.indexOfComponent(view.root)
        if (index != -1) {
            tabs.setTitleAt(index, title)
        }
    }
}

So there it is. You should be able to use this as a starting place for your next Griffon-based text editor, web browser, etc. I'm personally using in PSICAT to manage the open diagrams:

I zipped up all of the code from this post and made it available for download.

Griffon Tip: Silly SwingBuilder Tricks

2009-10-12T15:28:00.001-05:00

Here's two quick tips for working with radio buttons in SwingBuilder. Radio buttons are created using SwingBuilder's radioButton() method. To ensure only one is selected at a time, you must also associate them with a button group. Ideally SwingBuilder would allow you to nest your radioButton()s inside a buttonGroup(), e.g.

buttonGroup() {
    // doesn't work
    radioButton(text: 'Option 1')
    radioButton(text: 'Option 2')
}

Unfortunately this doesn't work as buttonGroup() does not support nesting. Marc Hedlund stumbled across this same issue and offers up one solution. I like his solution but it means creating a separate variable to hold the button group. I've found a bit nicer way to do it using Groovy's with keyword:

buttonGroup().with {
    add radioButton(text: 'Option 1')
    add radioButton(text: 'Option 2')
}

The advantage of this approach is that it conveys our intention better and we don't have to instance extra variables.

EDIT: the 'with' also passes in the created object so you could take it a step further and replace the add as well:

buttonGroup().with { group ->
    radioButton(text: 'Option 1', buttonGroup: group)
    radioButton(text: 'Option 2', buttonGroup: group)
}

As a bonus, here's how you can use mutual binding to keep a boolean in your model in sync with the radio buttons. Let's say your model looks something like so:

class DemoModel {
    ...
    @Bindable boolean option1 = true
    ...
}

Using mutual binding, you can keep the model synced up without any explicit management code:

buttonGroup().with {
    add radioButton(text: 'Option 1', selected: bind(source: model, sourceProperty: 'option1', mutual:true))
    add radioButton(text: 'Option 2', selected: bind { !model.option1 })
}

Obviously this will only work for cases where you have an either-or choice.

Griffon Tip: MVC Groups Revisited

2009-10-06T22:43:00.001-05:00

My last post about MVC groups caused a bit of confusion, so I thought I'd follow up with a bit of clarification and, as a bonus, I'll also throw in a new trick for working with MVC groups.

MVC Groups for Dialogs
The withMVC method I introduced in the previous post was meant to help manage the lifecycle of short-lived MVC groups, such as those used in simple dialogs. In cases like this, the MVC group only has to exist for as long as the dialog is visible. withMVC was very much inspired by File#withInputStream which handles opening and closing of the input stream like withMVC handles creating and destroying of the MVC group. Below is a complete example of showing an MVC group as a dialog:

NewProjectWizardModel.groovy

@Bindable class NewProjectWizardModel {
    String name
    String filePath
}

NewProjectWizardView.groovy

panel(id:'options', layout: new MigLayout('fill'), border: etchedBorder()) { 
    // project name
    label('Name:')
    textField(text: bind(source: model, sourceProperty:'name', mutual:true), constraints: 'growx, span, wrap')
    
    // directory
    label('Directory:')
    textField(text: bind(source: model, sourceProperty:'filePath', mutual:true), constraints:'width min(200px), growx')
    button(action: browseAction, constraints: 'wrap')
}

NewProjectWizardController.groovy

class NewProjectWizardController {
    def model
    def view

    void mvcGroupInit(Map args) {}

    def actions = [
        browse': { evt = null ->
            def file = Dialogs.showSaveDirectoryDialog("New Project", null, app.appFrames[0])
            if (file) { 
                model.filePath = file.absolutePath
                if (!model.name) { model.name = file.name }
            }
        }
    ]

    def show() {
        if (Dialogs.showCustomDialog("Create New Project", view.options, app.appFrames[0])) {
            // perform some logic
        }
    }
}

Dialogs.showCustomDialog() just uses JOptionPane to show a custom dialog with a component from the view. I can show this dialog in response to a menu option, a button click, or whatever with the snippet from last post:

withMVC('NewProjectWizard', 'newProject', [:]) { mvc ->
    def project = mvc.controller.show()
    if (project) {
        // do something
    }
}

Hopefully that's a bit more clear and shows how MVC groups can be used as dialogs. Now let's look at another use of MVC groups: embedding them as components in other views.

Embedding MVC Groups in Views
MVC groups work well as dialogs but they can also be embedded directly in the views of other MVC groups. This is useful because it allows you to create re-usable components that you can embed in your views as needed. Embedding is relatively straightforward:

widget(id:'sidePanel', buildMVCGroup('Project', 'project').view.root)

We build the MVC group and grab a component from the view (in this case it was called 'root' but it could be any id). If we need to access something in the group, we can reference it by id:

def model = app.models['id']
def view = app.views['id']
def controller = app.controllers['id']

Hopefully these tips help you use MVC groups a bit more effectively and bring a bit more modularity to your Griffon app.

Griffon Tip: MVC Groups

2009-10-05T20:10:00.001-05:00

Recently I've been doing some refactoring of PSICAT to make better use of Griffon's MVC Groups. I like MVC Groups because they provide self-contained building blocks that you can re-use throughout your application. Below are a few tips that might be useful to others:

Creating MVC Groups
MVC Groups are created with the 'griffon create-mvc ' command. This creates the appropriate files in the griffon-app directory structure and registers the MVC Group in Application.groovy.

When you want to use the new MVC group in your application, you have two ways to instantiate it: createMVCGroup and buildMVCGroup. Both methods instantiate the MVC group but differ in the ordering of parameters passed to them and in what they return:

List[model, view, controller] createMVCGroup(type, id, paramMap)
Map['model': model, 'view':view, 'controller':controller, ...] buildMVCGroup(paramMap, type, id)

So if I had created an MVC group: 'griffon create-mvc Diagram', I could instantiate it in my application with either:
def list = createMVCGroup('Diagram', 'diagram1', [:])
or
def map = buildMVCGroup([:], 'Diagram', 'diagram1')

I was initially a bit confused as to why the parameters map was moved to the first argument for buildMVCGroup but then I remembered that Groovy lets you pass named parameters to a method and collects them all as a map. This allows some nice syntactic sugar when creating an MVC group:
def map = buildMVCGroup('Diagram', 'diagram1', name: 'The Diagram', format: 'jpeg')

The other nifty thing is that Griffon automagically sets parameters passed when creating an MVC group to properties on the model (if applicable). In the example above, if I had a name and a format properties on my model, they would be set to the values I passed instead of me having to explicitly set them in mvcGroupInit.

Destroying MVC Groups
While probably not critical, it's good practice to dispose of your MVC groups when you're done working with them. You can do this by calling destroyMVCGroup with the id you passed when creating the group. I find myself using many short-lived MVC groups. To ensure I disposed of things properly, I whipped up a little method that simplifies things:

def withMVC(String type, String id, Map params, Closure closure) {
    closure(buildMVCGroup(params, type, id))
    destroyMVCGroup(id)
}

NOTE: you could do the same trick of moving the map to the first argument if you wanted to be able to pass 'loose' named parameters to the method.

You would call it in your code like so:

withMVC('NewSectionDialog', 'newSection', [ project: model.project ]) { mvc ->
    def section = mvc.controller.show()
    if (section) { model.projectSections << section }
}

This creates the MVC group, hands it to the closure you pass, and then destroys the group when done.

MVC groups are a nice feature of Griffon that let you create re-usable building blocks within your application. Hopefully this tip helps you use them more effectively.

Happy Birthday Griffon

2009-09-10T12:45:00.000-05:00

Well it looks like I'm a bit late to the party as James, Jim, Guillaume, and Andres already beat me to it, but I'll say it anyways: Happy Birthday Griffon! As birthdays are a natural time for reflection, I thought I'd look through my blog archives to find my first Griffon post. Although not one of my most original titles, My First Griffon App debuted on September 17, 2008 marking me as a happy Griffon user since nearly the beginning. Since then I've written several other Griffon apps and a few more blog posts about Griffon.

It's been fun watching not only the project develop but also the enthusiastic community spring up around it. As Jim mentioned in his post, Griffon's presence at JavaOne this year was a major highlight. The fact that the major IDEs are adding Griffon support and Griffon in Action is set to release this March shows that Griffon truly has arrived.

So congrats to everyone involved. If you haven't tried Griffon yet, what are you waiting for? The upcoming 0.2 release is shaping up to be the best yet!

Griffon Tip: Mac Look and Feel on Snow Leopard

2009-09-09T11:28:00.002-05:00

If you upgraded your Mac to Snow Leopard recently, you may have noticed that your Griffon apps no longer look the same as they used to on Leopard. Aside from a few color differences, the biggest giveaway will be that your app no longer uses the standard top menubar. Fortunately there is a simple fix if you want to use the more native look and feel. Change your griffon-app/lifecycle/Initialize.groovy file to include the 'mac' look and feel first:

import groovy.swing.SwingBuilder
import griffon.util.GriffonPlatformHelper

GriffonPlatformHelper.tweakForNativePlatform(app)
SwingBuilder.lookAndFeel('mac', 'nimbus', 'gtk', ['metal', [boldFonts: false]])

The crux of the problem is SwingBuilder tries various L&Fs in the order you specify them and stops when it finds one that works. Nimbus is the L&F for Java 6 but previous versions of Mac OS X shipped with Java 5. So SwingBuilder would try Nimbus and it would fail then it would try the Mac look and feel which would work. Snow Leopard ships with Java 6 so when SwingBuilder tried Nimbus, it worked and it never tried to set the Mac L&F.

EDIT: This will be the default behavior in the upcoming Griffon 0.2 release. So you will only need this fix if you have Griffon apps from pre-0.2 days.

Griffon Tip: Intercepting Window Closing Events

2009-09-01T09:55:00.002-05:00

When building an editor of sorts, it is often useful to intercept the window closing event so that you can prompt the user to save if the editor has unsaved changes. To do this in Griffon, we first need to change the defaultCloseOperation and define a windowClosing event handler on our application node in the view:

application(title:'Editor', size:[800,600], locationByPlatform: true,
    defaultCloseOperation: WindowConstants.DO_NOTHING_ON_CLOSE, 
    windowClosing: { evt -> 
        // check if the editor needs saving
    }) {
// the rest of your view code here
}

If we run our application now, we should notice that...the windowClosing event handler never runs. Turns out that there's one more step--we need to tell Griffon that we want to explicitly handle shutting down the application. We do this by setting autoShutdown=false in griffon-app/conf/Application.groovy Once this is set, we should notice that our windowClosing event handler runs. Since Griffon is no longer responsible for automatically shutting down our application, we must make sure to call app.shutdown() somewhere in our closing logic or the user won't be able to exit the app.

Building your Griffon app with Gradle

2009-08-19T18:54:00.009-05:00

In this post, I'm going to show how to build your Griffon apps with Gradle. Griffon already comes with a command line tool to build and interact with your app, so why would you want to do this? I can think of a few reasons:

you have an existing Gradle build you want to integrate with
you have a complex build or packaging scenario
you want to take advantage of Gradle's dependency management for your app

Prerequisites

The build.gradle file

Below is a generic Gradle build file for Griffon applications. Either copy and paste the code below or download it and place it in a build.gradle in your Griffon application folder.

// standard configuration
project.captureStandardOutput(LogLevel.INFO)
configurations { compile }

// add additional repositories here if you need them
repositories {
    //mavenRepo(urls:'http://download.java.net/maven/2/')
    mavenCentral()
}

// add any dependencies you want downloaded as part of the bootstrap process
dependencies {
    //compile 'com.google.collections:google-collections:1.0-rc2'
}

// this should be run once for the project
createTask('bootstrap') << {
    // download any dependencies and put them in lib/
    configurations['compile']?.files?.findAll { it.absolutePath.indexOf('unspecified') < 0 }.each { dep ->
        ant.copy(toDir: new File('lib'), file: dep)
    }

    // try to install any plugins
    new File('application.properties').eachLine { line ->
        if (line.startsWith("plugins")) {
            def plugin = line[8..-1].split('=')
            griffon "install-plugin ${plugin[0]} ${plugin[1]}"
        }
    }
}

// mimic the tasks provided by the Java plugin
createTask('clean') << { griffon 'clean' }
createTask('compile') << { griffon 'compile' }
createTask('libs') << { griffon 'package' }
createTask('test') << { griffon 'test-app' }
createTask('dists') << { griffon 'prod package' }

// additional run-related tasks
createTask('run-app') << { griffon 'run-app' }
createTask('debug-app') << { griffon 'run-app -debug' }
createTask('run-webstart') << { griffon 'run-webstart' }
createTask('run-applet') << { griffon 'run-applet' }

// call out to the griffon command
def griffon(target) {
    if (System.getProperty("os.name").toLowerCase().startsWith("win")) {
        ant.exec(executable: 'griffon.bat', dir: projectDir, failonerror: true) {
            arg(value: target)
        }
    } else {
        ant.exec(executable: 'griffon', dir: projectDir, failonerror: true) {
            arg(value: target)
        }
    }
}

// if you want to use the gradle wrapper
createTask('wrapper', type: Wrapper).configure {
    gradleVersion = '0.7'
}

To verify that everything is working, let's try running our app from Gradle (substitute gradle.bat if you are on Windows):
gradle run-app

Your Griffon app should run. If it doesn't, try invoking the Gradle command with the '-i' flag:
gradle -i run-app
to see what the problem is.

Multi-project Builds

So far it may seem like "What's the point?" We could already run our Griffon app with the griffon command. Where Gradle really starts to shine is when you start wanting to perform complex builds, such as building multiple Griffon apps at once or integrating with an existing build.

Gradle has good support for multi-project builds. Assuming you are using a directory layout like this:

settings.gradle
griffonapp1/
    build.gradle
    ...
griffonapp2/
    build.gradle
    ...

Building these multiple Griffon apps is as simple as listing them in the settings.gradle file:

include 'griffonapp1', 'griffonapp2'

Similarly, integrating with an existing build should be as simple as adding the name of your Griffon app to your existing settings.gradle file.

Complex Build/Packaging Scenarios

Griffon offers several packaging options with the Installer plugin. Sometimes you need more flexibility. For my PSICAT build, I wanted to package two Griffon apps together in a specific directory structure. The nice thing about Gradle build files is that they are just Groovy code and offer great integration with Ant. Below is a snippet from the PSICAT build that illustrates using Ant within Gradle:

// griffon create-all-launchers for PSICAT and SchemeEditor
griffon(psicat, 'create-all-launchers')
griffon(schemeEditor, 'create-all-launchers')

// build linux package
ant.copy(toDir: new File(linux, 'PSICAT')) {
    fileset(dir: new File(psicat, 'installer/linux/dist')) {
        include(name: 'bin/**')
        include(name: 'lib/**')
    }
}
ant.copy(toDir: new File(linux, 'PSICAT/tools/SchemeEditor')) {
    fileset(dir: new File(schemeEditor, 'installer/linux/dist')) {
        include(name: 'bin/**')
        include(name: 'lib/**')
    }
}
ant.chmod(perm: "+x", dir: new File(linux, 'PSICAT/bin'), includes: '*')
ant.chmod(perm: "+x", dir: new File(linux, 'PSICAT/tools/SchemeEditor/bin'), includes: '*')
ant.tar(destFile: new File(linux, "PSICAT-linux-${version}.tar.gz"), compression: 'gzip', basedir: linux, includes: 'PSICAT/**')
...

(if you're curious, the whole build.gradle file is here)

Cool Gradle Tricks

One cool feature of Gradle is that you can generate a 'wrapper' script that will download and bootstrap Gradle itself so that developers who download your source don't have to have Gradle installed to build your project.

The build.gradle file above is already setup to create this wrapper. You simply have to run the command:

gradle wrapper

and it will create four files in your project:

gradle-wrapper.jar
gradle-wrapper.properties
gradlew
gradlew.bat

If you include these files when you send someone your project, they can use the gradlew or gradlew.bat scripts as if they had Gradle installed:

./gradlew run-app

The other neat trick with Gradle is you can use its ability to pull from Maven repositories to bootstrap the dependencies of your Griffon app. This means you don't need to send someone your Griffon app with a bunch of JARs in the lib directory. Instead you can send them a Gradle-ized version of your app and tell them to run the command:

./gradlew bootstrap

This will download all of the dependencies you've listed in the build.gradle file and put them in the lib directory of your Griffon app. Specifying dependencies is as simple as adding a few lines to the dependencies section of the build file:

dependencies {
    compile 'com.google.collections:google-collections:1.0-rc2'
    compile 'org.slf4j:slf4j-api:1.5.8'
}

Conclusion

Hopefully this has given you a taste of using Gradle with Griffon. Both are awesome tools that really show off the power of the Groovy language. If there's enough interest in using Gradle with Griffon, we can make build.gradle a standard artifact created by the griffon create-app task.

Are Groovy Stacktraces a Showstopper?

2009-08-15T14:20:00.006-05:00

I've been part of an interesting debate on the merits of Groovy via Twitter this week. I'm reproducing it here because I think it's an interesting take on what potential users feel are make or break issues for a language, and to give others who may not have seen it the chance to weigh in without the 140 character limit. :)

(Sorry in advance. Reproducing a Twitter convo in a blog is kind of hideous)

[@philswenson] if you want to know why groovy sucks, click here: http://pastie.org/583115
[@joshareed] @philswenson It took me a second to diagnose your problem. Did you want Groovy to say: "Hey joker, actually import the classes you're using"
[@joshareed] @philswenson Apparently Ruby has some magical mind-reading interpreter/compiler that saves you from yourself?

[@philswenson] @joshareed why the extra 500 lines of crap? in java or jruby that stacktrace would have been 1/50 the size. that's my point
[@joshareed] @philswenson re: Java guess that's the trade off for Groovy dynamics. Ruby's (http://bit.ly/IagLJ) is not any better [Note: Nambu mangled the original link]

[@philswenson] @joshareed Ruby's stack dump = 67 lines. Groovy's = 222 lines. Not as different as I said, but still...
[@joshareed] @philswenson # of lines doesn't matter when you see the cause of the prob in the first 10 lines, does it? You don't have to keep reading...
[@joshareed] @philswenson and was the 67 vs 222 equivalent programs in each? I'd be interested in comparing the source of each

[@philswenson] @joshareed this was a case where the cause was obvious (it usually isn't). isn't groovy in large part about succinctness over java?
[@joshareed] @philswenson how much time do you spend looking at stacktraces that this is hang up for you? Write a 3 line .findAll{} to filter traces
[@joshareed] @philswenson and if you want to stick with stacktrace length as your argument then you won't be plain old Java traces with JRuby

From my perspective, this isn't a JRuby vs Groovy debate even though both languages were brought up in the conversation. Both are two different ways to skin a cat but they ultimately get you to the same place. Which you choose to use is more a matter of personal preference than the actual merits of the language.

What I'm more interested in is how important/serious is the stacktrace issue that Phil Swenson argues? I've been writing Groovy for a couple years now and can't remember (off the top of my head) a stacktrace from Groovy that was so confusing I couldn't figure out the problem. I'm sure there are some, especially when you start getting deep into the Groovy innards, but I'm guessing that's also the case with other languages on the JVM, like JRuby.

All dynamic languages built on the JVM introduce a certain level of indirection (by necessity), so are Groovy's stacktraces that much more hideous than JRuby's? What qualifies as hideous? I think length is a naive measure because the cause of the problem is likely to show up in the first dozen lines. Who cares if it goes on for another 200 lines? Length is also easily manipulated. You could write a simple findAll closure that sanitizes the stacktrace by removing all the lines that start with 'sun.reflect' or 'org.codehaus.groovy.runtime'. I have never once run into a problem that could be tracked to one of those lines in the stacktrace. So now does that make Groovy better than JRuby because the stacktrace is shorter?

I'm sure this is just a case of me being used to seeing Groovy stacktraces, so I'm used to feel of them and can quickly hone in on the cause of the problem. Likewise I'm sure Phil is used to [J]Ruby errors which probably look much different than Groovy's. I say probably because the two example put forth didn't look all that dissimilar IMO but were not from the same problem so it was an apples to oranges comparison.

Anyways, how important are stacktraces when making the decision to use a particular language? Are they important enough to declare one language's superiority over another? Are they worth focusing on to improve Groovy's ease-of-use?

Griffon Installer Plugin

2009-08-12T13:44:00.006-05:00

Last week Andres Almiray and I released a new version of the Griffon Installer plugin. This version adds support for creating native Mac app bundles for your Griffon application, as well a few refactorings to make the generated artifacts more consistent. In this post, I'm going to walk you through the process of creating a set of native launchers for your application. A future post will explain the process of creating installers.

Getting started using the installer plugin is simple:
griffon install-plugin installer

This will fetch the plugin and install it into your Griffon app. The plugin adds several new targets. All of the launcher-related targets are broken into two steps: a prepare step that copies all of the configuration files for that particular launcher into your application, and a create step that actually generates the native launcher. The prepare targets only need to be run once for your project. They give you a chance to tweak the launcher configuration files. The create targets are run every time you want to make a release.

The plugin provides launchers for the following platforms:

Linux
Mac
Windows
Executable Jar

The plugin also provides two other convenient targets, prepare-all-launchers and create-all-launchers which invoke the prepare- and create- targets for the four platforms above.

We'll use these convenience targets to create launchers for all platforms:
griffon prepare-all-launchers

This will create several new files for you:

installer/jar - the executable JAR launcher directory
installer/jsmooth - the Windows launcher directory
installer/linux - the Linux launcher directory
installer/mac - the Mac launcher directory
installer/jar/MANIFEST.MF - customize the manifest of the executable JAR
installer/jsmooth/$appName-icon.png - customize the .exe icon
installer/jsmooth/$appName.jsmooth - customize the .exe options
installer/mac/$appName.icns - customize the Mac icon

Once you are done customizing the configuration files, invoke the
griffon create-all-launchers target to actually create the launchers. This will create all of the launchers in the installer/$platform/dist directories and create archives of the launchers:

installer/jar/dist/$appName-$version.jar - the executable jar
installer/linux/dist/bin/$appName - the Linux launcher shell script
installer/linux/dist/$appName-linux-$version.zip - a zip of the Linux launcher
installer/mac/dist/$appName-$version.dmg - a disk image of the Mac launcher [only created if on a Mac]
installer/mac/dist/$appName.app - the Mac application bundle
installer/jsmooth/dist/$appName-windows-$version.zip - a zip of the Windows launcher
installer/jsmooth/dist/$appName.exe - the Windows launcher
installer/windows/dist/$appName-windows-$version.zip - copied from the JSmooth directory
installer/windows/dist/$appName.exe - copied from the JSmooth directory

As you make changes to your code, simply invoke griffon create-all-launchers and your latest changes will be packaged up. If you increment the version number of your app, the created artifacts will be updated to match.

Hopefully the Installer plugin makes it easy to distribute your Griffon applications to users on various platforms. In a future blog post, I'll cover creating installers for when a simple zip file is not enough.

Griffon Plugin Development

2009-07-14T20:41:00.010-05:00

I finally got to the point in my Griffon application development that I was ready to release my app. Griffon has a plugin for creating installers and platform-specific launchers, but it didn't generate a proper Mac .app bundle. Being a Mac user myself, I really wanted to be able to deploy my app natively on Macs, so I decided to scratch my itch and add support to the plugin. I learned a fair bit about Griffon plugin development, so keep reading if you're interested in tweaking existing plugins or creating new plugins.

First off, Griffon plugins are simply Griffon applications with a few extra bits not normally used in a standard application—scripts for providing command line targets e.g., griffon create-mac-launcher, and interception of Griffon events like eventCleanEnd. To get started with plugin development, you'll want to decide whether you want to create a new plugin or edit an existing one.

Setting up your Griffon plugin development environment is fairly straightforward. If you want to edit an existing plugin:


cd MyExistingApplication
griffon install-plugin installer

If you want to create a new plugin and install it in an existing application:


griffon create-plugin MyNewPlugin
cd MyExistingApplication
{edit application.properties to include a line 'plugin.my-new-plugin=0.1'}

The real trick for both of these scenarios is to link the plugin source to somewhere where you can edit it live. This makes testing a whole lot easier. I had to pull a few strings to get this tip, but I'm going to share it with you for free:
Go into your ~/.griffon/{griffon version}/projects/MyExistingApplication/plugins/ and either edit the plugin directly or symlink it to something more convenient:


cd ~/.griffon/{griffon version}/projects/MyExistingApplication/plugins/
rm -rf installer-0.3
ln -s ~/Workspace/griffon/griffon-installer installer-0.3

You just have to make sure the name and the version match what is in your MyExistingApplication/application.properties file. If you installed an existing plugin, you should be good to go. If you're creating a new plugin, it defaults to 0.1.

Once you have your plugin development environment setup, you're ready to go to town. Below are a few things I learned along the way. Most have to do with Gant scripting, which I wasn't very familiar with before this but is easy to pick up.

If you want to create a new command line option, e.g. griffon say-hello-world you need to create a new script in your plugin. You can do this by creating a SayHelloWorld.groovy file in the $plugin/scripts. Or as Andres points out in the comments a simple griffon create-script SayHelloWorld works as well. Even better! You also have to define a sayHelloWorld target in your script and make it the default target:


target(sayHelloWorld:"Says 'Hello World'") {
  // any Groovy is valid here
  println "Hello World!"
}
setDefaultTarget(sayHelloWorld)

Another useful feature is the ability to invoke targets in other strips, such as if you wanted to make sure the Griffon app was built before your task was executed. You can accomplish this by including the targets of other scripts in your script:


includeTargets << griffonScript("_GriffonPackage")  // script name minus the .groovy

target(doAfterBuild: "Build and then run me") {
  depends(checkVersion, packageApp, classpath)  // defined by _GriffonPackage.groovy
  packageApp()  // calls the packageApp target in _GriffonPackage.groovy
  println "The JARs in $basedir/staging should be fresh!" 
}

setDefaultTarget(doAfterBuild)

Finally, before this post gets too long-winded, I'll cover the topic of events. Events are fired by the Griffon core runtime and by plugins in response to actions in the workflow. They provide an extension mechanism for plugins who wish to respond to various actions. Your plugin can intercept events by providing a $plugin/scripts/_Events.groovy file which declares interest in specific events:


eventCleanEnd = {
  // jar
  ant.delete(dir:"${basedir}/installer/jar/dist", failonerror:false)
  ant.delete(dir:"${basedir}/installer/jar/classes", failonerror:false)
 
  // jsmooth
  ant.delete(dir:"${basedir}/installer/jsmooth/dist", failonerror:false)
 
  // linux
  ant.delete(dir:"${basedir}/installer/linux/dist", failonerror:false)
 
  // mac
  ant.delete(dir:"${basedir}/installer/mac/dist", failonerror:false)
 
  // windows
  ant.delete(dir:"${basedir}/installer/windows/dist", failonerror:false)
}

If you are developing your own plugin, you should consider publishing events so that other plugins can hook into your workflow. You can do this by including the _GriffonEvents targets in your script, or by including any script that itself includes _GriffonEvents:


includeTargets << griffonScript("_GriffonEvents")

target(myPluginTask: "") {
  event("MyPluginTaskStart", [])
  println "My Plugin Task!" 
  event("MyPluginTaskEnd", [])
}

setDefaultTarget(myPluginTask)

Thanks for following along. Any suggestions or questions are greatly appreciated!

Oh and if you're interested in the installer plugin stuff, check out: http://dev.psicat.org/ for some example outputs. These are not the exact outputs as I am bundling two Griffon apps--the main PSICAT one and a second one in tools/--but you get the idea.

Griffon Action Patterns

2009-07-02T18:27:00.006-05:00

I've been working on a decent-sized Griffon app recently and hit upon a pattern for managing actions that I thought might be useful to others. I began by following SwingPad's lead and declaring my actions separate (PSICATActions.groovy) from my view (PSICATView.groovy):


actions {
    action(id: 'exitAction', name: 'Exit', closure: controller.exit)
    action(id: 'openAction', name: 'Open', closure: controller.open)
    ...
}

which I import into my view:


// build actions
build(PSICATActions)

application(title:'PSICAT', size:[800,600], locationByPlatform: true, layout: new MigLayout('fill')) {
    ...
}

This is nice because it keeps all of your action definitions in one place and doesn't clutter your view code with them. When implementing the actions, the common pattern seems to define closures on your controller:


class PSICATController {
   def model
   def view

   void mvcGroupInit(Map args) {
       ...
   }

   def exit = { evt = null ->
       ...
   }

   def open = { evt = null ->
       ...
   }
}

This is fine but when you start getting many actions your controller becomes cluttered and it is difficult to tell what is an action implementation and what is regular controller logic. You also have to be careful with naming:


actions {
    action(id: 'exitAction', name: 'Exit', closure: controller.exit)
    action(id: 'openAction', name: 'Open', closure: controller.open)
    action(id: 'newAction', name: 'New', closure: controller.new) // ERROR
}

We are forced to break with our naming convention because 'new' is a reserved keyword. To combat the clutter and avoid naming issues, I've taken to declaring my action implementations in a map on the controller:


class PSICATController {
   def model
   def view

   void mvcGroupInit(Map args) {
       ...
   }

   def actions = [
       'exit': { evt = null -> ... },
       'open': { evt = null -> ... },
       'new':  { evt = null -> ... }  // no issues here
   ]
}

Using the map makes it easy to tell what is an action and what is not, and linking the action definitions to the implementations becomes elegant and intuitive:


actions {
    action(id: 'exitAction', name: 'Exit', closure: controller.actions['exit'])
    action(id: 'openAction', name: 'Open', closure: controller.actions['open'])
    action(id: 'newAction', name: 'New', closure: controller.actions['new'])
}

So far I have been pretty happy with this approach. What do other Griffon developers out there think?

Understudy

2009-02-27T12:46:00.002-06:00

A few days ago I stumbled across Understudy which lets you watch Hulu and Netflix from inside Front Row on your Mac. As there wasn't much on TV last night, I decided to test it out. I have an old MacBook Pro whose screen backlight died but is otherwise fully functional. I threw a copy of Understudy on it and hooked it up to the TV in the living room. Much to my delight it worked like a charm. I could use my Apple Remote to browse and navigate the shows and movies in my Hulu and Netflix queues. The only real issue I had was that I couldn't always trigger full screen mode in Hulu via the remote. This is a known bug so I expect it will eventually be fixed.

Updates

2009-02-24T10:47:00.003-06:00

My blogging has sort of fallen to the wayside since I started using Twitter. If you want to keep up with what I'm doing day to day (and don't mind a bit lower signal-to-noise ratio), check out my Twitter feed. You can follow along without signing up, but it's more fun if you have an account and we can interact.

Here's a whirlwind tour of the last three months for me:

Work

Work with ANDRILL is still going well. I seem to be doing something different each day, which is some times nice and some times a burden. I've got enough projects (both internal and external collaborations) going to keep me busy around the clock. I have to re-interview for my position in June (which I'm told is just a formality). Such is the way of university politics and being dependent on public funding, I guess.

My traveling has slowed down somewhat. I was in Lincoln in November, San Francisco in December for AGU, and Wellington, NZ this month for a meeting. I currently have no trips in the forseeable future, though they have a tendency to pop up on short notice.

I've got a manuscript submitted to Computers and Geosciences that reviewed favorably. Hopefully one more short round of revisions and it will be published. Then I can move on to the several papers that are in the 'needs revision' or 'just plain needs writing' stage. I've not been very good about forcing myself to write.

I helped write a fairly ambitious proposal for NSF's CDI solicitation. We're still waiting to hear how it reviewed, but it would be a lot of fun work if funded.

Personal

Things are still going well for Elizabeth at work. The last 3 months have been comparatively 'easy' for her. Easy being relative but she's not had as many rotations which require to spend the night at the hospital on call. Currently she's back into the call rotations, so she's gone every 4 days or so--she tells me it's payback for all of my travel.

The house is still standing. We had a minor pipe freeze a couple months back before we realized that there was an on/off lever for heating the basement. As you can probably guess, it was off. No problems since.

We're hoping to put in a nice concrete patio out back this spring. This step 1 of our renovation plans. Step 2 is to replace the bank of windows in the basement with a set of French doors to walk out onto the patio. Step 3 is to convert some space in the basement to a proper bathroom with a shower. Once all that is done, we can start looking at gutting and renovating the bathroom upstairs. Well down the road will be kitchen.

We're all excited here for spring. Bella likes the snow but it will be nice to be able to take her to the lake for a swim. It'll also be nice to setup the hammock again so I can blog from the backyard. I think it's going to be a busy spring cleaning up things around the yard that we didn't get done last fall.

I'm still brewing beer. Right now I've got a Strong Ale and a Brown Ale (my own recipe) bottled and the rest of the brown on tap. There's a Maibock fermenting away in one of my primaries and something (can't remember it offhand) on deck to brew probably in a week or two. I'm also still exploring single malt Scotches. I've got a bottle of Lagavulin 16YO that Elizabeth gave me for Christmas and a bottle of The Macallan 12YO open at the moment. Between the beer on tap, the whisky, and the new patio, this should be a fun place to grill out this summer.

That's probably enough for now. I need to blog more often, if only to avoid these long rambling posts.

RESTful Query URLs (Cont.)

2008-11-18T23:28:00.001-06:00

As a follow up on my previous RESTful Query URLs post, I'm going to be looking at implementing RESTful querying to the JSON repository I developed. The repository stores JSON documents and is searchable using a 'query-by-example' approach, e.g. you provide a document with the key/value combinations you're interested in and the repository returns all documents that match.

Since a query involves a template document, the simplest approach would be to provide a /search URI and let the client POST their query to that. My main hangup with this approach is the lack of linkability, e.g. I can't email/im/twitter a URI to the results of a query.

Doug talks about this in his recent REST via URI's and Body Representation blog post. In it, he suggests an approach where the client would POST the query/payload and recieve a 201 and a GETable URI with the results. This has some interesting implications (as Doug points out). How long does the /resource/request/[id] stay around? Presumably it could stick around indefinitely or until whatever is being queried changes. Do two clients POSTing the same body payload get the same results id. If you're going to support this, then you'll either have to query to see if the request has already been assigned an id or you're going to have to assign the ids based on the contents of the request, perhaps a SHA hash of the body payload. In either case, you're going to have to store the original request along with the id you've assigned to it.

I think this is why URI-encoding appeals to me: I don't have to keep any extra state around because I can re-create the request from the URI. This falls down when you need more expressive request capabilities than URI-encoding allows. I can also see an advantage in Doug's approach if the majority of your interactions are going to be workflow-based rather than single shot queries.

For the interface to the JSON repository, I chose to represent queries via the URI. I even went so far as to avoid using the query string, opting rather to put everything into the URI structure. The choice to do this was mainly one of exploration. I wanted to see if it offered any advantages (readability, cachability, simplicity) over using the query string.

The URIs take the form of: /collection[/term/value(s)]+ where term is either a direct property/key in the desired JSON document or a derived property (such as 'fulltext' which looks at the full text index of the document). The value section can either be a single value or a comma separated list of values. Some annotated examples include:

/and2 - return all documents in the and2 collection
/and2/1 - return the document with id = 1 in the and2 collection (special case)
/and2/type/image.SplitCore - return all documents with a type property of 'image.SplitCore'
/and2/fulltext/calcite,calcareous,carbonate - return any documents that contain 'calcite' OR 'calcareous' OR 'carbonate'
/and2/depth/100,200 - return any documents between the depth 100 and the depth 200. This changes the semantics of the comma operator as it no longer means the OR as it did with the fulltext term. If you pass in only one depth, it only returns documents at exactly that depth. If you pass in more than two depths, then the additional depths are ignored.

Multiple terms can be chained together:

/and2/type/image.SplitCore/depth/100,200 - return any documents of type 'image.SplitCore' AND between depth 100 and 200.
/and2/fulltext/calcite/fulltext/carbonate - return any documents containing 'calcite' and containing 'carbonate'

And I've added some special query operators:

/and2/type/!psicat.* - return any documents not of any psicat type, e.g. this would exclude documents with type properties of 'psicat.Interval', 'psicat.Unit', and 'psicat.Occurrence'.

What I like about this approach is that I can build a URI to just about any subset of documents in the collection (though it does require some prior knowledge of the structure). There are a few warts though. For one, it requires that URI components occur in pairs, so you can't peel back the URI like an onion: e.g. /and2/type/image.SplitCore is valid but /and2/type doesn't make sense. There is also an issue of canonicality, e.g. /and2/type/image.SplitCore/depth/100,200 will always return the same results as /and2/depth/100,200/type/image.SplitCore but they appear as separate resources to the caching layer. There's also aesthetics. I don't yet know how I feel about commas in the URL; they look weird to me. The scheme also doesn't pluralize the terms when multiple values are sent, e.g. /and2/types/image.SplitCore,image.WholeCore.

I'd love to hear any feedback on what you think of this approach.

Encounters at the End of the World

2008-11-18T22:24:00.003-06:00

I just got my copy of Encounters at the End of the World. If you're interested in Antarctica and what it's like to live and work there, then I highly recommend this movie. It was actually filmed while I was down there the first time, but I was far too busy to put in a cameo. :) It will probably come off as a little out there, but it's a good representation of the people and life down there.

ImageMagick DSL 2

2008-11-17T10:03:00.002-06:00

This is a quick post that shows another way to work with the ImageMagick DSL (and other Java DSLs). It comes from a trick I saw in a DSL talk given by Neal Ford. Basically you can use initializer blocks to construct objects in a bit less verbose way:


new ImageMagick() {{
 option("-rotate", "90");
 option("-resize", width + "x");
}}.run(in, out);

This still requires external variables, such as width, need to be declared final. So to wrap up, here's three different ways to invoke the DSL:

Standard:


ImageMagick convert = new ImageMagick();
convert.option("-rotate", "90");
convert.option("-resize", width + "x");
convert.run(in, out);

Method Chaining:


new ImageMagick().option("-rotate", "90").option("-resize", width + "x").run(in, out);

Initializer Block:


new ImageMagick() {{
 option("-rotate", "90");
 option("-resize", width + "x");
}}.run(in, out);

ImageMagick DSL

2008-11-07T09:58:00.004-06:00

I've been fighting with JAI/Java2D over the last day or two to manipulate (resize, crop, composite) some large images. I have working code that produces decent quality images, but I really have to crank up the heap space to avoid OutOfMemoryExceptions. If I try to process more than one or two images concurrently, OutOfMemoryExceptions are inevitable. Since this code is going to be called from a servlet, I'm expecting to handle multiple concurrent requests.

This is not a new problem and people have been tackling it in various ways. Since I was working in a server environment and have control over what applications are installed, I decided to use ImageMagick for the image manipulation. ImageMagick is great; I've used it quite often in various shell scripts.

There's basically two ways to work with ImageMagick from Java. You can use JMagick, a JNI layer over the ImageMagick interface, or you can use Runtime.exec() to call the ImageMagick command line application. I opted for the latter as it seemed simpler when I pushed the code from my Mac to my Linux server.

Since finding and invoking ImageMagick's convert command can be somewhat problematic, I decided to write a simple fluent API in Java to hide the details. The result allows you to invoke convert using method chaining:


ImageMagick convert = new ImageMagick(Arrays.asList("/opt/local/bin"));
convert.in(new File("in.jpeg"))
 .option("-fuzz", "15%")
 .option("-trim")
 .out(new File("out.jpeg")).run();
convert.option("-resize", "250x").run(new File("in.jpeg"), new File("out.jpeg"));

You create a new ImageMagick object. As a convenience, you can pass in a list of additional paths to check for the convert command in the event that it isn't on the default path. If the convert command can't be found, the constructor throws an IllegalArgumentException.

Once you have an ImageMagick object, you can execute convert by chaining various method calls, ending in a run(). run() returns true if the command succeeds, false otherwise.

In less than 200 lines of Java code, I had a much nicer way to interact with ImageMagick. A fun experiment would be to take the code and implement an even nicer DSL in Groovy. methodMissing() would allow fluent method chaining on steroids:


convert.fuzz("15%").trim().run(in, out)

As Guillaume Laforge tweeted, using metaprogramming, categories, and syntactic sugar like named parameters you could end up with a full blown DSL that looks like this:


convert fuzz:15.pct, trim: true, in: file, out:file

RESTful Query URLs

2008-10-31T18:53:00.005-05:00

The last couple of days I've been working on writing a RESTful JSON document database. While a number of these already exist (CouchDB, FeatherDB, DovetailDB, Persevere, JSONStore, etc.), I decided to write my own because I wanted a bit more control over the URL scheme used by the REST interface, and I needed the ability to tweak the search functionality to achieve decent performance on some common but complicated queries. All in all it was an interesting diversion. The actual server clocked in at about 1000 SLOC, with much of that boilerplate because I wrote it in Java/JDBC rather than Groovy/GroovySQL.

The most interesting problem came in designing the query scheme for the REST interface. There seems to be a couple different ways to implement it with no real consensus as to which is the "right" way. As with most things, I suspect it depends on how you've implemented other pieces of the architecture and even personal preference. Below I describe three approaches I considered. The nice thing with REST is there's nothing stopping you from implementing all of these approaches in your interface.

NB: I'm no REST expert so the information below is my observations rather than any best practices. I'd love for anyone who knows better to chime into the discussion.

POST query parameters/document
In this approach, you provide a search endpoint, say something unoriginal like '/search', and queries are POSTed to that URI. The query is either a set of form encoded key-value pairs or a search document using a schema shared between the client and server.

This approach seems closer to RPC than REST to me, but may be the best approach if your search functionality requires a more complex exchange of information than simple key-value pairs allow. The obvious downside to this approach is that there is no way to bookmark a query or email/IM a query to someone else. This approach also can't take advantage of the caching built into the HTTP spec.

GET query string
Similar to above, you expose a URI endpoint, possibly something like /search, and queries are sent to that endpoint with the parameters encoded in the query string of the URL, e.g. http://www.google.com/search?q=REST+query+string

This approach improves on the bookmarkability of searches, since all of the parameters are in the URL. However, the use of the query string may interfere with caching as described in Section 13.9 of the HTTP spec. Overall, I think there is nothing inherently un-RESTful about this approach, especially if you provide more resource-oriented URIs than /search, e.g. /documents?author=Reed. In my head, I interpret the latter as "give me all of the document resources but filter on the author Reed. Removing the query string will still give you a resource (or collection of resources in this case).

Where this approach falls down is when you start trying to represent hierarchical or taxonomic queries with the query string, e.g. http://lifeforms.org?k=kingdom&p=phylum&c=class&o=order&f=family&g=genus&s=species as described on the RestWiki.

Encoding query parameters into the URI structure
In this approach the query parameters are encoded directly into the URI structure, e.g. /documents/authors/Reed, rather than using the query string. Another example of is described at Stack Overflow.

This approach solves both the bookmarkability and the caching issues of the previous approaches, but can introduce some ambiguity, especially if your resources aren't strictly hierarchical in nature. The biggest stumbling block for me was this: looking at the URI /documents/authors/Reed, it's not immediately clear what will be returned. For example, if I sent you the URI /documents you might infer that you would get a list or the contents of some documents. From the URI /documents?author=Reed, you might infer that the resource(s) returned would be documents authored by Reed. So what might you expect to get from the URI /documents/authors/Reed? Information about the author Reed or all documents authored by Reed?

How important is this? I guess it's really up to you. A machine likely infers about as much from
/documents/authors/Reed as it does from /documents?author=Reed.

Core Gallery

2008-10-09T09:46:00.008-05:00

It seems like every couple of months I end up with a project that involves a fair amount of Javascript. Back in March it was working with Simile Timeline to visualize depth-based data. This time around, I wanted to create a lightweight way to visualize our drill core imagery. We already have full-featured visualization tools that scientists use, so I was looking to create something simple that would engage non-geologists.

The result is the Core Gallery. It shows an animated whole core image next to a split core image. Since the images are too large to display on the screen, there's a slider that lets you see different parts of the core. The page also displays some additional information about the core.

I'm really happy how it turned out. The page is 100% HTML, Javascript, and CSS. No Flash and no Java. For the Javascript, I'm using JQuery for no reason other than I wanted to see how it stacked up to other JS libraries I've used. It was perfect for this project and a treat to work with. Below I'm going to sketch out how various parts of the page are built.

Core Slider
The core slider is the most complicated part of the page. It uses the JQuery UI/Slider component. I used this screencast to help me acquaint myself with the slider. To achieve the highlighted core effect as the slider handle moves, I used two thumbnails of the core. One thumbnail is regular and one is washed out. I set the washed out thumbnail as a CSS background image on the slider element. I set the regular thumbnail as a CSS background image on the slider handle. The handle has a fixed size based on the height of the thumbnail vs. the height of the real core images so only part of the thumbnail is shown. From the slider's slide() callback, I simply update the CSS background-position property on the handle to ensure that handle's image is showing the same portion of the core as the underlying slider. I use this same technique to move the rotating whole core and split core images, taking the difference in image height between the thumbnail and the other core images into account.

Animated Whole Core Image
The slider was the most complicated but the animated whole core image was the most challenging. I wanted show the image animated in faux 3D. I initially started with a Java applet using JOGL. The applet worked on my Mac but not on Windows or Linux, so I abandoned it. I then got the idea to employ the CSS Sprites technique. So I used a tool to render the 3D whole core image 90 times each rotated by 4 degrees and montaged them together. Once I had this, it was simply a matter of setting up a Javascript Timer interval to fire every 50ms and move the image right by a fixed amount each time. This simulates animation fairly effectively. I keep track of the current rotation and vertical offset in global variables so the core keeps rotating when you move the slider.

Split Core Image
I use the same technique as on the slider handle to make the image track the slider's position.

Core Links
In the text description, it is possible to link to different parts of the core. This is a somewhat neat trick. To accomplish it, I wrap portions of the description text in span tags. Each span tag has an id attribute in the form of a ratio between 0.0 and 1.0. Using JQuery, I find these special span tags and add an onClick handler that updates the slider position based on the span's id attribute. So if the span had an id of 0.8, clicking on it would move the slider to the 80% position of the core. 0.0 takes you to the top and 1.0 takes you to the bottom.

Conclusion
Overall the Core Gallery turned out surprisingly well for being 100% browser-based. It took much less work than I originally envisioned thanks to JQuery. I'd definitely consider JQuery for future projects.

My First Griffon App

2008-09-17T16:43:00.005-05:00

Sorry it's been so long since I posted here. Work keeps me busy.

Recently I had been given several GB of raw data from our two most recent scientific drilling expeditions in Antarctica. This data needs a fair amount of quality control processing to turn it into a usable datasets for the scientists. To do this, I needed to write a tool for the drillers to interactively plot and explore the data to determine regions of interest. Given the recent buzz about Griffon, I thought I'd give it a try.

I started by downloading and installing Griffon. Once I had everything setup, I created an app:


griffon create-app DrillingAnalytics

If you've done any Grails development, this will be a familiar idiom to you. The results of this command is a straightforward directory structure, focused around the MVC pattern. You'll recognize directories for models, views, and controllers.

My next step was to flesh out my model. When you create an app, Griffon automatically creates a model class called ${app.name}Model (

DrillingAnalyticsModel for me) in the griffon-app/models directory.  The main purpose of my app is to plot time series data so I defined two fields, startDate and endDate in my model:

import groovy.beans.Bindable

class DrillingAnalyticsModel {
 @Bindable String startDate = "2006-11-07 00:00"
 @Bindable String endDate = "2006-11-08 00:00"
}



You'll notice the @Bindable annotations on these fields.  These fields will tie to these components in the UI, and the @Bindable annotation will automatically take care of keeping the UI in sync with the model via PropertyChangeEvents.

The model class is also where you can put other fields to maintain applications state:
 
 def plot = new CombinedDomainXYPlot(new DateAxis()) 
 def subplots = []
 def chart = new JFreeChart(null, JFreeChart.DEFAULT_TITLE_FONT, plot, false)


With the model sorted out, I moved on to developing the view.  As with model, Griffon creates a ${app.name}View class for you in the griffon-app/views directory.  Griffon puts the full power of SwingBuilder, SwingXBuilder, and GraphicsBuilder (with more on the way) at your fingertips for developing the UI. 

I spent the majority of my time on the UI.  It was a seemingly endless cycle of  tweaking the code and testing with griffon run-app to get it to look the way I wanted.  This is no knock on Griffon; writing Java UIs, especially by hand, just plain sucks.  

After far too long trying to get the standard Java layout managers to do what I want, I did myself a favor and downloaded MigLayout.  Despite not being built into SwingBuilder, MigLayout integrates nicely with SwingBuilder:

application(title:'Drilling Analytics', pack:true, locationByPlatform:true) {
   panel(layout: new MigLayout('fill')) {
    // chart panel
    widget(chartPanel, constraints:'span, grow')
    
    // our runs and time
    panel(layout: new MigLayout('fill'), border: titledBorder('Time'), constraints: 'grow 100 1') {
     scrollPane(constraints:'span 3 2, growx, h 75px') {
      runs = list(listData: model.mis.keySet().toArray())
     }
     label('Start:', constraints: 'right, gapbefore 50px')
     textField(id:"startDate", text: bind { model.startDate }, action: plotAction, constraints:'wrap, right, growx')
     label('End:', constraints: 'right, top')
     textField(id:"endDate", text: bind { model.endDate }, action: plotAction, constraints:'wrap, right, top, growx')
     label("+/-", constraints: 'right')
     textField(id:"padding", text: "30", constraints: 'growx')
     label("min")
     button(action: plotAction, constraints:'span 2, bottom, right')
    }
        
    // our plots panel
    panel(layout: new MigLayout(), border: titledBorder('Plots'), constraints: 'grow 100 1') {
     model.data.each { id, map ->
      checkBox(id: id, selected: false, action: plotAction, text: map.title, constraints:'wrap')
     }
    }
   }
}


SwingBuilder gets rid of all the boilerplate code and MigLayout makes it possible to code decent Java UIs by hand:


We've covered the Model and the View, now it's time to focus on the Controller.  The controller mediates between the model and view.  It contains all of the logic for handling events from the UI and manipulating the model.

One common pattern in the existing Griffon examples is the use of Swing Action objects to trigger actions from the UI.  My UI was pretty simple so I could reuse a single action on all of the components to refresh the plot:

actions {
 action(id: 'plotAction', 
  name: 'Update', 
  closure: controller.plot)
}


I put this code in my DrillingAnalyticsView class, but it could just as easily be defined in its own file and imported into the view via the build() method.  You'll notice that I give the action an id--plotAction--which I use to reference it from the components:

 button(action: plotAction, constraints:'span 2, bottom, right')


You can also see that the action just delegates to the controller.plot closure.  This is convenient because it keeps all of the logic in one place and the controller has access to both the model and view.  The actual code of the controller.plot is unremarkable.  The big consideration is to properly manage your threading.  Don't do long running actions in the EDT as it will freeze the UI, and don't update the UI from outside the EDT as Swing is not thread safe.  Andres Almiray has a good description of how Griffon makes this easy. 

Since my app is fairly niche (I doubt there's many of you visualizing drilling data), I'm not going to post the whole source code here.  However, I want to point out that the source code consists of just 327 lines of code, and that's including blank lines and comments!  The bulk of that code is the logic to query the database and update the JFreeChart plots.  This truly demonstrates how simple and easy it is to build an app with Griffon.  

If you're looking for more Griffon examples, check out the samples included in the samples directory of the Griffon distribution, and keep an eye on Griffon posts groovy.dzone.com



OSGi Command Line Applications
2008-08-06T13:15:00.004-05:00
I'm a big fan of OSGi.  One thing I always wanted to do but never got around to implementing until just recently was to be able to call services in an OSGi application from the command line.  I've often wanted to be able to script PSICAT instead of having to fire it up and interact with the GUI.  Turns out it's not all that difficult; you just need to sit down and do it.  The only snag I ran into was that I couldn't find an implementation-agnostic way of accomplishing this, so the code I'm going to show is for the Equinox OSGi implementation.  Though the same could easily be accomplished in Felix or likely other implementations with minor changes.

As with most things, there are multiple ways to skin a cat.  The route I chose was to embed Equinox in a Java app and mediate command line access through this class.  Fortunately, most of the work is already done for us via the EcliseStarter class (if you're on Felix, check out this).  Assuming Equinox is on your classpath, simply calling EclipseStarter#startup() will fire up the Equinox runtime.  More importantly, it will give you a BundleContext which you can use to interact with the OSGi framework.  Once we have a BundleContext, we can do interesting things like install and start additional bundles:

 public static void main(final String[] args) throws Exception {
  // start the framework
  context = EclipseStarter.startup(new String[0], null);

  // install all bundles
  installAllPlugins();

  // start our platform bundles
  startPlugin("org.eclipse.core.runtime");

  // start plugins
  for (Bundle b : context.getBundles()) {
   startPlugin(b.getSymbolicName());
  }
...


The final piece is to do the command line interaction.  For this, I created an interface that bundles can publish services under to make them available to the command line:

public interface ICommand {
 /**
  * Execute this command.
  * 
  * @param args
  *            the args.
  * @return the return value.
  */
 Object execute(String[] args) throws Exception;

 /**
  * Gets the help text that explains this command.
  * 
  * @return the help text.
  */
 String getHelp();
}


Unfortunately since there is a lot of classloader magic going on, we can't just get these ICommand classes from the service registry and invoke them directly (like we would do from inside OSGi).  The OSGi classes are on a different classloader than the one we started things on.  At first this may seem annoying but its actually a good thing--it means fools can't crash the OSGi implementation.  So we either can specify some classloader chicanery (osgi.parentClassloader=app) or we can invoke the commands via reflection.  I opted for this route because I was always taught not to mess with things you don't understand and the ClassLoader hierarchy under OSGi is definitely something I don't understand.  Here's the two applicable methods:

 private static Object invokeCommand(final String name, final String[] args)
   throws Exception {
  String filter = "(&(" + Constants.OBJECTCLASS + "="
    + ICommand.class.getName() + ")(name=" + name + "))";
  ServiceReference[] services = context.getAllServiceReferences(
    ICommand.class.getName(), filter);
  if ((services != null) && (services.length != 0)) {
   Object c = context.getService(services[0]);
   if (c != null) {
    Method m = c.getClass().getMethod("execute", String[].class);
    return m.invoke(c, (Object) args);
   }
  }
  return "Command not found: " + name;



private static Map getAllCommands() {
  Map commands = new LinkedHashMap();
  try {
   ServiceReference[] services = context.getAllServiceReferences(
     ICommand.class.getName(), null);
   if (services != null) {
    for (ServiceReference r : services) {
     Object c = context.getService(r);
     if (c != null) {
      try {
       Method m = c.getClass().getMethod("getHelp");
       commands.put((String) r.getProperty("name"),
         (String) m.invoke(c));
      } catch (SecurityException e) {
       // ignore
      } catch (IllegalArgumentException e) {
       // ignore
      } catch (NoSuchMethodException e) {
       // ignore
      } catch (IllegalAccessException e) {
       // ignore
      } catch (InvocationTargetException e) {
       // ignore
      }
     }
    }
   }
  } catch (InvalidSyntaxException e) {
   // should never happen
  }
  return commands;
 }


Not my finest hour, throwing Exception, but it should get you on your way.  It works like a charm in my app.

Cheers,
Josh


AT&T Update
2008-08-02T13:55:00.003-05:00
Well, since I bitched about AT&T last time, I suppose I should post something with some technical merit.  It'll be in the next post, so folks that want to read it don't have to read through this post.  For those of you interested, things aren't fully resolved with AT&T but Elizabeth's mom got on the phone with AT&T and put them in their place.  She took it to the AT&T National level and has direct lines to folks there that can actually get stuff done.  Supposedly everything is almost sorted, I just need to bring my iPhone in and get it re-programmed to my new number.  I say 'supposedly' because until the deal is actually done and it's been a month or two, I have absolutely no faith in AT&T.  It was a bit comical, though, because Elizabeth's mom got things sorted in like 20 minutes.  Both Elizabeth and I are dumbfounded after the numerous interactions with AT&T, both on the phone and in person, as to how she could be so persuasive.


AT&T == Lying, Deceitful, and Fraudulent
2008-07-31T22:28:00.004-05:00
So it's been a long time since I blogged and I really hate to be so negative but I had an absolute nightmare of a day dealing with AT&T today.  My birthday is coming up and Elizabeth thought it would be nice to get me an iPhone because I had been asking about them.  So begins the saga.  We previously had cell phones on Elizabeth's parent's AT&T Family Talk plan, which costs about $20/month for 2 lines.  Elizabeth called up AT&T to figure out what we had to do to get me an iPhone.  They informed us that all we needed to do was sign up for a new account and transfer our existing numbers to this account.  However, despite mentioning numerous times that we were doing this to purchase an iPhone and asking explicitly about the costs, we were told "pay the transfer fee of $18/line and sign up for a Family Talk plan @ $69.99 and then go into the Apple Store and they'll set up the iPhone data plan" and you'll be good to go. (And I make it sound easy, but this really entailed 2 hours on the phone and talking to several different people at AT&T).  Looking online we had a ballpark figure of around $150/month for the voice and data.  This is far more than the $20/month we had been paying but we were willing to shell out the money for our own account and for the iPhone plan.  

Fast forward to an hour later when we were in an Apple store trying to activate one of the few remaining iPhones.  Activation failed!  Apparently the new account got created as a business account instead of a domestic account and Apple couldn't activate the phone.  WTF?  Well we called up AT&T while at the mall and after another 30 minutes on the phone, got the new account switched to a domestic rather than business account.  We go back for a second time to activate the phone and they said we're not eligible.  So what everyone from AT&T neglected to mention was that despite signing up for a new account and a significant additional cost, by transferring our numbers we were still bound under the original contract.  Nevermind the fact that we 1) weren't switching companies and 2) we were actually bringing MORE money in for AT&T since we were going from paying $20/month for the next year to paying $150/month for the next 2 years.

That's all fine and dandy but what I don't understand is how they can sign us up for a new contract under different terms but hold us to the original contract.  They were all too happy to charge us $36 to transfer our numbers and commit us to paying $150/month for the next 2 years but when we want to get the iPhone at the discount all of a sudden the story is that we're still under the other contract and are not eligible for the phone at the reduced price.  Now, I can understand the transfer logic and I can understand the new account logic.  What I can't fathom is how they think they can enforce two contracts, with conflicting terms, at the same time?  Either the new account comes under the old contract terms, and I pay $20/month through September 2009 and no iPhone upgrade (not really a new account then) or the new account is treated like a new account at the new rate for the new time period and I'm eligible for the iPhone upgrade.  One or the other, but you can't have both!

But it doesn't end there.  I went ahead and signed up a new single account under my name to purchase an iPhone.  This meant having to get a new number.  After the Apple store, we walked down the hall in the mall and went into the AT&T corporate store thinking it might be a refreshing change from spending hours on the phone.  We explained the situation to the customer service rep there and he was all to happy to try and rectify the situation.  He said "sure, we can just transfer those numbers back to the original account and close the new account".  We were like, that's fine even though my old number/phone would basically go unused now that I had a new number.  It would save Elizabeth from having to change her number.  The AT&T rep couldn't do the transfer from his system because of that stupid business vs. domestic error so he called the corporate office to get things fixed.  He almost got it but then needed permission from Elizabeth's mom to add us back onto the Family Talk plan.  This makes sense, so I don't fault them for that.  Unfortunately we couldn't get Elizabeth's mom on the phone so we couldn't continue.  

It was at that point that the rep informed us that it would cost an additional $18/line to transfer the numbers back!  So we had to pay to transfer the lines to a new account, despite the fact that no one informed us that we wouldn't be able to purchase the iPhone at the reduced rate and now they wanted to charge us an additional $18/line to transfer back.  All that after spending the whole afternoon, from noon to 5PM either on the phone or in stores dealing with AT&T!  The rep, Andrew "Drew" at the Southdale AT&T store then proceeded to get in our face about the charges and be quite rude.  "Well it's not like we just went in and changed it without permission."  No, but you also were deceitful when you said that all we had to do was sign up for a new account and transfer our numbers and then we'd be good to go.

But the best is yet to come.  So we leave the store and Elizabeth immediately gets on the phone again with AT&T.  Once she actually gets to a live person, she explains the situation for the umpteenth time and then gets flak when she asks for a manager after the person on the other end won't help her.  After explaining the situation yet again, the manager seems sympathetic and is willing to waive the transfer fees.  She begins the process of transferring back and then magically says "we can't transfer back because lines in a new account can't be transferred for 60 days".  So the only thing you can do is transfer the 3 lines on her parents account to our account for 2 months and then transfer them to another account after that.  And guess what, that's $18/line for each line and then another $18/line to transfer off our account.  All because transferring back wasn't possible.  Gee, well our buddy Drew in the store seemed to think it was possible.  So yet again, AT&T comes up with these convenient rules.

So let's re-cap.  When you sign up a new account with AT&T and try to transfer your lines, beware that despite them taking significantly more than what you were paying before and binding you to an additional 2 years, they can and will choose to enforce the previous contract when it suits them.  So basically you're bound to 2 contracts and they've got you over a barrel by using whichever suits them at the time.  You should also not expect to be informed of absolutely anything, especially not contract terms when you sign up for your new old account.  Furthermore, what you can and can't do changes from person to person and from minute to minute.  Our buddy Drew was going to transfer us back and the manager on the phone was going to transfer us back but then randomly came up with this no-transfer during 60 days rule which conveniently nets AT&T an additional $108 in transfer charges.  The best part is, and what no one at AT&T seemed to grasp, was that it was in their interest to just give us a new account and let us sign up for the iPhone because it meant we went from paying $20/month to paying $150/month AND they had us for 2 full years!  I hope some AT&T investors stumble across this and realize how poorly managed the company is that they are throwing away money and souring customers.

So at this point, there's not much we can do.  We're going to let Elizabeth's mom talk to them and see if she can make any headway.  Tomorrow I'm going to file a complaint with the MN Attorney General and the Better Business Bureau for deceitful and fraudulent practices.  If Elizabeth's mom doesn't make any headway, I think we'll be contesting the charges with our credit card company and we'll have to see if it is worth filing in small claims court.  After that, I'm out of ideas.  The only advice I have is: steer clear of AT&T if you can.

Through it all, the Apple Store employees were helpful and pleasant to work with, even going so far as to try and cover for AT&T.  It was all too obvious, though, that AT&T was in the wrong.  They were truly apologetic that we had to go through such a mess, and didn't want this experience to sour our opinion of Apple and the iPhone.  No worries, though, as we got nothing but top notch service from Apple.

Time for bed, it's been a long day.


St. Petersburg, Russia
2008-07-10T03:38:00.003-05:00
Sorry for the lack of recent updates.  It seems like I've been on a tour for work: Lincoln, Potsdam, and now St. Petersburg, Russia all in the last month or so.  And if   

St. Petersburg is like no place I've ever been.  The diversity and contrast between buildings is amazing.  You'll be walking down the street and see buildings with huge golden domes and intricate architecture next to a no-nonsense, utilitarian building that looks like it has been abandoned.

Overall, I've had good luck with the people.  Most have been friendly and helpful.  The rest have been largely indifferent to my butchering the pronunciation of the few Russian words I've picked up via osmosis.

The biggest adjustment for me is the lack of smoking bans in public areas.  I was shocked when we arrived and I saw someone lighting a cigarette in the hotel lobby.  It's completely different from the US and is something I don't think I'd want to get used to.

I've had a hard time adjusting to the timezone.  It is 9 hours different from home but for whatever reason I haven't been sleeping very much and not on a regular schedule.  Part of the problem may be that there's very little darkness at night during the summer.  It usually gets dark around midnight and stays dark for 2 hours or so.  It's almost like my first weeks on the ice in Antarctica.

I'm looking forward to returning home on Saturday.  My flight is early Saturday morning and I arrive back in Minneapolis around 3:30PM if all goes to plan (though I'm not holding my breath with the state of air travel these days).  I have to quick rush home from the airport, change, and go to a wedding reception.  I doubt I'll make much of a party guest, but I should put in an appearance.  After that, I think I'll take a few days to settle back in and get on a normal schedule.  I think I'm home for all of a week before I have to pop over to DC for a quick meeting.  Then I think I'm going to do everything in my power to spend a full month at home in my new house.  Though we'll see what comes up.

Dasvidania.


Mercurial Push from IntelliJ
2008-06-26T17:55:00.005-05:00
I've been using IntelliJ on a recent project because of its Groovy and Mercurial support.  The Mercurial support worked quite well except for pushing changes to a remote repository over ssh.  A quick look at the Version Control Console revealed that things were getting hung up on: remote: ssh_askpass: exec(/usr/libexec/ssh-askpass): No such file or directory.  After confirming that there was no ssh-askpass on my Mac OS X Leopard system, I turned to Google.  After a few misses, I stumbled across Joe Mocker's blog post about VNC tunneled through SSH on OS X.  Embedded in that post is this shell/AppleScript:

#! /bin/sh

#
# An SSH_ASKPASS command for MacOS X
#
# Author: Joseph Mocker, Sun Microsystems

#
# To use this script:
#     setenv SSH_ASKPASS "macos-askpass"
#     setenv DISPLAY ":0"
#

TITLE=${MACOS_ASKPASS_TITLE:-"SSH"}

DIALOG="display dialog \"$@\" default answer \"\" with title \"$TITLE\""
DIALOG="$DIALOG with icon caution with hidden answer"

result=`osascript -e 'tell application "Finder"' -e "activate"  \
-e "$DIALOG" -e 'end tell'`

if [ "$result" = "" ]; then
exit 1
else
echo "$result" | sed -e 's/^text returned://' -e 's/, button returned:.*$//'
exit 0
fi


I dropped this code into a script at /usr/libexec/ssh-askpass and now when I push from IntelliJ:

Ugly, but it works.  Now I just wish that the IntelliJ Mercurial plugin would consult the .hg/hgrc file for the remote repository or at least remember the value I type in when I pushed the last time, so I don't have to type in some long ssh://user@host.org/path/to/the/repo every time.


PSICAT News
2008-06-12T04:50:00.003-05:00
Last week I was in Potsdam, Germany for a meeting with folks from ICDP, ESO, and CoreWall.  We were discussing integrating our various tools (PSICAT, Corelyzer, and the Drilling Information System) to create a turnkey technology platform for future ICDP and ESO drilling expeditions.  Each of the tools have been used successfully on multiple expeditions, but until now haven't interoperated with each other.  The meeting went well and we adjourned with an integration plan.

So what does this mean for PSICAT?  First off, if you're an existing user, you won't have to do anything different; PSICAT will continue to work exactly as it does today.  It will just have some new, optional features for integrating with the DIS and Corelyzer.  One positive side effect of this is that I will have some "official" time devoted to working on PSICAT.  Things have been so busy in the last couple of months that PSICAT development has been on the back burner.  PSICAT won't be my only project but it should get more attention than it is getting now.  It also means that PSICAT will be getting used all over the world on new drilling projects, which (I hope) leads to a larger user community and a better overall product. 

Exciting times!  Keep an eye on the PSICAT site for more updates.


GraphicsBuilder Update
2008-06-10T08:04:00.002-05:00
Just a quick update to my previous post about GraphicsBuilder.  All of the issues I had in Step #4 have been fixed, so you should no longer have to modify the pom.xml file or manually install Batik 1.7 jars.  The only remaining issue is the requirement on Java 1.6, but Andres recently posted that the next version of GraphicsBuilder will not require Java 1.6.


GraphicsBuilder Experimentation
2008-06-05T14:27:00.003-05:00
This afternoon I was experimenting with GraphicsBuilder on Groovy, and I'm blown away.  Andres has done an amazing job with GraphicsBuilder.

It took a bit of work to get setup because I wanted to play around with the SVG rendering support (which isn't included in the 0.5.1 package that is currently available).  Once I built from the trunk, it worked like a charm.  Below are the steps that I went through to set things up in case it is useful to anyone else.

Pre-requisites:
1) Java 6
Make sure you have Java 6 installed for your platform.  I'm on a Mac which doesn't have Java 6 support (grr...) so I went the SoyLatte route.

2) Groovy 1.5
Make sure you have Groovy installed.  I ran into troubles when I tried to use the 1.6 snapshot (ClassCastExceptions) so stick with 1.5 for now.

3) Maven 2
We'll be building GraphicsBuilder from the Subversion trunk, so we need to install Maven 2.

4) Subversion
Hopefully you already have the Subversion, but if not, grab and install it for you platform.

Instructions:
1) Setting up some environment variables at the commandline

export GROOVY_HOME=~/Source/groovy
export JAVA_HOME=/usr/local/java1.6
export PATH=$JAVA_HOME/bin:$GROOVY_HOME/bin:~/Source/maven/bin:$PATH


Obviously your paths will differ.  Once this is done, we should be able to run our java, groovy, and mvn commands without errors.

2) Download GraphicsBuilder from Subversion

svn co http://svn.codehaus.org/groovy-contrib/graphicsbuilder/trunk graphicsbuilder


3) Download the Batik 1.7 distribution
Download a copy of the Batik 1.7 distribution and unzip it into our graphicsbuilder directory.

4) Use Maven to build GraphicsBuilder

cd graphicsbuilder
mvn


If everything works for you, then proceed to the next step.  For me, I ran into two problems.  The first problem was that top level pom.xml referenced groovy-all-minimal as a dependency but the other pom.xml files referenced groovy-all as a dependency.  This caused Maven to complain about a missing version and to fail.  I fixed this by changing the top level pom.xml file to reference groovy-all:

--- pom.xml (revision 356)
+++ pom.xml (working copy)
@@ -77,7 +77,7 @@
  
      
         org.codehaus.groovy
-            groovy-all-minimal
+            groovy-all
         ${groovy-version}
      
      


This seemed to clear up Maven's problems and the build actually proceeded.

The other problem I ran into was that the Batik 1.7 jars weren't available in the Maven repositories so the build complained of missing dependencies.  Fortunately Maven will allow us to install the required jars locally:

mvn install:install-file -DgroupId=batik -DartifactId=batik-awt-util -Dversion=1.7 -Dpackaging=jar -Dfile=batik-1.7/lib/batik-awt-util.jar
mvn install:install-file -DgroupId=batik -DartifactId=batik-util -Dversion=1.7 -Dpackaging=jar -Dfile=batik-1.7/lib/batik-util.jar
mvn install:install-file -DgroupId=batik -DartifactId=batik-gui-util -Dversion=1.7 -Dpackaging=jar -Dfile=batik-1.7/lib/batik-gui-util.jar
mvn install:install-file -DgroupId=batik -DartifactId=batik-ext -Dversion=1.7 -Dpackaging=jar -Dfile=batik-1.7/lib/batik-ext.jar
mvn install:install-file -DgroupId=batik -DartifactId=batik-svggen -Dversion=1.7 -Dpackaging=jar -Dfile=batik-1.7/lib/batik-svggen.jar
mvn install:install-file -DgroupId=batik -DartifactId=batik-dom -Dversion=1.7 -Dpackaging=jar -Dfile=batik-1.7/lib/batik-dom.jar
mvn install:install-file -DgroupId=batik -DartifactId=batik-svg-dom -Dversion=1.7 -Dpackaging=jar -Dfile=batik-1.7/lib/batik-svg-dom.jar
mvn install:install-file -DgroupId=batik -DartifactId=batik-parser -Dversion=1.7 -Dpackaging=jar -Dfile=batik-1.7/lib/batik-parser.jar
mvn install:install-file -DgroupId=batik -DartifactId=batik-xml -Dversion=1.7 -Dpackaging=jar -Dfile=batik-1.7/lib/batik-xml.jar
mvn install:install-file -DgroupId=batik -DartifactId=batik-gvt -Dversion=1.7 -Dpackaging=jar -Dfile=batik-1.7/lib/batik-gvt.jar


After that, I was able to kick off mvn and build everything.

5) Install GraphicsBuilder into GROOVY_HOME
If you want to play around with GraphicsBuilder from the commandline, the easiest thing to do is to install it in your GROOVY_HOME:

cp */lib/*.jar $GROOVY_HOME/lib/
cp */target/*.jar $GROOVY_HOME/lib/
cp */src/lib/*.jar $GROOVY_HOME/lib/
cp */src/bin/* $GROOVY_HOME/bin/
chmod +x $GROOVY_HOME/bin/graphicsPad
chmod +x $GROOVY_HOME/bin/svg2groovy


You may also want to grab the build of GraphicsBuilder 0.5.1 because I think it may have included another jar or two that wasn't covered above (MultipleGradientPaint.jar, TimingFramework-1.0-groovy.jar, swing-worker.jar, and swingx-0.9.2.jar).

6) Play with it
You can either test it by running the graphicsPad application or writing a script that calls one of the renderers:

import groovy.swing.j2d.*
import groovy.swing.j2d.svg.*

def foo = {
  antialias('on')
  circle(cx:0, cy:0, radius:300, borderColor:'black', borderWidth:4) {
      multiPaint {
          colorPaint('orange')
          texturePaint(x:0, y:0, file:'/Users/jareed/Desktop/602.png')        
      }
      transformations {
       translate(x:500, y: 500)
      }
  }
  circle(cx:300, cy:300, radius:150, borderColor:'black', borderWidth:4) {
      multiPaint {
          colorPaint('red')
          texturePaint(x:0, y:0, file:'/Users/jareed/Desktop/602.png')        
      }
  }
}

def gr = new GraphicsRenderer()
def sr = new SVGRenderer()

gr.renderToFile("/Users/jareed/Desktop/test.png", 1000, 1000, foo)
sr.renderToFile("/Users/jareed/Desktop/test.svg", 1000, 1000, foo)


which generates:

Nothing earth shattering, but it will be potentially interesting once I flesh out the project I want to use it on for.  The best part, though, is that the SVG rendering works exactly as advertised.  I had tried to do SVG rendering in the past using Batik's Graphics2D implementation (from a SWT app) and I couldn't get the background images to show up.  Sweet!


Lincoln and Potsdam
2008-06-03T15:48:00.003-05:00
Sorry for the lack of updates.  I'm in Lincoln this week and then Potsdam, Germany next week for work.  If anyone happens to be reading from either of those places and wants to grab a beer, shoot me an email.  I'm hoping to get some work done on the flights, so hopefully I'll have stuff to blog about.

On a personal note, Elizabeth and I are in the process of buying a house.  We found one we liked and offered on it last week.  The offer was accepted and now we're just waiting for the sellers to do a few things and for our mortgage loan to come through.  The plan is to close on the 20th or sooner, so when I get back from Potsdam, it looks like I'll be getting my stuff packed up to move.

The house is in Golden Valley (about 2 miles north of where we currently live).  It's about 2000 sq. ft, with 4 bedrooms and 2 bathrooms.  It's a one story, with a finished walkout basement.  The lot is nice and large (for the city) and it's in what appears to be a nice, established neighborhood.  I'm super pumped about the move as it means I can start doing things around the house (when I'm home, that is).  I'll get some photos up when we start moving.


Units DSL in Groovy
2008-05-22T16:28:00.003-05:00
A while back I saw Guillaume Laforge's article about building a Groovy DSL for unit manipulations.  I recently needed to implement something similar in a project, so I decided to take Guillaume's code and update it a bit.  I wanted a nice way to package it up so I could quickly enable unit manipulation support on a particular class.  I also added a pair of methods to make things more flexible.  Here's my UnitDSL.groovy:

package org.psicat.model

import org.jscience.physics.amount.*
import javax.measure.unit.*

/**
 * A helper class for setting up the Units DSL
 */
class UnitDSL {
 private static boolean isEnabled = false;
 private UnitDSL() { /* singleton */ }
 
 /**
  * Initialize the Units DSL.
  */
 static enable() {
     // only initialize once
     if (isEnabled) return
 
     // mark ourselves as initialized
     isEnabled = true
  
     // enable inheritance on EMC
     ExpandoMetaClass.enableGlobally()
  
     // transform number properties into an mount of a given unit represented by the property
     Number.metaClass.getProperty = { String symbol -> Amount.valueOf(delegate, Unit.valueOf(symbol)) }
  
     // define opeartor overloading, as JScience doesn\'t use the same operation names as Groovy
     Amount.metaClass.static.valueOf = { Number number, String unit -> Amount.valueOf(number, Unit.valueOf(unit)) }
     Amount.metaClass.multiply = { Number factor -> delegate.times(factor) }
     Number.metaClass.multiply = { Amount amount -> amount.times(delegate) }
     Number.metaClass.div = { Amount amount -> amount.inverse().times(delegate) }
     Amount.metaClass.div = { Number factor -> delegate.divide(factor) }
     Amount.metaClass.div = { Amount factor -> delegate.divide(factor) }
     Amount.metaClass.power = { Number factor -> delegate.pow(factor) }
     Amount.metaClass.negative = { -> delegate.opposite() }
  
     // for unit conversions
     Amount.metaClass.to = { Amount amount -> delegate.to(amount.unit) }
     Amount.metaClass.to = { String unit -> delegate.to(Unit.valueOf(unit)) }
 }
 
 /**
  * Add Units support to the specified class.
  */
 static addUnitSupport(clazz) {
  clazz.metaClass.setProperty = { String name, value ->
     def metaProperty = clazz.metaClass.getMetaProperty(name)
     if (metaProperty) {
      if (metaProperty.type == Amount.class && value instanceof String) {
       metaProperty.setProperty(delegate, Amount.valueOf(value))
      } else {
       metaProperty.setProperty(delegate, value)
      }
     }
  }
 }
 
 /**
  * Remove Units support from the specified class.
  */
 static removeUnitSupport(clazz) {
  GroovySystem.metaClassRegistry.removeMetaClass(clazz)
 }
}


To enable the DSL, you have to call UnitDSL.enable().  This adds a few methods to the metaclasses on Number and Amount.  The majority of the code in enable() is a straight cut and paste job from Guillaume's article.  

I did add two methods.  The first:

Amount.metaClass.static.valueOf = { Number number, String unit -> Amount.valueOf(number, Unit.valueOf(unit)) }


allows you to create an Amount using a Number and a String, e.g. Amount.valueOf(1.5, "cm").

The other new method:

Amount.metaClass.to = { String unit -> delegate.to(Unit.valueOf(unit)) }

allows conversions with the unit specified as a String, e.g. 1.5.m.to("cm")

The final enhancement I added was to create a addUnitSupport(clazz) method.  This method overrides the setProperty() method of the passed class to support setting Amount properties as strings.  All assignments in the following scenario are valid:

class Foo {
    Amount bar
}

// test
UnitDSL.enable()
UnitDSL.addUnitSupport(Foo)

def foo = new Foo()
foo.bar = Amount.valueOf(3, SI.METER)
foo.bar = Amount.valueOf(3, "m")
foo.bar = Amount.valueOf("3m")
foo.bar = 3.m
foo.bar = "3m"


To use this code, you'll have to grab the latest JScience release.


Visualizer, Part 3: Poor Man's PDE Build
2008-05-21T21:11:00.006-05:00
This is the third in my series (Part 1, Part 2) of posts about Visualizer.  In this post I'll be talking about how to create a simplified PDE build.

As I mentioned in the previous post, Visualizer is built on OSGi.  My preferred development environment for doing any Java development, but especially OSGi development, is Eclipse because of its wonderful JDT and PDE tooling.  The PDE team has created an awesome environment for developing and managing OSGi bundles.  However, one of the requirements that I had for Visualizer was that anyone could download the source code and build it, regardless of their IDE or environment preferences.  PDE includes the ability to perform a headless build, but I didn't really want to expect the user to download Eclipse or to include a stripped down version of Eclipse in the Visualizer distribution just so the user could build it from the commandline.  So I set out to create a "Poor Man's PDE Build" using just Ant.

Actually building the plugins with Ant is relatively simple.  This short Ant file will build the plugin:

<?xml version="1.0" encoding="UTF-8"?>
<project default="build-plugin">
    <property name="src.dir" value="src"/>
    <property name="classes.dir" value="bin"/>
    <property file="META-INF/MANIFEST.MF"/>
    <property file="build.properties"/>
    
    <path id="classpath">
        <fileset dir="${dist.dir}">
            <include name="*.jar"/>
        </fileset>
        <fileset file="${osgi.framework}"/>
        <pathelement path="${java.class.path}"/>
    </path>
    
    <target name="init">
        <mkdir dir="${classes.dir}"/>
    </target>
    
    <target name="compile" depends="init">
        <echo message="Compiling the ${Bundle-SymbolicName} plugin"/>
        <javac srcdir="${src.dir}" destdir="${classes.dir}" classpathref="classpath" debug="true"/>
    </target>
    
    <target name="copy-resources">
        <echo message="Copying resources"/>
        <copy todir="${classes.dir}">
            <fileset dir="." includes="${bin.includes}"/>
        </copy>
    </target>
    
    <target name="build-plugin" depends="compile, copy-resources">
        <jar jarfile="${dist.dir}/${Bundle-SymbolicName}_${Bundle-Version}.jar" basedir="${classes.dir}" manifest="META-INF/MANIFEST.MF"/>
    </target>
    
    <target name="clean">
        <delete includeemptydirs="true">
            <fileset dir="${classes.dir}" includes="**/*"/>
        </delete>
    </target>
</project>


As you can see here, there really isn't much to the actual build.  The best part is we can use the META-INF/MANIFEST.MF and build.properties created when we're working in PDE to control the build.

For a single bundle with no dependencies, this effectively duplicates the PDE build process.  The difficulties comes in when you start having dependencies.  If you use the headless PDE build, it will sort out all the dependencies for you and build your bundles in the proper order.  

Implementing proper dependency resolution seemed awfully complicated, especially since PDE build already implements it.  Fortunately, Visualizer doesn't require complicated dependency resolution because I've structured the bundles in a logical order.  There are three levels of bundles: "core" which implement the main functionality, "ui" which implement the user interface to the core bundles, and "application" bundles that build on both the core and ui bundles.  

Armed with this knowledge, we can structure a three stage build process where we first build all of the core bundles then all of the ui bundles and then all of the application bundles.  To accomplish this, we have a master build.xml that calls out to the template build-plugin.xml file listed above using a subant task.


<target name="build-framework" depends="init">
    <!-- build org.andrill.visualizer, org.andrill.visualizer.services* -->
    <subant target="build-plugin" genericantfile="build-plugin.xml" failonerror="false">
        <property name="dist.dir" value="../${build.dir}"/>
        <property name="osgi.framework" value="../framework.jar"/>
        <dirset dir=".">
            <include name="org.andrill.visualizer"/>
            <include name="org.andrill.visualizer.services*"/>
        </dirset>
    </subant>
</target>


Here you can see we build first the org.andrill.visualizer bundle and then all of the org.andrill.visualizer.services bundles.  As we build each bundle, we copy the bundled JAR file to our dist.dir.  Each time a bundle is built, it creates its classpath from all of the JARs in dist.dar.  So even though there are dependencies among bundles, we are progressively fulfilling those dependencies by collecting the built bundles in dist.dir.

Once all of the "core" bundles are built, we can kick off the build of the ui bundles:


<target name="build-ui" depends="build-framework">
    <!-- build org.andrill.visualizer.ui* -->
    <subant target="build-plugin" genericantfile="build-plugin.xml" failonerror="false">
        <property name="dist.dir" value="../${build.dir}"/>
        <property name="osgi.framework" value="../framework.jar"/>
        <dirset dir=".">
            <include name="org.andrill.visualizer.ui*"/>
        </dirset>
    </subant>
</target>
  

Finally we can build all of the "application" bundles by excluding everything we've already built:

<target name="build-apps" depends="build-framework, build-ui">
    <subant target="build-plugin" genericantfile="build-plugin.xml" failonerror="false">
        <property name="dist.dir" value="../${build.dir}"/>
        <property name="osgi.framework" value="../framework.jar"/>
        <dirset dir=".">
            <include name="*.*"/>
            <exclude name="org.andrill.visualizer"/>
            <exclude name="org.andrill.visualizer.services*"/>
            <exclude name="org.andrill.visualizer.ui*"/>
            <exclude name="${build.dir}"/>
            <exclude name="${dist.dir}"/>
        </dirset>
    </subant>
</target>


You can check out the full build file at: build.xml and build-plugin.xml.

It's not nearly as neat as just kicking off a PDE build and letting it do all of the hard work of figuring out the dependencies for you.  However, I'm rather fond of my approach because it keeps me honest.  If I create a new bundle and it starts breaking the build, then I know I need to go back and make sure I've thought through the dependencies and am not trying to mix "core" code with "ui" code and such.  And there's no need to bundle Eclipse with the source to build the thing.


Open Participation in SimpleIssue
2008-05-12T09:16:00.004-05:00
When I started writing the simple issue tracker it was with the goal of learning more about Grails and coming away with a generally useful project.  I decided to blog it because I was hoping my insights might be useful to the community.  The response has been awesome, with lots of encouragement and lots of great ideas.

It strikes me that I can probably learn more from the community than just working away on my own.  So if you want to fix something I screwed up or you have ideas for new features or you just want to experiment, drop me a line and I'll add you to the SimpleIssue project out at Google Code.  Mike Hugo has already pointed out my embarrassing lack of tests and has offered to lend a hand.  This is great news for me because testing has never been my strong suit.  It will give me an opportunity to study how it's done.

SimpleIssue is a nights and weekends project for me, so I don't expect any particular level of participation.  If you've got ideas and want to contribute, then all the better for us.


Writing a Simple Issue Tracker in Grails, Part 2
2008-05-11T10:06:00.003-05:00
This is the long overdue follow up to my Writing a Simple Issue Tracker in Grails post.  In this post I'll be detailing how to add security with the JSecurity plugin.

So let's dive right in and work on securing our application.  I opted to use the JSecurity plugin for no other reason than I've used the Acegi plugin in the past and wanted to see how JSecurity compares.  You could use the Acegi plugin with a similar process and results.

Following the JSecurity Quick Start guide, we'll begin by installing the plugin and running the quick start script:

grails install-plugin jsecurity
grails quick-start


The quick start script created a few domain classes as well as a controller for logging in and out.  Now we need to setup an Administrator user and role to test things with.  We'll just cut and paste the Administrator role setup right from the Quick Start guide into our Bootstrap.groovy file:

import org.jsecurity.crypto.hash.Sha1Hash

class BootStrap {

    def init = {servletContext ->
        // Administrator user and role.
        def adminRole = new JsecRole(name: "Administrator").save()
        def adminUser = new JsecUser(username: "admin", passwordHash: new Sha1Hash("admin").toHex()).save()
        new JsecUserRoleRel(user: adminUser, role: adminRole).save()
    }

    def destroy = {
    }
}


With the administrator user in place, we can start securing our controllers.  JSecurity makes this drop dead simple by letting you specify rules in the conf/SecurityFilters.groovy file.  

Before we dive in and start writing our rules, let's think about what we want users to be able to do.  I'm going to keep things simple and have two classes of users: anonymous and administrators.  However, you could easily create a third class of users, authenticated, that would have more permissions than the anonymous users but less than the administrators.

Administrators should be able to create, edit, and delete Projects, Components, and Issues.  Additionally, Administrators should be able to access the admin controller.

Depending on how private you want your issue tracker, anonymous users might be able to list and show the Projects, Components, and Issues or they might not.  They may also create new issues or they may not.  To handle this, I'm going to create two new configuration parameters that control how secure the application is by adding the following lines to Config.groovy:

issue.secure.create = true
issue.secure.view = false


These configuration options will let us configure whether anonymous users can create issues (issue.secure.create = true) and whether anonymous users can view issues (issue.secure.view = false).  In the above example, creating a new issue is secured (requires the user be logged in) but viewing is not.

Now that we have an idea of our security rules, let's codify them in the conf/SecurityFilters.groovy file:

import org.codehaus.groovy.grails.commons.ApplicationHolder

class SecurityFilters {
    def filters = {

        // secure the project controller
        projectCreationAndEditing(controller: "project", action: "(create|edit|save|update|delete)") {
            before = {
                accessControl {
                    role("Administrator")
                }
            }
        }

        // secure the component controller
        componentCreationAndEditing(controller: "component", action: "(create|edit|save|update|delete)") {
            before = {
                accessControl {
                    role("Administrator")
                }
            }
        }

        // secure issue editing
        issueEditing(controller: "issue", action: "(edit|update|delete)") {
            before = {
                accessControl {
                    role("Administrator")
                }
            }
        }

        // secure admin controller
        admin(controller: "admin", action: "*") {
            before = {
                accessControl {
                    role("Administrator")
                }
            }
        }

        // secure creating issues if Config#issue.secure.create = true
        if (ApplicationHolder.application.config?.issue?.secure?.create) {
            issueCreation(controller: "issue", action: "(create|save)") {
                before = {
                    accessControl {
                        role("Administrator")
                    }
                }
            }
        }

        // secure viewing issues if Config#issue.secure.view = true
        if (ApplicationHolder.application.config?.issue?.secure?.view) {
            issueBrowsing(controller: "issue", action: "(show|list)") {
                before = {
                    accessControl {
                        role("Administrator")
                    }
                }
            }

            componentBrowsing(controller: "component", action: "(show|list)") {
                before = {
                    accessControl {
                        role("Administrator")
                    }
                }
            }

            projectBrowsing(controller: "project", action: "(show|list)") {
                before = {
                    accessControl {
                        role("Administrator")
                    }
                }
            }
        }
    }
}


So with that we should have a secured grails app.  Obviously there are plenty of things we can improve.  First off, there is no user management code.  We'll probably want to generate a controller to allow new users to register as well as to allow Administrators to add new users.  The views could be cleaned up quite a bit and we could add a custom logo.  Finally there are numerous other options that folks have suggested, including saved searches, internationalization, etc.

I've gone ahead and created a new Google Code project with the code from this article.  If you're interested in hacking on this code, let me know.  In the next installment, which I promise won't take a month, I'll be adding search/filtering support via the Searchable plugin and creating an API so I can create new issues from external applications.

Cheers,
Josh


Visualizer, Part 2: OSGi & Native Libraries
2008-05-06T19:40:00.005-05:00
This is the second in my series (Part 1) of posts about Visualizer.  In this post I'll be talking about packaging an OSGi bundle that includes native libraries.

Visualizer uses the OpenGL bindings provided by JOGL project to display images and data.  JOGL provides a series of platform-specific downloads that include a standard JAR file of Java classes and a set of native libraries for variety of platforms.  This complicates the deployment of Visualizer because we need to install the appropriate version of JOGL for the user's platform.  One option is to mimic JOGL and provide platform-specific builds of Visualizer that includes the appropriate version of JOGL.  This isn't ideal because it adds extra steps to the build process and can introduce confusion for users trying to figure out which version of Visualizer they should download.

Fortunately, there's another option: OSGi.  In a nutshell, OSGi is a component framework specification that allows you to assemble and manage applications as a collection of components (bundles).  I'm not really doing OSGi justice so if you don't know what it is, you owe it to yourself to check it out.  And odds are you've probably already used something built on OSGi because it seems to be everywhere these days.

Anyhow, OSGi elegantly solves our Visualizer deployment problem by allowing us to provide a single download.  We simply combine the Java classes and all of the platform-specific native libraries provided into a single bundle and OSGi will detect and extract the appropriate set of native libraries based on the user's platform.

The first step was to download all of the JOGL packages for the platforms you want to support.  I have users on Linux (32 & 64 bit), Mac OS X, and Windows (32 & 64 bit), so I downloaded all of these.  From these downloads, I kept one copy of the JOGL JAR files and collected all of the native libraries.

The next step was to use Eclipse's excellent PDE tooling to create a new "Plugin Project from existing JAR files".  I called it 'jogl' and pointed it at the JOGL JAR files.  It sucked in all of the class files and spat out an OSGi bundle.  If there were no native libraries, we'd be done.

Since we have native libraries, I copied them into the jogl bundle directory using a straightforward directory structure:


The 'native' directory structure is not required; you can use whatever makes sense to you.  As you can see from the screenshot, I've got a set of libraries for 5 platform/processor combinations.

The final step is to make the OSGi framework aware of the libraries so it can extract the appropriate libraries when it starts up.  This requires using the Bundle-NativeCode header in your bundle manifest:

Bundle-NativeCode: native/macosx/libgluegen-rt.jnilib;
 native/macosx/libjogl_cg.jnilib;
 native/macosx/libjogl_awt.jnilib;
 native/macosx/libjogl.jnilib;
 osname=mac os x;
 processor=x86;
 processor=ppc,
 native/linux/x86/libgluegen-rt.so;
 native/linux/x86/libjogl_cg.so;
 native/linux/x86/libjogl_awt.so;
 native/linux/x86/libjogl.so;
 osname=linux;
 processor=x86,
 native/linux/x86-64/libgluegen-rt.so;
 native/linux/x86-64/libjogl_cg.so;
 native/linux/x86-64/libjogl_awt.so;
 native/linux/x86-64/libjogl.so;
 osname=linux;
 processor=x86-64,
 native/windows/x86/gluegen-rt.dll;
 native/windows/x86/jogl_cg.dll;
 native/windows/x86/jogl_awt.dll;
 native/windows/x86/jogl.dll;
 osname=win32;
 processor=x86,
 native/windows/x86-64/gluegen-rt.dll;
 native/windows/x86-64/jogl_cg.dll;
 native/windows/x86-64/jogl_awt.dll;
 native/windows/x86-64/jogl.dll;
 osname=win32;
 processor=x86-64


One quick note: watch the whitespace when editing the bundle manifest.  The OSGi specification is explicit about where whitespace is allowed and where it is required.

With this final piece we can JAR up our class files, native libraries, and bundle manifest and we should be able to use it in any OSGi implementation.  When the OSGi implementation loads our bundle, it will extract the appropriate set of native libraries based on the user's osname and processor properties and make sure those libraries are available on the classpath.

I've tested this JOGL bundle on the Equinox implementation of OSGi across Mac, Windows, and Linux and it works great.  If anyone is interested, I can make the pre-built bundle of JOGL available for download.


Busy Last Couple of Weeks
2008-05-06T19:32:00.004-05:00
In the last 2.5 weeks, I was in the Bahamas for my wedding, Tallahassee for work, and in Iowa for a friend's wedding.  I've spent a total of maybe 36 hours at home during that time.  And to top it all off, my laptop went kaput while I was in the Bahamas and I'm waiting for a replacement.  The blogging should pick back up now that I'm home for a bit.


Bahamas
2008-04-25T11:00:00.002-05:00
Sorry for the radio silence, I'm currently in the Bahamas for my wedding and honeymoon.  I should be back to blogging next week.

Cheers,
Josh


Writing a Simple Issue Tracker in Grails, Part 1
2008-04-13T12:33:00.010-05:00
My project for the weekend was to write a simple issue tracking webapp in Grails.   I could have used something like Trac but that's overkill for my needs.  I just wanted something simple where my users could report issues and request new features.  I also wanted to add a few personal touches, which I'll show you along the way.

I'm going to assuming that you're not absolutely new to Grails and you've already got it installed and played around with it.  If that's not the case, I suggest checking out Scott Davis's Mastering Grails series of articles.  He's goes into far more detail than I do, so check them out.

Let's start by creating our project:
grails create-app simpleissue

First up is to define our domain models.  We're going to keep it simple with just three models: Project, Component, and Issue.  A Project has one or more Components, such as ui, documentation, etc.  A Component is associated with a single Project and has zero or more Issues associated with it.  The Issue object is associated with a Component and captures a bunch of information.

So let's lay down the code:
Project.groovy

class Project {
    // relationships
    static hasMany = [components: Component]

    // fields
    String name

    String toString() {
        return name
    }

    // constraints
    static constraints = {
        name()
        components()
    }
}

Component.groovy

class Component {
    // relationships
    static belongsTo = Project
    static hasMany = [issues: Issue]

    // fields
    Project project
    String name

    // override for nice display
    String toString() {
        return "${project} - ${name}"
    }

    // constraints
    static def constraints = {
        name()
        project()
        issues()
    }
}

Issue.groovy

class Issue {
    // relationships
    static belongsTo = Component

    // fields
    Component component
    String type
    String submitter
    String description
    String status = "New"
    Integer bounty
    Date dateCreated
    Date lastUpdated

    // constraints
    static constraints = {
        component()
        type(inList: ["Defect", "Feature"])
        submitter()
        description(size: 0..5000)
        status(inList: ["New", "Accepted", "Closed", "Won't Fix"])
        bounty(range:0..12)
    }
}


Most of the code is a pretty straightforward translation of our written description of the domain.  You may, however, notice a few peculiar constraints.  I've used a fair number of 'empty' constraints such as:

static def constraints = {
   name()
   project()
   issues()
}

By default, Grails treats all fields in the domain class as required.  I didn't want to change that, but I wanted to affect the order that the fields show up in a particular order in the web forms.  By specifying the constraint, even if it is empty, it'll show up in that order in our forms.  Of course, we could have customized the field order by hand directly in the view GSP code.

I also make use of the inList constraint to limit the fields to a specific set of values.  Our views will be generated with an HTML select drop down containing the list of values we've specified.

Finally, we specify our issue description as being size:0..5000.  This will ensure that there is plenty of space in the database for the description text.  If we hadn't specified this, the description would have been generated as a varchar(255).

With our domain classes in place, we can create our controllers and views to test things out:

grails generate-all Project
grails generate-all Component
grails generate-all Issue
grails run-app

Fire up your browser and test things out by visiting http://localhost:8080/simpleissue:
and our issue creation form:

Looks pretty decent for 5 minutes of work.  Poke around and test creating a project, component, and a few issues.

Customizing the Look

Now let's clean things up a bit and add some polish.  The first thing I want to do is have the index page show the list of issues.  We could copy and paste the code from the Issue List view or we can simply add a redirect to the top of our web-app/index.gsp file:

<% response.sendRedirect('issue/list') %>


The next thing I want to do is clean up the issue creation form.  A few of the values, such as status, dateCreated, lastUpdated don't need to be specified in the form.  We can go into grails-app/views/issue/create.gsp and remove those fields.

You may have noticed an odd field in the Issue domain class: bounty.  You might have expected to see a field for priority on the issue.  Instead, I chose to add a "beer bounty" field where the issue submitter could pledge a certain number of beers that I could redeem upon completion of the issue.  This is, in my opinion, far superior to simply assigning low, medium, high priorities to issues.

As a final customization, I want to convert the number of beers into little beer mug icons to make it easy to see the important issues to fix.  We'll do this by first copying the repeat example tag from the Dynamic Tag Libraries page of the documentation:
grails create-tag-lib Misc
This will create a grails-app/taglib/MiscTagLib.groovy file which we can add:

class MiscTagLib {
    def repeat = {attrs, body ->
        def i = Integer.valueOf(attrs["times"])
        def current = 0
        i.times {
            out << body(++current)
        }
    }
}


And we'll call it in our grails-app/views/issue/list.gsp:

<g:repeat times="${issue.bounty}">
    <img src="${createLinkTo(dir:'images', file:'beer.gif')}" alt="${issue.bounty} beers"/>
</g:repeat>


Here's a look at the final output:
In part 2, we're going to add in some security to prevent arbitrary user's from editing and deleting issues.  We'll also add in searching/filtering support with the Searchable plugin.

Cheers.


Searchable: Me Too!
2008-04-12T04:57:00.004-05:00
I've got to echo what seems to be the community consensus: use the Searchable plugin for Grails if you need to do any sort of searching or filtering.  It just works and it works damn well.  I'm using it to search/filter some lists of domain objects in my app with nice paging support.  I added the plugin to an existing, deployed application in less than an hour this afternoon.  The documentation was straightforward and easy to understand.

The only real trick I did was using the 'component' option for searching related domain classes.  Take my domain class below:

class SampleRequest {
    static searchable = true
    static hasMany = [samples: Sample]
    static mappedBy = [samples: "request"]
    static belongsTo = User

    // fields
    User investigator
    Hole hole
    Double top
    Double bottom
    String sampleType
    Integer samplesRequested = 1
    Double sampleSpacing = 0.0
    SampleGroup sampleGroup
    String notes = ""
    Date created = new Date()
    String status = STATE_NEW
    Integer priority = 1
}


By default, related domain classes such as User, Hole, and SampleGroup above are treated as references.  When I was searching for something like "micropaleo" which happens to be the name of a SampleGroup object, Searchable would return the actual SampleGroup but not all of the SampleRequest objects in that group.  Since I was mainly interested in the sample requests in that group, I simply changed my searchable definition to:

static searchable = {
    hole component:true
    investigator component:true
    sampleGroup component:true
}

and added static def searchable = true to my User, Hole, and SampleGroup domain classes.  So now when I search for something like "micropaleo" or "olney" the sample requests in that group or by that user are returned.

The best part is, my user's think I'm some sort of programming deity because they asked for search and I added it that same day.  Hopefully none of them read this blog and see how little work it was for me.

Cheers.


Visualizer, Part 1: Introduction
2008-04-10T00:07:00.009-05:00
On and off over the past couple of months, I've been working on an OpenGL visualization application, called Visualizer (original, I know), for viewing high resolution core imagery and data.  Visualizer is built on and with numerous technologies: Java, OSGi, JOGL, Eclipse, Ant, etc.  Along the way, I've collected a fair number of tips, tricks, and tidbits that I'm going to share here.  I hope to make Visualizer posts a regular feature of the blog, at least until the tidbits run out.  If you like it, please let me know by commenting.

A quick aside here: Those of you in geosciences might recognize this concept as Corelyzer.  My goal was to create a simplified, streamlined version of Corelyzer that would be easy to deploy on laptops/desktops; a sort of Corelyzer "lite" edition if you will. Visualizer doesn't have nearly as many features as the current version of Corelyzer, but it also doesn't suffer some of Corelyzer's warts.

I thought I'd start off with quick introduction of what Visualizer is for folks who aren't familiar with Corelyzer. During scientific geological drilling, hundreds or thousands of core samples are taken, imaged, and analyzed.  This results in several (tens of) gigabytes of data and imagery.  Collecting all of this data is of little use unless you have a way to view it.  This is where tools like Corelyzer and Visualizer come in.  They provide a way to view the images and data in context:


This picture illustrates Corelyzer in action.  Stretched across the two 30" cinema screens is a high resolution image of core drilled in Antarctica.  The user can zoom in or out and pan left or right to view the all of the imagery collected on the expedition.

So now that you know a bit of the background, let's look at the requirements (in no particular order) I had for Visualizer:
Cross platform - I have users on Mac, Windows, and Linux so I need a solution that will work for all of them.
Easy to deploy - Visualizer should work out of the box with no complicated setup for most users.
Easy to update - Needs a fairly automated mechanism to distribute updates and new features.
Open, extensible platform - Building on an open, extensible platform will allow me or others to quickly add new features or customize to a specific groups needs.
Reasonable performance - Visualizer should be able to handle 100m or so of imagery and data with reasonable performance on commodity hardware.
In the next blog post, I'll talk about the overall design of Visualizer and specific technologies employed.  Future blog entries will include: "OSGi & Native Libraries", "Poor Man's Eclipse PDE Build", "Nifty Ant Tricks", "Automated Update Checks", and anything else that I can think of.

Cheers.


Antarctica Service Medal
2008-04-08T19:50:00.005-05:00
Not too much technical content tonight.  I spent most of the afternoon trying to get OSGi working from a Java Webstart application without much luck.  It may be something that I revisit down the road, but for the time being distributing a zip with a set of shell scripts should work.  As part of the process, I also learned how to sign a Jar file, which will be useful in an upcoming project.

This morning I pushed out the first update to the Grails app I have in production.  Just before performing the actual update, I remembered to double check that my production DataSource was set to 'update' instead of 'create' or 'create-drop' the database.  It was lucky I checked because it was set to 'create' from when I was wrangling with MySQL.  I had already made a dump of the database, so I wouldn't have lost any data, but I was glad that I checked.

On the non-technical front, I received an Antarctica Service Medal in the mail for the time I spent in Antarctica:


I probably won't have too many occasions to actually wear it, but the medal is a nice memento of my time down there.

And the countdown has begun: 9 days until I'm in the Bahamas for my wedding and honeymoon.  Pretty crazy!


Pluggable Service Implementations in Grails
2008-04-06T09:45:00.007-05:00
This weekend I was working on a StorageService service in Grails.  The service takes a file, stores it somewhere, and then returns the URL where the file can be retrieved from.  I can envision having multiple implementations of this service: one that simply puts the file in a directory for something like Tomcat/Apache/Nginx to serve up, one that puts the contents of the file into a database blob and has a controller that retrieves it from the database and serves it up, and one that stores the file on Amazon S3.

I wanted to be able to easily specify which implementation to use without having to refactor my code and I wanted to be able to configure the implementation based upon the environment.  For development and integration testing, it would be easiest to test with a simple File-based service.  When I actually rolled it out to production, I would likely want to use something more scalable like Amazon's S3.

I didn't know how (if?) Grails handled this situation, so I decided to write my own.  I drew inspiration from Shawn Harstock's post about Fun with Grails Configuration Files for managing service configuration in the various environments.

The first thing I did was create a stub service, StorageService:

import org.codehaus.groovy.grails.commons.ApplicationHolder

class StorageService {
   boolean transactional = true
   boolean initialized = false
   def provider = null

   String store(File file) {
       // initialize our provider
       if (!initialized) {
           initializeProvider()
       }

       // use our provider or log the error
       if (provider == null) {
           // log the error
           return null
       } else {
           return provider.store(file)
       }
   }

   def initializeProvider() {
       initialized = true;

       // get our provider name
       def prefix = ApplicationHolder.application.config?.storage?.provider
       if (prefix == null) {
           prefix = "Default"
       } else {
           def index = prefix.lastIndexOf('.')
           if (index == -1) {
               prefix = prefix[0].toUpperCase() + prefix[1..-1]
           } else {
               prefix = prefix[0..index] + prefix[index+1].toUpperCase() + prefix[(index+2)..-1]
           }
       }

       // try creating the provider
       provider = Class.forName("${prefix}StorageProvider")?.newInstance()
   }
}

The service looks up and loads the storage provider class based on a configuration parameter.  For example, my Config.groovy file might look like this:

...
environments {
   development {
       storage {
           provider="file"
       }
   }
   test {
       storage {
           provider="org.andrill.mock"
       }
   }
   production {
       storage {
           provider="s3"
       }
   }

}
...


In development, the StorageService would try to load the class FileStorageProvider and delegate calls to it.  In testing, it would try to load the class org.andrill.MockStorageProvider and delegate calls to it.  And finally, in production it would try to use the S3StorageProvider.

My controllers don't need to know which provider implementation, they just declare a
StorageService storageService field, which Spring auto-wires for them, and go.  It works like a charm.

So my only question is, did I just re-invent the wheel?  Does Grails already have a mechanism for handling this?  If not, how would you accomplish the task?

Update: Check out David Hernández's explanation of how to do this with Spring.  No icky, ClassForName needed.  Nice work, David!

Cheers.


Grails and MySQL Connection Errors
2008-04-01T22:35:00.007-05:00
I ran into a few configuration problems when I tried to push a Grails webapp into production and make it talk to a MySQL database instead of the standard HSQL database.  These notes might be useful to someone who is pulling their hair out trying to get Grails talking to MySQL database.

My production machine is a CentOS 5 box running MySQL 5, Sun JDK 5, and Tomcat 5.5 (incidentally, my webapp's version happened to be 0.5 which rounds out the '5' theme nicely).

I started by pulling down the MySQL JDBC driver and dropping it into my lib directory.  I then updated the production section in my DataSource.groovy file:

...
 production {
       dataSource {
           pooling = true
           driverClassName = "com.mysql.jdbc.Driver"
           dbCreate = "create"
           url = "jdbc:mysql://localhost:3306/grails_webapp"
           username = "grailsuser"
           password = "XXXX"
       }
   }
...


I then grails wared things up on my local machine and copied it over to the production server.

Over on the production server, I connected to MySQL with the mysql command line utility to create my database and user:

create database grails_webapp;
grant all privileges on grails_webapp.* to 'grailsuser'@'localhost' identified by 'XXXX';
flush privileges;


I verified things worked by connecting to the new database as the newly created user and creating a table.

I dropped the war into my tomcat/webapps directory and watched the logs.  The webapp tries to start but quickly dies, leaving behind a log full of Communications link failure and Connection refused messages.  I could see that the MySQL JDBC driver was found, but for whatever reason it couldn't connect to the database server.

After double checking my DataSource.groovy file to make sure my URL, username, and password were all spelled correctly and connecting again locally from the command line to ensure that the database is indeed running, I started Googling.

Google revealed that one common problem is to forget to enable networking in MySQL.  I double checked my my.cnf and confirmed that I had enabled networking by specifying a port and an address to bind to.  This is where I noticed that the bound address wasn't localhost or 127.0.0.1, but the actual IP address of the machine.

Normally you would expect MySQL to only need to listen locally.  In my case, however, the production MySQL server replicates to a slave server for backup, so it needs to listen on the external interface.  The firewall locked down to prevent all connections to the MySQL port except from the slave server.

On a hunch, I went back and changed my database URL to jdbc:mysql://the.ip.addr.ess:3306/grails_webapp and tried re-deploying.  Things still failed, but with access denied errors instead of connection refused errors.  Progress.

The final piece of the puzzle was to go and grant my grailsuser access on all addresses instead of just localhost:

grant all privileges on grails_webapp.* to 'grailsuser'@'%' identified by 'XXXX';
flush privileges;


Once this was done, I re-deployed again and was greeted with an error free log.  Huzzah!


Back from Boston
2008-03-31T23:50:00.004-05:00
I made it back from Boston.  The trip was fun but I did not get as much done as I would have at home.  Things are looking to be pretty busy this week as I scramble to get caught up.  The wedding is in a few weeks.  10 days on the beach in the Bahamas.  That thought is what's getting me through the work and stress.

In other news, I'm starting a new project in my spare time.  I'm going to keep things under wraps for a bit as I don't have a solid time frame.  However, one thing I will mention is that it's going to be a webapp and it's going to contain a social component.  I'm hoping the development will yield a series of useful Groovy/Grails blog posts. 

I'd also really like to pull together a generic "web 2.0 template" project in Grails.  Something that would pull together plugins for security and contain User, Profile, Role-type domain classes, controllers, and views.  Ideally you should be able to use it to bootstrap a generic Web 2.0 site with all of the user registration and profile management taken care of.  Then you can start adding your specific domain classes to make the site useful.


Let's Get Crackin'
2008-03-28T17:33:00.002-05:00
We had dinner at the Barking Crab last night:


Definitely a highlight.  It was a really good dinner and I think we looked exceptionally good in our bibs.


Blogging from Boston
2008-03-27T13:46:00.003-05:00
I'm currently at the National Science Teacher Association (NSTA) national meeting in Boston.  I'm just here to help out with the ANDRILL booth and get some much needed face to face time with my co-workers (something we don't often get since I work remotely).  The trip has been fun so far, though not as productive as being at home.

I took a few minutes today to fix a bug in the Simile Timeline depth code--when scrolling the timeline, the overview bars would get out of sync.  If you're using the depth unit extension, please grab the latest version or check out the diff.  I've also updated the Simile Grails plugin, so you may want to update your version from the repository.  It fixes the same depth bug and updates the SimileTagLib to work a bit more independently from the domain classes.


Customizing Timeline Event Bubbles
2008-03-24T19:17:00.006-05:00
Tonight is a two for one special on blog posts.  I'm heading out to Boston tomorrow for the rest of the week, so I don't know how much time I'm going to have to blog while out there.  If you happen to be reading and are going to be in the Boston area (either for the NSTA meeting like me or whatever), shoot me an email and let's meet up for a beer.

In this post, I'm going to show you how to customize the rendering of event bubbles in Simile Timeline.  The default bubbles look pretty nice:
but they lack a few features that I needed, namely clicking on the thumbnail (if present) should take the user to the event link and the start and end depth should be rendered on a single line:

The markup for the event bubble is built programmatically in the fillInfoBubble() function on Timeline.DefaultEventSource.Event class.  You can find the code at the end of the sources.js file in the Timeline distribution.

To customize the event bubble rendering, we just need to modify the fillInfoBubble() function and make it generate the markup we want.   One option would be to go in and edit the sources.js file directly.  This would accomplish our goal but it would mean we'd be modifying the Timeline source code and we would have to patch the file every time it was changed in the Timeline trunk.

The second option is to replace the function "on the fly" by modifying the Timeline.DefaultEventSource.Event prototype in our own code.  The end is the same but we can do it without touching the Timeline sources.  To accomplish this, we just need to overwrite the function after sources.js loads:

(function() {
Timeline.DefaultEventSource.Event.prototype.fillInfoBubble = function (elmt, theme, labeller) {
// build our custom markup
};
})();


What this code does is create an anonymous function that runs immediately.  Inside that function, we replace the current fillInfoBubble() implementation on the Event class with our own.

This code is an improvement over editing sources.js directly, but we can do one better.  The problem with this approach is that we're replacing fillInfoBubble() for everyone.  We're breaking any clients that depend on the old implementation, which is just bad form.  What we'd really like to do is selectively apply our new implementation.  Fortunately that's easy enough to do:

(function() {
var default_fillInfo = Timeline.DefaultEventSource.Event.prototype.fillInfoBubble;
Timeline.DefaultEventSource.Event.prototype.fillInfoBubble = function (elmt, theme, labeller) {
   // if not a depth labeller, use the original
   if (!labeller.isDepthLabeller) {
       default_fillInfo.apply(this, arguments);
       return;
   }
 
   // build our custom markup
};
})();


In this updated snippet we save the old function implementation then we check to see if the labeller is a DepthLabeller.  If it isn't, we just pass it on to the original implementation, otherwise we use our customized implementation.

For completeness sake, here's the snippet I used to make the thumbnail image clickable and to render the depths on the same line:

if (image != null) {
   var img = doc.createElement("img");
   img.src = image;
   theme.event.bubble.imageStyler(img);
         
   if (link != null) {
       var a = doc.createElement("a");
       a.href = link;
       a.target = "_blank"
       a.appendChild(img);
       elmt.appendChild(a);
   } else {
       elmt.appendChild(img);
   }
}

var divTime = doc.createElement("div");
if (this.isInstant()) {
   divTime.appendChild(elmt.ownerDocument.createTextNode(labeller.labelPrecise(this.getStart())));
} else {
   divTime.appendChild(elmt.ownerDocument.createTextNode(labeller.labelPrecise(this.getStart())));
   divTime.appendChild(elmt.ownerDocument.createTextNode(" - "));
   divTime.appendChild(elmt.ownerDocument.createTextNode(labeller.labelPrecise(this.getEnd())));
}


Hope that was clear as mud!


Grails Query By Example Gotcha
2008-03-24T17:03:00.005-05:00
This is not actually a gotcha in Grails but more a gotcha in my domain class.  However, it seems like it might be a common enough scenario to trip up someone else.

I have a domain class which I am populating from an uploaded Excel spreadsheet.  Each row in the spreadsheet corresponds to a new instance of the domain class.  Since I anticipate that my users will try to upload spreadsheets with errors in some of the rows (not pessimistic, just realistic), I want my app to accept all rows that are properly formatted and provide useful error messages for the rows that are not.  I created a spreadsheet that exercised all of my error cases and ran into an unexpected outcome: every time I uploaded the spreadsheet I got duplicates of all of the valid rows in the spreadsheet. 

This was surprising because I was using the query by example facilities to check and see if the row was a duplicate.  Despite not changing the spreadsheet between uploads, I was still getting duplicates.  Then it dawned on me--I had a Date created = new Date() field in my domain class to keep track of when the domain class was created.  Simply adding an example.created = null to my query example solved the problem.

Like I said in the opening paragraph, not a gotcha in Grails.  The query by example stuff works exactly how I expect it to, you just have to watch out about initialized fields.


Grails Excel File Download
2008-03-21T20:55:00.007-05:00
This is the companion to my previous "Grails Excel File Upload" post.  In this post, I'll show how to create and send an Excel spreadsheet from your Grails webapp.

The first thing you'll need to do is go out and grab JExcelAPI if haven't already.   JExcelAPI will let you read and write Excel spreadsheets from Java.  Drop the jxl.jar in your webapp's lib directory.

In my app, I'm providing Excel download functionality from a couple different controllers, each with a different list of domain objects.  To enable the highest degree of re-use, I wrote a generic utility method for serializing an object to a row in an Excel spreadsheet:

 import jxl.*
 import jxl.write.*

...

 static def writeExcel(out, map, objects) {
     // create our workbook and sheet
     def workbook = Workbook.createWorkbook(out)
     def sheet = workbook.createSheet("Requests", 0)
 
     // walk through our map and write out the headers
     def c = 0
     map.each() { k, v ->
         // write out our header
         sheet.addCell(new Label(c, 0, v.toString()))
     
         // write out the value for each object
         def r = 1
         objects.each() { o ->
             if (o[k] != null) {
                 if (o[k] instanceof java.lang.Number) {
                     sheet.addCell(new Number(c, r, o[k]))
                 } else {
                     sheet.addCell(new Label(c, r, o[k].toString()))
                 }
             }
             r++
         }
         c++
     }
 
     // close
     workbook.write()
     workbook.close()
 }


Edit: See Ted's comment on this post about using eachWithIndex() instead of keeping track of the r and c vars yourself.

This method takes an OutputStream, like the ServletOutputStream you can get from response.outputStream; a map of property names and "nice" header titles; and a list of objects to write to the spreadsheet.

Then I call it from my controller with appropriate values:

     def header = [:]
     header.id = "Id"
     header.investigator = "Investigator"
     header.hole = "Hole"
     header.top = "Interval Top (mbsf)"
     header.bottom = "Interval Bottom (mbsf)"
     header.samplesRequested = "Samples Requested"
     header.sampleSpacing = "Sample Spacing (m)"
     header.sampleType = "Volume/Type"
     header.sampleGroup = "Group/Discipline"
     header.notes = "Notes"
     header.status = "Status"
     header.priority = "Priority"
 
     ExcelUtils.writeExcel(response.outputStream, header, SampleRequest.findAllByHole(hole))


The final piece of the puzzle is to make sure you set your content type and content disposition headers before you call the writeExcel() method:

    // set our header and content type
    response.setHeader("Content-disposition", "attachment; filename=${hole}-requests.xls")
    response.contentType = "application/vnd.ms-excel"


The nice part about the writeExcel() method is that it works with just about any list of objects and can easily be customized.  For example, in another place in my app, I let the user download all of the sample request domain objects associated with them.  To do that, I use nearly the same code:

     // set our header and content type
     response.setHeader("Content-disposition", "attachment; filename=${user}-requests.xls")
     response.contentType = "application/vnd.ms-excel"
 
     // define our header map
     def header = [:]
     header.id = "Id"
     header.hole = "Hole"
     header.top = "Interval Top (mbsf)"
     header.bottom = "Interval Bottom (mbsf)"
     header.samplesRequested = "Samples Requested"
     header.sampleSpacing = "Sample Spacing (m)"
     header.sampleType = "Volume/Type"
     header.sampleGroup = "Group/Discipline"
     header.notes = "Notes"
 
     ExcelUtils.writeExcel(response.outputStream, header, SampleRequest.findAllByUser(user))


I removed a few redundant/unimportant properties from my header map and call the writeExcel() method with the list of SampleRequests associated only with that user.


And the match is...
2008-03-20T16:19:00.002-05:00
Minneapolis.  We were pretty sure Elizabeth would match here since it is a big program and she had strong recommendations from a lot of the doctors in the program but it was still a little stressful.  With that hurdle out of the way, I'm off to Boston next week for the NSTA national meeting.  I'm just there helping out with the ANDRILL booth, so hopefully I can find plenty of time to work as things are shaping up to be a death march until I leave for the wedding on the 17th.  We're rolling out a new sample request management system on April 1, revamping the whole ANDRILL website right around then, and preparing for the SMS post-drilling workshop at the end of April.  I can't tell you how much I'm looking forward to spending a week on the beach in the Bahamas.


Grails Excel File Upload
2008-03-19T14:05:00.008-05:00
Just like most things in Grails, parsing data from an uploaded Excel file is relatively easy.  The first thing I did was go out and grab JExcelAPI.  JExcelAPI will let you read and write Excel spreadsheets from Java.  It doesn't support all of the advanced features (like charts) of Excel, but for my needs it works well.  You can also look at Apache POI for reading and writing more MS Office formats, and there's numerous other options if you're on Windows or want to plop down some money for a commercial library.

After dropping JExcelAPI into my lib directory, I went ahead and created a view to upload the spreadsheet.  The form can be as simple or as fancy as you want, just as long as you have a file input field:

<g:form action="upload" method="post" enctype="multipart/form-data">
<label for="file">File:</label>
<input type="file" name="file" id="file"/>
<input class="save" type="submit" value="Upload"/>
</g:form>
Now we need to create the upload method in our controller to parse the Excel file:


def upload = {
 // get our multipart
 MultipartHttpServletRequest mpr = (MultipartHttpServletRequest)request;
 CommonsMultipartFile file = (CommonsMultipartFile) mpr.getFile("file");
  
 // create our workbook
 Workbook workbook = Workbook.getWorkbook(file.inputStream)
 Sheet sheet = workbook.getSheet(0)

 def added = 0;
 def skipped = [];
 for (int r = 3; r < sheet.rows; r++) {
  // get our fields
  def top = sheet.getCell(0, r).contents
 
  def bottom = sheet.getCell(1, r).contents
  if (bottom == "") bottom = null
 
  def number = sheet.getCell(2, r).contents
  if (number == "") number = 1

  def spacing = sheet.getCell(3, r).contents
  if (spacing == "") spacing = 0.0
 
  def type = sheet.getCell(4, r).contents
  def notes = sheet.getCell(5, r).contents
 
  // check that we got a top and a type
  if (top == null || top == "") {
   // do nothing
  } else if ((new SampleRequest(
    "investigator":user,
    "hole":hole,
    "sampleGroup":group,
    "top":top,
    "bottom":bottom,
    "sampleType":type,
    "samplesRequested":number,
    "sampleSpacing":spacing,
    "notes":notes)).save()) {
   added++
  } else {
   skipped += (r + 1)
  }
 }
 workbook.close()

 // generate our flash message
 flash.message = "${added} sample request(s) added."
 if (skipped.size() > 0) {
  flash.message += "  Rows ${skipped.join(', ')} were skipped because they were incomplete or malformed"
 }
 redirect(controller:"home", action:"index")
}


My spreadsheet has a fairly simple format with fixed columns, some being optional and some required, so I hard coded the column indices.

In a future blog post, I'll show how to output an Excel spreadsheet with Grails and JExcelAPI.

Edit: Blogger seems to have eaten some of the code when I updated tags so I just updated it.


Screencasts on Macs
2008-03-18T17:47:00.003-05:00
I've found myself doing a lot of screencasts recently.  Screencasts are great for demonstrating software or techniques when you can't show the user directly.  In some ways, a good screencast is even better than a good live demonstration because the user can go back and watch the screencast any time they want.

For creating screencasts on my Mac, I use iShowU.  iShowU is one of the first pieces of software I purchased for my Mac.  It's nice looking and easy to use.  And it has a plethora of options to create videos in many formats and quality levels.  At $20, it's a steal.  I only wish it had an option to make me sound more intelligent and confident.  I usually end up doing several takes before I'm not appalled with how I sound on video.

iShowU doesn't encode to Flash videos for the web, so that may be a non-starter for you.  I usually just end up distributing the video file directly.


Anxious Anticipation
2008-03-17T08:56:00.004-05:00
This is a pretty important week for Elizabeth and I.  On Thursday we find out where she "matched" for her residency, which will dictate where we end up living for the next four or so years.  Her top choice was to stay here in Minneapolis.  The University of Minnesota has a really good program in what she is going into (MedPeds--a combination of Internal Medicine and Pediatrics).  Her second choice is Milwaukee, WI.  After that, the list includes Indianapolis, IN; Chicago, IL; Peoria, IL; and Rochester, NY. 

Obviously Minneapolis would be the simplest all around option.  We're already situated here, so we wouldn't have to worry about moving.  Oddly enough, Milwaukee is also intriguing to me.  Maybe it's the beer and brats.  It's also still close enough that we could get back to see friends and family for the holidays and long weekends.  The big downside to Milwaukee, or any place other than Minneapolis, is that we'd have to move on relatively short notice.  We find out on March 20th and have to be moved by June 15th or so.  Factor a wedding and a reception into that timeframe and we've only got about a month to find a place and get moved in.

We're both waiting for Thursday with bated breath.


Simile Grails IE Bug Fixed
2008-03-14T09:32:00.002-05:00
Just a quick post to let anyone interested in the Simile depth extension or Grails plugin that I fixed a small bug in IE that was preventing the depth stuff from loading.  It's been pushed to the repository.


Simile Timeline and Grails
2008-03-12T23:56:00.007-05:00
I just put the finishing touches on a Grails plugin for Simile Timeline.  There's already the RichUI Grails plugin which provides a nice wrapper for incorporating Timeline into your Grails app.  However, I couldn't use RichUI because it links to the libraries out at the Simile site instead of serving them up locally.  This causes cross-domain issues when using my depth extension.

Developing the plugin wasn't too terribly difficult and it was a great opportunity to dig into the internals of Grails a bit.  Its plugin architecture is pretty slick.  I actually created my plugin directly inside the plugins directory of a webapp I was working on.  This allowed me to develop on both the webapp and the plugin at the same time.  It's a pretty nice way to work when you have a pretty decent understanding of which pieces you want to go in your webapp and which you want to go in your plugin.

My Simile Grails plugin does a bit more than just packaging the Timeline javascript for serving locally.  I put together a handy SimileTagLib for including the libraries so you can simply throw a <simile:ajax/>
 and a <simile:timeline/>
 tag in your view and that will include the javascript libraries.  I ran into a bit of a stumbling block when trying to do this because the Javascript code is actually in the plugin.  Hardcoding the links wouldn't work because I would have to change the link every time I updated the plugin's version number (Grails installs a plugin as {plugin.name}-{plugin.version}).  Fortunately, you can look up the plugin's name and version at runtime:

def p = PluginManagerHolder?.pluginManager.getGrailsPlugin('simile')
return "<script type="text/javascript" src="${request?.contextPath}/plugins/${p.name}-${p.version}/js/${lib}.js?"></script>"

You may be wondering why I called it the Simile plugin instead of the Timeline plugin or what have you.  I actually made the plugin generic enough that if I have a need to later, I can package all of the Simile code in the same plugin.  The Timeline code already relies on the Simile Ajax library, so the plugin actually packages both libraries.

Along with the taglib, I also created some Timeline-related domain classes for working with timelines and events.  I used these domain classes to bootstrap the example webapp I was working on.  I simply did a:

grails create-app timelines
cd timelines
grails install-plugin simile
grails generate-all Timeline  (domain class provided by the plugin)
grails generate-all TimelineEvent  (ditto)
grails run-app

and I was up and running with a webapp that could create and edit timelines and events.  Actually displaying the timelines took a bit more customizing, but not all that much.

I've made the source code for the completed webapp available via a Mercurial repository.  The webapp gives you a CRUD style interface for creating timelines and timeline events.  It supports both date-type of timelines as you see in the examples out at the Timeline site, as well as depth-type of timelines with my depth extension.  I've put up a live demo on my development server at work.  Feel free to play around with it by creating your own timeline and adding events.  It'll stay up until as long as it isn't abused.


Simile Timeline
2008-03-10T20:10:00.007-05:00
Over the last week or so, I've been working on a webapp for managing expedition sample requests.  One of the bigger challenges has been to find an effective way to visualize the sample requests.  Traditional histograms and line graphs are an option, but they offer limited interactivity.  I was hoping to find something more "snazzy" to let the users really explore the data.

I mentioned this in passing to my friend Doug Fils and he pointed me to Simile Timeline.  Timeline is something I had seen in passing before but hadn't considered for this particular project.  Timeline provides a Google Maps-type interface for exploring time-based data.  I really liked the interface, but needed a way to put depth on the scale instead of time.  It turns out I wasn't the only one looking for different units.

After chiming in on enhancement request, the original reporter, Charlie Kershner, sent me an email with the solution he'd come up with: use years, decades, centuries, and millennia as proxies for metric depth units.  The conversion factors are all right, it's just the units that are wrong.  This is a really clever solution,   so I was glad Charlie passed it along.  Using it, I had a working demo in about an hour.

I decided to poke around in the Timeline code to see what it would take to properly support depths.  It is a really coherently designed tool with well defined interfaces for extension and customization.  I can't claim to be fully understand everything, but with a little monkey see, monkey do, and a bit of brute force, I got proper depth support:


This is the default rendering.  I think for my application, I'll end up customizing the event painters.  I need to find a way to simplify the rendering, especially in busy sections like this.  There's also a fair number of errors in the data that I need to go back and fix.  Being able to visualize it in this form makes it really easy to see the errors (Hint: those blue bars at the top and bottom of the main band represent typos in the data.  There were no multi-meter samples...)

I don't have a live demo for people to play with because I'm working with "live" data that isn't public.  However, there are numerous examples out at the Timeline site.   There were a few gotchas I ran into while developing and testing things:
You need to serve your own copy of Timeline because of cross-domain issues with mixing code from the Simile site and from a local extension.
You need to tell Timeline to use your new unit in two different places--when you create the event source and when you actually create the timeline.  Failure to do so will have you pulling out your hair chasing down errors.
You need to use your customized methods to create your bands
Not all of the rendering options (like showEventText: false) seem to work.  I think this is because I don't fully understand the rendering pipeline and how it is customized.  This is one area I'm going to be looking at in the future.
Here's an example of the Javascript to create the the example above.  Note the bolded areas that were mentioned in #2 and #3:

var tl;
function onLoad() {
var eventSource = new Timeline.DefaultEventSource(new SimileAjax.EventIndex(Timeline.DepthUnit));

 var theme = Timeline.ClassicTheme.create();
 theme.event.bubble.width = 320;
 theme.event.bubble.height = 220;
 var d = Timeline.DepthUnit.wrapDepth(0)
 var bandInfos = [
     Timeline.Depth.createBandInfo({
         eventSource:    eventSource,
         date:           d,
         width:          "80%",
         intervalUnit:   Timeline.DepthUnit.METER,
         intervalPixels: 100
     }),
     Timeline.Depth.createBandInfo({
         eventSource:    eventSource,
         date:           d,
         width:          "20%",
         intervalUnit:   Timeline.DepthUnit.DECAMETER,
         intervalPixels: 25,
         overview:       true,
         convertToBaseUnit: true
     })
 ];
 bandInfos[1].syncWith = 0;
 bandInfos[1].highlight = true;

 tl = Timeline.create(document.getElementById("tl"), bandInfos, Timeline.HORIZONTAL, Timeline.DepthUnit);
 tl.loadJSON("samples.js", function(json, url) {
     eventSource.loadJSON(json, url);
 });
}
var resizeTimerID = null;
function onResize() {
 if (resizeTimerID == null) {
     resizeTimerID = window.setTimeout(function() {
         resizeTimerID = null;
         tl.layout();
     }, 500);
 }
}

The code for the depth extension is available in a Mercurial repository where you can grab a zip of the most recent version.  Feel free to use it however you like.   I've offered to contribute the code to the main Timeline repository so future versions may not require you to host your own Timeline.


A Java on Linux Aside
2008-03-08T16:27:00.004-06:00
So I promised a technical post but I ran into a few problems Friday afternoon that ate several hours of my evening.  I was trying to deploy a Grails webapp onto a CentOS server of mine.  CentOS wasn't by choice, it was dictated to me.  I have much more experience with Debian and it's related derivatives like Ubuntu.  Anyhow, I was trying to deploy a Grails app but it required Java 1.5/5 and all that was installed was Java 1.4.2.  And to make things worse, it was the GCJ version of Java.  I understand the philosophical arguments behind what the GCJ folks' reasoning.  Sun's Java licensing terms wouldn't allow Linux distros to re-distribute the code.  The GCJ have put together a completely free, completely re-distributable version of Java.  The problem is, it just doesn't always work as well as Sun's JDK.  The blame is as much the application developers as it is GCJ.  But to make a long story short, my Grails webapp wouldn't work with "Java" as it is installed on CentOS.


This Java problem is/was (now that new versions of Java are open source) common to all distros.  One thing that surprised me about CentOS, was that they didn't have a repository with Java pre-built and packaged.  On Ubuntu, all I had to do was add a "non-free" repository and I could install the Sun JDK straightaway.  And, once installed, it integrated directly with the system.  On CentOS, I had to download the Java binary from Sun.  Then I had to download a source RPM from another place.  Then I had to manually edit the RPM spec file to match the version of the JDK I got.  And then I had to setup an RPM build environment and rebuild the Java RPM with the binary bits from Sun.  That ended up taking over an hour to get everything set up properly. 

Once I finally got the RPM built, I thought my troubles were over.  I manually installed the shiny new RPMs that I built, updated the alternatives, and checked to ensure that 'java -version' was calling the Sun JDK.  So done and done I thought, but I was wrong.  Imagine my surprise when I started up Tomcat 5 and it was still using the GCJ version of Java.  What gives?!  I made sure the default Java was the Sun JDK.  So then I decided I'd never need the GCJ Java, so I decided to uninstall it completely.  Imagine my surprise when Tomcat 5 decided it needed to uninstall itself with the GCJ Java. 

I fiddled around with it for another hour trying to get things to work with no luck.  So I ended up manually installing both the Sun JDK and Tomcat 5 in /opt and will manage it myself.  This finally got things to the point where I could deploy my Grails app however, I lose all of the system integration.  A user on the server can no longer just type java and expect it to work.  They have to fiddle with their path and their JAVA_HOME variables to make things work.  I guess the biggest disappointment to me was that I ended up settling on the solution that I had used several years ago while working on RedHat boxes.  After all these years, and the advent of their 'yum' package management system, and even generating packages by hand, you still can't get the Sun JDK package to integrate properly with RedHat/CentOS.

I'm not saying it's RedHat's or CentOS's fault.  I guess I've just had better experience with Debian/Ubuntu.  I don't know if it's me and my rusty experience with RedHat-based distros, the (lack of) documentation, or the package management system.  I'm sure it's a bit of all of them.

All in all, the Java situation is a mess on all of the major platforms.  I think Windows got their growing pains out of the way early with the whole Microsoft Java fiasco.  On Linux, it's been the inability to re-distribute the Sun JDK because of licensing issues.  That should improve now that the newer versions of Sun's JDK are open source.  Finally, the Java situation on Macs is wearing my patience pretty thin.  It's about time Apple gets a version of Java 6 out that runs on my laptop.  I've already shelled out the money to upgrade to Leopard with the expectation that I'd be able to get the next version of Java.

In the next post, I'll be talking about some work I've been doing with Simile Timeline.


Bacheloring it up
2008-03-06T22:42:00.003-06:00
Elizabeth went to Rochester today through Sunday.  She's having her final wedding dress fitting and a bridal shower.  That leaves me and the cat to fend for ourselves.  Tonight I had the boys over to watch some college basketball and to try out the cream stout Will and I just kegged.  Everyone agreed that it turned out well.

Tomorrow I really need to finish up some Grails work I've been doing and to get a few screencasts shot.  Since Elizabeth is gone, I'll probably end up working Saturday to catch up on some PSICAT work that I've fallen behind on.  Apologies to anyone who's waiting on me.  ANDRILL is keeping me busy.

Tomorrow I'll have a technical post.  I haven't decide what yet, but I'll figure something out.


Automatic change tracking with cron and Mercurial
2008-03-05T16:27:00.005-06:00
A while back I had to setup a system for sharing ANDRILL's expedition data among a geographically distributed group of scientists.  For simplicities sake, I settled on serving the data via HTTP (for read only access) and WebDAV (for read/write access).  It's not the most robust system, but it works, is relatively well supported across modern operating systems, and doesn't require users use some special software.

The only problem is that allowing read/write access to a large group of people means that something is going to get inadvertently changed or deleted.  To combat this, I needed an automatic way to track changes that would allow me to revert or rollback any accidents.

The first step was to put the expedition data into a standard version control system.  My VCS of choice is Mercurial, so I opted to use that.  The initial import took a bit of time because there was about 30GB of files.

Once I had all of the data safely imported into a Mercurial repository, I needed a way to automatically detect and commit changes.   I worked up a fairly simple shell script that gets run periodically by cron to check for and commit any changes:

#!/bin/sh

REPOS="/home/projects/sms/"
DATE=`date`

for r in $REPOS
do
 echo "Working on $r"

 # clean up temporary files
 find $r -type f -name "._*" \! -wholename "*.hg*" -exec rm {} \;

 # find changes
 hg addremove -R $r
 hg commit -m "Automated commit @ $DATE" -R $r
done


The script is relatively straightforward.  It will walk through a list of repositories (in this case, only 1 is configured).  Fore each repository, it first cleans any dangling WebDAV lock files outside of the Mercurial metadata directory.  Then it runs hg addremove to detect any added or deleted files.  Finally it commits the changes with an automated commit message containing the current date and time.

The script is configured to run periodically (hourly) by cron.  This has the side effect that if the same file is modified two separate times within the same hour window, the intermediate changes won't be caught.  For my needs, this wasn't a big deal.  You could always run the script more often to have a better chance of catching more changes.

The final piece of the puzzle was to send out a daily email to the science team to notify them of the changes.  For this, I developed another script that is run once a day:

#!/bin/sh

REPOS="/home/projects/sms/"
DATE=`date`

for r in $REPOS
do
 LOG=`hg log -d -1 --template '{rev}\n' -R $r`
 if [ -z "$LOG" ]; then
  echo "No changes"
 else
  echo -e "The following files have changed in the last 24 hours:\n" > /tmp/hg-tc-daily.log
  for c in $LOG
  do
   hg log --rev $c --template 'Changeset {rev} ({date|isodate}):\n' -R $r >> /tmp/hg-tc-daily.log
   hg status --rev `expr $c - 1`:$c -R $r >> /tmp/hg-tc-daily.log
   echo "" >> /tmp/hg-tc-daily.log
  done
  cat /home/projects/hg-tc/summary.legend >> /tmp/hg-tc-daily.log
  cat /tmp/hg-tc-daily.log | mail -s "Expedition Data Changes" foo@bar.com
 fi
done


If you can grok that in a single glance, you're a better shell scripter than me.  It took me a couple passes to write the script and get it working the way I wanted it to.  There's a bit of magic going on here, but I'll break it down.

This script walks through a set of repositories like in the previous script.  The first magical incantation we run into is:
LOG=`hg log -d -1 --template '{rev}\n' -R $r`

This line asks Mercurial to tell us the revision numbers for all changes in the last day (-d -1) and stores them, one to a line, in the LOG variable.  We then check to see whether the LOG variable contains anything.  If it doesn't, then we're done and can exit.  

If the LOG variable does contain something, then we need to send out an email to notify people of the changes.  We start building up our email in /tmp/hg-tc-log.  Next we walk through our list of revision numbers, and for each one we include some information using this incantation:

hg log --rev $c --template 'Changeset {rev} ({date|isodate}):\n' -R $r >> /tmp/hg-tc-daily.log
hg status --rev `expr $c - 1`:$c -R $r >> /tmp/hg-tc-daily.log


The first line simply prints out "Changeset 53 (2008-03-04 15:01 -0600):" or something similar for every changeset that occurred in the last 24 hours.  The second line prints out the list of changed files and the change type (added, deleted, modified) from the previous previous revision (--rev `expr $c - 1`:$c).  

Finally the script adds a helpful legend to the email and sends it out.

Overall, it's a bit hacky, but it works.  The team can continue to collaborate in a central location with the relative safety of using a VCS to track changes.  The best part is they don't have to do anything different; the change tracking is all automatic.  I've even had the occasion to revert inadvertent changes and deletions, so it was a good investment of an afternoon.


rotateLeftAndFlipInSitu
2008-03-04T19:16:00.004-06:00
Recently I was working with some large (200MB) vertically-oriented core images in Java.  I needed a way to rotate these images CCW so they were horizontally-oriented and flip them vertically for display on an OpenGL canvas.  It's a pretty easy task if you're using the Java2D API:


       // get our dimensions
       final int w = image.getWidth();
       final int h = image.getHeight();

       // create our new image and graphics
       final BufferedImage out = new BufferedImage(h, w, image.getType());
       final Graphics2D g2 = out.createGraphics();

       // setup up an affine transformation
       final AffineTransform at = AffineTransform.getRotateInstance(Math
               .toRadians(-90.0), h / 2, w / 2);
       at.translate((h - w) / 2, (w - h) / 2);
       g2.drawRenderedImage(image, at);
       g2.dispose();
       return out;


That snippet will rotate your image to the left.  If you twiddle with the affine transform, you can also get it to flip the image for you.  The only problem with this approach is that it requires you allocate a second image to hold the results of the transformation.  So rotating a 200MB image actually requires 400MB of memory (give or take).  This isn't a big deal if I only ever needed to rotate one large image at a time, but in the application, I didn't know how many large images the user was going to be working with at a time.  So I decided to see if I could do the rotation in O(1) space complexity.

It turns out that this is actually a pretty difficult problem.  It's been studied since the 70s under the guise of "in-situ transposition of a rectangular matrix".  Once I found this out, it became a personal quest to come up with a solution.  In the end, I was successful:


  // get our sample model and data buffer
  final SampleModel model = image.getSampleModel();
  final DataBuffer db = image.getRaster().getDataBuffer();

  final int w = model.getWidth();
  final int h = model.getHeight();
  final int[] t1 = new int[model.getNumBands()];
  final int[] t2 = new int[model.getNumBands()];

  // transpose
  int i, j, k, movesLeft;
  for (i = 0, movesLeft = w * h; movesLeft > 0; i++) {
   for (j = pred(i, w, h); j > i; j = pred(j, w, h)) {
    // spin
   }

   if (j < i) {
    continue;
   }
   for (k = i, j = pred(i, w, h); j != i; k = j, j = pred(j, w, h)) {
    // get our pixels
    model.getPixel(k % w, k / w, t1, db);
    model.getPixel(j % w, j / w, t2, db);

    // swap them
    model.setPixel(k % w, k / w, t2, db);
    model.setPixel(j % w, j / w, t1, db);
    --movesLeft;
   }
   --movesLeft;
  }

  // now return a new image with an updated sample model
  final WritableRaster raster = Raster.createWritableRaster(model
    .createCompatibleSampleModel(h, w), db, null);
  return new BufferedImage(image.getColorModel(), raster, image
    .isAlphaPremultiplied(), null);


with pred() defined as:


  return (i % h) * w + (i / h);


This algorithm works.  It'll rotate the image in place.  Thus, if you could load the image in the first place, you shouldn't run out of memory when you rotate it.  

With that being said, this approach is pretty naive.  There are numerous optimizations that could be implemented, from increasing the amount of temporary storage you use to optimizing the look up so you spend less time spinning.

In the end, I didn't end up using the code in production.  It was a fun exercise to work through, though.


Wedding Update
2008-03-04T18:40:00.004-06:00
Our wedding plans are progressing nicely.  The actual wedding is on April 19th in the Bahamas (Euleuthera Island).  It's going to be small; only a few family and friends.  We're all flying down there on Thursday, the 17th.  We'll have Thursday afternoon, all day Friday, and most of the day Saturday to relax and hang out.  The actual wedding ceremony is going to be pretty low key.  We're going to be married on the beach in late afternoon on Saturday.  For the honeymoon, Elizabeth and I have rented a little house on a private beach for the following week.  We return to the real world on the 27th, and a couple weeks after that, on May 16th, we're going to have a reception in St. Paul.


A blog reborn...
2008-03-04T18:09:00.003-06:00
It's about time I reclaim this blog from dormancy.  It's been a hectic couple of months since I returned from the ice.  Things, however, are progressing well so it's about time I start blogging again.  I'm changing the name of this blog to "Josh (Formerly) In Antarctica" as there are no Antarctic trips in the foreseeable future.  I'm going to keep the blog format the same--pretty informal with a mix of personal and technical entries.  I may even start tagging my posts so folks who are only interested in technical content don't have to wade through all of my posts.


Happy New Year
2008-01-03T10:28:00.000-06:00
Happy New Year to everyone out there! 

I made it home from Antarctica on Dec. 15th and I've been just relaxing and visiting family since then.  With the new year, it's time to start whittling away at the pile of work that has built up.  Even though the field portion of the expedition is over, my job doesn't change much--only the location where I'm working from.  There are fewer day-to-day surprises that pop up, but there is still plenty to do on the data management, software development, and web fronts.  And I guess that means I'd better get back to it!


So long, and thanks for all the...chicken
2007-12-13T14:07:00.000-06:00
In a few short hours, I'll be hopping on a C-17 cargo plane and heading home by way of New Zealand.  Leaving is somewhat bittersweet as I don't know when or even if I'll be back to Antarctica again.   People often ask me "Why Antarctica?"  It's such an amazing and intense place.  You end up living and working in such close quarters with people that you can't help but get to really know them.  The quality of friendships forged down here amazes me the most.  I feel very fortunate to be walking away from this experience with a bunch of really great friends.  The other great thing about Antarctica is that it has forced me out of my comfort zone and allowed me to learn a lot about myself and the world.  And you can't beat the scenery.  I've seen landscapes that take your breath away.

With all that said though, I'm really excited to get home and settle back into life.  Hopefully I'll be able to relax for a few days before starting back to work.    There's also a lot of work in the coming weeks and months to get Elizabeth and my wedding and reception arranged.  She's been handling all of the details while I've been gone so it's about time I go back and start lending a hand.

Some of you may remember that I had a bet going about not eating Frosty Boy.  I'm proud to say that I made it the whole season, as did Cara.  So yesterday we met at dinner and each had a bowl.  It was delicious, but I'm still glad I held out.

Cheers.


A harrowing journey
2007-12-09T20:03:00.000-06:00

Yesterday I barely returned from a traverse to a geological feature called Castle Rock.  The conditions were appalling.  The sun beat down from a clear blue sky and a cold wind howled down the ridge.  It was like hiking across a mirror under a full afternoon sun.  My face bore the brunt of the sun and wind's assault.  My sunglasses were the only thing that saved me from snow blindness.

In all seriousness, though, it was pretty foolish of me to forget to put sunscreen on while out hiking for 4 hours.  The sun down here ravages your face if you're out for any length of time.  And I'm paying the price today.  It's been a rather hard season on my face, with frostbite early on and sunburn now.  I'm going to come home and be pretty leather faced.  The trip yielded some nice photos, though:





Final Week
2007-12-07T01:43:00.000-06:00
This marks the start of my final week on ice.  We got up early this morning and saw a large group of ANDRILL folks off.  With this mass exodus, our numbers are cut in half.  It was hard seeing people go, but there's still a lot of work to be done and I will see many of them in Florida in a couple of months.

The last couple of days have been alternating between full on and relaxed.  I'm sitting in an office with Rich, Laura, and Leslie, so there is always something going on or someone coming and going.  Fortunately, when things do get stressed, someone busts out a joke and breaks the tension.

I'm looking forward to getting home and some R&R.  This season seems to have taken it out of me more than last, but last year everything was new and novel.  Anyhow, no pictures this post.  Hopefully I'll have some for the next post.  Cheers.


Antarctic Vehicles 4: Tucker Sno-Cat
2007-12-01T00:25:00.000-06:00
Yesterday I added another vehicle to my repertoire, the Tucker Sno-Cat:


We had to get out to the drill site for a video conference and the only vehicle available was this trusty Tucker.  It's slow like a Pisten Bully--our hand held GPS unit had us clocked in at 11 knots, which is just over 12 mph--but the ride was fairly smooth since it actually has a suspension.  The Tuckers must be quite durable as this one was manufactured in 1984--only 2 years after I was born.

On the way out to the site, we ran into some pretty dicey weather.  The snow was blowing so bad you couldn't see the next flag on the route until you were almost on top of it, so we had to drive by GPS.  It's frighteningly easy to get lost in conditions like that if you lose the flags.

When we were leaving the drill site, there was a group of penguins that came to check out the Tucker from a distance:


This may be the end of the Antarctic Vehicle series.  I think I've driven most of the ground vehicles available to the scientists, and I didn't have much luck convincing the helo pilot to let me fly it.  This is Tucker Trucker 072 signing off.


Wright Valley
2007-11-28T19:34:00.000-06:00
Things have been busy since I last wrote.  I made the transition back to the day shift.  It wasn't too terrible.  I did 18 hours up, 4 hours of sleep, then 24 hours up before the transition, so I just passed out for 12 hours from exhaustion and was on day shift schedule.

The first couple of days on day shift were a bit rough.  I was fairly tired and there are a lot more people and a lot more going on during the day.  I felt like everyone was coming at me and getting in my way in the Galley.  Things are better now, though, as I integrate more fully with the hustle and bustle of days.

Yesterday, I went on a fam trip to Wright Valley in the Dry Valleys.  The vistas were enough to  take my breath away.  My pictures don't do them justice, but I've included a few here:

This is a shot of the Air Devron Six Icefalls from our banking helicopter.  The icefalls were spectacular.  It looked like a waterfall frozen in time.  The ice that is spilling over is from the Polar Plateau, the ice sheet that covers most of the continent.

Here's another shot of the icefalls from the Dais.  In the foreground, you can see a feature called the Labyrinth.

This is another shot of the Labyrinth.  If you didn't see the snow in the background, you might think you were in the American southwest.

This is down in Wright Valley proper.  You can see some alpine glaciers spilling down the valley walls.

There's got to be worse places to sit down and take a rest.

This is an aerial shot from the helo on the flight back to McMurdo.


Random Pictures
2007-11-23T10:13:00.000-06:00




Happy Thanksgiving
2007-11-23T04:51:00.000-06:00
Thanks to all of you that sent me Thanksgiving wishes.  I hope everyone had a good meal, especially if it was turkey.  We don't get turkey dinner until tomorrow, but if last year is any indicator it will be good.


Mission Accomplished!
2007-11-21T04:14:00.000-06:00
We officially terminated HQ drilling last night at the depth of 1011.08 meters below sea floor (mbsf), just below our target depth of 1000 mbsf.  Hats off to the drillers!

A picture from the drill site to commemorate crossing the 1000 mbsf mark and one from the Crary night shift:

Even though we've hit our target depth, there is still a lot of work to be done.  A group of scientists went out to the drill site today to do measurements on the bore hole.  Over the next couple of days, they will be sending various instruments down the hole to collect data about the physical properties of the rocks.

Back here at Crary, we still have ~200m of core to log and sample.  About the time we get caught up, the drill site is going to be drilling another 50m or so to perform an experiment at the bottom of the hole, so we'll have to log that.  That should bring us to the beginning of December or so, where the focus will shift to writing up the results and on-ice report.  When that's done, we'll have to pack up all of the gear and get it ready to ship home.  Busy, busy, busy.


Snarky Jar
2007-11-17T07:51:00.000-06:00
We reached the point in the season where people, myself included, are getting tired and short tempered with people.  To combat this, Laura and I decided to create a "snarky jar" in our office:
If someone comes into the office and complains or makes a snarky comment, then they have to pony up some money to put in the jar.  The rules are the same for Laura, Rich, Leslie, and I.  The general going rate is $0.25/snarky comment or $1 for a particularly good rant.  I've already contributed almost $5.  At the end of the season, we're going to take the contents of the jar and treat ourselves at the bar.  And to protect the snarky jar from any thieves, we've got a guard:




Vehicles 3: Skidoo
2007-11-13T10:24:00.000-06:00
This is my third post in an unofficial series about the vehicles I've driven while down here.  I had to test some video conferencing hardware from the drill site, so Rich, Christina, and I hopped on skidoos and headed out there:

The skidoos are a bit inconvenient if you have a bunch of stuff to carry, but they make for a nice and quick ride.  If the weather is nice and the trail is in good condition, you can get moving at a pretty good clip.  We had full face helmets, which I'm obviously not wearing in the picture, so the wind and cold is not a problem.  The helmets also make you feel a bit safer, but I'm pretty sure if you fly off your skidoo at 60mph and hit the ice, the ice will win helmet or no helmet.

The drill site was pretty much same old, same old.  The VTC hardware worked on the first try.  A core came up while we were out there, so we got to watch the process pulling the core barrel up and extracting the core.  We were lucky to be there at the right time.  I also got a nice shot of Mt. Discovery illuminated by the sun:




One Month Left
2007-11-12T07:15:00.000-06:00
Sorry for the long time since the last post, it's been pretty busy around here.  We've had good weather these last couple of days, so the core has really been flowing in.  We're down to about 670 meters below sea floor.  With our current drilling pace, we're poised to reach our target depth in the next two weeks.  The season seems to have flown by.  There's less than a month left for most of the science team.  I'm leaving on the 14th, so I've got just over a month left.

I haven't had much time to do anything other than work, but I did sneak out the other morning and take a few pictures from hut point.  I particularly like this one:

It's of Vince's Cross, a memorial to George Vince who died in an accident nearby in 1902.

Tomorrow night I'm heading out to the drill site to test some VTC equipment.  It'll be on skidoo, so that should round out my Antarctic vehicle blog series.  Stay tuned.


Scott's Discovery Hut
2007-11-08T07:19:00.000-06:00
Well after the excitement of Condition 1 last night, today was bright and clear.  The same system that produced the storm last night is supposed to move back in tomorrow.  We took advantage of the break in the weather to go tour Scott's Discovery hut.  I've walked by in a hundred times but this is the first time I actually got to go inside it.  The hut was used primarily for storage, and as you can see in the photos below, they liked their biscuits:



While I was waiting to get into the hut, a plane was taking off from the ice runway headed to the Pole:

Ann Curry and her crew were on the plane.  If you look real close, you can see her waving to me.

Finally, when I was walking back to my room last night, I saw something that made me giggle:




Condition 1
2007-11-07T04:11:00.000-06:00
The helos were able to get out to the drill site and bring some core back this morning.  It's a good thing, too, because the weather turned nasty again:
This is the same view out my office window as the picture in the "A Cruel Mistress" post below.   We're officially in Condition 1, so we can't go to lunch until things clear up.


SSDD
2007-11-06T17:38:00.000-06:00
Not too much going on here.  The Today Show had a live broadcast yesterday and this morning.  We've been working away, though we've been hampered by the weather.  There have been no core flights the last two nights so now the sedimentologists are caught up.  If there is no flight today, we may not have much to do tonight.  They've been making great progress out at the drill site so they're getting pretty backed up.  At least the temperatures have been much warmer than last week when I froze my ears off.


A Cruel Mistress
2007-11-02T08:08:00.000-05:00
Things have been cold the last day or so, but I didn't realize how cold it was until I had to make an emergency dash down to the skidoo berm (don't ask, it's a story best told over a beer).  The berm is a fairly decent jaunt out onto the sea ice:
In my haste to get down there, I threw on my wool hat and big red and headed off at a run.  I remembered it being cold out when I came to work but I was totally unprepared for how cold and windy it was out on the ice.  I was only down there for 15 minutes or so but in that short amount of time I frost nipped both of my ears and one of my fingertips.  It's no surprise, either, considering it was -50F in the wind and I was running into it on my way back.  I've got feeling back in my fingertip and the swelling in my ears is subsiding.  They are quite red, so I look pretty comical, but I don't think they will blister.  Antarctica is a cruel mistress.

UPDATE: My ears are still pretty red, but doing better.  They look and feel like they've been severely sunburned.  This whole incident has given me a healthy respect for how quickly frost nip and frost bite can happen, even if you're wearing your hat, parka, and gloves.


Drill Site Visit, Part 2
2007-10-29T11:29:00.000-05:00
Last night we went out to the drill site for a tour of the drill rig.  There were only 4 of us on the trip so we got to skip the Pisten Bully and got to drive in style:

This badboy is a Ford pickup that's been outfitted with tracks.  It's a faster and much smoother ride than the Pisten Bully.  The only downside is that you can't turn the tracks when it is not moving, so backing up can be a tricky, especially in a tight space.

The drive out to the site was pretty uneventful.  There was a bit of blowing snow and drifting, which kept things interesting.  The transmission was also running a bit hot so I had to call into Mac Ops  Out at the camp, everything is pretty exposed so the wind was quite nasty.


Alissa gave us a nice tour of the drill rig and camp.  Even though I made a bunch of trips out to the drill site last year, this was the first time I got to go inside the rig.  It was pretty interesting, though a little anticlimactic since all you can see is a dirty hole in the ice with a pipe sticking out of it.




It's not all computers
2007-10-25T18:49:00.000-05:00
I spend the vast majority of my day in front of a computer doing various things, but I also like to pitch in and lend a hand, especially for the curators who work their tails off.  Here's a shot of me and Phill bringing down some core for the sedimentologists to describe:


As you can see I'm keeping up shaving.  My boyish good looks drive the ladies down here crazy.  Ok, so I made that last bit up. 

In Frosty Boy news, I almost broke down and had some the other night.  It was Chris's 50th birthday party so Leslie baked a decadent chocolate cake.  It was just begging to be topped with a dollop of Frosty Boy.  However, I held strong and settled for a glass of milk.


Condition 2 Blues
2007-10-23T07:50:00.000-05:00
We had a storm blow in Sunday night that made things nasty to be outside on Monday.  McMurdo was in condition 2 and everywhere else was in condition 1  for most of the day.  The drill site reported it was like living in a marshmallow--they couldn't even see the rig from camp.  The storm kept the grounded the helos, so the core was backed up at the drill site until this morning when they were finally able to fly.  They brought back over 60 meters of core.  From this you can probably guess that the drilling is progressing well.  The core has also been pretty interesting to my untrained eye.  There's been a fair amount of variability in the lithologies and it seems like every night yields some thing new and exciting.  I need to head back to it.  Cheers.


Still fighting the good fight
2007-10-20T19:47:00.000-05:00
I'm still fighting the good fight against Frosty Boy.  I wavered, just for a moment, when Kurt brought a brownie topped with Frosty Boy and slathered in hot fudge.  But I held strong and skipped dessert altogether.

The weather has been pretty variable.  We'll have a couple of nice days (>0F temps with not much wind), strung together and then followed by some downright nasty days.  It's never so much the temperature as it is the wind.  Today we had 30mph sustained winds which plunged the temperature to well below zero.

I think I'm fully adjusted to the night shift.  I've been less and less tired in the mornings.  I've even made it to the gym almost every day this week, which is no small feat because I've been working some long, 16 hour shifts.  Some of this is due to an increasing amount of core coming in and some due to some work back home.  Hopefully in a week or so things will settle down into a steady routine. 

I've been working on some new Corelyzer plugins which I hope to be able to roll out soon for people to check out.  Corelyzer is a visualization system that we use to look at our high-resolution core imagery.  Last year we were able to pull up the core images and some line graphs.  This year we've improved things by bringing in the lithology strip from PSICAT.  I'm working on visualizing other core data and imagery we are capturing.

Anyhow, I need to head to bed.  I've been up since 8PM last night (it's 2PM now) and I need to get up again in another couple of hours.  Cheers.


150,000 in ODM
2007-10-18T18:30:00.000-05:00
Well, in some non-Antarctica-related news, I got an email from my good friend Bill Ryan today.  He sent me a note to let me know that the ODM software him and I wrote when I did an internship at the Mayo Clinic just surpassed 150,000 CDs.  The purpose of the software was to incorporate outside digital media, such as digital x-rays, MRIs, etc., from other hospitals into the Mayo system.  So if a patient had some exams done elsewhere and the results were available in digital form, he wouldn't need to have the same exam re-done when he was referred to Mayo.

The actual ODM software isn't all that sophisticated--suck in the contents of a CD, make a digital archive copy, fire off the images to be processed by the imaging services.  All of the magic is done behind the scenes by the imaging services Bill wrote.  Nevertheless, it was a great experience to be able to work in a diverse team and I learned a lot.  And we must have done something right--they are rolling out the ODM software to Mayo's other two locations in Jacksonville and Scottsdale.

On the Antarctic front, things are starting to ramp up.  The core recovery has been steadily increasing.  Last night we had ~15m with 97% recovery.  The sedimentologists have been able to stay on top of the incoming core, and haven't run into too many problems, so I've been able to focus on getting a bunch of other stuff, like sample requests from the drill site, sorted out.  I'm looking forward to it slowing down some.  I'm averaging about 16-18 hours of work.  Sprinkle some gym time most days on top of that and you can quickly run yourself down.  And that's the last thing you want to do with all the bugs going around.  They've taken to quarantining people to their rooms to stop the spread of the flu.  So far I've been lucky. 

Anyhow, I had better grab some shut eye.


Pastels over Black Island
2007-10-16T14:32:00.000-05:00
Here's a shot of the sunrise over Black Island:



Switched Shifts
2007-10-15T19:44:00.000-05:00
Well, I survived the shift switching process.  Yesterday was difficult.  I ended up doing a 30 hour stretch awake.  By the end I was exhausted.  Last night was easier because I had started adjusting and there were more people around.  It turned into sort of a long day, though.  We had some last minute problems that kept me in the lab for an extra 4 hours.  So now I'm ready to collapse.  Tonight I'm going to try and catch a few pictures of the moon and the sunrise.  This morning there was a particularly beautiful sunrise but I couldn't be bothered to go out and take a few pictures of it.


Switching Shifts
2007-10-13T17:46:00.000-05:00
Things are progressing well down here.  Nearly all of the computers are setup, there's just a few loose ends to take care of.  The drilling is also progressing well.  The riser has been embedded and core should start arriving tomorrow night.  The arrival of the core means it's time for me to switch shifts.  I prescribe to the "stay up for 30 hours" method of switching shifts, so I'm going to try to make it to at least 10AM tomorrow before I go to bed.

In other news, I'm still Frosty Boy free.  It's actually easier than I thought it would be.  Cara doesn't appear to be wavering, so I suspect we'll end up splitting a 6 pack at the end of the season.


Drill Site
2007-10-11T22:33:00.000-05:00
A couple of days ago I drove out to the drill site to set up some computer systems.   The drill site is about 25 miles outside of McMurdo on the sea ice.  Because I was bringing a bunch cargo and a couple of people out, I took a Pisten Bully:

The Pisten Bully has a top speed of 13 mph with a tail wind and rolling down a very steep hill, so the 25 miles took over 2 hours.  It was pretty nice to get out of town and see some of the scenery.  In the picture above, Mt. Erebus, an active volcano, is over my shoulder.  It is too bad that there was some clouds obscuring the mountain.  The picture below shows the "road" out to the drill site:
That black speck on the horizon is the next flag.  It's a pretty desolate route.


Condition 2
2007-10-11T02:11:00.000-05:00
Today we woke up to condition 2 severe weather in Mactown.  Condition 2 is declared when one or more of the following conditions are met:
Sustained wind speed 48 knots to 55 knots
Wind chill temperature -75°F (-60°C) to -100°F (-73°C)
Visibility 1/4 mile to 100 feet
In our case, there was a lot of blowing snow causing very limited visibility.  Condition 2 is unpleasant to be out in, but it doesn't really affect us in the base.  We can still go about our day to day business.  I find the definition of Condition 2 rather telling--basically it says that wind up to 55MPH and wind chills up to (or rather down to) -75F are considered "normal".  I've thrown in a few pictures to give you an idea of what a storm looks like when it is blowing in:

There are mountains out there...somewhere.  And this is what it looks like with some mildly unpleasant weather in town:

It is still Condition 3 (normal) in the picture, but it wasn't too fun to be walking around in.


Just Say No to Frosty Boy
2007-10-07T01:32:00.000-05:00
Those of you who followed the blog last year may recognize this picture of me with a bowl of Frosty Boy.  This season I've decided to abstain from Frosty Boy.  It's oh so good, but not all that great for you.  For incentive, Cara and I have a bet going to see who can hold out the longest without having Frosty Boy.  The person who gives in first has to buy the other a six pack of beer.  Poor Cara, I don't think she realized the extent of my resolve when she bet me.  I'll blog about it when the inevitable happens and she gives in.

Besides avoiding Frosty Boy, I've also decided to groom myself (aka shave and get haircuts) this season.  This is mainly because I looked like crap most of the season last year.  Surprisingly a lot of people want me to grow it out again, but I have to pass because it won't even be filled it out by the time I leave.


Sunset
2007-10-06T19:31:00.000-05:00
One of the perks with being down here so early in the season this year is that I actually got to see a few Antarctic sunsets:


The picture doesn't really do the sunset justice.  The pastels are gorgeous, especially when you get them reflecting off of the clouds.  The sun is still setting but it's getting later and later.  Soon enough it will be light 24 hours a day.


I'm not dead (yet)
2007-10-05T00:46:00.000-05:00
Despite what my lack of posting might suggest, I'm not dead.  I actually arrived in Antarctica on Tuesday afternoon.  It's kind of spooky being back here because it feels like I left just a week ago.  We had absolutely gorgeous weather--only -10F--for our flight down and for our first couple of days on base.  I was able to walk around in just my jeans and Big Red parka for the first two days.  On Friday, the wind picked up and things got a lot colder around here.  I'm back to wearing my long underwear underneath my jeans and heavy fleece.

Things have been busy since we arrived which is the reason why I haven't been blogging--just too much to do.  As you can imagine, a lot of work goes into to setting up and running a fairly large (~60 person) science operation in Antarctica.  I'm happy that I only have to deal with the computer stuff, which is just a tiny part of the overall workload.  Most of our cargo arrived today, so I suspect I'll spend tomorrow setting up computers. 

Next week, I have to do a vehicle training so I can drive the various vehicles (trucks, mattracks, pisten bullies, and snowmobiles) down here.  I am also heading out to the drill site to set up a Corelyzer system out there so the drill site scientists can look at the core images we generate in McMurdo.  The other thing I'm hoping to do is test our video conferencing hardware.  We brought down a Polycom system for doing video conferences with groups back home.  The camera works, but the big concern is whether it will consume too much bandwidth.  I guess only time will tell.

I've got a few pictures that I wanted to include in this post, but I forgot my camera cable in my room.  So I'll include them in the next post, which will hopefully be tomorrow.  Cheers.


Botanical Gardens
2007-10-02T02:10:00.000-05:00
We went and got our gear on Tuesday.  Since I knew what to expect, the gear issue was pretty straightforward this year.  They simplified the number of things you have to wear on the plane.  So I can actually wear my own underwear-type of gear and then just throw on the wind pants and coat.  I also opted for some Carhart overalls rather than the standard issue wind pants.  I didn't have any problems with the wind pants last year, but the Carharts seem a bit more comfortable.  We'll see, though.  I shouldn't have the occasion to use them unless I make a bunch of trips out to the drill site on skidoos (which I'm hoping to do).

After the gear issue, we had a spot of tea and then headed back into town.  We spent most of the afternoon wandering around the botanical gardens and the Canterbury museum.  I snapped a few shots of flowers.  Actually, I snapped many shots of flowers, only a few turned out.  The first one is just a picture I liked:
And here's another...see if you can spot the hidden bird:



Christchurch
2007-09-30T21:25:00.000-05:00
I made it to Christchurch, NZ after a surprisingly uneventful trip.  A little Benadryl helped me sleep 8 of the 12.5 hours from LA to Auckland.  All of my bags arrived, which was a big relief. 

Christchurch is just as I remember it and I instantly fall back into old, familiar habits: making dinner plans with people to meet at the Dux for dinner, strolling along the Avon river on my way over to the fish and chips place, wandering around Cathedral Square watching the tourists snap photos.  The only change I've noticed is that one of my old haunts, the Asian food court where you could get a ton of food for a very reasonable price, appears to be under construction.  There's no indication of where it moved, so it looks like I'm S.O.L.

Tomorrow morning we head out to the Clothing Distribution Center (CDC) and try on our gear.  We're scheduled to fly to the ice on Wednesday.  There is a flight going out tomorrow morning.  I really hope they make it because if they boomerang, then our flight gets pushed back.  The more backed up we get, the less time we have to set things up when we get down there.  I'm starting to realize that time is of the essence.


Down to the wire...
2007-09-27T21:52:00.000-05:00
I'm back in Minneapolis after successful trips to Phoenix and Lincoln this week.  Phoenix was a personal trip.  The Bogotron invited me down for some R&R and let Mike and Margie cook for and entertain me.  Lincoln was a work trip.  We got a lot of stuff sorted out, and also ended up realizing there's a lot left to do.  That's OK, though.  As the Kiwis are fond of saying: "No worries."  On the vaguely work-related front, PSICAT has been getting a lot of attention recently so that's rewarding.

Tomorrow Elizabeth and I have a fair amount of packing and errands to run before we head our separate ways on Saturday.  Tomorrow is also my last chance to get some decent food.  I'm going to miss my Chipotle burritos, so I'll probably have to stop and have one for lunch.  I also think I'm going to cook some halibut steaks with a pineapple salsa for a nice, romantic dinner.

On Saturday I fly to Los Angeles by way of O'Hare, so that will be about as fun as a kick to the head.  Last year I almost missed my flight to New Zealand because nothing ever gets out of O'Hare on time.  I hoping for better luck this year.  I'm also hoping I win the seat lottery on the flight to New Zealand.  If I get stuck middle-middle, you can bet I'm going to be taking advantage of the complimentary beer and liquor.

In Christchurch, I'm staying at the Windsor Bed and Breakfast.  I stayed there last year and it was great.  The owners are really nice and it's centrally located for easy walking to places.  My two planned stops are this hole in the wall fish and chips place I found on my first visit to NZ and the Dux De Lux for a few beers.

I know a lot of people are following the blog again this year.  If there's something you're curious about or want to see, drop me a line via email or the comments and I'll do my best to put it in a blog post.  I also have a few invites for Dopplr, so if you do a lot of traveling yourself or have an intense burning desire to know where I am, let me know.  Cheers.


On the road again
2007-09-20T10:02:00.000-05:00
Today marks the beginning of my 3 month traveling tour.  The North American portion is making stops in Phoenix, Lincoln, Minneapolis, and the always lovely LAX.  The abroad portion includes stops in New Zealand and Antactica, my home during October, November, and half of December.  If you need to get a hold of me before the 29th, email and cell phone are your best bet.  After the 29th, email or snail mail.  This blog should receive regular attention, so if you're just interested in what I'm up to, check back.  Cheers.


Dr. Dobbs in Print!
2007-09-15T11:58:00.000-05:00

I got a welcome surprise today--they ran my article in the print version of Dr. Dobbs as well!  My two copies arrived this morning.  I had to snap a few photos:





Antarctica Bound (almost)
2007-09-14T09:37:00.000-05:00
Where has the summer gone?  I walked outside this morning and shivered because the temperature was in the 40s.  Which is rather silly considering where I'm going in two weeks, the weather will be far colder. 

It's been a long time since I last posted here.  I've been doing a lot of traveling recently for work, so I haven't been home all that much.  In the last month and a half, I've been in Lincoln, Tallahassee, and Germany.  And then next two weeks are shaping up to be no different.  I'm heading out to Phoenix to see my uncle on the 20th.  I return to Minneapolis on the 24th and then turn around and head to Lincoln for the 25th through the 27th.  I'm home on the 28th and then leave for Antarctica on the 29th.  Fortunately Elizabeth has been and continues to be a good sport.

Work has been progressing at a fast and furious pace.  There's a lot of stuff that I'd still like to get done before I ship out.  Every one, particularly in the Science Management Office, has been working their tails off to make this season even more successful than last year.  I'm just thinking we shouldn't have set the bar so high with last year's success!  :)

In other news, Dr. Dobb's Journal published an article I wrote about PSICAT.  It's a good introduction to the internals and technology behind PSICAT.


It Lives!!
2007-08-04T17:37:00.001-05:00

Today I bought and installed a draft beer tower on to the refrigerator that I converted to a kegerator.  Previously I was just dispensing the beer with a picnic tapper hanging on the inside of the door.  Now there's no need to open the fridge up unless you need to change out the keg inside.


Josh: 0, Bees: 1
2007-07-23T11:44:00.000-05:00
I had a little run in with some bees this weekend.  Elizabeth and I were walking through the woods on the northeast side of Cedar Lake yesterday morning.  I was messing around down in the water and Elizabeth had walked up some wooden stairs built into the hill.  All of a sudden she came running down to the water's edge yelling about bees.  Before I knew it we both were getting stung.  Needless to say we vacated the area at a run.  The total damage: 3 stings to Elizabeth and 2 to me.  All of Elizabeth's cleared up to the point that they were only a little swollen and a itchy, as did the one on my leg.  The one on my arm, however is a different matter:

It's been a solid 24 hours since I was stung so hopefully the swelling starts to go down soon.