Showing posts with label technical. Show all posts
Showing posts with label technical. Show all posts

Wednesday, August 19, 2009

Building your Griffon app with Gradle

In this post, I'm going to show how to build your Griffon apps with Gradle. Griffon already comes with a command line tool to build and interact with your app, so why would you want to do this? I can think of a few reasons:
  • you have an existing Gradle build you want to integrate with
  • you have a complex build or packaging scenario
  • you want to take advantage of Gradle's dependency management for your app
Prerequisites
  1. Griffon
  2. Gradle
The build.gradle file
Below is a generic Gradle build file for Griffon applications. Either copy and paste the code below or download it and place it in a build.gradle in your Griffon application folder.
// standard configuration
project.captureStandardOutput(LogLevel.INFO)
configurations { compile }

// add additional repositories here if you need them
repositories {
//mavenRepo(urls:'http://download.java.net/maven/2/')
mavenCentral()
}

// add any dependencies you want downloaded as part of the bootstrap process
dependencies {
//compile 'com.google.collections:google-collections:1.0-rc2'
}

// this should be run once for the project
createTask('bootstrap') << {
// download any dependencies and put them in lib/
configurations['compile']?.files?.findAll { it.absolutePath.indexOf('unspecified') < 0 }.each { dep ->
ant.copy(toDir: new File('lib'), file: dep)
}

// try to install any plugins
new File('application.properties').eachLine { line ->
if (line.startsWith("plugins")) {
def plugin = line[8..-1].split('=')
griffon "install-plugin ${plugin[0]} ${plugin[1]}"
}
}
}

// mimic the tasks provided by the Java plugin
createTask('clean') << { griffon 'clean' }
createTask('compile') << { griffon 'compile' }
createTask('libs') << { griffon 'package' }
createTask('test') << { griffon 'test-app' }
createTask('dists') << { griffon 'prod package' }

// additional run-related tasks
createTask('run-app') << { griffon 'run-app' }
createTask('debug-app') << { griffon 'run-app -debug' }
createTask('run-webstart') << { griffon 'run-webstart' }
createTask('run-applet') << { griffon 'run-applet' }

// call out to the griffon command
def griffon(target) {
if (System.getProperty("os.name").toLowerCase().startsWith("win")) {
ant.exec(executable: 'griffon.bat', dir: projectDir, failonerror: true) {
arg(value: target)
}
} else {
ant.exec(executable: 'griffon', dir: projectDir, failonerror: true) {
arg(value: target)
}
}
}

// if you want to use the gradle wrapper
createTask('wrapper', type: Wrapper).configure {
gradleVersion = '0.7'
}
To verify that everything is working, let's try running our app from Gradle (substitute gradle.bat if you are on Windows):
gradle run-app

Your Griffon app should run. If it doesn't, try invoking the Gradle command with the '-i' flag:
gradle -i run-app
to see what the problem is.

Multi-project Builds
So far it may seem like "What's the point?" We could already run our Griffon app with the griffon command. Where Gradle really starts to shine is when you start wanting to perform complex builds, such as building multiple Griffon apps at once or integrating with an existing build.

Gradle has good support for multi-project builds. Assuming you are using a directory layout like this:
settings.gradle
griffonapp1/
build.gradle
...
griffonapp2/
build.gradle
...
Building these multiple Griffon apps is as simple as listing them in the settings.gradle file:
include 'griffonapp1', 'griffonapp2'
Similarly, integrating with an existing build should be as simple as adding the name of your Griffon app to your existing settings.gradle file.

Complex Build/Packaging Scenarios
Griffon offers several packaging options with the Installer plugin. Sometimes you need more flexibility. For my PSICAT build, I wanted to package two Griffon apps together in a specific directory structure. The nice thing about Gradle build files is that they are just Groovy code and offer great integration with Ant. Below is a snippet from the PSICAT build that illustrates using Ant within Gradle:
// griffon create-all-launchers for PSICAT and SchemeEditor
griffon(psicat, 'create-all-launchers')
griffon(schemeEditor, 'create-all-launchers')

// build linux package
ant.copy(toDir: new File(linux, 'PSICAT')) {
fileset(dir: new File(psicat, 'installer/linux/dist')) {
include(name: 'bin/**')
include(name: 'lib/**')
}
}
ant.copy(toDir: new File(linux, 'PSICAT/tools/SchemeEditor')) {
fileset(dir: new File(schemeEditor, 'installer/linux/dist')) {
include(name: 'bin/**')
include(name: 'lib/**')
}
}
ant.chmod(perm: "+x", dir: new File(linux, 'PSICAT/bin'), includes: '*')
ant.chmod(perm: "+x", dir: new File(linux, 'PSICAT/tools/SchemeEditor/bin'), includes: '*')
ant.tar(destFile: new File(linux, "PSICAT-linux-${version}.tar.gz"), compression: 'gzip', basedir: linux, includes: 'PSICAT/**')
...
(if you're curious, the whole build.gradle file is here)

Cool Gradle Tricks
One cool feature of Gradle is that you can generate a 'wrapper' script that will download and bootstrap Gradle itself so that developers who download your source don't have to have Gradle installed to build your project.
The build.gradle file above is already setup to create this wrapper. You simply have to run the command:
gradle wrapper
and it will create four files in your project:
gradle-wrapper.jar
gradle-wrapper.properties
gradlew
gradlew.bat
If you include these files when you send someone your project, they can use the gradlew or gradlew.bat scripts as if they had Gradle installed:
./gradlew run-app

The other neat trick with Gradle is you can use its ability to pull from Maven repositories to bootstrap the dependencies of your Griffon app. This means you don't need to send someone your Griffon app with a bunch of JARs in the lib directory. Instead you can send them a Gradle-ized version of your app and tell them to run the command:
./gradlew bootstrap

This will download all of the dependencies you've listed in the build.gradle file and put them in the lib directory of your Griffon app. Specifying dependencies is as simple as adding a few lines to the dependencies section of the build file:
dependencies {
compile 'com.google.collections:google-collections:1.0-rc2'
compile 'org.slf4j:slf4j-api:1.5.8'
}
Conclusion
Hopefully this has given you a taste of using Gradle with Griffon. Both are awesome tools that really show off the power of the Groovy language. If there's enough interest in using Gradle with Griffon, we can make build.gradle a standard artifact created by the griffon create-app task.

Saturday, August 15, 2009

Are Groovy Stacktraces a Showstopper?

I've been part of an interesting debate on the merits of Groovy via Twitter this week. I'm reproducing it here because I think it's an interesting take on what potential users feel are make or break issues for a language, and to give others who may not have seen it the chance to weigh in without the 140 character limit. :)

(Sorry in advance. Reproducing a Twitter convo in a blog is kind of hideous)

[@philswenson] if you want to know why groovy sucks, click here: http://pastie.org/583115
[@joshareed] @philswenson It took me a second to diagnose your problem. Did you want Groovy to say: "Hey joker, actually import the classes you're using"
[@joshareed] @philswenson Apparently Ruby has some magical mind-reading interpreter/compiler that saves you from yourself?

[@philswenson] @joshareed why the extra 500 lines of crap? in java or jruby that stacktrace would have been 1/50 the size. that's my point
[@joshareed] @philswenson re: Java guess that's the trade off for Groovy dynamics. Ruby's (http://bit.ly/IagLJ) is not any better [Note: Nambu mangled the original link]

[@philswenson] @joshareed Ruby's stack dump = 67 lines. Groovy's = 222 lines. Not as different as I said, but still...
[@joshareed] @philswenson # of lines doesn't matter when you see the cause of the prob in the first 10 lines, does it? You don't have to keep reading...
[@joshareed] @philswenson and was the 67 vs 222 equivalent programs in each? I'd be interested in comparing the source of each

[@philswenson] @joshareed this was a case where the cause was obvious (it usually isn't). isn't groovy in large part about succinctness over java?
[@joshareed] @philswenson how much time do you spend looking at stacktraces that this is hang up for you? Write a 3 line .findAll{} to filter traces
[@joshareed] @philswenson and if you want to stick with stacktrace length as your argument then you won't be plain old Java traces with JRuby

From my perspective, this isn't a JRuby vs Groovy debate even though both languages were brought up in the conversation. Both are two different ways to skin a cat but they ultimately get you to the same place. Which you choose to use is more a matter of personal preference than the actual merits of the language.

What I'm more interested in is how important/serious is the stacktrace issue that Phil Swenson argues? I've been writing Groovy for a couple years now and can't remember (off the top of my head) a stacktrace from Groovy that was so confusing I couldn't figure out the problem. I'm sure there are some, especially when you start getting deep into the Groovy innards, but I'm guessing that's also the case with other languages on the JVM, like JRuby.

All dynamic languages built on the JVM introduce a certain level of indirection (by necessity), so are Groovy's stacktraces that much more hideous than JRuby's? What qualifies as hideous? I think length is a naive measure because the cause of the problem is likely to show up in the first dozen lines. Who cares if it goes on for another 200 lines? Length is also easily manipulated. You could write a simple findAll closure that sanitizes the stacktrace by removing all the lines that start with 'sun.reflect' or 'org.codehaus.groovy.runtime'. I have never once run into a problem that could be tracked to one of those lines in the stacktrace. So now does that make Groovy better than JRuby because the stacktrace is shorter?

I'm sure this is just a case of me being used to seeing Groovy stacktraces, so I'm used to feel of them and can quickly hone in on the cause of the problem. Likewise I'm sure Phil is used to [J]Ruby errors which probably look much different than Groovy's. I say probably because the two example put forth didn't look all that dissimilar IMO but were not from the same problem so it was an apples to oranges comparison.

Anyways, how important are stacktraces when making the decision to use a particular language? Are they important enough to declare one language's superiority over another? Are they worth focusing on to improve Groovy's ease-of-use?

Wednesday, August 12, 2009

Griffon Installer Plugin

Last week Andres Almiray and I released a new version of the Griffon Installer plugin. This version adds support for creating native Mac app bundles for your Griffon application, as well a few refactorings to make the generated artifacts more consistent. In this post, I'm going to walk you through the process of creating a set of native launchers for your application. A future post will explain the process of creating installers.

Getting started using the installer plugin is simple:
griffon install-plugin installer

This will fetch the plugin and install it into your Griffon app. The plugin adds several new targets. All of the launcher-related targets are broken into two steps: a prepare step that copies all of the configuration files for that particular launcher into your application, and a create step that actually generates the native launcher. The prepare targets only need to be run once for your project. They give you a chance to tweak the launcher configuration files. The create targets are run every time you want to make a release.

The plugin provides launchers for the following platforms:
  • Linux
  • Mac
  • Windows
  • Executable Jar
The plugin also provides two other convenient targets, prepare-all-launchers and create-all-launchers which invoke the prepare- and create- targets for the four platforms above.

We'll use these convenience targets to create launchers for all platforms:
griffon prepare-all-launchers

This will create several new files for you:
  • installer/jar - the executable JAR launcher directory
  • installer/jsmooth - the Windows launcher directory
  • installer/linux - the Linux launcher directory
  • installer/mac - the Mac launcher directory
  • installer/jar/MANIFEST.MF - customize the manifest of the executable JAR
  • installer/jsmooth/$appName-icon.png - customize the .exe icon
  • installer/jsmooth/$appName.jsmooth - customize the .exe options
  • installer/mac/$appName.icns - customize the Mac icon
Once you are done customizing the configuration files, invoke the
griffon create-all-launchers target to actually create the launchers. This will create all of the launchers in the installer/$platform/dist directories and create archives of the launchers:
  • installer/jar/dist/$appName-$version.jar - the executable jar
  • installer/linux/dist/bin/$appName - the Linux launcher shell script
  • installer/linux/dist/$appName-linux-$version.zip - a zip of the Linux launcher
  • installer/mac/dist/$appName-$version.dmg - a disk image of the Mac launcher [only created if on a Mac]
  • installer/mac/dist/$appName.app - the Mac application bundle
  • installer/jsmooth/dist/$appName-windows-$version.zip - a zip of the Windows launcher
  • installer/jsmooth/dist/$appName.exe - the Windows launcher
  • installer/windows/dist/$appName-windows-$version.zip - copied from the JSmooth directory
  • installer/windows/dist/$appName.exe - copied from the JSmooth directory
As you make changes to your code, simply invoke griffon create-all-launchers and your latest changes will be packaged up. If you increment the version number of your app, the created artifacts will be updated to match.

Hopefully the Installer plugin makes it easy to distribute your Griffon applications to users on various platforms. In a future blog post, I'll cover creating installers for when a simple zip file is not enough.

Tuesday, July 14, 2009

Griffon Plugin Development

I finally got to the point in my Griffon application development that I was ready to release my app. Griffon has a plugin for creating installers and platform-specific launchers, but it didn't generate a proper Mac .app bundle. Being a Mac user myself, I really wanted to be able to deploy my app natively on Macs, so I decided to scratch my itch and add support to the plugin. I learned a fair bit about Griffon plugin development, so keep reading if you're interested in tweaking existing plugins or creating new plugins.

First off, Griffon plugins are simply Griffon applications with a few extra bits not normally used in a standard application—scripts for providing command line targets e.g., griffon create-mac-launcher, and interception of Griffon events like eventCleanEnd. To get started with plugin development, you'll want to decide whether you want to create a new plugin or edit an existing one.

Setting up your Griffon plugin development environment is fairly straightforward. If you want to edit an existing plugin:

cd MyExistingApplication
griffon install-plugin installer


If you want to create a new plugin and install it in an existing application:

griffon create-plugin MyNewPlugin
cd MyExistingApplication
{edit application.properties to include a line 'plugin.my-new-plugin=0.1'}



The real trick for both of these scenarios is to link the plugin source to somewhere where you can edit it live. This makes testing a whole lot easier. I had to pull a few strings to get this tip, but I'm going to share it with you for free:
Go into your ~/.griffon/{griffon version}/projects/MyExistingApplication/plugins/ and either edit the plugin directly or symlink it to something more convenient:

cd ~/.griffon/{griffon version}/projects/MyExistingApplication/plugins/
rm -rf installer-0.3
ln -s ~/Workspace/griffon/griffon-installer installer-0.3

You just have to make sure the name and the version match what is in your MyExistingApplication/application.properties file. If you installed an existing plugin, you should be good to go. If you're creating a new plugin, it defaults to 0.1.

Once you have your plugin development environment setup, you're ready to go to town. Below are a few things I learned along the way. Most have to do with Gant scripting, which I wasn't very familiar with before this but is easy to pick up.

If you want to create a new command line option, e.g. griffon say-hello-world you need to create a new script in your plugin. You can do this by creating a SayHelloWorld.groovy file in the $plugin/scripts. Or as Andres points out in the comments a simple griffon create-script SayHelloWorld works as well. Even better! You also have to define a sayHelloWorld target in your script and make it the default target:

target(sayHelloWorld:"Says 'Hello World'") {
// any Groovy is valid here
println "Hello World!"
}
setDefaultTarget(sayHelloWorld)


Another useful feature is the ability to invoke targets in other strips, such as if you wanted to make sure the Griffon app was built before your task was executed. You can accomplish this by including the targets of other scripts in your script:

includeTargets << griffonScript("_GriffonPackage") // script name minus the .groovy

target(doAfterBuild: "Build and then run me") {
depends(checkVersion, packageApp, classpath) // defined by _GriffonPackage.groovy
packageApp() // calls the packageApp target in _GriffonPackage.groovy
println "The JARs in $basedir/staging should be fresh!"
}

setDefaultTarget(doAfterBuild)


Finally, before this post gets too long-winded, I'll cover the topic of events. Events are fired by the Griffon core runtime and by plugins in response to actions in the workflow. They provide an extension mechanism for plugins who wish to respond to various actions. Your plugin can intercept events by providing a $plugin/scripts/_Events.groovy file which declares interest in specific events:

eventCleanEnd = {
// jar
ant.delete(dir:"${basedir}/installer/jar/dist", failonerror:false)
ant.delete(dir:"${basedir}/installer/jar/classes", failonerror:false)

// jsmooth
ant.delete(dir:"${basedir}/installer/jsmooth/dist", failonerror:false)

// linux
ant.delete(dir:"${basedir}/installer/linux/dist", failonerror:false)

// mac
ant.delete(dir:"${basedir}/installer/mac/dist", failonerror:false)

// windows
ant.delete(dir:"${basedir}/installer/windows/dist", failonerror:false)
}


If you are developing your own plugin, you should consider publishing events so that other plugins can hook into your workflow. You can do this by including the _GriffonEvents targets in your script, or by including any script that itself includes _GriffonEvents:

includeTargets << griffonScript("_GriffonEvents")

target(myPluginTask: "") {
event("MyPluginTaskStart", [])
println "My Plugin Task!"
event("MyPluginTaskEnd", [])
}

setDefaultTarget(myPluginTask)


Thanks for following along. Any suggestions or questions are greatly appreciated!

Oh and if you're interested in the installer plugin stuff, check out: http://dev.psicat.org/ for some example outputs. These are not the exact outputs as I am bundling two Griffon apps--the main PSICAT one and a second one in tools/--but you get the idea.

Friday, October 31, 2008

RESTful Query URLs

The last couple of days I've been working on writing a RESTful JSON document database. While a number of these already exist (CouchDB, FeatherDB, DovetailDB, Persevere, JSONStore, etc.), I decided to write my own because I wanted a bit more control over the URL scheme used by the REST interface, and I needed the ability to tweak the search functionality to achieve decent performance on some common but complicated queries. All in all it was an interesting diversion. The actual server clocked in at about 1000 SLOC, with much of that boilerplate because I wrote it in Java/JDBC rather than Groovy/GroovySQL.

The most interesting problem came in designing the query scheme for the REST interface. There seems to be a couple different ways to implement it with no real consensus as to which is the "right" way. As with most things, I suspect it depends on how you've implemented other pieces of the architecture and even personal preference. Below I describe three approaches I considered. The nice thing with REST is there's nothing stopping you from implementing all of these approaches in your interface.

NB: I'm no REST expert so the information below is my observations rather than any best practices. I'd love for anyone who knows better to chime into the discussion.

POST query parameters/document
In this approach, you provide a search endpoint, say something unoriginal like '/search', and queries are POSTed to that URI. The query is either a set of form encoded key-value pairs or a search document using a schema shared between the client and server.

This approach seems closer to RPC than REST to me, but may be the best approach if your search functionality requires a more complex exchange of information than simple key-value pairs allow. The obvious downside to this approach is that there is no way to bookmark a query or email/IM a query to someone else. This approach also can't take advantage of the caching built into the HTTP spec.

GET query string
Similar to above, you expose a URI endpoint, possibly something like /search, and queries are sent to that endpoint with the parameters encoded in the query string of the URL, e.g. http://www.google.com/search?q=REST+query+string

This approach improves on the bookmarkability of searches, since all of the parameters are in the URL. However, the use of the query string may interfere with caching as described in Section 13.9 of the HTTP spec. Overall, I think there is nothing inherently un-RESTful about this approach, especially if you provide more resource-oriented URIs than /search, e.g. /documents?author=Reed. In my head, I interpret the latter as "give me all of the document resources but filter on the author Reed. Removing the query string will still give you a resource (or collection of resources in this case).

Where this approach falls down is when you start trying to represent hierarchical or taxonomic queries with the query string, e.g. http://lifeforms.org?k=kingdom&p=phylum&c=class&o=order&f=family&g=genus&s=species as described on the RestWiki.

Encoding query parameters into the URI structure
In this approach the query parameters are encoded directly into the URI structure, e.g. /documents/authors/Reed, rather than using the query string. Another example of is described at Stack Overflow.

This approach solves both the bookmarkability and the caching issues of the previous approaches, but can introduce some ambiguity, especially if your resources aren't strictly hierarchical in nature. The biggest stumbling block for me was this: looking at the URI /documents/authors/Reed, it's not immediately clear what will be returned. For example, if I sent you the URI /documents you might infer that you would get a list or the contents of some documents. From the URI /documents?author=Reed, you might infer that the resource(s) returned would be documents authored by Reed. So what might you expect to get from the URI /documents/authors/Reed? Information about the author Reed or all documents authored by Reed?

How important is this? I guess it's really up to you. A machine likely infers about as much from
/documents/authors/Reed as it does from /documents?author=Reed.

Wednesday, September 17, 2008

My First Griffon App

Sorry it's been so long since I posted here. Work keeps me busy.

Recently I had been given several GB of raw data from our two most recent scientific drilling expeditions in Antarctica. This data needs a fair amount of quality control processing to turn it into a usable datasets for the scientists. To do this, I needed to write a tool for the drillers to interactively plot and explore the data to determine regions of interest. Given the recent buzz about Griffon, I thought I'd give it a try.

I started by downloading and installing Griffon. Once I had everything setup, I created an app:

griffon create-app DrillingAnalytics


If you've done any Grails development, this will be a familiar idiom to you. The results of this command is a straightforward directory structure, focused around the MVC pattern. You'll recognize directories for models, views, and controllers.

My next step was to flesh out my model. When you create an app, Griffon automatically creates a model class called ${app.name}Model (DrillingAnalyticsModel for me) in the griffon-app/models directory. The main purpose of my app is to plot time series data so I defined two fields, startDate and endDate in my model:

import groovy.beans.Bindable

class DrillingAnalyticsModel {
@Bindable String startDate = "2006-11-07 00:00"
@Bindable String endDate = "2006-11-08 00:00"
}



You'll notice the @Bindable annotations on these fields. These fields will tie to these components in the UI, and the @Bindable annotation will automatically take care of keeping the UI in sync with the model via PropertyChangeEvents.

The model class is also where you can put other fields to maintain applications state:
 
def plot = new CombinedDomainXYPlot(new DateAxis())
def subplots = []
def chart = new JFreeChart(null, JFreeChart.DEFAULT_TITLE_FONT, plot, false)


With the model sorted out, I moved on to developing the view. As with model, Griffon creates a ${app.name}View class for you in the griffon-app/views directory. Griffon puts the full power of SwingBuilder, SwingXBuilder, and GraphicsBuilder (with more on the way) at your fingertips for developing the UI.

I spent the majority of my time on the UI. It was a seemingly endless cycle of tweaking the code and testing with griffon run-app to get it to look the way I wanted. This is no knock on Griffon; writing Java UIs, especially by hand, just plain sucks.

After far too long trying to get the standard Java layout managers to do what I want, I did myself a favor and downloaded MigLayout. Despite not being built into SwingBuilder, MigLayout integrates nicely with SwingBuilder:

application(title:'Drilling Analytics', pack:true, locationByPlatform:true) {
panel(layout: new MigLayout('fill')) {
// chart panel
widget(chartPanel, constraints:'span, grow')

// our runs and time
panel(layout: new MigLayout('fill'), border: titledBorder('Time'), constraints: 'grow 100 1') {
scrollPane(constraints:'span 3 2, growx, h 75px') {
runs = list(listData: model.mis.keySet().toArray())
}
label('Start:', constraints: 'right, gapbefore 50px')
textField(id:"startDate", text: bind { model.startDate }, action: plotAction, constraints:'wrap, right, growx')
label('End:', constraints: 'right, top')
textField(id:"endDate", text: bind { model.endDate }, action: plotAction, constraints:'wrap, right, top, growx')
label("+/-", constraints: 'right')
textField(id:"padding", text: "30", constraints: 'growx')
label("min")
button(action: plotAction, constraints:'span 2, bottom, right')
}

// our plots panel
panel(layout: new MigLayout(), border: titledBorder('Plots'), constraints: 'grow 100 1') {
model.data.each { id, map ->
checkBox(id: id, selected: false, action: plotAction, text: map.title, constraints:'wrap')
}
}
}
}


SwingBuilder gets rid of all the boilerplate code and MigLayout makes it possible to code decent Java UIs by hand:


We've covered the Model and the View, now it's time to focus on the Controller. The controller mediates between the model and view. It contains all of the logic for handling events from the UI and manipulating the model.

One common pattern in the existing Griffon examples is the use of Swing Action objects to trigger actions from the UI. My UI was pretty simple so I could reuse a single action on all of the components to refresh the plot:

actions {
action(id: 'plotAction',
name: 'Update',
closure: controller.plot)
}


I put this code in my DrillingAnalyticsView class, but it could just as easily be defined in its own file and imported into the view via the build() method. You'll notice that I give the action an id--plotAction--which I use to reference it from the components:

button(action: plotAction, constraints:'span 2, bottom, right')


You can also see that the action just delegates to the controller.plot closure. This is convenient because it keeps all of the logic in one place and the controller has access to both the model and view. The actual code of the controller.plot is unremarkable. The big consideration is to properly manage your threading. Don't do long running actions in the EDT as it will freeze the UI, and don't update the UI from outside the EDT as Swing is not thread safe. Andres Almiray has a good description of how Griffon makes this easy.

Since my app is fairly niche (I doubt there's many of you visualizing drilling data), I'm not going to post the whole source code here. However, I want to point out that the source code consists of just 327 lines of code, and that's including blank lines and comments! The bulk of that code is the logic to query the database and update the JFreeChart plots. This truly demonstrates how simple and easy it is to build an app with Griffon.

If you're looking for more Griffon examples, check out the samples included in the samples directory of the Griffon distribution, and keep an eye on Griffon posts groovy.dzone.com

Thursday, June 12, 2008

PSICAT News

Last week I was in Potsdam, Germany for a meeting with folks from ICDP, ESO, and CoreWall. We were discussing integrating our various tools (PSICAT, Corelyzer, and the Drilling Information System) to create a turnkey technology platform for future ICDP and ESO drilling expeditions. Each of the tools have been used successfully on multiple expeditions, but until now haven't interoperated with each other. The meeting went well and we adjourned with an integration plan.

So what does this mean for PSICAT? First off, if you're an existing user, you won't have to do anything different; PSICAT will continue to work exactly as it does today. It will just have some new, optional features for integrating with the DIS and Corelyzer. One positive side effect of this is that I will have some "official" time devoted to working on PSICAT. Things have been so busy in the last couple of months that PSICAT development has been on the back burner. PSICAT won't be my only project but it should get more attention than it is getting now. It also means that PSICAT will be getting used all over the world on new drilling projects, which (I hope) leads to a larger user community and a better overall product.

Exciting times! Keep an eye on the PSICAT site for more updates.

Wednesday, May 21, 2008

Visualizer, Part 3: Poor Man's PDE Build

This is the third in my series (Part 1, Part 2) of posts about Visualizer. In this post I'll be talking about how to create a simplified PDE build.

As I mentioned in the previous post, Visualizer is built on OSGi. My preferred development environment for doing any Java development, but especially OSGi development, is Eclipse because of its wonderful JDT and PDE tooling. The PDE team has created an awesome environment for developing and managing OSGi bundles. However, one of the requirements that I had for Visualizer was that anyone could download the source code and build it, regardless of their IDE or environment preferences. PDE includes the ability to perform a headless build, but I didn't really want to expect the user to download Eclipse or to include a stripped down version of Eclipse in the Visualizer distribution just so the user could build it from the commandline. So I set out to create a "Poor Man's PDE Build" using just Ant.

Actually building the plugins with Ant is relatively simple. This short Ant file will build the plugin:

<?xml version="1.0" encoding="UTF-8"?>
<project default="build-plugin">
<property name="src.dir" value="src"/>
<property name="classes.dir" value="bin"/>
<property file="META-INF/MANIFEST.MF"/>
<property file="build.properties"/>

<path id="classpath">
<fileset dir="${dist.dir}">
<include name="*.jar"/>
</fileset>
<fileset file="${osgi.framework}"/>
<pathelement path="${java.class.path}"/>
</path>

<target name="init">
<mkdir dir="${classes.dir}"/>
</target>

<target name="compile" depends="init">
<echo message="Compiling the ${Bundle-SymbolicName} plugin"/>
<javac srcdir="${src.dir}" destdir="${classes.dir}" classpathref="classpath" debug="true"/>
</target>

<target name="copy-resources">
<echo message="Copying resources"/>
<copy todir="${classes.dir}">
<fileset dir="." includes="${bin.includes}"/>
</copy>
</target>

<target name="build-plugin" depends="compile, copy-resources">
<jar jarfile="${dist.dir}/${Bundle-SymbolicName}_${Bundle-Version}.jar" basedir="${classes.dir}" manifest="META-INF/MANIFEST.MF"/>
</target>

<target name="clean">
<delete includeemptydirs="true">
<fileset dir="${classes.dir}" includes="**/*"/>
</delete>
</target>
</project>


As you can see here, there really isn't much to the actual build. The best part is we can use the META-INF/MANIFEST.MF and build.properties created when we're working in PDE to control the build.

For a single bundle with no dependencies, this effectively duplicates the PDE build process. The difficulties comes in when you start having dependencies. If you use the headless PDE build, it will sort out all the dependencies for you and build your bundles in the proper order.

Implementing proper dependency resolution seemed awfully complicated, especially since PDE build already implements it. Fortunately, Visualizer doesn't require complicated dependency resolution because I've structured the bundles in a logical order. There are three levels of bundles: "core" which implement the main functionality, "ui" which implement the user interface to the core bundles, and "application" bundles that build on both the core and ui bundles.

Armed with this knowledge, we can structure a three stage build process where we first build all of the core bundles then all of the ui bundles and then all of the application bundles. To accomplish this, we have a master build.xml that calls out to the template build-plugin.xml file listed above using a subant task.


<target name="build-framework" depends="init">
<!-- build org.andrill.visualizer, org.andrill.visualizer.services* -->
<subant target="build-plugin" genericantfile="build-plugin.xml" failonerror="false">
<property name="dist.dir" value="../${build.dir}"/>
<property name="osgi.framework" value="../framework.jar"/>
<dirset dir=".">
<include name="org.andrill.visualizer"/>
<include name="org.andrill.visualizer.services*"/>
</dirset>
</subant>
</target>


Here you can see we build first the org.andrill.visualizer bundle and then all of the org.andrill.visualizer.services bundles. As we build each bundle, we copy the bundled JAR file to our dist.dir. Each time a bundle is built, it creates its classpath from all of the JARs in dist.dar. So even though there are dependencies among bundles, we are progressively fulfilling those dependencies by collecting the built bundles in dist.dir.

Once all of the "core" bundles are built, we can kick off the build of the ui bundles:


<target name="build-ui" depends="build-framework">
<!-- build org.andrill.visualizer.ui* -->
<subant target="build-plugin" genericantfile="build-plugin.xml" failonerror="false">
<property name="dist.dir" value="../${build.dir}"/>
<property name="osgi.framework" value="../framework.jar"/>
<dirset dir=".">
<include name="org.andrill.visualizer.ui*"/>
</dirset>
</subant>
</target>


Finally we can build all of the "application" bundles by excluding everything we've already built:

<target name="build-apps" depends="build-framework, build-ui">
<subant target="build-plugin" genericantfile="build-plugin.xml" failonerror="false">
<property name="dist.dir" value="../${build.dir}"/>
<property name="osgi.framework" value="../framework.jar"/>
<dirset dir=".">
<include name="*.*"/>
<exclude name="org.andrill.visualizer"/>
<exclude name="org.andrill.visualizer.services*"/>
<exclude name="org.andrill.visualizer.ui*"/>
<exclude name="${build.dir}"/>
<exclude name="${dist.dir}"/>
</dirset>
</subant>
</target>


You can check out the full build file at: build.xml and build-plugin.xml.

It's not nearly as neat as just kicking off a PDE build and letting it do all of the hard work of figuring out the dependencies for you. However, I'm rather fond of my approach because it keeps me honest. If I create a new bundle and it starts breaking the build, then I know I need to go back and make sure I've thought through the dependencies and am not trying to mix "core" code with "ui" code and such. And there's no need to bundle Eclipse with the source to build the thing.

Monday, May 12, 2008

Open Participation in SimpleIssue

When I started writing the simple issue tracker it was with the goal of learning more about Grails and coming away with a generally useful project. I decided to blog it because I was hoping my insights might be useful to the community. The response has been awesome, with lots of encouragement and lots of great ideas.

It strikes me that I can probably learn more from the community than just working away on my own. So if you want to fix something I screwed up or you have ideas for new features or you just want to experiment, drop me a line and I'll add you to the SimpleIssue project out at Google Code. Mike Hugo has already pointed out my embarrassing lack of tests and has offered to lend a hand. This is great news for me because testing has never been my strong suit. It will give me an opportunity to study how it's done.

SimpleIssue is a nights and weekends project for me, so I don't expect any particular level of participation. If you've got ideas and want to contribute, then all the better for us.

Sunday, May 11, 2008

Writing a Simple Issue Tracker in Grails, Part 2

This is the long overdue follow up to my Writing a Simple Issue Tracker in Grails post. In this post I'll be detailing how to add security with the JSecurity plugin.

So let's dive right in and work on securing our application. I opted to use the JSecurity plugin for no other reason than I've used the Acegi plugin in the past and wanted to see how JSecurity compares. You could use the Acegi plugin with a similar process and results.

Following the JSecurity Quick Start guide, we'll begin by installing the plugin and running the quick start script:

grails install-plugin jsecurity
grails quick-start


The quick start script created a few domain classes as well as a controller for logging in and out. Now we need to setup an Administrator user and role to test things with. We'll just cut and paste the Administrator role setup right from the Quick Start guide into our Bootstrap.groovy file:

import org.jsecurity.crypto.hash.Sha1Hash

class BootStrap {

def init = {servletContext ->
// Administrator user and role.
def adminRole = new JsecRole(name: "Administrator").save()
def adminUser = new JsecUser(username: "admin", passwordHash: new Sha1Hash("admin").toHex()).save()
new JsecUserRoleRel(user: adminUser, role: adminRole).save()
}

def destroy = {
}
}


With the administrator user in place, we can start securing our controllers. JSecurity makes this drop dead simple by letting you specify rules in the conf/SecurityFilters.groovy file.

Before we dive in and start writing our rules, let's think about what we want users to be able to do. I'm going to keep things simple and have two classes of users: anonymous and administrators. However, you could easily create a third class of users, authenticated, that would have more permissions than the anonymous users but less than the administrators.

Administrators should be able to create, edit, and delete Projects, Components, and Issues. Additionally, Administrators should be able to access the admin controller.

Depending on how private you want your issue tracker, anonymous users might be able to list and show the Projects, Components, and Issues or they might not. They may also create new issues or they may not. To handle this, I'm going to create two new configuration parameters that control how secure the application is by adding the following lines to Config.groovy:

issue.secure.create = true
issue.secure.view = false


These configuration options will let us configure whether anonymous users can create issues (issue.secure.create = true) and whether anonymous users can view issues (issue.secure.view = false). In the above example, creating a new issue is secured (requires the user be logged in) but viewing is not.

Now that we have an idea of our security rules, let's codify them in the conf/SecurityFilters.groovy file:

import org.codehaus.groovy.grails.commons.ApplicationHolder

class SecurityFilters {
def filters = {

// secure the project controller
projectCreationAndEditing(controller: "project", action: "(create|edit|save|update|delete)") {
before = {
accessControl {
role("Administrator")
}
}
}

// secure the component controller
componentCreationAndEditing(controller: "component", action: "(create|edit|save|update|delete)") {
before = {
accessControl {
role("Administrator")
}
}
}

// secure issue editing
issueEditing(controller: "issue", action: "(edit|update|delete)") {
before = {
accessControl {
role("Administrator")
}
}
}

// secure admin controller
admin(controller: "admin", action: "*") {
before = {
accessControl {
role("Administrator")
}
}
}

// secure creating issues if Config#issue.secure.create = true
if (ApplicationHolder.application.config?.issue?.secure?.create) {
issueCreation(controller: "issue", action: "(create|save)") {
before = {
accessControl {
role("Administrator")
}
}
}
}

// secure viewing issues if Config#issue.secure.view = true
if (ApplicationHolder.application.config?.issue?.secure?.view) {
issueBrowsing(controller: "issue", action: "(show|list)") {
before = {
accessControl {
role("Administrator")
}
}
}

componentBrowsing(controller: "component", action: "(show|list)") {
before = {
accessControl {
role("Administrator")
}
}
}

projectBrowsing(controller: "project", action: "(show|list)") {
before = {
accessControl {
role("Administrator")
}
}
}
}
}
}


So with that we should have a secured grails app. Obviously there are plenty of things we can improve. First off, there is no user management code. We'll probably want to generate a controller to allow new users to register as well as to allow Administrators to add new users. The views could be cleaned up quite a bit and we could add a custom logo. Finally there are numerous other options that folks have suggested, including saved searches, internationalization, etc.

I've gone ahead and created a new Google Code project with the code from this article. If you're interested in hacking on this code, let me know. In the next installment, which I promise won't take a month, I'll be adding search/filtering support via the Searchable plugin and creating an API so I can create new issues from external applications.

Cheers,
Josh

Tuesday, May 06, 2008

Visualizer, Part 2: OSGi & Native Libraries

This is the second in my series (Part 1) of posts about Visualizer. In this post I'll be talking about packaging an OSGi bundle that includes native libraries.

Visualizer uses the OpenGL bindings provided by JOGL project to display images and data. JOGL provides a series of platform-specific downloads that include a standard JAR file of Java classes and a set of native libraries for variety of platforms. This complicates the deployment of Visualizer because we need to install the appropriate version of JOGL for the user's platform. One option is to mimic JOGL and provide platform-specific builds of Visualizer that includes the appropriate version of JOGL. This isn't ideal because it adds extra steps to the build process and can introduce confusion for users trying to figure out which version of Visualizer they should download.

Fortunately, there's another option: OSGi. In a nutshell, OSGi is a component framework specification that allows you to assemble and manage applications as a collection of components (bundles). I'm not really doing OSGi justice so if you don't know what it is, you owe it to yourself to check it out. And odds are you've probably already used something built on OSGi because it seems to be everywhere these days.

Anyhow, OSGi elegantly solves our Visualizer deployment problem by allowing us to provide a single download. We simply combine the Java classes and all of the platform-specific native libraries provided into a single bundle and OSGi will detect and extract the appropriate set of native libraries based on the user's platform.

The first step was to download all of the JOGL packages for the platforms you want to support. I have users on Linux (32 & 64 bit), Mac OS X, and Windows (32 & 64 bit), so I downloaded all of these. From these downloads, I kept one copy of the JOGL JAR files and collected all of the native libraries.

The next step was to use Eclipse's excellent PDE tooling to create a new "Plugin Project from existing JAR files". I called it 'jogl' and pointed it at the JOGL JAR files. It sucked in all of the class files and spat out an OSGi bundle. If there were no native libraries, we'd be done.

Since we have native libraries, I copied them into the jogl bundle directory using a straightforward directory structure:


The 'native' directory structure is not required; you can use whatever makes sense to you. As you can see from the screenshot, I've got a set of libraries for 5 platform/processor combinations.

The final step is to make the OSGi framework aware of the libraries so it can extract the appropriate libraries when it starts up. This requires using the Bundle-NativeCode header in your bundle manifest:

Bundle-NativeCode: native/macosx/libgluegen-rt.jnilib;
native/macosx/libjogl_cg.jnilib;
native/macosx/libjogl_awt.jnilib;
native/macosx/libjogl.jnilib;
osname=mac os x;
processor=x86;
processor=ppc,
native/linux/x86/libgluegen-rt.so;
native/linux/x86/libjogl_cg.so;
native/linux/x86/libjogl_awt.so;
native/linux/x86/libjogl.so;
osname=linux;
processor=x86,
native/linux/x86-64/libgluegen-rt.so;
native/linux/x86-64/libjogl_cg.so;
native/linux/x86-64/libjogl_awt.so;
native/linux/x86-64/libjogl.so;
osname=linux;
processor=x86-64,
native/windows/x86/gluegen-rt.dll;
native/windows/x86/jogl_cg.dll;
native/windows/x86/jogl_awt.dll;
native/windows/x86/jogl.dll;
osname=win32;
processor=x86,
native/windows/x86-64/gluegen-rt.dll;
native/windows/x86-64/jogl_cg.dll;
native/windows/x86-64/jogl_awt.dll;
native/windows/x86-64/jogl.dll;
osname=win32;
processor=x86-64


One quick note: watch the whitespace when editing the bundle manifest. The OSGi specification is explicit about where whitespace is allowed and where it is required.

With this final piece we can JAR up our class files, native libraries, and bundle manifest and we should be able to use it in any OSGi implementation. When the OSGi implementation loads our bundle, it will extract the appropriate set of native libraries based on the user's osname and processor properties and make sure those libraries are available on the classpath.

I've tested this JOGL bundle on the Equinox implementation of OSGi across Mac, Windows, and Linux and it works great. If anyone is interested, I can make the pre-built bundle of JOGL available for download.

Sunday, April 13, 2008

Writing a Simple Issue Tracker in Grails, Part 1

My project for the weekend was to write a simple issue tracking webapp in Grails. I could have used something like Trac but that's overkill for my needs. I just wanted something simple where my users could report issues and request new features. I also wanted to add a few personal touches, which I'll show you along the way.

I'm going to assuming that you're not absolutely new to Grails and you've already got it installed and played around with it. If that's not the case, I suggest checking out Scott Davis's Mastering Grails series of articles. He's goes into far more detail than I do, so check them out.

Let's start by creating our project:
grails create-app simpleissue

First up is to define our domain models. We're going to keep it simple with just three models: Project, Component, and Issue. A Project has one or more Components, such as ui, documentation, etc. A Component is associated with a single Project and has zero or more Issues associated with it. The Issue object is associated with a Component and captures a bunch of information.

So let's lay down the code:
Project.groovy

class Project {
// relationships
static hasMany = [components: Component]

// fields
String name

String toString() {
return name
}

// constraints
static constraints = {
name()
components()
}
}

Component.groovy

class Component {
// relationships
static belongsTo = Project
static hasMany = [issues: Issue]

// fields
Project project
String name

// override for nice display
String toString() {
return "${project} - ${name}"
}

// constraints
static def constraints = {
name()
project()
issues()
}
}

Issue.groovy

class Issue {
// relationships
static belongsTo = Component

// fields
Component component
String type
String submitter
String description
String status = "New"
Integer bounty
Date dateCreated
Date lastUpdated

// constraints
static constraints = {
component()
type(inList: ["Defect", "Feature"])
submitter()
description(size: 0..5000)
status(inList: ["New", "Accepted", "Closed", "Won't Fix"])
bounty(range:0..12)
}
}


Most of the code is a pretty straightforward translation of our written description of the domain. You may, however, notice a few peculiar constraints. I've used a fair number of 'empty' constraints such as:

static def constraints = {
name()
project()
issues()
}

By default, Grails treats all fields in the domain class as required. I didn't want to change that, but I wanted to affect the order that the fields show up in a particular order in the web forms. By specifying the constraint, even if it is empty, it'll show up in that order in our forms. Of course, we could have customized the field order by hand directly in the view GSP code.

I also make use of the inList constraint to limit the fields to a specific set of values. Our views will be generated with an HTML select drop down containing the list of values we've specified.

Finally, we specify our issue description as being size:0..5000. This will ensure that there is plenty of space in the database for the description text. If we hadn't specified this, the description would have been generated as a varchar(255).

With our domain classes in place, we can create our controllers and views to test things out:

grails generate-all Project
grails generate-all Component
grails generate-all Issue
grails run-app

Fire up your browser and test things out by visiting http://localhost:8080/simpleissue:
and our issue creation form:

Looks pretty decent for 5 minutes of work. Poke around and test creating a project, component, and a few issues.

Customizing the Look

Now let's clean things up a bit and add some polish. The first thing I want to do is have the index page show the list of issues. We could copy and paste the code from the Issue List view or we can simply add a redirect to the top of our web-app/index.gsp file:

<% response.sendRedirect('issue/list') %>


The next thing I want to do is clean up the issue creation form. A few of the values, such as status, dateCreated, lastUpdated don't need to be specified in the form. We can go into grails-app/views/issue/create.gsp and remove those fields.

You may have noticed an odd field in the Issue domain class: bounty. You might have expected to see a field for priority on the issue. Instead, I chose to add a "beer bounty" field where the issue submitter could pledge a certain number of beers that I could redeem upon completion of the issue. This is, in my opinion, far superior to simply assigning low, medium, high priorities to issues.

As a final customization, I want to convert the number of beers into little beer mug icons to make it easy to see the important issues to fix. We'll do this by first copying the repeat example tag from the Dynamic Tag Libraries page of the documentation:
grails create-tag-lib Misc
This will create a grails-app/taglib/MiscTagLib.groovy file which we can add:

class MiscTagLib {
def repeat = {attrs, body ->
def i = Integer.valueOf(attrs["times"])
def current = 0
i.times {
out << body(++current)
}
}
}


And we'll call it in our grails-app/views/issue/list.gsp:

<g:repeat times="${issue.bounty}">
<img src="${createLinkTo(dir:'images', file:'beer.gif')}" alt="${issue.bounty} beers"/>
</g:repeat>


Here's a look at the final output:
In part 2, we're going to add in some security to prevent arbitrary user's from editing and deleting issues. We'll also add in searching/filtering support with the Searchable plugin.

Cheers.

Saturday, April 12, 2008

Searchable: Me Too!

I've got to echo what seems to be the community consensus: use the Searchable plugin for Grails if you need to do any sort of searching or filtering. It just works and it works damn well. I'm using it to search/filter some lists of domain objects in my app with nice paging support. I added the plugin to an existing, deployed application in less than an hour this afternoon. The documentation was straightforward and easy to understand.

The only real trick I did was using the 'component' option for searching related domain classes. Take my domain class below:

class SampleRequest {
static searchable = true
static hasMany = [samples: Sample]
static mappedBy = [samples: "request"]
static belongsTo = User

// fields
User investigator
Hole hole
Double top
Double bottom
String sampleType
Integer samplesRequested = 1
Double sampleSpacing = 0.0
SampleGroup sampleGroup
String notes = ""
Date created = new Date()
String status = STATE_NEW
Integer priority = 1
}


By default, related domain classes such as User, Hole, and SampleGroup above are treated as references. When I was searching for something like "micropaleo" which happens to be the name of a SampleGroup object, Searchable would return the actual SampleGroup but not all of the SampleRequest objects in that group. Since I was mainly interested in the sample requests in that group, I simply changed my searchable definition to:

static searchable = {
hole component:true
investigator component:true
sampleGroup component:true
}

and added static def searchable = true to my User, Hole, and SampleGroup domain classes. So now when I search for something like "micropaleo" or "olney" the sample requests in that group or by that user are returned.

The best part is, my user's think I'm some sort of programming deity because they asked for search and I added it that same day. Hopefully none of them read this blog and see how little work it was for me.

Cheers.

Thursday, April 10, 2008

Visualizer, Part 1: Introduction

On and off over the past couple of months, I've been working on an OpenGL visualization application, called Visualizer (original, I know), for viewing high resolution core imagery and data. Visualizer is built on and with numerous technologies: Java, OSGi, JOGL, Eclipse, Ant, etc. Along the way, I've collected a fair number of tips, tricks, and tidbits that I'm going to share here. I hope to make Visualizer posts a regular feature of the blog, at least until the tidbits run out. If you like it, please let me know by commenting.

A quick aside here: Those of you in geosciences might recognize this concept as Corelyzer. My goal was to create a simplified, streamlined version of Corelyzer that would be easy to deploy on laptops/desktops; a sort of Corelyzer "lite" edition if you will. Visualizer doesn't have nearly as many features as the current version of Corelyzer, but it also doesn't suffer some of Corelyzer's warts.

I thought I'd start off with quick introduction of what Visualizer is for folks who aren't familiar with Corelyzer. During scientific geological drilling, hundreds or thousands of core samples are taken, imaged, and analyzed. This results in several (tens of) gigabytes of data and imagery. Collecting all of this data is of little use unless you have a way to view it. This is where tools like Corelyzer and Visualizer come in. They provide a way to view the images and data in context:


This picture illustrates Corelyzer in action. Stretched across the two 30" cinema screens is a high resolution image of core drilled in Antarctica. The user can zoom in or out and pan left or right to view the all of the imagery collected on the expedition.

So now that you know a bit of the background, let's look at the requirements (in no particular order) I had for Visualizer:
  • Cross platform - I have users on Mac, Windows, and Linux so I need a solution that will work for all of them.
  • Easy to deploy - Visualizer should work out of the box with no complicated setup for most users.
  • Easy to update - Needs a fairly automated mechanism to distribute updates and new features.
  • Open, extensible platform - Building on an open, extensible platform will allow me or others to quickly add new features or customize to a specific groups needs.
  • Reasonable performance - Visualizer should be able to handle 100m or so of imagery and data with reasonable performance on commodity hardware.
In the next blog post, I'll talk about the overall design of Visualizer and specific technologies employed. Future blog entries will include: "OSGi & Native Libraries", "Poor Man's Eclipse PDE Build", "Nifty Ant Tricks", "Automated Update Checks", and anything else that I can think of.

Cheers.

Tuesday, April 08, 2008

Antarctica Service Medal

Not too much technical content tonight. I spent most of the afternoon trying to get OSGi working from a Java Webstart application without much luck. It may be something that I revisit down the road, but for the time being distributing a zip with a set of shell scripts should work. As part of the process, I also learned how to sign a Jar file, which will be useful in an upcoming project.

This morning I pushed out the first update to the Grails app I have in production. Just before performing the actual update, I remembered to double check that my production DataSource was set to 'update' instead of 'create' or 'create-drop' the database. It was lucky I checked because it was set to 'create' from when I was wrangling with MySQL. I had already made a dump of the database, so I wouldn't have lost any data, but I was glad that I checked.

On the non-technical front, I received an Antarctica Service Medal in the mail for the time I spent in Antarctica:


I probably won't have too many occasions to actually wear it, but the medal is a nice memento of my time down there.

And the countdown has begun: 9 days until I'm in the Bahamas for my wedding and honeymoon. Pretty crazy!

Sunday, April 06, 2008

Pluggable Service Implementations in Grails

This weekend I was working on a StorageService service in Grails. The service takes a file, stores it somewhere, and then returns the URL where the file can be retrieved from. I can envision having multiple implementations of this service: one that simply puts the file in a directory for something like Tomcat/Apache/Nginx to serve up, one that puts the contents of the file into a database blob and has a controller that retrieves it from the database and serves it up, and one that stores the file on Amazon S3.

I wanted to be able to easily specify which implementation to use without having to refactor my code and I wanted to be able to configure the implementation based upon the environment. For development and integration testing, it would be easiest to test with a simple File-based service. When I actually rolled it out to production, I would likely want to use something more scalable like Amazon's S3.

I didn't know how (if?) Grails handled this situation, so I decided to write my own. I drew inspiration from Shawn Harstock's post about Fun with Grails Configuration Files for managing service configuration in the various environments.

The first thing I did was create a stub service, StorageService:

import org.codehaus.groovy.grails.commons.ApplicationHolder

class StorageService {
boolean transactional = true
boolean initialized = false
def provider = null

String store(File file) {
// initialize our provider
if (!initialized) {
initializeProvider()
}

// use our provider or log the error
if (provider == null) {
// log the error
return null
} else {
return provider.store(file)
}
}

def initializeProvider() {
initialized = true;

// get our provider name
def prefix = ApplicationHolder.application.config?.storage?.provider
if (prefix == null) {
prefix = "Default"
} else {
def index = prefix.lastIndexOf('.')
if (index == -1) {
prefix = prefix[0].toUpperCase() + prefix[1..-1]
} else {
prefix = prefix[0..index] + prefix[index+1].toUpperCase() + prefix[(index+2)..-1]
}
}

// try creating the provider
provider = Class.forName("${prefix}StorageProvider")?.newInstance()
}
}

The service looks up and loads the storage provider class based on a configuration parameter. For example, my Config.groovy file might look like this:

...
environments {
development {
storage {
provider="file"
}
}
test {
storage {
provider="org.andrill.mock"
}
}
production {
storage {
provider="s3"
}
}

}
...


In development, the StorageService would try to load the class FileStorageProvider and delegate calls to it. In testing, it would try to load the class org.andrill.MockStorageProvider and delegate calls to it. And finally, in production it would try to use the S3StorageProvider.

My controllers don't need to know which provider implementation, they just declare a
StorageService storageService field, which Spring auto-wires for them, and go. It works like a charm.

So my only question is, did I just re-invent the wheel? Does Grails already have a mechanism for handling this? If not, how would you accomplish the task?

Update: Check out David Hernández's explanation of how to do this with Spring. No icky, ClassForName needed. Nice work, David!

Cheers.

Tuesday, April 01, 2008

Grails and MySQL Connection Errors

I ran into a few configuration problems when I tried to push a Grails webapp into production and make it talk to a MySQL database instead of the standard HSQL database. These notes might be useful to someone who is pulling their hair out trying to get Grails talking to MySQL database.

My production machine is a CentOS 5 box running MySQL 5, Sun JDK 5, and Tomcat 5.5 (incidentally, my webapp's version happened to be 0.5 which rounds out the '5' theme nicely).

I started by pulling down the MySQL JDBC driver and dropping it into my lib directory. I then updated the production section in my DataSource.groovy file:

...
production {
dataSource {
pooling = true
driverClassName = "com.mysql.jdbc.Driver"
dbCreate = "create"
url = "jdbc:mysql://localhost:3306/grails_webapp"
username = "grailsuser"
password = "XXXX"
}
}
...


I then grails wared things up on my local machine and copied it over to the production server.

Over on the production server, I connected to MySQL with the mysql command line utility to create my database and user:

create database grails_webapp;
grant all privileges on grails_webapp.* to 'grailsuser'@'localhost' identified by 'XXXX';
flush privileges;


I verified things worked by connecting to the new database as the newly created user and creating a table.

I dropped the war into my tomcat/webapps directory and watched the logs. The webapp tries to start but quickly dies, leaving behind a log full of Communications link failure and Connection refused messages. I could see that the MySQL JDBC driver was found, but for whatever reason it couldn't connect to the database server.

After double checking my DataSource.groovy file to make sure my URL, username, and password were all spelled correctly and connecting again locally from the command line to ensure that the database is indeed running, I started Googling.

Google revealed that one common problem is to forget to enable networking in MySQL. I double checked my my.cnf and confirmed that I had enabled networking by specifying a port and an address to bind to. This is where I noticed that the bound address wasn't localhost or 127.0.0.1, but the actual IP address of the machine.

Normally you would expect MySQL to only need to listen locally. In my case, however, the production MySQL server replicates to a slave server for backup, so it needs to listen on the external interface. The firewall locked down to prevent all connections to the MySQL port except from the slave server.

On a hunch, I went back and changed my database URL to jdbc:mysql://the.ip.addr.ess:3306/grails_webapp and tried re-deploying. Things still failed, but with access denied errors instead of connection refused errors. Progress.

The final piece of the puzzle was to go and grant my grailsuser access on all addresses instead of just localhost:

grant all privileges on grails_webapp.* to 'grailsuser'@'%' identified by 'XXXX';
flush privileges;


Once this was done, I re-deployed again and was greeted with an error free log. Huzzah!

Monday, March 31, 2008

Back from Boston

I made it back from Boston. The trip was fun but I did not get as much done as I would have at home. Things are looking to be pretty busy this week as I scramble to get caught up. The wedding is in a few weeks. 10 days on the beach in the Bahamas. That thought is what's getting me through the work and stress.

In other news, I'm starting a new project in my spare time. I'm going to keep things under wraps for a bit as I don't have a solid time frame. However, one thing I will mention is that it's going to be a webapp and it's going to contain a social component. I'm hoping the development will yield a series of useful Groovy/Grails blog posts.

I'd also really like to pull together a generic "web 2.0 template" project in Grails. Something that would pull together plugins for security and contain User, Profile, Role-type domain classes, controllers, and views. Ideally you should be able to use it to bootstrap a generic Web 2.0 site with all of the user registration and profile management taken care of. Then you can start adding your specific domain classes to make the site useful.

Thursday, March 27, 2008

Blogging from Boston

I'm currently at the National Science Teacher Association (NSTA) national meeting in Boston. I'm just here to help out with the ANDRILL booth and get some much needed face to face time with my co-workers (something we don't often get since I work remotely). The trip has been fun so far, though not as productive as being at home.

I took a few minutes today to fix a bug in the Simile Timeline depth code--when scrolling the timeline, the overview bars would get out of sync. If you're using the depth unit extension, please grab the latest version or check out the diff. I've also updated the Simile Grails plugin, so you may want to update your version from the repository. It fixes the same depth bug and updates the SimileTagLib to work a bit more independently from the domain classes.

Monday, March 24, 2008

Customizing Timeline Event Bubbles

Tonight is a two for one special on blog posts. I'm heading out to Boston tomorrow for the rest of the week, so I don't know how much time I'm going to have to blog while out there. If you happen to be reading and are going to be in the Boston area (either for the NSTA meeting like me or whatever), shoot me an email and let's meet up for a beer.

In this post, I'm going to show you how to customize the rendering of event bubbles in Simile Timeline. The default bubbles look pretty nice:
but they lack a few features that I needed, namely clicking on the thumbnail (if present) should take the user to the event link and the start and end depth should be rendered on a single line:

The markup for the event bubble is built programmatically in the fillInfoBubble() function on Timeline.DefaultEventSource.Event class. You can find the code at the end of the sources.js file in the Timeline distribution.

To customize the event bubble rendering, we just need to modify the fillInfoBubble() function and make it generate the markup we want. One option would be to go in and edit the sources.js file directly. This would accomplish our goal but it would mean we'd be modifying the Timeline source code and we would have to patch the file every time it was changed in the Timeline trunk.

The second option is to replace the function "on the fly" by modifying the Timeline.DefaultEventSource.Event prototype in our own code. The end is the same but we can do it without touching the Timeline sources. To accomplish this, we just need to overwrite the function after sources.js loads:

(function() {
Timeline.DefaultEventSource.Event.prototype.fillInfoBubble = function (elmt, theme, labeller) {
// build our custom markup
};
})();


What this code does is create an anonymous function that runs immediately. Inside that function, we replace the current fillInfoBubble() implementation on the Event class with our own.

This code is an improvement over editing sources.js directly, but we can do one better. The problem with this approach is that we're replacing fillInfoBubble() for everyone. We're breaking any clients that depend on the old implementation, which is just bad form. What we'd really like to do is selectively apply our new implementation. Fortunately that's easy enough to do:

(function() {
var default_fillInfo = Timeline.DefaultEventSource.Event.prototype.fillInfoBubble;
Timeline.DefaultEventSource.Event.prototype.fillInfoBubble = function (elmt, theme, labeller) {
// if not a depth labeller, use the original
if (!labeller.isDepthLabeller) {
default_fillInfo.apply(this, arguments);
return;
}

// build our custom markup
};
})();


In this updated snippet we save the old function implementation then we check to see if the labeller is a DepthLabeller. If it isn't, we just pass it on to the original implementation, otherwise we use our customized implementation.

For completeness sake, here's the snippet I used to make the thumbnail image clickable and to render the depths on the same line:

if (image != null) {
var img = doc.createElement("img");
img.src = image;
theme.event.bubble.imageStyler(img);

if (link != null) {
var a = doc.createElement("a");
a.href = link;
a.target = "_blank"
a.appendChild(img);
elmt.appendChild(a);
} else {
elmt.appendChild(img);
}
}

var divTime = doc.createElement("div");
if (this.isInstant()) {
divTime.appendChild(elmt.ownerDocument.createTextNode(labeller.labelPrecise(this.getStart())));
} else {
divTime.appendChild(elmt.ownerDocument.createTextNode(labeller.labelPrecise(this.getStart())));
divTime.appendChild(elmt.ownerDocument.createTextNode(" - "));
divTime.appendChild(elmt.ownerDocument.createTextNode(labeller.labelPrecise(this.getEnd())));
}


Hope that was clear as mud!