Bahamas

Sorry for the radio silence, I'm currently in the Bahamas for my wedding and honeymoon. I should be back to blogging next week.

Cheers,
Josh

Writing a Simple Issue Tracker in Grails, Part 1

My project for the weekend was to write a simple issue tracking webapp in Grails. I could have used something like Trac but that's overkill for my needs. I just wanted something simple where my users could report issues and request new features. I also wanted to add a few personal touches, which I'll show you along the way.

I'm going to assuming that you're not absolutely new to Grails and you've already got it installed and played around with it. If that's not the case, I suggest checking out Scott Davis's Mastering Grails series of articles. He's goes into far more detail than I do, so check them out.

Let's start by creating our project:
grails create-app simpleissue

First up is to define our domain models. We're going to keep it simple with just three models: Project, Component, and Issue. A Project has one or more Components, such as ui, documentation, etc. A Component is associated with a single Project and has zero or more Issues associated with it. The Issue object is associated with a Component and captures a bunch of information.

So let's lay down the code:
Project.groovy

class Project {
    // relationships
    static hasMany = [components: Component]

    // fields
    String name

    String toString() {
        return name
    }

    // constraints
    static constraints = {
        name()
        components()
    }
}

Component.groovy

class Component {
    // relationships
    static belongsTo = Project
    static hasMany = [issues: Issue]

    // fields
    Project project
    String name

    // override for nice display
    String toString() {
        return "${project} - ${name}"
    }

    // constraints
    static def constraints = {
        name()
        project()
        issues()
    }
}

Issue.groovy

class Issue {
    // relationships
    static belongsTo = Component

    // fields
    Component component
    String type
    String submitter
    String description
    String status = "New"
    Integer bounty
    Date dateCreated
    Date lastUpdated

    // constraints
    static constraints = {
        component()
        type(inList: ["Defect", "Feature"])
        submitter()
        description(size: 0..5000)
        status(inList: ["New", "Accepted", "Closed", "Won't Fix"])
        bounty(range:0..12)
    }
}

Most of the code is a pretty straightforward translation of our written description of the domain. You may, however, notice a few peculiar constraints. I've used a fair number of 'empty' constraints such as:

static def constraints = {
   name()
   project()
   issues()
}

By default, Grails treats all fields in the domain class as required. I didn't want to change that, but I wanted to affect the order that the fields show up in a particular order in the web forms. By specifying the constraint, even if it is empty, it'll show up in that order in our forms. Of course, we could have customized the field order by hand directly in the view GSP code.

I also make use of the inList constraint to limit the fields to a specific set of values. Our views will be generated with an HTML select drop down containing the list of values we've specified.

Finally, we specify our issue description as being size:0..5000. This will ensure that there is plenty of space in the database for the description text. If we hadn't specified this, the description would have been generated as a varchar(255).

With our domain classes in place, we can create our controllers and views to test things out:

grails generate-all Project
grails generate-all Component
grails generate-all Issue
grails run-app

Fire up your browser and test things out by visiting http://localhost:8080/simpleissue:
and our issue creation form:

Looks pretty decent for 5 minutes of work. Poke around and test creating a project, component, and a few issues.

Customizing the Look

Now let's clean things up a bit and add some polish. The first thing I want to do is have the index page show the list of issues. We could copy and paste the code from the Issue List view or we can simply add a redirect to the top of our web-app/index.gsp file:

<% response.sendRedirect('issue/list') %>

The next thing I want to do is clean up the issue creation form. A few of the values, such as status, dateCreated, lastUpdated don't need to be specified in the form. We can go into grails-app/views/issue/create.gsp and remove those fields.

You may have noticed an odd field in the Issue domain class: bounty. You might have expected to see a field for priority on the issue. Instead, I chose to add a "beer bounty" field where the issue submitter could pledge a certain number of beers that I could redeem upon completion of the issue. This is, in my opinion, far superior to simply assigning low, medium, high priorities to issues.

As a final customization, I want to convert the number of beers into little beer mug icons to make it easy to see the important issues to fix. We'll do this by first copying the repeat example tag from the Dynamic Tag Libraries page of the documentation:
grails create-tag-lib Misc
This will create a grails-app/taglib/MiscTagLib.groovy file which we can add:

class MiscTagLib {
    def repeat = {attrs, body ->
        def i = Integer.valueOf(attrs["times"])
        def current = 0
        i.times {
            out << body(++current)
        }
    }
}

And we'll call it in our grails-app/views/issue/list.gsp:

<g:repeat times="${issue.bounty}">
    <img src="${createLinkTo(dir:'images', file:'beer.gif')}" alt="${issue.bounty} beers"/>
</g:repeat>

Here's a look at the final output:
In part 2, we're going to add in some security to prevent arbitrary user's from editing and deleting issues. We'll also add in searching/filtering support with the Searchable plugin.

Cheers.

Searchable: Me Too!

I've got to echo what seems to be the community consensus: use the Searchable plugin for Grails if you need to do any sort of searching or filtering. It just works and it works damn well. I'm using it to search/filter some lists of domain objects in my app with nice paging support. I added the plugin to an existing, deployed application in less than an hour this afternoon. The documentation was straightforward and easy to understand.

The only real trick I did was using the 'component' option for searching related domain classes. Take my domain class below:

class SampleRequest {
    static searchable = true
    static hasMany = [samples: Sample]
    static mappedBy = [samples: "request"]
    static belongsTo = User

    // fields
    User investigator
    Hole hole
    Double top
    Double bottom
    String sampleType
    Integer samplesRequested = 1
    Double sampleSpacing = 0.0
    SampleGroup sampleGroup
    String notes = ""
    Date created = new Date()
    String status = STATE_NEW
    Integer priority = 1
}

By default, related domain classes such as User, Hole, and SampleGroup above are treated as references. When I was searching for something like "micropaleo" which happens to be the name of a SampleGroup object, Searchable would return the actual SampleGroup but not all of the SampleRequest objects in that group. Since I was mainly interested in the sample requests in that group, I simply changed my searchable definition to:

static searchable = {
    hole component:true
    investigator component:true
    sampleGroup component:true
}

and added static def searchable = true to my User, Hole, and SampleGroup domain classes. So now when I search for something like "micropaleo" or "olney" the sample requests in that group or by that user are returned.

The best part is, my user's think I'm some sort of programming deity because they asked for search and I added it that same day. Hopefully none of them read this blog and see how little work it was for me.

Cheers.

Visualizer, Part 1: Introduction

On and off over the past couple of months, I've been working on an OpenGL visualization application, called Visualizer (original, I know), for viewing high resolution core imagery and data. Visualizer is built on and with numerous technologies: Java, OSGi, JOGL, Eclipse, Ant, etc. Along the way, I've collected a fair number of tips, tricks, and tidbits that I'm going to share here. I hope to make Visualizer posts a regular feature of the blog, at least until the tidbits run out. If you like it, please let me know by commenting.

A quick aside here: Those of you in geosciences might recognize this concept as Corelyzer. My goal was to create a simplified, streamlined version of Corelyzer that would be easy to deploy on laptops/desktops; a sort of Corelyzer "lite" edition if you will. Visualizer doesn't have nearly as many features as the current version of Corelyzer, but it also doesn't suffer some of Corelyzer's warts.

I thought I'd start off with quick introduction of what Visualizer is for folks who aren't familiar with Corelyzer. During scientific geological drilling, hundreds or thousands of core samples are taken, imaged, and analyzed. This results in several (tens of) gigabytes of data and imagery. Collecting all of this data is of little use unless you have a way to view it. This is where tools like Corelyzer and Visualizer come in. They provide a way to view the images and data in context:


This picture illustrates Corelyzer in action. Stretched across the two 30" cinema screens is a high resolution image of core drilled in Antarctica. The user can zoom in or out and pan left or right to view the all of the imagery collected on the expedition.

So now that you know a bit of the background, let's look at the requirements (in no particular order) I had for Visualizer:

• Cross platform - I have users on Mac, Windows, and Linux so I need a solution that will work for all of them.
• Easy to deploy - Visualizer should work out of the box with no complicated setup for most users.
• Easy to update - Needs a fairly automated mechanism to distribute updates and new features.
• Open, extensible platform - Building on an open, extensible platform will allow me or others to quickly add new features or customize to a specific groups needs.
• Reasonable performance - Visualizer should be able to handle 100m or so of imagery and data with reasonable performance on commodity hardware.

In the next blog post, I'll talk about the overall design of Visualizer and specific technologies employed. Future blog entries will include: "OSGi & Native Libraries", "Poor Man's Eclipse PDE Build", "Nifty Ant Tricks", "Automated Update Checks", and anything else that I can think of.

Cheers.

Antarctica Service Medal

Not too much technical content tonight. I spent most of the afternoon trying to get OSGi working from a Java Webstart application without much luck. It may be something that I revisit down the road, but for the time being distributing a zip with a set of shell scripts should work. As part of the process, I also learned how to sign a Jar file, which will be useful in an upcoming project.

This morning I pushed out the first update to the Grails app I have in production. Just before performing the actual update, I remembered to double check that my production DataSource was set to 'update' instead of 'create' or 'create-drop' the database. It was lucky I checked because it was set to 'create' from when I was wrangling with MySQL. I had already made a dump of the database, so I wouldn't have lost any data, but I was glad that I checked.

On the non-technical front, I received an Antarctica Service Medal in the mail for the time I spent in Antarctica:


I probably won't have too many occasions to actually wear it, but the medal is a nice memento of my time down there.

And the countdown has begun: 9 days until I'm in the Bahamas for my wedding and honeymoon. Pretty crazy!

Pluggable Service Implementations in Grails

This weekend I was working on a StorageService service in Grails. The service takes a file, stores it somewhere, and then returns the URL where the file can be retrieved from. I can envision having multiple implementations of this service: one that simply puts the file in a directory for something like Tomcat/Apache/Nginx to serve up, one that puts the contents of the file into a database blob and has a controller that retrieves it from the database and serves it up, and one that stores the file on Amazon S3.

I wanted to be able to easily specify which implementation to use without having to refactor my code and I wanted to be able to configure the implementation based upon the environment. For development and integration testing, it would be easiest to test with a simple File-based service. When I actually rolled it out to production, I would likely want to use something more scalable like Amazon's S3.

I didn't know how (if?) Grails handled this situation, so I decided to write my own. I drew inspiration from Shawn Harstock's post about Fun with Grails Configuration Files for managing service configuration in the various environments.

The first thing I did was create a stub service, StorageService:

import org.codehaus.groovy.grails.commons.ApplicationHolder

class StorageService {
   boolean transactional = true
   boolean initialized = false
   def provider = null

   String store(File file) {
       // initialize our provider
       if (!initialized) {
           initializeProvider()
       }

       // use our provider or log the error
       if (provider == null) {
           // log the error
           return null
       } else {
           return provider.store(file)
       }
   }

   def initializeProvider() {
       initialized = true;

       // get our provider name
       def prefix = ApplicationHolder.application.config?.storage?.provider
       if (prefix == null) {
           prefix = "Default"
       } else {
           def index = prefix.lastIndexOf('.')
           if (index == -1) {
               prefix = prefix[0].toUpperCase() + prefix[1..-1]
           } else {
               prefix = prefix[0..index] + prefix[index+1].toUpperCase() + prefix[(index+2)..-1]
           }
       }

       // try creating the provider
       provider = Class.forName("${prefix}StorageProvider")?.newInstance()
   }
}

The service looks up and loads the storage provider class based on a configuration parameter. For example, my Config.groovy file might look like this:

...
environments {
   development {
       storage {
           provider="file"
       }
   }
   test {
       storage {
           provider="org.andrill.mock"
       }
   }
   production {
       storage {
           provider="s3"
       }
   }

}
...

In development, the StorageService would try to load the class FileStorageProvider and delegate calls to it. In testing, it would try to load the class org.andrill.MockStorageProvider and delegate calls to it. And finally, in production it would try to use the S3StorageProvider.

My controllers don't need to know which provider implementation, they just declare a
StorageService storageService field, which Spring auto-wires for them, and go. It works like a charm.

So my only question is, did I just re-invent the wheel? Does Grails already have a mechanism for handling this? If not, how would you accomplish the task?

Update: Check out David Hernández's explanation of how to do this with Spring. No icky, ClassForName needed. Nice work, David!

Cheers.

Grails and MySQL Connection Errors

I ran into a few configuration problems when I tried to push a Grails webapp into production and make it talk to a MySQL database instead of the standard HSQL database. These notes might be useful to someone who is pulling their hair out trying to get Grails talking to MySQL database.

My production machine is a CentOS 5 box running MySQL 5, Sun JDK 5, and Tomcat 5.5 (incidentally, my webapp's version happened to be 0.5 which rounds out the '5' theme nicely).

I started by pulling down the MySQL JDBC driver and dropping it into my lib directory. I then updated the production section in my DataSource.groovy file:

...
 production {
       dataSource {
           pooling = true
           driverClassName = "com.mysql.jdbc.Driver"
           dbCreate = "create"
           url = "jdbc:mysql://localhost:3306/grails_webapp"
           username = "grailsuser"
           password = "XXXX"
       }
   }
...

I then grails wared things up on my local machine and copied it over to the production server.

Over on the production server, I connected to MySQL with the mysql command line utility to create my database and user:

create database grails_webapp;
grant all privileges on grails_webapp.* to 'grailsuser'@'localhost' identified by 'XXXX';
flush privileges;

I verified things worked by connecting to the new database as the newly created user and creating a table.

I dropped the war into my tomcat/webapps directory and watched the logs. The webapp tries to start but quickly dies, leaving behind a log full of Communications link failure and Connection refused messages. I could see that the MySQL JDBC driver was found, but for whatever reason it couldn't connect to the database server.

After double checking my DataSource.groovy file to make sure my URL, username, and password were all spelled correctly and connecting again locally from the command line to ensure that the database is indeed running, I started Googling.

Google revealed that one common problem is to forget to enable networking in MySQL. I double checked my my.cnf and confirmed that I had enabled networking by specifying a port and an address to bind to. This is where I noticed that the bound address wasn't localhost or 127.0.0.1, but the actual IP address of the machine.

Normally you would expect MySQL to only need to listen locally. In my case, however, the production MySQL server replicates to a slave server for backup, so it needs to listen on the external interface. The firewall locked down to prevent all connections to the MySQL port except from the slave server.

On a hunch, I went back and changed my database URL to jdbc:mysql://the.ip.addr.ess:3306/grails_webapp and tried re-deploying. Things still failed, but with access denied errors instead of connection refused errors. Progress.

The final piece of the puzzle was to go and grant my grailsuser access on all addresses instead of just localhost:

grant all privileges on grails_webapp.* to 'grailsuser'@'%' identified by 'XXXX';
flush privileges;

Once this was done, I re-deployed again and was greeted with an error free log. Huzzah!