Executing SQL queries with GORM

Yesterday at work somebody asked me how to execute a SQL query through GORM. I didn't remember at the moment how exactly how to do it, but I remembered that Hibernate's Session class had a method called executeSQLQuery.

This is a silly example to show how we have done it. Lets say you have a domain class 'MyDomainClass' and inside a service (MyService) you need to execute a given SQL query. It could be like this:

  class MyDomainClass {
      String name
      Integer age
  }

  class MyService {
      Integer getAgeByExecutingSQL() {
         return MyDomainClass.withSession { session ->
            session.executeSQLQuery("select age from my_domain_class").get()
         }
      }
  }

Accessing Hibernate session through withSession method we can access the method I was mentioning above. Then is just a matter of creating a valid SQL and of course keep in mind that the query doesn't return domain class instances but arrays of data.

New conditional expressions in Groovy

Many times I've heard someone to say it would be great to have some more control expressions. The good thing in Groovy is that because its syntax you can build your own expressions like if they were already part of the language.

I've created a couple of my own inspired by some other languages out there:

Unless

Unless is a typical expression in other languages like Ruby and sometimes in Groovy it's just annoying to use the not operator. Let's say for example we had this piece of code:

if (!companyIsBankrupt) {
     invest()
}

Wouldn't be nicer to see...

unless(companyIsBankrupt()) {
     invest()
}

Of course it may depend on the context but I think is more readable and clearer than the first attempt. Also this expression not only serves as conditional but because it's been implemented as a method it could return a value, so the latest example could become:

def investment = unless(companyIsBankrupt()) {
     invest()
}

Where

This expression is more complex because it has an internal state and more embedded conditional expressions. It is inspired by the where expression in Haskell.

check(weight: 60) {
    when { weight <= underweight } then { "You're underweight" }
    when { weight <= normal }      then { "You're normal" }
    when { weight <= fat }         then { "You're fat" }
    otherwise { "You have a strange composition" }
    where {
       underweight = 50
       normal = 70
       fat  = 90
    }
}

You can initialize the whole expression with a map and then use the when-then branches to execute the right one taking into account set values within the where clause.

When-then clauses are half way between if-else expressions and switch-case expressions but there is something unique about this type of expression, pre-set values within the scope of the expression and shared among all when-then expressions.

My initial implementation is still a proof of concep and still lacks of some desirable constraints:

• Initialization values at check() method should be immutable values
• Values created within the where block should be immutable too

I forgot to mention that the same way unless expression could return a value this check-where expression can do it as well so it won't be any problem to assign the execution of the where clause to a given variable:

def message = check(weight: 60) {
    when { weight <= underweight } then { "You're underweight" }
    when { weight <= normal }      then { "You're normal" }
    when { weight <= fat }         then { "You're fat" }
    otherwise { "You have a strange composition" }
    where {
       underweight = 50
       normal = 70
       fat  = 90
    }
}

Resources

• Github repository for both samples
• Unless reference in Ruby
• Where reference in Haskell

Searchable plugin: Search by a domain relationship

I don't remember last time I used Searchable Plugin, there must be like a year or so. Anyway I was trying to map a Product/I18nProductDescription domain classes to be able to search by the product description and then return the product id in order to build the link to the product detail.
I was using the default configuration and doing the queries through searchable controller to test my mappings. These were the initial mappings:

class Product {

    static hasMany = [descriptions: I18NDescription]   

}

class I18nDescription {

   static belongsTo = [product: Product]
   static searchable = true

   Locale locale
   String title
   String description

}

The problem was that if you map as "searchable" only I18nDescription, the index doesn't store the relationship, then you won't be able to do "productDescription.product.id" and you'll get a NPE (I guess this is caused because instances are not taken from the hibernate session but from lucene index).

I read the documentation and I found the "component" term.

class Product {

   static hasMany = [descriptions: I18NDescription]   
   static searchable = {
      descriptions component: true
   }

}

class I18nDescription {

   static belongsTo = [product: Product]
   static searchable = {
      product component: true
   }

   Locale locale
   String title
   String description

}

So I mapped my to domain classes as components of each other. But if I stopped there I would get double of results, because both domain classes are pointing to each other right? So I need to "disable" for searching one of them. To do so, one of them should have an option to tell it's not going to have it's one index but is going to be part of another search index.

class Product {

   static hasMany = [descriptions: I18NDescription]   
   static searchable = {
      descriptions component: true
      mapping {
        root false
      }
   }

}

class I18nDescription {

   static belongsTo = [product: Product]
   static searchable = {
      product component: true
      mapping {
        root true
      }
   }

   Locale locale
   String title
   String description

}

That way you can search by title or description and be able to get the reference of the product without having to do anything else.

References

• Searchable plugin
• Searchable Mapping Options
• Searchable components

Testing a command object without a controller

Sometimes you just want to test a command object in a Spock specification without having to use @TestFor with an existing controller class passed as an argument.

Back in time (last week or so XD ) I used to do it this way

import spock.lang.Specification
import grails.test.mixin.TestFor

@TestFor(SomeControllerClass)
class MyCommandObjectSpec extends Specification {
   // ... your stuff
}

But the thing is that Grails wraps all its mocking functionality in "mixins"  and the way you can apply those mixins is through the @TestMixin annotation.

Normally you don't notice this behavior because the annotation @TestFor does that under the hood, but if you had to apply controller mocking behavior in your specification you should be using the @TestMixin annotation with the ControllerUnitTestMixin directly:

import spock.lang.Specification
import grails.test.mixin.TestMixin
import grails.test.mixin.web.ControllerUnitTestMixin

@TestMixin(ControllerUnitTestMixin)
class MyCommandObjectSpec extends Specification {
  //... your stuff
}

Happy testing :)

Generating an Inner class using Groovy AST transformations

Oh man! This was hard to find XD!

During the weekend I've been trying to create a simple AST transformation that generates an inner class. I dove into Groovy's documentation and googled during the last two days, but it seemed nobody has struggled with the same issue.

In the end I achieved the goal, and I thought it would be nice to post it in case anyone could have the same problem :)

The transformation takes a method annotated with @MoveToInner and moves it to an inner class. The inner class will have the name passed as parameter to the annotation.

Here is the code:

package com.github.groovy.astextras.local.inner

import org.codehaus.groovy.ast.ASTNode
import org.codehaus.groovy.ast.MethodNode
import org.codehaus.groovy.ast.ClassNode
import org.codehaus.groovy.ast.AnnotationNode

import org.codehaus.groovy.ast.builder.AstBuilder

import org.codehaus.groovy.control.SourceUnit
import org.codehaus.groovy.control.CompilePhase
import org.codehaus.groovy.control.CompilationUnit
import org.codehaus.groovy.control.CompilerConfiguration

import org.codehaus.groovy.transform.ASTTransformation
import org.codehaus.groovy.transform.GroovyASTTransformation

/**
 * This transformation takes the method annotated with @MoveToInner and effectively
 * moves it to an inner class which has the name of the value given as value to the
 * former annotation.
 *
 * This way a code like the following:
 * 
 *   package com.github.groovy.astextras.local.inner
 *
 *   class Something {
 *     @MoveToInner("Foo")
 *     def myMethod() {
 *       return "Hello John"
 *     }
 *   }
 * 
*
 * Will become...
 *
 * 
 *   package com.github.groovy.astextras.local.inner
 *
 *   class Something {
 *
 *      class Foo {
 *       def myMethod() {
 *          return "Hello John"
 *       }
 *      }
 *
 *   }
 * 
*
 *
 */
@GroovyASTTransformation(phase = CompilePhase.INSTRUCTION_SELECTION)
class MoveToInnerAst implements ASTTransformation {

    static final VALUE = 'value'
    static final DOLLAR = '$'
    static final PUBLIC = ClassNode.ACC_PUBLIC

    void visit(ASTNode[] astNodes, SourceUnit sourceUnit) {

        if (!checkNodes(astNodes)) return

        def annotationNode = astNodes[0]
        def methodNode = astNodes[1]
        def declaringClass = methodNode.declaringClass

        def innerClassName = annotationNode.members.getAt(VALUE)?.text
        def outerClassName = declaringClass.name

        def outerClassNode = createOuterClass(outerClassName)
        def innerClassNode = createInnerClass(outerClassName, innerClassName)

        innerClassNode.addMethod(cloneNode(methodNode))

        def compilerConfiguration = sourceUnit.getAST().getUnit().config
        def compilationUnit =
            new CompilationUnit(compilerConfiguration).with {
                addClassNode(outerClassNode)
                addClassNode(innerClassNode)
                compile()
            }

    }

    /**
     * This method checks that the nodes passed as parameters are the ones we
     * want to visit
     *
     * @param astNodes The nodes we may want to process
     * @return true if the nodes are the ones we were looking for, false otherwise
     */
    def checkNodes(ASTNode[] astNodes) {

        astNodes &&
        astNodes[0] &&
        astNodes[1] &&
        astNodes[0] instanceof AnnotationNode &&
        astNodes[0].classNode?.name == MoveToInner.class.name &&
        astNodes[1] instanceof MethodNode

    }

    /**
     * This method creates an inner class
     *
     * @param qualifiedOuterClassName qualified outer class name (with the name of the package)
     * @param simpleInnerClassName name of the inner class (without the name of the package)
     * @return an instance of an InnerClassNode
     */
    def createInnerClass(String qualifiedOuterClassName, String simpleInnerClassName) {

        def innerClassFullName = qualifiedOuterClassName + DOLLAR + simpleInnerClassName

        new AstBuilder().buildFromSpec {
            innerClass(innerClassFullName, PUBLIC) {
                classNode(qualifiedOuterClassName, PUBLIC) {
                    classNode Object
                    interfaces { classNode GroovyObject }
                    mixins { }
                }
                classNode Object
                interfaces { classNode GroovyObject }
                mixins { }
            }
        }.first()

    }

    /**
     * This method creates an empty class node with the qualified name passed as parameter
     *
     * @param qualifiedClassNodeName The qualified name of the ClassNode we want to create
     * @return a new ClassNode instance
     */
    def createOuterClass(String qualifiedClassNodeName) {

        new AstBuilder().buildFromSpec {
            classNode(qualifiedClassNodeName, PUBLIC) {
                classNode Object
                interfaces { classNode GroovyObject }
                mixins { }
            }
        }.first()

    }

    /**
     * This method clones the method node passed as parameter
     *
     * @param methodNode the MethodNode instance we want to clone from
     * @return a cloned instance of the node passed as parameter
     */
    def cloneNode(MethodNode methodNode) {

        new MethodNode(
            methodNode.name,
            methodNode.modifiers,
            methodNode.returnType,
            methodNode.parameters,
            methodNode.exceptions,
            methodNode.code
        )

    }

}

The hidden gem of the solution was the following lines:

def compilerConfiguration = sourceUnit.getAST().getUnit().config
        def compilationUnit =
            new CompilationUnit(compilerConfiguration).with {
                addClassNode(outerClassNode)
                addClassNode(innerClassNode)
                compile()
            }

Those lines are responsible to create the .class files for the inner and the outer class. Before using those lines I was unable to make them visible for the rest of the classes. If anybody knows how to do it in any other way, please tell me, I'm all ears.

You can also get the compiler compilation configuration with the shorter expression:

   def compilerConfiguration = classNode.compileUnit.config

 UPDATE (Thanks to Andres Almiray) :

Andres pointed out that the solution was even easier. It was unnecessary to get the compilation unit. The only thing I had to do was:

 void visit(ASTNode[] astNodes, SourceUnit sourceUnit) {

        if (!checkNodes(astNodes)) return

        def annotationNode = astNodes[0]
        def methodNode = astNodes[1]
        def declaringClass = methodNode.declaringClass

        def innerClassName = annotationNode.members.getAt(VALUE)?.text
        def outerClassName = declaringClass.name

        def outerClassNode = createOuterClass(outerClassName)
        def innerClassNode = createInnerClass(outerClassName, innerClassName)

        innerClassNode.addMethod(cloneNode(methodNode))

        methodNode.declaringClass.module.addClass(innerClassNode) 
 }

As I mentioned earlier you can find this code and more AST examples at Github. 

Exposing Grails URLMappings as JSON

I'm currently working in a REST API. The endpoints are increasing really fast and we haven't decided how to document it yet. Until that happens I was wondering it would be nice to know how to access URLMappings data at runtime.

We've created a controller where the URLMappings structure is exposed. Only thing we have to do is to access to "grailsApplication" and "grailsUrlMappingsHolder" beans.

import grails.converters.JSON

class RootController extends BaseApiController {
   
      def grailsApplication
      def grailsUrlMappingsHolder
  
      def listApiUrls() {
  
          def urlMappings = grailsUrlMappingsHolder.urlMappings.collect { [
  
                  name: it.mappingName?:'',
                  url: it.urlData.logicalUrls.first(),
                  methods: it.parameterValues,
                  parameters: it.constraints.propertyName
 
          ] }
  
  
          def root = [
  
              metadata: grailsApplication.metadata,
              urlMappings: urlMappings
  
          ]
  
          render root as JSON
  
      }
  
  }    

We put all the information in a map and convert it to JSON and....

{
  "metadata": {
    "app.version": "0.1",
    "app.grails.version": "2.2.4",
    "app.name": "yump-api"
  },
  "urlMappings": 
  [
    {
      "name": "",
      "url": "500",
      "methods": {
        "action": "handleValidationException",
        "controller": "exception"
      },
      "parameters": 
      [
      ]
    },
    {
      "name": "",
      "url": "500",
      "methods": {
        "action": "handleApiException",
        "controller": "exception"
      },
      "parameters": 
      [
      ]
    }...

Here we have a basic version of a API documentation with a few lines :)

Groovy and C++ through JNA

Last week I attended a talk at Kaleidos about integrating/extending Python with C++ (Great talk by the way).

Because there's a kind of healthy competition inside Kaleidos.net I'm thinking about coding some C++ again (I probably did my last line of c++ like 10 years ago but WTF let's do this!!! :P)

When talking about integrating C++ with the JVM (as far as I know) there's only two choices JNI and JNA.

I googled for a while and I got to the point where I decided to go for JNA instead of JNI. The reason was the JNI looked more cumbersome than JNA.

First step: Create your C++ code:

Well the first step was to create my C++ code. That's easier said than done. Like I said my C++ is more than rusty. Anyway I came up with the following sample code, don't take it too seriously:

//Algorithms.h

class Algorithms {         
   
     public:
       int getMax(int*,int);
 
 };

//Algorithms.cpp

#include "Algorithms.h"    
   
extern "C" int getMax(int* numbers,int numbersLength) {
   
   Algorithms instance;     
   return instance.getMax(numbers,numbersLength);
 
}
  
int Algorithms::getMax(int* numbers,int numbersLength) {
  
   int max = 0;
  
   for (int i = 0; i < numbersLength ; i++) { 
     int next = numbers[i];
     if (next > max){
       max = next; 
     }
   }
  
   return max;
  
}

The most important detail here is to externalize the code with the 'extern "C"' part.

There's a useful tool called objdump you can use to inspect the resulting library in order to find out which functions are accesible from outside.

By the way, the c++ project is compiled using Gradle cpp plugin;)

Second step: Invoke your code!

Then I wrote the following code in the groovy console:

@Grab( 'net.java.dev.jna:jna:3.4.0' )
import com.sun.jna.Library
import com.sun.jna.Native
 
System.setProperty("jna.library.path","/pathToTheLibraryDir")
 
interface Algorithms extends Library {
    public int getMax(int[] numbers,int numbersLength);
}

def instance = Native.loadLibrary( 'cpp',Algorithms)
def numbers = [1,2,3,4] as int[]
instance.getMax(numbers,numbers.size())

Thinks to take into account.

• The jna property is one of the three ways of telling JNA where the C++ library is. You can check the other two at the Github JNA page.
• You can invoke the script without specifying the location of the library if you executed groovyConsole in the same dir.

Thanks to the people of MadridJUG mailing list. They saved me a lot of time.

References

• Groovy entry about using JNA: http://blog.bloidonia.com/post/26134471586/using-jna-with-groovy
• JNA page at GitHub: https://github.com/twall/jna

Grails pitfalls: Don't do flush:true when you actually want a transaction (and II)

Today somebody at work ask me about some statements I made in the previous entry "Grails pitfalls: Don't do flush:true when you actually want a transaction".

He thought that when doing

company.save(flush:true)

It didn't matter it was within or outside a transaction, the transaction will eventually commit the transaction event tough there were an exception on the way.

I didn't know that either until I worked in a project where I needed to return the number of persisted items just after persisting a new one but just before doing some critical calculation that could raise an exception and eventually rollback the entire transaction.

Think about it, if the domain wasn't persisted and "committed" (notice the quotes) I couldn't count the right number of persisted entities, but What if the subsequent calculation failed? I had already saved the domain class even tough it was incorrect to do so. It was a dead alley until I had a conversation with a senior engineer. More or less it was like the following:

Mario: Ciaran I can't save the instance and see it in the following line
Ciaran: Then flush:true the save statement
Mario: What? But save(flush:true) commits the statement.
Ciaran: Nop, it makes the instructions visible for the rest of the transaction
Mario: Ummm, I don't quite follow you but let's test it anyway.

That was the most difficult task, How to test that assertion?

Let's say we have the following service method:

    def saveAndFail(Company company) {
      company.user = springSecurityService.currentUser
      company.save(flush:true)
      throw new IllegalArgumentException("Ahhhh")
    }

I did the following test:

   package whatever

   import static org.junit.Assert.assertThat
   import static org.hamcrest.CoreMatchers.notNullValue
   import static org.hamcrest.CoreMatchers.is
     
   import grails.plugin.spock.IntegrationSpec
   import org.codehaus.groovy.grails.plugins.springsecurity.SpringSecurityUtils
         
  class CompanyServiceFlushIntegrationSpec extends IntegrationSpec{
       
    def companyService
         
    def "Test flush:true"(){ 
      setup: "Building a valid company instance"
        def user = validUser.save(failOnError:true,flush:true)
        def company = new Company(
          user: user,
          name: "name",
          companyCode : "companyCode"
        )
      when:"Saving a valid company instance"
          def savedCompany = SpringSecurityUtils.doWithAuth("username"){
              companyService.saveAndFail(company)
          }
      then: "The result should be the saved company"
        thrown(Exception)
        assertThat(Company.count(),is(0))
    }
        
    def getValidUser(){
      def validUser = new ApplicationUser(
        name: "name",
        username: "username",
        password: "password",
        locale: Locale.getDefault()
      ) 
    }       
        
 }

But it didn't work. I got:

Failure:  Test flush:true(whatever.CompanyServiceFlushIntegrationSpec)
|  java.lang.AssertionError: 
Expected: is <0>
     got: <1>

Let's analyze how the test worked. First, we should know that by default integration tests are transactional by default. That means that at the end of every test method if there is any transaction it will be rolled back....AT THE END. In our method we threw an exception "catched" by our thrown(Exception) statement. So there was no rollback there, so it makes sense that at line number 28 the assertion fails because the exception has not been propagated outside the test method, so that the count statement is aware of the persisted company because of the flush:true statement.

But I want to prove that the transaction actually does a rollback because of the exception we put on purpose. We can only do that establishing our own transaction scope. To do this is neccesary to declare the test as "not transactional", and then to wrap the service's invocation as it will happen at runtime.

   package whatever

   import static org.junit.Assert.assertThat
   import static org.hamcrest.CoreMatchers.notNullValue
   import static org.hamcrest.CoreMatchers.is
     
   import grails.plugin.spock.IntegrationSpec
   import org.codehaus.groovy.grails.plugins.springsecurity.SpringSecurityUtils
         
  class CompanyServiceFlushIntegrationSpec extends IntegrationSpec{
       
    static transactional = false

    def companyService
         
    def "Test flush:true"(){ 
      setup: "Building a valid company instance"
        def user = validUser.save(failOnError:true,flush:true)
        def company = new Company(
          user: user,
          name: "name",
          companyCode : "companyCode"
        )
      when:"Saving a valid company instance"
          Company.withTransaction{
            def savedCompany = SpringSecurityUtils.doWithAuth("username"){
              companyService.saveAndFail(company)
            }
          }
      then: "The result should be the saved company"
        thrown(Exception)
        assertThat(Company.count(),is(0))
    }
        
    def getValidUser(){
      def validUser = new ApplicationUser(
        name: "name",
        username: "username",
        password: "password",
        locale: Locale.getDefault()
      ) 
    }       
        
 }

And voila!

| Tests PASSED - view reports in...

I had no companies saved even though I used flush:true.

Clojure & Groovy: Exposing classes and methods

I'm trying to learn a new language: Clojure. It's really different than anything I've faced before...and I like it. But eventually I'd like to use it in some other apps coded in some other JVM languages such as Groovy ;)

In order to do that I'm using a Gradle project having different modules written in different JVM languages (Java, Scala, Clojure), and a Groovy module with Spock specs to test them all.

So I came up with the following Clojure file:

 (ns polyglot.clojure.sample
     (:gen-class              
      :name polyglot.clojure.sample.ClojureUtils
      :methods [
         [lowerit [String] String]
         #^{:static true} [upperit [String] String] 
       ]                      
     )
     (:require [clojure.string :as st])
  )   
        
  (defn -lowerit [this message]
    (st/lower-case message))                                                                                                                                
  
  (defn -upperit [message]   
    (st/upper-case message))

And then I was able to test it with the following Spock specification:

package polyglot
       
   import polyglot.clojure.sample.ClojureUtils
   import spock.lang.Specification
   
   class UseClojureGromGroovySpec extends Specification{
   
     def "Getting the message using cars"(){
       setup: "Creating a car"
        def car = new Car(brand:"Seat",model:"Leon")
        def util = new ClojureUtils()   
      when: "Initializing Clojure"
        def instanceMessage = util.lowerit(car.brand)
        def staticMessage = ClojureUtils.upperit(car.brand) 
      then: "The message should be like the following"
        instanceMessage == "seat"                                                                                                                           
        staticMessage == "SEAT"
    }
  }

What I have learnt so far is:

•  File location:

(ns polyglot.clojure.sample

Points out to the clojure file. In this sample the file was /polyglot/clojure/sample.clj

• Class name:

:name polyglot.clojure.sample.ClojureUtils

To be able to tell Clojure which Class to create, you have to specify the whole path, the "qualified name" so to speak. It's a little bit annoying to repeat the package when it could have been guessed from the namespace (ns attribute). But maybe I'm wrong and it's just that I don't know how to do it yet. Following the docs:

"The package-qualified name of the class to be generated"

• Specifying instance methods:

:methods [
         [lowerit [String] String]
       ] 

This line exposes the method lowerit, method having an String as parameter. It should return an String as well. The method is implemented as:

(defn -lowerit [this message]
    (st/lower-case message))      

Notice that instance and static methods, both are implemented with an "-" symbol. Following the Clojure documentation :

"...Given a generated class org.mydomain.MyClass with a method named mymethod, gen-class will generate an implementation that looks for a function named by (str prefix mymethod) (default prefix: "-")..."

• Specifying static methods:

:methods [         
         #^{:static true} [upperit [String] String] 
       ] 

The way of exposing static methods has a different syntax in the methods: block, but same way of implementing the method.

  (defn -upperit [message]   
    (st/upper-case message))

• Importing third party libraries:

Because I first started using Clojure with the REPL I forgot some assumptions, such as REPL imports some namespaces by default. So I first try to implement lowerit as:

(defn -lowerit [this message]
    (clojure.string/lower-case message))  

That failed because the compiler could find the String type and even less the lower-case method. So after some search I found out how to do it. Before trying to use the String class I had to import it in the required: block (You can give the library an alias in order to use it through your implementation).

(:require [clojure.string :as st])

References:

• Polyglot Gradle Project (Github): I built it to know how to mix different JVM languages.
• Clojuredocs: A good Clojure reference site

Grails pitfalls: Controllers and how to reuse code

There're a lot of common repetitive tasks that arises when coding controllers, services...etc. I've seen many times repeating code that has to do with pagination.

def list() {
    params.max = params.max ?: grailsApplication.config.list.default.maxElements
  
    def result = GeographicArea.findAll(params)        
    render view: 'list', model: [list:result]
}

It's really painful to repeat the code to establish default maximum or minimum number of results every time we have to deal with some pagination. It's easier to create a method maybe in some parent class, to be able to call it from those classes inheriting that class. We can use inheritance, mixins, AST...etc.

The pros of using inheritance is you have the behavior because of compilation (AST would be the same here) while doing Mixins would have a performance penalty.

Many times my favorite choice is to inherit a BaseController where I include most of the boilerplate code.

class BaseController{
  
 def configurationService

    def getPaginationParams(){
      if (!params.max){
        params.max = configurationService.defaulMaximumListElements()

        //params.min...etc
      }
      params
    }
  
  }

And then I make the child class to extend the BaseController class

class GeographicAreaController extends BaseController{  
    def list() {    
       def result = GeographicArea.findAll(paginationParams)        
       render view: 'list', model: [list:result]
      }   
}

Grails pitfalls: Inheriting a parent class doesn't mean a new table

Sometimes we need to audit our domain classes. And sometimes you open a couple of classes and realize auditing fields have been copied in every class:

  class Product{
     Date dateCreated
     Date lastUpdated
  }

  class Category{
     Date dateCreated
     Date lastUpdated
  }

As you may imagine, this could improve a lot. But most of newbies don't want to create a new class and make the previous classes inherit from it because they think they will be creating a new table.

But... Did you know you can inherit from a class without creating a new table?

The solution is really simple, the parent class should be in src/groovy and it should be an abstract class. Then you can extend that class. The child class will have all the parent's fields but without the penalty of having a new table.

  // src/main/groovy/myapp/BaseEntity.groovy
  abstract class BaseEntity{
     Date dateCreated
     Date lastUpdated
  }

  // grails-app/domain/myapp/Product.groovy
  class Product extends BaseEntity { }

  // grails-app/domain/myapp/Category.groovy
  class Category extends BaseEntity{ }

Grails pitfalls: Don't do flush:true when you actually want a transaction

Putting persistence code in controllers leads to poor maintenance, reduces the chance of reusing code, and makes controllers extremely hard to test.

One of the first problems a junior programmer experience is to realize the domain class hasn't been saved when he/she expected.

After explaining to them the best way of approaching this type of situations is to put the persistence code in the service layer, they ignore you because they find they can force the commit doing:

  product.save(flush:true)

For what else would be this parameter for? (Sarcasm ;)) If you were only saving a given instance without any relationship...still I wouldn't do that. At least forget about the use of flush:true in that context and use withTransaction:

  Product.withTransaction{
    product.save()
  }

I can see a transaction going on here. Let's say we have two given transactions trying to do something at once.

  Product.withTransaction{
    product.status = ACTIVE
    product.save(flush:true) // without this line the total number will be all of them but this one
    Product.countByByStatus(ACTIVE)
  }

In the previous code without forcing flush:true we would have been omitting the product we were saving in our transaction. First time you see this code you could be thinking, OMG they are committing before the end of the transaction.

Sort answer would be: we ARE NOT committing anything yet, we are sending to the database all instructions until this point (flushing) allowing other transactions in their way to see the our transaction status. If something goes wrong before the transaction ends then changes won't persist.

The problem is that when there's no transaction boundaries the default behavior of flush:true is to commit changes. But if you do establish the boundaries, we would expect the commit/rollback to happen once we exit the transaction's scope.

UPDATE: Somebody at work asked me how to prove this statement. I've created an entry to explain this a little bit more.

Using flush:true the way people normally use it is not right, but is even funnier when they realize they can't do

product.delete(flush:true)

All these could have been avoided if people were used to put their persistence code in services. There are a couple of benefits when using services:

• You no longer have to worry about opening and closing a transaction. Convention over configuration.

A simplistic view of transactions in Grail's services would be that by default the code within a service's method is executed within a transaction. If the method throws an exception then that code will be rolled back. If everything went ok then all changes will be commited. Although you could but now you don't have to declare explicitly the boundaries of the transaction.

class ProductService{

   def addProductToCart(Product p, Cart c){ // transaction starts here

      //... code within a transaction

   } // transaction ends here (if everything went ok ;) )

}

EXPLANATORY UPDATE:

To be accurate the transaction begins not at the beginning of the method but at the service's invocation time. What does it means? It means that when you're invoking a service's method let's say from the controller, a Spring interceptor, intercepts the call and wraps that call inside a given transaction scope.

One more thing to keep in mind is that the default propagation of transactions is REQUIRED, that means that if we're calling a transactional service's method from another transactional method the latter uses the former transaction scope, or in cause there were none it creates a new one. That explains why we can call some other service methods from our service method without committing previous statements, because there were a pre-existent transaction.

You can always change the default propagation policy using the @Transactional annotation from Spring in your service methods, or using Spring AOP advices to configure transactional scopes.

Anyway if you want to dive into this topic please read "Spring Declarative Transaction Management".

• It keeps separated your view logic (controllers) from your persistence code (services)

It's clear that less code is always easier to review. Hence if you could keep separated view logic from business logic everything starts looking clearer.  Let's see an example:

If you have the following code:

class ContractController{

  def saveContract(Contract contract,Address address){

    if(contract.validate()){       

       contract.deliveryAddress = address.save(flush:true)   
       contract.save(flush:true,failOnError:true)  

       render(view:'ok')    

    } else {

       render(view:'ko')

    }

  }

}

If you had to unit test this code you should have to mock every domain class behavior (Contract and Address), and sometimes that's really a pain in the ass. Besides the fact that the address could have been saved without succeeding on doing the same with the contract instance. That type of inconsistencies are really dangerous.

Wouldn't be better to split the code like this:

class ContractController{

  def contractService 

  def saveContract(Contract contract,Address address){

    if(c.validate()){   

       contractService.saveContract(contract,address)
       render(view:'ok')    

    } else {

       render(view:'ko')

    }

  }

}

class ContractService{

   def saveContract(Contract c,Address address){

       c.deliveryAddress = address.save()
       c.save(failOnError:true) 

   }

}

Now is easier to mock the service's method, and you only have to focus on testing the view logic.

Grails Tip: Different URL depending on user's role

"We need to redirect users depending on their roles". OMG how I was supposed to do this? Well after doing some search on the usual site, I found this:

http://omarello.com/2011/09/grails-custom-target-urls-after-login/

My solution is based on the previous blog entry. The only thing I've added is using Config.groovy to register different ROLE/URL cases. I thought it worthed sharing it.

Let's say we want to redirect users with role ROLE_MANAGER  to /users/search and users with role ROLE_PROVIDER to /invoices/search. So I came up with the following lines in my Config.groovy

authenticationurl{
 mappings{
  ROLE_MANAGER.controller='users'
  ROLE_MANAGER.action='search'
  ROLE_PROVIDER.controller='invoices'
  ROLE_PROVIDER.action='search'
 }
}

The nice thing about configuration files in Grails is that you can build maps just sharing the same root, in other words, I can ask which controller and action should be used for a manager accessing to the key ROLE_MANAGER:

def map = authenticationurl.mappings.ROLE_MANAGER
def controller = map.controller // 'users'
def action = map.action // 'search'

Taking into account that premise, I built my own AuthenticationHandler. This handler takes the authenticated user's role and use it as key to look for its correspondent controller/action:

import org.codehaus.groovy.grails.plugins.springsecurity.AjaxAwareAuthenticationSuccessHandler
import org.springframework.security.core.Authentication

import org.apache.commons.logging.LogFactory

import javax.servlet.http.HttpServletRequest
import javax.servlet.http.HttpServletResponse

import static org.codehaus.groovy.grails.plugins.springsecurity.SpringSecurityUtils.ifAllGranted


class AuthenticationHandler extends AjaxAwareAuthenticationSuccessHandler {

 static log = LogFactory.getLog(AuthenticationHandler)

 def grailsLinkGenerator 
 def configurationService
 def springSecurityService

 @Override
 protected String determineTargetUrl(HttpServletRequest request,HttpServletResponse response) {
  /* Taking configuration mappings */ 
  def authMappings = configurationService.authenticationHandlerUrlMappings
  def defaultUrl = super.determineTargetUrl(request, response)
  if(log.isDebugEnabled()){
   log.debug("Default URL: $defaultUrl")
  }
  /* Taking the first match or the targetUrl  */
  return authMappings.findResult{k,v->
   if(ifAllGranted(k)){
    /* Here's better to get it absolute */
    grailsLinkGenerator.link(v << [absolute:true])
   }
  } ?: defaultUrl 
 }

}

The line with the authMapping variable assignment is doing grailsApplication.config. authenticationurl.mappings underneath. The value is itself a map of maps.

The method determineTargetUrl loops through the map looking for the first result matching the user's role. If there's an entry the user will be redirected to the configured URL otherwise the default URL will be used (normally it is the same URL where the user was when he tried to authenticate himself).

Notice that I'm using the service grailsLinkGenerator to generate the proper link passing the controller and action arguments to it. This works the same as if we were using the tag in any gsp. This service is available by default in Grails.

Ah of course don't forget to register your authentication handler in the resources.groovy file

authenticationSuccessHandler(AuthenticationHandler) {
        /* Reusing the security configuration */
        def conf = SpringSecurityUtils.securityConfig
        /* Configuring the bean */
        requestCache = ref('requestCache')
        redirectStrategy = ref('redirectStrategy')
        springSecurityService = ref('springSecurityService')
  grailsLinkGenerator = ref('grailsLinkGenerator')
  configurationService = ref('configurationService')
        defaultTargetUrl = conf.successHandler.defaultTargetUrl
        alwaysUseDefaultTargetUrl = conf.successHandler.alwaysUseDefault
        targetUrlParameter = conf.successHandler.targetUrlParameter
        ajaxSuccessUrl = conf.successHandler.ajaxSuccessUrl
        useReferer = conf.successHandler.useReferer
    }

Some ideas about processing a text file with Groovy

I really hated Java when reading a file. At least when using BufferedReader you could always abstract reading every line with readLine(), and now with JDK7 you can even forget about taking care of closing the stream (related article at DZone), great!! But...Can Groovy do better? I hope so :P

A couple of days ago I was looking into Groovy's java.util.File JDK documentation trying to find out the easiest way to iterate through a text file collecting information on the way.

I started doing my research with the following data:

JANUARY     100.23
FEBRUARY    23.34
MARCH       45.56
APRIL       67
MAY         78.2
JUNE        23.3
JULY        92.2
AUGUST      802.2
SEPTEMBER   87.3
OCTOBER     2.2
NOVEMBER    3.2
DECEMBER    150.4

The research

Firstly I started looping through every line in the file using the Java approach:

def file2Process = new File("/pathtofile/file.txt")
def acc = 0
def javaReader = file2Process.newReader()
while((next = javaReader.readLine()) != null){     
    acc += next.split(/\s{1,}/)?.getAt(1)?.toDouble()?:0
}

javaReader.close()
println "Java (1)->$acc"

Not bad, BufferedReader is a nice abstraction but the while statement has too many things, a declaration, assignation, and a condition altogether.

Afterwards, I don't know where it came from, I tried to iterate the reader returned from newReader() method (added by Groovy to the File class). As you may guess It looked nicer:

def reader = file2Process.newReader()
def perLine = {line-> line.split(/\s{1,}/)?.getAt(1)?.toDouble()?:0 }

def totalAmount = reader.collect(perLine).sum()

reader.close()

println "Groovy (1)->$totalAmount"

So I guess all readers are iterable, Aren't they? I don't remember they were in Java, or maybe I just forgot it. But still I had to close the reader explicitly. Common! JDK7 already do that, you can do it better! Don't you?

Yes, and it was my fault, I didn't notice about the withXXX methods added to the java.util.File class in Groovy. These methods receive a closure as parameter. Inside that closure you can use the underlying reader or stream and at the end of the closure the method takes care about closing the reader/stream.

def perLine = {line-> line.split(/\s{1,}/)?.getAt(1)?.toDouble()?:0 }
def result = file2Process.withReader{r->
    r.collect(perLine).sum()
}

println "Groovy (2)->$result"

The good thing about this is that you can still use it with JDK6 (However I encourage you to move to JDK7).

The "real" mission

The task that I needed to do was to process all lines of a text file gathering different chunks of information.

Also motivated by the rant around "functional vs OOP" I wanted to process the file in a "functional" way, which means not to use temporary variables outside the scope of the closure (aka no side effects).

That's why I found really impressive the idea that readers could be iterable. If readers were iterable I would be able to use collection methods like "inject" to populate a map with different types of information collected along the file.

Of course the file I'm showing here has little to do with the real one, with thousands of lines (and dozens of fields) coming in from a legacy Cobol system, but it works for the shake of the explanation.

So lets say I wanted to return a map with different values

• Total number of lines
• Total amount
• Total amount by quarter

I came up with this solution:

def firstQ = ['JAN','FEB','MAR','APRIL']
def secondQ = ['MAY','JUN','JUL','AUG']
def thirdQ = ['SEP','OCT','NOV','DEC']

def data = file2Process.withReader{r->
    r.inject([q1:0,q2:0,q3:0,lines:0,total:0]){map,val->
        def lineInfo = val.split(/\s{1,}/)
        def month = lineInfo?.getAt(0)?.take(3)
        def amount = lineInfo?.getAt(1)?.toBigDecimal()
        switch(month){
            case firstQ:
               map.q1 += amount
            break
            case secondQ:
               map.q2 += amount
            break
            case thirdQ:
               map.q3 += amount
            break
        }
        map.total += amount
        map.lines++
     /* Don't forget to return the map */
        map
    }
}

Which returns the map:

[q1:169.13, q2:995.9, q3:243.1, lines:12, total:1475.13]

One complaint

• Because in the real file I had to do some validations that involved the use of financial algorithms, I missed Gpars for that. But I wasn't able to make Gpars' foldParallel(..) method to work the same way as the inject(...) method does. I've sent the question to the mailing list and I'll update the entry as soon as I get an answer.

Resources

• Groovy Input Output: http://groovy.codehaus.org/Input+Output
• Groovy Streams: http://groovy.codehaus.org/JN2025-Streams

Take a look at how to transform a given file's lines with transformLine(...) method, it sprang to mind that maybe if I had to pre-process a given file I'd require that type of behavior sometime.

Groovy Xml Series: JAXB

Although the Groovy way of dealing with XML is simply great, sometimes it's nice to use Jaxb for things such as schema validation or maybe for establishing what fields are involved in the process and what others are not.

This is not going to be an entry of JAXB specially when there are hundreds of blog entries and even books covering the topic widely.

I'm going to create a very simple example to unmarshall a given object instance to XML. So here we have the XML source

                                                                                                            
                       
                       
                   
                          
                   Manuel De Cervantes
                       
                   
                   
                  JD Salinger
                      
                  
                  
                  Lewis Carroll
              
                   
                         
                  Manuel De Cervantes
                      
                     
                     
  

Before doing anything, just remind you that the code is available at Github.

And now we should be creating the classes which are going to map the XML fragments in which I'm interested.

• Author

package github.groovy.xml.jaxb                                                                                                          
                              
   import javax.xml.bind.annotation.*
   
   @XmlRootElement(name="author")
   @XmlAccessorType(XmlAccessType.FIELD)
   class Author{              
   
       @XmlAttribute Long id  
      @XmlElement String name
        
 } 

• Book

 package github.groovy.xml.jaxb                                                                                                          
                              
   import javax.xml.bind.annotation.*
   
   @XmlRootElement(name="book")
   @XmlAccessorType(XmlAccessType.FIELD)
   class Book{                
   
       @XmlAttribute Long id  
      @XmlElement String title
      @XmlElement Author author
          
 }

Basically I'm using the following annotations:

• @XmlRootElement: for be able to marshal the object without having to include it in a another object having this annotation. Only objects annotated with this annotation can be marshaled directly.
• @XmlAccessorType(XmlAccessType.FIELD): Because we're using Groovy, and there is "a lot of magic" at runtime you should tell JAXB to stick to the fields, otherwise it will try to unmarshall crazy things. 
• @XmlElement: To tell JAXB what's going to be a tag element
• @XmlAttribute: To tell JAXB what's going to be an attribute element

Unmarshalling (XML --> Object)

Well let's say we have an XML full of books and authors and we want to unmarshal them into objects. If we wanted to unmarshall the XML above to get only books and authors, we should pre-process the XML to get rid of the response/data tags. And only then unmarshal the remaining XML.

import javax.xml.bind.*

//...Omitted code

      def "Unmarshalling the first book with boilerplate code"(){
          setup: "Building the unmarshaller"
              def jaxbContext= JAXBContext.newInstance(Book)
              def unmarshaller = jaxbContext.createUnmarshaller()
          and: "Filtering the xml"
              def response = new XmlSlurper().parse(xmlFile) 
          and: "Getting only the first book"
              def firstBook = response.'**'.find{it.name() =='book'}
              def jaxbSource= XmlUtil.serialize(firstBook)
              def newXmlSource = new StringReader(jaxbSource)
          when: "Unmarshalling the source" 
              def jaxbBook = unmarshaller.unmarshal(newXmlSource)
          then: "We make sure the conversion took place"
              jaxbBook instanceof Book        
          and: "Checking Book properties"
              jaxbBook.title 
              jaxbBook.author
              jaxbBook.author.id
      }

I've created an utility class to avoid most of the repeated code when creating a Marshaller/Unmarshaller:

package github.groovy.xml.jaxb

import javax.xml.bind.*
import github.groovy.xml.util.ResourcesUtil

/**
 * This class helps us to handle marshalling and unmarshalling of JAXB objects
**/
class JaxbUtils {

 def object2MarshalType
 def source2Unmarshall 
 
 def marshal(object){
  object2MarshalType = object.getClass()
  this
 }

 def to(File file){
  throwIfNull(object2Marshal)
  throwIfNull(file)  

  buildMarshaller(object2MarshalType).marshal(object2MarshalType,file)
 }

 def unmarshal(source){
  source2Unmarshall = source 
  this
 }

 def to(Class anyType){
  throwIfNull(source2Unmarshall)
  throwIfNull(anyType)

  buildUnmarshaller(anyType).unmarshal(source2Unmarshall)
 }

 def buildMarshaller(Class type){
  def jaxbContext= JAXBContext.newInstance(type)
  def marshaller = jaxbContext.createMarshaller()

  marshaller
 }

 def buildUnmarshaller(Class type){
  def jaxbContext= JAXBContext.newInstance(type)
  def unmarshaller = jaxbContext.createUnmarshaller()

  unmarshaller
 }


 def throwIfNull(value,message="You should have provided any value"){
  if (!value){
   throw new Exception(message)
  }
 }

}

So the previous code becomes:

      def "Unmarshalling the first book the good way"(){
          setup: "Parsing the document"
              def response = new XmlSlurper().parse(xmlFile)
          and: "Getting only the first book"
              def firstBook = response.'**'.find{it.name() =='book'}
              def jaxbSource= XmlUtil.serialize(firstBook)
          when: "Convert to Jaxb object"
              def newXmlSource = new StringReader(jaxbSource)
              def jaxbBook = unmarshal(newXmlSource).to(Book)
          then: "We make sure the conversion took place"
              jaxbBook instanceof Book
          and: "Checking Book properties"
              jaxbBook.title
              jaxbBook.author
              jaxbBook.author.id
      }

Marshalling (Object --> XML)

I was a little bit lazy and I didn't build any example for marshalling an object to XML. Maybe I will update this entry later on with a working example.

Resources

• JAXB Official Page: http://jaxb.java.net 
• I found this entry using Groovy, JAX-WS and JAXB altogether, highly advisable: http://weblogs.java.net/blog/lamineba/archive/2012/05/20/building-restful-web-services-jax-rs-jaxb-and-groovy

Groovy Xml Series: Index

So far I've written a couple of entries about Groovy and Xml and although I've taken care of setting the tags in order to be able to find the entries easily, but I thought it would be better to have an index with all the entries altogether.

• Groovy Xml Series: Parsing Xml
• Groovy Xml Series: Querying Xml with GPath
• Groovy Xml Series: Printing Xml
• Groovy Xml Series: Manipulating Xml
• Groovy Xml Series: Templating

This way if I added a new related entry there would be a common place to look for this topic.

Groovy Xml Series: Templating

I guess in many web applications there're times when you are going to return the same XML estructure with little changes every time. So you would be looking for a general template with some placeholders to substitute at runtime.

How to this with Groovy: groovy.text.XmlTemplateEngine to the rescue :-)

The following example's been inspired in an entry of MrHaki talking about the topic. The first we have to build is the template.

Template

     def tpl = ''' 
          
              
                  
                      books.eachWithIndex{book,index->
                                      
                               
                                  
                                        
                                   
                                  book.author
                              
                          
                      }
                  
              
          
      '''

Several things to take into account when creating the template:

• Notice that I've used single quotes. If I had used double quotes I'd have been in trouble at runtime because Groovy would have tried to substitute the placeholders beforehand.
• Adding "gsp" namespace: Adding this namespace we'll be able to use scriptlet and expression tags. I'll explain both later.
• Groovy expressions: Like if we were using GString expressions we can use expressions like ${index}. Again notice all variables are evaluated differently than GString instances.

Scriptlets

Basically we're going to use the tag when trying to build complex estructures or looping through a  set of values. In the example:

    books.eachWithIndex{book,index->
        // Nested code
    }

Here we were looping through a list of books. Notice how we're going to be able to access to the variables created in that snippet inside the scope of the tag.

Expressions

Following the documentation the tag should be used for code fragments which produce output. In this example we want to add the book's author as the text value of the author's node.

              
       book.author
    

Groovy Expressions

We can be using the expressions ${expression} and $variable to add expressions to the document.

Invoking XmlTemplateEngine

Finally we should put all pieces together and set all variables in the template. For that we will be invoking the XmlTemplateEngine instance and passing a map with the variables we want to substitute in the template.

This is a method taken from a Spock specification created for testing the XmlTemplateEngine functionality.

     def "Creating a new xml using the template and some bindings"(){
          setup: "Building some data"
              def books = (1..10).collect{
                  [title:"Book${it}",author:"Author${it}"]
              }
          when: "Creating the engine instance and compiling the template"
              def engine = new XmlTemplateEngine()
              def templ = engine.createTemplate(tpl)
          and: "Binding the values with the template"
              def bindings = [books:books]
          and: "Parsing the template with the included bindings"
              def writable = templ.make(bindings)
           /* I want to see the output in the test report */
              println writable
          and: "Parsing the result to check the outcoming xml"
              def response = new XmlSlurper().parseText(writable.toString())
          then: "We should have a document with 10 books"
              response.value.books.book.size() == 10
      }

See how first we create a template instance from the XmlTemplateEngine instance.

def engine = new XmlTemplateEngine()
     def templ = engine.createTemplate(tpl)

Then we pass the books as a parameter to the template. That will give us a writable instance.

      def bindings = [books:books]
      def writable = templ.make(bindings)

At the end we only parse the document again for testing purposes.

      def response = new XmlSlurper().parseText(writable.toString())
      response.value.books.book.size() == 10

Resources

• Entry Github location: https://github.com/mariogarcia/xmlgroovy
• Groovy Templating: http://groovy.codehaus.org/Groovy+Templates
• XmlTemplateEngine: http://groovy.codehaus.org/api/groovy/text/XmlTemplateEngine.html
• Mr Haki Sample: http://mrhaki.blogspot.ie/2009/10/groovy-goodness-using-template-engines.html 

Groovy Xml Series: Manipulating Xml

The Xml

In this entry I would like to review the different ways of adding / modifying / removing nodes using XmlSlurper or XmlParser. The xml we are going to be handling is the following:

      def xml = """                                                                                                                       
          
              
                  
                      
                          
                          Manuel De Cervantes
                      
                  
              
          
      """

Adding nodes

The main difference between XmlSlurper and XmlParser is that when former creates the nodes they won't be available until the document's been evaluated again, so you should parse the transformed document again in order to be able to see the new nodes. So keep that in mind when choosing any of both approaches.

If you needed to see a node right after creating it then XmlParser should be your choice, but if you're planning to do many changes to the XML and send the result to another process maybe XmlSlurper would be more efficient.

You can't create a new node directly using the XmlSlurper instance, but you can with XmlParser. The way of creating a new node from XmlParser is through its method createNode(..)

        def "Adding a new tag to a node"(){
          setup: "Building an instance of XmlParser"
              def parser = new XmlParser()
          and: "Parsing the xml"
              def response = parser.parseText(xml)
          when: "Adding a tag to response"
              def numberOfResults = parser.createNode(
                  response,
                  new QName("numberOfResults"),
                  [:]
              )   
          and: "Setting the node's value"
              numberOfResults.value = "1"
          then: "We should be able to find the new node"
              response.numberOfResults.text() == "1"
      }

The createNode() method receives the following parameters:

• parent node (could be null)
• The qualified name for the tag (In this case we only use the local part without any namespace)
• A map with the tag's attributes (None in this particular case)

Anyway you won't normally be creating a node from the parser instance but from the parsed Xml instance. That is from a Node or a GPathResult instance.

Take a look at the next example. We are parsing the xml with XmlParser and then creating a new node from the parsed document's instance (Notice the method here is slightly different in the way it receives the parameters):

def "Adding a new tag to a node with the node instance"(){
          setup: "Building an instance of XmlParser"
              def parser = new XmlParser()
          and: "Parsing the xml"
              def response = parser.parseText(xml)
          when: "Appending the tag to the current node"
              response.appendNode(
                  new QName("numberOfResults"),
                  [:],
                  "1"
              )
          then: "We should be able to find it"    
              response.numberOfResults.text() == "1"
      }

When using XmlSlurper GPathResult instances don't have createNode() method.

Modifying / Removing nodes

We know how to parse the document, add new nodes, now I want to change a given node's content. Let's start using XmlParser and Node. This example changes the first book information to actually another book.

     
     def "Replacing a node"(){
          setup: "Building the parser and parsing the xml"
              def response = new XmlParser().parseText(xml)
          when: "Replacing the book 'Don Xijote' with 'To Kill a Mockingbird'"
           /* Use the same syntax as groovy.xml.MarkupBuilder */
              response.value.books.book[0].replaceNode{
                  book(id:"3"){
                      title("To Kill a Mockingbird")  
                      author(id:"3","Harper Lee")     
                  }
              }
          and: "Looking for the new node"
              def newNode = response.value.books.book[0]
          then: "Checking the result"
              newNode.name() == "book"        
              newNode.@id == "3"
              newNode.title.text() == "To Kill a Mockingbird"
              newNode.author.text() == "Harper Lee"
           /* Don't know why I have to look for the first id */
              newNode.author.@id.first() == "3"
      }

When using replaceNode() the closure we pass as parameter should follow the same rules as if we were using groovy.xml.MarkupBuilder (See resources section for more information):

tagName(attribute:attributeValue){
    nestedTag("stringcontent")
  /// etc
}

Here the same example with XmlSlurper:

def "Replacing a node"(){
          setup: "Parsing the document"
              def response = new XmlSlurper().parseText(xml) 
          when: "Replacing the book 'Don Xijote' with 'To Kill a Mockingbird'"
           /* Use the same syntax as groovy.xml.MarkupBuilder */
              response.value.books.book[0].replaceNode{
                  book(id:"3"){
                      title("To Kill a Mockingbird")  
                      author(id:"3","Harper Lee")     
                  }          
              }              
          and: "Asserting the lazyness"
              assert response.value.books.book[0].title.text() == "Don Xijote"
          and: "Rebuild the document"
           /* That mkp is a special namespace used to escape away from the normal building mode 
              of the builder and get access to helper markup methods 
              'yield', 'pi', 'comment', 'out', 'namespaces', 'xmlDeclaration' and 
              'yieldUnescaped' */
              def result = new StreamingMarkupBuilder().bind{mkp.yield response}.toString()
              def changedResponse = new XmlSlurper().parseText(result)
          then: "Looking for the new node"
              assert changedResponse.value.books.book[0].title.text() == "To Kill a Mockingbird"
      }

Notice how using XmlSlurper we have to parse the transformed document again in order to find the created nodes. In this particular example could be a little bit annoying isn't it?

Finally both parsers also use the same approach for adding a new attribute to a given attribute. This time again the difference is whether you want the new nodes to be available right away or not. First XmlParser:

     def "Adding a new attribute to a node"(){
          setup: "Building an instance of XmlParser"
              def parser = new XmlParser()    
          and: "Parsing the xml"
              def response = parser.parseText(xml)
          when: "Adding an attribute to response"
              response.@numberOfResults = "1" 
          then: "We should be able to see the new attribute"
              response.@numberOfResults == "1"
      }

And XmlSlurper:

def "Adding a new attribute to a node"(){
          setup: "Parsing the document"
              def response = new XmlSlurper().parseText(xml)
          when: "adding a new attribute to response"
              response.@numberOfResults = "2"
          then: "In attributes the node is accesible right away"
              response.@numberOfResults == "2"
      }

But, hold on a second! The XmlSlurper example didn't need to evaluate again the transformation did it? You're right. When adding a new attribute doing a new evaluation is not necessary either way.

Resources

• Code at Github: https://github.com/mariogarcia/xmlgroovy
• Using MarkupBuilder: http://groovy.codehaus.org/Creating+XML+using+Groovy's+MarkupBuilder 
• Groovy processing XML: http://groovy.codehaus.org/Processing+XML

Groovy Xml Series: Printing Xml

I started reviewing my knowledge of Groovy's xml capabilities due to an entry in the Groovy user's mailing list.

Somebody asked about how to print a GPath query result. Because I knew about the GPath API, I suggested to start building the output from the available methods, I mean, if you had a Node / GPathResult (Depending on the parser you used to parse the xml: XmlParser/XmlSlurper respectively) representing

value

You could print that node with the following statement:

println "<${node.name()}>${node.text()}</{node.name()}>"

But luckily somebody mentioned we could be doing the same thing using groovy.xml.XmlUtil class. It has several static methods to serialize the xml fragment from several type of sources (Node,GPathResult,String...)

Thus I've added a sample to my xmlgroovy project at Github to remember how I can for example print out the result of a GPath result. This is a slightly changed version in order to execute it on the GroovyConsole:

       

import groovy.xml.XmlUtil

def xml = """
                                                                                                            
                       
                       
                   
                          
                   Manuel De Cervantes
                       
                   
                   
                  JD Salinger
                      
                  
                 
                  Lewis Carroll
                      
                   
                         
                  Manuel De Cervantes
              
          
                     
  
"""
 
   def response = new XmlParser().parseText(xml)
   def nodeToSerialize = response.'**'.find{it.name() == 'author'}
   def nodeAsText = XmlUtil.serialize(nodeToSerialize)

   println nodeAsText

In this example I'm parsing the xml and and printing out only the author's node I was interested in. And it looks like:

Manuel De Cervantes

Missing

• XmlNodePrinter : If you are using Node instances because you've parsed the document with XmlParser you can use XmlNodePrinter. You can check out the entry of MrHaki's blog where he covered the topic.

Groovy Xml Series: Querying Xml with GPath

The most common way of querying XML in Groovy is using GPath. The entry from the official page:

"GPath is a path expression language integrated into Groovy which allows parts of nested structured data to be identified. In this sense, it has similar aims and scope as XPath does for XML. The two main places where you use GPath expressions is when dealing with nested POJOs or when dealing with XML"

So it's similar to XPath expressions and you can use it not only with XML but also with POJO classes. Ok, so lets begin.

Given the following xml:

 
                                                                                                               
                       
                       
                   
                          
                   Manuel De Cervantes
                       
                   
                   
                  JD Salinger
                      
                  
                  
                  Lewis Carroll
                      
                   
                         
                  Manuel De Cervantes
              
          
                     
   

Node's text content

First thing we are going to do is to get a value using POJO's notation. Lets get the first book's author's name (Code is available at Github.).

      def "Using POJO notation: Getting a node using POJOs notation a.b.c"(){
          setup: "Parsing the document"
              def response = new XmlSlurper().parse(xmlFile) 
          when: "Trying to get a given node using the a.b.c notation"
              def authorNode = response.value.books.book[0].author
          then: "We can check the author's value"
              authorNode.text() == 'Manuel De Cervantes'
      }

So first we parse the document with XmlSlurper (The xmlFile is a variable of type java.io.File) and the we have to consider the returning value as the root of the XML document, so in this case is "response".

So that's why we start traversing the document from response and then value.books.book[0].author. Note that in XPath the node arrays starts in [1] instead of [0], but because GPath is Java-based it starts in [0] index.

GPathResult (XmlSlurper) and Node (XmlParser)

In the end we'll have the instance of the "author" node and because we wanted the text inside that node we are going to call the text() method.  The "author" node is an instance of GPathResult type and text() a method giving us the content of that node as a String.

When using GPath with an xml parsed with XmlSlurper we'll have as a result a GPathResult object. GPathResult has many other convenient methods to convert the text inside a node to any other type such as:

• toInteger()
• toFloat()
• toBigInteger()
• ...

All these methods try to convert an String to a certain type.

If we were using a XML parsed with XmlParser we could be dealing with instances of type Node. But still all the actions applied to GPathResult in these examples could be applied to a Node as well. Creators of both parsers took into account GPath compatibility.

Attribute's content

Next step is to get the some values from a given node's attribute. In the following sample we want to get the first book's author's id. We'll be using two different approaches. Let's see the code first:

         def "Using POJO notation: Getting an attribute's value using POJOs notation a.b.c"(){
          setup: "Parsing the document"
              def response = new XmlSlurper().parse(xmlFile) 
          when: "Trying to get a given node using the a.b.c notation"
              def firstBook = response.value.books.book[0]
              def firstAuthorIdNode1 = firstBook.author.@id
              def firstAuthorIdNode2 = firstBook.author['@id']
          then: "Getting the id's value"
              firstAuthorIdNode1.toInteger() == 1
              firstAuthorIdNode2.toInteger() == 1
      }

Again we first parse the document and then using the POJO's notation we get the first book node. Now take a look at the first expression:

• firstBook.author.@id
• firstBook.author['@id']

I specially like the former type of notation because is more straight forward, and meaningful.  The latter is more like using an instance of a map (which I guess it should be eventually).

Speeding things up: "breadfirst()" and "depthfirst()"

If you ever have used XPath you have been using the expressions like

• "//" : Look everywhere
• "/following-sibling::othernode" : Look for a node "othernode" in the same level

More or less we have their conterparts in Gpath with the methods breadfirst() and depthfirst(). The first example shows a simple use of breadfirst(). The creators of this methods created a shorter syntax for it using '*'.

        def "Using '*': Getting a node using breadthFirst operator '*'"(){                                                              
          setup: "Parsing the document"
              def response = new XmlSlurper().parse(xmlFile)
          when: "Looking for the node having the name 'book'"
          and: "with attribute id equals to 2"
           /* You can use the breadthFirst operator to look among a group 
              of nodes at the same level */
              def catcherInTheRye = response.value.books.'*'.find{node-> 
               /* node.@id == 2 could be expressed as node['@id'] == 2 */
                  node.name() == 'book' && node.@id == '2'
              }
          then: "Getting the author's value"
              catcherInTheRye.title.text() == 'Catcher in the Rye'
      }

This Spock specification looks for any node at the same level as "books" node first, and only if it couldn't find the node we were looking for then it will look deeper in the tree, always taking into account the given the expression inside the closure.

That expression says "Look for any node with a tag name equals 'book' and having an id with a value of '2'".

Today I woke up very lazy and I'd like to look for a given value without caring where it might be. The only thing I know is that I need the id of the author "Lewis Carroll" . How do I do that? using depthFirst()

        def "Using '**': Getting a node using depthFirst operator '**'"(){
          setup: "parsing the document"
              def response = new XmlSlurper().parse(xmlFile)                                                                              
          when: "Using the deptFirst operator we can look for something"
          and: "it doesn't matter how deep the node is"
          and: "Let's say we want to look for the book's id of the book written by Lewis Carrol"
           /* Beware of the name I used for the closure's parameter. It may look like 
              the ** is too smart, but it isn't. It's just that I'm sure only books will 
              match the query. To avoid any confusion I'd rather use 'node' */
              def bookId = response.'**'.find{book->
                  book.author.text() == 'Lewis Carroll'
              }.@id
          then: "The bookId should be 3"
              bookId == "3"
      }

Definitely is shorter that using the POJO notation isn't it? depthfirst() is the same as looking something "everywhere in the tree from this point down". In this case we've used the method find(Closure cl) to find just the first occurrence.

What if we want to collect all book's titles?

      
     def "Using '**': Collecting all titles"(){
          setup: "parsing the document"
              def response = new XmlSlurper().parse(xmlFile)
          when: "Looking for all titles within the document"
              def titles = response.'**'.findAll{node-> node.name() == 'title'}*.text()
          then: "There should be only four"
              titles.size() == 4
      }

I've mentioned there are some useful methods that convert a node's value to an integer,float...etc. Those methods could be convenient when doing comparisons like this:

      def "Using findAll: Collecting all titles"(){
          setup: "parsing the document"
              def response = new XmlSlurper().parse(xmlFile)
          when: "Looking for all titles with an id greater than 2"
              def titles = response.value.books.book.findAll{book->
               /* You can use toInteger() over the GPathResult object */
                  book.@id.toInteger() > 2
              }*.title
          then: "There should be only two"
              titles.size() == 2
      }

In this case the number 2 has been hardcoded but imagine that value could have come from any other source (Gorm id's...etc)

Resources

• Groovy GPath page: http://groovy.codehaus.org/GPath
• Github samples: https://github.com/mariogarcia/xmlgroovy