Fullmenu null

 

18 April 2018

GORM (Groovy o Grails?) Object Relational Mapping, http://gorm.grails.org/, is the library from Grails who allow us to access and manage our database. At the beginning it only worked with Hibernate but recently it works with Neo4J, NoSQL as MongoDB, etc
Neo4j is a graph database management system developed by Neo4j, Inc. Described by its developers as an ACID-compliant transactional database with native graph storage and processing, Neo4j is the most popular graph database according to DB-Engines ranking. (https://en.wikipedia.org/wiki/Neo4j)
The Panama Papers are 11.5 million leaked documents that detail financial and attorney–client information for more than 214,488 offshore entities. The documents, some dating back to the 1970s, were created by, and taken from, Panamanian law firm and corporate service provider Mossack Fonseca, and were leaked in 2015 by an anonymous source. (https://en.wikipedia.org/wiki/Panama_Papers)

In this example we’ll write a simple Groovy Script showing how to search into Panama Papers Neo4j database. Instead of write a semi-complex Cipher Query we’ll use GORM capabilities.

We will use the Docker image ryguyrg/neo4j-panama-papers instead of install a Neo4j database and populate it with the data, because it’s ready to use in only a few minutes (obviously you need Docker to run it):

docker run -p 7474:7474 -p 7687:7687 \      //(1)
        -d                          \       //(2)
        --name panama                \      //(3)
        -v $(pwd)/panama-papers-data:/data  //(4)
        ryguyrg/neo4j-panama-papers         //(5)
docker logs -f panama
  1. We need to link two ports, 7474 for browser and 7687 for bolt driver

  2. Launch in background

  3. name the container as panama (or whatever you want)

  4. to persist our changes in a local folder in case we remove the container

  5. the public Docker image

After launch the container we execute a docker command to inspect the logs of the container. In this way we can see when the import process has finished. After them we need to change the password for the default user, so you need to open a browser and navigate to http://localhost:7474 . Login with the default user/password (neo4j/neo4j) and change the password.

In our case we’ll use panama as password

Dependencies

import groovy.transform.EqualsAndHashCode
@GrabConfig(systemClassLoader=true)
@Grab('org.slf4j:slf4j-api:1.7.10')
@Grab('org.grails:grails-datastore-gorm-neo4j:6.1.6.RELEASE')

import groovy.transform.ToString
import grails.gorm.annotation.Entity as EntityAnnotation
import org.grails.datastore.gorm.neo4j.Neo4jDatastore

Domain

The Panama Papers’s domain is very simple. It has only 4 entities but a lot of relationships.

In the next diagram we’ll show only a few of them.

diag 91516a665d817ab80fe353813d724a4f

GORM

@EntityAnnotation //(1)
@ToString
class Officer  {
    String name
    String valid_until
    String icij_id

    static hasMany = [  //(2)
            director_of : Entity
    ]
}

@EntityAnnotation
@ToString
class Entity {
    String sourceID
    String address
    String former_name
    String jurisdiction
    String struck_off_date
    String service_provider
    String countries
    String jurisdiction_description
    String valid_until
    String ibcRUC
    String original_name
    String name
    String inactivation_date
    String country_codes
    String incorporation_date
    String status
}

@EntityAnnotation
@ToString
class Intermediary {
    String sourceID
    String valid_until
    String name
    String country_codes
    String countries
    String status
}

@EntityAnnotation
@ToString
class Address {
    String address
}
  1. As we’ll use a domain object 'Entity' we rename the annotation from GORM as EntityAnnotation

  2. We’ll declare relationships between entities as hasMany. In this example we’ll declare only 1 relationship

Command line arguments

Our script will search into the database looking for Officers or Entities by similar name passed as argument. Also we need to configure the connection parameters as the url, user and password.

@Option(names= ["-h", "--help"], usageHelp= true, description= "Show this help message and exit.")
@Field boolean helpRequested

@Option(names = ["-b", "--bolt"], description = "url for the bolt driver")
@Field private String url = 'bolt://localhost:7687'

@Option(names = ["-u", "--user"], description = "user to neo4j connection")
@Field private String user = "neo4j"

@Option(names = ["-p", "--password"], description = "password to neo4j connection")
@Field private String password = "password"


@Option(names = ["-a", "--action"], description = ["action to perform as:", "searchOfficer, searchEntity"])
@Field private String action = "searchOfficer"

@Parameters(arity="0", paramLabel="SUBJECT", description="The subject to use.")
@Field String subject

Configuration

Map getConfiguration(){
    [
            'grails.neo4j.url': url,
            'grails.neo4j.username': user,
            'grails.neo4j.password': password
    ]
}


Neo4jDatastore datastore
void initDataStore(){
    datastore = new Neo4jDatastore(configuration, Officer, Entity, Intermediary, Address)
}

We need to configure the url, user and password and a list of the domain objects we want to GORM use.

Simple Query

Once initialized the datasource, we can use GORM to perfom simple querys:

println "We have ${Officer.count()} officers"   //(1)
println "We have ${Entity.count()} entities"
println "We have ${Intermediary.count()} intermediaries"

Officer.withTransaction {   //(2)

    def pilarBorBon = Officer.findByName('Pilar de Borbón') //(3)

    println "We have an interesting Officer $pilarBorBon.name"

    pilarBorBon.director_of.each {  //(4)
        println "\t $it.name"
    }
}
  1. perfom some Count

  2. as we are using a script we need to open a session with the database

  3. findByXXXX is added by GORM automatically

  4. we can navigate using relationships

Complex Query

We can do more complicate querys using where for example to find similar Officer by similar names.

void searchOfficer(String search) {
    Officer.withTransaction {

        Officer.where {
            name ==~ "%${search}%"
            join 'director_of'      //(1)

        }.list().eachWithIndex { Officer officer, int idx ->
            println "$idx: $officer.name"

            officer.director_of.each{ director_of ->
                println "\t $director_of.name, $director_of.address"
            }
        }
    }
}
  1. using join we instruct to GORM to fetch all objects in the same query

With this method we can instruct to Neo4J to look for officer without know the name:

groovy Neo4jGorm.groovy -p panama Macron

0: Macron Associate Company
         Craftlighting Inc., null

groovy Neo4jGorm.groovy -p panama Trump

0: Pacific Trump Enterprises Ltd
1: Trumptech (Far East) Limited
2: New Trump Technology Limited

groovy Neo4jGorm.groovy -p panama Kirchner

0: Guillermo Kirchner
         Meck Investments, Ltd, Corporate Solutions Inc. Corporate Solutions LLC 520 Brickell Key Dr   Ste 1403 Miami, Fl  33131 USA RT BVI

And you? are you in the Panama Papers ?


Script
//tag::dependencias[]

import groovy.transform.EqualsAndHashCode
@GrabConfig(systemClassLoader=true)
@Grab('org.slf4j:slf4j-api:1.7.10')
@Grab('org.grails:grails-datastore-gorm-neo4j:6.1.6.RELEASE')

import groovy.transform.ToString
import grails.gorm.annotation.Entity as EntityAnnotation
import org.grails.datastore.gorm.neo4j.Neo4jDatastore
//end::dependencias[]
@Grab('info.picocli:picocli:2.0.3')

@picocli.groovy.PicocliScript


import groovy.transform.Field
import static picocli.CommandLine.*

//tag::picocli[]
@Option(names= ["-h", "--help"], usageHelp= true, description= "Show this help message and exit.")
@Field boolean helpRequested

@Option(names = ["-b", "--bolt"], description = "url for the bolt driver")
@Field private String url = 'bolt://localhost:7687'

@Option(names = ["-u", "--user"], description = "user to neo4j connection")
@Field private String user = "neo4j"

@Option(names = ["-p", "--password"], description = "password to neo4j connection")
@Field private String password = "password"


@Option(names = ["-a", "--action"], description = ["action to perform as:", "searchOfficer, searchEntity"])
@Field private String action = "searchOfficer"

@Parameters(arity="0", paramLabel="SUBJECT", description="The subject to use.")
@Field String subject
//end::picocli[]

//tag::domain[]
@EntityAnnotation //(1)
@ToString
//tag::plantuml[]
class Officer  {
    String name
    String valid_until
    String icij_id

    //tag::noplantuml[]
    static hasMany = [  //(2)
            director_of : Entity
    ]
    //end::noplantuml[]
}
//end::plantuml[]

@EntityAnnotation
@ToString
//tag::plantuml[]
class Entity {
    String sourceID
    String address
    String former_name
    String jurisdiction
    String struck_off_date
    String service_provider
    String countries
    String jurisdiction_description
    String valid_until
    String ibcRUC
    String original_name
    String name
    String inactivation_date
    String country_codes
    String incorporation_date
    String status
}
//end::plantuml[]

@EntityAnnotation
@ToString
//tag::plantuml[]
class Intermediary {
    String sourceID
    String valid_until
    String name
    String country_codes
    String countries
    String status
}
//end::plantuml[]

@EntityAnnotation
@ToString
//tag::plantuml[]
class Address {
    String address
}
//end::plantuml[]

//end::domain[]

//tag::configuration[]
Map getConfiguration(){
    [
            'grails.neo4j.url': url,
            'grails.neo4j.username': user,
            'grails.neo4j.password': password
    ]
}


Neo4jDatastore datastore
void initDataStore(){
    datastore = new Neo4jDatastore(configuration, Officer, Entity, Intermediary, Address)
}
//end::configuration[]

//tag::complexQuery[]
void searchOfficer(String search) {
    Officer.withTransaction {

        Officer.where {
            name ==~ "%${search}%"
            join 'director_of'      //(1)

        }.list().eachWithIndex { Officer officer, int idx ->
            println "$idx: $officer.name"

            officer.director_of.each{ director_of ->
                println "\t $director_of.name, $director_of.address"
            }
        }
    }
}
//end::complexQuery[]


initDataStore()

//tag::simpleQuery[]
println "We have ${Officer.count()} officers"   //(1)
println "We have ${Entity.count()} entities"
println "We have ${Intermediary.count()} intermediaries"

Officer.withTransaction {   //(2)

    def pilarBorBon = Officer.findByName('Pilar de Borbón') //(3)

    println "We have an interesting Officer $pilarBorBon.name"

    pilarBorBon.director_of.each {  //(4)
        println "\t $it.name"
    }
}
//end::simpleQuery[]

if( subject)
    this."$action"(subject)