Groovy scripts with Groovy

by Andrés Viedma

scripts with

Restless Java lover

Software dinosaurmore than 20 years nagging

Working at

Usual suspect of MadridGUG and MadridJUG

Arquitecture at

Service DB

TService

TuentiNgDBs

tuenti-ng

Micro serviceTService



Arquitecture at

Service DB

TService

TuentiNgDBs

tuenti-ng




PHP Script

About Groovy

● Dynamic Language, prepared for scripting

● Runs in the JVM

o Lots of Java libraries

o Can use the code of our Java services

● Source code “almost” compatible with Java

o Main difference: == operator means equals()

o If you make your script in Java, it’ll WORK

● Other “cool” features, mainly inspired by Python

Arquitecture at

Service DB

TService

TuentiNgDBs

tuenti-ng




Arquitecture at

Service DB

TService

TuentiNgDBs

tuenti-ng




Groovy Script

Arquitecture at

Service DB

TService

TuentiNgDBs

tuenti-ng




Groovy Script

Mix your Production code with direct accesses to the

DB, config, TServices...

Hello World (Java, but works also as Groovy)

import utils.*;

World world = new World();world.setId("mongo");world.setName("Mongo");

System.out.println("Hello " + world.getName() + "!");

package utils;

public class World { private String id; private String name;

public String getId() { return id; } public void setId(String id) { this.id = id; } public String getName() { return name; } public void setName(String name) { this.name = name; }}

$GRSCRIPTS/helloworld.groovy $GRSCRIPTS/utils/World.groovy

Hello World (Groovy style)

def world = new World(id: "mongo", name: "Mongo")println "Hello ${world.name}!"

class World { String id String name}

$GRSCRIPTS/helloworld.groovy

$ cd $GRSCRIPTS$ groovy helloworld.groovyHello Mongo!$
















Closures / collectionsdef worlds = [ new World(id: "mongo", name: "Mongo"), new World(id: "mars", name: "Mars"), new World(id: "wonderland", name: "Wonderland")] as ArrayList

def heroesByWorld = [ mongo: ["Flash Gordon", "Prince Barin"], mars: ["John Carter"], wonderland: ["Alice", "Queen of Hearts"]]

heroesByWorld.each { worldEntry -> def worldId = worldEntry.key def world = worlds.find { it.id == worldId } println "*** World: ${world.name}"

def heroList = worldEntry.value heroList.each { hero -> println hero }}

















Functions and scope of variablesdef worlds = [ … ]def heroesByWorld = [ … ]

eachWorld { world, heroes -> println "*** World: ${world.name}" heroes.each { hero -> println hero }}

void eachWorld(Closure closure) { heroesByWorld.each { worldEntry -> def worldId = worldEntry.key def world = worlds.find { it.id == worldId } def heroList = worldEntry.value closure(world, heroList) }}













● worlds = [ … ]

● @Field def worlds = [ … ]

● @Field List<World> worlds = [ … ]

Grape: libraries from Maven repos@Grab(group='org.apache.commons', module='commons-email', version='1.3.3')

import org.apache.commons.mail.*

def usr, pwd, toAddress = [ … ]

println "Creating email object"Email email = new SimpleEmail( hostName: "smtp.googlemail.com", smtpPort: 465, authenticator: new DefaultAuthenticator(usr, pwd), SSLOnConnect: true, from: usr, subject: "Groovy mail!", msg: "You're hot and you know it ... :-)" )

email.addTo(toAddress);

println "Sending email..."email.send()println "OK"

<ivysettings> <settings defaultResolver="downloadGrapes"/> <resolvers> <chain name="downloadGrapes"> <filesystem name="cachedGrapes"> <ivy pattern="${user.home}/.groovy/grapes/[organisation]/[module]/ivy-[revision].xml"/> <artifact pattern="${user.home}/.groovy/grapes/[organisation]/[module]/[type]s/[artifact]-[revision].[ext]"/> </filesystem> <ibiblio name="codehaus" root="http://repository.codehaus.org/" m2compatible="true"/> <ibiblio name="ibiblio" m2compatible="true"/> <ibiblio name="java.net2" root="http://download.java.net/maven/2/" m2compatible="true"/> </chain> </resolvers></ivysettings>

Grape: default config





Grape: add your repository

<ivysettings> <settings defaultResolver="downloadGrapes"/>

<credentials host="nexus.xxx.xxx" realm="Sonatype Nexus Repository Manager" username="xxxx" passwd="xxxx"/> <property name="nexus-root" value="http://nexus.xxx.xxx/content/groups/public/"/>

<resolvers> <chain name="downloadGrapes"> <ibiblio name="nexus" root="${nexus-root}" m2compatible="true" /> (...) </chain> </resolvers></ivysettings>

http://nexus.xxx.xxx/content/groups/public/%22/

Grape: add your repository



<resolvers> <chain name="downloadGrapes"> <ibiblio name="nexus" root="${nexus-root}" m2compatible="true" /> (...) </chain> </resolvers></ivysettings>

http://nexus.xxx.xxx/content/groups/public/%22/

Grape: updatable snapshots



<resolvers> <ibiblio name="downloadGrapes" root="${nexus-root}" m2compatible="true" /> </resolvers></ivysettings>

Grape: updatable snapshots



<resolvers> <ibiblio name="downloadGrapes" root="${nexus-root}" m2compatible="true" /> </resolvers></ivysettings>

Databases: Sql object@Grab('mysql:mysql-connector-java')@GrabConfig(systemClassLoader=true)import groovy.sql.Sql

def sql = Sql.newInstance("jdbc:mysql://localhost:3306/simfonics_charging", "tuenti_developer", "lalala", "com.mysql.jdbc.Driver")sql.resultSetConcurrency = java.sql.ResultSet.CONCUR_UPDATABLE

def version = 2def newVersion = 3def type = "BUNDLEDEF"

sql.eachRow("""select * from charging_element_definition where version = ${version} and element_type = ${type}""" ) { row -> if (row.element_id.startsWith("PG0.")) { println "Bundle ${row.element_id} version ${row.version}" + " to ${newVersion}" row.version = newVersion }}













Script arguments: CliBuilder

def cli = new CliBuilder(usage:'simpleHtmlServer -p PORT -d DIRECTORY')cli.with { h longOpt:'help', 'Usage information' p longOpt:'port',argName:'port', args:1, type:Number.class,'Def is 8080' d longOpt:'dir', argName:'directory', args:1, 'Default is .'} def opts = cli.parse(args)if(!opts) returnif(opts.help) { cli.usage() return}

def directory = opts.ddef port = opts.port(...)







Create your own utility classes

Grapes in reusable files

import utils.*

PlatformLibs.load()

(...)

package utils

@Grab('commons-logging:commons-logging:1.1.3')@Grab('org.apache.logging.log4j:log4j-api:2.1')@Grab('...')(...)

class PlatformLibs { static void load() {}}

(Script) utils/PlatformLibs.groovy

“Clean” Services Clients

import utils.*

TServiceClient subscriptionService = new TServiceClient( base: "http://xxxxx/SubscriptionService/")

def statusData = subscriptionService.getSubscriptionStatus([id: 80644970])

println "Result of the call: ${statusData}"println "Subscription status: ${statusData.status}"

● Generic, equivalent to CURLs

● Using dynamic Groovy features

● Methods of the service are used transparently as if it were a “real” client interface

“Clean” clients trick@Grab(group='org.codehaus.groovy.modules.http-builder', module='http-builder', version='0.7')(...)class TServiceClient {

(...) def methodMissing(String name, args) { def jsonResult = jsonCall(name, args)

(...) }

String jsonCall(String name, args) { def http = getHttpClient() def json = http.request(POST, JSON) { req -> body = [ "jsonrpc": "2.0", "id": callId, "method": interface + "." + version + "." + name, "params": [ "params": args, "gid": gid, "rid": rid ] ] } return json }}

“Clean” clients trick@Grab(group='org.codehaus.groovy.modules.http-builder', module='http-builder', version='0.7')(...)class TServiceClient {

(...) def methodMissing(String name, args) { def jsonResult = jsonCall(name, args)

(...) }

String jsonCall(String name, args) { def http = getHttpClient() def json = http.request(POST, JSON) { req -> body = [ "jsonrpc": "2.0", "id": callId, "method": interface + "." + version + "." + name, "params": [ "params": args, "gid": gid, "rid": rid ] ] } return json }}

Classes for named access to the environment

import utils.*import groovy.sql.*

def chargingService = TuentiEnv.jsc.prod.chargingdef balanceData = chargingService.getBalance([id: 80644970], "es")

Sql chargingDb = TuentiEnv.jsc.prod.charging.sql

● Easy named access to:

o Databases

o Service clients

o For every environment

Environment helperpackage utils

@GrabConfig(systemClassLoader=true, initContextClassLoader=true)@Grab('mysql:mysql-connector-java')@GrabExclude('xml-apis:xml-apis')import groovy.sql.Sql

class TuentiEnv {(... properties with common configuration ...)

static void initConsole() { Object.metaClass.tu = TuentiEnv.metaClass Object.metaClass.sim = TuentiEnv.simfonics Object.metaClass.jsc = TuentiEnv.jsc (...)

Object.metaClass.pretty = { obj ->(new groovy.json.JsonBuilder(obj)).toPrettyString() } Object.metaClass.tservice = { String base, String

iface = null -> return new TServiceClient(base: base,

iface: iface) } }}







iface: iface) } }}







iface: iface) } }}

Services Console: easily test your services

aviedma@dev6:~$ groovysh

Groovy Shell (2.2.2, JVM: 1.6.0_26)Type ':help' or ':h' for help.---------------------------------------------------------------------groovy:000> utils.TuentiEnv.initConsole()===> nullgroovy:000> jsc.prod.charging.getBalance([id: 80644970], "es")===> [amount:[amountInMillieuros:0], isAvailable:true]groovy:000> pretty jsc.prod.subscription.getSubscriptionStatus([id: 80644970])===> { "isAvailable": true, "lastUpdateTime": 1423501862, "status": "active"}groovy:000>







Tracking scripts execution

import utils.*

TuScript.track("Say hello to the world”)

println "Hello world!"(...)

● A single line at the beginning of the script

o Logs start-end time and duration of the run (even if it fails or is killed)

o Without changes in the scripts, it can be used to register the script execution for monitoring

Tracking scripts executionclass TuScript { public static void track(String description) { if (metadata != null) { finishCurrentScript() } else { addShutdownHook { finishCurrentScript() } captureSignal("INT") captureSignal("TERM") } metadata = new ScriptMetadata(description: description) logScriptStart() } public static void finishCurrentScript() { if (metadata != null) {

logScriptEnd() } metadata = null }

private static void captureSignal(String signal) { def oldHandler = Signal.handle(new Signal(signal), [handle: { sig -> logSignal(signal) if (oldHandler) oldHandler.handle(sig) }] as SignalHandler); } (...)







Split execution in stepssql.eachRow("""

<loooooong SQL> """) { row -> processRow(row)}

void processRow(row) { println "${row.id}: ${row.name} ${row.surname}" (... lots and lots of processing ...)}

Split execution in stepssql.eachRow("""

<loooooong SQL> """) { row -> processRow(row)}


Split execution in stepsvoid step1(String file) { Dump dump = new Dump(file) dump.withWriter(fieldNames) { out -> sql.eachRow(""" <loooooong SQL> """) { row -> out.insert(row) } }}

void step2(String file) { Dump dump = new Dump(file) dump.eachRow { out -> processRow(row) }}








Same processing code, no changes needed

Running Shell commandsdef process = 'ls /home/andres'.execute()def procOutput = new InputStreamReader(process.in)procOutput.eachLine { line -> println line}process.waitFor()println "** Return code: ${process.exitValue()}"



def process = 'echo "Hola caracola"'.execute()println process.text

def process = ['echo', 'Hola caracola'].execute()




def process = 'ls /home/andres/*'.execute()

def process = ['sh', '-c', 'ls /home/andres/*'].execute()




def process = 'ls /home/andres/*'.execute()

def process = ['sh', '-c', 'ls /home/andres/*'].execute()

def process = 'ls'.execute()

def process = 'ls'.execute([], new File('/home/andres'))

Running Shell commandsclass Shell { File currentDir = new File('.') final Map environment = [:] Process run(String command) { return ['sh', '-c', command].execute( environment.collect { "${it.key}=${it.value}" }, currentDir) } String runAndGet(String command) { def proc = run(command) proc.waitFor() return proc.text }}

Shell shell = new Shell(currentDir: new File("/home/andres/temp"))shell.environment["GREETINGS"] = "Hi!"

print shell.runAndGet('echo $GREETINGS')print shell.runAndGet("ls i*")print shell.runAndGet('echo "Hola caracola"')

Running Shell commandsclass Shell { File currentDir = new File('.') final Map environment = [:] Process run(String command) { return ['sh', '-c', command].execute( environment.collect { "${it.key}=${it.value}" }, currentDir) } String runAndGet(String command) { def proc = run(command) proc.waitFor() return proc.text }}

Shell shell = new Shell(currentDir: new File("/home/andres/temp"))shell.environment["GREETINGS"] = "Hi!"

print shell.runAndGet('echo $GREETINGS')print shell.runAndGet("ls i*")print shell.runAndGet('echo "Hola caracola"')

Thanks!

Questions?