Grails lucenecacherdfperformance

BLAZING FAST PERFORMANCEwith Spring Cache, Lucene, GPars, RDF and Grails

WHAT’S A TRIPLE?

subject

predicate

object

WHAT’S A TRIPLE?

subject

predicate

object

“Imatinib”

type

Pharmaceutical Drug

LINKING OPEN DRUG DATAhttp://esw.w3.org/HCLSIG/LODD

WHAT’S A TRIPLE?http://www4.wiwiss.fu-berlin.de/drugbank/page/drugs/DB00619

subject predicate object

DB00619 label “Imatinib”

DB00619 type drugbank:drugs

DB00619 brandName “Gleevec”

DB00619 drugbank:target targets:17

WHAT’S A TRIPLE?http://www4.wiwiss.fu-berlin.de/drugbank/page/targets/17

subject predicate object

DB00619 drugbank:target targets/17

targets/17 type drugbank:drugs

targets/17 label “Proto-oncogene tyrosine-protein kinase ABL1”

targets/17 geneName “ABL1”

RELATIONSHIPS

Gene

Disease

Predicate

Legend

Compound

ABL1

Leukemia

Imatinib

associatedGene

possibleDrug

target

PERFORMANCE

Type-ahead search

select id, label from targetswhere label like ‘%${queryValue}%’

select id, label from targetswhere label like ‘%${queryValue}%’

SELECT ?uri ?label WHERE { ?uri rdfs:label ?label . ?uri rdf:type drugbank:targets . FILTER regex(?label, '\\\\Q${queryValue}\\\\E', 'i')"}

select id, label from targetswhere label like ‘%${queryValue}%’

SELECT ?uri ?label WHERE { ?uri rdfs:label ?label . ?uri rdf:type drugbank:targets . FILTER regex(?label, '\\\\Q${queryValue}\\\\E', 'i')"}

40 million records

Index the data (up front performance hit)

Search (really really really really fast)

BuildConfig.groovy: runtime 'org.apache.lucene:lucene-core:3.0.1'

-OR-

grails install-plugin searchable

DE-DUPE

Lucene Extension

Index the (RDF) data

Search (really really really really fast)

Understands Triples (or structured data)

subject predicate object

DB00619 label “Imatinib”

SirenTupleQuery tupleQuery = new SirenTupleQuery()tupleQuery.add(createCellQuery(‘label’, SirenTupleConstraintPosition.PREDICATE), SirenTupleClause.Occur.MUST)

tupleQuery.add(createCellQuery(‘imatinib’, SirenTupleConstraintPosition.OBJECT), SirenTupleClause.Occur.MUST)

GRAILS SPRING CACHE

• grails install-plugin springcache

• Add @Cacheable (and or @CacheFlush) annotation to services / controllers

@Cacheable('somecachename') def slow(String name) { log.info "resolving $name" Thread.sleep(2000) return "took a long time to resolve ${name}" }

GPARS• BuildConfig.groovy:

runtime 'org.codehaus.gpars:gpars:0.10' runtime 'org.coconut.forkjoin.jsr166y:jsr166y:070108'

• http://www.gpars.org/guide/index.html

•Note: hibernate session is not available on GPars threads; you need to get one yourself (use DomainClass.withTransaction)

•Data Parallelism

•Map-Reduce, Fork-Join, and many more...

GPARSvoid resolve(scoreDocs){ scoreDocs.each {scoreDoc ->

//do something in single thread } }

void resolveWithPool(scoreDocs){ GParsPool.withPool { scoreDocs.eachParallel {scoreDoc ->

//do something in parallel } }}

OTHER HANDY TOOLS• Grails Melody (Java Melody) (monitoring)

• Perf4j (logging performance)

• Solr (lucene search server)

• Grails Console Plugin (web based console)