Infinispan in 50 minutesSanne Grinovero
Sanne Grinovero•Senior Software Engineer at Red Hat•Hibernate team
• lead of Hibernate Search• Hibernate OGM
•Infinispan• Search, Query and Lucene integrations
•Apache Lucene•JGroups
Who’s this guy?
Agenda•What is Infinispan•API•Key features•Three use cases•The path ahead
Infinispanl Open source highly scalable data grid platform
l Distribution or Replicationl Sync or Asyncl Transactionall Persists contents using a CacheLoader
− Write-through or write-behind− Shared or per cluster node
l Hibernate second-level cachel State of the art eviction strategies
Distributed Data Structure
Storage Engine (RAM
+Overflow)
Transaction
Query
Map/Reduce
Monitoring
Transport (JGroups)
Memcached Hotrod Rest
Node (JVM process)
Storage Engine (RAM
+Overflow)
Transaction
Query
Map/Reduce
Monitoring
Transport (JGroups)
Memcached Hotrod Rest
Node (JVM process)
TCP/UDP
RemoteClient
TCPTCP
High Availability•Memory is volatile•Make redundant copies
•Total replication (Replication Mode)•Partial replication (Distribution Mode)
• numOwners•Topology changes
•Nodes will crash!•Re-arrange state
Elasticity•Expect
•Node additions•Node removals
•Topology changes •are totally consistent•do not "stop the world"
API•Key-value store
• java.util.concurrent.ConcurrentHashMap•JSR-107 compliant•CDI support•Spring support
Integration•Hibernate integration
•2nd level cache•Lucene integration
•Query• index storage
•ModeShape•CapeDwarf•CacheStores
•Cassandra, MongoDB, ...
Key features•Transactions•Performance•Persistence•Map/Reduce•Querying
Big Data / Fast Data ?
Transactions•JTA transactions support
•XA or Synchronization based enlistment• recovery!
•Local transactions (batching)•Transactional modes
•optimistic •pessimistic• coming: ergonomics!
Persistence/Cache Store•Used for
•durability• increased storage•warm caches
•Various implementations•HBase, Cassandra, BDBJE, JDBC, MongoDB, File, Cloud,
Remote / Infinispan, ...•Extensible
•easy!
Network Performance & Reliability
Map/Reduce example
Map/Reduce
Map/Reduce
Infinispan on cloudsl Cloud-friendly autodiscovery strategies
l S3_PING, JDBC_PING, TCP_PINGl Fully dynamic clustering
l Add or kill nodes on the flyl Using a CacheLoader, scale back to zero nodes without data loss
Querying•Based on Hibernate Search/Lucene•Objects in the grid are indexed
• index management
Annotate your model@Indexedpublic class Book implements Serializable {
@Field String title; @Field String author; @Field String editor;
public Book(String title, String author, String editor) { this.title = title; this.author = author; this.editor = editor; }
}
Run a QuerySearchManager qf = Search.getSearchManager(cache); Query query = qf.buildQueryBuilderForClass(Book.class) .get() .phrase() .onField("title") .sentence("in action") .createQuery(); List<Object> list = qf.getQuery(query).list();
Use cases•Local cache•Cluster of caches•Autonomous data store
•access protocols
Local cache
Features of a local cache•Eviction•Expiry•Write through, write behind•Preloading•Notifications•Statistics
Local cache not good enough...
Cluster of caches
Limitations of embedded mode
•Client is affected by cache topology changes•Shared resources•Tier management
• incompatible JVM tuning•security•garbage collection
•Non-JVM clients
Cache Servers
Client/Server Protocols•REST•Memcached•Hotrod
• “proprietary”• java, python, ruby
Lucene Index Stored in Infinispan
Single Node Performance Idea
Multi-Node Setup
Performance Warnings• Set Lucene's maximum segment size to fit in LuceneDirectory chunk_size to avoid readlocks• Verify blob sizes fit in JGroups network packets• Check for CacheStores “sweet spot” size
The Path Ahead• 5.1.x.Final• 5.2 - February 2013
• non-blocking state transfer• cross-site replication• rolling upgrades for hotrod-clients• map/reduce marked stable
• 5.3 - Spring 2013• state transfer between sites
• 6.0• Beyond the current amazing performance• Scalable index writing• Advanced CacheStores
JBoss Data Grid (JDG)•JBoss Data Grid•Build on top of Infinispan•http://www.jboss.com/edg
Thank you!•infinispan.blogspot.com•@Infinispan•@SanneGrinovero