Thursday, August 12, 2010

Infinispan as Hibernate L2 Cache

We're now using Infinispan as a Hibernate L2 cache. There was only one obstacle: locking of our persistent entities.

Formerly we were using EHCache in a single-server configuration only. We switched to Atomikos + Infinispan and got the app working again. So far so good.

Then I ran some load tests. Uh - oh, exceptions all over the place, mostly Infinispan TimeoutExceptions. I'm using the same lock timeout as in the old EHCache version of the system.

In the end this boiled down to long running processing that was in fact causing lock timeouts. This didn't happen in EHCache possibly because it only supports CacheConcurrencyStrategy.READ_WRITE? I'm not sure; we never pinned down exactly why this arose only after switching to Infinispan.

The app is running very well after sorting through this issue. We've completed load testing and in a single server scenario we get just about exactly the same performance we saw with EHCache.

I wanted to use the Infinispan CacheManager for some app-specific (non-hibernate) caching as well, so I wrote a SpringInfinispanRegionFactory that let me spring-create the CacheManager and wire it to both Hibernate and to my other classes.

The only remaining problem is that we needed a JTA implementation. We went with Atomikos and we aren't happy with it. Bitronix looks good, but doesn't support Infinispan, not even using the last-resource gambit. Jotm is probably OK but the XAPool connection pool that it supports is widely agreed to be awful. Any brilliant suggestions?

4 comments:

  1. I should mention that the app is Tomcat-based and we would like to keep it that way, thus the hassle with JTA implementations.

    I should also mention that the JBoss transaction manager has a good reputation. The problem is that they don't seem to have an XA Connection Pool that can be spring-configured, so I'm not sure how you're ever supposed to use it in a real-world Spring configured app.

    I can't understand why Spring doesn't build their own transaction manager. It fits with their no-app-server-required platform so well. I also don't see why I want their tcServer product when it has so little to offer beyond vanilla Tomcat, but if it had a pre-integrated JTA and JMS solution it would be a done deal.

    ReplyDelete
  2. In my current project I basically "succeeded" with Bitronix + Infinispan integration (Tomcat webapp + Spring + Hibernate + Infinispan). I did small integration layer as Infinispan guys documented here: http://community.jboss.org/wiki/ImplementingstandaloneJPAJTAHibernateapplicationoutsideJ2EEserverusingInfinispan2ndlevelcache but stress testing whole system sometimes runs into weird exceptions. Before I tried with Atomikos but these guys don't recommend to use it with PostgreSQL (but we are using it). So I'm wondering if you have any updates on that? What kind of solution really works for our environments?

    ReplyDelete
  3. Andriy we're working towards truly massive scalability and we have come to the conclusion that an approach that relies on Hibernate's L2 cacheing isn't viable for us in the long term.

    We're currently working on a novel way of using Infinispan as an entity cache and collections cache along with Hibernate but with Hibernate's L2 cache disabled. I'll write up a blog post when we have it working properly; it will be a few months.

    This is no small endeavor and certainly not recommended for most projects by the way.

    ReplyDelete
  4. try NCache which also offers its support for NHibernate L2 cache. Ncache is a distributed cache with separate editions for .NET and Java applications.

    ReplyDelete