Startpagina > English, java, work > JPA with versioning and full text search – Mixing Hibernate Envers with elasticsearch

JPA with versioning and full text search – Mixing Hibernate Envers with elasticsearch

I’m working on a project where we need to search the data the ‘google way’ and keep a history of every change in the data. Since a requirement is that we have to store the data in an sql database I started with Hibernate JPA. Hibernate Envers was added for versioning. For the Google search (or just full text search) I needed something with Lucene in the background. Hibernate Search seemed like a good combination.

Pretty soon I found out that Envers and Search don’t mix very well and a little search on the Hibernate forum confirmed it [1]. Envers and Search are great products, don’t get me wrong, but this time it didn’t work out.

Furthermore it’s good to know that I’m also using Spring, which can mess up things pretty bad. Again it’s a lethal cocktail, nothing to do with the quality of the frameworks.

My problem

Since Hibernate Search has annotations to define your Lucene documents and automatically updates the index I needed a replacement for both. So the first step is something like an event listener and the second step a new search framework.

Event listeners

Hibernate Search acts on certain events in the database: create, update and delete. This is done automatically, but now I need to do this manually. Instead of doing it in my Dao classes (with the chance of forgetting to do it somewhere) I was sure there was an easy solution for it.

After some searching I started with the @PostPersist en @PostUpdate annotations from JPA. Bad idea with Spring, since JPA controls the annotated classes and not Spring. There was no way I could inject my indexing service. Trying to fix this problem messed up things a lot more. I ended up with the @Configurable annotation and AspectJ. When simple things get so complicated it’s usually a good idea to hit undo a few times and start over again.
After some googlin’ I found out about the Service Provider Interface in Hibernate [2]. A feature available since version 4, great and simple!

The first step is adding a class where the listeners are configured:

@Component
public class HibernateListenersConfigurer {

@Autowired
private EntityManagerFactory entityManagerFactory;

@Autowired
private ShipEntityListener listener;

@PostConstruct
public void registerListeners() {
HibernateEntityManagerFactory hibernateEntityManagerFactory = (HibernateEntityManagerFactory) this.entityManagerFactory;
SessionFactoryImpl sessionFactoryImpl = (SessionFactoryImpl) hibernateEntityManagerFactory.getSessionFactory();
EventListenerRegistry registry = sessionFactoryImpl.getServiceRegistry().getService(EventListenerRegistry.class);
registry.getEventListenerGroup(EventType.POST_COMMIT_INSERT).appendListener(listener);
registry.getEventListenerGroup(EventType.POST_COMMIT_UPDATE).appendListener(listener);
registry.getEventListenerGroup(EventType.POST_DELETE).appendListener(listener);
}

}

This is basically a class with a @PostConstruct annotation. The method annotated with this annotation will be excecuted on startup. I register the 3 events I want actions on to the EventListenerRegistry and I’m ready for the next step: the ShipEntityListener (our product is dealing with Ships).

@Component
public class ShipEntityListener implements PostInsertEventListener, PostUpdateEventListener, PostDeleteEventListener {

private static final Logger log = LoggerFactory.getLogger(ShipEntityListener.class);

@Autowired
private ShipIndexDao shipIndexDao;

@Override
public void onPostInsert(PostInsertEvent event) {
Object entity = event.getEntity();
if (entity instanceof Ship) {
shipIndexDao.addShip((Ship) entity);
}
}

@Override
public void onPostUpdate(PostUpdateEvent event) { … }

@Override
public void onPostDelete(PostDeleteEvent event) { … }

}

Just implement the interfaces here and you’re done. Now when a create, update or delete happens. The ShipIndexDao doesn’t exist yet, but you get the idea. Now we need a new search framework.

Possible candidates

I heard a lot about Apache Solr so I gave that a shot. The documentation gave me headache so I moved it to the bottom of my list of frameworks to research. A good product with poor documentation still is a poor product!
Compass looked nice, but it appeared to be a bit outdated. After some searching I found that its successor was called elasticsearch. The documentation looked nice and there was an active community. Only drawback so far was that you needed to start a separate server for it.

Integrating elasticsearch in my project and with Spring

Since I don’t like to start a separate server while developing I search around and found something made by a guy called David Pilato: elasticsearch integration with Spring [3], great! It seems programming is glueing some pieces together nowadays!

I said something about easy conversion. Since Lucene Documents are basically maps I decided to go with maps. A future project might be to create annotations for it, but for now this works.

Envers and elasticsearch

My first idea was to store the revision number of Envers in elasticsearch. Unfortunately it was too big a hassle to get this number, so I went with storing a sequence number. In our project we know the order of the changes in the data so it’s easy to link those two together.

Conclusion

I hope this article gives you enough information. For me it’s a bit like documenting my project. Feel free to ask for clarifications, I’ll be happy to improve this article.
During my search I found a lot of differences between versions of frameworks. It’s a good thing frameworks are fixed/get new features but you have to be real careful which features are available. Hibernate Envers for example is now integrated in Hibernate Core so the latest Envers documentation can be found there instead of version 3.6 I came across a lot.
I really like Elasticsearch. Sometimes it’s a bit difficult to find out how to do things, but there are loads of sample applications and a very active community.
I miss the Hibernate Search annotations, but working with a Map is okay for now. Creating a index from a database filled with data is still a problem to be fixed, but we would’ve had the same problem with Hibernate Search.

Sources

[1] Using Hibernate Search with Hibernate Envers
[2] Spring Managed Event Listeners With Jpa
[3] Spring factories for Elasticsearch

About these ads
Categorieën:English, java, work Tags:, , ,
  1. Rizzi
    10 juli 2014 om 22:10

    Hi. Nice article. One of my first question was: How to perform a full-text search with this integration? Is the full-text search returning the elastic search entity or returning the jpa entity?

  2. Jeroen van Wilgenburg
    11 juli 2014 om 11:33

    Thanks.

    It depends on what you want to do. In my project we returned the elasticsearch entity and only used the database to store the unindexed data. This was a relatively small dataset and we stored all the fields in elasticsearch.
    When you have a large database or have to run subsequent queries it might be a better idea to return the jpa-entity (and store the primary key from the database in elasticsearch to link both entities)

  3. Rizzi
    11 juli 2014 om 14:05

    Hi. Thanks for your answer.

    I have a working jpa project and try to add full-text search to some entity classes.
    I created extra “index” entities for this classes, because there are some fields in this classes not have to get indexed. My first try was to use spring data elastic search to manage the index parallel on required places, but your idea to use the listener is much less expensive work.
    One problem is to get only the id’s of matching entries from elastic search to load them from jpa and the second, that every entity update request to elastic search create a new dataset, instead to update the existing one. Have to do with “(and store the primary key from the database in elasticsearch to link both entities)” in your answer, i think.

    greets Rizzi

  1. No trackbacks yet.

Geef een reactie

Vul je gegevens in of klik op een icoon om in te loggen.

WordPress.com logo

Je reageert onder je WordPress.com account. Log uit / Bijwerken )

Twitter-afbeelding

Je reageert onder je Twitter account. Log uit / Bijwerken )

Facebook foto

Je reageert onder je Facebook account. Log uit / Bijwerken )

Google+ photo

Je reageert onder je Google+ account. Log uit / Bijwerken )

Verbinden met %s

Volg

Ontvang elk nieuw bericht direct in je inbox.

%d bloggers like this: