Posts Tagged ‘elasticsearch’

Running your elasticsearch integration tests with JUnit 5, Karate and TestContainers (Docker)

Earlier this year I wrote an article on how to run your integration tests with an embedded elasticsearch. When upgrading to elasticsearch 7 this method didn’t work (yet). An alternative (and maybe even better) method is using Testcontainers to run elasticsearch in a Docker container. I will also show how you can leverage Karate to do your integration testing.

Read more…


Running your JUnit 5 integration test with an embedded elasticsearch on a random port (and optionally Spring Boot)

With recent versions of elasticsearch (5+) the learning curve for an integration test became a bit steeper but will result in a cleaner solution in the end. In this article I will describe how to set up your test with JUnit 5 to run your elasticsearch integration tests. I will also discuss how to make it work with Spring-Boot Test.

Read more…

Understanding Spark parameters – A step by step guide to tune your Spark job

15 February 2015 1 comment

After using Spark for a few months we thought we had a pretty good grip on how to use it. The documentation of Spark appeared pretty decent and we had acceptable performance on most jobs. On one Job we kept hitting limits which were much lower than with that Jobs predecessor (Storm). When we did some research we found out we didn’t understand Spark as good as we thought.
My colleague Jethro pointed me to an article by Gerard Maas and I found another great article by Michael Noll. Combined with the Spark docs and some Googlin’ I wrote this article to help you tune your Spark Job. We improved our throughput by 600% (and then the elasticsearch cluster became the new bottle neck)

Read more…

Boosting with wildcards in elasticsearch

While preparing my presentation I discovered that boosting with wildcards wasn’t working. I kind of promised to blog about it, so here it is šŸ™‚

Read more…

Categories: English, java, work Tags:

My NLJUG presentation on elasticsearch

Last thursday I gave a talk about elasticsearch on the Dutch Java User Group conference (JFall) called “Full text search met ElasticSearch in de praktijk”. Yes it’s in Dutch but I still want to share it with you, it might contain some useful tips and tricks.

How sharding in elasticsearch makes scoring a little less accurate and what to do about it

11 September 2013 3 comments

Currently I’m using a small dataset (about 3500 records) on ElasticSearch and saw some strange scoring. Hits that should have exactly the same score had _almost_ the same score. Almost the same is kind of a problem since we sort our data on score and name. After some researching it appeared sharding was the issue here. With only one shard the problem is solved. If you want to know how I found out and what alternatives you have please stay tuned.

updated @ Sep 13: Zachary Tong pointed me to an article with a much better solution (read: the right solution šŸ˜‰ ). I added the second to last paragraph to explain it.

Read more…

Categories: English, java, work Tags: ,

Diacritics in ElasticSearch – Fix it in two places and also watch your encoding while your at it

A few weeks ago I solved an issue with ElasticSearch where some search results with diacritics didn’t show up in the results. At least, I thought I did. There are quite a few places where it can go wrong. In this article I’ll explain these places and how to fix the issues.

Read more…

Categories: English, java, work Tags: ,