Archive
Archive for June, 2016
At-least-once delivery with Kafka and Spark – Improve the reliability of your job
28 June 2016
Leave a comment
The default behaviour of a Spark-job is losing data when it explodes. This isn’t a bad choice per se, but at my current project we need higher reliability. In this article I’ll talk about at least once delivery with the Spark write ahead log. This article is focussed on Kafka, but can also be applied to other Receivers.
Advertisements