Archive

Posts Tagged ‘classloading’

How to remotely reload classes with Spring Developer Tools without opening extra ports

With Spring Developer Tools it is possible to reload classes on a local or remote machine from within your IDE without using JPDA. In this article I’ll show you how to do this and how to prevent this from happening on your production environment.

Read more…

Advertisements

HTTP/2 Server Push with JBoss undertow – And how to monitor things

With HTTP/2 it’s possible to deliver data at a client before the client even asks for the data. This will significantly improve latency and perceived download speed. Last hack day at JPoint we spent some time with HTTP/2 server push. I will show you how to get it running on your machine and show some tools to monitor/prove all this goodness.

Read more…

Categories: English, java, work Tags: ,

How to fix the classpath in Spark – Rebuilding Spark to get rid of old jars

After fixing an earlier classpath problem a new and more difficult problem showed up. Again the evil-doer is and ancient version of Jackson.
Unfortunately all classes are loaded from one big container with Spark/Hadoop/Yarn, this causes a lot of problems. The spark.files.userClassPathFirst option is still ‘experimental’ which, in our case, meant it just didn’t work. But again, we found a solution. Our system engineers also wanted a reproducible solution so in the end it’s just a small recipe.

Read more…

Categories: English, java, work Tags: , , , ,

Using a Spring Context within Hadoop Mappers and Reducers with environment specific property files

Bootstrapping a Spring Context from the classpath is quite simple, even from Mappers/Reducers in Hadoop. When you want to use enviroment specific property files it gets a bit harder. In our Mappers and Reducers we rely heavily on Spring Beans who perform the magic. Besides that we load different properties per environment (ie a MongoDB). After several iterations we came up with a nice solution. You can also leave out the Spring part and use this solution to broadcast Properties to all your Mappers/Reducers.

Read more…

Overriding Hadoop jars with MapReduce v2 – How to fix the classpath

We ran into a nasty problem with Hadoop when we wanted to use a ‘new’ version of Jackson. Hadoop decided to throw a NoSuchMethodError. It appeared Hadoop uses an ancient version of Jackson. Some terrible memories of classloading and JBoss EAP came into my head. After I was calmed down by some colleagues I ultimately found a solution.

Read more…

Categories: English, java, work Tags: , , ,