Left, side of this chart, VSZ (virtual memory) and RSS (RAM) over time (obtained via ps) for a process using poor implementation of KafkaClient in java, which is creating a new kafka client per GET request. This is bad. Right side of the chart, current performance once I fixed the…
Very quick note for future reference. Please ignore. Change loglevel in spark Easy peasy, you can do it programatically in the application like: spark.sparkContext.setLogLevel("WARN") Change loglevel in yarn This one took a while to find, you can just run spark-submit while previously exporting this envvar: export YARN_…
Took me a while to figure it out, but it seems that in golang you cant re-read from an http response. I found here a way to solve it. For debugging purposes, I had to be able to print the raw response as well as decoding it to json, to…
A few weeks ago I read an article in which Timo Grossenbacher showed how he managed to plot, in my opinion, one of the most beautiful maps I have ever seen. So I went and tried to replicate it. First of all, here is the map. As usual, you can…
This post serves two purposes; First, it will hopefully get indexed by google and will help future US citizens living in Spain. Second, it will be yet another example of how bad things are in Spain regarding Open Data, Open Access and Transparency. For those that come looking for that…