TL;DR I made a map displaying weather and traffic road conditions for Spain that is easier to use, nicer and faster than the official Spanish Government map. As usual (1, 2), I keep being disappointed with the Spanish Government Open Data policies. You can check out the map here.…
Airflow is one of the most widely used Schedulers currently in the tech industry. Initially developed at Airbnb, a few years ago it became an Apache foundation project, quickly becoming one of the foundation top projects. It is a direct competitor of other schedulers such as Spotify's Luigi or newer…
These are the steps I took to install Microsoft's cool Gradient Boosted Models library, LightGBM Step 1. Install CUDA I am not going to explain this step because it is easy to find. Step 2. Install Boost sudo apt-get install libboost-all-dev Step 3. Clone LightGBM and build with CUDA enabled…
If you run pyspark and see this error (it happens in scala-shell as well): Error while instantiating ‘org.apache.spark.sql.hive.HiveSessionState’ The solution is easy, yet ridiculous. 1. Create the folder /tmp/hive 2. Give it chmod permissions sudo chmod -R 777 /tmp/hive Found here…
Yesterday I spent a few hours dealing with what I like to call "the edges of StackOverflow". By that I mean those situations in which you are trying to solve a programming problem (mostly a bug) and you have no idea why its happening, and even worse, no amount of…