Shellshock Bash Vulnerability has been keeping me busy last few days. Shellshock is a nasty bug that causes problems. Because ePanorama.net is running on Linux server, I has to do the necessary steps to keep the site safe.
First step was to analyze the problem: Does my system have vulnerable bas version? The answer is yes to pretty much every Linux system around (bug was introduced over 20 years ago) unless they have been specifically update for this. So I am pretty sure I have the problem. Running tests told I had the problem.
Does it pose serious danger is next question? There are several attack surfaces. The SSH attack surface is not very dangerous in my case, because using it needs the attacker to have login credentials for the system. There are very few login credentials in use and they have already quite much rights, so user trying to use to bug to get more rights is not a serious thread to worry much about. The more serious is HTTP attack surface, where anyone could include bash commands to HTTP headers… This site still uses some services implemented with CGI-BIN interface and bash (some of them put originally on-line 15 years). This attack vector needed something to be done quickly. I verified with CVE-2014-6271/CVE-2014-7169 test page that I really has the problem.
The alternatives to solve the problem are fixing the bash to a working version and/or using some form of protection that blocks dangerous request from getting into the system. So I first checked if the problem could be easily solved out just by running Linux update. In this case this did not work that easily.
So I had to try the other option, which was to block the potentially dangerous traffic. Red Hat article Bash Code Injection Vulnerability via Specially Crafted Environment Variables (CVE-2014-6271, CVE-2014-7169) article give instructions how to block potentially dangerous traffic using mod_security and IPtables. I started with the simple IPtables trick, it worked but caused some collateral damage blocking something that should get through. That was the first cure, the benefit to damages ratio was acceptable.
This give some time to work on real fix, which was to update bash. To do that I needed to figure out the best way to get the new bash version and make sure things work well with it. I ended up compiling it from the source code similarly to methods described at https://shellshocker.net/. Then some testing with the new bash version. The services turned to work well with it. Then some manual hacking to get it replace all the ways the old bash could be called. And then testing that there is no issues with new bash. Things worked well. So now I can again run the system without that problematic firewall rule to block potentially dangerous HTTP requests.
Things are working well again. The plus-side of this whole story is that I have learned more on bash, IPtables and mod_security. Things have worked pretty well despite all the work going on (not any serious problems). On the weekend there was also a short service outage due Amazon’s massive messy EC2 reboot.