Handling the Heartbleed Bug

So when news of the Heartbleed Bug hit the tech news sites and security bulletins I decided to read up on it when I got home from work and go from there. While it’s true that some news sites aimed at the general public have gone rather over the top about this and in many cases got their facts and recommendations plain wrong, my evening reading was not that enjoyable – this is indeed a very serious bug and is likely to affect everyone in some way shape or form. Note that I said affect, and not compromise – this is important – there was and still is no evidence that the bug was being exploited in the wild.

For those with a technical inclination, the register has done a nice little round up of how the bug works. The non technical among you may find this heartbleed cartoon illustration over on XKCD more useful.

Anyway, I thought I’d share with you the steps I have taken and will be taking going forward to help mitigate the effects of the bug. I’ve laid out the steps in order, note that I’ve focused on external services first – this was deliberate – no sense in using external services to help me fix my own infrastructure if they were not patched yet.

External Infrastructure

  • Verified with my hosting company that their servers were not affected. Xilo confirmed to all customers that their OpenSSL version was prior to the appearance of the bug. This ensured all keys stored in their infrastructure and used by me are secure
  • Ensured that my cloud based password management system was unaffected. LastPass were very open and detailed with their customers in explaining that despite their servers having been affected and subsequently patched and their certificates re-issued, a combination of passwords never being sent to their systems without an additional layer of encryption to which they never see the key and Perfect Forward Secrecy, customer data was and remains secure.
  • Verified that my certificate provider’s online portal is currently patched and protected. StartSSL have confirmed that their infrastructure was never vulnerable to the attack.

Own Infrastructure

  • Patched all servers I administer that were vulnerable – check yours here

    apt-get update
    apt-get upgrade

  • Re-generated all SSH host keys on vulnerable hosts (remember to leave the pass phrase blank)

    ssh-keygen -t dsa -f /etc/ssh/ssh_host_dsa_key
    ssh-keygen -t rsa -f /etc/ssh/ssh_host_rsa_key

  • Revoked certificates that were in use on previously vulnerable hosts. Some providers may charge for this but you’re not fully secure without it!
  • Re-issued certificates that had been revoked using new keys. Generate these and CSR as follows

    openssl genrsa -out server.key 4096
    openssl req -new -key server.key -out server.csr

  • Restarted all services using OpenSSL. In many cases this will be done by the package manager during the upgrade process but if you’ve changed keys etc. a restart will be needed to pick this up

    /etc/init.d/ssh restart
    /etc/init.d/apache2 restart

Passwords

  • Given that changing a password prior to a site admin both patching the server and revoking/re-issuing their certificate is a pointless exercise, I plan on relying on a combination of the LastPass checker and conversations with web masters directly about if I should change passwords or not. I’m expecting a large number to need changing but hopefully it can be done at a relaxed pace over the next few weeks as and when news becomes available regarding the vulnerability or otherwise of a site.

Finance

  • Confirmed, via the LastPass checker, that PayPal have never been vulnerable due to the software they use for SSL
  • While it’s possible that credit card numbers, bank details and indeed other personal details may have been leaked via the bug, I deemed that replacing bank cards etc. was overkill. After all most banks offer fraud cover and my subscription to credit report monitoring includes tools to help manage any increased risk of identity theft. In any case, many personal details sent in this way are not easily changed, in fact only a credit card number can really be replaced – bank details, national insurance number, address etc. are all set in stone.

Hopefully this overview is helpful to those who are confused!

Comments (2)    

Cannot update WordPress post

I came across an old post on my blog the other day that was getting a lot of hits. When I viewed the page I decided that the code snippet in the article could do with an increase in text size to make it more readable.

I went into the admin panel and added an in-line style attribute to the code tag in question. When I saved the page WordPress reported that the post had been updated, however the text in the editor screen reverted back to what it had been prior to my change. I tried numerous times to save and tried different browsers but try as I might, the change just wouldn’t “stick”.

I usually consider myself pretty adept at solving WordPress related issues but this one was stumping me so I did what all developers do when they are in a tight spot and I googled it. Most posts turned up were completely unrelated, but one article suggested that the issue may be symptomatic of a corrupted table.

A quick command as the MySQL root user had my answer and effected the fix into the bargain

[kieran@www:~/public_html]$ mysqlcheck --check --databases wordpress --tables wp_posts -u root -p
Enter password:
wordpress.wp_posts
warning : Table is marked as crashed
error : Key in wrong position at page 132096
error : Corrupt

I re-ran the same command to make sure it had effected a fix (it isn’t clear from the output)

[kieran@www:~/public_html]$ mysqlcheck --check --databases wordpress --tables wp_posts -u root -p
Enter password:
wordpress.wp_posts OK

Now updating the offending post works first time without issue. I suspect, although can’t be sure, that the corruption occurred during a recent upgrade of the server when not all the services, MySQL included, got shut down gracefully.

Comments (1)    

SecurityException in Application

When carrying out a recent upgrade of suPHP I encountered the following error in my apache error logs and PHP pages were not being served.

SecurityException in Application.cpp:511: Unknown Interpreter: php

Turns out the fix is a simple one; the syntax of the config file has change between versions and it’s simple a case of replacing the following lines:

[handlers]
;Handler for php-scripts
x-httpd-php=php:/usr/bin/php-cgi

;Handler for CGI-scripts
x-suphp-cgi=execute:!self

With the following (note the addition of double quotes)

[handlers]
;Handler for php-scripts
x-httpd-php="php:/usr/bin/php-cgi"

;Handler for CGI-scripts
x-suphp-cgi="execute:!self"

A quick bounce of apache and you should be back and kicking

Comments off    

Removing kernel from OpenVZ Container

When I recently upgraded an OpenVZ container in-situ over SSH, I had a few issues with the upgrade path. One of these was that the upgrade process called for the installation of a new kernel, however a container doesn’t need one and in my case couldn’t handle installing one either and thus caused the upgrade process to error out.

The fix was to force remove the kernel, both from the dpkg directory structure (in this case I copied the files to /tmp/ just in case) and using dpkg it’s self. The following commands as root:

mv /var/lib/dpkg/info/linux-image-3.2.0-58-generic.* /tmp/
dpkg --remove --force-remove-reinstreq linux-image-3.2.0-58-generic

Resuming the dist-upgrade after this process was a success

Comments off    

Family Christmas 2013

Family Christmas 2013

A collection of snaps from yet another fun-filled family Christmas

Comments off    

Narita Express

On a recent trip to Japan we were very nearly caught out by a timetabling anomaly for the Narita Express airport train service from Shinjuku so I thought I’d share it on the blog to save others some of the anxiety of arriving so close to check in closure time!

Basically the frequency of the service depends on the time of day. Up to and including 8am, the service runs almost every 20 minutes (great). After 10am it’s roughly every hour until nearly 8pm (not so great but manageable with planning). However from 8am there is nearly a 2 hour gap until the next service at 9:40am! This means that if your flight is at 11am or earlier, you must catch the 8am train or resort to other means of getting to the airport (including taking the Yamanote line to Tokyo and picking up the express from there where services are more frequent).

In short, either check the timetable carefully for your chosen boarding point taking into account your flight time or elect to always travel from Tokyo station rather than any of the other possible starting points as there are always regular services from this station.

A cursory search on Hyperdia would also seem to indicate that taking the Skyliner service from Nippori, accessed from Shinjuku via the Yamanote line, is quicker than the Narita Express anyway, and more frequent, providing an almost consistent 20 minute interval departure time throughout the day with an end to end journey time of just over an hour, compared with the express which is over an hour and a half door to door.

Comments off    

Joy of full width

Screens are getting bigger, but specifically they are getting wider. This realisation has, over time, been nagging at me and got me thinking about the idea of a serious overhaul for this site. Despite trying a number of different options over the past month I decided that perhaps it wasn’t the radical that was required, but the more practical.

Behold then, the same fundamental design but sporting wider pages, more flowing text, larger images and more tweaks to come. Those with wide screens or even just plain old rectangular ones should be rather pleased – do let me know your thoughts in comments.

Comments (2)    

Venice

Venice

A short summer jaunt to the picturesque Italian city of Venice

Comments (1)    

Romania

Romania

A flying visit to Romania, taking in some of the sights in and around Bucharest

Comments off    

Four80East – Off Duty

Four80East - Off Duty Album CoverHot off the press and into my new albums playlist on spotify this week, Four80East’s latest offering, Off Duty. Frequently blasting out of my car speakers as I embark on another long trip somewhere, Four80East have once again made an easy home in my hi-fi. Their unmistakable style still shining strong and true but the fresh nature of the music on this pressing simply can’t be ignored.

After a few minutes on the turntable I can’t help but draw distinct parallels with Nocturnal, the sultry undertones and softer rhythm section lending its self strongly to that laid back, late night lounge listening with a long tall one and a summer breeze drifting across the terrace.

It’s not all mellow grooves though; the first track on the album, The Walker, brings in a distinct crisp modern edge on the lyrics side of things and Nothing is Written, a track around the mid section of the album, has a distinctly Eastern fusion flavour to it which I feel is certainly a new angle by the group but a foray which seems to have paid off.

It’s rare I find myself consistently impressed by a Jazz group’s offerings over a number of consecutive years but I do believe that Four80East are strongly bucking the trend. For fans who remember their sounds in the noughties I’d recommend checking out tracks from this album “Cashed Out” and “The Walker”, while those seeking something a little more up to date from the group will thoroughly enjoy “Sandbar” and “Gare Du Nord”.

A welcome addition to any jazz-head’s collection, a purchase of this album won’t disappoint.

Comments (4)    

Next entries » | « Previous entries