As a trusted security partner, our phones have been blowing up the past 24 hours with clients calling to ask us about the Heartbleed bug found in the OpenSSL library. It’s been all over the news, and some of the brightest security minds out there are throwing around really scary words like “catastrophic” and “doomsday”. We’ve been delving into the details the last few days, and working in cooperation with our friends at Codenomicon, the security vendor that discovered the bug.
Short version? There’s no doubt this is a serious vulnerability, and it’s incredibly widespread, both of which do make this a scary one. Still, there are facts and then there’s hyperbole, so let’s see if we can boil down some facts.
We’ve studied the problem, we’ve consulted with other security experts, we’ve gone through the remediation process ourselves, and we’ve worked with several clients to help them fix their own sites, so hopefully we can clear up some of the noise, and tell you what you really need to know in a simple, concise form. Without further ado, here is our take on “The Heartbleed Bug – what you really need to know.”
What is it?
The Heartbleed bug is an information disclosure vulnerability in the OpenSSL library, which is used by many web servers for SSL (secure socket layer) capability. OpenSSL is very popular, and it’s used to secure much of the Internet’s traffic. SSL is used to encrypt sensitive data in transit and is most familiar to non-technical users as the source of the padlock or green address bar that users see when conducting transactions over the web.
OpenSSL is also used in a lot of other places you might not think of – such as medical devices, automotive software, and critical infrastructure systems. Although the Heartbleed vulnerability was publicly announced on April 7, the affected OpenSSL versions have been in use for more than two years, which means there’s the distinct possibility that it’s already been exploited by malicious hackers. While many large companies do not use OpenSSL for their primary website, as this list demonstrates, they may still use it elsewhere within their organization.
Most competent system administrators already know of the issue, have determined if there was a risk to their system, and implemented patches to fix the issue yesterday. Most will also generate new SSL Certificates and revoke their old ones just to be on the safe side.
What systems does it affect?
- The two major web servers affected are Apache and NGINX, but the problem can surface in any service that incorporates the OpenSSL library.
- Technical details: The versions of OpenSSL affected are in the 1.0.1 branch. Major services besides HTTPS that utilize this library are the STARTTLS part of the SMTP email protocol, the POP3S email protocol, and IMAPS email protocol among others.
In (mostly) plain English, how does it work?
- When an attacker exploits this vulnerability, the web server sends 64k of data from its private memory area to the attacker over the network.
- Since the 64k of data comes from the server’s private memory space, it can contain the private key for the SSL cert, a password, or a session key.
- Since all SSL keys in use today are less that 1k, this disclosure will eventually find the private key if run repeatedly.
What’s the danger?
There are several major concerns:
- The most immediate is that the flaw could enable attackers to compromise usernames and passwords, as possession of the key would allow a hacker to decrypt any sensitive data in transit.
- The bigger concern is that if the server is not properly configured for something called “perfect forward security”, this private key can also be used to decrypt previous (e.g., logged) traffic. The theft of the SSL keys and certificate information from servers could make it easier for bad actors to launch man-in-the-middle attacks and phishing sites (when combined with DNS spoofing).
Yikes! What do I do to fix it?! (This gets a little nerdy…)
The remediation process is as follows:
- Inventory all systems running OpenSSL version 1.0.1 and newer.
- Patch the OpenSSL library OR recompile it with the heartbeat feature disabled.
- Verify that the system is patched using a vulnerability scanner.
- Rekey all the SSL certs on affected servers and revoke the old certs. The reason for rekeying and revoking the old certificate is to prevent the attacker from continuing to decrypt traffic even after the vulnerability is patched.
- Implement perfect forward secrecy on all servers that don’t already have it.
- Change all secondary key material including passwords, session cookies, and session keys for the site or service protected by SSL.
- Have the remediation process independently validated.
Codenomicon says Fixed OpenSSL has been released and should be deployed now across websites vulnerable to the bug. They also advise operating system and appliance vendors – as well as independent software vendors – to remediate the problem as soon as possible.
Longer term, once you’ve verified that the remediation process was properly completed, you may wish to consider employing a third-party monitoring solution (if you’re not already) to look for any indicators of confidential or critical data being published or traded online by unauthorized sources, particularly if you’re in the financial services industry or critical infrastructure.
Finally, consider the possibility that you’ve already been breached as part of your Incident Response plans, and develop messaging to notify customers accordingly in the event that you have.
So, what should I communicate to my customers now (or anyone else we deal with who might be impacted)?
The answer depends on if you’re using OpenSSL, the type of system you have and the sensitivity of the data it is protecting. That being said, we think it’s a good idea to make it clear that if your organization was vulnerable, the vulnerability is patched and the possibly disclosed key is no longer valid.
Although we haven’t seen any indications of data being leaked or exploited yet, it doesn’t mean that it hasn’t happened. The flaw has been around for nearly two years, and there is no easy way to detect a compromise. As security researchers have demonstrated in the past 24 hours with tests on thousands of websites that use OpenSSL, the threat is very real. The bigger risk is that encrypted traffic may have already been compromised in the past few years without the knowledge of website or application administrators.
If you are using OpenSSL and your organization relies on usernames and passwords to collect or provide access to information or conduct transactions, let customers or partners know what you’ve done to remediate the risk and suggest they change their usernames and passwords. Here is a good example from LastPass that explains why customers are not at risk and what steps they’ve taken to address this issue, and another from Drupal.org. If you’re sending a “Heartbleed” password reset email, please don’t include a login link.
Depending on the nature of your business, you may want to do a force password reset. Soundcloud, for example, posted a notice this evening that reads, “Due to the ‘Heartbleed’ security vulnerability, we’ll be signing everyone out of their accounts later on today. Please check that you know your sign-in details, you’ll need them to access your account.”
Wait, I need more information!
Codenomicon has set up an excellent website, Heartbleed.com, which details the exploit and offers answers to a number of other frequently asked questions.
If you have questions about how to remediate the problem or would like additional information on monitoring services for finding leaked data, we invite you to get in touch with your account manager, or contact our experts.