Tuesday, April 16, 2013

Linode disappoints ...

It all started on April 12th with an email from Linode:

Linode administrators have discovered and blocked suspicious activity on the Linode network. This activity appears to have been a coordinated attempt to access the account of one of our customers. This customer is aware of this activity and we have determined its extent and impact. We have found no evidence that any Linode data of any other customer was accessed. In addition, we have found no evidence that payment information of any customer was accessed.

etc … etc

I did not think much of it. After all, if something is on the Internet, it is going to be attacked sooner or later. Heck, I get daily notifications of people trying to access sinan at yahoo dot com and my security logs are full of attempts to log on to my VPS using well known user names. Keep in mind: Don't allow password logins for SSH, and don't copy your private key to the server.

So, yesterday brings some rumblings, including this email from a person I trust:

Subject: Linode's been hacked


Then, this. Then, this.

At the heart of the issue is a chat log and a directory listing of a web server. As I go to bed, there is no real information from Linode.

While a vulnerability that allows someone access to the public directory of a web server would hardly be news, the chat log also includes more serious claims.

This morning brought an update from Linode.

this group gained access to a web server, parts of our source code, and ultimately, our database. We have been working around the clock since discovering this vulnerability. Our investigation reveals that this group did not have access to any other component of the Linode infrastructure, including access to the host machines or any other server or service that runs our infrastructure.

Credit card numbers in our database are stored in encrypted format, using public and private key encryption. The private key is itself encrypted with passphrase encryption and the complex passphrase is not stored electronically. Along with the encrypted credit card, the last four digits are stored in clear text to assist in lookups and for display on things like your Account tab and payment receipt emails. We have no evidence decrypted credit card numbers were obtained.

At this point, I do not fully understand all the ramifications of this breach, but it is making me very uncomfortable.

I have been singing Linode's praises for such a long time that this was a real blow to my image of them. I am not sure how to proceed right now, but the praise is on hold for the moment.

Update: 2013/05/28: I decided to stay with Linode after some consideration.


  1. That can happen to about every company on the web. The difference is how they deal with it and how honest they are about it to their customers.

  2. After reading the news on Linode, I actually immediately thought of you and what are you going to blog next re: Linode. :-)

    1. Yeah, I was feeling so good about them lately. I look at the names people are pushing in the wake of this incident, and I just do not get a good feeling from the alternatives. Any recommendations?

      Right now, I am in wait and see mode. I know everyone wants them to go spill everything, but I can see the need for being cautious and slow. In the mean time, I realize I need to put together a mechanism that will allow me to move from one provider to another without much down time. Right now, if I do that, it will end up being a tedious process.

  3. I'm disappointed because they're posting this on their blog and not emailing customers. I had no idea there was an update until I saw it here.

  4. much depends on the crypto they use for the cc numbers. Do we know which pci-dss compliance level they claim?