Johnson vs Acronis

The story goes something like this: Originally, we used external hard drives plugged in via USB to our servers to back up our servers. This worked well, except that we had two external drives and 7 servers. About this time I started working this job and immediately set up a MediaWiki website for storing documentation and a Mantis Bug Tracker website for storing information about ongoing projects–I think I’ll try to write a post about that at some point as well. Anyways, these two websites originally ran on my personal webserver in Prosser, but we decided to run them on a company-owned server instead, and that this new machine could act as both a backup server and a webserver for those two websites.

This presented a challenge, however. We chose a different physical location for the backup server to add some level of off-site storage, but we wanted to ensure it was properly encrypted. At first, we tried to use sshfs along with host-keys to authenticate our servers. This worked well for short connections, but due to either sshfs or network misbehavior, this proved completely unreliable for the kind of long-term(weeks long) connections we wanted to have between these machines. This in turn caused backups to fail, which is both highly frustrating and completely unacceptable.

Happily(we thought), Acronis released a new edition of the TrueImage software, Acronis TrueImage Echo. This supported both  AES encryption and FTP storage. While SFTP would have been preferable, we decided that AES encryption along with a very minimal whitelist of IPs allowed to connect to FTP ports would be sufficient. In the process of testing this new setup out, I discovered that Acronis was failing to AES encrypt files transmitted via FTP. This is a major deal since AES is really the only reason that FTP was acceptable at all.

This is where the story gets really interesting. We contacted Acronis about this issue, and after an initial volley of ‘You must be doing something wrong’(a typical Acronis assumption, sadly) and subsequent, ‘Oh, um, heh, can you please send some more log files our way?’ After finally convincing them of the issue(which they did say they had verified) we heard nothing. Nothing for about three months. My old boss emailed them asking what the status was, and we were told the the fixes were forthcoming, though no hard deadlines of any kind were sent, and we have (again) not heard from them in over six weeks. Here is part of their actual response:

We have contacted our developers and they explained that the fix is one of the complex ones to implement. Currently we are doing our best to expedite the development.

According to our development team the case is still open which means the fix is not approved and will not be implemented in the coming update.

After their noncommital answers and our utter frustration, we reported the problem to several security websites, notably Secunia. Secunia immediately picked up on it, and after further description of the problem, set an initial disclosure date of July 23, 2008. Apparently, Acronis failed to respond to them as well, so Secunia emailed us back asking if we’d like to further delay the disclosure or release it immediately. We picked immediately. Secunia released the disclosure a couple of days ago, and it’s now available at the link referenced at the end of this post.

As far as my commentary on this goes, Acronis fails for several reasons here:

1. Failure to design a network backup with the realities of networks in mind. Implementing a backup client where the only options for data backup are FTP and SMB is not merely shameful, it’s irresponsible. Asymmetric key cryptography works, and it works well. SFTP is not an exotic protocol, it’s been a default part on nearly every Linux install for years.
2. Failure to design a robust encryption method. I’m not sure if their AES was tied directly into functions that were only called open writing a file to disk or what the failure here was, but something is completely amiss if one mode of operation correctly encrypts data and another mode fails to. Even if this was the case, an ethical choice without expending really any work is to add a dialog box warning of the situation. A simple, ‘Use of FTP and use of encryption are mutually exclusive, please choose FTP carefully.
3. Failure to identify major issues and deal with them appropriately. Acronis should have issued this advisory to all registered customers immediately upon verifying it. This is an easy enough problem for an administrator to work around, but they have to know about it first. They should have also fixed it in the four months since then. I’m not sure what they’ve been up to here. Saying that a fix is complicated is not an excuse for anything, ever. AES is complex, but it’s already been written, and it’s already been written in spite of it’s complexity. If your software is that complex, that’s your fault, not your paying customers’. If your developers are incapable and incompetent, that’s again not your paying customer’s fault, it’s yours.

In the end, our holdover solution until we can deploy a new solution(read that last part as: Until Travis can convince management to ditch Acronis in spite of the financial expenditure) is to set up a VPN with OpenVPN, only run our FTP on the backup server’s OpenVPN IP address, and completely discourage it’s use to anyone who will listen.

Secunia Advisory 30856