How to set up Crashplan Headless on Ubuntu Linux

Problem / Outcome Summary

• This article will show you how to install Crashplan Headless on Ubuntu Linux
  • (To Learn how to Install Crashplan on Ubuntu Linux Desktop (with a GUI) please click here)
• Please see the ‘Summary Overview’ tab below for a high level view of the objectives this ‘howto’ will achieve.

Important Update

17 Jan 2016 – There are currently issues with the new CrashPlan client / server mix for version 4.5.2 in this configuration.  In particular, the guide works, but some hours down the track it breaks.  I am addressing this and will update this guide when complete.

23 Jan 2016 – Fixed.  Please see comments section at bottom of article for details.

Why might I want to do this?

• To backup any data, centralised from your network or specific to your computer to the Crashplan Cloud (this saves money as you don’t need a family plan)
• To create a backup ‘target’ for other computers on your network to backup to
• Because a Crashplan headless (no GUI) style of computer performs better and is more stable

Foreword

To be clear, what does Crashplan Headless do?

Simply put, Crashplan gives you good automatic options for backing up your data.  These can be at a cost, or at no cost depending on which situation suits you best.

What are the current backup features of Crashplan Headless?

Crashplan works on Windows, Mac OS X and Linux.  It also has unofficial Crashplan headless ports for a number of NAS devices including QNAP, Synology and Netgear’s ReadyNAS.

Other notable features of Crashplan include a truly unlimited backup for a very low cost and that it retains any and all deleted versions of your files forever.

Something particularly important in this day and age, is Crashplan’s ability to support client side encryption.  Client side encryption means that Crashplan has an extra level of security than most online backup systems do today.

With client side encryption, Code42 (the Crashplan company) do not hold your decryption key themselves, you do.  Client side encryption makes it nearly impossible for your data to end up being used for purposes other than it was intended because only you know the encryption key.  I say ‘nearly impossible’, because there’s always a chance with technology that someone such as with the now famous NSA / FBI and Apple encryption battle.   However with this feature the data is most definitely safer from e.g. identity theft than it will be on your own hard drive at home. Be warned however,  if you use client side encryption and then lose your key, your backup is useless and you would be unable to retrieve your Crashplan data).

For me, client side encryption is a mandatory requirement – I do not put personal data on the internet such as e.g. Inland Revenue Department or Social Security numbers, banking numbers etc.  Whether you realise it or not, these will likely be on your hard drive somewhere, which means you’re probably backing them up to the internet.

A full comparison list of features for Crashplan can be found here.

Plans

There are three tier’s of Plans available with Crashplan.  Crashplan Home, Crashplan Business and Crashplan Enterprise.

Crashplan Home Options:

• Free: Backup to local assets such as hard drives and computers owned by you
• Free: Backup to remote assets such as friends computers, or computers you own that are connected to the internet
• Not Free: Individual / Family Plans listed here

Crashplan Business / Crashplan Enterprise

Some free options do exist in the Business / Enterprise plans too, specifically a peer to peer option in the enterprise product.

Remember, for a full comparison of all the options, see the Crashplan comparison here.

Please note: Storing your personal information on the internet is highly risky.  Globally, the new generation of internet users have become complacent with persona data such as that submitted to social sharing  sites so far with with little consequence.  However the impacts of this are still being worked out through legal and moral systems.  If you store any data that could be considered sensitive to you, or misinterpreted by others, it is strongly recommended to use a client side encryption key.

Do you REALLY want to pay someone else to backup your data?

As you have seen above, there ARE free options available, within the Crashplan application.  These will work if you set up any kind of Crashplan software including Crashplan headless on Ubuntu.  It is very rare to get such a full featured and reliable application as Crashplan is, especially one that is so simple to use and includes very usable free options.

Tip for OS X customers: Mac OS X customers in particular should note, that Crashplan removes the limitations of the Mac built in backup.  The shipping version of OS X is limited to Apples Time Machine backup only.  Strictly speaking, Time machine is designed only to do a full system backup, or a full user backup.  Crashplan offers a much higher quality of backup (and a free alternative) that allows you to select a smaller subset of backup files (such as your documents folder) and enables you to back it up elsewhere on your network according to the free options above.

The three most likely scenarios requiring recovery of your data are:

1. Accidental deletion of a file or folder
2. Loss of data due to theft or fire
3. Hard Drive Crash

All of these scenarios are covered by CrashPlan free, provided that that scenario 2 is hosted in a remote location that would not be affected by the same fire or theft.  My suggestion is to make a mutually beneficial deal with a friend to use each others spare hard drive space (or buy each other a hard drive to use), then share that in Crashplan so that you have off site backups.  This would actually get you 99% safety.  The other 1% is in the unusual but possible case that your friends computer hard drive dies and e.g. your computer is stolen at the same time.  Not common, but not unheard of.  To mitigate this scenario, you could of course add a third friend to backup to.

So if I can backup for free, why would I pay to backup to the Crashplan cloud?

There are a few reasons for this:

CrashPlan only costs you $5 per month for what is essentially a guaranteed recovery.  However there are some other items that are also worth noting.

1. Restoration speed – Internet connections in countries like New Zealand and Australia, do not commonly have fast upload speeds.  If you want to restore your data from a friends computer en-masse, it will likely be slower than using Crashplan due to their limited upload (typically about 1Mb/s in these countries).  Make sure you check your friends upload speed.  Of course this won’t matter for just a few files, only for many files.
2. Enterprise data security – Crashplan utilise data centers that maintain your data in monitored conditions to protect against various threats such as overheating, security hacks and viruses such as Cryptolocker.  If your data is stolen from your friends house, you would want to be double sure you have client side encryption before they start trolling through your personal photo’s and bank account numbers.
3. Availability – This is potentially the worst one.  Most people switch their computers off at night and this may happen at the time you turn yours on.  This can result in backups not being completed (ever), especially considering how long it can take with the poor upload speed some customers have and the quantity/size of backup data many of us now have on our computers.  Make sure you check the progress indicator to see how your backup is tracking over several days.
4. Unlimited backups – Your friends hard drive space is eventually going to run out, (and so is yours if they are backing up to your computer too).  Crashplan really does offer a truly unlimited backup.

What is the performance like?

The performance will entirely depend on your scenario and also on your client as I found out here.  Certain types of files can be compressed which means that they will upload faster.  If you’re uploading to a friends house, they may have a slow connection, or you may have a slow upload capability.  ADSL customers in NZ and Australia will only have about a 1Mb/s upload speed which would take months to upload a fairly typical 1TB of disk space.  VDSL or fibre customers on the other hand may take days or even weeks for the same data.  One option in Crashplan that does make a difference is the de-duplication option in the advanced settings.  Set this to minimum for more speed if you don’t have a powerful processor (especially if you run client side encryption), although the environmentalist in me says you should leave this alone for reasons I won’t explain here (it uses less hard disk by doing some clever stuff).  That said, make sure you test it out yourself in both scenarios as de-duplication means you don’t have to upload similar data in different files and this could also be of benefit with a slow connection if you have a lot of duplication.

Pre-Requisites

Software Dependencies

• Ubuntu Linux

Hardware Dependencies

• An internet modem / router capable of performing NAT

Tools Required

• SSH Access
• A recent web browser

Other Dependencies

• A working internet connection

High Level Summary Steps

The below lists the high level summary of steps we’re about to take during this howto.

• Download and extract the Crashplan Application
• Install and configure the Crashplan Application
• Connect to your Crashplan Headless on Ubuntu Service

Implementation

Download and extract the Crashplan Application

1. wget https://download2.code42.com/installs/linux/install/CrashPlan/CrashPlan_4.5.2_Linux.tgz
2. sudo apt-get update
3. tar zxvf
4. tar zxvf CrashPlan_4.3.0_Linux.tgz

Install and configure the Crashplan application

1. become root
2. su
3. password
4. cd Crashplan-install/
5. ./install.sh

(If you want to uninstall a previous installation, just type ./uninstall -i /usr/local/crashplan)

Welcome to the Crashplan Installer.
Press enter to continue with installation.

1. Press Enter

Validating environment…
detected root permissions
You must review and agree to the EULA before installation.
Press enter to read the EULA. (Space scrolls quicker)

1. Press Enter

Do you accept and agree to be bound by the EULA? (yes/no)

1. Type ‘Yes’ and press <Enter>

Directory Locations

Press <Enter> on all the defaults below, taking particular note of number 4 as this is where ‘friends’ data is stored (or data from your other computers doing local backups.  You may like to make it somewhere that has plenty of free space.

1. What parent directory do you wish to install CrashPlan into? [/usr/local]
2. /usr/local/crashplan does not exist. Create /usr/local/crashplan? (y/n) [y]
3. What directory do you wish to link the CrashPlan executable to? [/usr/local/bin]
4. What directory do you wish to store backups in? [/usr/local/var/crashplan] /mnt/backups
5. /mnt/backups does not exist. Create /mnt/backups? (y/n) [y] y
6. What directory contains your SYSV init scripts? [/etc/init.d]
7. What directory contains your runlevel init links? [/etc/rc2.d]

The installer then confirms your selections in a list, press y if correct, n if not so that you can do them again.

Your selections:
Crashplan will install to: /usr/local/crashplan
And put links to binaries in: /usr/local/bin
And store datas in: /mnt/backups
Your init.d dir is: /etc/init.d
Your current runlevel directory is: /etc/rc2.d
Is this correct? (y/n) [y] y

The Java Runtime Environment is then downloaded from the internet and uses the locations you have just specified.  Wait for this to complete.
downloading the JRE using wget
–2015-08-02 16:24:48– http://download.code42.com/installs/proserver/jre/jre-7u45-linux-x64.tgz
Resolving download.code42.com (download.code42.com)… 216.17.8.19
Connecting to download.code42.com (download.code42.com)|216.17.8.19|:80… connected.
HTTP request sent, awaiting response… 301 Moved Permanently
Location: https://download2.code42.com/installs/proserver/jre/jre-7u45-linux-x64.tgz [following]
–2015-08-02 16:24:48– https://download2.code42.com/installs/proserver/jre/jre-7u45-linux-x64.tgz
Resolving download2.code42.com (download2.code42.com)… 216.17.8.57
Connecting to download2.code42.com (download2.code42.com)|216.17.8.57|:443… connected.
HTTP request sent, awaiting response… 200 OK
Length: 46843538 (45M) [application/octet-stream]
Saving to: ‘jre-7u45-linux-x64.tgz’
100%[======================================>] 46,843,538 4.16MB/s in 12s
2015-08-02 16:25:01 (3.66 MB/s) – ‘jre-7u45-linux-x64.tgz’ saved [46843538/46843538]
Java Installed.
Unpacking /home/username/CrashPlan-install/./CrashPlan_4.3.0.cpi …
60490 blocks

Also note the number of files that linux will watch.  You will likely have more files than this so you will need to adjust it.
Your Linux system is currently configured to watch 8192 files in real time.
We recommend using a larger value; see the CrashPlan support site for details
Starting CrashPlan Engine … Using standard startup
OK
CrashPlan has been installed and the Service has been started automatically.

If you want to check that CrashPlan is running on your Ubuntu Headless computer, simply enter:

1. cd /usr/local/crashplan/bin (lately changed to /usr/local/bin
2. ./CrashPlanEngine –status

You should see a message saying the CrashPlan engine is running

Note as above if you have a lot of files, you’ll need to update the Java heap size – see CrashPlan’s article on this here.

Also, to update the number of files your linux machine can watch enter the below:

echo 1000000 > /proc/sys/fs/file-max (assuming you have 1,000,000 files to watch in your backup (I do).

Connect to your Crashplan Headless on Ubuntu service

When you install a Crashplan headless system, you’ll want to connect the Crashplan GUI to it, from another computer.  This will allow you to configure your Crashplan backup folders (and other things) easily.  When you’re finished, the changes are saved to the headless computer.

Make sure you have installed the Crashplan application to your local client computer from the Crashplan downloads page.  A guide for doing this in an Ubuntu GUI can be found here.

There is varying information about what you must and must not do to set this up.  Some information suggests you do not need to copy the ‘ui.info’ file, some says you do, some says you need an SSH port redirect, some doesn’t mention it at all.  The truth is I’ve had mixed results.  Often nothing works permanently.  Until recently, I did not need to create an SSH tunnel, then I did and now that’s stopped working again.  This is the way it’s likely to stay while Crashplan oddly insists it’s unsupported but details how to do it at the same time.  Occasionally the ui_info file gets auto detected making people believe it is not needed, but in most cases you will need to do it.  As far as SSH port redirects, try it without, and if you can’t get it to work, try it with the tunneling.

Backup and copy the .ui_info file from the Ubuntu headless computer to the appropriate directory on the local computer (if you’re using mac, you can use the following commands – note be careful with the spaces, if in doubt you can put ” around it).  For windows, use PuTTY, for Linux just use same as Mac method.

Mac Method

On your Mac, backup your .ui_info file

1. mv /Library/Application Support/CrashPlan/.ui_info /Library/Application Support/CrashPlan/.ui_info_original

Copy the Headless .ui_info file to your mac

1. scp username@yourheadlessIP:/var/lib/crashplan/.ui_info /Users/yourmacusername
2. sudo cp /Users/yourmacusername/.ui_info /Library/Application Support/CrashPlan/

(Current .ui_info file locations can be found on the Code42 support page here).

Point your mac Crashplan application to the headless server

1. vi /Applications/CrashPlan.app/Contents/Resources/Java/conf/ui.properties
2. Change the #serviceHost line to the ip address of your headless server, remove the ‘#’ and save the file

If you need ssh tunneling

On a Mac

1. sudo vi /Library/Application Support/CrashPlan/.ui_info
2. change the line that reads #servicePort from 4243 to 4200 and remove the # to uncomment the line
3. ssh -L 4200:localhost:4243 user@yourheadlessipaddress

Start the local Crashplan app and this should now be working.

Final Word

• To date, I have found no reason to change other settings such as port settings in the ui.properties file.  I’m sure some people will have their reasons.
• You can always change the IP address back should you wish to manage the CrashPlan service on your local computer.

So there you have it, a fully functioning Crashplan backup that allows centrally hosted files to be backed up to Crashplan.  This saves money because you don’t need to employ a more expensive licence provided you all save files to a single location on your home Ubuntu Server or NAS.

**Looking for creating a manual backup process?  Why not read our article on how to create a manual backup process and why you need one, here.

Marshalleq