Background
So this personal project began when I was looking for a home remote (aka offsite) backup solution that was flexible and cheap. What ever hardware I was looking for had to also double as my home NAS and triple as my simple linux box; in the that order (first being most important third being least). I started by hunting for the cheapest computer I could buy that was brand new. I looked at Rasberry Pi, CuBox, Gumstix, Adruino, PandaBoard, PogoPlug, SheevaPlug, Drobos, ReadyNas and a few more. While searching for a small embedded system computer I focused on the caveat that I wanted to run Ubuntu as my Operating System (OS). I love Ubuntu’s package management and community support, specifically for ARM, which is the type of processor integrated in most of these embedded systems. Which one to get? Rasberry Pi was most appealing to me because of price, and CuBox for performance/formfactor, but neither where ‘to-market’ in full production force, which always makes me leery about purchasing a product. By chance one of my coworkers, who is a college senior focused on embedded systems, had a perfect device that I could borrow, play with, and test; a BeagleBone! I then spent some time and got Ubuntu 11.04 and 12.04 LTS running on the BeagleBone.
Now with a device and OS, I was ready to focus on how to backup my data. I needed to get an offsite back going but I wanted to accomplish this cheaply, safely, securely and reliably. This part has caveats to, the first being cost and the second being trust. I personally do not trust ‘the cloud’, meaning I don’t trust other companies with my personal private data (such as Carbonite, Amazon, Mozy or any other online backup solution). Eventually I think I will begin to trust ‘the cloud’ but not this year or next! 🙂 On the flip side I do have some good friends that I trust much more than a big company so I was content figuring a solution to push data to their houses. I had three ideas/approaches to this, all which required some significant work.
1. Write a daemon based on Rsync
2. Write a daemon based on Duplicity
3. Compile CrashPlan application to run on the BeagleBone
I am somewhat comfortable writing small simple daemons in bash scripting and thought I would use Rsync to do the job since I was familiar with it. Problem was it did not satisfy my ‘security sense’ so I then turned to Duplicity for the basis of my daemon. Wrote the daemon and while testing it I noticed that Duplicity did some things to eat storage space that I didn’t want to live with. Noticed some other people had similar, unfixed issues while exercising my ‘google-fu’ so I dropped pursuing a Duplicity based daemon and decided to jump in on idea number three. I like CrashPlan because they have commercial support, a robust ‘veted’ product and most of all, a free product that lets you utilize friends as backup destinations. Therefore I chose to roll with it.
I found a lot of write-ups on how to get crashplan running on other ARM systems but none specific to the BeaglBone, so I hope this helps someone out there. Also I am by no means a programmer or linux OS developer so I mainly referenced two significant sources, cited here and here, to help me accomplish the compiling and detailed configuration. Nuf credit to dem, BigUp!
Enough background blab, here is the step by step guide to getting CrashPlan installed and working on the BeagleBone (aka on Ubuntu 12.04 for ARM).
Installation Guide/Walkthrough
Please check out my quick-guide if you just want a condensed set of all commands and configuration file changes; else brief explanation of steps are below.
This guide is based on installing CrashPlan 3.2, which automatically updates to the latest version. I installed this on a clean installation of Ubuntu 12.04 LTS. I assume you have Ubuntu running on the BeagleBone and you’re logged in. If not check out this site to help you get Ubuntu up and going on your BeagleBone.
note: all commands are run in the shell and I begin running all commands in my user’s home directory (~/). My user is ‘ubuntu’ therefore my home directory is /home/ubuntu/
First thing we need to do is install Java JRE, JDK and JNI Libaries:
sudo apt-get install openjdk-6-jre-headless openjdk-6-jdk libjna-java
I tried java 7 and ran into issues, so I would stick with the above, but if you can get it going more power to yuh!
Next download and install CrashPlan with the following selections:
CrashPlan will install to: /opt/crashplan
And put links to binaries in: /usr/local/bin
And store datas in: /opt/crashplan/manifest
Your init.d dir is: /etc/init.d
Your current runlevel directory is: /etc/rc2.d
wget http://download.crashplan.com/installs/linux/install/CrashPlan/CrashPlan_3.2_Linux.tgz
tar xfzv CrashPlan_3.2_Linux.tgz
cd CrashPlan-install
sudo ./install.sh
After you have finished going through the install script CrashPlan will have claimed to start, but in reality it doesn’t. CrashPlan does not start because the install only comes with libraries compiled for x86 based systems and we are running on an ARM based system. Therefore the java virtual machine pukes when trying to load a couple essential libraries. These libraries are JTUX and MD5. We will compile each of these.
Before we start compiling make sure the following packages are installed by running the following commands:
sudo apt-get install make gcc unzip gunzip
Download the patch and source code for JTUX and run the following commands. I found this patch file on a number of other sources for multiple ARM based CrashPlan installs, so I tried it on the BeagleBone and it worked!
cd ~/
wget http://forum.excito.net/download/file.php?id=145 -O jtux.PS3-YDL6.1.patch.txt.gz
gunzip jtux.PS3-YDL6.1.patch.txt.gz
wget http://basepath.com/aup/jtux/jtux.tar.gz
tar xfzv jtux.tar.gz
cd jtux
patch < ~/jtux.PS3-YDL6.1.patch.txt
vi Makefile
Find the line in the MakeFile and update/replace the path as shown in the following:
JAVA_INCLUDE=/usr/lib/jvm/java-6-openjdk-armhf/include/
Compile JTUX, move the newly compiled .so file into the CrashPlan directory and modify its permissions by running the following commands. note: this overwrites the existing x86 compiled .so file.
make
sudo mv ./libjtux.so /opt/crashplan/libjtux.so
sudo chmod 644 /opt/crashplan/libjtux.so
Now time to compile the MD5 library. Download source and run the following commands to compile, move the new .so file to the CrashPlan directory and modified permissions.
cd ~/
wget ftp://sourceware.org/pub/libffi/libffi-3.0.10.tar.gz
tar zxf libffi-3.0.10.tar.gz
cd libffi-3.0.10
CFLAGS="$CFLAGS -O3" ./configure --host=arm-none-linux-gnueabi --prefix=$DEST --disable-static
make clean && make
export FFI=`pwd`
cd ..
wget http://www.twmacinta.com/myjava/fast-md5-2.7.1.zip
unzip fast-md5-2.7.1.zip
cd fast-md5
gcc -O3 -shared -fPIC -I. -I/usr/lib/jvm/java-6-openjdk-armhf/include src/lib/arch/linux_x86/MD5.c -o libmd5.so
sudo mv /opt/crashplan/libmd5.so /opt/crashplan/libmd5.so.bak
sudo cp ./libmd5.so /opt/crashplan/
sudo chmod 644 /opt/crashplan/libmd5.so
That’s it! CrashPlan can now be started and will run as the root user and should also automatically start every time the system boots. If you are content consider yourself done with this guide and run the following command to start CrashPlan (the blue pill). But, if you want to see some other niceties I’ve implemented stay in wonderland and I’ll show you how deep the rabbit hole goes (the red pill).
sudo service crashplan start
Optional Setup and Config ….aka red pill
Your still reading! Way to enter the Matrix! So as I mentioned above in the background I wanted this solution to first be my backup solution but also wanted it to double as my home NAS. So this is the part where I do some stuff to make ‘my backup solution’ more ‘resilient’ and configure the device to be my home NAS. Some of these specifics/ideas came from the references I cited in the background and others are just my doing. Here is a summery of additional setup that I found useful.
1. Configure CrashPlan to run as a none root user
2. Configure your system to watch more files in the files system (for CrashPlan)
3. Install and setup AutoFS for auto mounting your external storage device
4. Configure your system so that it updates its time regularly with NTP
Configure CrashPlan to run as a none root user
CrashPlan is a java program that automatically updates itself. This is a good and bad thing. Good from the fact that you will always be running the latest version and that you do not have to manually update the program. However this is bad from the point of view that you allow another entity to install and run whatever code they want on your box. I am very trusting to let CrashPlan do this, but I would like to limit them as much as possible; which is why i do not run the program as the root user (which is what happens by default). Here is how to do it.
First change the ownership of the crashplan application files to the specific username that you want crashplan to run as (in my case the user is ubuntu).
sudo chown -R ubuntu:ubuntu /opt/crashplan/
Next edit the crashplan start up script. So that it starts the application as the user specified.
sudo vi /etc/init.d/crashplan
Take all the instances (lines) where it says “$SCRIPTNAME start”, “$SCRIPTNAME stop” etc. (five of them) and modify them so that they read like below.
#!/bin/sh
# Linux Standard Base comments
### BEGIN INIT INFO
# Provides: CrashPlanEngine
# Required-Start: $local_fs $network $remote_fs
# Required-Stop: $local_fs $network $remote_fs
# Should-Start:
# Should-Stop:
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: CrashPlan Engine
# Description: CrashPlan backup service engine
### END INIT INFO
#############################################################
# Init script for CrashPlanEngine
#############################################################
# Defaults
SCRIPTNAME=/opt/crashplan/bin/CrashPlanEngine
case "$1" in
start)
su ubuntu -c "$SCRIPTNAME start"
;;
stop)
su ubuntu -c "$SCRIPTNAME stop"
;;
restart)
su ubuntu -c "$SCRIPTNAME restart"
;;
force-reload)
su ubuntu -c "$SCRIPTNAME force-reload"
;;
status)
su ubuntu -c "$SCRIPTNAME status"
;;
*)
echo "Usage: $0 <start|stop|restart|force-reload|status>" >&2
exit 3
;;
esac
exit 0
And you’ re done. Kill and start crashplan (or reboot) and crashplan should start as the specified user.
Configure your system to watch more files in the files system (for CrashPlan)
This allows for your system to be more comprehensive about file changes, ultimately increasing the accuracy of crashplan backups. Edit the following file.
sudo vi /etc/sysctl.conf
Add the following line towards the end of the file and save (I believe the default value is 8192).
fs.inotify.max_user_watches=1048576
Done.
Install and setup AutoFS for auto mounting your external storage device
TBD
Configure your system so that it updates its time regularly with NTP
Any good system needs accurate time so that logging, troubleshooting, syncronization and post analysis can be done. This sets up a daily cron job for the quick ntp update command to run. Create the following file.
sudo vi /etc/cron.daily/ntpdate
Places this in file.
#!/bin/bash
ntpdate ntp.ubuntu.com
Make it executible.
sudo chmod 755 /etc/cron.daily/ntpdate
Run the below line and select the timezone.
sudo dpkg-reconfigure tzdata
Done. Your system should ntp sync its time daily with public Ubuntu internet time servers.
Conclusion
I hope this guide helps someone out there. I felt that I have used so many online resources it is about time I started contributing back to the online community. Hopefully this has a little impact. I think this is a good, cheap, robust remote data back up solution for anyone with a decent amount of data. If nothing at lease it will serve as a reference to me in the future.
crashbone 