50gb of “cloud” space with Box, automatically sync’d on Ubuntu/Linux with webdav and unison

Box (used to be box.net) is an online storage service that’s been around for a while – it’s quite popular and gives you more storage space than Dropbox usually does. And, if you’ve got an Android or iPhone, getting the mobile Box app (for Box.com) unlocks 50gb of storage immediately, for free. 50GB of storage. Unfortunately, it doesn’t have a linux syncing client yet – wouldn’t it be nice if you could create your own?

Contents

• Intro
• Signup for Box!
• Mount the Box folder locally using webdav
• Setup Unison to keep a local folder and your Box folder in sync
• Create a cron job to run unison automagically

Box supports webdav; webdav is an extension to HTTP that allows handling of remote filesystems – thankfully, there are a number of packages in linux available to let you mount these webdav folders locally and have them behave as local folders. However, performance on these usually is a bit slow (each read/write involves a few round-trips to the web server) so what we’ll get here is:

• a locally mounted webdav folder that will have the “live” contents of what Box has, but quite slow
• a local copy of the contents of the webdav folder that gets kept in sync with the online Box copy every, or automatically when you change a file – but because it’s a local copy, it’s no different to normal files

(Anywhere I’ve used vim, feel free to use your editor of choice – just replace it with something like gedit if you’d rather a gui)

Sign up for Box!

So, first up, you’ll need a Box account.

Just visit box.com to create one, or signup from your mobile (then you get 50gb space from the getgo!). Once you’ve got your account all set up and confirmed, continue!

Mount the Box folder locally using webdav

Next we need to mount the Box web folder as a webdav folder – the davfs2 package will do the job:

sudo apt-get install davfs2

Now create somewhere for the webdav folder to mount:

sudo mkdir /media/box.net

Then we’ll add an entry in fstab (the file that Ubuntu reads to work out what’s mountable)

sudo vim /etc/fstab

and put this in it at the end:

https://www.box.net/dav /media/box.net davfs defaults,rw,user,noauto 0 0

Now we’ll change the davfs config so that users in the users group can mount the webdav share whenever they need, and also to disable the use of file locks – they don’t seem to work properly with the Box.com webdav service

sudo vim /etc/davfs2/davfs2.conf

and put this in it (there might already be something similar to this somewhere in the file – just replace it with this):

dav_group users
use_locks 0

Next, we need to add our user to the users group – (my user’s called seb, replace seb with you obviously!)

sudo addgroup seb users

Now we need to store our Box.com username and password so that it doesn’t prompt every time we try and use it:

sudo vim /etc/davfs2/secrets

and put this at the end:

https://www.box.net/dav <your_user> <your_password>

At this point, everything should be ready to test the webdav setup; run this to mount it:

mount /media/box.net

If that worked, then you should be able to open a filemanager (like nautilus or thunar) and visit /media/box.net and view the contents of your Box account! However, you might notice that it’s really not very responsive – and sometimes freezes up the file manager… which is what the next section will fix.

Setup Unison to keep a local folder and your Box folder in sync

Unison is a 2-way synchronization tool that can do all sorts of powerful things; here though we’re just going to use it to keep 2 folders in sync with each other – I’m going to put my local Box folder in /home/seb/box.net and keep it in sync with the existing /media/box.net. So first make sure you’ve got the box.net folder:

cd ~
mkdir box.net

Now you need to install unison (if you’ve not got it already) then run unison once to create the .unison config folder:

sudo apt-get install unison
unison # if you've already used unison before to create a profile, this might run it! So be careful...

You should now have a .unison folder in your home directory. We’re going to create a new unison profile to keep our folders in sync:

vim .unison/box.prf

and put this in it (obviously again replacing seb with your username!):

root = /home/seb/box.net
root = /media/box.net

ignore = Name *~
ignore = Name .*~

auto = true

retry = 2

logfile = /tmp/unisonlog

batch = true

That profile ignores any files with filenames ending in ~ (like vim and emacs’ backup files), sets automatic and batchmode to true (so it’ll try and do everything automatically, without asking for confirmation all the time) and write out a log of what happened to /tmp/unisonlog.

So let’s try it:

unison -ui text box

This runs unison in non-gui mode (thanks for Onoman for the heads up!) with the “box” profile. This should print out a load of stuff about what it’s doing, what it’s copying, conflicts etc, then write something (hopefully!) like:

UNISON 2.32.52 finished propagating changes at 08:30:46 on 24 Feb 2012
Saving synchronizer state
Synchronization complete at 08:30:46

If it did, then the contents if your box.net account should be in the box.net folder in your home directory. Brilliant.

So, now you can edit files, move them around, change, rename, whatever files in the /home//box.net foder, run unison box, then your box account will update itself. Likewise, you can put files in your box account on your phone, upload files through the website, email files in, run unison box again and your box.net folder in your home dir should update itself. AMAZING!

It’s a bit of a pain in the bum having to manually run unison everytime we want updates though…

Create a cron job to run unison automagically

Cron lets your system run commands automatically in the background on timed intervals, without you having to do anything. So we’ll create a cron job to automatically run our unison box command every hour. You can fiddle with that number to suit, but bear in mind that if you need some files sync’d immediately, just run unison box again.

Update: as mentioned by Markus in the comments, we really want to prevent unison from running more than one copy at a time; this can be arranged by creating a wrapper script that uses flock to run unison for you, and making sure there’s only one copy by creating a lock file:

/home/seb/bin/unison_wrapper.sh:

#!/bin/bash
flock -n /var/lock/my_unison_lock unison -ui text box

flock is a tool that is in Ubuntu by default – the manpage description is flock - manage locks from shell scripts which is exactly what we want; the -n specifies the name of the lock file to monitor; if another copy of our script is run but the lock is already locked, flock returns immediately and doesn’t run another (potentially clobbering) copy of unison.

So, now that we have the wrapper script setup, edit your crontab (your user’s list of scheduled cron jobs):

crontab -e

and put this at the bottom of it:
0 * * * * /home/seb/bin/unison_wrapper.sh

Cron’s format is a bit odd – it’s:

m    h    dom    mon    dow    command

So the crontab line we put in will run /home/seb/bin/unison_wrapper.sh on the 0th minute of * hours (every hour), * days of the month, * months, * days of the week – which is exactly what we want.

And with that, you should be done! Feel free to drop me a line if you have any problems…