{"id":1215,"date":"2013-12-04T20:46:23","date_gmt":"2013-12-05T01:46:23","guid":{"rendered":"http:\/\/swildow.darktech.org\/wp\/?p=1215"},"modified":"2013-12-04T20:47:22","modified_gmt":"2013-12-05T01:47:22","slug":"50gb-of-cloud-space-with-box-automatically-syncd-on-ubuntulinux-with-webdav-and-unison","status":"publish","type":"post","link":"http:\/\/www.wildow.com\/blog\/?p=1215","title":{"rendered":"50gb of \u201ccloud\u201d space with Box, automatically sync\u2019d on Ubuntu\/Linux with webdav and unison"},"content":{"rendered":"<h2><a href=\"http:\/\/seb.so\/50gb-of-cloud-space-with-box-automatically-syncd-on-linux-with-webdav\/\">50gb of \u201ccloud\u201d space with Box, automatically sync\u2019d on Ubuntu\/Linux with webdav and unison<\/a><\/h2>\n<footer><i>Posted on:<\/i>\u00a0<time datetime=\"2013-12-03T22:34:16+00:00\">February 24th, 2012<\/time>\u00a0<i>by<\/i>\u00a0Seb<\/footer>\n<footer><a href=\"http:\/\/seb.so\/50gb-of-cloud-space-with-box-automatically-syncd-on-linux-with-webdav\/\">http:\/\/seb.so\/50gb-of-cloud-space-with-box-automatically-syncd-on-linux-with-webdav\/<\/a><\/footer>\n<div>\n<p>Box (used to be box.net) is an online storage service that\u2019s been around for a while \u2013 it\u2019s quite popular and gives you more storage space than Dropbox usually does. And, if you\u2019ve got an Android or iPhone, getting the mobile Box app (for Box.com) unlocks 50gb of storage immediately, for free. 50GB of storage. Unfortunately, it doesn\u2019t have a linux syncing client yet \u2013 wouldn\u2019t it be nice if you could create your own?<\/p>\n<p><!--more--><\/p>\n<hr \/>\n<h4>Contents<\/h4>\n<ul>\n<li><a href=\"http:\/\/seb.so\/50gb-of-cloud-space-with-box-automatically-syncd-on-linux-with-webdav\/#box_intro\">Intro<\/a><\/li>\n<li><a href=\"http:\/\/seb.so\/50gb-of-cloud-space-with-box-automatically-syncd-on-linux-with-webdav\/#box_signup\">Signup for Box!<\/a><\/li>\n<li><a href=\"http:\/\/seb.so\/50gb-of-cloud-space-with-box-automatically-syncd-on-linux-with-webdav\/#box_mountwebdav\">Mount the Box folder locally using webdav<\/a><\/li>\n<li><a href=\"http:\/\/seb.so\/50gb-of-cloud-space-with-box-automatically-syncd-on-linux-with-webdav\/#box_unison\">Setup Unison to keep a local folder and your Box folder in sync<\/a><\/li>\n<li><a href=\"http:\/\/seb.so\/50gb-of-cloud-space-with-box-automatically-syncd-on-linux-with-webdav\/#box_cronjob\">Create a cron job to run unison automagically<\/a><\/li>\n<\/ul>\n<p><a name=\"box_intro\"><\/a><br \/>\nBox supports webdav; webdav is an extension to HTTP that allows handling of remote filesystems \u2013 thankfully, there are a number of packages in linux available to let you mount these webdav folders locally and have them behave as local folders. However, performance on these usually is a bit slow (each read\/write involves a few round-trips to the web server) so what we\u2019ll get here is:<\/p>\n<ul>\n<li>a locally mounted webdav folder that will have the \u201clive\u201d contents of what Box has, but quite slow<\/li>\n<li>a local copy of the contents of the webdav folder that gets kept in sync with the online Box copy every, or automatically when you change a file \u2013 but because it\u2019s a local copy, it\u2019s no different to normal files<\/li>\n<\/ul>\n<p>(Anywhere I\u2019ve used\u00a0<code>vim<\/code>, feel free to use your editor of choice \u2013 just replace it with something like\u00a0<code>gedit<\/code>\u00a0if you\u2019d rather a gui)<\/p>\n<hr \/>\n<p><a name=\"box_signup\"><\/a><\/p>\n<h3>Sign up for Box!<\/h3>\n<p>So, first up, you\u2019ll need a Box account.<\/p>\n<p>Just visit\u00a0<a href=\"http:\/\/www.box.com\/\" target=\"_blank\">box.com<\/a>\u00a0to create one, or signup from your mobile (then you get 50gb space from the getgo!). Once you\u2019ve got your account all set up and confirmed, continue!<\/p>\n<hr \/>\n<p><a name=\"box_mountwebdav\"><\/a><\/p>\n<h3>Mount the Box folder locally using webdav<\/h3>\n<p>Next we need to mount the Box web folder as a webdav folder \u2013 the\u00a0<code>davfs2<\/code>\u00a0package will do the job:<\/p>\n<p><code>sudo apt-get install davfs2<\/code><\/p>\n<p>Now create somewhere for the webdav folder to mount:<\/p>\n<p><code>sudo mkdir \/media\/box.net<\/code><\/p>\n<p>Then we\u2019ll add an entry in fstab (the file that Ubuntu reads to work out what\u2019s mountable)<\/p>\n<p><code>sudo vim \/etc\/fstab<\/code><\/p>\n<p>and put this in it at the end:<\/p>\n<p><code>https:\/\/www.box.net\/dav \/media\/box.net davfs defaults,rw,user,noauto 0 0<\/code><\/p>\n<p>Now we\u2019ll change the davfs config so that users in the\u00a0<code>users<\/code>\u00a0group can mount the webdav share whenever they need, and also to disable the use of file locks \u2013 they don\u2019t seem to work properly with the Box.com webdav service<\/p>\n<p><code>sudo vim \/etc\/davfs2\/davfs2.conf<\/code><\/p>\n<p>and put this in it (there might already be something similar to this somewhere in the file \u2013 just replace it with this):<\/p>\n<pre>dav_group users\r\nuse_locks 0<\/pre>\n<div>(Thanks to\u00a0<a href=\"http:\/\/johnreid.it\/2009\/09\/26\/mount-a-webdav-folder-in-ubuntu-linux\/\" target=\"_blank\">johnreid.it<\/a>\u00a0for the pointers about use_locks)<\/div>\n<p>Next, we need to add our user to the users group \u2013 (my user\u2019s called\u00a0<code>seb<\/code>, replace seb with you obviously!)<\/p>\n<p><code>sudo addgroup seb users<\/code><\/p>\n<p>Now we need to store our Box.com username and password so that it doesn\u2019t prompt every time we try and use it:<\/p>\n<p><code>sudo vim \/etc\/davfs2\/secrets<\/code><\/p>\n<p>and put this at the end:<\/p>\n<p><code>https:\/\/www.box.net\/dav &lt;your_user&gt; &lt;your_password&gt;<\/code><\/p>\n<p>At this point, everything should be ready to test the webdav setup; run this to mount it:<\/p>\n<p><code>mount \/media\/box.net<\/code><\/p>\n<div>(it might say something strange about\u00a0<code>system option in user configuration file<\/code>\u00a0\u2013 I don\u2019t really know why that is, but it doesn\u2019t appear to break anything)<\/div>\n<div>(it might also say\u00a0<code>program is not setuid root<\/code>\u00a0\u2013 you can fix this by running\u00a0<code>sudo chmod u+s \/sbin\/mount.davfs<\/code>\u00a0then running the mount command again)<\/div>\n<p>If that worked, then you should be able to open a filemanager (like nautilus or thunar) and visit \/media\/box.net and view the contents of your Box account! However, you might notice that it\u2019s really not very responsive \u2013 and sometimes freezes up the file manager\u2026 which is what the next section will fix.<\/p>\n<hr \/>\n<p><a name=\"box_unison\"><\/a><\/p>\n<h3>Setup Unison to keep a local folder and your Box folder in sync<\/h3>\n<p>Unison is a 2-way synchronization tool that can do all sorts of powerful things; here though we\u2019re just going to use it to keep 2 folders in sync with each other \u2013 I\u2019m going to put my local Box folder in\u00a0<code>\/home\/seb\/box.net<\/code>\u00a0and keep it in sync with the existing\u00a0<code>\/media\/box.net<\/code>. So first make sure you\u2019ve got the\u00a0<code>box.net<\/code>\u00a0folder:<\/p>\n<pre>cd ~\r\nmkdir box.net<\/pre>\n<p>Now you need to install unison (if you\u2019ve not got it already) then run unison once to create the .unison config folder:<\/p>\n<pre>sudo apt-get install unison\r\nunison # if you've already used unison before to create a profile, this might run it! So be careful...<\/pre>\n<p>You should now have a .unison folder in your home directory. We\u2019re going to create a new unison profile to keep our folders in sync:<\/p>\n<p><code>vim .unison\/box.prf<\/code><\/p>\n<p>and put this in it (obviously again replacing seb with your username!):<\/p>\n<pre>root = \/home\/seb\/box.net\r\nroot = \/media\/box.net\r\n\r\nignore = Name *~\r\nignore = Name .*~\r\n\r\nauto = true\r\n\r\nretry = 2\r\n\r\nlogfile = \/tmp\/unisonlog\r\n\r\nbatch = true<\/pre>\n<p>That profile ignores any files with filenames ending in ~ (like vim and emacs\u2019 backup files), sets automatic and batchmode to true (so it\u2019ll try and do everything automatically, without asking for confirmation all the time) and write out a log of what happened to \/tmp\/unisonlog.<\/p>\n<p>So let\u2019s try it:<\/p>\n<p><code>unison -ui text box<\/code><\/p>\n<p>This runs unison in non-gui mode (thanks for Onoman for the heads up!) with the \u201cbox\u201d profile. This should print out a load of stuff about what it\u2019s doing, what it\u2019s copying, conflicts etc, then write something (hopefully!) like:<\/p>\n<pre>UNISON 2.32.52 finished propagating changes at 08:30:46 on 24 Feb 2012\r\nSaving synchronizer state\r\nSynchronization complete at 08:30:46<\/pre>\n<p>If it did, then the contents if your box.net account should be in the box.net folder in your home directory. Brilliant.<\/p>\n<p>So, now you can edit files, move them around, change, rename, whatever files in the \/home\/\/box.net foder, run\u00a0<code>unison box<\/code>, then your box account will update itself. Likewise, you can put files in your box account on your phone, upload files through the website, email files in, run\u00a0<code>unison box<\/code>\u00a0again and your box.net folder in your home dir should update itself. AMAZING!<\/p>\n<p>It\u2019s a bit of a pain in the bum having to manually run unison everytime we want updates though\u2026<\/p>\n<hr \/>\n<p><a name=\"box_cronjob\"><\/a><\/p>\n<h3>Create a cron job to run unison automagically<\/h3>\n<p>Cron lets your system run commands automatically in the background on timed intervals, without you having to do anything. So we\u2019ll create a cron job to automatically run our\u00a0<code>unison box<\/code>\u00a0command every hour. You can fiddle with that number to suit, but bear in mind that if you need some files sync\u2019d immediately, just run\u00a0<code>unison box<\/code>\u00a0again.<\/p>\n<p>Update: as mentioned by Markus in the comments, we really want to prevent unison from running more than one copy at a time; this can be arranged by creating a wrapper script that uses flock to run unison for you, and making sure there\u2019s only one copy by creating a lock file:<\/p>\n<p><code>\/home\/seb\/bin\/unison_wrapper.sh<\/code>:<\/p>\n<pre>#!\/bin\/bash\r\nflock -n \/var\/lock\/my_unison_lock unison -ui text box<\/pre>\n<p><code>flock<\/code>\u00a0is a tool that is in Ubuntu by default \u2013 the manpage description is\u00a0<code>flock - manage locks from shell scripts<\/code>\u00a0which is exactly what we want; the\u00a0<code>-n<\/code>\u00a0specifies the name of the lock file to monitor; if another copy of our script is run but the lock is already\u00a0<i>locked<\/i>, flock returns immediately and doesn\u2019t run another (potentially clobbering) copy of unison.<\/p>\n<p>So, now that we have the wrapper script setup, edit your crontab (your user\u2019s list of scheduled cron jobs):<\/p>\n<p><code>crontab -e<\/code><\/p>\n<p>and put this at the bottom of it:<br \/>\n<code>0 * * * * \/home\/seb\/bin\/unison_wrapper.sh<\/code><\/p>\n<p>Cron\u2019s format is a bit odd \u2013 it\u2019s:<\/p>\n<pre>m    h    dom    mon    dow    command<\/pre>\n<p>So the crontab line we put in will run\u00a0<code>\/home\/seb\/bin\/unison_wrapper.sh<\/code>\u00a0on the 0th minute of * hours (every hour), * days of the month, * months, * days of the week \u2013 which is exactly what we want.<\/p>\n<p>And with that, you should be done! Feel free to drop me a line if you have any problems\u2026<\/p>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>50gb of \u201ccloud\u201d space with Box, automatically sync\u2019d on Ubuntu\/Linux with webdav and unison Posted on:\u00a0February 24th, 2012\u00a0by\u00a0Seb http:\/\/seb.so\/50gb-of-cloud-space-with-box-automatically-syncd-on-linux-with-webdav\/ Box (used to be box.net) is an online storage service that\u2019s been around for a while \u2013 it\u2019s quite popular and &#8230; <a class=\"more-link\" href=\"http:\/\/www.wildow.com\/blog\/?p=1215\">Read More &raquo;<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[12],"tags":[],"_links":{"self":[{"href":"http:\/\/www.wildow.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/1215"}],"collection":[{"href":"http:\/\/www.wildow.com\/blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.wildow.com\/blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.wildow.com\/blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.wildow.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1215"}],"version-history":[{"count":2,"href":"http:\/\/www.wildow.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/1215\/revisions"}],"predecessor-version":[{"id":1217,"href":"http:\/\/www.wildow.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/1215\/revisions\/1217"}],"wp:attachment":[{"href":"http:\/\/www.wildow.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1215"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.wildow.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1215"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.wildow.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1215"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}