How to sync files between computers securely and fast
Behold, the end of the time of the sneakernet has come! For a long time, I have wanted to sync some files back and forth between my various computers. So far, I have been using AFP over TCP/IP, FTP or sneakernet with USB dongles to transfer files between machines. The downside to this way of syncing files is that I have to manually copy any file I have changed from one machine to the other. It is easy to get conused about wich machine ha the newest version of a file. A disadvantage of using AFP over TCP/IP is that I have to have file sharing (or netatalk on linux) enabled all the time on one or more machines, wich might be a bad idea since it is probably not that hard to crack the password and get full access to my files. Or I have to turn it on and off every time I am tranferring some files back and forth.
The other option, so far, has been using one of the many cloud services out there. The problem with cloud services is that they are slow, since they have to transfer all your files from machine A to a server on the internet (or «the cloud») before machine B can download the files from that server. Even with a fast internet connection at both ends, this is much slower than just transferring the files manually over a local area network (LAN) from one machine to the other. It might also be slower than sneakernet.
The other problem with cloud services is privacy. DropBox was recently hacked and since it encrypts every dropbox folder of every dropbox client with the same encryption key, all data uploaded to DropBox was suddenly public information. Even if the companies behind your cloud service might have better security in place, you have no garantee that the data you upload to their servers will not be sold to marketing companies. You better read those terms of service… How much do you really trust Apple, Microsoft, Google, Dropbox or Cannonical?
The brilliant solution to the syncing problem is Bittorrent Sync. It is a new syncing service using peer-to-peer connections with the bittorrent protocol. If you are syncing two machines on the same local area network, these two machines will connect to each other and sync the data between folders on the two machines. This is much faster than any cloud service, since it doesn't have to upload any data to any server on the internet before downloading from the same server.
If the machines syncing folders are situated in different locations, the connection is made over the internet from the first machine to the second machine directly. Since no server is involved this is also faster than the cloud services.
If more than two machines are syncing a folder, the bittorrent protocol will make sure that data gets conveyed as fast as possible from the machines that have got the data to the other ones syncing the same folder, with different mchines contributing different parts of the folder, just as when you download files with a normal bittorrent client program.
Since all connections are made between the computers syncing the folder and no server in «the cloud» is involved, there are no third parties that will have a copy of the data transmitted. The connections between machines are encrypted, wich means that the data is not easy to sniff out either. Every folder you sync gets a random generated «secret», a key that you have to input at all the other machines syncing the same folder. By inserting the secret, the other machine gets to know wich machine(s) to connect to and wich folder to sync.
With some cloud services you have to put the files you want to sync in a special folder, but with Bittorrent sync, you can sync any folder on one machine with any folder on another machine. So, if you for instance want to keep your LibreOffice settings synced between a couple of Macs, you could sync the LibreOffice folder inside Application Support folder in your Libraries folder. Or you could update programs only on one machine and sync the program folder to all the other machines to avoid having to download the same programs more than once from the internet.
Another advantage of Bittorrent Sync is that there are no limitations on the amount of data you can sync, since there are no servers in «the cloud» that could limit your capacity. There are no extra fees for transferring more data than a certain amount and no need to buy extra space on cloud servers. However, If you want to, you may limit the upload and download bandwith BitTorrent Sync is allowd to use.
There is also a possibility to sync only one way. A scenario where this might be usefull is if you want to share some photos with some friends, but don't want to let them add their own files to the folder to be synced back. There is also a possibility to make a one-time «secret» that will let a person sync a folder only within 24 hours of the generation of the secret. These secrets might be one-way (read only) or full access.
At this point, Bittorrent Sync is only available for Mac OS X, Windows and Linux, but Bittorrent labs are working on making clients for mobile phones and tablets as well. The Linux version is managed through a webbrowser in stead of the program having its own GUI. Check out this blog post for the lastest Ubuntu and Debian user packages. Many thanks to tuxpoldo for making the packages!