Refactoring
to
maintainability

Data store/backup tool

2021-06-01
Tools

The Setup

For a long time I have used a double “backup”/duplication system: Dropbox has been time the main system I have been using to duplicate my data across my different devices. But, for a subset of data, I have been using SpiderOak One due to their encryption at rest.

But, I started realizing that SpiderOak was having some issues with the sync (some files it would not sync, sometimes it would refuse to sync even when I knew there were new files set on the server). This could be related to their Linux clients. I don’t know. So I decided to look for an alternative.

Syncthing

Thanks to ex-colleagues I have found Syncthing, a tool to synchronize folders between different devices. My first impressions of the tool are good. It is relatively easy to use, seems to allow for quite a few options.

If you want to sync everything between all of your devices, as I want to do right now, you can use the default folder to be shared. But if you want to share differently based on the device, you want to think how you want to structure your folders.

One of the interesting options that I have found is that you can setup a folder to only send files when synchronizing or to only receive files, so they can be set as a broadcaster of changes or the ultimate resting place for files.

An issue, though, is, because there is no central server, you need to have your devices on and running Syncthing at the same time. If you had an Android phone/tablet, you could use that as the bridge, but otherwise needs special consideration of your time.

The RAID-5 incident

Years ago, the operations people of some company which will not be named lost some information for a client. And at some point they asked an ex-colleague of mine (a developer) if RAID-5 was enough as a backup. You would think face-palm. But is not like Dropbox, SpiderOak One or Syncthing are not the multi-device equivalent of Raid-5. And guess which person that writes a blog lost a couple of files because of that?

One of the multiple options of Syncthing is to have a folder keeping versions of files (in fact, it has multiple options, which is quite nice). With versioning it would move deleted (and changed) files to an .stversion directory. But I don’t think I want my different computers to be the resting place of backups, as they are always liable to be reformatted/reinstalled.

As a personal user a backup system maybe is not as important, if I have replication through multiple devices. But should I look at something better than that?. Thinking, for example, about all my pictures; I would be sad if I lost some of them.

The “Cloud” is the default place where to look for setting up something nowadays. Though, as I mentioned before the tooling that I have used is clearly not for backups. There are options like Azue Backup, or just go straight up with an archive system using AWS S3 or Azure Archive and use some other software to do the backups.

The other options is having my own NAS used as a backup, which is interesting as I never have done anything similar. I have colleagues that have their own NAS setup, so help is available. Of course, you don’t get the enterprise grade backup that you would have otherwise. But would allow me to play with different setups and strategies.


Return to Index

Unless otherwise stated, all posts are my own and not associated with any other particular person or company.

For all code - MIT license
All other text - Creative Commons Attribution-ShareAlike 4.0 International License
Creative Commons BY-SA license image