10. Backup & Restore¶
Currently, Kopano provides three ways of restoring items:
- Through the softdelete restore system
- Using the brick-level backup system
- With a full database backup (coupled with a backup of the attachment directory)
10.1. Softdelete restore¶
The softdelete restore can be used by MAPI clients with the Restore deleted items dialog to restore deleted items such as Kopano WebApp. This covers most accidental deletions and allows users to directly restore without interaction or supervision from IT administration.
Items that are deleted by the user (by emptying the deleted items folder) are simply placed in the deleted items cache. This means that the item will not actually be removed from the database until the retention time of the item has expired. This expiration time in can be specified in the server.cfg configuration time and it set to 30 days by default.
In the following overview, which possibilities can be performed by whom, and when it is most likely used, can be seen.
Table 10.1. Recovery options
| Restore request | % of time spent | Backup solution | Performer |
|---|---|---|---|
| Items < 30 days old | 80 % | Softdelete system | User and Administrator |
| Items >= 30 days old | 10 % | Bricklevel | Administrator |
| Items from a specific sender | 5 % | Bricklevel | Administrator |
| Items over a specific time period | 3 % | Bricklevel | Administrator |
| Disaster recovery | 2 % | MySQL Dump + Attachments | Administrator |
As can be seen, the most common restore request can be performed by the user itself.
When older messages are requested to be restored, the Administrator will need to consult alternatives to the softdelete backup method. It is not possible to restore a single item with a MySQL dump, so this is the point where the kopano-backup tool steps in.
The bricklevel backups from the kopano-backup tool contain not enough information for disaster recovery. A complete dump of the MySQL database will be needed to perform this type of recovery.
10.2. Full database dump¶
All the data that is stored by Kopano Server is stored within a MySQL database and on the filesystem for attachments. This means that for a disaster recovery, all that is needed is a full backup/restore of the database in question together with the attachments configured to be stored within server.cfg. This can be done in many ways, but we will explain two ways of doing a good backup here. There are certain recommendations on how to make the backup to safely get all data in the state required for a consistent restore.
10.2.1. SQL dump through mysqldump¶
The contents of an entire Kopano database can be saved to a file by using the mysqldump command. There are, however, some options that are important in this case: the --single-transaction option should always be specified to mysqldump. When this is done, it will cause mysqldump to write a single snapshot of the database to disk. This will make sure that any writes done in the database during the backup will not be backed up. In effect, the dump that is made is a ‘snapshot’ of the database at the moment that the dump started. Additionally the option --routines should be specified to backup Kopano’s stored procedures.
When using mysqldump, it is very important not to do any table locking. This means that the --opt option and --lock-tables should never be used while dumping a Kopano database. The reason is that these options will ‘lock’ the tables while they are being dumped to disk, causing any accesses to the database to ‘freeze’ while the backup runs. This is firstly unnecessary and secondly may cause emails that are arriving during backup to bounce (depending on the MTA settings).
A simple:
mysqldump -p --single-transaction --routines <database> -r <dumpfile>.sql
will make a consistent dump of the database.
10.2.2. Binary data dump via LVM Snapshotting¶
This technique uses the ‘LVM Snapshot’ feature to effectively ‘freeze’ a binary view of the database file, while the database keeps running. This ‘frozen’ view is then simply binary copied to a remote server. This works because innodb makes sure that a single snapshot of a database will always be coherent (ie. It will be able to recover the database when mysql is started up on this dataset.)
As setting up LVM and configuring LVM for snapshots is a complex process, we refer the user to the LVM documentation and tools on how to set up an LVM volume for the MySQL data, and how to create and delete snapshot partitions.
10.2.3. Attachments backup¶
When using the attachments storage outside the database, make sure that these attachments are also backed-up.
Some backup methods that can be used to backup the attachments:
- Rsync
- Copy all files to external backup server or external attached hard-drive
- Use of a (commercial) backup agent for Linux, like SEP, Bacula, Arkeia or others
10.3. Brick-level backups¶
kopano-backup is the brick-level backup tool for Kopano Core. This tool will write a backup of users and stores onto the local filesystem. Currently kopano-backup supports three modes of operation:
- initial backup - which includes emails, calendar items, contacts, etc., as well as user settings such as rules, permissions and WebApp settings.
- incremental backup - which is automatically choosen, when the backup directory already contains a backup set for a given user/store.
- purge - which removes items from the backup data, that have previously been deleted from the users store.
For additional options kopano-backup includes a man page and a built-in help text, these can be retrieved by calling man kopano-backup and kopano-backup –help.
In compared to the old way of storing backups in zarafa-backup, kopano-backup now uses a berkeley DB to store serialised data and a folder hierarchy remsembling the mailbox layout, instead of a binary file with position markers in an index file. As result data exported by kopano-backup is now more stable and less prone to corruption. A downside of this new approach is that by updating the Berkeley DB file instead of adding a file for each increment, backups of the exported data are not very efficient. Until KC-627 has been implemented we recommend the use of filesystem snapshots if a versioned backup of the data exported by kopano-backup is needed.
10.3.1. kopano-backup vs. zarafa-backup¶
A brief comparison of usage differences in usage between kopano-backup and zarafa-backup.
| Task | kopano-backup | zarafa-backup |
|---|---|---|
| Refresh backup through deleting backup and all increments | not needed anymore as all transactions are recorded via ics and stored in Berkeley DB | should be done on a regular base |
| Storage of data | Folder hierarchy resembling the mailbox layout, serialised data in Berkeley DB files | One binary file and an index file, increments into separate binary files |
| off-site backup and rotation | Until KC-627 no efficient way to only transfer increments, filesystem snapshots should be used | possible, while creating increments previous index files should be kept |
Please note that for disaster recovery it is always recommended to use a MySQL dump over the bricklevel data.