For additional guidance, check out our community articles detailing the process of migrating from your current platform to Carbonio CE.
What is the point of having a backup?
There are many reasons why it is worth and important to have a backup, but especially an efficient backup system that really works. For example, you need it to restore data after loss caused by hardware/software failure or due to accidental deletion. Other situations where backup could be important are:
- data retention policy (i.e. recovery of an earlier version than the current one)
- protection against internal/external security threats
- legal requirements and compliance with institutional regulations
- data transfer (migration) requirements
It is essential that the backup system is verifiable. This is achieved by constantly checking the integrity of the item in the destination and its conformity with the original. This is important to guarantee the certainty of being able to restore the original data in case of need.
Typically, a backup system plans to schedule a series of copies of the data at a destination other than the operational one.
RealTimeScanner
A first and important innovation introduced by the Zextras backup module is the inclusion of a component called RealTimeScanner, which leads to the elimination of the concept of scheduling.
But what is it exactly?
The RealTimeScanner is a process, or rather a daemon, constantly active on the mailbox servers, capable of intercepting events such as the arrival, creation or modification of an item and then writing the changes to the backup destination.
The backup management of the Zextras module separates the metadata (record on the DataBase) from the BLOB (file on the filesystem) allowing a backup copy to be made in a local folder or in an external space. The replication of the structure of the element, its characteristics (metadata) and its history is also guaranteed, thus allowing the entire history to be preserved, with the possibility of restoring the data and its state in a specific temporal position.
Below is a small diagram highlighting the items that are backed up, what they are backed up to and some of the operations that can be performed.
The BLOB contains the essential part of an email (like header or body). Once copied to the backup destination, it takes on the name of its digest (identifier contained in the reference metadata corresponding to the checksum of the element). This ‘translation’ performs two tasks:
- It allows the integrity of the copied data to be checked.
- it allows the deduplication system to store only one version of the element when it is referenced by several objects; this avoids redundant copies of files and improves backup speed.
The metadata contains information about the item and the transactions that affected it: for example, the name of the folder into which an e-mail was moved or the date and time of this movement.
Both the blob and the metadata are copied to specific folders; the path to these folders (backup path) is generated on the basis of the identifier of the account to which the data belongs.
Backup Export
The backup export creates a snapshot of the mail system, ready to be used in case of migration or Disaster Recovery. Optimization of disk utilization, I/O speed and transfer times is achieved through deduplication and compression of the exported data. All accounts, items, distributions lists, Class of Services and domains configuration are exported.
It can also operate consistently on an active server, with no impact on service delivery.
The Zextras backup module allows an export of the entire infrastructure or of a single domain, thus allowing import into another server running Zextras Suite (regardless of the version of Zimbra or the operating system) and allowing the infrastructures to be updated.
Maintenance and Control Operations
The management of the backup module and its related operations can be carried out by the system administrator both from the administration portal and from the CLI which implements specific commands for this module.
The main operations that an administrator can perform are:
- SmartScan
- Purge
- CoherencyCheck
Let’s look at them in detail
Smartscan
SmartScan is the primary consistency check for the integrity of the backup system. The name “smart” comes from the fact that it does not act on the entire backup data set, but only on the accounts that have changed since the last SmartScan was performed. This allows a significant reduction in scanning time.
In addition to this type of scanning, SmartScan updates any obsolete entries, creates any type of item not yet present in the backup and marks as deleted any item found in the backup and not in the Zimbra datastore. Plus, it updates all configuration metadata in the backup so that domains, accounts, COSs and server configurations are archived along with a dump of all LDAP data and configurations.
By default, a SmartScan is scheduled to run at night. Only once a week, on a day chosen by the administrator, a cleanup operation is performed along with the SmartScan on deleted items that have exceeded their retention period and are therefore permanently removed.
Purge
Purge is a cleanup operation that removes from the backup path any deleted items that have exceeded the retention time defined by the data retention policy.
If a BLOB type item is referenced by one or more metadata items, the data will not be purged.
Its execution can be scheduled or manually initiated from the administration console or CLI.
If the retention period of the data has been set to infinite retention the Purge operation will be immediately terminated and no removal will be performed.
CoherencyCheck
This coherency check performs a more in-depth check than SmartScan.
While SmartScan works incrementally, Coherency Check performs a thorough check of all metadata and BLOBs in the backup path. Its purpose is to verify the integrity of the backup and it is specifically designed to detect damaged metadata and/or BLOBs.
If errors are detected, any orphaned or damaged metadata / BLOBs will be moved to a dedicated directory within the backup path by re-running the Coherency Check with the ‘fixBackup’ option.
The CoherencyCheck is started automatically if a previous SmartScan has detected a damaged item.
It is recommended to run the check periodically (approximately every 3/6 months) for verification purposes. It is also strongly recommended to run it after system crashes or storage failures.
Recovery Operations
The Zextras Backup module provides the system administrator with various recovery functions. Below we will see what the main ones are:
- Item Restore
Allows you to restore a single item from the backup of a given account by creating a new version of it if the item is already present in the production environment.
- Undelete
Allows an administrator to restore all the items deleted from a mailbox in a specific period, restoring them within a folder of the mailbox itself.
- Restore on new
Allows you to restore the contents and preferences of a mailbox to a specific point in time, to a completely new account.
The source account is not changed in any way, so you can restore one or more items belonging to a user’s account without actually restoring the entire mailbox. When performing this type of restore, you can hide the newly created account from the global address list as a security measure. The restored items will be created in the current main archive unless the “Respect HSM policy” box is checked.
If the account contained in the backup has been deleted in the production environment, a Restore on New command executed on the last available time point will restore the mailbox as it was when the mailbox was deleted.
- External Restore
Adds all COSs, domains, users, and items stored in a backup path or external backup to the current mailserver. It is mainly used for disaster recovery and migrations.
- Restore BLOB
This mode is used in case of complete corruption of a production volume in order to restore all missing and damaged BLOBs.
Synthesis
When you go to analyze a backup plan, there are some very important indicators that allow you to measure the effectiveness of the planned strategy:
- RPO – Recovery Point Objective
It represents the distance between the moment when the last backup copy is made and the moment when the event causing data loss occurs. The lower the RPO, the higher the percentage of recovered data.
- RTO – Recovery Time Objective
It expresses the amount of time elapsed between the disaster event and the restoration of the services provided.
- Data Retention Period
Institutional rules or company policies may impose not only the preservation of data for a specific period of time, but also the obligation to delete data beyond that period.
- Atomicity of Backup Transactions
A transaction is an atomic unit when it has no intermediate stages: it can conclude successfully or fail. The Zextras backup module manages transactions in this way. In fact, they can be successful, producing a consistent copy of the data in the backup destination, or they can be rolled back without the failed transaction having any effect on the system.
The use of RealTimeScanner and its ability to copy any object on the backup destination almost instantly gives the system administrator the ability to recover from a single deleted message to the entire domain, leading to a reduction of the RPO to zero, removing the need to put the system in a maintenance state and allowing to perform recovery operations while users are using the platform.
Backup space is reduced by 50% compared to the operational filesystem.
A complete and constant control of the archiving storage is ensured through the automatic deletion of the backup data once the retention time threshold has been exceeded (which is, we remind you, customizable) and the definitive elimination of the data beyond the time period defined by the adopted regulations is guaranteed.
The recovery procedures, which can be used both from a graphical environment and from the CLI, are independent from the operating systems, from the types of filesystems and from the Zimbra versions used, allowing their use also for import purposes from external platforms or, on the contrary, for total/partial migration to other infrastructures. They do not require additional software or specialized technical skills on the system administrator side, drastically reducing recovery times (RTO).
