With this article, we would like to give you an overview of the features and structure of the backup path in Zextras.
We are now going to have a look on how data organization works on a zimbra/zextras mailserver. Zimbra has a SQL backend. You can see how it is structured by reading this article. Understanding the backup path, is also important to better comprehend how coeherency check works.
ZIMBRA – METADATA (database)
It’s included in
mail_item table inside
mboxgroup% database. This last one is obtained by performing a modal division of the account id (“11” in the example shown next). Below you can find a complete example:
mailbox_id: 11 id: 260 type: 5 folder_id: 2 locator: 1 blob_digest: w5JqLTjHOR6pKyxxYySrNIslzaPj8kMG5PIYXWt9M7E= unread: 1 flags: 0 tags: 0 tag_names: NULL sender: email@example.com recipients: user02 subject: test3s name: NULL metadata: d1:f0:1:s16:firstname.lastname@example.org:t25:user02 email@example.com:vi10ee mod_metadata: 110 change_date: 1609258597 mod_content: 110
ZIMBRA – BLOB (store)
Starting from the previous metadata the physical file (blob) that it references is located on the storage in:
- 11 is the mailbox id
- 260 is the element id
- 110 is the mod_content (the element counter)
ZEXTRAS – METADATA (backup/account)
In this sample case the metadata is copied to:
- de2e9181-a1b6-4523-91d8-28782760442f is the id for the account that received the email. It can be got using this command: zmprov ga firstname.lastname@example.org zimbraId
- 60 is the id for the backup folder, computed è l’id della cartella di backup calcolata by bitshifting the element id
- 260 is the element id
ZEXTRAS – DIGEST (backup/items)
In this example scenario the physical file (BLOB) is copied to:
Where w5JqLTjHOR6pKyxxYySrNIslzaPj8kMG5PIYXWt9M7E is the blob_digest.
As you can see from the query shown in the previous lines, the identifier of the blob_digest can be obtained by querying the relevant database.
The so called “Backup Path” is the place where Zextras Backup data are stored. It can be both:
- A local directory or mountpoint
- A third party S3 volume
Metadata and Blobs
Zextras Backup separates BLOBs from METADATA, where:
- BLOB is a file containing the item itself (eg. header and body of an email)
- METADATA is a file containing all the information and transaction of an item (eg. the folder where it is stored and when it has been moved there)
This division is made to provide faster search and interaction times along with increased security. Both BLOBs and Metadata are stored in the same backup location, but in different subdirectories
Whether the backup is local or remote, the system will read from and write to the path in real time via the RealTime Scanner, during scheduled operations such as SmartScan and Purge, and during restores.
Most of the read and write operations involve item metadata, which is constantly updated with each new change, while BLOBs are only written when a new item is created and during restores. On average, metadata represents 10% of total backup disk usage but 80% of read operations.
Backup Path Structure
Both local and remote Backup Paths use the same basic structure:
/opt/zimbra/backup/zextras/ |-- accounts |-- backupstat |-- items |-- lock_[serverID] `-- server
accountsis a directory where all Backup Metadata are stored. In case of local backup path it contains a subfolder for each account on the server with the name of the Zimbra ID for that account. In case of remote backup it contains a compressed file for each account on the server with the name of the Zimbra account.
backupstatis a file that contains the basic backup usage information.
itemsis a directory that includes all item BLOBs
lock_[serverID]is a file that marks the backup path as owned by the server and keeps other servers from using the same backup path.
serveris a directory that holds server and other configurations like LDAP backups. All done by SmartScan.