Zextras Backup Path | Zimbra

With this article, we would like to give you an overview of the features and structure of the backup path in Zextras.

Data Organization

We are now going to have a look on how data organization works on a zimbra/zextras mailserver. Zimbra has a SQL backend. You can see how it is structured by reading this article. Understanding the backup path, is also important to better comprehend how coeherency check works.

ZIMBRA – METADATA (database)

It’s included in mail_item table inside mboxgroup% database. This last one is obtained by performing a modal division of the account id (“11” in the example shown next). Below you can find a complete example:

mailbox_id: 11
id: 260
type: 5
folder_id: 2
locator: 1
blob_digest: w5JqLTjHOR6pKyxxYySrNIslzaPj8kMG5PIYXWt9M7E=
unread: 1
flags: 0
tags: 0
tag_names: NULL
sender: user01@example.com
recipients: user02
subject: test3s
name: NULL
metadata: d1:f0:1:s16:user01@example.com1:t25:user02 user02@example.com1:vi10ee
mod_metadata: 110
change_date: 1609258597
mod_content: 110

ZIMBRA – BLOB (store)

Starting from the previous metadata the physical file (blob) that it references is located on the storage in:

/opt/zimbra/store/0/11/msg/0/260-110.msg

Where

• 11 is the mailbox id
• 260 is the element id
• 110 is the mod_content (the element counter)

ZEXTRAS – METADATA (backup/account)

In this sample case the metadata is copied to:

/opt/zimbra/backup/zextras/accounts/de2e9181-a1b6-4523-91d8-28782760442f/items/60/260

Where

• de2e9181-a1b6-4523-91d8-28782760442f is the id for the account that received the email. It can be got using this command: zmprov ga user@example.com zimbraId
• 60 is the id for the backup folder, computed è l’id della cartella di backup calcolata by bitshifting the element id
• 260 is the element id

ZEXTRAS – DIGEST (backup/items)

In this example scenario the physical file (BLOB) is copied to:

/opt/zimbra/backup/zextras/items/w5/w5JqLTjHOR6pKyxxYySrNIslzaPj8kMG5PIYXWt9M7E=

Where w5JqLTjHOR6pKyxxYySrNIslzaPj8kMG5PIYXWt9M7E is the blob_digest.

As you can see from the query shown in the previous lines, the identifier of the blob_digest can be obtained by querying the relevant database.

Backup Path

The so called “Backup Path” is the place where Zextras Backup data are stored. It can be both:

• A local directory or mountpoint
• A third party S3 volume

Metadata and Blobs

Zextras Backup separates BLOBs from METADATA, where:

• BLOB is a file containing the item itself (eg. header and body of an email)
• METADATA is a file containing all the information and transaction of an item (eg. the folder where it is stored and when it has been moved there)

This division is made to provide faster search and interaction times along with increased security. Both BLOBs and Metadata are stored in the same backup location, but in different subdirectories

Whether the backup is local or remote, the system will read from and write to the path in real time via the RealTime Scanner, during scheduled operations such as SmartScan and Purge, and during restores.

Most of the read and write operations involve item metadata, which is constantly updated with each new change, while BLOBs are only written when a new item is created and during restores. On average, metadata represents 10% of total backup disk usage but 80% of read operations.

Backup Path Structure

Both local and remote Backup Paths use the same basic structure:

/opt/zimbra/backup/zextras/
|-- accounts
|-- backupstat
|-- items
|-- lock_[serverID]
`-- server

Where:

• accounts is a directory where all Backup Metadata are stored. In case of local backup path it contains a subfolder for each account on the server with the name of the Zimbra ID for that account. In case of remote backup it contains a compressed file for each account on the server with the name of the Zimbra account.
• backupstat is a file that contains the basic backup usage information.
• items is a directory that includes all item BLOBs
• lock_[serverID] is a file that marks the backup path as owned by the server and keeps other servers from using the same backup path.
• server is a directory that holds server and other configurations like LDAP backups. All done by SmartScan.