Items Deduplication and File Compression for Zimbra with Zextras Suite | Zimbra

Zextras suite item deduplication and file compression allow you to save disk space by compressing files and storing only one copy of each item. It will save you a lot of space especially in large servers and make a significant difference financially in the long run.

Item deduplication and file compression are extremely useful as they reduce the space needed to store files. Storage devices have limited space, so using these techniques allows you to store more data in the same space. It will also help in transferring files, migration, backup size, and restore time. But the most important aspect might be the economic advantage, as you pay less for electricity, space, maintenance, and storage devices. Another impact would be on human-computer interactions as transferring smaller files is quicker, directly impacting user experience.

These functionalities are most requested by large enterprises that archiving email-based correspondence can grow to an incredible size.

Item Deduplication

Deduplication is another feature of Zextras storage management that stores each item only once and it will only be referenced if it is needed in other locations. This leads to huge savings in terms of space especially in servers with a large number of objects.

Advantages

Considering the explosive growth of data nowadays, the most important benefits of deduplication are performance enhancement and affordability. Reducing storage device costs, floor space, cooling, power consumption, and labor all contribute to affordability since less equipment to manage and operate means less cost.

Deduplication frees up storage space to be used more efficiently, it also eliminates the need for regular storage expansion, helps with your server performance. Corporates can take advantage of this optimization to remove constraints on data growth, which leads to improving their service levels and their competitiveness.

Zimbra’s rudimentary deduplication is tied to a limited cache while Zextras storage management’s deduplication continues to work regardless of any cache or timing.

How Much It Saves

The amount of space saved by deduplication depends on the amount of duplicated data on your server that can vary by the characteristics and access patterns of the data.

Nonetheless, you can have a rough idea of your environment’s potential to create duplication. For example, files created by office employees often contain redundant data and are frequently copied (especially for a big corporation that sends emails internally to all the employees). Such environments can benefit the most from deduplication.

Deduplication ratio is the proportion of logical data to physical space needed to store the deduplicated data. For example, a 10:9 deduplication ratio is quite a fair assumption based on experiments on Zimbra server data. It means that 10 units of logical data (10TB) are stored in 9 units of physical disk space (9TB) that eliminates the need for an additional 1TB. In other words, the Zextras suite on average saves around 10% of storage space.

File Comression

A real-time compression is implemented in Zextras storage management utilizing the gzip algorithm. Storing a corporation’s data and its backups that rapidly grow to an incredible size costs you new expensive storage devices unless you take advantage of data compression.

Advantages

All the advantages you read about deduplication are also true for compression since these benefits come from the saving storage space and both the features share this property.

Compression allows you to use your storage space more efficiently. It also immensely delays the need for storage expansion and gives you more time to better estimate your needs for expansion.

How Much It Saves

The theoretical compression ratio of this algorithm can be as high as 10:1 which saves around 90% of the used space. However, estimating the compression ratio in practice is more complicated.

The compression ratio is a function of the length and structure of the data being compressed. It means that two files with the same sequence length can have different compressed sizes according to their sequence structures. Therefore, it’s not easy to estimate the compression ratio of different Zimbra servers with different properties.

Nonetheless, practical experiments have shown that a 10:6 compression ratio is quite achievable based on the type of data on a Zimbra server. It means that 10 units of logical data(10TB) are stored in 6 units of physical disk space (6TB) that eliminates the need for an additional 4TB. In other words, the Zextras suite on average saves around 35% of storage space.

How To Use

Zextras suite allows you to use deduplication and compression features in a very intuitive way. For instance, you simply specify the name of a volume to enable compression or browsing duplicated items.

There are two ways to manage deduplication and compression:

• Administrative zimlet
• Command-line interface

These are some examples of using the deduplication and compression features that show how easy and intuitive they are.

Zimlet

To perform deduplication:

• Log into Zimbra administration console,
• Go to Zextras on the left pane,
• Select Powerstore,
• Select the volume,
• Click Deduplicate.

To enable compression on a volume, you can simply use Enable Compression button in the zimlet and Select the Compression Threshold.

CLI

Deduplication operation can be run in the CLI using zxsuite powerstore doDeduplicate command and specifying the name of the target volume.

For example, the following command starts deduplication on volume myVolume:

zxsuite powerstore doDeduplicate myVolume

Deduplication is also recommended after performing migration or importing large data.

For compression, you can use the optional attributes volume_compressed and compression_threshold_bytes as follows.

You can enable the compression when creating a new volume using doCreateVolume command as follows:

zxsuite powerstore doCreateVolume FileBlob {volume_name} {primary|secondary|index} {volume_path} [attr1 value1 [attr2 value2...]]

for example:

zxsuite powerstore doCreateVolume FileBlob volumeName secondary /path/to/store volume_compressed true compression_threshold_bytes 4096

Alternatively, on an already existing volume, you can modify these options as follows:

zxsuite powerstore doUpdateVolume FileBlob {current_volume_name} [attr1 value1 [attr2 value2...]]

zxsuite powerstore doUpdateVolume FileBlob CurrentVolumeName volume_compressed true compression_threshold_bytes 4096

Report

When you finish the scan and deduplication is complete, the Zextras suite generates a report indicating the amount of storage you saved.

For example, the following report shows a saving of 20.25GB:

Current Pass (Digest Prefix):  63/64
 Checked Mailboxes:             148/148
 Deduplicated/duplicated Blobs: 64864/128907
 Already Deduplicated Blobs:    75678
 Skipped Blobs:                 0
 Invalid Digests:               0
 Total Space Saved:             20.25 GB

Scheduling

With the Zextras suite, you can also configure schedule deduplication in a specific period. This process is smart, meaning that it ignores previously deduplicated data and works only on the data added after the last deduplication session that makes the process a lot faster.

For more information, we strongly suggest reading the Zextras documentation webpage.