I am trying to enable and setup a backup using the following Documentation for Carbonio CE 24.7.1 on Ubuntu 22.04, and then rsync the backup directory to another identical VM in a different geographical location to test a restore.
https://docs.zextras.com/carbonio/html/admincli/backup.html#
However, none of the commands in this document work as described. ($ carbonio config set server $(zmhostname) ZxBackup_DestPath /opt/carbonio-backup). Even using the zmprov commands, they do not properly work.
In addition, the referenced Admin GUI does not contain the described section either. "Smartscan can be run manually from the CLI or configured from the Admin Panel (
Am I missing something?
Is this Documentation current?
Migrating a Zimbra server seemed very straight forward and work very well. Would something like this work for Carbonio?
In Zimbra, I ran two identical VMs in two different physical locations (PROD and BACKUP). Meaning, I built and configured a new install of Zimbra, migrated all of my Users and Data and validated it working as expected. Then cloned the PROD VM and copied it to a different geographical datacenter and considered it my BACKUP.
NOTE: I run both servers in two different locations but DNS, PTR and MX only point to mail.domain.com (PROD) while I do have separate DNS, PTR and MX on standby pointing to mail2.domain.com (BACKUP), if I ever need them.
Now I can manually push PROD data using a few steps in rsync, during off hours, leaving the BACKUP in near real-time identical status at all times. This method only takes less than 15-mins with 33 users and over 150GB of data. I can even do this in cron at 4am.
Below are just the highlights, not all the required steps are included which can be found here https://wiki.zimbra.com/wiki/ZCS_to_ZCS_rsync_Migration. But for 10 years, it worked flawlessly each time.
PUSH PROD LDAP AND MESSAGE BLOB TO BACKUP.
1. STOP ZIMBRA ON PROD and BACKUP:
$ zmcontrol stop
2. BACKUP PROD LDAP (as Zimbra) AND COPY TO BACKUP (As Root)
$ /opt/zimbra/libexec/zmslapcat /backup (as Zimbra)
# scp /backup/ldap.bak mail2.domain.com:/backup (as Root)
3. COPY PROD MYSQL TO BACKUP (as Root)
# rsync --delete -axvzKHS /opt/zimbra/db/data/* mail2.domain.com:/opt/zimbra/db/data/
4. COPY MESSAGE BLOB TO BACKUP (as Zimbra)
$ rsync -e ssh --delete -axvzKHS /opt/zimbra/store/* zimbra@mail2.domain.com:/opt/zimbra/store/
$ rsync -e ssh --delete -axvzKHS /opt/zimbra/index/* zimbra@mail2.domain.com:/opt/zimbra/index/
5. START ZIMBRA ON PROD
$ zmcontrol start
ENABLE THE BACKUP WITH THE LATEST LDAP.
6. REMOVE EXISTING LDAP ON BACKUP (as Root)
# cd /opt/zimbra/data/ldap
# mv -f mdb /backup/old/
# mkdir -p mdb/db
# chown -R zimbra:zimbra /opt/zimbra/data/ldap
7. IMPORT LATEST LDAP ON BACKUP (as Zimbra)
$ /opt/zimbra/libexec/zmslapadd /backup/ldap.bak
8. START ZIMBRA BACKUP (as Zimbra)
$ zmcontrol start
NOW BOTH PROD AND BACKUP AND IN-SYNC.
I know it's not perfect but it's a straight forward, cost effective way to sleep at night knowing my backup is only one-day old and readily available.
Again, I would love to get the 'out of the box' Backup to work in Carbonio but in absence of that, would a similar rsync methodology work for Carbonio?
Thank you,
Hippiecool
@hippiecool
Hi,
The documentation you are following is for Carbonio not for CE. You can find the CE documentation here:
https://docs.zextras.com/carbonio-ce/html/index.html
But, backup feature is not available in Carbonio CE. For migration from Zimbra to Carbonio CE, you can take a look at this:
Regards,
Sharif
I am trying to enable and setup a backup using the following Documentation for Carbonio CE 24.7.1 on Ubuntu 22.04, and then rsync the backup directory to another identical VM in a different geographical location to test a restore.
https://docs.zextras.com/carbonio/html/admincli/backup.html#
However, none of the commands in this document work as described. ($ carbonio config set server $(zmhostname) ZxBackup_DestPath /opt/carbonio-backup). Even using the zmprov commands, they do not properly work.
In addition, the referenced Admin GUI does not contain the described section either. "Smartscan can be run manually from the CLI or configured from the Admin Panel (
Am I missing something?
Is this Documentation current?
Thank you for the reply. I figured that was the case.
I am already up and running with carbonio-ce. Now I am trying to design the best way to ensure its properly backed up. You never answered my question regarding the rsync method I was using for Zimbra. Would it work for carbonio-ce? I have a feeling it might actually work but I will test and report back for anyone who would like to use it.
Hippiecool
Hi,
RSYNC is a universal tool used in many backup scenarios especially in Zimbra OSE. Frankly, I did not answer it because I haven't tried it and most importantly if our forum users contribute in this kind of scenarios then the community would be more benefited. Ultimately, It is a community edition product so not all the features will be available in it. But I firmly believe our users can cover most of the lacking requirements using their experience and creativity. If we want to make this product sustainable, then the community has to come forward.
It would be very nice of you if you could share your experiments and results.
Regards,
Sharif
I couldn't agree more. FOSS is only as good as the community that contributes to it. I will definitely share my results.
Best,
hippiecool
This procedure will RSYNC all user and ldap data between two Carbonio-CE servers. It's designed for a single-server installation. You can use this to perform a one-time MIGRATION of Carbonio to a new Carbonio server, or as a ROUTINE BACKUP TOOL or BUSINESS CONTINGENCY PLAN to keep two geographically located servers in near-real-time synchronization (SERVER_PROD ---> SERVER_BACKUP).
ASSUMPTIONS:
SERVER_PROD is a VM that was built as a new single-server installation, provisioned with users and data, tested and validated as working as expected with the proper SSL, DNS, PTR and MX records. Then SERVER_PROD_VM was cloned as SERVER_BACKUP_VM and moved to another geographic datacenter. Both servers are currently running in their respective locations but DNS, PTR and MX records only point to SERVER_PROD.
Carbonio versions are IDENTICAL on BOTH servers. I update and upgrade each server weekly to keep them fresh. In general, OS versions typically do not matter (especially with a MIGRATION to a new server) but in my case these VMs are identical to make life easy (same commercial SSL too).
PRE-REQUISITES:
Ensure you can successfully ssh between the two servers (as zextras and root). You may need to adjust your firewall ports, NAT and rules to make this work. This will include any physical outside-perimeter device or OS firewall that is enabled.
Again, make sure Carbonio versions are identical on BOTH PROD and BACKUP servers (OS versions do not matter).
$ zmcontrol -v (as zextras)
PERFORM A ONE-TIME SETUP ON BOTH SERVERS – (as root) (make a backup directory on each server)
$ su
# cd /
# mkdir backup
# mkdir backup/old
# chown -R zextras:zextras /backup
# exit
---------------------- YOU ARE NOW READY TO BEGIN: --------------------------------------
This procedure requires you to stop Carbonio in PROD and BACKUP. Depending on the size of your Message Blob, it could take 5-20 minutes or in some cases longer. Best to perform this after-hours or when no users are currently using Carbonio (PROD). NOTE: The more often you do this procedure on a routine basis, the faster the synchronization will be. Because rsync uses compression and focuses on changes, a 200-500MB Message Blob can be completed between 5-10 minutes with adequate internet bandwidth.
--------------------- BEGIN TASKS ON BOTH SERVERS ------------------------------------
1.0 STOP CARBONIO ON BOTH PROD AND BACKUP SERVERS (as zextras). Carbonio cannot be running on either server.
$ zmcontrol stop
--------------------- END TASKS ON BOTH SERVERS --------------------------------------
------------------------- BEGIN PROD SERVER TASKS ------------------------------------
NOTE: XXX.XXX.XXX.XXX = WAN IP ADDRESS OF YOUR BACKUP SERVER
1.1 BACKUP LDAP (as zextras)
$ /opt/zextras/libexec/zmslapcat /backup
1.2 COPY LDAP TO BACKUP (as root)
# scp /backup/ldap.bak XXX.XXX.XXX.XXX:/backup
2.1 RSYNC MYSQL TO BACKUP (as root)
# rsync --delete -axvzKHS /opt/zextras/db/data/* XXX.XXX.XXX.XXX:/opt/zextras/db/data/
3.1 RSYNC MESSAGE BLOB STORE TO BACKUP (as zextras)
$ rsync -e ssh --delete -axvzKHS /opt/zextras/store/* XXX.XXX.XXX.XXX:/opt/zextras/store/
3.1 RSYNC MESSAGE BLOB INDEX TO BACKUP (as zextras)
$ rsync -e ssh --delete -axvzKHS /opt/zextras/index/* XXX.XXX.XXX.XXX:/opt/zextras/index/
4.1 START CARBONIO (as zextras)
$ zmcontrol start
-------------------- END PROD SERVER TASKS ----------------------------------
--------------------BEGIN BACKUP SERVER TASKS -------------------------------
1.0 REMOVE EXISTING LDAP (as root)
# cd /opt/zextras/data/ldap
# mv -f mdb /backup/old/
# mkdir -p mdb/db
# chown -R zextras:zextras /opt/zextras/data/ldap
2.0 IMPORT LATEST LDAP ON BACKUP (as zextras)
$ /opt/zextras/libexec/zmslapadd /backup/ldap.bak
OPTIONAL:
2.1 FIX PERMISSIONS (as root)
# /opt/zextras/libexec/zmfixperms -extended
3.0 START CARBONIO (as zextras)
$ zmcontrol start
--------------------END BACKUP SERVER TASKS -------------------------------
------------------ BOTH PROD AND BACKUP AND IN-SYNC -------------------
That's it, now both PROD and BACKUP servers are synchronized!
Depending how often you choose to perform these tasks, it's a straight forward, cost effective way to sleep at night knowing that your backup is in near-real-time and readily available in-case of a disaster.
------------------------------------------------------------------------------
NOTE:
If you made any customization's in PROD to these files, you will need to copy them as well and re-apply the changes to the BACKUP server.
/opt/zextras/conf/postfix_blacklist
/opt/zextras/conf/postfix_rbl_override
/opt/zextras/conf/amavisd.conf.in
/opt/zextras/conf/localconfig.xml
Regards,
hippiecool
@hippiecool I didn't have time to test it yet, but at a first look it sounds a nice piece of work! Thanks for your contribution!