Running Carbonio CE, carbonio-core 4.5.12-1jammy, Ubuntu 22.04.5.
Webmail/admin login fails with 502 Bad Gateway on GET /zx/login/v3/auth/config. Traced this to the carbonio-proxy upstream carbonio-auth, which routes to 127.78.0.1:20008 (an Envoy sidecar). The sidecar itself is up and listening, but nothing is registered behind it.
What I've confirmed:
consul catalog servicesshows nocarbonio-authentry at all (not unhealthy — completely absent)- No
.hclfile for it exists in/etc/zextras/service-discover/(onlycarbonio-proxy.hclreferences the name, as a destination) - No systemd unit
carbonio-auth-sidecar.serviceexists anywhere on disk dpkg -S carbonio-auth-sidecarreturns no matching packageapt-cache search carbonio | grep authandapt list -a carbonio-auth*show onlycarbonio-auth-ui(a frontend module), no backend service- Not served directly by mailboxd either (
curl localhost:8080/zx/login/v3/auth/config→ 404) - No trace of any process ever attempting to register this service name in
mailbox.log pending-setups -aruns clean, nothing pending- Full
zmcontrol restartand individual service restarts don't resolve it - Repointing the proxy upstream to
carbonio-user-management(the service that appears to actually handle auth-related SOAP requests) also didn't resolve it — reverted that change
What's healthy: everything else — mail delivery, IMAP, SMTP, LDAP, MTA, proxy itself, mailbox backend. This is isolated to the webmail/admin login flow specifically.
Found one other user with the exact same symptom (community.zextras.com/forum/carbonio-general-thread/carbonio-auth-problem/, Feb 2024) with no public resolution shown.
Is carbonio-auth supposed to be provided by a specific package? Was it possibly deprecated/replaced by carbonio-user-management in a later version and this proxy template just wasn't updated to match? Any pointers appreciated.
- Install missing packages
apt install -y carbonio-catalog carbonio-user-management
Then run pending-setups -a (needs the cluster credential password) — you may need to run it twice until all scripts complete.
- Nginx workaround for dead upstreams
The zx (port 8742) and carbonio-auth (port 20008) upstreams are legacy — no service listens there anymore. The admin/login apps call /zx/login/v1/config and /zx/auth/supported which time out. Add these location blocks before the existing location ^~ /zx/ in your admin nginx config (in both .admin and .admin.default files under /opt/zextras/conf/nginx/includes/):
location ^~ /zx/login/v1/config
{
return 200 '{"adminConsolePublicUrl":"https://your-server:6071","publicUrl":"https://your-server","zimbraDomainName":"your-domain","authMethods":["zimbra"]}';
}
location ^~ /zx/auth/supported
{
return 200 '{"maxApiVersion":2,"authMethods":["zimbra"]}';
}
Then reload nginx:
su - zextras -c 'zmproxyctl reload'
- i18n fix (if admin login shows 404 for locale files)
ln -sf removed link removed link
carbonio-catalog or carbonio-user-management as dependencies. The nginx templates still reference dead upstreams from the old architecture (pre-envoy). This should be fixed by Zextras in the upgrade scripts and package dependencies. Hope this saves you the hours of digging it took me! ---
Best Regards
Herbert
Thanks for the detailed writeup, Herbert — very helpful starting point. I dug into this further and found I’m in a slightly different situation than the “missing packages” scenario:
• carbonio-catalog and carbonio-user-management are both already installed (apt-get install -s confirms “already the newest version” for both)
• Their service-discover registration files exist and are enabled (/etc/zextras/service-discover/carbonio-catalog.hcl, carbonio-user-management.hcl)
• Their pending-setups scripts show as already successfully completed: /etc/zextras/pending-setups.d/done/421-carbonio-catalog.sh and done/121-carbonio-user-management.sh
• But there’s no systemd unit for either service, and nothing is listening on the address/port their own .hcl file specifies (confirmed carbonio-catalog should be at 127.78.0.28:10000 — nothing there)
So the setup appears to have run successfully at some point, but the actual services aren’t running now and haven’t self-recovered. Running pending-setups -a again shows “no pending setups to run” — nothing left for it to do, since these are already marked complete.
Following up on my earlier post — this resolved on its own after running a routine apt-get upgrade today that included libc6 (2.35-0ubuntu3.13 → 2.35-0ubuntu3.14) and related core library packages (libc-bin, libc-dev-bin, libc-devtools, libc6-dev, locales, locales-all). During that upgrade, needrestart prompted to restart a batch of services including carbonio-catalog.service and carbonio-user-management.service — services that, before this, had no systemd unit file at all on this system (confirmed via systemctl status returning nothing, and nothing listening on their expected ports).
After the upgrade, both now show as genuinely active (running), with real processes, “catalog ready”/“user-management ready” status messages, and confirmed listening on their expected addresses (carbonio-catalog at 127.78.0.28:10000, matching its own .hcl service-discovery config).
So it looks like whatever was missing (proper systemd unit registration for these two components) got installed/corrected as a dependency of this later package update, separate from the original install path. Didn’t need the pending-setups -a re-run or any manual intervention in the end — just needed a later upgrade cycle to catch up.
Environment: carbonio-core and related packages current as of 2026-07-28, Ubuntu 22.04.5.