Hi all,
I'm attempting to use the install_carbonio_ce_singleserver_ubuntu.sh found here on a fresh install (never had Carbonio or anything else on it before). I'm using a fresh install of Ubuntu 22.04 server with a static IP address.
Release:
root@mail:~# lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 22.04.5 LTS Release: 22.04 Codename: jammy
Static IP (Internal), I'll NAT applicable ports from the firewall into the server when it's installed and online:
root@mail:~# ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.111.30 netmask 255.255.255.0 broadcast 192.168.111.255
ether 9a:d9:50:f4:ef:9e txqueuelen 1000 (Ethernet)
RX packets 365224 bytes 1671372424 (1.6 GB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 254361 bytes 17829023 (17.8 MB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
When I run the script, it gets to this point and throws the following error:
Initializing ldap...Connection timed out at /opt/zextras/libexec/ldapinit.pm line 122. panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x50 pc=0x78c27e]
More Screen output:
----------------------------------
| Starting Carbonio Bootstrap... |
----------------------------------
--------------------------------------------------------------------------------
The system is going to start configuring all the packages.
If you are installing Carbonio CE please refer to every single package license
and to the Github repo for more informations on open source licenses:
https://github.com/Zextras
If you are also installing Carbonio please refer to every single package
license, to the Github repo for more informations on open source licenses:
https://github.com/Zextras and to the Zextras EULA for the proprietary
software: https://zextras.com/licenses
--------------------------------------------------------------------------------
Operations logged to /tmp/zmsetup.20241007-154727.log
Bootstrap mode: new install
Installing LDAP configuration database...done.
Setting defaults...done.
Checking for port conflicts
Saving config in /opt/zextras/config.13763...done.
Operations logged to /tmp/zmsetup.20241007-154727.log
Setting local config values...done.
Setting up CA...done.
Deploying CA to /opt/zextras/conf/ca ...done.
Creating SSL carbonio-appserver certificate...done.
Creating new carbonio-directory-server SSL certificate...done.
Creating new carbonio-mta SSL certificate...done.
Creating new carbonio-proxy SSL certificate...done.
Installing mailboxd SSL certificates...done.
Installing MTA SSL certificates...done.
Installing LDAP SSL certificate...done.
Installing Proxy SSL certificate...done.
Setting replication password...done.
Setting Postfix password...done.
Setting amavis password...done.
Setting nginx password...done.
Initializing ldap...Connection timed out at /opt/zextras/libexec/ldapinit.pm line 122.
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x50 pc=0x78c27e]
goroutine 1 [running]:
github.com/go-ldap/ldap.(*Conn).nextMessageID(...)
/home/agent/go/pkg/mod/github.com/go-ldap/ldap@v3.0.3+incompatible/conn.go:235
github.com/go-ldap/ldap.(*Conn).SimpleBind(0x0, 0xc000096e28)
/home/agent/go/pkg/mod/github.com/go-ldap/ldap@v3.0.3+incompatible/bind.go:54 +0x13e
github.com/go-ldap/ldap.(*Conn).Bind(0xc00002cea0?, {0xc00002d0c0?, 0xc000096f10?}, {0xc000013db0?, 0x828d60?})
/home/agent/go/pkg/mod/github.com/go-ldap/ldap@v3.0.3+incompatible/bind.go:116 +0x53
github.com/zextras/service-discover/pkg/carbonio.connect(0xc00007c900, 0x65?)
/tmp/service-discover/pkg/carbonio/ldap.go:216 +0x1eb
github.com/zextras/service-discover/pkg/carbonio.(*ldapContext).QueryAllServersWithService(0x8c9ade?, {0x8be909, 0x10})
/tmp/service-discover/pkg/carbonio/ldap.go:149 +0x65
github.com/zextras/service-discover/cmd/server/command/setup.(*Setup).isFirstInstance(0xc000170018, {0x96d450, 0xc000030ec0})
/tmp/service-discover/cmd/server/command/setup/common.go:211 +0x7b
github.com/zextras/service-discover/cmd/server/command/setup.(*Setup).Run(0xc000170018, 0xc000170008)
/tmp/service-discover/cmd/server/command/setup/common.go:175 +0x1b9
reflect.Value.call({0x88b9e0?, 0xc000170018?, 0x8b5700?}, {0x8b6d2a, 0x4}, {0xc000010e70, 0x1, 0x1?})
/usr/lib/go/src/reflect/value.go:596 +0xca6
reflect.Value.Call({0x88b9e0?, 0xc000170018?, 0xc0000978b8?}, {0xc000010e70?, 0x8b5700?, 0xc0000977f8?})
/usr/lib/go/src/reflect/value.go:380 +0xb9
github.com/alecthomas/kong.callFunction({0x88b9e0?, 0xc000170018?, 0xc000097a30?}, 0xc00019f680)
/home/agent/go/pkg/mod/github.com/alecthomas/kong@v0.9.0/callbacks.go:98 +0x45a
github.com/alecthomas/kong.(*Context).RunNode(0xc000146180, 0xc0001720f0, {0xc000097ef8, 0x1, 0x828d60?})
/home/agent/go/pkg/mod/github.com/alecthomas/kong@v0.9.0/context.go:774 +0x80e
github.com/alecthomas/kong.(*Context).Run(0xc000146180?, {0xc000097ef8?, 0xc00014ff08?, 0x10?})
/home/agent/go/pkg/mod/github.com/alecthomas/kong@v0.9.0/context.go:799 +0x132
main.main()
/tmp/service-discover/cmd/server/server.go:66 +0x8bc
I ran a search here on the forums and found this workaround, which I tried and then ran carbonio-bootstrap. It did not resolve the issue.
It's also asking for the cluster password, which I do not have at the moment. Looking at /etc/zextras/service-discover, I do not appear to have a cluster-credentials.tar.gpg file.
root@mail:/etc/zextras/service-discover# ls -al total 132 drwxr-xr-x 3 root root 4096 Oct 7 16:11 . drwxr-xr-x 6 root root 4096 Oct 7 15:45 .. -rw-r--r-- 1 root root 481 Aug 27 10:04 carbonio-clamav.hcl -rw-r--r-- 1 localadmin localadmin 328 Sep 19 10:06 carbonio-clamav-signature-provider.hcl -rw-r--r-- 1 root root 303 Sep 11 09:40 carbonio-docs-connector-db.hcl -rw-r--r-- 1 root root 527 Aug 27 10:24 carbonio-docs-connector.hcl -rw-r--r-- 1 root root 299 Sep 11 09:45 carbonio-files-db.hcl -rw-r--r-- 1 root root 1787 Sep 23 15:38 carbonio-files.hcl -rw-r--r-- 1 root root 301 Sep 18 09:40 carbonio-mailbox-admin.hcl -rw-r--r-- 1 root root 268 Sep 5 16:11 carbonio-mailbox-db.hcl -rw-r--r-- 1 root root 1074 Sep 18 09:40 carbonio-mailbox.hcl -rw-r--r-- 1 root root 314 Sep 18 09:40 carbonio-mailbox-nslookup.hcl -rw-r--r-- 1 root root 265 Aug 27 11:46 carbonio-message-broker.hcl -rw-r--r-- 1 root root 541 Jul 26 16:06 carbonio-message-dispatcher-auth.hcl -rw-r--r-- 1 root root 199 Aug 26 16:39 carbonio-message-dispatcher-db.hcl -rw-r--r-- 1 root root 728 Jul 26 16:06 carbonio-message-dispatcher-http.hcl -rw-r--r-- 1 root root 274 Jul 26 16:06 carbonio-message-dispatcher-xmpp.hcl -rw-r--r-- 1 localadmin localadmin 387 Aug 27 16:01 carbonio-mta.hcl -rw-r--r-- 1 root root 658 Sep 10 13:04 carbonio-preview.hcl -rw-r--r-- 1 root root 113 Aug 27 22:11 carbonio-prometheus-consul-exporter.hcl -rw-r--r-- 1 root root 113 Aug 27 22:12 carbonio-prometheus-mysqld-exporter.hcl -rw-r--r-- 1 root root 112 Aug 27 22:11 carbonio-prometheus-nginx-exporter.hcl -rw-r--r-- 1 root root 111 Aug 27 22:11 carbonio-prometheus-node-exporter.hcl -rw-r--r-- 1 root root 115 Aug 27 22:11 carbonio-prometheus-openldap-exporter.hcl -rw-r--r-- 1 localadmin localadmin 2413 Sep 19 10:06 carbonio-proxy.hcl -rw-r--r-- 1 root root 362 Sep 5 16:12 carbonio-storages.hcl -rw-r--r-- 1 root root 305 Sep 11 09:46 carbonio-tasks-db.hcl -rw-r--r-- 1 root root 780 Aug 27 08:29 carbonio-tasks.hcl -rw-r--r-- 1 root root 603 Sep 10 14:30 carbonio-user-management.hcl -rw-r--r-- 1 root root 554 Sep 24 15:39 carbonio-videoserver.hcl -rw-r--r-- 1 root root 197 Aug 26 16:39 carbonio-ws-collaboration-db.hcl -rw-r--r-- 1 root root 1727 Aug 30 10:52 carbonio-ws-collaboration.hcl drwxr-xr-x 2 root root 4096 Oct 7 15:45 templates
So.... help please?
More info:
root@mail:/etc/zextras/service-discover# systemctl enable carbonio-openldap
Created symlink /etc/systemd/system/carbonio-directory-server.target.wants/carbonio-openldap.service → /lib/systemd/system/carbonio-openldap.service.
Created symlink /etc/systemd/system/carbonio-ce.target.wants/carbonio-openldap.service → /lib/systemd/system/carbonio-openldap.service.
Unit /lib/systemd/system/carbonio-openldap.service is added as a dependency to a non-existent unit carbonio-ce.target.
Created symlink /etc/systemd/system/carbonio.target.wants/carbonio-openldap.service → /lib/systemd/system/carbonio-openldap.service.
Unit /lib/systemd/system/carbonio-openldap.service is added as a dependency to a non-existent unit carbonio.target.
root@mail:/etc/zextras/service-discover# systemctl status carbonio-openldap
○ carbonio-openldap.service - Carbonio OpenLDAP Daemon
Loaded: loaded (/lib/systemd/system/carbonio-openldap.service; enabled; vendor preset: enabled)
Active: inactive (dead)
root@mail:/etc/zextras/service-discover# systemctl restart carbonio-openldap
root@mail:/etc/zextras/service-discover# systemctl status carbonio-openldap
● carbonio-openldap.service - Carbonio OpenLDAP Daemon
Loaded: loaded (/lib/systemd/system/carbonio-openldap.service; enabled; vendor preset: enabled)
Active: active (running) since Mon 2024-10-07 16:43:06 UTC; 1s ago
Process: 25781 ExecStartPre=/opt/zextras/bin/systemd-envscript (code=exited, status=0/SUCCESS)
Main PID: 25942 (slapd)
Tasks: 2 (limit: 19027)
Memory: 342.3M
CPU: 10.532s
CGroup: /system.slice/carbonio-openldap.service
└─25942 /opt/zextras/common/libexec/slapd -d 0 -l LOCAL0 -h "ldap://mail.[domainredacted.com]:389 ldapi:///" -F /opt/zextras/data/ldap/config
Oct 07 16:43:02 mail.[domainredacted.com] systemd[1]: Starting Carbonio OpenLDAP Daemon...
Oct 07 16:43:06 mail.[domainredacted.com] systemd[1]: Started Carbonio OpenLDAP Daemon.
Oct 07 16:43:06 mail.[domainredacted.com] slapd[25942]: @(#) $OpenLDAP: slapd 2.5.18 (Aug 27 2024 12:00:31) $
root@78cac1d6050b:/tmp/openldap/src/openldap-2.5.18/servers/slapd
Oct 07 16:43:06 mail.[domainredacted.com] slapd[25942]: sql_select option missing
Oct 07 16:43:06 mail.[domainredacted.com] slapd[25942]: auxpropfunc error no mechanism available
Oct 07 16:43:06 mail.[domainredacted.com] slapd[25942]: _sasl_plugin_load failed on sasl_auxprop_plug_init for plugin: sql
Oct 07 16:43:06 mail.[domainredacted.com] slapd[25942]: slapd starting
Specifically this caught my attention:
Oct 07 16:43:06 mail.[domainredacted.com] slapd[25942]: sql_select option missing Oct 07 16:43:06 mail.[domainredacted.com] slapd[25942]: auxpropfunc error no mechanism available Oct 07 16:43:06 mail.[domainredacted.com] slapd[25942]: _sasl_plugin_load failed on sasl_auxprop_plug_init for plugin: sql
I'm seeing similar issues with LDAP after an update. Right now my email server is bricked due to this unfortunately.
@anomaly0617 Sorry to hear you have those problems, honestly I never experienced a SIGSEGV in Carbonio installation... Apparently, the script fails because it can not find the (initial) openldap configuration file. However, since the carbonio-openldap service is running, the installer should have (somehow) recovered. I assume that installation failed, can you confirm?
It is strange that you do not have the cluster-credentials.tar.gpg, but the password is also printed on screen as the last task of the script (along with the PostgreSQL one).
Yyou can always retrieve the Carbonio Mesh (service discover) password, at /var/lib/service-discover/password.
Hi David,
@anomaly0617 Sorry to hear you have those problems, honestly I never experienced a SIGSEGV in Carbonio installation... Apparently, the script fails because it can not find the (initial) openldap configuration file.
Can you tell me what the path should be to the initial openldap file and I can verify that it does or does not exist?
However, since the carbonio-openldap service is running, the installer should have (somehow) recovered. I assume that installation failed, can you confirm?
Yes, the installation most definitely failed. I've tried it multiple times, now from two different Ubuntu Server 22.04.5 installations. It fails at the same point. This last time I also took a snapshot of the server just before I ran the script, so I can return to that point and reproduce the errors, if that's helpful. If you or someone else from the install team wants to PM me, I can revert the server to the snapshot and arrange for you/them to have SSH access to the server temporarily, so you guys can see what I see when it runs through the script. I think resolving the script issue would help a lot of people to get this issue resolved and get Carbonio working on new installs again.
It is strange that you do not have the cluster-credentials.tar.gpg, but the password is also printed on screen as the last task of the script (along with the PostgreSQL one).
I don't think it makes it to this point to display the passwords. My standard practice is that when information like that is presented, I immediately copy and paste it into a file in the /root directory with 0600 permissions and then use gpg to encrypt the file.
Yyou can always retrieve the Carbonio Mesh (service discover) password, at /var/lib/service-discover/password.
Checking /var/lib/service-discover, the directory is empty. Here's the direct output:
Last login: Thu Oct 10 14:02:56 2024 root@mail:~# cd /var/lib/service-discover/ root@mail:/var/lib/service-discover# ls -al total 8 drwxr-x--- 2 service-discover service-discover 4096 Oct 7 15:47 . drwxr-xr-x 53 root root 4096 Oct 7 19:15 .. root@mail:/var/lib/service-discover#
So there's definitely something up with the install script. Just a matter of getting to the bottom of it. 🙂
Thanks!,
Paul
I am seeing the exact same error in my attempted installs on the same system. I have posted here 4 days ago https://community.zextras.com/forum/carbonio-setup/service-discover-not-running/but my post is still awaiting moderation?
Bootstrap mode: new install
Installing LDAP configuration database...done.
Setting defaults...done.
Checking for port conflicts
Saving config in /opt/zextras/config.21291...done.
Operations logged to /tmp/zmsetup.20241008-212449.log
Setting replication password...done.
Setting Postfix password...done.
Setting amavis password...done.
Setting nginx password...done.
Initializing ldap...Connection refused at /opt/zextras/libexec/ldapinit.pm line 122.
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x50 pc=0x78c27e]
goroutine 1 [running]:
github.com/go-ldap/ldap.(*Conn).nextMessageID(...)
/home/agent/go/pkg/mod/github.com/go-ldap/ldap@v3.0.3+incompatible/conn.go:235
github.com/go-ldap/ldap.(*Conn).SimpleBind(0x0, 0xc00009ee28)
/home/agent/go/pkg/mod/github.com/go-ldap/ldap@v3.0.3+incompatible/bind.go:54 +0x13e
github.com/go-ldap/ldap.(*Conn).Bind(0xc0000e8e20?, {0xc0000e9040?, 0xc00009ef10?}, {0xc0000bdce8?, 0x828d60?})
/home/agent/go/pkg/mod/github.com/go-ldap/ldap@v3.0.3+incompatible/bind.go:116 +0x53
github.com/zextras/service-discover/pkg/carbonio.connect(0xc0000be8a0, 0x65?)
/tmp/service-discover/pkg/carbonio/ldap.go:216 +0x1eb
github.com/zextras/service-discover/pkg/carbonio.(*ldapContext).QueryAllServersWithService(0x8c9ade?, {0x8be909, 0x10})
/tmp/service-discover/pkg/carbonio/ldap.go:149 +0x65
github.com/zextras/service-discover/cmd/server/command/setup.(*Setup).isFirstInstance(0xc0001b4018, {0x96d450, 0xc0000a6e80})
/tmp/service-discover/cmd/server/command/setup/common.go:211 +0x7b
github.com/zextras/service-discover/cmd/server/command/setup.(*Setup).Run(0xc0001b4018, 0xc0001b4008)
/tmp/service-discover/cmd/server/command/setup/common.go:175 +0x1b9
reflect.Value.call({0x88b9e0?, 0xc0001b4018?, 0x8b5700?}, {0x8b6d2a, 0x4}, {0xc0000b8e58, 0x1, 0x1?})
/usr/lib/go/src/reflect/value.go:596 +0xca6
reflect.Value.Call({0x88b9e0?, 0xc0001b4018?, 0xc00009f8b8?}, {0xc0000b8e58?, 0x8b5700?, 0xc00009f7f8?})
/usr/lib/go/src/reflect/value.go:380 +0xb9
github.com/alecthomas/kong.callFunction({0x88b9e0?, 0xc0001b4018?, 0xc00009fa30?}, 0xc0001e3680)
/home/agent/go/pkg/mod/github.com/alecthomas/kong@v0.9.0/callbacks.go:98 +0x45a
github.com/alecthomas/kong.(*Context).RunNode(0xc000180180, 0xc0001b60f0, {0xc00009fef8, 0x1, 0x828d60?})
/home/agent/go/pkg/mod/github.com/alecthomas/kong@v0.9.0/context.go:774 +0x80e
github.com/alecthomas/kong.(*Context).Run(0xc000180180?, {0xc00009fef8?, 0xc00018bf08?, 0x10?})
/home/agent/go/pkg/mod/github.com/alecthomas/kong@v0.9.0/context.go:799 +0x132
main.main()
/tmp/service-discover/cmd/server/server.go:66 +0x8bc
gpg: can't open '/etc/zextras/service-discover/cluster-credentials.tar.gpg': No such file or directory
gpg: decrypt_message failed: No such file or directory
tar: This does not look like a tar archive
tar: consul-acl-secret.json: Not found in archive
tar: Exiting with failure status due to previous errors
@stefanodavid bumping this thread.
(There's a minimum 50 characters required so I'm including this useless text here)
Paul
@anomaly0617 Hi Paul, sorry for the late reply. Unfortunately, we have no resources to make a direct support like you are asking.
I also asked a few colleagues, who routinely install Carbonio with the script, about SIGSEGV occurrences, but they never experienced it. Other possible reasons for that crash I can think of are: a hardware (RAM) problem, or you are trying to install on a container, which is no supported installation (mostly due to service discover not working on containers).
Hmm. It shouldn't be either of those things. So, here's what I'm going to do... I'm going to stand up a new virtual machine and do the bare minimum of commands - like, I'll set the hostname, fqdn, and static IP address for the machine, but I'm not going to install any extra tools like cockpit or midnight commander or anything else - then run the script. And then see if I get the same error. I'll do what I can to log the entire process so if it does screw up, we and "rewind" and see what happens.
I can reproduce it with (depressingly) great reliability.
I can spin up a different host and see if I get the same error using different physical RAM
I will happily provide access to such a VM (it's in a sandbox until I am happy anyway) or perform any specific tests requested.
I am keen to see this potential Zimbra replacement "in the flesh" so to speak.
I can reproduce it with (depressingly) great reliability.
I can spin up a different host and see if I get the same error using different physical RAM
I will happily provide access to such a VM (it's in a sandbox until I am happy anyway) or perform any specific tests requested.
I am keen to see this potential Zimbra replacement "in the flesh" so to speak.
I got one working earlier this year, and was happy with it. Took an upgrade in September and it broke it.
Then I went to stand up a new server after trying and failing to resolve the Carbonio Mesh / service-discover errors and LDAP errors I was seeing all over the place. Now I can't get a new install to complete successfully.
Like you, I'm happy to give the ZExtras team access to a server in a sandbox to play in, if they are willing to do so. I just want to see the problems resolved.
Paul
So, let me invite you to join us on Telegram @CarbonioMail and try our installation script. It's fixed there.
It has been tested for many people and, AFAIK, it's working flawlessly.
Regards
I just rebooted my Ubuntu 22.04 server after routine maintenance and Carbonio fails to start AGAIN! Ldap is broken every since the latest version upgrade and I'm starting to lose patience with this software.
Starting directory server...Done. Starting config service...Failed. Starting zmconfigd...Failed to start zmconfigd. Starting mailbox...Done. Starting memcached...Done. Starting proxy...Done. Starting amavis...Done. Starting antispam...Done. Starting antivirus...Done. Starting opendkim...Done. Starting mta...Failed. Starting milter server...failed. zmmilterctl failed to start Starting saslauthd...done. postfix/postlog: starting the Postfix mail system Starting stats...Done. Starting service webapp...Done.
Then I check the status and it works:
zextras@mail:~$ zmcontrol status
Host mail.myriad.ca
amavis Running
antispam Running
antivirus Running
directory-server Running
mailbox Running
memcached Running
mta Running
opendkim Running
proxy Running
service webapp Running
service-discover Running
stats Running
config service Running
Hi @anahuac
thanks, I appreciate the invitation, unfortunately I don't do telegram.
Is the solution public? May I ask you to post it here please?
warm regards