rabbitmq-server crypto issue after upgrade to 24.7

I have exactly the same problem on RHEL8. I have posted in the announcement thread but it still waits moderation.

Job for carbonio-message-broker.service failed because the control process exited with error code. Looking at the service log (journalctl -u carbonio-message-broker.service):

=ERROR REPORT==== 24-Jul-2024::10:05:46.646484 ===
 Unable to load crypto library. Failed with error:
 "load, Library load-call unsuccessful (227)."
 =WARNING REPORT==== 24-Jul-2024::10:05:46.847359 ===
 The on_load function for module crypto returned:
 {error,{load,"Library load-call unsuccessful (227)."}}
 =CRASH REPORT==== 24-Jul-2024::10:05:46.848519 ===
   crasher:
     initial call: credentials_obfuscation_svc:init/1
     pid: <0.128.0>
     registered_name: []
     exception error: undefined function crypto:strong_rand_bytes/1
       in function  credentials_obfuscation_svc:check/3 (src/credentials_obfuscation_svc.erl, line 197)
       in call from credentials_obfuscation_svc:init_state/0 (src/credentials_obfuscation_svc.erl, line 169)
       in call from gen_server:init_it/2 (gen_server.erl, line 962)
       in call from gen_server:init_it/6 (gen_server.erl, line 917)
     ancestors: [credentials_obfuscation_sup,<0.126.0>]
     message_queue_len: 0
     messages: []
     links: [<0.127.0>]
     dictionary: []
     trap_exit: false
     status: running
     heap_size: 987
     stack_size: 28
     reductions: 296
   neighbours:
 =SUPERVISOR REPORT==== 24-Jul-2024::10:05:47.029920 ===
     supervisor: {local,credentials_obfuscation_sup}
     errorContext: start_error
     reason: {undef,
                 [{crypto,strong_rand_bytes,[128],[]},
                  {credentials_obfuscation_svc,check,3,
                      [{file,"src/credentials_obfuscation_svc.erl"},
                       {line,197}]},
                  {credentials_obfuscation_svc,init_state,0,
                      [{file,"src/credentials_obfuscation_svc.erl"},
                       {line,169}]},
                  {gen_server,init_it,2,[{file,"gen_server.erl"},{line,962}]},
                  {gen_server,init_it,6,[{file,"gen_server.erl"},{line,917}]},
                  {proc_lib,init_p_do_apply,3,
                      [{file,"proc_lib.erl"},{line,241}]}]}
     offender: [{pid,undefined},
                {id,credentials_obfuscaton_svc},
                {mfargs,{credentials_obfuscation_svc,start_link,[]}},
                {restart_type,permanent},
                {significant,false},
                {shutdown,5000},
                {child_type,worker}]

It seems like an issue with the supplied erlang installation and SSL. Try to load the crypto module does not work:

su - zextras
erl
# in the erl shell try to run crypto version
crypto:version().
=ERROR REPORT==== 24-Jul-2024::10:31:41.809455 ===
Unable to load crypto library. Failed with error:
"load, Library load-call unsuccessful (227)."

=WARNING REPORT==== 24-Jul-2024::10:31:41.812732 ===
The on_load function for module crypto returned:
{error,{load,"Library load-call unsuccessful (227)."}}

** exception error: undefined function crypto:version/0

Same issue here. I'm running Ubuntu 22.04.4

Any idea?

Same here, Carbonio CE 24.5.0 on Ubuntu 20.04 (or RHEL 8) to Carbonio CE 24.7.0 on Ubuntu 22.04.

Followed the upgrade instructions. Then read Sharif's Carbonio – Upgrade Ubuntu from 20.04 to 22.04 and needed to edit the files in /etc/apt/sources.list.d

but pending-setups -a gives me Job for carbonio-message-broker.service failed because the control process exited with error code.

systemctl status message-broker.service

carbonio-message-broker.service - Carbonio message broker daemon
Loaded: loaded (/lib/systemd/system/carbonio-message-broker.service; enabled; vendor preset: enabled)
Drop-In: /etc/systemd/system/carbonio-message-broker.service.d
└─override.conf
Active: activating (auto-restart) (Result: exit-code) since Wed 2024-07-24 18:30:51 UTC; 7s ago
Process: 114325 ExecStart=/opt/zextras/common/bin/rabbitmq-server (code=exited, status=1/FAILURE)
Main PID: 114325 (code=exited, status=1/FAILURE)

same here while trying a fresh install under 2204 🙁 Hoping for a solution

For ubuntu 20.04.6, try to downgrade carbonio-erlang, internal test show that the basic functions working, no chat.

Same issue, Carbonio CE 24.5.0 on Ubuntu 22.04 to Carbonio CE 24.7.0 on Ubuntu 22.04.

For chat, try downgrade carbonio-message-dispatcher

Same problem as the guys here. Carbonio CE 24.5.0 on Ubuntu 22.04 upgraded to Carbonio CE 24.7.0.

Same problem here.

The same problem also shows for mongooseim. The only thing that doesn't work seems to be chat which nobody uses here, but still, the services are constantly restarted and puking into the logs.

I do think it has to do with the OpenSSL update, the erlang stuff must probably somehow recompiled, but I don't know how to do that or if that can be done at all without some extra development libs.

Has anybody found a solution yet?

Hello, I have the same problem. Since I haven't made many configurations post-installation, I was thinking about reinstalling directly, but seeing that several people have had the same issue, I suppose a solution will be provided soon.

Thank you!

I have the same issue, Carbonio CE 24.5.0 on Ubuntu 22.04 to Carbonio CE 24.7.0 on Ubuntu 22.04.

Same issue here. 

10:20:46.460 [error] Unable to load crypto library. Failed with error:
"load, Library load-call unsuccessful (227)."

10:20:46.463 [warning] The on_load function for module crypto returned:
{error,{load,"Library load-call unsuccessful (227)."}}
when=2024-07-24T10:20:46.463095+00:00 level=warning pid=<0.430.0> at=: unstructured_log="The on_load function for mod>
{error,{load,\"Library load-call unsuccessful (227).\"}}\
"

Any idea?

Got the same issue:

Config entry written: service-defaults/carbonio-message-broker
Config entry written: service-intentions/carbonio-message-broker
Configuration reload triggered
Job for carbonio-message-broker.service failed because the control process exited with error code.
See "systemctl status carbonio-message-broker.service" and "journalctl -xeu carbonio-message-broker.service" for details.
Script for message broker terminated with error
Setup script failed, keeping it
-------------journalctl -xeu carbonio-message-broker.service--------------------root@mail:~# journalctl -xeu carbonio-message-broker.service
Jul 26 16:35:45 mail.uhlhost.net rabbitmq-server[1675984]:                               [{file,"gen_server.erl"},{line,917}]},
Jul 26 16:35:45 mail.uhlhost.net rabbitmq-server[1675984]:                           {proc_lib,init_p_do_apply,3,
Jul 26 16:35:45 mail.uhlhost.net rabbitmq-server[1675984]:                               [{file,"proc_lib.erl"},{line,241}]}]}}},
Jul 26 16:35:45 mail.uhlhost.net rabbitmq-server[1675984]:              {credentials_obfuscation_app,start,[normal,[]]}}
Jul 26 16:35:45 mail.uhlhost.net rabbitmq-server[1675984]:     type: transient
Jul 26 16:35:47 mail.uhlhost.net rabbitmq-server[1675984]: Kernel pid terminated (application_controller) ("{application_start_failure,credentials_obfuscation,{{shutd>
Jul 26 16:35:47 mail.uhlhost.net rabbitmq-server[1675984]: 
Jul 26 16:35:47 mail.uhlhost.net rabbitmq-server[1675984]: Crash dump is being written to: erl_crash.dump...done
Jul 26 16:35:47 mail.uhlhost.net systemd[1]: carbonio-message-broker.service: Main process exited, code=exited, status=1/FAILURE
░░ Subject: Unit process exited
░░ Defined-By: systemd
░░ Support:  http://www.ubuntu.com/support 
░░ 
░░ An ExecStart= process belonging to unit carbonio-message-broker.service has exited.
░░ 
░░ The process' exit code is 'exited' and its exit status is 1.
Jul 26 16:35:47 mail.uhlhost.net systemd[1]: carbonio-message-broker.service: Failed with result 'exit-code'.
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support:  http://www.ubuntu.com/support 
░░ 
░░ The unit carbonio-message-broker.service has entered the 'failed' state with result 'exit-code'.
Jul 26 16:35:47 mail.uhlhost.net systemd[1]: Failed to start Carbonio message broker daemon.
░░ Subject: A start job for unit carbonio-message-broker.service has failed
░░ Defined-By: systemd
░░ Support:  http://www.ubuntu.com/support 
░░ 
░░ A start job for unit carbonio-message-broker.service has finished with a failure.
░░ 
░░ The job identifier is 75695 and the job result is failed.

I have done following, and my server was running. 

I then run all other pending setups manually and excluded the broker one.

su - zextras
zmcontrol status
zmcontrol restart

I also noticed that upon restarting the services, milter service refused to stop, this behaviour was not present in my case prior to uprade.

Stopping milter server...failed.

Also the number of my defunct / Zombie processes increased:

root@mail:/etc/carbonio# ps -eo pid,ppid,stat,cmd | grep Z
   2439    2120 Z    [consul] <defunct>
   2464    2133 Z    [consul] <defunct>
   2529    2130 Z    [consul] <defunct>
   2568    2125 Z    [consul] <defunct>
   2617    2123 Z    [consul] <defunct>
   2643    2117 Z    [consul] <defunct>
1674511 1674455 Z    [consul] <defunct>
1674653 1674587 Z    [consul] <defunct>
1674816 1674759 Z    [consul] <defunct>
1674960 1674933 Z    [consul] <defunct>
1675138 1675100 Z    [consul] <defunct>
1675286 1675258 Z    [consul] <defunct>
1714819 1714776 Z    [consul] <defunct>
1715248 1715214 Z    [consul] <defunct>
1715548 1715519 Z    [consul] <defunct>
1715974 1715933 Z    [consul] <defunct>
1716179 1716151 Z    [consul] <defunct>
1716475 1716389 Z    [consul] <defunct>
1716484 1716374 Z    [consul] <defunct>
1716511 1716429 Z    [consul] <defunct>
1716727 1716702 Z    [consul] <defunct>
1716938 1716902 Z    [consul] <defunct>
1717170 1717103 Z    [consul] <defunct>
1717340 1717318 Z    [consul] <defunct>
1717506 1717487 Z    [consul] <defunct>
1717772 1717740 Z    [consul] <defunct>