The mailbox.log has lot of entries like:

2024-02-14 11:08:51,595 ERROR   mailbox - Error during mailbox creation
com.zimbra.common.service.ServiceException: system failure: unable to read SQL statements from /opt/zextras/mailboxd/../db/create_database.sql
	at com.zimbra.cs.mailbox.MailboxManager.createMailboxInternal(MailboxManager.java:905) ~
	at com.zimbra.cs.mailbox.MailboxManager.createMailbox(MailboxManager.java:878) ~
	at com.zimbra.cs.mailbox.MailboxManager.getMailboxByAccountId(MailboxManager.java:387) ~
	at com.zimbra.cs.mailbox.MailboxManager.getMailboxByAccountId(MailboxManager.java:342) ~
	at com.zimbra.cs.mailbox.MailboxManager.getMailboxByAccount(MailboxManager.java:279) ~
	at com.zimbra.cs.mailbox.MailboxManager.getMailboxByAccount(MailboxManager.java:235) ~
Caused by: java.io.FileNotFoundException: /opt/zextras/mailboxd/../db/create_database.sql (No such file or directory)
com.zimbra.common.service.ServiceException: system failure: unable to read SQL statements from /opt/zextras/mailboxd/../db/create_database.sql
	at com.zimbra.cs.mailbox.MailboxManager.createMailboxInternal(MailboxManager.java:905) ~
	at com.zimbra.cs.mailbox.MailboxManager.createMailbox(MailboxManager.java:878) ~
	at com.zimbra.cs.mailbox.MailboxManager.getMailboxByAccountId(MailboxManager.java:387) ~
	at com.zimbra.cs.mailbox.MailboxManager.getMailboxByAccountId(MailboxManager.java:342) ~
	at com.zimbra.cs.mailbox.MailboxManager.getMailboxByAccount(MailboxManager.java:279) ~
	at com.zimbra.cs.mailbox.MailboxManager.getMailboxByAccount(MailboxManager.java:235) ~
Caused by: java.io.FileNotFoundException: /opt/zextras/mailboxd/../db/create_database.sql (No such file or directory)
2024-02-14 11:09:28,016 INFO    mailbox - total space required to copy items = 34342

And the audit.log has nothing except account addition, disable and close entries.

Looks like mailbox is not getting created due to some sql file non availability. I am not able to find this file anywhere in the whole server.

But can't still understand why the domain is not getting deleted??!!

Best regards,

Raghav

Could you please check mailbox.log or audit.log for any hints?

Thanks,

Sharif

After rebooting the server and restarting all services as you suggested, systemctl list-unit carbonio* is showing all units.

However, I am still not able to delete the non functional domain. Force delete despite leaving for 30 minutes, there is no success. Only the account is getting closed (Soft Delete).

root@mail:/home/ubuntu# systemctl list-units carbonio*
  UNIT                                               LOAD   ACTIVE SUB     DESCRIPTION                                     
  carbonio-clamav-sidecar.service                    loaded active running Carbonio clamav sidecar proxy                   
  carbonio-clamav-signature-provider-sidecar.service loaded active running NGINX Carbonio ClamAV signature provider sidecar
  carbonio-docs-connector-sidecar.service            loaded active running Carbonio docs connector sidecar proxy           
  carbonio-docs-connector.service                    loaded active running Carbonio docs connector daemon                  
  carbonio-docs-editor-sidecar.service               loaded active running Carbonio Docs server sidecar proxy              
  carbonio-docs-editor.service                       loaded active running Carbonio Docs Server                            
  carbonio-files-db-sidecar.service                  loaded active running Carbonio Files database sidecar                 
  carbonio-files-sidecar.service                     loaded active running Carbonio Files sidecar proxy                    
  carbonio-files.service                             loaded active running Carbonio Files daemon                           
  carbonio-mailbox-sidecar.service                   loaded active running Mailbox carbonio proxy sidecar                  
  carbonio-mta-sidecar.service                       loaded active running Carbonio mta sidecar proxy                      
  carbonio-preview-sidecar.service                   loaded active running Carbonio Preview sidecar proxy                  
  carbonio-preview.service                           loaded active running Carbonio Preview daemon                         
  carbonio-prometheus-consul-exporter.service        loaded active running Carbonio Prometheus Consul exporter             
  carbonio-prometheus-mysqld-exporter.service        loaded active running Carbonio Prometheus MySQL exporter              
  carbonio-prometheus-nginx-exporter.service         loaded active running Carbonio Prometheus NGINX exporter              
  carbonio-prometheus-node-exporter.service          loaded active running Carbonio Prometheus Node Exporter               
  carbonio-prometheus-openldap-exporter.service      loaded active running Carbonio Prometheus OpenLDAP exporter           
  carbonio-proxy-sidecar.service                     loaded active running NGINX carbonio proxy sidecar                    
  carbonio-storages-sidecar.service                  loaded active running Storages-CE sidecar proxy                       
  carbonio-storages.service                          loaded active running Carbonio Storages-CE daemon                     
  carbonio-tasks-db-sidecar.service                  loaded active running Carbonio Tasks database sidecar                 
  carbonio-tasks-sidecar.service                     loaded active running Carbonio Tasks sidecar proxy                    
  carbonio-user-management-sidecar.service           loaded active running Carbonio User Management sidecar proxy          
  carbonio-user-management.service                   loaded active running Carbonio User Management daemon                 
  carbonio.service                                   loaded active exited  LSB: Carbonio Services                          

LOAD   = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB    = The low-level unit activation state, values depend on unit type.

26 loaded units listed. Pass --all to see loaded but inactive units, too.
To show all installed unit files use 'systemctl list-unit-files'.

Best regards,

Raghav

Hi,

systemctl list-units carbonio* should show carbonio unit files like this:

root@mail:~# systemctl list-units carbonio*
  UNIT                                               LOAD   ACTIVE SUB     DESCRIPTION
  carbonio-clamav-sidecar.service                    loaded active running Carbonio clamav sidecar proxy
  carbonio-clamav-signature-provider-sidecar.service loaded active running NGINX Carbonio ClamAV signature provider sidecar
  carbonio-docs-connector-db-sidecar.service         loaded active running Carbonio Docs Connector database sidecar
  carbonio-docs-connector-sidecar.service            loaded active running Carbonio docs connector sidecar proxy
  carbonio-docs-connector.service                    loaded active running Carbonio docs connector daemon
  carbonio-docs-editor-sidecar.service               loaded active running Carbonio Docs server sidecar proxy
  carbonio-docs-editor.service                       loaded active running Carbonio Docs Server
  carbonio-files-db-sidecar.service                  loaded active running Carbonio Files database sidecar
  carbonio-files-sidecar.service                     loaded active running Carbonio Files sidecar proxy
  carbonio-files.service                             loaded active running Carbonio Files daemon
  carbonio-mailbox-db-sidecar.service                loaded active running Carbonio Mailbox database sidecar
  carbonio-mailbox-sidecar.service                   loaded active running Mailbox carbonio proxy sidecar
  carbonio-message-broker-sidecar.service            loaded active running Carbonio message broker sidecar proxy
  carbonio-message-broker.service                    loaded active running Carbonio message broker daemon
  carbonio-message-dispatcher-auth-sidecar.service   loaded active running Carbonio Message Dispatcher auth sidecar proxy
  carbonio-message-dispatcher-auth.service           loaded active running Carbonio Message Dispatcher auth daemon
  carbonio-message-dispatcher-db-sidecar.service     loaded active running Carbonio Message Dispatcher database sidecar
  carbonio-message-dispatcher-http-sidecar.service   loaded active running Carbonio Message Dispatcher HTTP sidecar proxy
  carbonio-message-dispatcher-xmpp-sidecar.service   loaded active running Carbonio Message Dispatcher XMPP sidecar proxy
  carbonio-message-dispatcher.service                loaded active running Carbonio Message Dispatcher daemon
  carbonio-mta-sidecar.service                       loaded active running Carbonio mta sidecar proxy
  carbonio-preview-sidecar.service                   loaded active running Carbonio Preview sidecar proxy
  carbonio-preview.service                           loaded active running Carbonio Preview daemon
  carbonio-prometheus-consul-exporter.service        loaded active running Carbonio Prometheus Consul exporter
  carbonio-prometheus-mysqld-exporter.service        loaded active running Carbonio Prometheus MySQL exporter
  carbonio-prometheus-nginx-exporter.service         loaded active running Carbonio Prometheus NGINX exporter
  carbonio-prometheus-node-exporter.service          loaded active running Carbonio Prometheus Node Exporter
  carbonio-prometheus-openldap-exporter.service      loaded active running Carbonio Prometheus OpenLDAP exporter
  carbonio-proxy-sidecar.service                     loaded active running NGINX carbonio proxy sidecar
  carbonio-storages-sidecar.service                  loaded active running Storages-CE sidecar proxy
  carbonio-storages.service                          loaded active running Carbonio Storages-CE daemon
  carbonio-tasks-db-sidecar.service                  loaded active running Carbonio Tasks database sidecar
  carbonio-tasks-sidecar.service                     loaded active running Carbonio Tasks sidecar proxy
  carbonio-user-management-sidecar.service           loaded active running Carbonio User Management sidecar proxy
  carbonio-user-management.service                   loaded active running Carbonio User Management daemon
  carbonio-videoserver-sidecar.service               loaded active running Carbonio Videoserver sidecar proxy
  carbonio-videoserver.service                       loaded active running Zextras Videoserver
  carbonio-ws-collaboration-db-sidecar.service       loaded active running Carbonio Workstream Collaboration database sidecar
  carbonio-ws-collaboration-sidecar.service          loaded active running Carbonio Workstream Collaboration sidecar proxy
  carbonio-ws-collaboration.service                  loaded active running Carbonio Workstream Collaboration daemon
● carbonio.service                                   loaded failed failed  LSB: Carbonio Services

LOAD   = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB    = The low-level unit activation state, values depend on unit type.

41 loaded units listed. Pass --all to see loaded but inactive units, too.
To show all installed unit files use 'systemctl list-unit-files'.
root@mail:~#

Try rebooting the server, then

systemctl restart carbonio-message-broker
systemctl restart carbonio-videoserver
systemctl restart carbonio-ws-collaboration
systemctl restart carbonio-message-dispatcher
systemctl restart carbonio-tasks.service
systemctl restart carbonio-prometheus-nginx-exporter.service

Check again after all services get up and running:

zmcontrol status

Regards,

Sharif

Thanks for your quick response.

We also upgraded following the same procedure given the same link. Upgrade process went through smoothly. There were no issues.

But when the users of the one of our test domains, tried to login they started seeing the error message shown in the previous message above.

OS : Ubuntu 20.04.06 LTS

Coming to outputs of the command you have given:

1. systemctl list-units carbonio*

loaded units listed. Pass --all to see loaded but inactive units, too.
To show all installed unit files use 'systemctl list-unit-files'.

2. consul members

Node                                 Address                    Status  Type   Build      Protocol  DC  Segment
server-mail-exzatech-net  172.31.xx.xxx:8301   alive    server  1.10.12  2             dc1  <all>

3. systemctl status service-discover.service

Loaded:    loaded (/lib/systemd/system/service-discover.service; enabled; vendor preset: enabled)
      Active: active (running) since Tue 2024-02-13 10:31:58 UTC; 22h ago
       Docs: https://docs.zextras.com/
Main PID: 1949 (consul)
       asks: 12 (limit: 18945)
 Memory: 87.1M
  CGroup: /system.slice/service-discover.service
                └─1949 /usr/bin/consul agent -bootstrap-expect 1 -config-dir /etc/zextras/service-discover/ -server -retry-join=mail.exzatech.net

4. zmcontrol status

  amavis                Running
  antispam             Running
  antivirus              Running
  directory-server  Running
  mailbox               Running
  memcached        Running
  mta                     Running
  opendkim            Running
  proxy                  Running
  service webapp  Running
  service-discover Running
  stats                   Running
  config service    Running

Hope this helps in identifying the issue.

Best regards,

Raghav

Hi,

Sorry to hear about the trouble.

I just checked by upgrading from 23.12.0 to 24.1.0 with 3 domains. All of the are working and I simply followed the official doc https://docs.zextras.com/carbonio-ce/html/release/upgrade.html

I even deleted one of them.

What OS you are using?
Also share the output below commands:

systemctl list-units carbonio*
consul members
systemctl status service-discover.service
zmcontrol status

Regards,

Sharif

We recently upgraded from Zextras Carbonio CE from 23.12 to 24.1, post which barring 2 domains, on our Carbonio CE server the third and fourth stopped working. When the user logged in, it started showing "Some technical issues occurred while processing your request. Please try to refresh the page or go back to login page". All domains were working before this upgrade. Now I am not even able to delete the non-working domain and its users.

Can some one from community help us?

Best regards,

Raghav