Anybody any idea why connections are set to 127.78.0.2? We are not using this range and I wonder why Carbonio is trying to use this IP.
Oct 07 13:31:05 mail systemd[1]: carbonio-mta-sidecar.service: Main process exited, code=exited, status=1/FAILURE
Oct 07 13:31:05 mail systemd[1]: carbonio-mta-sidecar.service: Failed with result 'exit-code'.
Oct 07 13:31:05 mail systemd[1]: carbonio-docs-editor-sidecar.service: Main process exited, code=exited, status=1/FAILURE
Oct 07 13:31:05 mail systemd[1]: carbonio-docs-editor-sidecar.service: Failed with result 'exit-code'.
Oct 07 13:31:05 mail java[33940]: 13:31:05.645 [main] ERROR com.zextras.carbonio.files.Boot - Service stopped unexpectedly: org.postgresql.util.PSQLException: Connection to 127.78.0.2:20000 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
Oct 07 13:31:05 mail java[33940]: Exception in thread "main" java.lang.NullPointerException
Oct 07 13:31:05 mail java[33940]: at com.zextras.carbonio.files.Boot.boot(Boot.java:63)
Oct 07 13:31:05 mail java[33940]: at com.zextras.carbonio.files.Boot.main(Boot.java:33)
Oct 07 13:31:05 mail systemd[1]: carbonio-files.service: Main process exited, code=exited, status=1/FAILURE
Oct 07 13:31:05 mail systemd[1]: carbonio-files.service: Failed with result 'exit-code'.
Oct 07 13:31:06 mail systemd[1]: service-discover.service: Scheduled restart job, restart counter is at 90.
Oct 07 13:31:06 mail systemd[1]: Stopped Service discover server for Zextras.
Oct 07 13:31:06 mail systemd[1]: Starting Service discover server for Zextras...
Oct 07 13:31:06 mail service-discoverd[34012]: local service-discover server NOT present in ldap/zimbraServiceEnabled attribute mail.xxxx.nl
Oct 07 13:31:06 mail systemd[1]: service-discover.service: Main process exited, code=exited, status=236/SMACK_PROCESS_LABEL
Oct 07 13:31:06 mail systemd[1]: service-discover.service: Failed with result 'exit-code'.
Oct 07 13:31:06 mail systemd[1]: Failed to start Service discover server for Zextras.
Anybody any idea why connections are set to 127.78.0.2? We are not using this range and I wonder why Carbonio is trying to use this IP.
Oct 07 13:31:05 mail systemd[1]: carbonio-mta-sidecar.service: Main process exited, code=exited, status=1/FAILURE
Oct 07 13:31:05 mail systemd[1]: carbonio-mta-sidecar.service: Failed with result 'exit-code'.
Oct 07 13:31:05 mail systemd[1]: carbonio-docs-editor-sidecar.service: Main process exited, code=exited, status=1/FAILURE
Oct 07 13:31:05 mail systemd[1]: carbonio-docs-editor-sidecar.service: Failed with result 'exit-code'.
Oct 07 13:31:05 mail java[33940]: 13:31:05.645 [main] ERROR com.zextras.carbonio.files.Boot - Service stopped unexpectedly: org.postgresql.util.PSQLException: Connection to 127.78.0.2:20000 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
Oct 07 13:31:05 mail java[33940]: Exception in thread "main" java.lang.NullPointerException
Oct 07 13:31:05 mail java[33940]: at com.zextras.carbonio.files.Boot.boot(Boot.java:63)
Oct 07 13:31:05 mail java[33940]: at com.zextras.carbonio.files.Boot.main(Boot.java:33)
Oct 07 13:31:05 mail systemd[1]: carbonio-files.service: Main process exited, code=exited, status=1/FAILURE
Oct 07 13:31:05 mail systemd[1]: carbonio-files.service: Failed with result 'exit-code'.
Oct 07 13:31:06 mail systemd[1]: service-discover.service: Scheduled restart job, restart counter is at 90.
Oct 07 13:31:06 mail systemd[1]: Stopped Service discover server for Zextras.
Oct 07 13:31:06 mail systemd[1]: Starting Service discover server for Zextras...
Oct 07 13:31:06 mail service-discoverd[34012]: local service-discover server NOT present in ldap/zimbraServiceEnabled attribute mail.xxxx.nl
Oct 07 13:31:06 mail systemd[1]: service-discover.service: Main process exited, code=exited, status=236/SMACK_PROCESS_LABEL
Oct 07 13:31:06 mail systemd[1]: service-discover.service: Failed with result 'exit-code'.
Oct 07 13:31:06 mail systemd[1]: Failed to start Service discover server for Zextras.
Hi @horizon,
The Carbonio community edition you're using is quite old, we have implemented several changes to the service discover in the meantime. I suggest updating the package to the latest available version (both carbonio-*, service-discover, and pending setup), verifying DNS resolution, and trying again.
By the stack trace, it seems the LDAP is not properly working and/or the service discover cannot obtain the list of Mesh servers.
About 127.78.0.2; this is the internal network used by the service mesh.
Hello Arman,
I am using the latest versions.....
After a reboot of Ubuntu, still the same issue:
Oct 10 12:17:10 mail systemd[1]: service-discover.service: Scheduled restart job, restart counter is at 17.
Oct 10 12:17:10 mail systemd[1]: Stopped Service discover server for Zextras.
Oct 10 12:17:10 mail systemd[1]: Starting Service discover server for Zextras...
Oct 10 12:17:10 mail service-discoverd[10213]: local service-discover server NOT present in ldap/zimbraServiceEnabled attribute mail.xxxxx.n l
Oct 10 12:17:10 mail systemd[1]: service-discover.service: Main process exited, code=exited, status=236/SMACK_PROCESS_LABEL
Oct 10 12:17:10 mail systemd[1]: service-discover.service: Failed with result 'exit-code'.
Oct 10 12:17:10 mail systemd[1]: Failed to start Service discover server for Zextras.
Oct 10 12:17:11 mail java[10181]: 12:17:11.180 [main] ERROR com.zextras.carbonio.files.Boot - Service stopped unexpectedly: org.postgresql.util.P SQLException: Connection to 127.78.0.2:20000 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP co nnections.
Also these errors:
Oct 10 12:20:57 mail consul[14933]: Failed looking up sidecar proxy info for carbonio-docs-connector: Get "http://127.0.0.1:8500/v1/agent/services": dial tcp 127.0.0.1:8500: c onnect: connection refused
Oct 10 12:20:57 mail systemd[1]: carbonio-docs-connector-sidecar.service: Main process exited, code=exited, status=1/FAILURE
Oct 10 12:20:57 mail systemd[1]: carbonio-docs-connector-sidecar.service: Failed with result 'exit-code'.
Oct 10 12:20:57 mail consul[14935]: Failed looking up sidecar proxy info for carbonio-files: Get "http://127.0.0.1:8500/v1/agent/services": dial tcp 127.0.0.1:8500: connect: c onnection refused
Oct 10 12:20:57 mail consul[14934]: Failed looking up sidecar proxy info for carbonio-files-db: Get "http://127.0.0.1:8500/v1/agent/services": dial tcp 127.0.0.1:8500: connect : connection refused
Oct 10 12:20:57 mail systemd[1]: carbonio-files-sidecar.service: Main process exited, code=exited, status=1/FAILURE
Oct 10 12:20:57 mail systemd[1]: carbonio-files-sidecar.service: Failed with result 'exit-code'.
Oct 10 12:20:57 mail systemd[1]: carbonio-files-db-sidecar.service: Main process exited, code=exited, status=1/FAILURE
Oct 10 12:20:57 mail systemd[1]: carbonio-files-db-sidecar.service: Failed with result 'exit-code'.
Oct 10 12:20:57 mail consul[14936]: Failed looking up sidecar proxy info for carbonio-preview: Get "http://127.0.0.1:8500/v1/agent/services": dial tcp 127.0.0.1:8500: connect: connection refused
Oct 10 12:20:57 mail systemd[1]: carbonio-preview-sidecar.service: Main process exited, code=exited, status=1/FAILURE
Oct 10 12:20:57 mail systemd[1]: carbonio-preview-sidecar.service: Failed with result 'exit-code'.
Oct 10 12:20:57 mail consul[14937]: Failed looking up sidecar proxy info for carbonio-storages: Get "http://127.0.0.1:8500/v1/agent/services": dial tcp 127.0.0.1:8500: connect : connection refused
Oct 10 12:20:57 mail systemd[1]: carbonio-storages-sidecar.service: Main process exited, code=exited, status=1/FAILURE
Oct 10 12:20:57 mail systemd[1]: carbonio-storages-sidecar.service: Failed with result 'exit-code'.
Oct 10 12:20:59 mail systemd[1]: service-discover.service: Scheduled restart job, restart counter is at 32.
Oct 10 12:20:59 mail systemd[1]: Stopped Service discover server for Zextras.
Oct 10 12:20:59 mail systemd[1]: Starting Service discover server for Zextras...
Oct 10 12:20:59 mail service-discoverd[14976]: local service-discover server NOT present in ldap/zimbraServiceEnabled attribute mail.xxxx.nl
Oct 10 12:20:59 mail systemd[1]: service-discover.service: Main process exited, code=exited, status=236/SMACK_PROCESS_LABEL
Oct 10 12:20:59 mail systemd[1]: service-discover.service: Failed with result 'exit-code'.
Oct 10 12:20:59 mail systemd[1]: Failed to start Service discover server for Zextras.
Oct 10 12:20:59 mail systemd[1]: carbonio-user-management.service: Scheduled restart job, restart counter is at 30.
Version info:
zextras@mail:~$ zmcontrol -v
Release 22.7.2 Community Edition.
zextras@mail:~$ zmprov gs mail.xxxx.nl zimbraServiceEnabled
# name mail.xxxxx.nl
zimbraServiceEnabled: amavis
zimbraServiceEnabled: antivirus
zimbraServiceEnabled: antispam
zimbraServiceEnabled: opendkim
zimbraServiceEnabled: stats
zimbraServiceEnabled: mta
zimbraServiceEnabled: directory-server
zimbraServiceEnabled: memcached
zimbraServiceEnabled: logger
zimbraServiceEnabled: proxy
zimbraServiceEnabled: mailbox
zimbraServiceEnabled: zimbra
zimbraServiceEnabled: zimbraAdmin
zimbraServiceEnabled: service
zimbraServiceEnabled: zimlet
@stefanodavid same issue, service-discover not start after carbonio update, from simple apt update.
oct 14 09:08:44 carbonio zmconfigd[3843571]: Watchdog: service antivirus status is OK.
oct 14 09:08:44 carbonio zmconfigd[3843571]: File "/opt/zextras/common/lib/jylibs/state.py", line 796, in rewriteConfig for line in open(fr).readlines():
oct 14 09:08:44 carbonio zmconfigd[3843571]: Rewrite failed: [Errno 2] No such file or directory: '/opt/zextras/mailboxd/etc/zimbra.web.xml.in' ([Errno 2] No such fil>
oct 14 09:08:44 carbonio zmconfigd[3843571]: File "/opt/zextras/common/lib/jylibs/state.py", line 796, in rewriteConfig for line in open(fr).readlines():
oct 14 09:08:44 carbonio zmconfigd[3843571]: Rewrite failed: [Errno 2] No such file or directory: '/opt/zextras/mailboxd/etc/jetty-env.xml.in' ([Errno 2] No such file>
oct 14 09:08:44 carbonio zmconfigd[3843571]: All rewrite threads completed in 0.01 sec
oct 14 09:08:44 carbonio zmconfigd[3843571]: All restarts completed in 0.00 sec
Executing /etc/zextras/pending-setups.d/carbonio-files.sh
Insert the cluster credential password:
Error determining policy ID: Get "http://127.0.0.1:8500/v1/acl/policies": dial tcp 127.0.0.1:8500: connect: connection refused
Setup failed: Cannot update policy for carbonio-files-policy
Setup script failed, keeping it
oct 14 08:24:18 carbonio systemd[1]: service-discover.service: Main process exited, code=exited, status=236/SMACK_PROCESS_LABEL
oct 14 08:24:18 carbonio systemd[1]: service-discover.service: Failed with result 'exit-code'.
oct 14 08:24:18 carbonio systemd[1]: Failed to start Service discover server for Zextras.
oct 14 08:24:33 carbonio systemd[1]: service-discover.service: Scheduled restart job, restart counter is at 290692.
oct 14 08:24:33 carbonio systemd[1]: Stopped Service discover server for Zextras.
oct 14 08:24:33 carbonio systemd[1]: Starting Service discover server for Zextras...
oct 14 08:24:33 carbonio service-discoverd[3810018]: local service-discover server NOT present in ldap/zimbraServiceEnabled attribute carbonio.example.com
oct 14 08:24:33 carbonio systemd[1]: service-discover.service: Main process exited, code=exited, status=236/SMACK_PROCESS_LABEL
oct 14 08:24:33 carbonio systemd[1]: service-discover.service: Failed with result 'exit-code'.
oct 14 08:24:33 carbonio systemd[1]: Failed to start Service discover server for Zextras.
Hi @horizon @linuxcuba,
Did you configure discover service on localhost? If yes it can't work , you need to use your network private ip.
Change ip from this file and restart discover-service ( systemctl restart service-discover ) :
nano /etc/zextras/service-discover/config.json -> to change ip systemctl restart service-discover -> to restart service systemctl status service-discover -> to check status
For zmconfig error try this :
su - zextras carbonio ms `zmhostname` +zimbraServiceInstalled 'service-discover' +zimbraServiceEnabled 'service-discover'
@mgarbo is config by default
/etc/zextras/service-discover/config.json
{
"bind_addr": "10.10.10.10"
}
zextras@carbonio:~$ carbonio ms `zmhostname` +zimbraServiceInstalled 'service-discover' +zimbraServiceEnabled 'service-discover'
ERROR: zclient.IO_ERROR (invoke Connection refused, server: localhost) (cause: java.net.ConnectException Connection refused)
nothing worked for me.
@linuxcuba please paste result of these commands :
as root : ps faux |grep ldap hostname -f
If ldap server is not running you have the root cause of your problem .
Try to resolve hostname then try to bypass mailbox command with :
carbonio -l ms `zmhostname` +zimbraServiceInstalled 'service-discover' +zimbraServiceEnabled 'service-discover'
@mgarbo ldap ir running non problem
root@carbonio:~# ps faux |grep ldap root 68995 0.0 0.0 8836 656 pts/1 S+ 10:26 0:00 \_ grep --color=auto ldap zextras 562 0.0 0.4 84093536 43832 ? Ssl 09:41 0:02 /opt/zextras/common/libexec/slapd -l LOCAL0 -u zextras -h ldap://carbonio.corvanit.com:389 ldapi:/// -F /opt/zextras/data/ldap/config zextras 4395 0.0 0.1 26744 17444 ? S 09:42 0:02 /usr/bin/perl -w /opt/zextras/libexec/zmstat-ldap root@carbonio:~# netstat -ptln | grep 389 tcp 0 0 10.10.10.10:389 0.0.0.0:* LISTEN 562/slapd root@carbonio:~# hostname -f carbonio.corvanit.com
@mgarbo this worked.
zmprov -l ms `zmhostname` +zimbraServiceInstalled 'service-discover' +zimbraServiceEnabled 'service-discover'
service-discorver is up and running, There are no pending-setups to run. Exiting!
now zimbra webapp and others is no running.
zextras@carbonio:~$ zmcontrol status
Host carbonio.corvanit.com
amavis Running
antispam Running
antivirus Running
directory-server Running
logger Running
mailbox Running
memcached Running
mta Running
opendkim Running
proxy Running
service webapp Running
service-discover Running
stats Running
zimbra webapp Stopped
zmmailboxdctl is not running.
zimbraAdmin webapp Stopped
zmmailboxdctl is not running.
zimlet webapp Stopped
zmmailboxdctl is not running.
zmconfigd Running
mailbox up and run, but stopping immediately.
zextras@carbonio:~$ zmcontrol restart
Host carbonio.corvanit.com
Stopping zmconfigd...Done.
Stopping zimlet webapp...Done.
Stopping zimbraAdmin webapp...Done.
Stopping zimbra webapp...Done.
Stopping service webapp...Done.
Stopping stats...Done.
Stopping mta...Done.
Stopping cbpolicyd...Done.
Stopping opendkim...Done.
Stopping amavis...Done.
Stopping antivirus...Done.
Stopping antispam...Done.
Stopping proxy...Done.
Stopping memcached...Done.
Stopping mailbox...Done.
Stopping logger...Done.
Stopping directory-server...Done.
Host carbonio.corvanit.com
Starting directory server...Done.
Starting zmconfigd...Done.
Starting logger...Done.
Starting mailbox...Done.
Starting memcached...Done.
Starting proxy...Done.
Starting amavis...Done.
Starting antispam...Done.
Starting antivirus...Done.
Starting opendkim...Done.
Starting mta...Done.
Starting stats...Done.
Starting service webapp...Done.
Starting zimbra webapp...Done.
Starting zimbraAdmin webapp...Done.
Starting zimlet webapp...Done.
zextras@carbonio:~$ zmcontrol status
Host carbonio.corvanit.com
amavis Running
antispam Running
antivirus Running
directory-server Running
logger Running
mailbox Running
memcached Running
mta Running
opendkim Running
proxy Running
service webapp Running
service-discover Running
stats Running
zimbra webapp Running
zimbraAdmin webapp Running
zimlet webapp Running
zmconfigd Running
zextras@carbonio:~$ zmcontrol status
Host carbonio.corvanit.com
amavis Running
antispam Running
antivirus Running
directory-server Running
logger Running
mailbox Stopped
zmmailboxdctl is not running.
memcached Running
mta Running
opendkim Running
proxy Running
service webapp Stopped
zmmailboxdctl is not running.
service-discover Running
stats Running
zimbra webapp Stopped
zmmailboxdctl is not running.
zimbraAdmin webapp Stopped
zmmailboxdctl is not running.
zimlet webapp Stopped
zmmailboxdctl is not running.
zmconfigd Running
@linuxcuba Check your mailbox log for errors :
/opt/zextras/log/mailbox.log /opt/zextras/log/zmmailboxd.out
Did you change mailbox port on carbonio-bootstrap command?
@mgarbo not not change nothing.
-- The unit carbonio-preview-sidecar.service has entered the 'failed' state with result 'exit-code'.
[WARN] agent: Check socket connection failed: check=service:carbonio-docs-editor-sidecar-proxy:1 error="dial tcp 127.0.0.1:21001: connect: connection refused"
[WARN] agent: Check is now critical: check=service:carbonio-docs-editor-sidecar-proxy:1
INFO [uvicorn.access.send:431] 127.0.0.1:60156 - "GET /health/ready/ HTTP/1.1" 200"
[WARN] agent: Check socket connection failed: check=service:carbonio-mta-sidecar-proxy:1 error="dial tcp 127.0.0.1:21005: connect: connection refused"
[WARN] agent: Check is now critical: check=service:carbonio-mta-sidecar-proxy:1
[WARN] agent: Check is now critical: check=service:carbonio-files
[WARN] agent: Check socket connection failed: check=service:carbonio-docs-editor error="dial tcp [::1]:10000: connect: connection refused"
[WARN] agent: Check is now critical: check=service:carbonio-docs-editor
[WARN] agent: Check socket connection failed: check=service:carbonio-mailbox error="dial tcp [::1]:8080: connect: connection refused"
[WARN] agent: Check is now critical: check=service:carbonio-mailbox
[WARN] agent: Check is now critical: check=service:carbonio-user-management
oct 14 10:49:26 carbonio service-discoverd[112337]: 2022-10-14T10:49:26.870+0200 [ERROR] agent.http: Request error: method=GET url=/v1/agent/services from=127.0.0.1:34162 error="ACL not found"
oct 14 10:49:26 carbonio consul[113045]: Failed looking up sidecar proxy info for carbonio-user-management: Unexpected response code: 403 (ACL not found)
oct 14 10:49:26 carbonio service-discoverd[112337]: 2022-10-14T10:49:26.874+0200 [ERROR] agent.http: Request error: method=GET url=/v1/agent/services from=127.0.0.1:34164 error="ACL not found"
oct 14 10:49:26 carbonio consul[113044]: Failed looking up sidecar proxy info for carbonio-docs-editor: Unexpected response code: 403 (ACL not found)
oct 14 10:49:26 carbonio systemd[1]: carbonio-user-management-sidecar.service: Main process exited, code=exited, status=1/FAILURE
-- The job identifier is 221447874.
oct 14 09:30:55 carbonio java[3910358]: Exception in thread "main" com.sun.xml.ws.wsdl.parser.InaccessibleWSDLException: 2 counts of InaccessibleWSDLException.
oct 14 09:30:55 carbonio java[3910358]: java.net.ConnectException: Connection refused
oct 14 09:30:55 carbonio java[3910358]: java.net.ConnectException: Connection refused
oct 14 09:30:55 carbonio java[3910358]: at com.sun.xml.ws.wsdl.parser.RuntimeWSDLParser.tryWithMex(RuntimeWSDLParser.java:275)
oct 14 09:30:55 carbonio java[3910358]: at com.sun.xml.ws.wsdl.parser.RuntimeWSDLParser.parse(RuntimeWSDLParser.java:246)
oct 14 09:30:55 carbonio java[3910358]: at com.sun.xml.ws.wsdl.parser.RuntimeWSDLParser.parse(RuntimeWSDLParser.java:209)
oct 14 09:30:55 carbonio java[3910358]: at com.sun.xml.ws.wsdl.parser.RuntimeWSDLParser.parse(RuntimeWSDLParser.java:178)
oct 14 09:30:55 carbonio java[3910358]: at com.sun.xml.ws.client.WSServiceDelegate.parseWSDL(WSServiceDelegate.java:364)
oct 14 09:30:55 carbonio java[3910358]: at com.sun.xml.ws.client.WSServiceDelegate.<init>(WSServiceDelegate.java:322)
oct 14 09:30:55 carbonio java[3910358]: at com.sun.xml.ws.client.WSServiceDelegate.<init>(WSServiceDelegate.java:231)
oct 14 09:30:55 carbonio java[3910358]: at com.sun.xml.ws.client.WSServiceDelegate.<init>(WSServiceDelegate.java:212)
oct 14 09:30:55 carbonio java[3910358]: at com.sun.xml.ws.client.WSServiceDelegate.<init>(WSServiceDelegate.java:208)
oct 14 09:30:55 carbonio java[3910358]: at com.sun.xml.ws.spi.ProviderImpl.createServiceDelegate(ProviderImpl.java:119)
oct 14 09:30:55 carbonio java[3910358]: at javax.xml.ws.Service.<init>(Service.java:112)
oct 14 09:30:55 carbonio java[3910358]: at javax.xml.ws.Service.create(Service.java:765)
oct 14 09:30:55 carbonio java[3910358]: at client.SoapClient.lambda$init$0(SoapClient.java:54)
oct 14 09:30:55 carbonio java[3910358]: at java.base/java.util.stream.Streams$RangeIntSpliterator.forEachRemaining(Streams.java:104)
oct 14 09:30:55 carbonio java[3910358]: at java.base/java.util.stream.IntPipeline$Head.forEach(IntPipeline.java:593)
oct 14 09:30:55 carbonio java[3910358]: at client.SoapClient.init(SoapClient.java:52)
oct 14 09:30:55 carbonio java[3910358]: at com.zextras.carbonio.user_management.Boot.main(Boot.java:30)
oct 14 09:30:55 carbonio systemd[1]: carbonio-user-management.service: Main process exited, code=exited, status=1/FAILURE
-- The unit carbonio-files-db-sidecar.service has entered the 'failed' state with result 'exit-code'.
oct 14 09:32:08 carbonio java[3911729]: 09:32:08.161 [main] ERROR com.zextras.carbonio.files.Boot - Service stopped unexpectedly: org.postgresql.util.PSQLException: Connection to 127.78.0.2:20000 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
oct 14 09:32:08 carbonio java[3911729]: Exception in thread "main" java.lang.NullPointerException
oct 14 09:32:08 carbonio java[3911729]: at com.zextras.carbonio.files.Boot.boot(Boot.java:63)
oct 14 09:32:08 carbonio java[3911729]: at com.zextras.carbonio.files.Boot.main(Boot.java:33)
oct 14 09:32:08 carbonio systemd[1]: carbonio-files.service: Main process exited, code=exited, status=1/FAILURE