Notifications
Clear all
Topic starter
11/27/2023 13:28
Hello!,
If anybody have a problem with GUI webmail / admin panel after upgrade Carbonio from 23.10.0 to 23.11.0? I do everything following the https://docs.zextras.com/carbonio/html/release/upgrade.html but after all and restart Carbonio it looks like:
So, it's almost empty and you can't do anything.
In logs, after refresh URLs I have f.e:
lusterLoadAssignment lis 26 14:06:47 poczta.XXXXXXXXXXX consul[7593]: [2023-11-26 14:06:47.140][7593][info][upstream] [source/server/lds_api.cc:78] lds: add/update listener 'public_listener:0.0.0.0:21013' lis 26 14:06:50 poczta.XXXXXXXXXXX carbonio-preview-start.sh[2035]: "[2023-11-26 14:06:50,110] INFO [uvicorn.access.send:478] 127.0.0.1:54700 - "GET /health/ready/ HTTP/1.1" 200" lis 26 14:06:54 poczta.XXXXXXXXXXX prometheus[1254]: ts=2023-11-26T13:06:54.046Z caller=consul.go:295 level=error component="discovery manager scrape" discovery=consul config=process msg="Error retrieving datacenter name" err="Unexpected response code: 403 (Permission denied)" lis 26 14:06:55 poczta.XXXXXXXXXXX carbonio-preview-start.sh[2030]: "[2023-11-26 14:06:55,113] INFO [uvicorn.access.send:478] 127.0.0.1:54710 - "GET /health/ready/ HTTP/1.1" 200" lis 26 14:06:55 poczta.XXXXXXXXXXX consul[7601]: [2023-11-26 14:06:55.623][7601][warning][config] [source/common/config/grpc_subscription_impl.cc:119] gRPC config: initial fetch timed out for type.googleapis.com/envoy.config.endpoint.v3.C lusterLoadAssignment lis 26 14:06:55 poczta.XXXXXXXXXXX consul[7601]: [2023-11-26 14:06:55.623][7601][warning][config] [source/common/config/grpc_subscription_impl.cc:119] gRPC config: initial fetch timed out for type.googleapis.com/envoy.config.endpoint.v3.C lusterLoadAssignment lis 26 14:06:55 poczta.XXXXXXXXXXX consul[7601]: [2023-11-26 14:06:55.629][7601][info][upstream] [source/server/lds_api.cc:78] lds: add/update listener 'public_listener:0.0.0.0:21008' lis 26 14:07:00 poczta.XXXXXXXXXXX carbonio-preview-start.sh[2035]: "[2023-11-26 14:07:00,115] INFO [uvicorn.access.send:478] 127.0.0.1:35028 - "GET /health/ready/ HTTP/1.1" 200" lis 26 14:07:02 poczta.XXXXXXXXXXX carbonio-files[12542]: 14:07:02.412 [Xmemcached-Reactor-0] ERROR remoting - Reactor dispatch events error lis 26 14:07:02 poczta.XXXXXXXXXXX carbonio-files[12542]: java.io.IOException: Connect to 10.0.60.24:11211 fail,Connection refused lis 26 14:07:02 poczta.XXXXXXXXXXX carbonio-files[12542]: at net.rubyeye.xmemcached.impl.MemcachedConnector.onConnect(MemcachedConnector.java:426) lis 26 14:07:02 poczta.XXXXXXXXXXX carbonio-files[12542]: at com.google.code.yanf4j.nio.impl.Reactor.dispatchEvent(Reactor.java:317) lis 26 14:07:02 poczta.XXXXXXXXXXX carbonio-files[12542]: at com.google.code.yanf4j.nio.impl.Reactor.run(Reactor.java:177) lis 26 14:07:05 poczta.XXXXXXXXXXX carbonio-preview-start.sh[2030]: "[2023-11-26 14:07:05,117] INFO [uvicorn.access.send:478] 127.0.0.1:35030 - "GET /health/ready/ HTTP/1.1" 200" lis 26 14:07:09 poczta.XXXXXXXXXXX prometheus[1254]: ts=2023-11-26T13:07:09.047Z caller=consul.go:295 level=error component="discovery manager scrape" discovery=consul config=process msg="Error retrieving datacenter name" err="Unexpected response code: 403 (Permission denied)" lis 26 14:07:10 poczta.XXXXXXXXXXX carbonio-preview-start.sh[2030]: "[2023-11-26 14:07:10,120] INFO [uvicorn.access.send:478] 127.0.0.1:42502 - "GET /health/ready/ HTTP/1.1" 200" lis 26 14:07:15 poczta.XXXXXXXXXXX carbonio-preview-start.sh[2030]: "[2023-11-26 14:07:15,122] INFO [uvicorn.access.send:478] 127.0.0.1:42516 - "GET /health/ready/ HTTP/1.1" 200" lis 26 14:07:20 poczta.XXXXXXXXXXX carbonio-preview-start.sh[2035]: "[2023-11-26 14:07:20,124] INFO [uvicorn.access.send:478] 127.0.0.1:51398 - "GET /health/ready/ HTTP/1.1" 200" lis 26 14:07:24 poczta.XXXXXXXXXXX prometheus[1254]: ts=2023-11-26T13:07:24.048Z caller=consul.go:295 level=error component="discovery manager scrape" discovery=consul config=process msg="Error retrieving datacenter name" err="Unexpected response code: 403 (Permission denied)" lis 26 14:07:25 poczta.XXXXXXXXXXX carbonio-preview-start.sh[2035]: "[2023-11-26 14:07:25,127] INFO [uvicorn.access.send:478] 127.0.0.1:51404 - "GET /health/ready/ HTTP/1.1" 200" lis 26 14:07:30 poczta.XXXXXXXXXXX carbonio-preview-start.sh[2035]: "[2023-11-26 14:07:30,129] INFO [uvicorn.access.send:478] 127.0.0.1:53696 - "GET /health/ready/ HTTP/1.1" 200" lis 26 14:07:35 poczta.XXXXXXXXXXX carbonio-preview-start.sh[2035]: "[2023-11-26 14:07:35,131] INFO [uvicorn.access.send:478] 127.0.0.1:53706 - "GET /health/ready/ HTTP/1.1" 200" lis 26 14:07:39 poczta.XXXXXXXXXXX prometheus[1254]: ts=2023-11-26T13:07:39.049Z caller=consul.go:295 level=error component="discovery manager scrape" discovery=consul config=process msg="Error retrieving datacenter name" err="Unexpected response code: 403 (Permission denied)" lis 26 14:07:40 poczta.XXXXXXXXXXX carbonio-preview-start.sh[2035]: "[2023-11-26 14:07:40,133] INFO [uvicorn.access.send:478] 127.0.0.1:60178 - "GET /health/ready/ HTTP/1.1" 200" lis 26 14:07:45 poczta.XXXXXXXXXXX carbonio-preview-start.sh[2035]: "[2023-11-26 14:07:45,136] INFO [uvicorn.access.send:478] 127.0.0.1:60186 - "GET /health/ready/ HTTP/1.1" 200" lis 26 14:07:50 poczta.XXXXXXXXXXX carbonio-preview-start.sh[2035]: "[2023-11-26 14:07:50,139] INFO [uvicorn.access.send:478] 127.0.0.1:48892 - "GET /health/ready/ HTTP/1.1" 200" lis 26 14:07:54 poczta.XXXXXXXXXXX prometheus[1254]: ts=2023-11-26T13:07:54.050Z caller=consul.go:295 level=error component="discovery manager scrape" discovery=consul config=process msg="Error retrieving datacenter name" err="Unexpected response code: 403 (Permission denied)" lis 26 14:07:55 poczta.XXXXXXXXXXX carbonio-preview-start.sh[2035]: "[2023-11-26 14:07:55,142] INFO [uvicorn.access.send:478] 127.0.0.1:48900 - "GET /health/ready/ HTTP/1.1" 200" lis 26 14:08:00 poczta.XXXXXXXXXXX carbonio-preview-start.sh[2035]: "[2023-11-26 14:08:00,145] INFO [uvicorn.access.send:478] 127.0.0.1:38096 - "GET /health/ready/ HTTP/1.1" 200" lis 26 14:08:02 poczta.XXXXXXXXXXX carbonio-files[12542]: 14:08:02.413 [Xmemcached-Reactor-0] ERROR remoting - Reactor dispatch events error lis 26 14:08:02 poczta.XXXXXXXXXXX carbonio-files[12542]: java.io.IOException: Connect to 10.0.60.24:11211 fail,Connection refused lis 26 14:08:02 poczta.XXXXXXXXXXX carbonio-files[12542]: at net.rubyeye.xmemcached.impl.MemcachedConnector.onConnect(MemcachedConnector.java:426) lis 26 14:08:02 poczta.XXXXXXXXXXX carbonio-files[12542]: at com.google.code.yanf4j.nio.impl.Reactor.dispatchEvent(Reactor.java:317) lis 26 14:08:02 poczta.XXXXXXXXXXX carbonio-files[12542]: at com.google.code.yanf4j.nio.impl.Reactor.run(Reactor.java:177) lis 26 14:08:05 poczta.XXXXXXXXXXX carbonio-preview-start.sh[2030]: "[2023-11-26 14:08:05,154] INFO [uvicorn.access.send:478] 127.0.0.1:38108 - "GET /health/ready/ HTTP/1.1" 200" lis 26 14:08:09 poczta.XXXXXXXXXXX prometheus[1254]: ts=2023-11-26T13:08:09.051Z caller=consul.go:295 level=error component="discovery manager scrape" discovery=consul config=process msg="Error retrieving datacenter name" err="Unexpected response code: 403 (Permission denied)" lis 26 14:08:10 poczta.XXXXXXXXXXX carbonio-preview-start.sh[2035]: "[2023-11-26 14:08:10,161] INFO [uvicorn.access.send:478] 127.0.0.1:45970 - "GET /health/ready/ HTTP/1.1" 200" lis 26 14:08:14 poczta.XXXXXXXXXXX consul[7569]: [2023-11-26 14:08:14.734][7569][warning][config] [bazel-out/k8-opt/bin/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:101] DeltaAggregatedResources gRPC config stream closed: 13, lis 26 14:08:14 poczta.XXXXXXXXXXX consul[7601]: [2023 ....... lis 26 14:08:14 poczta.XXXXXXXXXXX consul[7620]: [2023-11-26 14:08:14.735][7620][warning][config] [bazel-out/k8-opt/bin/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:101] DeltaAggregatedResources gRPC config stream closed: 13, lis 26 14:08:14 poczta.XXXXXXXXXXX prometheus[1254]: ts=2023-11-26T13:08:14.736Z caller=consul.go:522 level=error component="discovery manager scrape" discovery=consul config=blackbox datacenter=dc1 msg="Error refreshing service" service=carbonio-user-management-sidecar-proxy tags= err="Unexpected response code: 500 (rpc error: code = Canceled desc = grpc: the client connection is closing)" lis 26 14:08:14 poczta.XXXXXXXXXXX prometheus[1254]: ts=2023-11-26T13:08:14.736Z caller=consul.go:522 level=error component="discovery manager scrape" discovery=consul config=blackbox datacenter=dc1 msg="Error refreshing service" service=carbonio-mailbox-sidecar-proxy tags= err="Unexpected response code: 500 (rpc error: code = Canceled desc = grpc: the client connection is closing)" lis 26 14:08:14 poczta.XXXXXXXXXXX prometheus[1254]: ts=2023-11-26T13:08:14.736Z caller=consul.go:522 level=error component="discovery manager scrape" discovery=consul config=blackbox datacenter=dc1 msg="Error refreshing service" service=carbonio-files-sidecar-proxy tags= err="Unexpected response code: 500 (rpc error: code = Canceled desc = grpc: the client connection is closing)" lis 26 14:08:14 poczta.XXXXXXXXXXX prometheus[1254]: ts=2023-11-26T13:08:14.736Z caller=consul.go:522 level=error component="discovery manager scrape" discovery=consul config=blackbox datacenter=dc1 msg="Error refreshing service" service=carbonio-files-db-sidecar-proxy tags= err="Unexpected response code: 500 (rpc error: code = Canceled desc = grpc: the client connection is closing)" lis 26 14:08:14 poczta.XXXXXXXXXXX prometheus[1254]: ts=2023-11-26T13:08:14.736Z caller=consul.go:522 level=error component="discovery manager scrape" discovery=consul config=blackbox datacenter=dc1 msg="Error refreshing service" service=carbonio-mta-sidecar-proxy tags= err="Unexpected response code: 500 (rpc error: code = Canceled desc = grpc: the client connection is closing)" lis 26 14:08:14 poczta.XXXXXXXXXXX promet
11/28/2023 10:25
@dart This seems a problem with memcached. Can you check whether it is running/configured correctly?
Topic starter
11/28/2023 11:02
not now, but it is production server :> but it is clue... Now, in 23.10.0 everything works, but memcached listens at:
LISTEN 0 1024 127.0.1.1:11211 0.0.0.0:* users:(("memcached",pid=15517,fd=22))
But in 23.11.0 it try to connect to interface IP address:
lis 26 14:05:02 poczta.zeto.lublin.pl carbonio-files[12542]: java.io.IOException: Connect to 10.0.60.24:11211 fail,Connection refused
But there's something else strange. It refers to carbonio-files, but i don't use to `files`. It means, carbonioFeatureFilesEnabled is `false` for all COS. Hmm....
Topic starter
12/14/2023 11:44
hello,
And have somebody similar problem? Yesterday I tried upgrade again, but before it I set `carbonio prov ms `zmhostname` zimbraMemcachedBindAddress 10.0.60.24`, that memcached listen on IP interface, in stead of `lo` (I checked it). But after upgrade, situation is this same 🙁