Forums
Zextras Carbonio
Carbonio General
Problem after upgra...
 
Notifications
Clear all

Problem after upgrade to 23.11.0

 
Carbonio General
Last Post by dart 10 months ago
4 Posts
2 Users
0 Reactions
10.5 K Views
RSS
 dart
(@dart)
Joined: 11 months ago
Posts: 11
Topic starter 11/27/2023 13:28  

Hello!,

If anybody have a problem with GUI webmail / admin panel after upgrade Carbonio from 23.10.0 to 23.11.0? I do everything following the https://docs.zextras.com/carbonio/html/release/upgrade.html but after all and restart Carbonio it looks like:

 

So, it's almost empty and you can't do anything.

In logs, after refresh URLs I have f.e:

lusterLoadAssignment
lis 26 14:06:47 poczta.XXXXXXXXXXX consul[7593]: [2023-11-26 14:06:47.140][7593][info][upstream] [source/server/lds_api.cc:78] lds: add/update listener 'public_listener:0.0.0.0:21013'
lis 26 14:06:50 poczta.XXXXXXXXXXX carbonio-preview-start.sh[2035]: "[2023-11-26 14:06:50,110] INFO [uvicorn.access.send:478] 127.0.0.1:54700 - "GET /health/ready/ HTTP/1.1" 200"
lis 26 14:06:54 poczta.XXXXXXXXXXX prometheus[1254]: ts=2023-11-26T13:06:54.046Z caller=consul.go:295 level=error component="discovery manager scrape" discovery=consul config=process msg="Error retrieving datacenter name" err="Unexpected 
response code: 403 (Permission denied)"
lis 26 14:06:55 poczta.XXXXXXXXXXX carbonio-preview-start.sh[2030]: "[2023-11-26 14:06:55,113] INFO [uvicorn.access.send:478] 127.0.0.1:54710 - "GET /health/ready/ HTTP/1.1" 200"
lis 26 14:06:55 poczta.XXXXXXXXXXX consul[7601]: [2023-11-26 14:06:55.623][7601][warning][config] [source/common/config/grpc_subscription_impl.cc:119] gRPC config: initial fetch timed out for type.googleapis.com/envoy.config.endpoint.v3.C
lusterLoadAssignment
lis 26 14:06:55 poczta.XXXXXXXXXXX consul[7601]: [2023-11-26 14:06:55.623][7601][warning][config] [source/common/config/grpc_subscription_impl.cc:119] gRPC config: initial fetch timed out for type.googleapis.com/envoy.config.endpoint.v3.C
lusterLoadAssignment
lis 26 14:06:55 poczta.XXXXXXXXXXX consul[7601]: [2023-11-26 14:06:55.629][7601][info][upstream] [source/server/lds_api.cc:78] lds: add/update listener 'public_listener:0.0.0.0:21008'
lis 26 14:07:00 poczta.XXXXXXXXXXX carbonio-preview-start.sh[2035]: "[2023-11-26 14:07:00,115] INFO [uvicorn.access.send:478] 127.0.0.1:35028 - "GET /health/ready/ HTTP/1.1" 200"
lis 26 14:07:02 poczta.XXXXXXXXXXX carbonio-files[12542]: 14:07:02.412 [Xmemcached-Reactor-0] ERROR remoting - Reactor dispatch events error
lis 26 14:07:02 poczta.XXXXXXXXXXX carbonio-files[12542]: java.io.IOException: Connect to 10.0.60.24:11211 fail,Connection refused
lis 26 14:07:02 poczta.XXXXXXXXXXX carbonio-files[12542]:         at net.rubyeye.xmemcached.impl.MemcachedConnector.onConnect(MemcachedConnector.java:426)
lis 26 14:07:02 poczta.XXXXXXXXXXX carbonio-files[12542]:         at com.google.code.yanf4j.nio.impl.Reactor.dispatchEvent(Reactor.java:317)
lis 26 14:07:02 poczta.XXXXXXXXXXX carbonio-files[12542]:         at com.google.code.yanf4j.nio.impl.Reactor.run(Reactor.java:177)
lis 26 14:07:05 poczta.XXXXXXXXXXX carbonio-preview-start.sh[2030]: "[2023-11-26 14:07:05,117] INFO [uvicorn.access.send:478] 127.0.0.1:35030 - "GET /health/ready/ HTTP/1.1" 200"
lis 26 14:07:09 poczta.XXXXXXXXXXX prometheus[1254]: ts=2023-11-26T13:07:09.047Z caller=consul.go:295 level=error component="discovery manager scrape" discovery=consul config=process msg="Error retrieving datacenter name" err="Unexpected 
response code: 403 (Permission denied)"
lis 26 14:07:10 poczta.XXXXXXXXXXX carbonio-preview-start.sh[2030]: "[2023-11-26 14:07:10,120] INFO [uvicorn.access.send:478] 127.0.0.1:42502 - "GET /health/ready/ HTTP/1.1" 200"
lis 26 14:07:15 poczta.XXXXXXXXXXX carbonio-preview-start.sh[2030]: "[2023-11-26 14:07:15,122] INFO [uvicorn.access.send:478] 127.0.0.1:42516 - "GET /health/ready/ HTTP/1.1" 200"
lis 26 14:07:20 poczta.XXXXXXXXXXX carbonio-preview-start.sh[2035]: "[2023-11-26 14:07:20,124] INFO [uvicorn.access.send:478] 127.0.0.1:51398 - "GET /health/ready/ HTTP/1.1" 200"
lis 26 14:07:24 poczta.XXXXXXXXXXX prometheus[1254]: ts=2023-11-26T13:07:24.048Z caller=consul.go:295 level=error component="discovery manager scrape" discovery=consul config=process msg="Error retrieving datacenter name" err="Unexpected 
response code: 403 (Permission denied)"
lis 26 14:07:25 poczta.XXXXXXXXXXX carbonio-preview-start.sh[2035]: "[2023-11-26 14:07:25,127] INFO [uvicorn.access.send:478] 127.0.0.1:51404 - "GET /health/ready/ HTTP/1.1" 200"
lis 26 14:07:30 poczta.XXXXXXXXXXX carbonio-preview-start.sh[2035]: "[2023-11-26 14:07:30,129] INFO [uvicorn.access.send:478] 127.0.0.1:53696 - "GET /health/ready/ HTTP/1.1" 200"
lis 26 14:07:35 poczta.XXXXXXXXXXX carbonio-preview-start.sh[2035]: "[2023-11-26 14:07:35,131] INFO [uvicorn.access.send:478] 127.0.0.1:53706 - "GET /health/ready/ HTTP/1.1" 200"
lis 26 14:07:39 poczta.XXXXXXXXXXX prometheus[1254]: ts=2023-11-26T13:07:39.049Z caller=consul.go:295 level=error component="discovery manager scrape" discovery=consul config=process msg="Error retrieving datacenter name" err="Unexpected 
response code: 403 (Permission denied)"
lis 26 14:07:40 poczta.XXXXXXXXXXX carbonio-preview-start.sh[2035]: "[2023-11-26 14:07:40,133] INFO [uvicorn.access.send:478] 127.0.0.1:60178 - "GET /health/ready/ HTTP/1.1" 200"
lis 26 14:07:45 poczta.XXXXXXXXXXX carbonio-preview-start.sh[2035]: "[2023-11-26 14:07:45,136] INFO [uvicorn.access.send:478] 127.0.0.1:60186 - "GET /health/ready/ HTTP/1.1" 200"
lis 26 14:07:50 poczta.XXXXXXXXXXX carbonio-preview-start.sh[2035]: "[2023-11-26 14:07:50,139] INFO [uvicorn.access.send:478] 127.0.0.1:48892 - "GET /health/ready/ HTTP/1.1" 200"
lis 26 14:07:54 poczta.XXXXXXXXXXX prometheus[1254]: ts=2023-11-26T13:07:54.050Z caller=consul.go:295 level=error component="discovery manager scrape" discovery=consul config=process msg="Error retrieving datacenter name" err="Unexpected 
response code: 403 (Permission denied)"
lis 26 14:07:55 poczta.XXXXXXXXXXX carbonio-preview-start.sh[2035]: "[2023-11-26 14:07:55,142] INFO [uvicorn.access.send:478] 127.0.0.1:48900 - "GET /health/ready/ HTTP/1.1" 200"
lis 26 14:08:00 poczta.XXXXXXXXXXX carbonio-preview-start.sh[2035]: "[2023-11-26 14:08:00,145] INFO [uvicorn.access.send:478] 127.0.0.1:38096 - "GET /health/ready/ HTTP/1.1" 200"
lis 26 14:08:02 poczta.XXXXXXXXXXX carbonio-files[12542]: 14:08:02.413 [Xmemcached-Reactor-0] ERROR remoting - Reactor dispatch events error
lis 26 14:08:02 poczta.XXXXXXXXXXX carbonio-files[12542]: java.io.IOException: Connect to 10.0.60.24:11211 fail,Connection refused
lis 26 14:08:02 poczta.XXXXXXXXXXX carbonio-files[12542]:         at net.rubyeye.xmemcached.impl.MemcachedConnector.onConnect(MemcachedConnector.java:426)
lis 26 14:08:02 poczta.XXXXXXXXXXX carbonio-files[12542]:         at com.google.code.yanf4j.nio.impl.Reactor.dispatchEvent(Reactor.java:317)
lis 26 14:08:02 poczta.XXXXXXXXXXX carbonio-files[12542]:         at com.google.code.yanf4j.nio.impl.Reactor.run(Reactor.java:177)
lis 26 14:08:05 poczta.XXXXXXXXXXX carbonio-preview-start.sh[2030]: "[2023-11-26 14:08:05,154] INFO [uvicorn.access.send:478] 127.0.0.1:38108 - "GET /health/ready/ HTTP/1.1" 200"
lis 26 14:08:09 poczta.XXXXXXXXXXX prometheus[1254]: ts=2023-11-26T13:08:09.051Z caller=consul.go:295 level=error component="discovery manager scrape" discovery=consul config=process msg="Error retrieving datacenter name" err="Unexpected 
response code: 403 (Permission denied)"
lis 26 14:08:10 poczta.XXXXXXXXXXX carbonio-preview-start.sh[2035]: "[2023-11-26 14:08:10,161] INFO [uvicorn.access.send:478] 127.0.0.1:45970 - "GET /health/ready/ HTTP/1.1" 200"
lis 26 14:08:14 poczta.XXXXXXXXXXX consul[7569]: [2023-11-26 14:08:14.734][7569][warning][config] [bazel-out/k8-opt/bin/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:101] DeltaAggregatedResources gRPC 
config stream closed: 13,
lis 26 14:08:14 poczta.XXXXXXXXXXX consul[7601]: [2023

.......

lis 26 14:08:14 poczta.XXXXXXXXXXX consul[7620]: [2023-11-26 14:08:14.735][7620][warning][config] [bazel-out/k8-opt/bin/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:101] DeltaAggregatedResources gRPC config stream closed: 13,
lis 26 14:08:14 poczta.XXXXXXXXXXX prometheus[1254]: ts=2023-11-26T13:08:14.736Z caller=consul.go:522 level=error component="discovery manager scrape" discovery=consul config=blackbox datacenter=dc1 msg="Error refreshing service" service=carbonio-user-management-sidecar-proxy tags= err="Unexpected response code: 500 (rpc error: code = Canceled desc = grpc: the client connection is closing)"
lis 26 14:08:14 poczta.XXXXXXXXXXX prometheus[1254]: ts=2023-11-26T13:08:14.736Z caller=consul.go:522 level=error component="discovery manager scrape" discovery=consul config=blackbox datacenter=dc1 msg="Error refreshing service" service=carbonio-mailbox-sidecar-proxy tags= err="Unexpected response code: 500 (rpc error: code = Canceled desc = grpc: the client connection is closing)"
lis 26 14:08:14 poczta.XXXXXXXXXXX prometheus[1254]: ts=2023-11-26T13:08:14.736Z caller=consul.go:522 level=error component="discovery manager scrape" discovery=consul config=blackbox datacenter=dc1 msg="Error refreshing service" service=carbonio-files-sidecar-proxy tags= err="Unexpected response code: 500 (rpc error: code = Canceled desc = grpc: the client connection is closing)"
lis 26 14:08:14 poczta.XXXXXXXXXXX prometheus[1254]: ts=2023-11-26T13:08:14.736Z caller=consul.go:522 level=error component="discovery manager scrape" discovery=consul config=blackbox datacenter=dc1 msg="Error refreshing service" service=carbonio-files-db-sidecar-proxy tags= err="Unexpected response code: 500 (rpc error: code = Canceled desc = grpc: the client connection is closing)"
lis 26 14:08:14 poczta.XXXXXXXXXXX prometheus[1254]: ts=2023-11-26T13:08:14.736Z caller=consul.go:522 level=error component="discovery manager scrape" discovery=consul config=blackbox datacenter=dc1 msg="Error refreshing service" service=carbonio-mta-sidecar-proxy tags= err="Unexpected response code: 500 (rpc error: code = Canceled desc = grpc: the client connection is closing)"
lis 26 14:08:14 poczta.XXXXXXXXXXX promet

   
Quote
 stefanodavid
(@stefanodavid)
Joined: 3 years ago
Posts: 203
11/28/2023 10:25  

@dart This seems a problem with memcached. Can you check whether it is running/configured correctly?


   
ReplyQuote
 dart
(@dart)
Joined: 11 months ago
Posts: 11
Topic starter 11/28/2023 11:02  

not now, but it is production server :> but it is clue... Now, in 23.10.0 everything works, but memcached listens at:

LISTEN     0      1024             127.0.1.1:11211             0.0.0.0:*     users:(("memcached",pid=15517,fd=22))

But in 23.11.0 it try to connect to interface IP address:

lis 26 14:05:02 poczta.zeto.lublin.pl carbonio-files[12542]: java.io.IOException: Connect to 10.0.60.24:11211 fail,Connection refused

But there's something else strange. It refers to carbonio-files, but i don't use to `files`. It means, carbonioFeatureFilesEnabled is `false` for all COS. Hmm....


   
ReplyQuote
 dart
(@dart)
Joined: 11 months ago
Posts: 11
Topic starter 12/14/2023 11:44  

hello,

And have somebody similar problem? Yesterday I tried upgrade again, but before it I set `carbonio prov ms `zmhostname` zimbraMemcachedBindAddress 10.0.60.24`, that memcached listen on IP interface, in stead of `lo` (I checked it). But after upgrade, situation is this same 🙁


   
ReplyQuote
Forum Jump:
  Previous Topic
Next Topic