Consul/Envoy TLS pr...
 
Notifications
Clear all

Consul/Envoy TLS problem after upgrade

3 Posts
2 Users
0 Reactions
322 Views
(@cvidal)
Joined: 13 years ago
Posts: 21
Topic starter  

I migrated Carbonio 23 to 24.5 on a Centos-8 platform. I have 5 servers, and started by migrating the database node. I decided to upgrade the OS too to CentOS-9. I created a new VM with CentOS-9, installed the Carbonio packages I had in the old server, then exported/imported LDAP and Postgres, and copied the directories /etc/zextras, /etc/carbonio/, /var/lib/service-discover and /opt/zextras/conf. Everything started fine eventually, and the other servers were running fine.

Then I upgraded the other 4 servers (proxy+mta, video, docs and mailbox), but left them in CentOS-8.

The problem I have is that the mailbox server cannot login to the Postgres database after the update. I checked the DB login/password via consul and they are correct. Tracing the communication between the envoy sidecars with Wireshark it seems that the TLS negotiation fails.

Is it possible to re-initialize the consul/envoy relationship between the mailbox server and the database server? If yes, how? Where are the certificates stored?

Thank you for any clues!!


   
Quote
(@cvidal)
Joined: 13 years ago
Posts: 21
Topic starter  

I finally found the problem! There is an inconsistency in the file /etc/carbonio/mailbox-db/service-discover/intentions.json, it should allow 'carbonio-advanced' to access the database. This is how it looks like now:

{
  "Kind": "service-intentions",
  "Name": "carbonio-mailbox-db",
  "Sources": [
    {
      "Name": "carbonio-mailbox",
      "Action": "allow"
    },
    {
      "Name": "carbonio-advanced",
      "Action": "allow"
    }
  ]
}

  This was the error log in "/opt/zextras/log/mailbox.log":

2024-05-29 09:53:32,956 INFO [main] [] extensions - url jdbc:postgresql://127.78.0.25:20000/core?adaptiveFetch=....
2024-05-29 09:53:32,956 INFO [main] [] extensions - ServerNames 127.78.0.25
2024-05-29 09:53:32,957 INFO [main] [] extensions - PortNumbers 20000
2024-05-29 09:53:33,008 ERROR [main] [] extensions - Error getting connection from data source org.postgresql.ds.PGSimpleDataSource@21ae26b0

The port 20000 IP 127.78.0.25 corresponds to carbonio-advanced (according to  /etc/zextras/service-discover/carbonio-advanced.hcl). So "mailbox" is trying to access the Postgres "core" database thru the envoy "carbonio-advanced", that was not authorized by consul.

This inconsistency comes because I installed Carbonio DB on RHEL9, while the rest of the cluster is on RHEL8.

I checked the RHEL8 configuration file and it is correct. It also adds carbonio-storages.

So, it is my fault, but it is good to know that you have to use the same OS version in all the nodes!


   
ReplyQuote
(@sharif)
Admin
Joined: 2 years ago
Posts: 591
 

Posted by: @cvidal

The port 20000 IP 127.78.0.25 corresponds to carbonio-advanced (according to  /etc/zextras/service-discover/carbonio-advanced.hcl). So "mailbox" is trying to access the Postgres "core" database thru the envoy "carbonio-advanced", that was not authorized by consul.

This inconsistency comes because I installed Carbonio DB on RHEL9, while the rest of the cluster is on RHEL8.

I checked the RHEL8 configuration file and it is correct. It also adds carbonio-storages.

So, it is my fault, but it is good to know that you have to use the same OS version in all the nodes!

 

@ cvidal

Hi,

Thank you so much for sharing the details.

I believe it would be helpful for many of our users.  🙏

Regards,

Sharif


   
ReplyQuote