Cannot add an appliance to cluster

[alp]

Hi all,

There is an interesting which I faced in my client environment. There are two physical appliance in the environment. One of them is new model and the other one is older. I followed the standard procedure to make them cluster.

• I configured ntp settings.
• I created new CA certificate from appliance interface and upload the other appliance.
• They are on the same subnet so there was nothing to block communication between them.

After these configurations, I try to add an appliance from GUI but error message popped out.

"Stack trace: com.scur.k.shared.exceptions.KClientServerException: Node "74007980-BB40-11E9-906E-0012795D9712" reports STATUS_ERROR_CONNECT:
co_distribute_add_cluster_node: ssl failure on message socket 15 while sending request - last action: ssl connect"

To be sure about services I checked mwg-coordinator services in both appliance and they were up and running. Then I try telnet between each other and it was successful. 

I captured the trafic for 12346 port in the appliance and show that connection is dropped after server hello. TLS version was 1.3 therefore I could not see certificate details.

Let's assume one the appliance is A and the other one is B.

To check certificate details I used openssl s_client. When I started to communication from appliance A to B with openssl tool, communication is hung at connected state. When I try to same thing in vice versa, results was same. But, when I try to start communication from Appliance A to A, communication was successfull. Then, I decided to use other appliance except from these and it completed handshake successfully with both appliances. 

After these I decided to start handshake from appliance A to B with TLS 1.1 with openssl but these time service did not like TLS version.

Do you have any idea what can cause this behaviour?

[fw_mon]

Hello @alp 

the TLS handshake on your screenshot is incomplete.

Do both appliance have the same MWG version? Upgrade them to the latest version and try again.

[alp]

They have same version. When I started a  handshake from other linux systems, it is completed successfullly. But when  I try to start it from one appliance to another. It stack just like this.

[fw_mon]

try to restart mwg-coordinator and also check mwg-coordinator.errors.log

[alp]

I have already restart both mwg-coordinator services. Also, I have checked the error. logs. I attached the log below.

[fw_mon]

yeah, I see, a ssl error, it correlates with the openssl output..

Try to reupload a cluster cert to both appliances and check it using UI.

[alp]

I did it many times but results were same. When I try to use another linux system to test connection to port 12346 connection was successful. 

[alp · Accepted solution]

The problem was MTU difference between switch and appliances. After changing them to default value(1500) problem is solved.