Error: Transport endpoint is not connected

[nashcoop]

Can someone shine a light on what this error is about?  I'm seeing repeated errors on my dashboard from my director appliance in the cluster which says "Failed to synchronize policy to SaaS (Origin: saas-connector, ID: 3201)" and then when I look in the "mwg-saas-connector.errors.log" I see the error below. This started a few days ago, and I didn't make any configuration changes to the appliance prior to this.  I'm also not having any problem running a manual policy sync.

[2023-03-13 00:13:59.710 -04:00] [SaaS Connector] [InternalError] 'Failed to add /cfg/com.scur.mainaction.block.11364: 401'
[2023-03-13 00:13:59.751 -04:00] [SaaS Connector] [InternalError] 'Failed to add /cfg/com.scur.engine.streamdetector.16393: 401'
[2023-03-13 00:13:59.827 -04:00] [SaaS Connector] [InternalError] 'Failed to add /cfg/com.scur.engine.volquota.506: 401'
[2023-03-13 03:32:48.985 -04:00] [SaaS Connector] [InternalError] 'Failed to add /cfg/com.scur.mainaction.block.11364: 401'
[2023-03-13 03:32:48.988 -04:00] [SaaS Connector] [InternalError] 'Failed to add /cfg/com.scur.mainaction.block.11376: 401'
[2023-03-13 04:18:30.762 -04:00] [SaaS Connector] [InternalError] 'Failed to add /cfg/com.scur.engine.compositeopener.1: 401'
[2023-03-13 04:18:30.764 -04:00] [SaaS Connector] [InternalError] 'Failed to add /cfg/com.scur.engine.billing.4575: 401'
[2023-03-13 04:18:30.768 -04:00] [SaaS Connector] [InternalError] 'Failed to add /cfg/com.scur.engine.antivirus.4589: 401'
[2023-03-13 04:18:30.775 -04:00] [SaaS Connector] [InternalError] 'Failed to add /cfg/com.scur.engine.safesearchenforcer.11498: 401'
[2023-03-13 04:18:30.943 -04:00] [SaaS Connector] [InternalError] 'Failed to add /cfg/com.scur.engine.sslclientcontext.4585: 401'
[2023-03-13 07:37:15.065 -04:00] [SaaS Connector] [InternalError] 'Failed to add /cfg/com.scur.mainaction.block.11376: 401'
[2023-03-13 07:37:15.095 -04:00] [SaaS Connector] [InternalError] 'Failed to add /cfg/com.scur.mainaction.block.4572: 401'
[2023-03-13 07:37:15.155 -04:00] [SaaS Connector] [InternalError] 'Failed to add /cfg/com.scur.mainaction.block.11377: 401'
[2023-03-13 07:37:15.156 -04:00] [SaaS Connector] [InternalError] 'Failed to add /cfg/com.scur.mainaction.block.15139: 401'
[2023-03-13 07:37:15.236 -04:00] [SaaS Connector] [InternalError] 'Failed to add /cfg/com.scur.mainaction.block.4571: 401'
[2023-03-13 08:07:41.017 -04:00] [SaaS Connector] [InternalError] 'Failed to add /cfg/com.scur.engine.volquota.506: 401'
[2023-03-13 08:07:41.168 -04:00] [SaaS Connector] [InternalError] 'Failed to add /cfg/com.scur.mainaction.block.11376: 401'
[2023-03-13 08:17:51.647 -04:00] [SaaS Connector] [InternalError] 'errno: 104 - 'Connection reset by peer' on socketfd -1 Failed to add /cfg/com.scur.engine.streamdetector.16393: could not establish connection'
[2023-03-13 08:17:51.732 -04:00] [SaaS Connector] [InternalError] 'errno: 104 - 'Connection reset by peer' on socketfd -1 Failed to add /cfg/com.scur.mainaction.block.11377: could not establish connection'
[2023-03-13 08:17:51.874 -04:00] [SaaS Connector] [InternalError] 'errno: 104 - 'Connection reset by peer' on socketfd -1 Failed to add /cfg/com.scur.mainaction.block.4574: could not establish connection'
[2023-03-13 09:03:51.743 -04:00] [SaaS Connector] [InternalError] 'Failed to add /rules/gwrs: 401'
[2023-03-13 09:03:51.744 -04:00] [SaaS Connector] [InternalError] 'Failed to add /cfg/com.scur.mainaction.block.11377: 401'
[2023-03-13 09:03:51.744 -04:00] [SaaS Connector] [InternalError] 'Failed to add /cfg/com.scur.engine.volquota.506: 401'
[2023-03-13 10:50:49.180 -04:00] [SaaS Connector] [InternalError] 'Failed to add /cfg/com.scur.engine.streamdetector.16393: 401'
[2023-03-13 10:50:49.196 -04:00] [SaaS Connector] [InternalError] 'Failed to add /cfg/com.scur.mainaction.block.11370: 401'
[2023-03-13 10:50:49.229 -04:00] [SaaS Connector] [InternalError] 'Failed to add /cfg/com.scur.engine.volquota.506: 401'
[2023-03-13 10:50:49.367 -04:00] [SaaS Connector] [InternalError] 'Failed to add /rules/gwrs: 401'
[2023-03-13 11:36:45.561 -04:00] [SaaS Connector] [InternalError] 'Failed to connect to host: policysync.skyhigh.cloud. errno: 107 - 'Transport endpoint is not connected' on socketfd 9 Failed to query SaaS version: could not establish connection'
[2023-03-13 11:52:01.547 -04:00] [SaaS Connector] [InternalError] 'Failed to connect to host: policysync.skyhigh.cloud. errno: 107 - 'Transport endpoint is not connected' on socketfd 15 Failed to add /cfg/com.scur.engine.enablesslscanner.4573: could not establish connection'
[2023-03-13 11:52:01.547 -04:00] [SaaS Connector] [InternalError] 'Failed to connect to host: policysync.skyhigh.cloud. errno: 107 - 'Transport endpoint is not connected' on socketfd 12 Failed to add /cfg/com.scur.engine.antivirus.4589: could not establish connection'
[2023-03-13 11:52:01.552 -04:00] [SaaS Connector] [InternalError] 'Failed to connect to host: policysync.skyhigh.cloud. errno: 107 - 'Transport endpoint is not connected' on socketfd 18 Failed to add /cfg/com.scur.engine.safesearchenforcer.11498: could not establish connection'
[2023-03-13 11:52:13.051 -04:00] [SaaS Connector] [InternalError] 'Failed to connect to host: policysync.skyhigh.cloud. errno: 107 - 'Transport endpoint is not connected' on socketfd 12 Failed to add /cfg/com.scur.engine.antivirus.4589: could not establish connection'
[2023-03-13 11:52:13.062 -04:00] [SaaS Connector] [InternalError] 'Failed to connect to host: policysync.skyhigh.cloud. errno: 107 - 'Transport endpoint is not connected' on socketfd 18 Failed to add /cfg/com.scur.engine.safesearchenforcer.11498: could not establish connection'
[2023-03-13 12:07:40.418 -04:00] [SaaS Connector] [InternalError] 'Failed to connect to host: policysync.skyhigh.cloud. errno: 107 - 'Transport endpoint is not connected' on socketfd 8 Failed to query registered connector: could not establish connection'
[2023-03-13 12:07:50.978 -04:00] [SaaS Connector] [InternalError] 'Failed to connect to host: policysync.skyhigh.cloud. errno: 107 - 'Transport endpoint is not connected' on socketfd 8 Failed to create temporary configuration: could not establish connection'
[2023-03-13 12:08:02.885 -04:00] [SaaS Connector] [InternalError] 'Failed to connect to host: policysync.skyhigh.cloud. errno: 107 - 'Transport endpoint is not connected' on socketfd 16 Failed to add /cfg/com.scur.engine.enablesslscanner.4587: could not establish connection'
[2023-03-13 12:08:02.885 -04:00] [SaaS Connector] [InternalError] 'Failed to connect to host: policysync.skyhigh.cloud. errno: 107 - 'Transport endpoint is not connected' on socketfd 13 Failed to add /cfg/com.scur.engine.compositeopener.1: could not establish connection'
[2023-03-13 12:08:02.893 -04:00] [SaaS Connector] [InternalError] 'Failed to connect to host: policysync.skyhigh.cloud. errno: 107 - 'Transport endpoint is not connected' on socketfd 14 Failed to add /cfg/com.scur.engine.antivirus.4589: could not establish connection'
[2023-03-13 12:08:02.895 -04:00] [SaaS Connector] [InternalError] 'Failed to connect to host: policysync.skyhigh.cloud. errno: 107 - 'Transport endpoint is not connected' on socketfd 11 Failed to add /rules/gwrs: could not establish connection'
[2023-03-13 12:08:02.895 -04:00] [SaaS Connector] [InternalError] 'Failed to connect to host: policysync.skyhigh.cloud. errno: 107 - 'Transport endpoint is not connected' on socketfd 15 Failed to add /cfg/com.scur.engine.safesearchenforcer.11498: could not establish connection'
[2023-03-13 12:08:14.404 -04:00] [SaaS Connector] [InternalError] 'Failed to connect to host: policysync.skyhigh.cloud. errno: 107 - 'Transport endpoint is not connected' on socketfd 14 Failed to add /cfg/com.scur.engine.antivirus.4589: could not establish connection'
[2023-03-13 12:08:14.406 -04:00] [SaaS Connector] [InternalError] 'Failed to connect to host: policysync.skyhigh.cloud. errno: 107 - 'Transport endpoint is not connected' on socketfd 15 Failed to add /cfg/com.scur.engine.safesearchenforcer.11498: could not establish connection'
[2023-03-13 15:21:51.044 -04:00] [SaaS Connector] [InternalError] 'Failed to add /cfg/com.scur.mainaction.block.11376: 401'  

[fw_mon]

Hello @nashcoop 

have you found a solution for your issue? Or was able Support help you? The error message is clear, the WG cannot connect to policysync.skyhigh.cloud

It could be a temporarely network issue on the skyhigh side or some network misconfiguration on your side.

Let me know, I can provide troubleshooting steps.

[nashcoop]

I haven't received any feedback from support.  Ticket was opened yesterday 4-23752545761.  The problem began roughly one week ago, so it doesn't appear to be temporary.  I don't have any problem running a manual sync to policysync.skyhigh.cloud .  I'm not sure where the source of these sync errors is coming from.  I'm unaware of any auto sync configured to reach out to that address. The error is occurring anywhere between 4 and 20 times per day.

[fw_mon]

first please check if your location is not affected by the current service degradation: status.skyhighsecurity.com/swg/

how you describing it, the issue is not temporary, it's better to proceed with support.

[kelcie_1]

Was there any resolution to this? I have same error.

[nashcoop]

Looking back at the SR info the resolution involved engineers changing something on the back end. I would recommend opening a support ticket. I added some of the comments below from the engineer who handled the ticket

• "Engineering has identified the root cause for this issue and is actively working to resolve/mitigate for you. More updates to follow as I receive them."
• "Engineering has informed me that they have implemented a workaround that should've significantly mitigated this issue, and that they are actively working to fully resolve. You should see far fewer sync's failing, please let me know what you're seeing now on your end!"
  
  
• "As far as details go, I will have to follow up with engineering. The updates we have received from them indicated that it was related to the Okta rate limit for admin authentication, but that is all I have at this time. Allow me some time and I'll see what else I can learn for you."