Subset of inbound calls not being handled

Root Cause: The issue was caused by an incorrectly formed header in BYE requests, which triggered an infinite forwarding loop of SIP messages on our SIP load balancing service. This resulted in the server repeatedly forwarding messages to itself, causing resource exhaustion on the affected machines. During these periods, processing of incoming INVITE requests were delayed on affected servers.

Time window of impact:
We saw the first instances of these delays happen at 2025-12-28 19:00:00 UTC. While we pushed the first patches to mitigate this issue to majorly impacted regions on 2026-01-01 04:00:00 UTC, the fix was rolled out globally on 2026-01-02 19:00:00 UTC.

Scope of impact: The incident had a much lower impact vs the 5% assessment we made during the incident based on limited data. The incident is now categorized as degraded service as opposed to a partial outage. In terms of actual impact to calls, out of the small percentage of calls that rang longer (stats below), a smaller subset of them could have timed out depending on what the timeout settings of the inbound calling trunk were.

Regions affected: Eastern US

Daily impact (percentage of calls that rang longer than 2s - all times in UTC):

1. 2025-12-28: 0.076%
2. 2025-12-29: 0.061%
3. 2025-12-30: 0.103%
4. 2025-12-31: 0.0046%

Worst impacted hourly windows (percentage of calls that rang longer than 2s - all times in UTC):

1. 2025-12-28 19:00 + 1h: 1.01%
2. 2025-12-29 18:00 + 1h: 0.32%
3. 2025-12-30 16:00 + 1h: 0.37%
4. 2025-12-31 19:00 + 1h: 0.38%

Mitigations:

• We already applied a quick fix to mitigate the issue over the past few days.
• We will be rolling out code changes to handle this edge case more comprehensively by mid next week.
• More aggressive monitoring to detect delays in “SIP 100 Trying” responses is being worked on.