SIP INVITE requests unacknowledged in the EU

Incident Report for LiveKit

Postmortem

Typically, SIP INVITE messages fit within the standard Ethernet MTU. In cases where a large number of headers or metadata are included, an INVITE can exceed the MTU and be fragmented. While such large messages are ideally sent over TCP, some providers may still send them over UDP.

On 2026-01-08 06:11 UTC, we deployed updated firewall rules in our UK and EU regions as part of our regular security review. During this update, a configuration that allows fragmented packets was unintentionally omitted. As a result, larger SIP INVITE messages (inbound calls) sent over UDP were dropped.

This issue affected a limited subset of customers in the UK and EU regions, accounting for approximately 1.5% of total inbound calls in the affection regions during the highest point of traffic for the day.

The issue was identified and the firewall rules were rolled back at 2026-01-08 20:17 UTC.

We already maintain end-to-end tests that cover large, fragmented packets. However, these tests were previously run only in a subset of regions. To prevent similar incidents in the future, we are expanding these tests to run in every cluster and to execute immediately after firewall and security configuration updates are applied.

Posted Jan 30, 2026 - 16:20 PST

Resolved

This incident has been resolved. We will share a postmortem shortly.
Posted Jan 28, 2026 - 13:45 PST

Monitoring

A fix has been deployed and we are monitoring the results.
Posted Jan 28, 2026 - 12:22 PST

Identified

We have identified the source of the issue and are deploying a fix.
Posted Jan 28, 2026 - 12:09 PST

Update

This issue is affecting customers sending UDP SIP traffic that exceeds the standard SIP MTU size. Switching to TCP is an immediate way to mitigate the problem.
Posted Jan 28, 2026 - 11:46 PST

Investigating

We are continuing to investigate this issue.
Posted Jan 28, 2026 - 11:32 PST

Monitoring

We identified an issue in the UK related to inbound calls with large UDP INVITE messages that exceed the MTU. This affected 0.04% of inbound calls in the region. We have updated a configuration for handling fragmented packets in that cluster, and the issue was mitigated as of 16:10 UTC.

The impact was limited to the UK; this issue was not observed in other clusters.
Posted Jan 28, 2026 - 09:50 PST

Investigating

We are investigating reports of SIP INVITE requests not receiving responses in some EU regions. The impact appears to be very low. We will provide further details as they become available.
Posted Jan 28, 2026 - 08:45 PST
This incident affected: Regional SIP (Europe Central - SIP, Europe South - SIP, United Kingdom - SIP).