LiveKit Cloud Dashboard Down

Root Cause
A gap in our DNS update process resulted in a misconfiguration of the DNS for livekit.io, causing several A records to be overwritten. This resulted in an incorrect IP address being returned for livekit.io. All real-time services - calls, RTC, and hosted agents - were unaffected as they operate on separate domains.
We do have DNS monitoring in place, but it was not configured to page on-call. As a result, the issue was identified by LiveKit engineering after a short delay.
Timeline

• 2026-04-03 17:46 UTC: DNS configuration change applied; resolution begins failing for livekit.io.
• 2026-04-03 17:52 UTC: Issue identified by LiveKit.
• 2026-04-03 18:15 UTC: Root cause identified and fix applied.
• 2026-04-03 18:18 UTC: Fix fully propagated; services restored. Total duration: ~32 minutes.

Mitigations

• Enforce tighter automated guardrails around DNS updates.
• Enable paging on existing DNS health checks so any future issues are caught even sooner.