On October 11, 2024, at 18:30 UTC, Groove applications encountered intermittent problems due to a database deadlock. This situation arose when multiple transactions attempted to access the same resource, causing them to become stuck. Consequently, the system stalled, and incoming requests were blocked as they awaited the completion of the deadlocked transactions.
To address this, we implemented a timeout for database operations, which terminated blocked transactions after a specified duration. The issue was completely resolved by 20:03 UTC that same day. There was no risk or compromise to user data during this incident.