Increase Query Latency due to Abnormally High Load

Incident Report for Explo Status

Postmortem

Upon investigating our query runner, we found issues with high memory usage under extreme load that exponentially backed up our query queue. We have resolved issues here, and have additionally increased resources to handle higher load on our query runners.

Posted Mar 01, 2024 - 21:48 UTC

Resolved

Abnormally high load on our dashboards caused increased query latency for all customers. External effect was dashboards timing out or otherwise taking longer than expected to load data

Posted Mar 01, 2024 - 20:30 UTC