We have implemented a fix and confirmed that run failures have returned to their historic norms as of 4PM Eastern / 9PM UTC on Friday, January 13. If your runs are experiencing unusual wait times, are stalling on startup, or are exiting with an infrastructure-related error message please contact Support via chat or email support@getdbt.com with the affected Run ID(s) for further investigation.
Posted Jan 14, 2023 - 13:06 EST
Monitoring
We have identified additional root causes for the cancellations with infrastructure-related errors, most of which are occurring around 7PM and 7AM Eastern (Midnight and Noon UTC), and have implemented fixes that we believe will mitigate the issue.
Our team is actively monitoring for additional runs failing in this way. If you are unsure whether your run failure with an infrastructure-related error message is in scope for this incident, please start a support chat or email support@getdbt.com.
Posted Jan 12, 2023 - 17:00 EST
This incident affected: North America (N. Virginia) (Scheduled Jobs).