We have implemented a fix and confirmed that run failures have returned to their historic norms as of 4PM Eastern. If runs after this time continue to fail with an infrastructure related error message, please contact support@getdbt.com.
Posted Jan 11, 2023 - 17:41 EST
Update
We have identified that most runs launched from 10:50AM Eastern until approximately 11:26AM Eastern in the US Multitenant instance of dbt Cloud were stuck in a starting state and will be canceled 30 minutes after the scheduled start time. This issue has been fixed and most runs are proceeding as expected.
We are continuing to investigate spikes in runs failing due to an infrastructure issue at the top of each hour.
Posted Jan 11, 2023 - 11:43 EST
Update
We are aware of a problem with all new runs stalling in the startup phase but not progressing. We are actively and urgently investigating the root cause and will provide an update as soon as we have additional information.
Posted Jan 11, 2023 - 11:10 EST
Investigating
We are aware of a problem with some runs failing with an error message related to an underlying infrastructure issue. A re-run of the job should clear the error. We are actively investigating the root cause.
Posted Jan 11, 2023 - 10:51 EST
This incident affected: North America (N. Virginia) (Scheduled Jobs).