Materialization & Scheduled Exports Degraded Performance
Incident
Report for Sigma Computing
Postmortem
Sigma Engineering has completed the postmortem of this incident. A Root Cause Analysis (RCA) is available on the Sigma Community site: Postmortem For April 20, 2023 Materialization and Scheduled Export Incident - Sigma AWS & Sigma GCP
Resolved
Current Status: We have implemented mitigation measures and are monitoring their effect closely. The occurrence of errors has declined substantially. Apologies for any inconvenience caused.
Impacted Cloud Regions & Services: AWS & GCP
User Experience: We expect that the rate of failures for materializations and scheduled exports to be substantially decreased. Users should not need to manually sync their databases/schemas/tables anymore at this time, changes done in the warehouse should automatically show in Sigma again. Navigating newly-created connections should also be functional.
Incident Start Time: Approximately 11:50 UTC April 20, 2023
Incident End Time: Approximately 12:30am UTC April 21, 2023
We will perform a root cause analysis (RCA) and publish our findings in a postmortem note to the Sigma Community site in coming days. You will receive a notification when that occurs.
Impacted Cloud Regions & Services: AWS & GCP
User Experience: We expect that the rate of failures for materializations and scheduled exports to be substantially decreased. Users should not need to manually sync their databases/schemas/tables anymore at this time, changes done in the warehouse should automatically show in Sigma again. Navigating newly-created connections should also be functional.
Incident Start Time: Approximately 11:50 UTC April 20, 2023
Incident End Time: Approximately 12:30am UTC April 21, 2023
We will perform a root cause analysis (RCA) and publish our findings in a postmortem note to the Sigma Community site in coming days. You will receive a notification when that occurs.
Update
Current Status: We have implemented mitigation measures and are monitoring their effect closely. The occurrence of errors has declined substantially. As a consequence of this mitigation, users may need to manually sync their databases/schemas/tables if changes done in the warehouse are not showing in Sigma. Additionally, navigating newly-created connections may be slow and their search functionality will not work. Apologies for any inconvenience caused.
Impacted Cloud Regions & Services: AWS & GCP
User Experience: Users would have experienced sporadic failures for materializations and scheduled exports. Users may need to manually sync their databases/schemas/tables if changes done in the warehouse are not showing in Sigma. Additionally, navigating newly-created connections may be slow and their search functionality will not work.
Incident Start Time: Approximately 11:50am UTC April 20, 2023
Impacted Cloud Regions & Services: AWS & GCP
User Experience: Users would have experienced sporadic failures for materializations and scheduled exports. Users may need to manually sync their databases/schemas/tables if changes done in the warehouse are not showing in Sigma. Additionally, navigating newly-created connections may be slow and their search functionality will not work.
Incident Start Time: Approximately 11:50am UTC April 20, 2023
Monitoring
Current Status: We have implemented mitigation measures and are monitoring their effect closely. The occurrence of errors has declined substantially.
Impacted Cloud Regions & Services: AWS & GCP
User Experience: Users would have experienced sporadic failures for materializations and scheduled exports.
Incident Start Time: Approximately 11:50am UTC April 20, 2023
Impacted Cloud Regions & Services: AWS & GCP
User Experience: Users would have experienced sporadic failures for materializations and scheduled exports.
Incident Start Time: Approximately 11:50am UTC April 20, 2023
Investigating
Current Status: We are investigating technical issues leading to degraded performance for both materializations and scheduled exports. This is currently our highest priority. Apologies for any inconvenience caused.
Impacted Cloud Regions & Services: AWS & GCP
User Experience: Users may experience sporadic failures for materializations and scheduled exports now and in the past
Incident Start Time: Approximately 11:50am UTC April 20, 2023
Impacted Cloud Regions & Services: AWS & GCP
User Experience: Users may experience sporadic failures for materializations and scheduled exports now and in the past
Incident Start Time: Approximately 11:50am UTC April 20, 2023
This incident affected: GCP (Sigma - GCP (US)) and AWS (Sigma - AWS (US)).