Incorrect timestamps returned from warehouse for some customers
Incident
Report for Sigma Computing
Postmortem
Sigma Engineering has completed the postmortem of this incident. A Root Cause Analysis (RCA) is available on the Sigma Community site: Postmortem for Aug 10th, 2023: Incorrect results for timezone-based calculations
Resolved
Backdated Incident: Some customers observed that timestamps in data returned from the warehouse were incorrect. Specifically the timestamp was off by the UTC offset (hours the customer timezone is ahead or behind UTC). Customers may have also observed that date_trunc() would return incorrect values. This was due to a change made during a code refactoring; no behavioral difference was anticipated, but certain customer configurations experienced the described discrepancy.
Impacted Cloud Regions & Services: Snowflake and DataBricks customers with a particular combination of timezone configuration in Sigma and their warehouse were impacted by this issue.
Note the refactored code path was enabled for some customers on August 7, and then progressively enabled for more customers and warehouses over several days; notably:
- August 8, 17:30 UTC, code path enabled for all Databricks customers
- August 10, 15:58 UTC, code path enabled for all Snowflake customers
- August 10, 18:49 UTC, code path disabled for all Snowflake customers
- August 14, 15:40 UTC, code path disabled for all customers
Incident Start Time: 2023-08-07 18:10 UTC
Incident End Time: 2023-08-14 15:40 UTC
We will perform a root cause analysis (RCA) and publish our findings in a postmortem note to the Sigma Community site in coming days.
Impacted Cloud Regions & Services: Snowflake and DataBricks customers with a particular combination of timezone configuration in Sigma and their warehouse were impacted by this issue.
Note the refactored code path was enabled for some customers on August 7, and then progressively enabled for more customers and warehouses over several days; notably:
- August 8, 17:30 UTC, code path enabled for all Databricks customers
- August 10, 15:58 UTC, code path enabled for all Snowflake customers
- August 10, 18:49 UTC, code path disabled for all Snowflake customers
- August 14, 15:40 UTC, code path disabled for all customers
Incident Start Time: 2023-08-07 18:10 UTC
Incident End Time: 2023-08-14 15:40 UTC
We will perform a root cause analysis (RCA) and publish our findings in a postmortem note to the Sigma Community site in coming days.