Resolved -
All historical tasks have now been replayed and completed, so this incident will be closed.
For any customers that are still experiencing issues they feel may be related to this incident, please reach out to Atlassian Support for additional assistance.
Dec 5, 02:20 UTC
Monitoring -
Data syncing within the impacted features should now be fully operational.
Some final historical tasks are still replaying, and we will update again once these tasks have been completed and the issue is fully resolved.
Dec 3, 01:08 UTC
Update -
Impact
Customers using Jira Service Management and the Talent product experienced interruptions due to missing or delayed data updates. This issue impacted the availability of new hire employee data, preventing the creation of onboarding plans. The disruption persisted for more than 18 hours, affecting all Talent customers.
Current Status
Normal data processing remains restored, following initial hotfix implementation. Reprocessing the backlog of events continues at an accelerated pace. The Asia-Pacific Southeast (APSE) region has been reprocessed as of 1240 UTC and is complete. The EU Central region has also finished as of 1740 UTC, with no resync required. The last remaining affected region is US East, which is expected to reprocess within the next 3-4 hours.
Next Steps
No further action is required at this time for the APSE and EU Central regions. Atlassian Engineering is actively monitoring reprocessing job status on US East. The next update will occur at 2300 UTC.
Dec 2, 20:04 UTC
Update -
We released a fix, and internal monitoring shows services now process current events as expected.
Reprocessing the historical backlog of events from the incident start is ongoing. We estimate it will take about 7.5 hours for the APSE region and 10 hours for the EU Central region. For the PROD-EAST region, we do not yet have an ETA since reprocessing has not started, but the team estimates it will be brief due to fewer events.
We will provide an update within 10 hours or when reprocessing completes, whichever is sooner.
Dec 2, 09:57 UTC
Update -
A fix has now been released for this issue, and our internal monitoring is showing that services are now processing events as expected.
A historical backlog of events exists from the start of the incident, and we are now processing those events.
We will provide further update within 4 hours, or when processing is completed, whichever is sooner.
Dec 2, 06:09 UTC
Update -
Our team has now identified a potential root cause for this incident and is actively working on a hot-fix release to resolve the issue.
We will provide further update when the expected fix is available, or within 4 hours, whichever is sooner.
Dec 2, 01:59 UTC
Update -
Our team continues to actively investigate the cause of this issue.
We are now deploying additional monitoring to help us understand the root cause of this issue in more detail and will provide further update within 4 hours or as soon as a resolution becomes available.
Dec 2, 00:17 UTC
Update -
Impact
Customers using Jira Service Management and Talent have encountered an issue where New Hire Employee data is unavailable in Talent. This situation blocks the creation of onboarding plans for new hires. At present, all Talent customers are affected by this synchronization issue, which has persisted for more than 18 hours.
Current Status and Next Steps
The engineering team continues to work on the fix. Once we have more details to share about their findings and fixes, a new Update will be posted in a least 2 hours from now.
Dec 1, 22:16 UTC
Identified -
We are investigating reports of intermittent errors for ALL Talent Cloud customers. The Data Ingestion item is failing, causing sync issues.
We will provide another update within the next 2 hours.
Dec 1, 20:24 UTC
