Was the Jira Outage the Last Atlassian Problem?
Last Updated on May 8, 2023
A month ago it was impossible to imagine that hundreds of customers could lose access to such apps as Jira. What is more, it was absolutely unacceptable to think that the outage could last for almost three weeks. However, as the practice shows, everything can happen. So, let’s look in detail at the problems which customers had to face when Atlassian was overcoming its greatest Jira outage.
April’s Jira outage – what was going on?
On the first day, when the Jira outage happened nobody knew for sure how long it could last and the Atlassian engineers had been releasing the same message on their Jira Software Status “We continue to work on the defined processes to the resolution of the issues impacting some customers” with no explanations for almost eight days. On the ninth day, Atlassian finally sent emails to their customers that they weren’t able to restore the services at that moment but they were working and they needed up to 2 more weeks to solve the problem. Let’s repeat – 2 MORE WEEKS – how this will impact your team?
The impact on customers. Was it disastrous?
But what about Jira customers? Did they suffer from the Atlassian outage? Unfortunately, yeap. About 400 customers lost their connection during this Jira outage while Atlassian engineers were trying to fix the problem. Companies had to put up with the consequences as many of them didn’t back up their sites. Some of them were trying to use the Service Management tool for their helpdesk, which meant that their helpdesk wasn’t working during this Jira outage. Moreover, some companies had nothing to do but delay, reschedule, and re-plan their projects, which, of course, brought them a lot of financial inconveniences and a reputation loss.
Jira down? Get your team back to tasks within minutes with the first professional backup for Jira Cloud, Jira Service Management, and Jira Work Management.
What was the problem which lead to the longest Jira outage?
But let’s get back to the problem of this Atlassian outage that lasted for almost two weeks. As it turned out, according to the Atlassian engineer, Sri Viswanath’s explanation-post, the Jira outage happened because of a human mistake. Well, we are all people and such a thing can happen, so it’s always better to keep in mind a backup plan. While the engineers were updating the app with the code, something went wrong. As it turned out due to the lack of communication between the teams, which were in charge of the process ”The script was executed with the wrong execution mode and the wrong list of IDs. The result was that sites for approximately 400 customers were improperly deleted” he explained.
As it was hard to restore all the sites automatically in a bulk, the engineers had to make a number of steps which, obviously, took time. The restoration process was semi-automated and they needed to restore some customer data manually for each site. Once again, restore websites one by one manually!
Are there any other threats?
After such a massive Atlassian outage customers probably started to think if there is any other threat to their websites. So, recently, according to the information taken from Bleeping Computer, Atlassian has informed their Jira and Jira Service Management customers that a critical authentication bypass vulnerability in Seraph, which is a security framework for Atlassian web application, has affected their products. It means that there is no proper protection for customers’ login and logout requests and an attacker can remotely send a hack HTTP request to vulnerable computers and bypass authentication. That’s why Atlassian advises their customers to update their products to the versions that include this security protection.
Can situations like that be prevented?
Definitely, yes! The answer to this question is backup, as it is always the best solution to eliminate the negative effects of outages, security flaws, human errors, or cyberattacks. In the situation with the Jira outage, if users had a backup plan, they could continue their work almost without noticing the problem. All they needed to do is to restore their entire Jira environment to the new account, which would take them only a few minutes, and continue their work peacefully.
In the case of an authentication vulnerability, backup is essential as well. Imagine the situation when an attacker with full access to your account wipes out all the Jira data – probably hours of work (and money) with just one click. While it sounds scary, now we see it’s possible. With GitProtect.io on the place and your data backed up on secure external storage (local, any S3 compatible cloud, or both) you can restore your data to another Jira Cloud account and get your projects back from any point-in-time.
Just like with traditional backup, here at GitProtect.io we say there are two types of people – those who do Jira backup and those who will start doing Jira backup. Which one are you?
