GitLab Backup to S3
Last Updated on March 5, 2024
I regularly browse through various job offers, even when I am not looking for any new challenges. I do it to see what is going on in the market, what companies from various industries are doing, and what technologies are currently hot. What has been striking in many offers for a long time is “migration to cloud”. Well, keeping data or services in the cloud is nothing new, but it is still not quite common and many companies are still going through such migration or are planning to do so.
One may ask “but what about this?” Well, it opens up a lot of opportunities for us, but at the same time, it creates new dangers. It is also an interesting topic from the backup and maintenance perspective. Such copies should be available anytime and anywhere, so the cloud should be a thing that grabs our attention even more.
What’s backup, precious?
First, let’s clarify what backup actually is. In a few words, it’s not just a copy of data taken and stored elsewhere so that it may be used to restore the original data. It should have some “special” features to be called a proper backup:
- automation and scheduler
- encryption
- versioning
- data retention
- recovery process
- scalability
Our backup solution must meet these requirements so that we can feel safe. Our procedures should take into account all these aspects and should always be up-to-date. Unfortunately, it can be a problem for many companies. If we decide on our own solution, there can arise some problems that we didn’t expect at the beginning. We could build a dedicated team to operate, create and manage our backups and their possible restoration, but this is an expensive approach in the long term perspective. Another solution is to use third-party backup tools. Though, we need to know the pros and cons of both solutions and make the right decisions to ensure an appropriate level of security.
Are you switching to a DevSecOps operation model? Remember to secure your code with the first professional GitHub, Bitbucket, GitLab, and Jira backup.
However, already-made solutions have a crucial advantage. They are made by experts in this area. Such tools are created to deliver the highest level of functionality and security. If our business is not related to IT security itself then I suggest we share this responsibility with someone who knows this topic better than we do.
Check our comparison of self-made git backup script vs. repository backup software
AWS S3
S3, or Simple Storage Service, is one of the most popular services from Amazon. It offers industry-leading scalability, data availability, security, and performance. The method of operation and payment for the service are important – we pay only as much as we use. And we don’t have to worry about scalability here – this is one of the hallmarks of Amazon services. We can expand the scale of our operation at any time and we will not even notice it. Thanks to this approach, it is a good solution for both large corporations and small startups.
Other storage types
AWS S3 is a good choice, this is also the topic of this article, but it is worth adding that this is not the only storage type of course. Our backup should be stored somewhere and we need to consider some financial and licensing matters. For example, let’s imagine that due to some law regulations our specific data cannot be stored in a particular country. It is possible to face such an issue so we need to be aware of where our data is.
In GitProtect we can choose between two default regions: Amsterdam, the Netherlands, or Virginia, the United States. We can also use other storage types like cloud-based external services or our own hosting. It is a good idea to use the aforementioned AWS service to create a secure GitLab S3 backup.
Creating a backup plan in GitProtect
It is an incredibly simple process. GitProtect is a user-friendly tool with a modern GUI so we need only two steps to create a GitLab S3 backup plan. First, of course, we need to link our GitLab account and provide proper credentials. After that, we can select which repositories we want to add to our backup plan in GitProtect..
The next step is to select which data we want to protect. Our projects contain not only a code repository but also most metadata, like wikis, issues, webhooks, merge requests, labels, pipelines, and so on. You can even backup LFS. Let’s decide what we want to keep for a particular plan.
After that, we need to choose a storage type and the schedule. It shouldn’t take more than a few minutes. GitProtect is a multi-storage system and therefore we can add as many storage instances as we want. We can store our data locally or in the cloud via many different services, even GitProtect Cloud and easily meet the backup 3-2-1 rule. GitProtect Cloud Storage is included in every plan so if you want to bring your own S3 cloud, you can use it for backup replication purposes and increase the security level of your data.
Our goal was to create a GitLab backup to S3 so when creating a new plan, we need to choose where to store it. It looks like this:
As you can see in the picture below, to add the S3 storage you need to authenticate, provide the service URL and bucket name. That’s it – your S3 cloud is added and you can use it to store your automatic backups there.
Backup as a CI/CD process
Last but not least. DevSecOps approach is a current trend in IT. Everyone has (or would like to have) a Continuous Integration process implemented in their organization. After all, automation is the key. GitProtect meets this goal. Through an API we have the possibility to customize backup to match our company’s workflow. Thanks to that we can be sure that every change in the code is reliably protected and easily recoverable from any point in time and roll back to previous versions in the event of failure or human mistakes. Thanks to the unlimited retention you can restore your copies even from more than 365 days back. As you can see, you can use it for archive purposes. Moreover, you easily meet the shared responsibility model and compliance requirements.
Conclusion
Cloud services are the current standard. With that, backup as an integral part of our business operations should also go to the cloud. And the GitProtect tool makes it possible for us. We can very easily define the proper plan and storage type to create a GitLab backup to S3 for all our repositories and related metadata.
