{"id":3013,"date":"2022-06-02T07:39:10","date_gmt":"2022-06-02T07:39:10","guid":{"rendered":"https:\/\/gitprotect.io\/blog\/?p=3013"},"modified":"2024-11-20T15:02:05","modified_gmt":"2024-11-20T15:02:05","slug":"github-backup-to-s3","status":"publish","type":"post","link":"https:\/\/gitprotect.io\/blog\/github-backup-to-s3\/","title":{"rendered":"GitHub Backup to S3"},"content":{"rendered":"\n

Codebase expansion and the growing complexity of IT environments are good reasons to keep your GitHub backup in S3-compatible storage. Especially if you plan to boost data security while scaling<\/strong>
your storage horizontally with no physical upgrades needed.<\/strong><\/p>\n\n\n\n\n\n\n\n

From a business perspective, that is a step in the right direction. S3 storage efficiently streams disaster recovery strategies aligned with the 3-2-1<\/strong>, 3-2-1-1-0<\/strong>, or 4-3-2<\/strong> backup rules. It translates directly into downtime and data loss risk reduction.<\/p>\n\n\n\n\n\n

S3 to backup GitHub repos<\/strong><\/h2>\n\n\n\n

Cost efficiency is a key advantage of S3-compatible service. Its pay-as-you-go models allow GitHub repos to be scaled as they grow. S3 systems offer tiered plans for frequently accessed data to remain in faster tiers. Older backups are moved to cheaper, cold storage.<\/p>\n\n\n\n

GitHub backup to S3 allows users to take advantage of versioning, encoding services, and cross-region replication. These features enhance protection against accidental deletion or corruption. <\/p>\n\n\n\n

At the same time, access policies (e.g., IAM roles and IAM users) enable fine-grained control over who can view, modify, or delete the stored data. This supports companies in their part of the Shared Responsibility Model (SRM)<\/a>.<\/p>\n\n\n

\n
\"Shared<\/figure><\/div>\n\n\n

AWS S3, Azure Blob, or Google Cloud Storage for your Git repositories?<\/strong><\/h2>\n\n\n\n

When choosing the right S3 provider, matching their offering to the company’s data protection policies is essential.<\/p>\n\n\n\n

Although it may seem trivial, such an approach, followed by optimal cost management, allows us to back up all our repositories efficiently and without losing data. So, what are the options?<\/p>\n\n\n\n

AWS S3<\/strong><\/h3>\n\n\n\n

Amazon’s service is recognized for scalability and security. Features also include versioning, encryption, and cross-region replication. AWS S3 integrates seamlessly with GitHub Actions for automated backups. It makes the service perfect for enterprises needing reliable cloud storage for every repo they develop.<\/p>\n\n\n\n

Google Cloud Storage (S3-like)<\/strong><\/h3>\n\n\n\n

Supporting multi-region redundancy and automatic class transitions, Google Cloud Storage may be one of the best choices for GitHub backups. It works well with Python scripts. GCS features include strong encryption and analytics for backup monitoring.<\/p>\n\n\n\n

Azure Blob Storage<\/strong><\/h3>\n\n\n\n

Azure Blob is the choice for users mainly operating in the Microsoft ecosystem. Azure provides immutable storage and automated tiering for better cost optimization. The solution is highly protected. It supports compliance needs, making it helpful in backing up GitHub information.<\/p>\n\n\n\n

Backblaze B2<\/strong><\/h3>\n\n\n\n

B2 is an affordable S3-compatible storage solution offering simple backups with no egress fees. It’s ideal for users looking to store GitHub repos securely and efficiently while minimizing costs.<\/p>\n\n\n\n

Wasabi<\/strong><\/h3>\n\n\n\n

Wasabi stands out for its consistent pricing and affordable, high-durability S3-compatible storage. The offer does not include additional charges for egress or API calls, making it an excellent choice for long-term GitHub information storage.<\/p>\n\n\n\n

Each of the listed providers develop unique benefits for backing up GitHub repos, ensuring that saved data is safe and easy to recover from all objects and in any location.<\/p>\n\n\n\n

Setting up GitHub backups to S3<\/strong><\/h2>\n\n\n\n

In general, backing up GitHub repos to S3-compatible storage appears simple. Still, it requires thoughtful planning considering security and scalability.<\/p>\n\n\n\n

Let’s analyze some standard methods for setting up automatic backups of the GitHub to S3.<\/p>\n\n\n\n

Utilizing AWS CLI to create GitHub backups<\/strong><\/h3>\n\n\n\n

As a reminder, AWS CLI is a command-line feature. It can automate GitHub backups to an Amazon S3 object. To automate the process, you can create a Python script or Bash script.<\/p>\n\n\n\n

Here is an example of a script code:<\/p>\n\n\n\n

#!\/bin\/bash\n# Set environment variables for S3\n\nexport AWS_ACCESS_KEY_ID=your-access-key\nexport AWS_SECRET_ACCESS_KEY=your-secret-key\nexport S3_BUCKET=your-target-bucket\n\n# Synchronize GitHub repo folder with the bucket (S3)\naws s3 sync \/path\/to\/your\/local\/repository s3:\/\/$S3_BUCKET\/backup\/ --delete<\/code><\/pre>\n\n\n\n
As you can see, the script syncs the local folder with an S3 bucket. This way, any changes are reflected in the backup. You can set up the task to run daily with the cron jobs. Another option is to integrate it into your CI\/CD pipeline for continuous backups.<\/p>\n\n\n\n
Backup automation with GitHub Actions<\/strong><\/h3>\n\n\n\n
Using GitHub Action is another way to back up GitHub folders. Using this automation solution, users can run workflows triggered by repository events, for example, a push or pull request. By integrating Amazon’s command line into the GitHub Action workflow, you are free to automatically back up selected repositories to an S3 bucket.<\/p>\n\n\n\n
And here is how you can create a GitHub Action code example:<\/p>\n\n\n\n
on:\n  push:\n     branches:\n       - main\n\njobs:\n   backup:\n     runs-on: ubuntu-latest\n     steps:          \n\t- name: Checkout Repository\n            uses: actions\/checkout@v2 \n\t- name: Install AWS CLI\n            run: sudo apt-get install awscli -y\n\n\t- name: Sync Repository to S3\n            env:\n\t \tAWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}\n\t \tAWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}\t \n \t \tS3_BUCKET: your-target-bucket\n      \t    run: aws s3 sync . s3:\/\/$S3_BUCKET\/backup\/<\/code><\/pre>\n\n\n\n
In such a case, every push to the main branch triggers a GitHub Action workflow. It uses CLI to upload the latest version of the repo to the specified S3 bucket.<\/p>\n\n\n\n
Custom Python code for GitHub backup<\/strong><\/h3>\n\n\n\n
Some developers prefer more control over the process. A Python script helps automate the backup process. Its boto3 library allows for interaction with Amazon S3. This opens a way to set up a custom backup solution for your GitHub information clone.<\/p>\n\n\n\n
The mentioned Python code example may look like this:<\/p>\n\n\n\n
import boto3\nimport os\n\n# AWS credentials from environment variables\ns3 = boto3.client('s3',\n   aws_access_key_id=os.getenv('AWS_ACCESS_KEY_ID'),\n   aws_secret_access_key=os.getenv('AWS_SECRET_ACCESS_KEY'))\n\n# Path to the local repository folder\nlocal_repo_path = '\/path\/to\/your\/local\/repository'\n\n# Upload data to S3 bucket\nfor root, dirs, files in os.walk(local_repo_path):\n   for file in files:\n       s3.upload_file(os.path.join(root, file), 'your-target-bucket', f'backup\/{file}')<\/code><\/pre>\n\n\n\n
Efficient GitHub backups – Best practices<\/strong><\/h3>\n\n\n\n
It\u2019s no surprise that storing backups in S3 is recommended to follow at least some practices to enhance data protection and compliance<\/a>. To put it short, among them are:<\/p>\n\n\n\n
Data encryption<\/strong><\/h4>\n\n\n\n
At a minimum, use server-side encryption (SSE-S3 or SSE-KMS) to protect the information in the created S3 bucket (or objects).<\/p>\n\n\n\n
IAM roles and policies<\/strong><\/h4>\n\n\n\n
Use the least privilege principle granting users (IAM user) and roles (IAM role) minimum permissions required to perform any given task.<\/p>\n\n\n\n
Versioning and Object Lock<\/strong><\/h4>\n\n\n\n
By enabling these features you can safeguard backups from accidental deletion of corruption.<\/p>\n\n\n\n
Cross-Region Replication (CRR)<\/strong><\/h4>\n\n\n\n
CRR will help you establish redundancy and disaster recovery across multiple regions.<\/p>\n\n\n\n
Lifecycle policies<\/strong><\/h4>\n\n\n\n
Move old backups to cheaper options (like S3 Glacier in AWS) to automate storage cost optimization.<\/p>\n\n\n
\n
\"GitHub<\/a><\/figure><\/div>\n\n\n
GitProtect – a middle ground between GitHub backup and S3 bucket<\/strong><\/h2>\n\n\n\n
Another good practice is incorporating software for managing GitHub backups (or other Git environments) to automate the process. Such a solution eliminates the need for manual activities and reduces the risk of losing (or leaking) data.<\/p>\n\n\n\n
GitProtect is a prime example of a reliable tool for managing GitHub backups (and other Git environments). It offers comprehensive disaster recovery services, ensuring 360-degree cyber resilience and compliance.<\/p>\n\n\n\n
As an ISO 27k and SOC 2 Type II certified tool, GitProtect offers a safe way to back up and restore all or select data. That includes repositories and metadata:<\/p>\n\n\n\n