GitLab performs backup restoration and/or failover tests quarterly to confirm the reliability and integrity of system backups and/or recovery operations.
Context
By validating system backups/recovery operations in the event of an actual disaster or other disruption to service; we will have greater proficiency in restoring service to customers.
Scope
This control applies to all systems within our production environment. The production environment includes all endpoints and cloud assets used in hosting GitLab.com and its subdomains. This may include third-party systems that support the business of GitLab.com.
Ownership
Control Owner: Infrastructure Team
Process owner: Infrastructure Team
Guidance
This guidance is a two-parter, provide evidence demonstrating:
Documentation of incident response plan, including:
Roles and responsibilities
Specific procedures
Business recovery and continuity procedures
Backup processes
Legal requirements for reporting compromises
Coverage and responses for all critical systems
Reference/inclusion of incident response procedures from the payment brands
Sample of previously reported incidents documentation
Additional control information and project tracking
Non-public information relating to this security control as well as links to the work associated with various phases of project work can be found in the Resilience Testing control issue.
Examples of evidence an auditor might request to satisfy this control:
A copy of GitLab's backup, disaster recovery, and incident response processes
Documentation showing the testing of backup and disaster recovery procedures
Testing the efficiency of GitLab backup and recovery process (at a minimum) on a quarterly basis.
Policy Reference
A summary of our backup strategy is maintained here.
Database disaster recovery is documented in the GitLab Handbook.
Database backup recovery testing is implemented in the form of an automated pipeline.
Snapshot restoration has been manually verified and handbook updates reflecting such testing and verification merged.
GCP snapshot procedures can be found here. A manual verification of this process was done here.
Incident management is documented in the GitLab Handbook. The documentation contains the following components:
Incident runbooks are available and maintained in the runbooks project.
