Skip to content

Commit f4cec15

Browse files
authored
Merge pull request #4470 from ClickHouse/cloud-disaster-recovery
Guide: Cloud disaster recovery guide
2 parents 644098e + 2d0f7ee commit f4cec15

File tree

18 files changed

+461
-142
lines changed

18 files changed

+461
-142
lines changed

docs/cloud/features/08_backups.md

Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,101 @@
1+
---
2+
sidebar_label: 'Backups'
3+
slug: /cloud/features/backups
4+
title: 'Backups'
5+
keywords: ['backups', 'cloud backups', 'restore']
6+
description: 'Provides an overview of backup features in ClickHouse Cloud'
7+
doc_type: 'reference'
8+
---
9+
10+
import Image from '@theme/IdealImage';
11+
import ScalePlanFeatureBadge from '@theme/badges/ScalePlanFeatureBadge';
12+
import EnterprisePlanFeatureBadge from '@theme/badges/EnterprisePlanFeatureBadge'
13+
import backup_chain from '@site/static/images/cloud/manage/backup-chain.png';
14+
15+
Database backups provide a safety net by ensuring that if data is lost for any unforeseen reason, the service can be restored to a previous state from the last successful backup.
16+
This minimizes downtime and prevents business critical data from being permanently lost.
17+
18+
## Backups {#backups}
19+
20+
### How backups work in ClickHouse Cloud {#how-backups-work-in-clickhouse-cloud}
21+
22+
ClickHouse Cloud backups are a combination of "full" and "incremental" backups that constitute a backup chain. The chain starts with a full backup, and incremental backups are then taken over the next several scheduled time periods to create a sequence of backups. Once a backup chain reaches a certain length, a new chain is started. This entire chain of backups can then be utilized to restore data to a new service if needed. Once all backups included in a specific chain are past the retention time frame set for the service (more on retention below), the chain is discarded.
23+
24+
In the screenshot below, the solid line squares show full backups and the dotted line squares show incremental backups. The solid line rectangle around the squares denotes the retention period and the backups that are visible to the end user, which can be used for a backup restore. In the scenario below, backups are being taken every 24 hours and are retained for 2 days.
25+
26+
On Day 1, a full backup is taken to start the backup chain. On Day 2, an incremental backup is taken, and we now have a full and incremental backup available to restore from. By Day 7, we have one full backup and six incremental backups in the chain, with the most recent two incremental backups visible to the user. On Day 8, we take a new full backup, and on Day 9, once we have two backups in the new chain, the previous chain is discarded.
27+
28+
<Image img={backup_chain} size="lg" alt="Backup chain example in ClickHouse Cloud" />
29+
30+
### Default backup policy {#default-backup-policy}
31+
32+
In the Basic, Scale, and Enterprise tiers, backups are metered and billed separately from storage.
33+
All services will default to one daily backup with the ability to configure more, starting with the Scale tier, via the Settings tab of the Cloud console.
34+
Each backup will be retained for at least 24 hours.
35+
36+
See ["Review and restore backups"](/cloud/manage/backups/overview) for further details.
37+
38+
## Configurable backups {#configurable-backups}
39+
40+
<ScalePlanFeatureBadge feature="Configurable Backups" linking_verb_are="True"/>
41+
42+
ClickHouse Cloud allows you to configure the schedule for your backups for **Scale** and **Enterprise** tier services. Backups can be configured along the following dimensions based on your business needs.
43+
44+
- **Retention**: The duration of days, for which each backup will be retained. Retention can be specified as low as 1 day, and as high as 30 days with several values to pick in between.
45+
- **Frequency**: The frequency allows you to specify the time duration between subsequent backups. For instance, a frequency of "every 12 hours" means that backups will be spaced 12 hours apart. Frequency can range from "every 6 hours" to "every 48 hours" in the following hourly increments: `6`, `8`, `12`, `16`, `20`, `24`, `36`, `48`.
46+
- **Start Time**: The start time for when you want to schedule backups each day. Specifying a start time implies that the backup "Frequency" will default to once every 24 hours. Clickhouse Cloud will start the backup within an hour of the specified start time.
47+
48+
:::note
49+
The custom schedule will override the default backup policy in ClickHouse Cloud for your given service.
50+
51+
In some rare scenarios, the backup scheduler will not respect the **Start Time** specified for backups. Specifically, this happens if there was a successful backup triggered < 24 hours from the time of the currently scheduled backup. This could happen due to a retry mechanism we have in place for backups. In such instances, the scheduler will skip over the backup for the current day, and will retry the backup the next day at the scheduled time.
52+
:::
53+
54+
See ["Configure backup schedules"](/cloud/manage/backups/configurable-backups) for steps to configure your backups.
55+
56+
## Bring Your Own Bucket (BYOB) Backups {#byob}
57+
58+
<EnterprisePlanFeatureBadge/>
59+
60+
ClickHouse Cloud allows exporting backups to your own cloud service provider (CSP) account storage (AWS S3, Google Cloud Storage, or Azure Blob Storage).
61+
If you configure backups to your own bucket, ClickHouse Cloud will still take daily backups to its own bucket.
62+
This is to ensure that we have at least one copy of the data to restore from in case the backups in your bucket get corrupted.
63+
For details of how ClickHouse Cloud backups work, see the [backups](/cloud/manage/backups/overview) docs.
64+
65+
In this guide, we walk through how you can export backups to your AWS, GCP, Azure object storage, as well as how to restore these backups in your account to a new ClickHouse Cloud service.
66+
We also share backup / restore commands that allow you to export backups to your bucket and restore them.
67+
68+
:::note Cross-region backups
69+
Users should be aware that any usage where backups are being exported to a
70+
different region in the same cloud provider will incur [data transfer](/cloud/manage/network-data-transfer)
71+
charges.
72+
73+
Currently, we do not support cross-cloud backups, nor backup / restore for services utilizing [Transparent Data Encryption (TDE)](/cloud/security/cmek#transparent-data-encryption-tde) or for regulated services.
74+
:::
75+
76+
See ["Export backups to your own Cloud account"](/cloud/manage/backups/export-backups-to-own-cloud-account) for examples of how to take full and incremental backups to AWS, GCP, Azure object storage as well as how to restore from the backups.
77+
78+
### Backup options {#backup-options}
79+
80+
To export backups to your own cloud account, you have two options:
81+
82+
<VerticalStepper headerLevel="h5">
83+
84+
##### Via Cloud Console UI {#via-ui}
85+
86+
External backups can be [configured in the UI](/cloud/manage/backups/backup-restore-via-ui).
87+
By default, backups will then be taken daily (as specified in the [default backup policy](/cloud/features/backups#default-backup-policy)).
88+
However, we also support [configurable](/cloud/manage/backups/configurable-backups) backups to your own cloud account, which allows for setting a custom schedule.
89+
It is important to note that all backups to your bucket are full backups with no relationship to other previous or future backups.
90+
91+
##### Using SQL commands {#using-commands}
92+
93+
You can use [SQL commands](/cloud/manage/backups/backup-restore-via-commands) to export backups to your bucket.
94+
95+
</VerticalStepper>
96+
97+
:::warning
98+
ClickHouse Cloud will not manage the lifecycle of backups in customer buckets.
99+
Customers are responsible for ensuring that backups in their bucket are managed appropriately for adhering to compliance standards as well as managing cost.
100+
If the backups are corrupted, they will not be able to be restored.
101+
:::

docs/cloud/features/08_backups/backups-to-own-cloud/_category_.yml

Lines changed: 0 additions & 3 deletions
This file was deleted.

docs/cloud/features/08_backups/backups-to-own-cloud/export-backups-to-own-cloud-account.md

Lines changed: 0 additions & 57 deletions
This file was deleted.

docs/cloud/features/08_backups/configurable-backups.md

Lines changed: 0 additions & 46 deletions
This file was deleted.

docs/cloud/features/08_backups/index.md

Lines changed: 0 additions & 13 deletions
This file was deleted.

docs/cloud/features/08_backups/overview.md renamed to docs/cloud/guides/backups/01_review-and-restore-backups.md

Lines changed: 6 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
sidebar_label: 'Overview'
2+
sidebar_label: 'Review and restore backups'
33
sidebar_position: 0
44
slug: /cloud/manage/backups/overview
55
title: 'Overview'
@@ -17,29 +17,13 @@ import backup_usage from '@site/static/images/cloud/manage/backup-usage.png';
1717
import backup_restore from '@site/static/images/cloud/manage/backup-restore.png';
1818
import backup_service_provisioning from '@site/static/images/cloud/manage/backup-service-provisioning.png';
1919

20-
# Backups
20+
# Review and restore backups
2121

22-
Database backups provide a safety net by ensuring that if data is lost for any unforeseen reason, the service can be restored to a previous state from the last successful backup. This minimizes downtime and prevents business critical data from being permanently lost. This guide covers how backups work in ClickHouse Cloud, what options you have to configure backups for your service, and how to restore from a backup.
23-
24-
## How backups work in ClickHouse Cloud {#how-backups-work-in-clickhouse-cloud}
25-
26-
ClickHouse Cloud backups are a combination of "full" and "incremental" backups that constitute a backup chain. The chain starts with a full backup, and incremental backups are then taken over the next several scheduled time periods to create a sequence of backups. Once a backup chain reaches a certain length, a new chain is started. This entire chain of backups can then be utilized to restore data to a new service if needed. Once all backups included in a specific chain are past the retention time frame set for the service (more on retention below), the chain is discarded.
27-
28-
In the screenshot below, the solid line squares show full backups and the dotted line squares show incremental backups. The solid line rectangle around the squares denotes the retention period and the backups that are visible to the end user, which can be used for a backup restore. In the scenario below, backups are being taken every 24 hours and are retained for 2 days.
29-
30-
On Day 1, a full backup is taken to start the backup chain. On Day 2, an incremental backup is taken, and we now have a full and incremental backup available to restore from. By Day 7, we have one full backup and six incremental backups in the chain, with the most recent two incremental backups visible to the user. On Day 8, we take a new full backup, and on Day 9, once we have two backups in the new chain, the previous chain is discarded.
31-
32-
<Image img={backup_chain} size="md" alt="Backup chain example in ClickHouse Cloud" />
33-
34-
*Example backup scenario in Clickhouse Cloud*
35-
36-
## Default backup policy {#default-backup-policy}
37-
38-
In the Basic, Scale, and Enterprise tiers, backups are metered and billed separately from storage. All services will default to one daily backup with the ability to configure more, starting with the Scale tier, via the Settings tab of the Cloud console. Each backup will be retained for at least 24 hours.
22+
This guide covers how backups work in ClickHouse Cloud, what options you have to configure backups for your service, and how to restore from a backup.
3923

4024
## Backup status list {#backup-status-list}
4125

42-
Your service will be backed up based on the set schedule, whether it is the default daily schedule or a [custom schedule](./configurable-backups.md) picked by you. All available backups can be viewed from the **Backups** tab of the service. From here, you can see the status of the backup, the duration, as well as the size of the backup. You can also restore a specific backup using the **Actions** column.
26+
Your service will be backed up based on the set schedule, whether it is the default daily schedule or a [custom schedule](/cloud/manage/backups/configurable-backups) picked by you. All available backups can be viewed from the **Backups** tab of the service. From here, you can see the status of the backup, the duration, as well as the size of the backup. You can also restore a specific backup using the **Actions** column.
4327

4428
<Image img={backup_status_list} size="md" alt="List of backup statuses in ClickHouse Cloud" border/>
4529

@@ -174,8 +158,8 @@ Legacy Plans: For customers on legacy plans, default daily backups retained for
174158

175159
## Configurable backups {#configurable-backups}
176160

177-
If you want to set up a backups schedule different from the default backup schedule, take a look at [Configurable Backups](./configurable-backups.md).
161+
If you want to set up a backups schedule different from the default backup schedule, take a look at [Configurable Backups](/cloud/manage/backups/configurable-backups).
178162

179163
## Export backups to your own cloud account {#export-backups-to-your-own-cloud-account}
180164

181-
For users wanting to export backups to their own cloud account, see [here](./backups-to-own-cloud/export-backups-to-own-cloud-account.md).
165+
For users wanting to export backups to their own cloud account, see [here](/cloud/manage/backups/export-backups-to-own-cloud-account).

0 commit comments

Comments
 (0)