Merge branch 'main' into add-databrain-integration-documentation

Author: Blargian

.github/workflows/badges-notify.yml

@@ -13,7 +13,7 @@ jobs:
 
     steps:
       - name: Checkout repository
-        uses: actions/checkout@v4
+        uses: actions/checkout@v5
 
       - name: Make script executable
         run: chmod +x scripts/badger.sh

.github/workflows/build-search.yml

@@ -20,10 +20,10 @@ jobs:
 
     steps:
       - name: Checkout Repository
-        uses: actions/checkout@v4
+        uses: actions/checkout@v5
 
       - name: Set up Node.js
-        uses: actions/setup-node@v4
+        uses: actions/setup-node@v6
         with:
           node-version: '20' 
 

.github/workflows/check-build.yml

@@ -19,7 +19,7 @@ jobs:
         check_type: [spellcheck, kbcheck, md-lint, glossary-check]
     steps:
       # Add setup steps per check here
-      - uses: actions/checkout@v4
+      - uses: actions/checkout@v5
       - name: Install Aspell
         if: matrix.check_type == 'spellcheck'
         run: sudo apt-get update && sudo apt-get install -y aspell aspell-en
@@ -31,7 +31,7 @@ jobs:
           uv python install 3.12 --verbose
       - name: Setup md-lint environment
         if: matrix.check_type == 'md-lint'
-        uses: actions/setup-node@v3
+        uses: actions/setup-node@v6
         with:
           node-version: 20
       - name: Install markdownlint-cli2

.github/workflows/pull-request.yaml

@@ -21,10 +21,10 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Checkout repository
-        uses: actions/checkout@v3
+        uses: actions/checkout@v5
 
       - name: Set up Node.js
-        uses: actions/setup-node@v3
+        uses: actions/setup-node@v6
         with:
           node-version: '20'  # Adjust Node.js version as needed
           cache: 'yarn'       # Cache yarn dependencies

.github/workflows/trademark-cla-approval.yml

@@ -170,7 +170,7 @@ jobs:
 
       - name: Check out repository
         if: success() && steps.process-comment.outputs.pr_number != ''
-        uses: actions/checkout@v4
+        uses: actions/checkout@v5
         with:
           fetch-depth: 0
           # For forked PRs, we need to fetch from the fork and checkout the SHA

.github/workflows/trademark-cla-notice.yml

@@ -25,7 +25,7 @@ jobs:
           private-key: "${{ secrets.WORKFLOW_AUTH_PUBLIC_PRIVATE_KEY }}"
 
       - name: Check out code
-        uses: actions/checkout@v4
+        uses: actions/checkout@v5
         with:
           fetch-depth: 0
           # Use the GitHub App token if available, otherwise fallback to GITHUB_TOKEN

.github/workflows/vale-linter.yml

@@ -20,7 +20,7 @@ jobs:
     timeout-minutes: 10
     steps:
       - name: Checkout repository
-        uses: actions/checkout@v3
+        uses: actions/checkout@v5
         with:
           fetch-depth: 0
           path: .

docs/best-practices/_snippets/_avoid_mutations.md

@@ -6,6 +6,10 @@ When a mutation is issued, ClickHouse schedules the creation of new **mutated pa
 
 For large datasets, this can produce a substantial spike in disk I/O and degrade overall cluster performance. Unlike merges, mutations can't be rolled back once submitted and will continue to execute even after server restarts unless explicitly cancelled—see [`KILL MUTATION`](/sql-reference/statements/kill#kill-mutation).
 
+:::tip Monitoring the number of active or queued mutations in ClickHouse
+For how to monitor the number of active or queued mutations refer to the following [knowledge base article](/knowledgebase/view_number_of_active_mutations).
+:::
+
 Mutations are **totally ordered**: they apply to data inserted before the mutation was issued, while newer data remains unaffected. They do not block inserts but can still overlap with other ongoing queries. A SELECT running during a mutation may read a mix of mutated and unmutated parts, which can lead to inconsistent views of the data during execution. ClickHouse executes mutations in parallel per part, which can further intensify memory and CPU usage, especially when complex subqueries (like x IN (SELECT ...)) are involved.
 
 As a rule, **avoid frequent or large-scale mutations**, especially on high-volume tables. Instead, use alternative table engines such as [ReplacingMergeTree](/guides/replacing-merge-tree) or [CollapsingMergeTree](/engines/table-engines/mergetree-family/collapsingmergetree), which are designed to handle data corrections more efficiently at query time or during merges. If mutations are absolutely necessary, monitor them carefully using the system.mutations table and use `KILL MUTATION` if a process is stuck or misbehaving. Misusing mutations can lead to degraded performance, excessive storage churn, and potential service instability—so apply them with caution and sparingly.

docs/cloud/features/08_backups.md

@@ -0,0 +1,101 @@
+---
+sidebar_label: 'Backups'
+slug: /cloud/features/backups
+title: 'Backups'
+keywords: ['backups', 'cloud backups', 'restore']
+description: 'Provides an overview of backup features in ClickHouse Cloud'
+doc_type: 'reference'
+---
+
+import Image from '@theme/IdealImage';
+import ScalePlanFeatureBadge from '@theme/badges/ScalePlanFeatureBadge';
+import EnterprisePlanFeatureBadge from '@theme/badges/EnterprisePlanFeatureBadge'
+import backup_chain from '@site/static/images/cloud/manage/backup-chain.png';
+
+Database backups provide a safety net by ensuring that if data is lost for any unforeseen reason, the service can be restored to a previous state from the last successful backup.
+This minimizes downtime and prevents business critical data from being permanently lost.
+
+## Backups {#backups}
+
+### How backups work in ClickHouse Cloud {#how-backups-work-in-clickhouse-cloud}
+
+ClickHouse Cloud backups are a combination of "full" and "incremental" backups that constitute a backup chain. The chain starts with a full backup, and incremental backups are then taken over the next several scheduled time periods to create a sequence of backups. Once a backup chain reaches a certain length, a new chain is started. This entire chain of backups can then be utilized to restore data to a new service if needed. Once all backups included in a specific chain are past the retention time frame set for the service (more on retention below), the chain is discarded.
+
+In the screenshot below, the solid line squares show full backups and the dotted line squares show incremental backups. The solid line rectangle around the squares denotes the retention period and the backups that are visible to the end user, which can be used for a backup restore. In the scenario below, backups are being taken every 24 hours and are retained for 2 days.
+
+On Day 1, a full backup is taken to start the backup chain. On Day 2, an incremental backup is taken, and we now have a full and incremental backup available to restore from. By Day 7, we have one full backup and six incremental backups in the chain, with the most recent two incremental backups visible to the user. On Day 8, we take a new full backup, and on Day 9, once we have two backups in the new chain, the previous chain is discarded.
+
+<Image img={backup_chain} size="lg" alt="Backup chain example in ClickHouse Cloud" />
+
+### Default backup policy {#default-backup-policy}
+
+In the Basic, Scale, and Enterprise tiers, backups are metered and billed separately from storage.
+All services will default to one daily backup with the ability to configure more, starting with the Scale tier, via the Settings tab of the Cloud console.
+Each backup will be retained for at least 24 hours.
+
+See ["Review and restore backups"](/cloud/manage/backups/overview) for further details.
+
+## Configurable backups {#configurable-backups}
+
+<ScalePlanFeatureBadge feature="Configurable Backups" linking_verb_are="True"/>
+
+ClickHouse Cloud allows you to configure the schedule for your backups for **Scale** and **Enterprise** tier services. Backups can be configured along the following dimensions based on your business needs.
+
+- **Retention**: The duration of days, for which each backup will be retained. Retention can be specified as low as 1 day, and as high as 30 days with several values to pick in between.
+- **Frequency**: The frequency allows you to specify the time duration between subsequent backups. For instance, a frequency of "every 12 hours" means that backups will be spaced 12 hours apart. Frequency can range from "every 6 hours" to "every 48 hours" in the following hourly increments: `6`, `8`, `12`, `16`, `20`, `24`, `36`, `48`.
+- **Start Time**: The start time for when you want to schedule backups each day. Specifying a start time implies that the backup "Frequency" will default to once every 24 hours.  Clickhouse Cloud will start the backup within an hour of the specified start time.
+
+:::note
+The custom schedule will override the default backup policy in ClickHouse Cloud for your given service.
+
+In some rare scenarios, the backup scheduler will not respect the **Start Time** specified for backups. Specifically, this happens if there was a successful backup triggered < 24 hours from the time of the currently scheduled backup. This could happen due to a retry mechanism we have in place for backups. In such instances, the scheduler will skip over the backup for the current day, and will retry the backup the next day at the scheduled time.
+:::
+
+See ["Configure backup schedules"](/cloud/manage/backups/configurable-backups) for steps to configure your backups.
+
+## Bring Your Own Bucket (BYOB) Backups {#byob}
+
+<EnterprisePlanFeatureBadge/>
+
+ClickHouse Cloud allows exporting backups to your own cloud service provider (CSP) account storage (AWS S3, Google Cloud Storage, or Azure Blob Storage).
+If you configure backups to your own bucket, ClickHouse Cloud will still take daily backups to its own bucket.
+This is to ensure that we have at least one copy of the data to restore from in case the backups in your bucket get corrupted.
+For details of how ClickHouse Cloud backups work, see the [backups](/cloud/manage/backups/overview) docs.
+
+In this guide, we walk through how you can export backups to your AWS, GCP, Azure object storage, as well as how to restore these backups in your account to a new ClickHouse Cloud service.
+We also share backup / restore commands that allow you to export backups to your bucket and restore them.
+
+:::note Cross-region backups
+Users should be aware that any usage where backups are being exported to a
+different region in the same cloud provider will incur [data transfer](/cloud/manage/network-data-transfer)
+charges.
+
+Currently, we do not support cross-cloud backups, nor backup / restore for services utilizing [Transparent Data Encryption (TDE)](/cloud/security/cmek#transparent-data-encryption-tde) or for regulated services.
+:::
+
+See ["Export backups to your own Cloud account"](/cloud/manage/backups/export-backups-to-own-cloud-account) for examples of how to take full and incremental backups to AWS, GCP, Azure object storage as well as how to restore from the backups.
+
+### Backup options {#backup-options}
+
+To export backups to your own cloud account, you have two options:
+
+<VerticalStepper headerLevel="h5">
+
+##### Via Cloud Console UI {#via-ui}
+
+External backups can be [configured in the UI](/cloud/manage/backups/backup-restore-via-ui).
+By default, backups will then be taken daily (as specified in the [default backup policy](/cloud/features/backups#default-backup-policy)).
+However, we also support [configurable](/cloud/manage/backups/configurable-backups) backups to your own cloud account, which allows for setting a custom schedule.
+It is important to note that all backups to your bucket are full backups with no relationship to other previous or future backups.
+
+##### Using SQL commands {#using-commands}
+
+You can use [SQL commands](/cloud/manage/backups/backup-restore-via-commands) to export backups to your bucket.
+
+</VerticalStepper>
+
+:::warning
+ClickHouse Cloud will not manage the lifecycle of backups in customer buckets.
+Customers are responsible for ensuring that backups in their bucket are managed appropriately for adhering to compliance standards as well as managing cost.
+If the backups are corrupted, they will not be able to be restored.
+:::