Skip to content

Commit 4a92d80

Browse files
authored
Merge branch 'main' into rsickles-patch-12
2 parents 344ad85 + 35d3940 commit 4a92d80

File tree

19 files changed

+5468
-1055
lines changed

19 files changed

+5468
-1055
lines changed

docs/chdb/api/python.md

Lines changed: 3517 additions & 0 deletions
Large diffs are not rendered by default.

docs/chdb/index.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@ You can use it when you want to get the power of ClickHouse in a programming lan
2727

2828
chDB has the following language bindings:
2929

30-
* [Python](install/python.md)
30+
* [Python](install/python.md) - [API Reference](api/python.md)
3131
* [Go](install/go.md)
3232
* [Rust](install/rust.md)
3333
* [NodeJS](install/nodejs.md)
@@ -37,7 +37,7 @@ chDB has the following language bindings:
3737
## How do I get started? {#how-do-i-get-started}
3838

3939
* If you're using [Go](install/go.md), [Rust](install/rust.md), [NodeJS](install/nodejs.md), [Bun](install/bun.md) or [C and C++](install/c.md), take a look at the corresponding language pages.
40-
* If you're using Python, see the [getting started developer guide](getting-started.md). There are also guides showing how to do common tasks like:
40+
* If you're using Python, see the [getting started developer guide](getting-started.md) or the [chDB on-demand course](https://learn.clickhouse.com/user_catalog_class/show/1901178). There are also guides showing how to do common tasks like:
4141
* [JupySQL](guides/jupysql.md)
4242
* [Querying Pandas](guides/querying-pandas.md)
4343
* [Querying Apache Arrow](guides/querying-apache-arrow.md)
@@ -68,6 +68,7 @@ chDB delivers exceptional performance across different scenarios:
6868

6969
- Read the full story about the birth of the chDB project on [blog](https://clickhouse.com/blog/chdb-embedded-clickhouse-rocket-engine-on-a-bicycle)
7070
- Read about chDB and its use cases on the [Blog](https://clickhouse.com/blog/welcome-chdb-to-clickhouse)
71+
- Take the [chDB on-demand course](https://learn.clickhouse.com/user_catalog_class/show/1901178)
7172
- Discover chDB in your browser using [codapi examples](https://antonz.org/trying-chdb/)
7273
- More examples see (https://github.com/chdb-io/chdb/tree/main/examples)
7374

docs/chdb/install/index.md

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -8,11 +8,11 @@ doc_type: 'landing-page'
88

99
Instructions for how to get setup with chDB are available below for the following languages and runtimes:
1010

11-
| Language |
12-
|----------------------------------------|
13-
| [Python](/chdb/install/python) |
14-
| [NodeJS](/chdb/install/nodejs) |
15-
| [Go](/chdb/install/go) |
16-
| [Rust](/chdb/install/rust) |
17-
| [Bun](/chdb/install/bun) |
18-
| [C and C++](/chdb/install/c) |
11+
| Language | API Reference |
12+
|----------------------------------------|-------------------------------------|
13+
| [Python](/chdb/install/python) | [Python API](/chdb/api/python) |
14+
| [NodeJS](/chdb/install/nodejs) | |
15+
| [Go](/chdb/install/go) | |
16+
| [Rust](/chdb/install/rust) | |
17+
| [Bun](/chdb/install/bun) | |
18+
| [C and C++](/chdb/install/c) | |

docs/cloud/features/04_infrastructure/byoc.md

Lines changed: 29 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -357,6 +357,7 @@ Contact support to schedule maintenance windows. Please expect a minimum of a we
357357
## Observability {#observability}
358358

359359
### Built-in monitoring tools {#built-in-monitoring-tools}
360+
ClickHouse BYOC provides several approaches for various use cases.
360361

361362
#### Observability dashboard {#observability-dashboard}
362363

@@ -378,8 +379,20 @@ You can customize a dashboard using metrics from system tables like `system.metr
378379

379380
<br />
380381

382+
#### Access the BYOC Prometheus stack {#prometheus-access}
383+
ClickHouse BYOC deploys a Prometheus stack on your Kubernetes cluster. You may access and scrape the metrics from there and integrate them with your own monitoring stack.
384+
385+
Contact ClickHouse support to enable the Private Load balancer and ask for the URL. Please note that this URL is only accessible via private network and does not support authentication
386+
387+
**Sample URL**
388+
```bash
389+
https://prometheus-internal.<subdomain>.<region>.aws.clickhouse-byoc.com/query
390+
```
391+
381392
#### Prometheus Integration {#prometheus-integration}
382393

394+
**DEPRECATED: ** Please use the Prometheus stack integration in the above section instead. Besides the ClickHouse Server metrics, it provides more metrics including the K8S metrics and metrics from other services.
395+
383396
ClickHouse Cloud provides a Prometheus endpoint that you can use to scrape metrics for monitoring. This allows for integration with tools like Grafana and Datadog for visualization.
384397

385398
**Sample request via https endpoint /metrics_all**
@@ -419,8 +432,22 @@ ClickHouse_CustomMetric_TotalNumberOfErrors{hostname="c-jet-ax-16-server-43d5baj
419432
A ClickHouse username and password pair can be used for authentication. We recommend creating a dedicated user with minimal permissions for scraping metrics. At minimum, a `READ` permission is required on the `system.custom_metrics` table across replicas. For example:
420433

421434
```sql
422-
GRANT REMOTE ON *.* TO scraping_user
423-
GRANT SELECT ON system.custom_metrics TO scraping_user
435+
GRANT REMOTE ON *.* TO scrapping_user;
436+
GRANT SELECT ON system._custom_metrics_dictionary_custom_metrics_tables TO scrapping_user;
437+
GRANT SELECT ON system._custom_metrics_dictionary_database_replicated_recovery_time TO scrapping_user;
438+
GRANT SELECT ON system._custom_metrics_dictionary_failed_mutations TO scrapping_user;
439+
GRANT SELECT ON system._custom_metrics_dictionary_group TO scrapping_user;
440+
GRANT SELECT ON system._custom_metrics_dictionary_shared_catalog_recovery_time TO scrapping_user;
441+
GRANT SELECT ON system._custom_metrics_dictionary_table_read_only_duration_seconds TO scrapping_user;
442+
GRANT SELECT ON system._custom_metrics_view_error_metrics TO scrapping_user;
443+
GRANT SELECT ON system._custom_metrics_view_histograms TO scrapping_user;
444+
GRANT SELECT ON system._custom_metrics_view_metrics_and_events TO scrapping_user;
445+
GRANT SELECT(description, metric, value) ON system.asynchronous_metrics TO scrapping_user;
446+
GRANT SELECT ON system.custom_metrics TO scrapping_user;
447+
GRANT SELECT(name, value) ON system.errors TO scrapping_user;
448+
GRANT SELECT(description, event, value) ON system.events TO scrapping_user;
449+
GRANT SELECT(description, labels, metric, value) ON system.histogram_metrics TO scrapping_user;
450+
GRANT SELECT(description, metric, value) ON system.metrics TO scrapping_user;
424451
```
425452

426453
**Configuring Prometheus**

docs/cloud/features/05_admin_features/upgrades.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ When you create a service, the [`compatibility`](/operations/settings/settings#c
3030

3131
The `compatibility` setting allows you to use default values of settings from previous versions. When your service is upgraded to a new version, the version specified for the `compatibility` setting does not change. This means that default values for settings that existed when you first created your service will not change (unless you have already overridden those default values, in which case they will persist after the upgrade).
3232

33-
You cannot manage the `compatibility` setting for your service. You must [contact support](https://clickhouse.com/support/program) if you would like to change the version set for your `compatibility` setting.
33+
You cannot manage the service-level default `compatibility` setting for your service. You must [contact support](https://clickhouse.com/support/program) if you would like to change the version set for your service's default `compatibility` setting. However, you can override the `compatibility` setting at the user, role, profile, query, or session level using standard ClickHouse setting mechanisms such as `SET compatibility = '22.3'` in a session or `SETTINGS compatibility = '22.3'` in a query.
3434

3535
## Maintenance mode {#maintenance-mode}
3636

docs/cloud/guides/security/cloud_access_management/saml-sso-setup.md

Lines changed: 5 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,7 @@ We recommend setting up a **direct link to your organization** in addition to yo
3939

4040
1. Sign in to your [ClickHouse Cloud](https://console.clickhouse.cloud) organization.
4141

42-
<Image img={samlOrgId} size="md" alt="Organization ID" />
42+
<Image img={samlOrgId} size="md" alt="Organization ID" force/>
4343
4444
3. In the lower left corner, click on your organization name under **Organization**.
4545

@@ -197,7 +197,7 @@ You will configure two App Integrations in Okta for each ClickHouse organization
197197

198198
12. On the **Sign On** tab for your new app, click the **View SAML setup instructions** button.
199199

200-
<Image img={samlOktaSetup} size="md" alt="Okta SAML Setup Instructions" />
200+
<Image img={samlOktaSetup} size="md" alt="Okta SAML Setup Instructions" force/>
201201

202202
13. Gather these three items and go to Submit a Support Case above to complete the process.
203203
- Identity Provider Single Sign-On URL
@@ -215,7 +215,7 @@ You will configure one SAML app in Google for each organization and must provide
215215

216216
1. Go to your Google Admin console (admin.google.com).
217217

218-
<Image img={samlGoogleApp} size="md" alt="Google SAML App" />
218+
<Image img={samlGoogleApp} size="md" alt="Google SAML App" force/>
219219

220220
2. Click **Apps**, then **Web and mobile apps** on the left.
221221

@@ -272,7 +272,7 @@ Azure (Microsoft) SAML may also be referred to as Azure Active Directory (AD) or
272272

273273
5. Enter a name and select **Integrate any other application you don't find in the gallery (Non-gallery)**, then click **Create**.
274274

275-
<Image img={samlAzureApp} size="md" alt="Azure Non-Gallery App" />
275+
<Image img={samlAzureApp} size="md" alt="Azure Non-Gallery App" force/>
276276

277277
6. Click **Users and groups** on the left and assign users.
278278

@@ -298,7 +298,7 @@ Azure (Microsoft) SAML may also be referred to as Azure Active Directory (AD) or
298298
| (A) email | Basic | user.mail |
299299
| (U) /identity/claims/name | Omitted | user.mail |
300300

301-
<Image img={samlAzureClaims} size="md" alt="Attributes and Claims" />
301+
<Image img={samlAzureClaims} size="md" alt="Attributes and Claims" force/>
302302

303303
12. Gather these two items and go to Submit a Support Case above to complete the process:
304304
- Login URL
@@ -366,4 +366,3 @@ Security is our top priority when it comes to authentication. For this reason, w
366366
- **All users assigned to your app via your IdP must have the same email domain.** If you have vendors, contractors or consultants you would like to have access to your ClickHouse account, they must have an email address with the same domain (e.g. user@domain.com) as your employees.
367367

368368
- **We do not automatically link SSO and non-SSO accounts.** You may see multiple accounts for your users in your ClickHouse user list even if they are using the same email address.
369-

docs/cloud/reference/03_billing/01_billing_overview.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -184,6 +184,10 @@ A ClickHouse Credit is a unit of credit toward Customer's usage of ClickHouse Cl
184184

185185
Legacy pricing information can be found [here](https://clickhouse.com/pricing?legacy=true).
186186

187+
:::note
188+
If you are being billed through Stripe then you will see that 1 CHC is equal to \$0.01 USD on your Stripe invoice. This is to allow accurate billing on Stripe due to their limitation on not being able to bill fractional quantities of our standard SKU of 1 CHC = \$1 USD.
189+
:::
190+
187191
### How is compute metered? {#how-is-compute-metered}
188192

189193
ClickHouse Cloud meters compute on a per-minute basis, in 8G RAM increments.

docs/deployment-guides/replication-sharding-examples/01_1_shard_2_replicas.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,7 @@ for i in {01..02}; do
6666
done
6767
```
6868

69-
Add the following `docker-compose.yml` file to the `clickhouse-cluster` directory:
69+
Add the following `docker-compose.yml` file to the `cluster_1S_2R` directory:
7070

7171
```yaml title="docker-compose.yml"
7272
version: '3.8'

docs/integrations/data-ingestion/clickpipes/index.md

Lines changed: 20 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -86,11 +86,29 @@ Steps:
8686
## Adjusting ClickPipes advanced settings {#clickpipes-advanced-settings}
8787
ClickPipes provides sensible defaults that cover the requirements of most use cases. If your use case requires additional fine-tuning, you can adjust the following settings:
8888

89-
- **Streaming max insert wait time**: Configures the maximum wait period before inserting data into the ClickHouse cluster. Applies to streaming ClickPipes (e.g., Kafka, Kinesis).
90-
- **Object storage polling interval**: Configures how frequently ClickPipes checks object storage for new data. Applies to object storage ClickPipes (e.g., S3, GCS).
89+
### Object Storage ClickPipes {#clickpipes-advanced-settings-object-storage}
90+
91+
| Setting | Default value | Description |
92+
|------------------------------------|---------------|---------------------------------------------------------------------------------------|
93+
| `Max insert bytes` | 10GB | Number of bytes to process in a single insert batch. |
94+
| `Max file count` | 100 | Maximum number of files to process in a single insert batch. |
95+
| `Max threads` | auto(3) | [Maximum number of concurrent threads](/operations/settings/settings#max_threads) for file processing. |
96+
| `Max insert threads` | 1 | [Maximum number of concurrent insert threads](/operations/settings/settings#max_insert_threads) for file processing. |
97+
| `Min insert block size bytes` | 1GB | [Minimum size of bytes in the block](/operations/settings/settings#min_insert_block_size_bytes) which can be inserted into a table. |
98+
| `Max download threads` | 4 | [Maximum number of concurrent download threads](/operations/settings/settings#max_download_threads). |
99+
| `Object storage polling interval` | 30s | Configures the maximum wait period before inserting data into the ClickHouse cluster. |
100+
| `Parallel distributed insert select` | 2 | [Parallel distributed insert select setting](/operations/settings/settings#parallel_distributed_insert_select). |
101+
| `Parallel view processing` | false | Whether to enable pushing to attached views [concurrently instead of sequentially](/operations/settings/settings#parallel_view_processing). |
102+
| `Use cluster function` | true | Whether to process files in parallel across multiple nodes. |
91103

92104
<Image img={cp_advanced_settings} alt="Advanced settings for ClickPipes" size="lg" border/>
93105

106+
### Streaming ClickPipes {#clickpipes-advanced-settings-streaming}
107+
108+
| Setting | Default value | Description |
109+
|------------------------------------|---------------|---------------------------------------------------------------------------------------|
110+
| `Streaming max insert wait time` | 5s | Configures the maximum wait period before inserting data into the ClickHouse cluster. |
111+
94112
## Error reporting {#error-reporting}
95113
ClickPipes will store errors in two separate tables depending on the type of error encountered during the ingestion process.
96114
### Record Errors {#record-errors}

docs/integrations/data-ingestion/dbms/postgresql/connecting-to-postgresql.md

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,11 +14,14 @@ import ExperimentalBadge from '@theme/badges/ExperimentalBadge';
1414

1515
This page covers following options for integrating PostgreSQL with ClickHouse:
1616

17-
- using [ClickPipes](/integrations/clickpipes/postgres), the managed integration service for ClickHouse Cloud powered by PeerDB.
18-
- using [PeerDB](https://github.com/PeerDB-io/peerdb), an open-source CDC tool specifically designed for PostgreSQL database replication to both self-hosted ClickHouse and ClickHouse Cloud.
1917
- using the `PostgreSQL` table engine, for reading from a PostgreSQL table
2018
- using the experimental `MaterializedPostgreSQL` database engine, for syncing a database in PostgreSQL with a database in ClickHouse
2119

20+
:::tip
21+
We recommend using [ClickPipes](/integrations/clickpipes/postgres), a managed integration service for ClickHouse Cloud powered by PeerDB.
22+
Alternatively, [PeerDB](https://github.com/PeerDB-io/peerdb) is available as an an open-source CDC tool specifically designed for PostgreSQL database replication to both self-hosted ClickHouse and ClickHouse Cloud.
23+
:::
24+
2225
## Using the PostgreSQL table engine {#using-the-postgresql-table-engine}
2326

2427
The `PostgreSQL` table engine allows **SELECT** and **INSERT** operations on data stored on the remote PostgreSQL server from ClickHouse.

0 commit comments

Comments
 (0)