Skip to content

Commit 2882c7e

Browse files
authored
Merge branch 'main' into fix_broken_anchors_1
2 parents fd5be54 + 487d2cf commit 2882c7e

File tree

1,118 files changed

+5514
-2750
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

1,118 files changed

+5514
-2750
lines changed
Lines changed: 17 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,43 +1,52 @@
11

2+
import GCS_bucket_1 from '@site/static/images/integrations/data-ingestion/s3/GCS-bucket-1.png';
3+
import GCS_bucket_2 from '@site/static/images/integrations/data-ingestion/s3/GCS-bucket-2.png';
4+
import GCS_create_service_account_key from '@site/static/images/integrations/data-ingestion/s3/GCS-create-a-service-account-key.png';
5+
import GCS_create_service_account_0 from '@site/static/images/integrations/data-ingestion/s3/GCS-create-service-account-0.png';
6+
import GCS_create_service_account_a from '@site/static/images/integrations/data-ingestion/s3/GCS-create-service-account-a.png';
7+
import GCS_create_service_account_2 from '@site/static/images/integrations/data-ingestion/s3/GCS-create-service-account-2.png';
8+
import GCS_create_service_account_3 from '@site/static/images/integrations/data-ingestion/s3/GCS-create-service-account-3.png';
9+
import GCS_guide_key from '@site/static/images/integrations/data-ingestion/s3/GCS-guide-key.png';
10+
211
<details>
312
<summary>Create GCS buckets and an HMAC key</summary>
413

514
### ch_bucket_us_east1 {#ch_bucket_us_east1}
615

7-
![Add a bucket](@site/docs/integrations/data-ingestion/s3/images/GCS-bucket-1.png)
16+
<img src={GCS_bucket_1} alt="Creating a GCS bucket in US East 1" />
817

918
### ch_bucket_us_east4 {#ch_bucket_us_east4}
1019

11-
![Add a bucket](@site/docs/integrations/data-ingestion/s3/images/GCS-bucket-2.png)
20+
<img src={GCS_bucket_2} alt="Creating a GCS bucket in US East 4" />
1221

1322
### Generate an Access key {#generate-an-access-key}
1423

1524
### Create a service account HMAC key and secret {#create-a-service-account-hmac-key-and-secret}
1625

1726
Open **Cloud Storage > Settings > Interoperability** and either choose an existing **Access key**, or **CREATE A KEY FOR A SERVICE ACCOUNT**. This guide covers the path for creating a new key for a new service account.
1827

19-
![Add a bucket](@site/docs/integrations/data-ingestion/s3/images/GCS-create-a-service-account-key.png)
28+
<img src={GCS_create_service_account_key} alt="Generating a service account HMAC key in GCS" />
2029

2130
### Add a new service account {#add-a-new-service-account}
2231

2332
If this is a project with no existing service account, **CREATE NEW ACCOUNT**.
2433

25-
![Add a bucket](@site/docs/integrations/data-ingestion/s3/images/GCS-create-service-account-0.png)
34+
<img src={GCS_create_service_account_0} alt="Adding a new service account in GCS" />
2635

2736
There are three steps to creating the service account, in the first step give the account a meaningful name, ID, and description.
2837

29-
![Add a bucket](@site/docs/integrations/data-ingestion/s3/images/GCS-create-service-account-a.png)
38+
<img src={GCS_create_service_account_a} alt="Defining a new service account name and ID in GCS" />
3039

3140
In the Interoperability settings dialog the IAM role **Storage Object Admin** role is recommended; select that role in step two.
3241

33-
![Add a bucket](@site/docs/integrations/data-ingestion/s3/images/GCS-create-service-account-2.png)
42+
<img src={GCS_create_service_account_2} alt="Selecting IAM role Storage Object Admin in GCS" />
3443

3544
Step three is optional and not used in this guide. You may allow users to have these privileges based on your policies.
3645

37-
![Add a bucket](@site/docs/integrations/data-ingestion/s3/images/GCS-create-service-account-3.png)
46+
<img src={GCS_create_service_account_3} alt="Configuring additional settings for the new service account in GCS" />
3847

3948
The service account HMAC key will be displayed. Save this information, as it will be used in the ClickHouse configuration.
4049

41-
![Add a bucket](@site/docs/integrations/data-ingestion/s3/images/GCS-guide-key.png)
50+
<img src={GCS_guide_key} alt="Retrieving the generated HMAC key for GCS" />
4251

4352
</details>

docs/_snippets/_S3_authentication_and_bucket.md

Lines changed: 34 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,20 @@
1+
import s3_1 from '@site/static/images/_snippets/s3/s3-1.png';
2+
import s3_2 from '@site/static/images/_snippets/s3/s3-2.png';
3+
import s3_3 from '@site/static/images/_snippets/s3/s3-3.png';
4+
import s3_4 from '@site/static/images/_snippets/s3/s3-4.png';
5+
import s3_5 from '@site/static/images/_snippets/s3/s3-5.png';
6+
import s3_6 from '@site/static/images/_snippets/s3/s3-6.png';
7+
import s3_7 from '@site/static/images/_snippets/s3/s3-7.png';
8+
import s3_8 from '@site/static/images/_snippets/s3/s3-8.png';
9+
import s3_9 from '@site/static/images/_snippets/s3/s3-9.png';
10+
import s3_a from '@site/static/images/_snippets/s3/s3-a.png';
11+
import s3_b from '@site/static/images/_snippets/s3/s3-b.png';
12+
import s3_c from '@site/static/images/_snippets/s3/s3-c.png';
13+
import s3_d from '@site/static/images/_snippets/s3/s3-d.png';
14+
import s3_e from '@site/static/images/_snippets/s3/s3-e.png';
15+
import s3_f from '@site/static/images/_snippets/s3/s3-f.png';
16+
import s3_g from '@site/static/images/_snippets/s3/s3-g.png';
17+
import s3_h from '@site/static/images/_snippets/s3/s3-h.png';
118

219
<details>
320
<summary>Create S3 buckets and an IAM user</summary>
@@ -10,85 +27,85 @@ In this procedure, we'll be creating a service account user, not a login user.
1027

1128
2. In "users", select **Add users**
1229

13-
![create_iam_user_0](@site/docs/_snippets/images/s3/s3-1.png)
30+
<img src={s3_1} alt="create_iam_user_0"/>
1431

1532
3. Enter the user name and set the credential type to **Access key - Programmatic access** and select **Next: Permissions**
1633

17-
![create_iam_user_1](@site/docs/_snippets/images/s3/s3-2.png)
34+
<img src={s3_2} alt="create_iam_user_1"/>
1835

1936
4. Do not add the user to any group; select **Next: Tags**
2037

21-
![create_iam_user_2](@site/docs/_snippets/images/s3/s3-3.png)
38+
<img src={s3_3} alt="create_iam_user_2"/>
2239

2340
5. Unless you need to add any tags, select **Next: Review**
2441

25-
![create_iam_user_3](@site/docs/_snippets/images/s3/s3-4.png)
42+
<img src={s3_4} alt="create_iam_user_3"/>
2643

2744
6. Select **Create User**
2845

2946
:::note
3047
The warning message stating that the user has no permissions can be ignored; permissions will be granted on the bucket for the user in the next section
3148
:::
3249

33-
![create_iam_user_4](@site/docs/_snippets/images/s3/s3-5.png)
50+
<img src={s3_5} alt="create_iam_user_4"/>
3451

3552
7. The user is now created; click on **show** and copy the access and secret keys.
3653
:::note
3754
Save the keys somewhere else; this is the only time that the secret access key will be available.
3855
:::
3956

40-
![create_iam_user_5](@site/docs/_snippets/images/s3/s3-6.png)
57+
<img src={s3_6} alt="create_iam_user_5"/>
4158

4259
8. Click close, then find the user in the users screen.
4360

44-
![create_iam_user_6](@site/docs/_snippets/images/s3/s3-7.png)
61+
<img src={s3_7} alt="create_iam_user_6"/>
4562

4663
9. Copy the ARN (Amazon Resource Name) and save it for use when configuring the access policy for the bucket.
4764

48-
![create_iam_user_7](@site/docs/_snippets/images/s3/s3-8.png)
65+
<img src={s3_8} alt="create_iam_user_7"/>
4966

5067
### Create an S3 bucket {#create-an-s3-bucket}
5168
1. In the S3 bucket section, select **Create bucket**
5269

53-
![create_s3_bucket_0](@site/docs/_snippets/images/s3/s3-9.png)
70+
<img src={s3_9} alt="create_s3_bucket_0"/>
5471

5572
2. Enter a bucket name, leave other options default
5673
:::note
5774
The bucket name must be unique across AWS, not just the organization, or it will emit an error.
5875
:::
5976
3. Leave `Block all Public Access` enabled; public access is not needed.
6077

61-
![create_s3_bucket_2](@site/docs/_snippets/images/s3/s3-a.png)
78+
<img src={s3_a} alt="create_s3_bucket_2"/>
6279

6380
4. Select **Create Bucket** at the bottom of the page
6481

65-
![create_s3_bucket_3](@site/docs/_snippets/images/s3/s3-b.png)
82+
<img src={s3_b} alt="create_s3_bucket_3"/>
6683

6784
5. Select the link, copy the ARN, and save it for use when configuring the access policy for the bucket.
6885

6986
6. Once the bucket has been created, find the new S3 bucket in the S3 buckets list and select the link
7087

71-
![create_s3_bucket_4](@site/docs/_snippets/images/s3/s3-c.png)
88+
<img src={s3_c} alt="create_s3_bucket_4"/>
7289

7390
7. Select **Create folder**
7491

75-
![create_s3_bucket_5](@site/docs/_snippets/images/s3/s3-d.png)
92+
<img src={s3_d} alt="create_s3_bucket_5"/>
7693

7794
8. Enter a folder name that will be the target for the ClickHouse S3 disk and select **Create folder**
7895

79-
![create_s3_bucket_6](@site/docs/_snippets/images/s3/s3-e.png)
96+
<img src={s3_e} alt="create_s3_bucket_6"/>
8097

8198
9. The folder should now be visible on the bucket list
8299

83-
![create_s3_bucket_7](@site/docs/_snippets/images/s3/s3-f.png)
100+
<img src={s3_f} alt="create_s3_bucket_7"/>
84101

85102
10. Select the checkbox for the new folder and click on **Copy URL** Save the URL copied to be used in the ClickHouse storage configuration in the next section.
86103

87-
![create_s3_bucket_8](@site/docs/_snippets/images/s3/s3-g.png)
104+
<img src={s3_g} alt="create_s3_bucket_8"/>
88105

89106
11. Select the **Permissions** tab and click on the **Edit** button in the **Bucket Policy** section
90107

91-
![create_s3_bucket_9](@site/docs/_snippets/images/s3/s3-h.png)
108+
<img src={s3_h} alt="create_s3_bucket_9"/>
92109

93110
12. Add a bucket policy, example below:
94111
```json

docs/_snippets/_add_remote_ip_access_list_detail.md

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,15 @@
1+
import ip_allow_list_check_list from '@site/static/images/_snippets/ip-allow-list-check-list.png';
2+
import ip_allow_list_add_current_ip from '@site/static/images/_snippets/ip-allow-list-add-current-ip.png';
3+
14
<details>
25
<summary>Manage your IP Access List</summary>
36

47
From your ClickHouse Cloud services list choose the service that you will work with and switch to **Settings**. If the IP Access List does not contain the IP Address or range of the remote system that needs to connect to your ClickHouse Cloud service, then you can resolve the problem with **Add IPs**:
58

6-
![Check to see if the service allows traffic](@site/docs/_snippets/images/ip-allow-list-check-list.png)
9+
<img src={ip_allow_list_check_list} class="image" alt="Check to see if the service allows traffic" />
710

811
Add the individual IP Address, or the range of addresses that need to connect to your ClickHouse Cloud service. Modify the form as you see fit and then **Save**.
912

10-
![Add your current IP address](@site/docs/_snippets/images/ip-allow-list-add-current-ip.png)
13+
<img src={ip_allow_list_add_current_ip} class="image" alt="Add your current IP address" />
1114

1215
</details>
13-

docs/_snippets/_clickhouse_mysql_cloud_setup.mdx

Lines changed: 14 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,22 @@
1+
import mysql_1 from '@site/static/images/_snippets/mysql1.png';
2+
import mysql_2 from '@site/static/images/_snippets/mysql2.png';
3+
import mysql_3 from '@site/static/images/_snippets/mysql3.png';
4+
import mysql_4 from '@site/static/images/_snippets/mysql4.png';
5+
import mysql_5 from '@site/static/images/_snippets/mysql5.png';
6+
17
<br/>
28
1. After creating your ClickHouse Cloud Service, on the `Connect your app` screen, select MySQL from the drop down.
39
<br/>
410

511
<div class="eighty-percent">
6-
![Credentials screen - Prompt](./images/mysql1.png)
12+
<img src={mysql_1} class="image" alt="Credentials screen - Prompt" />
713
</div>
814

9-
2. Toggle the switch to enable the MySQL interface for this specific service. This will expose port `3306` for this service and prompt you with your MySQL connection screen that include your unique MySQL username.
1015

11-
<br/>
16+
2. Toggle the switch to enable the MySQL interface for this specific service. This will expose port `3306` for this service and prompt you with your MySQL connection screen that include your unique MySQL username.
17+
1218
<div class="eighty-percent">
13-
![Credentials screen - Enabled MySQL](./images/mysql2.png)
19+
<img src={mysql_2} class="image" alt="Credentials screen - Enabled MySQL" />
1420
</div>
1521
<br/>
1622

@@ -20,20 +26,20 @@ Alternatively, in order to enable the MySQL interface for an existing service:
2026

2127
<br/>
2228
<div class="eighty-percent">
23-
![Connection screen - Prompt MySQL](./images/mysql3.png)
29+
<img src={mysql_3} class="image" alt="Connection screen - Prompt MySQL" />
2430
</div>
2531
<br/>
2632

2733

2834
4. Select MySQL from the `Connect With` drop down.
2935

3036
<br/>
31-
![Connection screen - Prompt MySQL](./images/mysql4.png)
37+
<img src={mysql_4} class="image" alt="Connection screen - Prompt MySQL" />
3238
<br/>
3339

34-
5. Toggle the switch to enable the MySQL interface for this specific service. This will expose port `3306` for this service and prompt you with your MySQL connection screen that include your unique MySQL username.
40+
5. Toggle the switch to enable the MySQL interface for this specific service. This will expose port `3306` for this service and prompt you with your MySQL connection screen that include your unique MySQL username.
3541

36-
![Connection screen - MySQL Enabled](./images/mysql5.png)
42+
<img src={mysql_5} class="image" alt="Connection screen - MySQL Enabled" />
3743

3844
## Creating multiple MySQL users in ClickHouse Cloud {#creating-multiple-mysql-users-in-clickhouse-cloud}
3945

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
1+
import cloud_connect_button from '@site/static/images/_snippets/cloud-connect-button.png';
2+
import connection_details_https from '@site/static/images/_snippets/connection-details-https.png';
3+
14
To connect to ClickHouse with HTTP(S) you need this information:
25

36
- The HOST and PORT: typically, the port is 8443 when using TLS or 8123 when not using TLS.
@@ -8,10 +11,10 @@ To connect to ClickHouse with HTTP(S) you need this information:
811

912
The details for your ClickHouse Cloud service are available in the ClickHouse Cloud console. Select the service that you will connect to and click **Connect**:
1013

11-
![ClickHouse Cloud service connect button](@site/docs/_snippets/images/cloud-connect-button.png)
14+
<img src={cloud_connect_button} class="image" alt="ClickHouse Cloud service connect button" />
1215

13-
Choose **HTTPS**, and the details are available in an example `curl` command.
16+
Choose **HTTPS**, and the details are available in an example `curl` command.
1417

15-
![ClickHouse Cloud HTTPS connection details](@site/docs/_snippets/images/connection-details-https.png)
18+
<img src={connection_details_https} class="image" alt="ClickHouse Cloud HTTPS connection details" />
1619

1720
If you are using self-managed ClickHouse, the connection details are set by your ClickHouse administrator.
Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
1+
import cloud_connect_button from '@site/static/images/_snippets/cloud-connect-button.png';
2+
import connection_details_native from '@site/static/images/_snippets/connection-details-native.png';
3+
14
To connect to ClickHouse with native TCP you need this information:
25

36
- The HOST and PORT: typically, the port is 9440 when using TLS, or 9000 when not using TLS.
@@ -8,10 +11,10 @@ To connect to ClickHouse with native TCP you need this information:
811

912
The details for your ClickHouse Cloud service are available in the ClickHouse Cloud console. Select the service that you will connect to and click **Connect**:
1013

11-
![ClickHouse Cloud service connect button](@site/docs/_snippets/images/cloud-connect-button.png)
14+
<img src={cloud_connect_button} class="image" alt="ClickHouse Cloud service connect button" />
1215

13-
Choose **Native**, and the details are available in an example `clickhouse-client` command.
16+
Choose **Native**, and the details are available in an example `clickhouse-client` command.
1417

15-
![ClickHouse Cloud Native TCP connection details](@site/docs/_snippets/images/connection-details-native.png)
18+
<img src={connection_details_native} class="image" alt="ClickHouse Cloud Native TCP connection details" />
1619

1720
If you are using self-managed ClickHouse, the connection details are set by your ClickHouse administrator.
Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,19 @@
1+
import cloud_connect_to_sql_console from '@site/static/images/_snippets/cloud-connect-to-sql-console.png';
2+
import createservice8 from '@site/static/images/_snippets/createservice8.png';
3+
14
:::tip SQL console
25
If you need a SQL client connection, your ClickHouse Cloud service has an associated web based SQL console; expand **Connect to SQL console** below for details.
36
:::
47

58
<details>
69
<summary>Connect to SQL console</summary>
710

8-
From your ClickHouse Cloud services list, click on a service.
11+
From your ClickHouse Cloud services list, click on a service.
912

10-
![Connect to SQL Console](@site/docs/_snippets/images/cloud-connect-to-sql-console.png)
13+
<img src={cloud_connect_to_sql_console} class="image" alt="Connect to SQL Console" />
1114

1215
This will redirect you to the SQL console.
1316

14-
![SQL Console](@site/docs/_snippets/images/createservice8.png)
17+
<img src={createservice8} class="image" alt="SQL Console" />
1518

1619
</details>
Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,5 @@
1+
import cloud_service_action_menu from '@site/static/images/_snippets/cloud-service-actions-menu.png';
2+
13
Select your service, followed by `Data souces` -> `Predefined sample data`.
24

3-
![Cloud service Actions menu](@site/docs/_snippets/images/cloud-service-actions-menu.png)
5+
<img src={cloud_service_action_menu} class="image" alt="Cloud service Actions menu" />

docs/chdb/guides/index.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,3 +13,12 @@ from YAML frontmatter fields title, slug, description. If you've found an error
1313
in the table of contents, please edit the frontmatter of the files directly.
1414
-->
1515

16+
| Page | Description |
17+
|-----|-----|
18+
| [How to query Parquet files](/docs/chdb/guides/querying-parquet) | Learn how to query Parquet files with chDB. |
19+
| [How to query data in an S3 bucket](/docs/chdb/guides/querying-s3) | Learn how to query data in an S3 bucket with chDB. |
20+
| [Using a clickhouse-local database](/docs/chdb/guides/clickhouse-local) | Learn how to use a clickhouse-local database with chDB |
21+
| [How to query Pandas DataFrames with chDB](/docs/chdb/guides/pandas) | Learn how to query Pandas DataFrames with chDB |
22+
| [JupySQL and chDB](/docs/chdb/guides/jupysql) | How to install chDB for Bun |
23+
| [How to query a remote ClickHouse server](/docs/chdb/guides/query-remote-clickhouse) | In this guide, we'll learn how to query a remote ClickHouse server from chDB. |
24+
| [How to query Apache Arrow with chDB](/docs/chdb/guides/apache-arrow) | In this guide, we'll learn how to query Apache Arrow tables with chDB |

0 commit comments

Comments
 (0)