You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
description: 'Describes how to use asynchronous inserts into ClickHouse as an alternative best practice to batching'
6
6
---
7
7
8
+
import Image from '@theme/IdealImage';
8
9
import asyncInsert01 from '@site/static/images/cloud/bestpractices/async-01.png';
9
10
import asyncInsert02 from '@site/static/images/cloud/bestpractices/async-02.png';
10
11
import asyncInsert03 from '@site/static/images/cloud/bestpractices/async-03.png';
11
-
import Image from '@theme/IdealImage';
12
12
13
13
Inserting data into ClickHouse in large batches is a best practice. It saves compute cycles and disk I/O, and therefore it saves money. If your use case allows you to batch your inserts external to ClickHouse, then that is one option. If you would like ClickHouse to create the batches, then you can use the asynchronous INSERT mode described here.
Copy file name to clipboardExpand all lines: docs/cloud/bestpractices/partitioningkey.md
+2Lines changed: 2 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,6 +9,8 @@ import Image from '@theme/IdealImage';
9
9
import partitioning01 from '@site/static/images/cloud/bestpractices/partitioning-01.png';
10
10
import partitioning02 from '@site/static/images/cloud/bestpractices/partitioning-02.png';
11
11
12
+
# Choose a Low Cardinality Partitioning Key
13
+
12
14
When you send an insert statement (that should contain many rows - see [section above](/optimize/bulk-inserts)) to a table in ClickHouse Cloud, and that
13
15
table is not using a [partitioning key](/engines/table-engines/mergetree-family/custom-partitioning-key.md) then all row data from that insert is written into a new part on storage:
Once you are logged in, ClickHouse Cloud starts the onboarding wizard which walks you through creating a new ClickHouse service. You will initially be requested to [select a plan](/cloud/manage/cloud-tiers):
By default, the scale tier will create 3 replicas each with 4 VCPUs and 16 GiB RAM. [Vertical autoscaling](/manage/scaling#vertical-auto-scaling) will be enabled by default in the Scale tier.
67
62
68
63
Users can customize the service resources if required, specifying a minimum and maximum size for replicas to scale between. When ready, select `Create service`.
Congratulations! Your ClickHouse Cloud service is up and running and onboarding is complete. Keep reading for details on how to start ingesting and querying your data.
@@ -83,9 +76,7 @@ There are 2 ways to connect to ClickHouse:
83
76
84
77
For getting started quickly, ClickHouse provides a web-based SQL console to which you will be redirected on completing onboarding.
@@ -110,20 +99,16 @@ That's it - you are ready to start using your new ClickHouse service!
110
99
111
100
Press the connect button from the navigation menu. A modal will open offering the credentials to your service and offering you a set of instructions on how to connect with your interface or language clients.
If you can’t see your language client, you may want to check our list of [Integrations](/integrations).
105
+
If you can't see your language client, you may want to check our list of [Integrations](/integrations).
119
106
120
107
## 3. Add data {#3-add-data}
121
108
122
109
ClickHouse is better with data! There are multiple ways to add data and most of them are available on the Data Sources page, which can be accessed in the navigation menu.
@@ -137,9 +122,7 @@ You can upload data using the following methods:
137
122
138
123
[ClickPipes](http://clickhouse.com/docs/integrations/clickpipes) is a managed integration platform that makes ingesting data from a diverse set of sources as simple as clicking a few buttons. Designed for the most demanding workloads, ClickPipes's robust and scalable architecture ensures consistent performance and reliability. ClickPipes can be used for long-term streaming needs or one-time data loading job.
139
124
140
-
<divclass="eighty-percent">
141
-
<img src={select_data_source} class="image" alt="Select data source" />
142
-
</div>
125
+
<Imageimg={select_data_source}size="md"alt='Select data source'border/>
143
126
<br/>
144
127
145
128
### Add data using the SQL Console {#add-data-using-the-sql-console}
@@ -202,7 +185,6 @@ For a deep dive into core ClickHouse concepts, see ["Core Concepts"](../../manag
202
185
203
186
#### Insert data into your table {#insert-data-into-your-table}
204
187
205
-
206
188
You can use the familiar [`INSERT INTO TABLE`](../../sql-reference/statements/insert-into.md) command with ClickHouse, but it is important to understand that each insert into a [`MergeTree`](/engines/table-engines/mergetree-family/mergetree.md) table causes a **part** to be created in storage.
207
189
208
190
:::tip ClickHouse best practice
@@ -236,9 +218,7 @@ SELECT * FROM helloworld.my_first_table
236
218
237
219
You can also connect to your ClickHouse Cloud service using a command-line tool named [**clickhouse client**](/interfaces/cli). Click `Connect` on the left menu to access these details. From the dialog select `Native` from the drop-down:
import endpoints_testquery from '@site/static/images/cloud/sqlconsole/endpoints-testquery.png';
10
11
import endpoints_savequery from '@site/static/images/cloud/sqlconsole/endpoints-savequery.png';
11
12
import endpoints_configure from '@site/static/images/cloud/sqlconsole/endpoints-configure.png';
@@ -46,23 +47,23 @@ limit 10
46
47
47
48
Note that this query contains a parameter (`year`). The SQL console query editor automatically detects ClickHouse query parameter expressions and provides an input for each parameter. Let's quickly run this query to make sure that it works:
48
49
49
-
<imgsrc={endpoints_testquery}alt="Test the example query"/>
50
+
<Imageimg={endpoints_testquery}size="md"alt="Test the example query"/>
50
51
51
52
Next step, we'll go ahead and save the query:
52
53
53
-
<imgsrc={endpoints_savequery}alt="Save example query"/>
54
+
<Imageimg={endpoints_savequery}size="md"alt="Save example query"/>
54
55
55
56
More documentation around saved queries can be found [here](/cloud/get-started/sql-console#saving-a-query).
56
57
57
58
### Configuring the Query API Endpoint {#configuring-the-query-api-endpoint}
58
59
59
60
Query API endpoints can be configured directly from query view by clicking the **Share** button and selecting `API Endpoint`. You'll be prompted to specify which API key(s) should be able to access the endpoint:
After selecting an API key, the query API endpoint will automatically be provisioned. An example `curl` command will be displayed so you can send a test request:
After you've sent your first request, a new button should appear immediately to the right of the **Share** button. Clicking it will open a flyout containing monitoring data about the query:
import insights_overview from '@site/static/images/cloud/sqlconsole/insights_overview.png';
10
11
import insights_latency from '@site/static/images/cloud/sqlconsole/insights_latency.png';
11
12
import insights_recent from '@site/static/images/cloud/sqlconsole/insights_recent.png';
@@ -20,34 +21,31 @@ The **Query Insights** feature makes ClickHouse's built-in query log easier to u
20
21
21
22
After selecting a service, the **Monitoring** navigation item in the left sidebar should expand to reveal a new **Query insights** sub-item. Clicking on this option opens the new Query insights page:
The stat boxes at the top represent some basic top-level query metrics over the selected period of time. Beneath it, we’ve exposed three time-series charts representing query volume, latency, and error rate broken down by query kind (select, insert, other) over a selected time window. The latency chart can be further adjusted to display p50, p90, and p99 latencies:
28
+
The stat boxes at the top represent some basic top-level query metrics over the selected period of time. Beneath it, we've exposed three time-series charts representing query volume, latency, and error rate broken down by query kind (select, insert, other) over a selected time window. The latency chart can be further adjusted to display p50, p90, and p99 latencies:
Recent queries can be filtered and sorted by any available field. The table can also be configured to display or hide additional fields such as tables, p90, and p99 latencies.
38
39
39
40
## Query drill-down {#query-drill-down}
40
41
41
42
Selecting a query from the recent queries table will open a flyout containing metrics and information specific to the selected query:
As we can see from the flyout, this particular query has been run more than 3000 times in the last 24 hours. All metrics in the **Query info** tab are aggregated metrics, but we can also view metrics from individual runs by selecting the **Query history** tab:
0 commit comments