You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
## Community monitoring solutions {#community-monitoring}
2
+
3
+
The ClickHouse community has developed comprehensive monitoring solutions that integrate with popular observability stacks. [ClickHouse Monitoring](https://github.com/duyet/clickhouse-monitoring) provides a complete monitoring setup with pre-built dashboards. This open source project offers a quick-start approach for teams looking to implement ClickHouse monitoring with established best practices and proven dashboard configurations.
4
+
5
+
:::note
6
+
Like other direct database monitoring approaches, this solution queries ClickHouse system tables directly, which prevents instances from idling and impacts cost optimization.
import AdvancedDashboard from '@site/static/images/cloud/manage/monitoring/advanced_dashboard.png';
3
+
import NativeAdvancedDashboard from '@site/static/images/cloud/manage/monitoring/native_advanced_dashboard.png';
4
+
5
+
### Direct Grafana plugin integration {#direct-grafana}
6
+
7
+
The ClickHouse data source plugin for Grafana enables visualization and exploration of data directly from ClickHouse using system tables. This approach works well for monitoring performance and creating custom dashboards for detailed system analysis.
8
+
For plugin installation and configuration details, see the ClickHouse [data source plugin](/integrations/grafana). For a complete monitoring setup using the Prometheus-Grafana mix-in with pre-built dashboards and alerting rules, see [Monitor ClickHouse with the new Prometheus-Grafana mix-in](https://clickhouse.com/blog/monitor-with-new-prometheus-grafana-mix-in).
9
+
10
+
### Direct Datadog Integration {#direct-datadog}
11
+
12
+
Datadog offers a Clickhouse Monitoring plugin for its agent which queries system tables directly. This integration provides comprehensive database monitoring with cluster awareness through clusterAllReplicas functionality.
13
+
:::note
14
+
This integration is not recommended for ClickHouse Cloud deployments due to incompatibility with cost-optimizing idle behavior and operational limitations of the cloud proxy layer.
15
+
:::
16
+
17
+
### Using system tables directly {#system-tables}
18
+
19
+
Users can perform deep query performance analysis by connecting to ClickHouse system tables, particularly `system.query_log` and querying directly. Using either the SQL console or clickhouse client, teams can identify slow queries, analyze resource usage, and track usage patterns across the organization.
20
+
21
+
**Query Performance Analysis**
22
+
23
+
Users can use the system table query logs to perform Query Performance Analysis.
24
+
25
+
**Example query**: Find the top 5 long-running queries across all cluster replicas:
26
+
27
+
```sql
28
+
SELECT
29
+
type,
30
+
event_time,
31
+
query_duration_ms,
32
+
query,
33
+
read_rows,
34
+
tables
35
+
FROM clusterAllReplicas(default, system.query_log)
36
+
WHERE event_time >= (now() - toIntervalMinute(60)) AND type='QueryFinish'
import AdvancedDashboard from '@site/static/images/cloud/manage/monitoring/advanced_dashboard.png';
3
+
import NativeAdvancedDashboard from '@site/static/images/cloud/manage/monitoring/native_advanced_dashboard.png';
4
+
5
+
## Integration examples {#examples}
6
+
7
+
External integration allows organizations to maintain established monitoring workflows, leverage existing team expertise with familiar tools, and integrate ClickHouse monitoring with broader infrastructure observability without disrupting current processes or requiring significant retraining investments.
8
+
Teams can apply existing alerting rules and escalation procedures to ClickHouse metrics, while correlating database performance with application and infrastructure health within a unified observability platform. This approach maximizes ROI on current monitoring setups and enables faster troubleshooting through consolidated dashboards and familiar tooling interfaces.
9
+
10
+
### Grafana Cloud monitoring {#grafana}
11
+
12
+
Grafana provides ClickHouse monitoring through both direct plugin integration and Prometheus-based approaches. The Prometheus endpoint integration maintains operational separation between monitoring and production workloads while enabling visualization within existing Grafana Cloud infrastructure. See [Grafana's ClickHouse documentation](https://grafana.com/docs/grafana-cloud/monitor-infrastructure/integrations/integration-reference/integration-clickhouse/) for configuration guidance.
13
+
14
+
### Datadog monitoring {#datadog}
15
+
Datadog is developing a dedicated API integration that will provide proper cloud service monitoring while respecting service idling behavior. In the interim, teams can use the OpenMetrics integration approach via ClickHouse Prometheus endpoints for operational separation and cost-efficient monitoring. For configuration guidance, see [Datadog's Prometheus and OpenMetrics integration documentation](https://docs.datadoghq.com/integrations/openmetrics/).
16
+
17
+
### ClickStack {#clickstack}
18
+
19
+
ClickStack is ClickHouse's recommended observability solution for deep system analysis and debugging, providing a unified platform for logs, metrics, and traces using ClickHouse as the storage engine. This approach relies on HyperDX, the ClickStack UI, connecting directly to the system tables inside your ClickHouse instance.
20
+
HyperDX ships with a ClickHouse focused dashboard with tabs for Selects, Inserts, and Infrastructure. Teams can also use Lucene or SQL syntax to search system tables and logs, as well as create custom visualizations via Chart Explorer for detailed system analysis.
21
+
This approach is ideal for debugging complex issues, performance analysis, and deep system introspection rather than real-time production alerting.
22
+
23
+
:::note
24
+
Note that this approach will wake idle services as HyperDX queries the system tables directly.
Copy file name to clipboardExpand all lines: docs/best-practices/using_data_skipping_indices.md
+7-1Lines changed: 7 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -13,7 +13,7 @@ import Image from '@theme/IdealImage';
13
13
import building_skipping_indices from '@site/static/images/bestpractices/building_skipping_indices.gif';
14
14
import using_skipping_indices from '@site/static/images/bestpractices/using_skipping_indices.gif';
15
15
16
-
Data skipping indices should be considered when previous best practices have been followed i.e. types are optimized, a good primary key has been selected and materialized views have been exploited.
16
+
Data skipping indices should be considered when previous best practices have been followed i.e. types are optimized, a good primary key has been selected and materialized views have been exploited. If you're new to skipping indices, [this guide](/optimize/skipping-indexes) is a good place to start.
17
17
18
18
These types of indices can be used to accelerate query performance if used carefully with an understanding of how they work.
19
19
@@ -251,3 +251,9 @@ WHERE (CreationDate > '2009-01-01') AND (ViewCount > 10000000)
251
251
We also show an animation how the minmax skipping index prunes all row blocks that cannot possibly contain matches for the `ViewCount` > 10,000,000 predicate in our example query:
0 commit comments