Skip to content

Commit 3fb00b4

Browse files
gingerwizardgingerwizard
committed
Event patterns for clickstack
1 parent ce23311 commit 3fb00b4

File tree

4 files changed

+188
-0
lines changed

4 files changed

+188
-0
lines changed

‎docs/use-cases/observability/clickstack/deployment/hyperdx-clickhouse-cloud.md‎

Lines changed: 148 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -68,6 +68,154 @@ curl -O https://raw.githubusercontent.com/ClickHouse/clickhouse-docs/refs/heads/
6868
<details>
6969
<summary>otel-cloud-config.yaml</summary>
7070
```yaml file=docs/use-cases/observability/clickstack/deployment/_snippets/otel-cloud-config.yaml
71+
receivers:
72+
otlp/hyperdx:
73+
protocols:
74+
grpc:
75+
include_metadata: true
76+
endpoint: '0.0.0.0:4317'
77+
http:
78+
cors:
79+
allowed_origins: ['*']
80+
allowed_headers: ['*']
81+
include_metadata: true
82+
endpoint: '0.0.0.0:4318'
83+
processors:
84+
transform:
85+
log_statements:
86+
- context: log
87+
error_mode: ignore
88+
statements:
89+
# JSON parsing: Extends log attributes with the fields from structured log body content, either as an OTEL map or
90+
# as a string containing JSON content.
91+
- set(log.cache, ExtractPatterns(log.body, "(?P<0>(\\{.*\\}))")) where
92+
IsString(log.body)
93+
- merge_maps(log.attributes, ParseJSON(log.cache["0"]), "upsert")
94+
where IsMap(log.cache)
95+
- flatten(log.attributes) where IsMap(log.cache)
96+
- merge_maps(log.attributes, log.body, "upsert") where IsMap(log.body)
97+
- context: log
98+
error_mode: ignore
99+
conditions:
100+
- severity_number == 0 and severity_text == ""
101+
statements:
102+
# Infer: extract the first log level keyword from the first 256 characters of the body
103+
- set(log.cache["substr"], log.body.string) where Len(log.body.string)
104+
< 256
105+
- set(log.cache["substr"], Substring(log.body.string, 0, 256)) where
106+
Len(log.body.string) >= 256
107+
- set(log.cache, ExtractPatterns(log.cache["substr"],
108+
"(?i)(?P<0>(alert|crit|emerg|fatal|error|err|warn|notice|debug|dbug|trace))"))
109+
# Infer: detect FATAL
110+
- set(log.severity_number, SEVERITY_NUMBER_FATAL) where
111+
IsMatch(log.cache["0"], "(?i)(alert|crit|emerg|fatal)")
112+
- set(log.severity_text, "fatal") where log.severity_number ==
113+
SEVERITY_NUMBER_FATAL
114+
# Infer: detect ERROR
115+
- set(log.severity_number, SEVERITY_NUMBER_ERROR) where
116+
IsMatch(log.cache["0"], "(?i)(error|err)")
117+
- set(log.severity_text, "error") where log.severity_number ==
118+
SEVERITY_NUMBER_ERROR
119+
# Infer: detect WARN
120+
- set(log.severity_number, SEVERITY_NUMBER_WARN) where
121+
IsMatch(log.cache["0"], "(?i)(warn|notice)")
122+
- set(log.severity_text, "warn") where log.severity_number ==
123+
SEVERITY_NUMBER_WARN
124+
# Infer: detect DEBUG
125+
- set(log.severity_number, SEVERITY_NUMBER_DEBUG) where
126+
IsMatch(log.cache["0"], "(?i)(debug|dbug)")
127+
- set(log.severity_text, "debug") where log.severity_number ==
128+
SEVERITY_NUMBER_DEBUG
129+
# Infer: detect TRACE
130+
- set(log.severity_number, SEVERITY_NUMBER_TRACE) where
131+
IsMatch(log.cache["0"], "(?i)(trace)")
132+
- set(log.severity_text, "trace") where log.severity_number ==
133+
SEVERITY_NUMBER_TRACE
134+
# Infer: else
135+
- set(log.severity_text, "info") where log.severity_number == 0
136+
- set(log.severity_number, SEVERITY_NUMBER_INFO) where log.severity_number == 0
137+
- context: log
138+
error_mode: ignore
139+
statements:
140+
# Normalize the severity_text case
141+
- set(log.severity_text, ConvertCase(log.severity_text, "lower"))
142+
resourcedetection:
143+
detectors:
144+
- env
145+
- system
146+
- docker
147+
timeout: 5s
148+
override: false
149+
batch:
150+
memory_limiter:
151+
# 80% of maximum memory up to 2G, adjust for low memory environments
152+
limit_mib: 1500
153+
# 25% of limit up to 2G, adjust for low memory environments
154+
spike_limit_mib: 512
155+
check_interval: 5s
156+
connectors:
157+
routing/logs:
158+
default_pipelines: [logs/out-default]
159+
error_mode: ignore
160+
table:
161+
- context: log
162+
statement: route() where IsMatch(attributes["rr-web.event"], ".*")
163+
pipelines: [logs/out-rrweb]
164+
exporters:
165+
debug:
166+
verbosity: detailed
167+
sampling_initial: 5
168+
sampling_thereafter: 200
169+
clickhouse/rrweb:
170+
database: ${env:CLICKHOUSE_DATABASE}
171+
endpoint: ${env:CLICKHOUSE_ENDPOINT}
172+
password: ${env:CLICKHOUSE_PASSWORD}
173+
username: ${env:CLICKHOUSE_USER}
174+
ttl: 720h
175+
logs_table_name: hyperdx_sessions
176+
timeout: 5s
177+
retry_on_failure:
178+
enabled: true
179+
initial_interval: 5s
180+
max_interval: 30s
181+
max_elapsed_time: 300s
182+
clickhouse:
183+
database: ${env:CLICKHOUSE_DATABASE}
184+
endpoint: ${env:CLICKHOUSE_ENDPOINT}
185+
password: ${env:CLICKHOUSE_PASSWORD}
186+
username: ${env:CLICKHOUSE_USER}
187+
ttl: 720h
188+
timeout: 5s
189+
retry_on_failure:
190+
enabled: true
191+
initial_interval: 5s
192+
max_interval: 30s
193+
max_elapsed_time: 300s
194+
extensions:
195+
health_check:
196+
endpoint: :13133
197+
service:
198+
pipelines:
199+
traces:
200+
receivers: [otlp/hyperdx]
201+
processors: [memory_limiter, batch]
202+
exporters: [clickhouse]
203+
metrics:
204+
receivers: [otlp/hyperdx]
205+
processors: [memory_limiter, batch]
206+
exporters: [clickhouse]
207+
logs/in:
208+
receivers: [otlp/hyperdx]
209+
exporters: [routing/logs]
210+
logs/out-default:
211+
receivers: [routing/logs]
212+
processors: [memory_limiter, transform, batch]
213+
exporters: [clickhouse]
214+
logs/out-rrweb:
215+
receivers: [routing/logs]
216+
processors: [memory_limiter, batch]
217+
exporters: [clickhouse/rrweb]
218+
71219
```
72220
</details>
73221

‎docs/use-cases/observability/clickstack/event_patterns.md‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
---
2+
slug: /use-cases/observability/clickstack/event_patterns
3+
title: 'Event Patterns with ClickStack'
4+
sidebar_label: 'Event Patterns'
5+
pagination_prev: null
6+
pagination_next: null
7+
description: 'Event Patterns with ClickStack'
8+
---
9+
10+
import Image from '@theme/IdealImage';
11+
import event_patterns from '@site/static/images/use-cases/observability/event_patterns.png';
12+
import event_patterns_highlight from '@site/static/images/use-cases/observability/event_patterns_highlight.png';
13+
14+
Event patterns in ClickStack allow you to quickly make sense of large volumes of logs or traces by automatically clustering similar messages together, so instead of digging through millions of individual events, you only need to review a small number of meaningful groups.
15+
16+
<Image img={event_patterns} alt="Event patterns" size="lg"/>
17+
18+
This makes it much easier to spot which errors or warnings are new, which are recurring, and which are driving sudden spikes in log volume. Because the patterns are generated dynamically, you don't need to define regular expressions or maintain parsing rules - ClickStack adapts to your events automatically, regardless of format.
19+
20+
Beyond incident response, this high-level view also helps you identify noisy log sources that can be trimmed to reduce cost, discover the different types of logs a service produces, and more quickly answer whether the system is already emitting the signals you care about.
21+
22+
23+
## Accessing event patterns {#accessing-event-patterns}
24+
25+
Event patterns are available directly through the **Search** panel in ClickStack.
26+
27+
From the top-left **Analysis Mode** selector, choose **Event Patterns** to switch from the standard results table to a clustered view of similar events.
28+
29+
<Image img={event_patterns_highlight} alt="Event patterns" size="lg"/>
30+
31+
32+
This provides an alternative to the default **Results Table** which allows users to scroll through every individual log or trace.
33+
34+
## Recommendations {#recommendations}
35+
36+
Event patterns are most effective when applied to **narrowed subsets** of your data. For example, filtering down to a single service before enabling event patterns will usually surface more relevant and interesting messages than applying patterns across thousands of services at once.
37+
38+
They are also particularly powerful for summarizing error messages, where repeated errors with varying IDs or payloads are grouped into concise clusters.
39+
40+
For a live example, see how event patterns are used in the [Remote Demo Dataset](/use-cases/observability/clickstack/getting-started/remote-demo-data#identify-error-patterns).

‎static/images/use-cases/observability/event_patterns.png‎

1.07 MB
Loading

‎static/images/use-cases/observability/event_patterns_highlight.png‎

1.1 MB
Loading

0 commit comments

Comments
 (0)