Skip to content

Commit 1c48f18

Browse files
committed
use external file for code snippets
1 parent 620ae80 commit 1c48f18

File tree

1 file changed

+43
-1
lines changed
  • docs/use-cases/observability/clickstack/ingesting-data

1 file changed

+43
-1
lines changed

docs/use-cases/observability/clickstack/ingesting-data/collector.md

Lines changed: 43 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -162,7 +162,49 @@ The following configuration shows collection of this [unstructured log file](htt
162162
Note the use of operators to extract structure from the log lines (`regex_parser`) and filter events, along with a processor to batch events and limit memory usage.
163163

164164
```yaml file=code_snippets/ClickStack/config-unstructured-logs-with-processor.yaml
165-
place holder text
165+
receivers:
166+
filelog:
167+
include:
168+
- /opt/data/logs/access-unstructured.log
169+
start_at: beginning
170+
operators:
171+
- type: regex_parser
172+
regex: '^(?P<ip>[\d.]+)\s+-\s+-\s+\[(?P<timestamp>[^\]]+)\]\s+"(?P<method>[A-Z]+)\s+(?P<url>[^\s]+)\s+HTTP/[^\s]+"\s+(?P<status>\d+)\s+(?P<size>\d+)\s+"(?P<referrer>[^"]*)"\s+"(?P<user_agent>[^"]*)"'
173+
timestamp:
174+
parse_from: attributes.timestamp
175+
layout: '%d/%b/%Y:%H:%M:%S %z'
176+
#22/Jan/2019:03:56:14 +0330
177+
processors:
178+
batch:
179+
timeout: 1s
180+
send_batch_size: 100
181+
memory_limiter:
182+
check_interval: 1s
183+
limit_mib: 2048
184+
spike_limit_mib: 256
185+
exporters:
186+
# HTTP setup
187+
otlphttp/hdx:
188+
endpoint: 'http://localhost:4318'
189+
headers:
190+
authorization: <YOUR_INGESTION_API_KEY>
191+
compression: gzip
192+
193+
# gRPC setup (alternative)
194+
otlp/hdx:
195+
endpoint: 'localhost:4317'
196+
headers:
197+
authorization: <YOUR_API_INGESTION_KEY>
198+
compression: gzip
199+
service:
200+
telemetry:
201+
metrics:
202+
address: 0.0.0.0:9888 # Modified as 2 collectors running on same host
203+
pipelines:
204+
logs:
205+
receivers: [filelog]
206+
processors: [batch]
207+
exporters: [otlphttp/hdx]
166208
```
167209

168210
Note the need to include an [authorization header containing your ingestion API key](#securing-the-collector) in any OTLP communication.

0 commit comments

Comments
 (0)