Skip to content

Commit c6a7a39

Browse files
author
Robert Fekete
committed
Documents worker-partition-buckets()
And adds it to every destination that has worker-partition-key()
1 parent 1e64efb commit c6a7a39

File tree

10 files changed

+31
-0
lines changed

10 files changed

+31
-0
lines changed

content/chapter-destinations/clickhouse/_index.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -309,6 +309,8 @@ By default, sending data to ClickHouse doesn't propagate the type information of
309309
310310
*Description:* The username used for authentication.
311311
312+
{{< include-headless "chunk/option-destination-worker-partition-buckets.md" >}}
313+
312314
<a id="worker-partition-key"></a>
313315
{{< include-headless "chunk/option-destination-http-worker-partition-key.md" >}}
314316

content/chapter-destinations/configuring-destinations-http-nonjava/reference-destination-http-nonjava/_index.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -422,6 +422,8 @@ In case the server on the specified URL returns a redirect request, {{% param "p
422422

423423
{{% include-headless "chunk/option-destination-http-use-system-cert-store.md" %}}
424424

425+
{{< include-headless "chunk/option-destination-worker-partition-buckets.md" >}}
426+
425427
<a id="worker-partition-key"></a>
426428
{{< include-headless "chunk/option-destination-http-worker-partition-key.md" >}}
427429

content/chapter-destinations/destination-loki/_index.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -167,6 +167,8 @@ loki(
167167

168168
*Description:* The URL of the Loki endpoint, including the gRPC listen port of your Loki deployment.
169169

170+
{{< include-headless "chunk/option-destination-worker-partition-buckets.md" >}}
171+
170172
<a id="worker-partition-key"></a>
171173
{{< include-headless "chunk/option-destination-http-worker-partition-key.md" >}}
172174

content/chapter-destinations/destination-syslog-ng-otlp/_index.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -72,6 +72,8 @@ The `syslog-ng-otlp()` destination has the following options.
7272

7373
*Description:* The URL of the {{% param "product.abbrev" %}} receiver.
7474

75+
{{< include-headless "chunk/option-destination-worker-partition-buckets.md" >}}
76+
7577
<a id="worker-partition-key"></a>
7678
{{< include-headless "chunk/option-destination-http-worker-partition-key.md" >}}
7779

content/chapter-destinations/google-bigquery/_index.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -193,6 +193,8 @@ Alternatively, you can set the schema with the [`protobuf-schema()`](#protobuf-s
193193

194194
*Description:* The URL of the Google BigQuery where the logs are sent.
195195

196+
{{< include-headless "chunk/option-destination-worker-partition-buckets.md" >}}
197+
196198
{{< include-headless "chunk/option-destination-http-worker-partition-key.md" >}}
197199

198200
{{< include-headless "chunk/option-destination-threaded-workers.md" >}}

content/chapter-destinations/google-pubsub-grpc/_index.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -176,6 +176,8 @@ auth(
176176

177177
*Description:* An alias for [`service_endpoint`](#service-endpoint).
178178

179+
{{< include-headless "chunk/option-destination-worker-partition-buckets.md" >}}
180+
179181
<a id="worker-partition-key"></a>
180182
{{< include-headless "chunk/option-destination-http-worker-partition-key.md" >}}
181183

content/chapter-destinations/opentelemetry/_index.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -111,6 +111,8 @@ log non_otel_to_otel_tls {
111111

112112
*Description:* The URL of the OpenTelemetry receiver.
113113

114+
{{< include-headless "chunk/option-destination-worker-partition-buckets.md" >}}
115+
114116
<a id="worker-partition-key"></a>
115117
{{< include-headless "chunk/option-destination-http-worker-partition-key.md" >}}
116118

content/chapter-nonsequential-processing/_index.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,3 +57,7 @@ log {
5757
```
5858
5959
Staring with {{< product >}} version 4.17, you can use the `batch-size()` option to specify how many consecutive messages should be processed by a single `parallelize()` worker. This ensures that this many messages preserve their order on the destination side, and also improves `parallelize()` performance. A value around 100 is recommended for `batch-size()`. Default value: `0` (batching is disabled).
60+
61+
<!-- FIXME mention/explain
62+
{{< include-headless "chunk/option-destination-worker-partition-buckets.md" >}}
63+
-->
Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
---
2+
---
3+
<!-- This file is under the copyright of Axoflow, and licensed under Apache License 2.0, except for using the Axoflow and AxoSyslog trademarks. -->
4+
## worker-partition-buckets()
5+
6+
| | |
7+
| -------- | ------ |
8+
| Type: | template |
9+
| Default: | |
10+
11+
*Description:* The `worker-partition-buckets()` option determines the number of worker threads used for the `worker-partition-key()`. Note that the number set by `worker-partition-buckets()` should be lower than the number of `workers()`.

content/whats-new/_index.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,8 @@ weight: 10
1515
- You can now set a suffix for your S3 objects using the [`object-key-suffix()` option]({{< relref "/chapter-destinations/destination-s3/_index.md#object-key-suffix" >}}).
1616
- You can change the default checksum settings for S3 compatible solutions that don't support checksums using the [`use-checksum()` option]({{< relref "/chapter-destinations/destination-s3/_index.md#use-checksum" >}}).
1717

18+
- The `http()` and other threaded destinations now have a [`worker-partition-buckets()` option]({{< relref "/chapter-destinations/configuring-destinations-http-nonjava/reference-destination-http-nonjava/_index.md#worker-partition-buckets" >}}) that determines the number of worker threads used for the `worker-partition-key()`.
19+
1820
## Version 4.19 (2025-10-15)
1921

2022
- The [`dict_to_pairs`]({{< relref "/filterx/function-reference.md#dict-to-pairs" >}}) FilterX function can convert a dictionary to a list of pairs.

0 commit comments

Comments
 (0)