Skip to content

Commit 797a840

Browse files
max-cxmax-cx
authored
Merge pull request #101728 from max-cx/OBSDOCS-2517
OBSDOCS-2517: Modularize otel-collector-exporters.adoc
2 parents c553b28 + b1ffd6e commit 797a840

12 files changed

+516
-462
lines changed

‎modules/otel-exporters-aws-cloudwatch-exporter.adoc‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * observability/otel/otel-collector/otel-collector-exporters.adoc
4+
5+
:_mod-docs-content-type: REFERENCE
6+
[id="otel-exporters-aws-cloudwatch-exporter_{context}"]
7+
= AWS CloudWatch Logs Exporter
8+
9+
[role="_abstract"]
10+
The AWS CloudWatch Logs Exporter sends logs data to the Amazon CloudWatch Logs service and signs requests by using the AWS SDK for Go and the default credential provider chain.
11+
12+
:FeatureName: The AWS CloudWatch Logs Exporter
13+
include::snippets/technology-preview.adoc[]
14+
15+
.OpenTelemetry Collector custom resource with the enabled AWS CloudWatch Logs Exporter
16+
[source,yaml]
17+
----
18+
# ...
19+
config:
20+
exporters:
21+
awscloudwatchlogs:
22+
log_group_name: "<group_name_of_amazon_cloudwatch_logs>" # <1>
23+
log_stream_name: "<log_stream_of_amazon_cloudwatch_logs>" # <2>
24+
region: <aws_region_of_log_stream> # <3>
25+
endpoint: <protocol><service_endpoint_of_amazon_cloudwatch_logs> # <4>
26+
log_retention: <supported_value_in_days> # <5>
27+
# ...
28+
----
29+
<1> Required. If the log group does not exist yet, it is automatically created.
30+
<2> Required. If the log stream does not exist yet, it is automatically created.
31+
<3> Optional. If the AWS region is not already set in the default credential chain, you must specify it.
32+
<4> Optional. You can override the default Amazon CloudWatch Logs service endpoint to which the requests are forwarded. You must include the protocol, such as `https://`, as part of the endpoint value. For the list of service endpoints by region, see link:https://docs.aws.amazon.com/general/latest/gr/cwl_region.html[Amazon CloudWatch Logs endpoints and quotas] (AWS General Reference).
33+
<5> Optional. With this parameter, you can set the log retention policy for new Amazon CloudWatch log groups. If this parameter is omitted or set to `0`, the logs never expire by default. Supported values for retention in days are `1`, `3`, `5`, `7`, `14`, `30`, `60`, `90`, `120`, `150`, `180`, `365`, `400`, `545`, `731`, `1827`, `2192`, `2557`, `2922`, `3288`, or `3653`.
34+
35+
[role="_additional-resources"]
36+
.Additional resources
37+
* link:https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/WhatIsCloudWatchLogs.html[Amazon CloudWatch Logs User Guide: What is Amazon CloudWatch Logs?]
38+
* link:https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html#specifying-credentials[AWS SDK for Go Developer Guide: Specifying Credentials]
39+
// * link:https://docs.aws.amazon.com/sdk-for-go/v2/developer-guide/configure-auth.html[AWS SDK for Go v2 Developer Guide: Configure Authentication]
40+
* link:https://docs.aws.amazon.com/general/latest/gr/cwl_region.html[AWS General Reference: Amazon CloudWatch Logs endpoints and quotas]

‎modules/otel-exporters-aws-emf-exporter.adoc‎

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * observability/otel/otel-collector/otel-collector-exporters.adoc
4+
5+
:_mod-docs-content-type: REFERENCE
6+
[id="otel-exporters-aws-emf-exporter_{context}"]
7+
= AWS EMF Exporter
8+
9+
[role="_abstract"]
10+
The AWS EMF Exporter converts the following OpenTelemetry metrics datapoints to the AWS CloudWatch Embedded Metric Format (EMF):
11+
12+
- `Int64DataPoints`
13+
- `DoubleDataPoints`
14+
- `SummaryDataPoints`
15+
16+
The EMF metrics are then sent directly to the Amazon CloudWatch Logs service by using the `PutLogEvents` API.
17+
18+
One of the benefits of using this exporter is the possibility to view logs and metrics in the Amazon CloudWatch console at link:https://console.aws.amazon.com/cloudwatch/[].
19+
20+
:FeatureName: The AWS EMF Exporter
21+
include::snippets/technology-preview.adoc[]
22+
23+
.OpenTelemetry Collector custom resource with the enabled AWS EMF Exporter
24+
[source,yaml]
25+
----
26+
# ...
27+
config:
28+
exporters:
29+
awsemf:
30+
log_group_name: "<group_name_of_amazon_cloudwatch_logs>" # <1>
31+
log_stream_name: "<log_stream_of_amazon_cloudwatch_logs>" # <2>
32+
resource_to_telemetry_conversion: # <3>
33+
enabled: true
34+
region: <region> # <4>
35+
endpoint: <protocol><endpoint> # <5>
36+
log_retention: <supported_value_in_days> # <6>
37+
namespace: <custom_namespace> # <7>
38+
# ...
39+
----
40+
<1> You can use the `log_group_name` parameter to customize the log group name or set the default `+/metrics/default+` value or the following placeholders:
41+
+
42+
The `+/aws/metrics/{ClusterName}+` placeholder is for searching for the `ClusterName` or `+aws.ecs.cluster.name+` resource attribute in the metrics data and replacing it with the actual cluster name.
43+
+
44+
The `+{NodeName}+` placeholder is for searching for the `NodeName` or `+k8s.node.name+` resource attribute.
45+
+
46+
The `+{TaskId}+` placeholder is for searching for the `TaskId` or `+aws.ecs.task.id+` resource attribute.
47+
+
48+
If no resource attribute is found in the resource attribute map, the placeholder is replaced by the `undefined` value.
49+
<2> You can use the `log_stream_name` parameter to customize the log stream name or set the default `+otel-stream+` value or the following placeholders:
50+
+
51+
The `+{ClusterName}+` placeholder is for searching for the `+ClusterName+` or `+aws.ecs.cluster.name+` resource attribute.
52+
+
53+
The `+{ContainerInstanceId}+` placeholder is for searching for the `+ContainerInstanceId+` or `+aws.ecs.container.instance.id+` resource attribute. This resource attribute is valid only for the AWS ECS EC2 launch type.
54+
+
55+
The `+{NodeName}+` placeholder is for searching for the `+NodeName+` or `+k8s.node.name+` resource attribute.
56+
+
57+
The `+{TaskDefinitionFamily}+` placeholder is for searching for the `+TaskDefinitionFamily+` or `+aws.ecs.task.family+` resource attribute.
58+
+
59+
The `+{TaskId}+` placeholder is for searching for the `TaskId` or `+aws.ecs.task.id+` resource attribute in the metrics data and replacing it with the actual task ID.
60+
+
61+
If no resource attribute is found in the resource attribute map, the placeholder is replaced by the `undefined` value.
62+
<3> Optional. Converts resource attributes to telemetry attributes such as metric labels. Disabled by default.
63+
<4> The AWS region of the log stream. If a region is not already set in the default credential provider chain, you must specify the region.
64+
<5> Optional. You can override the default Amazon CloudWatch Logs service endpoint to which the requests are forwarded. You must include the protocol, such as `https://`, as part of the endpoint value. For the list of service endpoints by region, see link:https://docs.aws.amazon.com/general/latest/gr/cwl_region.html[Amazon CloudWatch Logs endpoints and quotas] (AWS General Reference).
65+
<6> Optional. With this parameter, you can set the log retention policy for new Amazon CloudWatch log groups. If this parameter is omitted or set to `0`, the logs never expire by default. Supported values for retention in days are `1`, `3`, `5`, `7`, `14`, `30`, `60`, `90`, `120`, `150`, `180`, `365`, `400`, `545`, `731`, `1827`, `2192`, `2557`, `2922`, `3288`, or `3653`.
66+
<7> Optional. A custom namespace for the Amazon CloudWatch metrics.
67+
68+
[role="_additional-resources"]
69+
.Additional resources
70+
* link:https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Embedded_Metric_Format_Specification.html[Amazon CloudWatch User Guide: Specification: Embedded metric format]
71+
* link:https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_PutLogEvents.html[Amazon CloudWatch Logs API Reference: PutLogEvents]
72+
* link:https://docs.aws.amazon.com/general/latest/gr/cwl_region.html[AWS General Reference: Amazon CloudWatch Logs endpoints and quotas]

‎modules/otel-exporters-aws-xray-exporter.adoc‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * observability/otel/otel-collector/otel-collector-exporters.adoc
4+
5+
:_mod-docs-content-type: REFERENCE
6+
[id="otel-exporters-aws-xray-exporter_{context}"]
7+
= AWS X-Ray Exporter
8+
9+
[role="_abstract"]
10+
The AWS X-Ray Exporter converts OpenTelemetry spans to AWS X-Ray Segment Documents and then sends them directly to the AWS X-Ray service. The AWS X-Ray Exporter uses the `PutTraceSegments` API and signs requests by using the AWS SDK for Go and the default credential provider chain.
11+
12+
:FeatureName: The AWS X-Ray Exporter
13+
include::snippets/technology-preview.adoc[]
14+
15+
.OpenTelemetry Collector custom resource with the enabled AWS X-Ray Exporter
16+
[source,yaml]
17+
----
18+
# ...
19+
config:
20+
exporters:
21+
awsxray:
22+
region: "<region>" # <1>
23+
endpoint: <protocol><endpoint> # <2>
24+
resource_arn: "<aws_resource_arn>" # <3>
25+
role_arn: "<iam_role>" # <4>
26+
indexed_attributes: [ "<indexed_attr_0>", "<indexed_attr_1>" ] # <5>
27+
aws_log_groups: ["<group1>", "<group2>"] # <6>
28+
request_timeout_seconds: 120 # <7>
29+
# ...
30+
----
31+
<1> The destination region for the X-Ray segments sent to the AWS X-Ray service. For example, `eu-west-1`.
32+
<2> Optional. You can override the default Amazon CloudWatch Logs service endpoint to which the requests are forwarded. You must include the protocol, such as `https://`, as part of the endpoint value. For the list of service endpoints by region, see link:https://docs.aws.amazon.com/general/latest/gr/cwl_region.html[Amazon CloudWatch Logs endpoints and quotas] (AWS General Reference).
33+
<3> The Amazon Resource Name (ARN) of the AWS resource that is running the Collector.
34+
<4> The AWS Identity and Access Management (IAM) role for uploading the X-Ray segments to a different account.
35+
<5> The list of attribute names to be converted to X-Ray annotations.
36+
<6> The list of log group names for Amazon CloudWatch Logs.
37+
<7> Time duration in seconds before timing out a request. If omitted, the default value is `30`.
38+
39+
[role="_additional-resources"]
40+
.Additional resources
41+
* link:https://docs.aws.amazon.com/xray/latest/devguide/aws-xray.html[AWS X-Ray Developer Guide: What is AWS X-Ray?]
42+
* link:https://docs.aws.amazon.com/sdk-for-go/api/index.html[AWS Documentation: AWS SDK for Go API Reference]
43+
* link:https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html#specifying-credentials[AWS SDK for Go Developer Guide: Specifying Credentials]
44+
// * link:https://docs.aws.amazon.com/sdk-for-go/v2/developer-guide/configure-auth.html[AWS SDK for Go v2 Developer Guide: Configure Authentication]
45+
* link:https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html[AWS Identity and Access Management User Guide: IAM roles]

‎modules/otel-exporters-debug-exporter.adoc‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * observability/otel/otel-collector/otel-collector-exporters.adoc
4+
5+
:_mod-docs-content-type: REFERENCE
6+
[id="otel-exporters-debug-exporter_{context}"]
7+
= Debug Exporter
8+
9+
[role="_abstract"]
10+
The Debug Exporter prints traces and metrics to the standard output.
11+
12+
.OpenTelemetry Collector custom resource with the enabled Debug Exporter
13+
[source,yaml]
14+
----
15+
# ...
16+
config:
17+
exporters:
18+
debug:
19+
verbosity: detailed # <1>
20+
sampling_initial: 5 # <2>
21+
sampling_thereafter: 200 # <3>
22+
use_internal_logger: true # <4>
23+
service:
24+
pipelines:
25+
traces:
26+
exporters: [debug]
27+
metrics:
28+
exporters: [debug]
29+
# ...
30+
----
31+
<1> Verbosity of the debug export: `detailed`, `normal`, or `basic`. When set to `detailed`, pipeline data are verbosely logged. Defaults to `normal`.
32+
<2> Initial number of messages logged per second. The default value is `2` messages per second.
33+
<3> Sampling rate after the initial number of messages, the value in `sampling_initial`, has been logged. Disabled by default with the default `1` value. Sampling is enabled with values greater than `1`. For more information, see the page for the link:https://pkg.go.dev/go.uber.org/zap/zapcore?utm_source=godoc#NewSamplerWithOptions[sampler function in the `zapcore` package] on the Go Project's website.
34+
<4> When set to `true`, enables output from the Collector's internal logger for the exporter.

‎modules/otel-exporters-file-exporter.adoc‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * observability/otel/otel-collector/otel-collector-exporters.adoc
4+
5+
:_mod-docs-content-type: REFERENCE
6+
[id="otel-exporters-file-exporter_{context}"]
7+
= File Exporter
8+
9+
[role="_abstract"]
10+
The File Exporter writes telemetry data to files in persistent storage and supports file operations such as rotation, compression, and writing to multiple files. With this exporter, you can also use a resource attribute to control file naming. The only required setting is `path`, which specifies the destination path for telemetry files in the persistent-volume file system.
11+
12+
:FeatureName: The File Exporter
13+
include::snippets/technology-preview.adoc[]
14+
15+
.OpenTelemetry Collector custom resource with the enabled File Exporter
16+
[source,yaml]
17+
----
18+
# ...
19+
config: |
20+
exporters:
21+
file:
22+
path: /data/metrics.json # <1>
23+
rotation: # <2>
24+
max_megabytes: 10 # <3>
25+
max_days: 3 # <4>
26+
max_backups: 3 # <5>
27+
localtime: true # <6>
28+
format: proto # <7>
29+
compression: zstd # <8>
30+
flush_interval: 5 # <9>
31+
# ...
32+
----
33+
<1> The file-system path where the data is to be written. There is no default.
34+
<2> File rotation is an optional feature of this exporter. By default, telemetry data is exported to a single file. Add the `rotation` setting to enable file rotation.
35+
<3> The `max_megabytes` setting is the maximum size a file is allowed to reach until it is rotated. The default is `100`.
36+
<4> The `max_days` setting is for how many days a file is to be retained, counting from the timestamp in the file name. There is no default.
37+
<5> The `max_backups` setting is for retaining several older files. The default is `100`.
38+
<6> The `localtime` setting specifies the local-time format for the timestamp, which is appended to the file name in front of any extension, when the file is rotated. The default is the Coordinated Universal Time (UTC).
39+
<7> The format for encoding the telemetry data before writing it to a file. The default format is `json`. The `proto` format is also supported.
40+
<8> File compression is optional and not set by default. This setting defines the compression algorithm for the data that is exported to a file. Currently, only the `zstd` compression algorithm is supported. There is no default.
41+
<9> The time interval between flushes. A value without a unit is set in nanoseconds. This setting is ignored when file rotation is enabled through the `rotation` settings.

‎modules/otel-exporters-kafka-exporter.adoc‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * observability/otel/otel-collector/otel-collector-exporters.adoc
4+
5+
:_mod-docs-content-type: REFERENCE
6+
[id="otel-exporters-kafka-exporter_{context}"]
7+
= Kafka Exporter
8+
9+
[role="_abstract"]
10+
The Kafka Exporter exports logs, metrics, and traces to Kafka. This exporter uses a synchronous producer that blocks and does not batch messages. You must use it with batch and queued retry processors for higher throughput and resiliency.
11+
12+
.OpenTelemetry Collector custom resource with the enabled Kafka Exporter
13+
[source,yaml]
14+
----
15+
# ...
16+
config:
17+
exporters:
18+
kafka:
19+
brokers: ["localhost:9092"] # <1>
20+
protocol_version: 2.0.0 # <2>
21+
topic: otlp_spans # <3>
22+
auth:
23+
plain_text: # <4>
24+
username: example
25+
password: example
26+
tls: # <5>
27+
ca_file: ca.pem
28+
cert_file: cert.pem
29+
key_file: key.pem
30+
insecure: false # <6>
31+
server_name_override: kafka.example.corp # <7>
32+
service:
33+
pipelines:
34+
traces:
35+
exporters: [kafka]
36+
# ...
37+
----
38+
<1> The list of Kafka brokers. The default is `+localhost:9092+`.
39+
<2> The Kafka protocol version. For example, `+2.0.0+`. This is a required field.
40+
<3> The name of the Kafka topic to read from. The following are the defaults: `+otlp_spans+` for traces, `+otlp_metrics+` for metrics, `+otlp_logs+` for logs.
41+
<4> The plain text authentication configuration. If omitted, plain text authentication is disabled.
42+
<5> The client-side TLS configuration. Defines paths to the TLS certificates. If omitted, TLS authentication is disabled.
43+
<6> Disables verifying the server's certificate chain and hostname. The default is `+false+`.
44+
<7> ServerName indicates the name of the server requested by the client to support virtual hosting.

‎modules/otel-exporters-load-balancing-exporter.adoc‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * observability/otel/otel-collector/otel-collector-exporters.adoc
4+
5+
:_mod-docs-content-type: REFERENCE
6+
[id="otel-exporters-load-balancing-exporter_{context}"]
7+
= Load Balancing Exporter
8+
9+
[role="_abstract"]
10+
The Load Balancing Exporter consistently exports spans, metrics, and logs according to the `routing_key` configuration.
11+
12+
:FeatureName: The Load Balancing Exporter
13+
include::snippets/technology-preview.adoc[leveloffset=+1]
14+
15+
.OpenTelemetry Collector custom resource with the enabled Load Balancing Exporter
16+
[source,yaml]
17+
----
18+
# ...
19+
config:
20+
exporters:
21+
loadbalancing:
22+
routing_key: "service" # <1>
23+
protocol:
24+
otlp: # <2>
25+
timeout: 1s
26+
resolver: # <3>
27+
static: # <4>
28+
hostnames:
29+
- backend-1:4317
30+
- backend-2:4317
31+
dns: # <5>
32+
hostname: otelcol-headless.observability.svc.cluster.local
33+
k8s: # <6>
34+
service: lb-svc.kube-public
35+
ports:
36+
- 15317
37+
- 16317
38+
# ...
39+
----
40+
<1> The `routing_key: service` exports spans for the same service name to the same Collector instance to provide accurate aggregation. The `routing_key: traceID` exports spans based on their `traceID`. The implicit default is `traceID` based routing.
41+
<2> The OTLP is the only supported load balancing protocol. All options of the OTLP exporter are supported.
42+
<3> You can configure only one resolver.
43+
<4> The static resolver distributes the load across the listed endpoints.
44+
<5> You can use the DNS resolver only with a Kubernetes headless service.
45+
<6> The Kubernetes resolver is recommended.

‎modules/otel-exporters-otlp-exporter.adoc‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * observability/otel/otel-collector/otel-collector-exporters.adoc
4+
5+
:_mod-docs-content-type: REFERENCE
6+
[id="otel-exporters-otlp-exporter_{context}"]
7+
= OTLP Exporter
8+
9+
[role="_abstract"]
10+
The OTLP gRPC Exporter exports traces and metrics by using the OpenTelemetry protocol (OTLP).
11+
12+
.OpenTelemetry Collector custom resource with the enabled OTLP Exporter
13+
[source,yaml]
14+
----
15+
# ...
16+
config:
17+
exporters:
18+
otlp:
19+
endpoint: tempo-ingester:4317 # <1>
20+
tls: # <2>
21+
ca_file: ca.pem
22+
cert_file: cert.pem
23+
key_file: key.pem
24+
insecure: false # <3>
25+
insecure_skip_verify: false # # <4>
26+
reload_interval: 1h # <5>
27+
server_name_override: <name> # <6>
28+
headers: # <7>
29+
X-Scope-OrgID: "dev"
30+
service:
31+
pipelines:
32+
traces:
33+
exporters: [otlp]
34+
metrics:
35+
exporters: [otlp]
36+
# ...
37+
----
38+
<1> The OTLP gRPC endpoint. If the `+https://+` scheme is used, then client transport security is enabled and overrides the `insecure` setting in the `tls`.
39+
<2> The client-side TLS configuration. Defines paths to TLS certificates.
40+
<3> Disables client transport security when set to `true`. The default value is `false` by default.
41+
<4> Skips verifying the certificate when set to `true`. The default value is `false`.
42+
<5> Specifies the time interval at which the certificate is reloaded. If the value is not set, the certificate is never reloaded. The `reload_interval` accepts a string containing valid units of time such as `ns`, `us` or `µs`, `ms`, `s`, `m`, `h`.
43+
<6> Overrides the virtual hostname of authority such as the authority header field in requests. You can use this for testing.
44+
<7> Headers are sent for every request performed during an established connection.

0 commit comments

Comments
 (0)