Skip to content

Commit 8969cff

Browse files
author
Robert Fekete
committed
Adds s3 object-key-suffix and related naming clarifications
1 parent ff0a311 commit 8969cff

File tree

2 files changed

+33
-2
lines changed

2 files changed

+33
-2
lines changed

content/chapter-destinations/destination-s3/_index.md

Lines changed: 29 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,14 @@ s3(
6060

6161
All of these strategies can be used individually, or together.
6262

63+
The name of the object can be further modified by the following options:
64+
65+
- [`object-key-suffix()`](#object-key-suffix)
66+
- `compression()`
67+
68+
<!-- object-key()object-key-timestamp()maxo-object-size()object-key-suffix().gz(if compression is enabled) -->
69+
70+
6371
## Upload options
6472

6573
{{% param "product.abbrev" %}} uploads objects using the multipart upload API. {{% param "product.abbrev" %}} composes chunks locally. When a chunk reaches the size set in `chunk-size()` (by default 5 MiB), the chunk is uploaded. When an object is finished, the multipart upload is completed and S3 merges the chunks.
@@ -119,7 +127,9 @@ If you configure an invalid value, the default is used.
119127
| Type: | boolean |
120128
| Default: | `no` |
121129

122-
*Description:* Setting `compression(yes)` enables gzip compression, and implicitly adds a `.gz` suffix to the created object's key. You can set the level of the compression using the `compresslevel()` option (0-9).
130+
*Description:* Setting `compression(yes)` enables gzip compression, and implicitly adds a `.gz` suffix to the very end of the created object's key. You can set the level of the compression using the `compresslevel()` option (0-9).
131+
132+
{{< include-headless "chunk/destination-s3-object-name.md" >}}
123133

124134
## compresslevel()
125135

@@ -174,6 +184,8 @@ Available in {{< product >}} 4.8 and later.
174184

175185
*Description:* The maximal size of the S3 object. If an object reaches this size, {{% param "product_name" %}} appends an index ("-1", "-2", ...) to the end of the object key and starts a new object after rotation.
176186

187+
{{< include-headless "chunk/destination-s3-object-name.md" >}}
188+
177189
## max-pending-uploads()
178190

179191
| | |
@@ -193,7 +205,20 @@ Available in {{< product >}} 4.8 and later.
193205
| Type: | template |
194206
| Default: | N/A |
195207

196-
*Description:* The [object key](https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-keys.html) (or key name), which uniquely identifies the object in an Amazon S3 bucket. Note that a suffix may be appended to this object key depending on the [naming strategies](#creating-objects) used. Example: `my-logs/${HOSTNAME}/`.
208+
*Description:* The [object key](https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-keys.html) (or key name), which uniquely identifies the object in an Amazon S3 bucket. Note that a suffix may be appended to this object key depending on the [naming strategies](#creating-objects) and other options used. Example: `my-logs/${HOSTNAME}/`.
209+
210+
## object-key-suffix()
211+
212+
| | |
213+
| -------- | --------- |
214+
| Type: | template |
215+
| Default: | empty string |
216+
217+
Available in {{< product >}} 4.20 and later.
218+
219+
*Description:* A suffix added to the object key.
220+
221+
{{< include-headless "chunk/destination-s3-object-name.md" >}}
197222

198223
## object-key-timestamp()
199224

@@ -204,6 +229,8 @@ Available in {{< product >}} 4.8 and later.
204229

205230
*Description:* The `object-key-timestamp()` option can be used to set a datetime-related template, which is appended to the end of the object key, for example: `"${R_MONTH_ABBREV}${R_DAY}"`. When a log message arrives with a newer timestamp template resolution, the previous timestamped object gets finished and a new one is started with the new timestamp. If an older message arrives, it doesn`t reopen the old object, but starts a new object with the key having an index appended to the old object.
206231

232+
{{< include-headless "chunk/destination-s3-object-name.md" >}}
233+
207234
{{< include-headless "chunk/option-persist-name.md" >}}
208235

209236
## region()
Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
---
2+
---
3+
<!-- This file is under the copyright of Axoflow, and licensed under Apache License 2.0, except for using the Axoflow and AxoSyslog trademarks. -->
4+
For a summary of how the different options affect the name of the object, see [Creating objects](#creating-objects).

0 commit comments

Comments
 (0)