Skip to content

Conversation

@runderwo
Copy link
Contributor

@runderwo runderwo commented Oct 13, 2025

Previously, there was no connection from the HTTP client created by the AWS library to the built-in HTTP debugging callbacks. This change creates that path, so that HTTP traffic created by AWS HTTP clients can be debugged in the usual way (by enabling HTTP debugging at build time and supplying config variables in the output block).

Without this change there is zero visibility into AWS HTTP sessions even when HTTP debugging is enabled, which violates the principle of least surprise to the user, and also as a result it's difficult to get help from AWS support because they tend to blame the client rather than the service until malformed requests are ruled out.

Testing
Before we can approve your change; please submit the following in a comment:

  • Example configuration file for the change

To enable HTTP debugging output for an AWS service, add one or more of the following to an output section:

      _debug.http.request_headers: true
      _debug.http.request_payload: true
      _debug.http.response_headers: true
      _debug.http.response_payload: true
  • Debug log output from testing the change

Relevant log snippet (sanitized):

[2025/10/13 16:06:09] [debug] [output:kinesis_firehose:kinesis_firehose.XXX] firehose:PutRecordBatch: events=5, payload=2729 bytes
[2025/10/13 16:06:09] [debug] [output:kinesis_firehose:kinesis_firehose.XXX] Sending log records to delivery stream XXX
[2025/10/13 16:06:09] [debug] [upstream] KA connection #569 to us-east-1.aws.x.y.z:15173 is connected
[2025/10/13 16:06:09] [debug] [http_client] not using http_proxy for header
[2025/10/13 16:06:09] [debug] [http] request headers
POST / HTTP/1.1
Host: us-east-1.aws.x.y.z:1234
Content-Length: 2729
User-Agent: aws-fluent-bit-plugin
Content-Type: application/x-amz-json-1.1
X-Amz-Target: Firehose_20150804.PutRecordBatch
x-amz-date: 20251013T160609Z
Authorization: AWS4-HMAC-SHA256 Credential=XXX
Connection: keep-alive


[2025/10/13 16:06:09] [debug] [http] request payload (2729 bytes)
{"DeliveryStreamName":"atlas-kinesis-splunk","Records":[{"Data":"<snip>"},{"Data":"<snip>"},{"Data":"<snip>"},{"Data":"<snip>"}]}
[2025/10/13 16:06:09] [debug] [http] response headers
HTTP/1.1 200 OK
x-amzn-RequestId: xxx
x-amz-id-2: yyy
Content-Type: application/x-amz-json-1.1
Content-Length: 1259
Date: Mon, 13 Oct 2025 16:06:09 GMT


[2025/10/13 16:06:09] [debug] [http] response payload (1259 bytes)
{"Encrypted":false,"FailedPutCount":0,"RequestResponses":[{"RecordId":"<snip>"},{"RecordId":"<snip>"},{"RecordId":"<snip>"},{"RecordId":"<snip>"},{"RecordId":"<snip>"}]}
  • Attached Valgrind output that shows no leaks or memory corruption was found
==362092== 
==362092== HEAP SUMMARY:
==362092==     in use at exit: 13,874 bytes in 49 blocks
==362092==   total heap usage: 247,903 allocs, 247,854 frees, 58,923,676 bytes allocated
==362092== 
==362092== LEAK SUMMARY:
==362092==    definitely lost: 0 bytes in 0 blocks
==362092==    indirectly lost: 0 bytes in 0 blocks
==362092==      possibly lost: 0 bytes in 0 blocks
==362092==    still reachable: 13,874 bytes in 49 blocks
==362092==         suppressed: 0 bytes in 0 blocks
==362092== Reachable blocks (those to which a pointer was found) are not shown.
==362092== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==362092== 
==362092== For lists of detected and suppressed errors, rerun with: -s
==362092== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

Documentation

  • Documentation required for this feature

I can't tell if the HTTP debug callback configuration keys are documented somewhere; I had to find them and guess how to use them by reading code.

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

Summary by CodeRabbit

  • New Features

    • Added HTTP debug hooks to the AWS client for richer request diagnostics.
  • Refactor

    • Integrated AWS HTTP debug initialization into the Firehose output flow.
  • Bug Fixes

    • Initialization now fails early if HTTP debug setup cannot be established.
  • Chores

    • Ensure debug context is created at client init and properly cleaned up on destruction.

✏️ Tip: You can customize this high-level summary in your review settings.

@coderabbitai
Copy link

coderabbitai bot commented Oct 13, 2025

Note

Other AI code review bot(s) detected

CodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review.

Walkthrough

Added a new http_cb_ctx callback pointer to struct flb_aws_client. The callback is created during client init and destroyed on teardown. HTTP client debug is enabled during requests and the Firehose plugin performs debug setup during initialization, aborting init on setup failure.

Changes

Cohort / File(s) Summary
Public AWS client struct update
include/fluent-bit/flb_aws_util.h
Added field struct flb_callback *http_cb_ctx; to struct flb_aws_client (placed after debug_only, guarded by #ifdef FLB_HAVE_HTTP_CLIENT_DEBUG).
AWS client lifecycle and request wiring
src/aws/flb_aws_util.c
Added #include for flb_http_client_debug.h; flb_aws_client_create() initializes http_cb_ctx via flb_callback_create("aws client") and returns NULL on failure; flb_aws_client_destroy() calls flb_callback_destroy() on http_cb_ctx; request_do() calls flb_http_client_debug_enable(c, aws_client->http_cb_ctx) under #ifdef FLB_HAVE_HTTP_CLIENT_DEBUG.
Firehose plugin: HTTP debug setup
plugins/out_kinesis_firehose/firehose.c
Added #include for flb_http_client_debug.h; in cb_firehose_init calls flb_http_client_debug_setup(ctx->firehose_client->http_cb_ctx, &ins->properties) behind #ifdef FLB_HAVE_HTTP_CLIENT_DEBUG, logs and aborts init on negative return. Also declares/uses flb_http_client_debug_setup interaction.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  participant App as Caller
  participant Factory as flb_aws_client_create()
  participant CB as flb_callback_create()
  participant Client as flb_aws_client

  App->>Factory: create aws client
  Factory->>CB: flb_callback_create("aws client")
  alt callback created
    CB-->>Factory: cb_ctx
    Factory-->>Client: return client (http_cb_ctx set)
  else creation failed
    CB-->>Factory: NULL
    Factory-->>App: NULL (error)
  end
Loading
sequenceDiagram
  autonumber
  participant Client as flb_aws_client
  participant Req as request_do()
  participant HTTP as HTTP client

  Client->>Req: perform request
  alt FLB_HAVE_HTTP_CLIENT_DEBUG
    Req->>Req: flb_http_client_debug_enable(c, Client.http_cb_ctx)
  end
  Req->>HTTP: send request (debug possibly enabled)
  HTTP-->>Req: response
  Req-->>Client: return result
Loading
sequenceDiagram
  autonumber
  participant Plugin as Firehose init
  participant Client as flb_aws_client
  participant Setup as flb_http_client_debug_setup()

  Plugin->>Client: access http_cb_ctx
  Plugin->>Setup: flb_http_client_debug_setup(Client.http_cb_ctx, &props)
  alt setup ok
    Setup-->>Plugin: 0
    Plugin-->>Plugin: continue init
  else setup failed
    Setup-->>Plugin: <0
    Plugin-->>Plugin: log error / abort init
  end
Loading

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

  • Verify public API addition in include/fluent-bit/flb_aws_util.h.
  • Check allocation failure handling and cleanup in flb_aws_client_create() and flb_aws_client_destroy().
  • Confirm conditional compilation and correct use of flb_http_client_debug_enable and flb_http_client_debug_setup in source files.

Suggested reviewers

  • koleini
  • fujimotos

Poem

I twitch my whiskers, find the hook,
A tiny callback joins the book.
It hops in requests, watches the stream,
Setup, teardown — a tidy dream.
I bound away with a happy look. 🐰✨

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main change: enabling HTTP debugging for AWS client requests in the Kinesis Firehose output plugin, which is the core objective of the changeset.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 81ac146 and 44481ed.

📒 Files selected for processing (2)
  • include/fluent-bit/flb_aws_util.h (1 hunks)
  • src/aws/flb_aws_util.c (4 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
  • include/fluent-bit/flb_aws_util.h
  • src/aws/flb_aws_util.c
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (31)
  • GitHub Check: pr-windows-build / call-build-windows-package (Windows 32bit, x86, x86-windows-static, 3.31.6)
  • GitHub Check: pr-windows-build / call-build-windows-package (Windows 64bit (Arm64), amd64_arm64, -DCMAKE_SYSTEM_NAME=Windows -DCMA...
  • GitHub Check: pr-windows-build / call-build-windows-package (Windows 64bit, x64, x64-windows-static, 3.31.6)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_ARROW=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_COMPILER_STRICT_POINTER_TYPES=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=Off, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=Off, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_THREAD=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_MEMORY=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_MEMORY=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_THREAD=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_COVERAGE=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_UNDEFINED=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=Off, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SMALL=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SMALL=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_UNDEFINED=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_ADDRESS=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_ADDRESS=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=Off, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=On, 3.31.6, gcc, g++)
  • GitHub Check: PR - fuzzing test
  • GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, gcc, g++, ubuntu-24.04, clang-14)
  • GitHub Check: pr-compile-centos-7
  • GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, clang, clang++, ubuntu-24.04, clang-14)
  • GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, gcc, g++, ubuntu-22.04, clang-12)
  • GitHub Check: pr-compile-without-cxx (3.31.6)
  • GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, clang, clang++, ubuntu-22.04, clang-12)

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 2b16b24 and 7e4d2c2.

📒 Files selected for processing (2)
  • include/fluent-bit/flb_aws_util.h (1 hunks)
  • src/aws/flb_aws_util.c (3 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
src/aws/flb_aws_util.c (1)
include/fluent-bit/flb_mem.h (1)
  • flb_free (126-128)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (30)
  • GitHub Check: pr-windows-build / call-build-windows-package (Windows 64bit (Arm64), amd64_arm64, -DCMAKE_SYSTEM_NAME=Windows -DCMA...
  • GitHub Check: pr-windows-build / call-build-windows-package (Windows 32bit, x86, x86-windows-static, 3.31.6)
  • GitHub Check: pr-windows-build / call-build-windows-package (Windows 64bit, x64, x64-windows-static, 3.31.6)
  • GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_ADDRESS=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_ARROW=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=Off, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_MEMORY=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_MEMORY=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_THREAD=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=Off, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_COVERAGE=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=Off, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_THREAD=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=Off, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_UNDEFINED=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SMALL=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_UNDEFINED=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SMALL=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=On, 3.31.6, gcc, g++)
  • GitHub Check: pr-compile-centos-7
  • GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_ADDRESS=On, 3.31.6, gcc, g++)
  • GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, clang, clang++, ubuntu-22.04, clang-12)
  • GitHub Check: pr-compile-without-cxx (3.31.6)
  • GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, clang, clang++, ubuntu-24.04, clang-14)
  • GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, gcc, g++, ubuntu-24.04, clang-14)
  • GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, gcc, g++, ubuntu-22.04, clang-12)
  • GitHub Check: PR - fuzzing test
🔇 Additional comments (2)
include/fluent-bit/flb_aws_util.h (1)

105-107: LGTM!

The new callback field is appropriately placed in the struct with a clear comment describing its purpose.

src/aws/flb_aws_util.c (1)

395-396: LGTM!

The callback is correctly wired into the HTTP client context for debugging purposes.

@edsiper
Copy link
Member

edsiper commented Oct 13, 2025

  • why is this change needed ?
  • what's the end-user outcome ?
  • fix DCO (sign-off)

@edsiper
Copy link
Member

edsiper commented Nov 21, 2025

here we will need to correct the commit history, these are the files being modified:

image

as we can see, there are two interfaces being modified: aws_utils and out_kinesis_firehose plugin, so the commits must be splitted as:

  • aws: utils: ...
  • out_kinesis_firehose: ...

Signed-off-by: Ryan Underwood <ryan.underwood@mongodb.com>
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (2)
src/aws/flb_aws_util.c (2)

272-272: Use C-style comment syntax.

The // comment style should be replaced with /* */ for consistency with the project's C coding standards.

Based on learnings.

Apply this diff:

-    client->http_cb_ctx = flb_callback_create("aws client");
+    client->http_cb_ctx = flb_callback_create("aws client"); /* Callbacks context */

301-303: Fix inconsistent indentation (tabs vs. spaces).

Lines 301 and 303 use tab characters while line 302 and the surrounding code use spaces. This creates inconsistent indentation that can cause issues with some editors and violates consistent code formatting.

Apply this diff to use consistent space indentation:

-	if (aws_client->http_cb_ctx) {
-            flb_callback_destroy(aws_client->http_cb_ctx);
-	}
+        if (aws_client->http_cb_ctx) {
+            flb_callback_destroy(aws_client->http_cb_ctx);
+        }
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 6a56e7e and 73d6a85.

📒 Files selected for processing (3)
  • include/fluent-bit/flb_aws_util.h (1 hunks)
  • plugins/out_kinesis_firehose/firehose.c (2 hunks)
  • src/aws/flb_aws_util.c (4 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
  • include/fluent-bit/flb_aws_util.h
  • plugins/out_kinesis_firehose/firehose.c
🧰 Additional context used
🧠 Learnings (9)
📚 Learning: 2025-08-31T12:46:11.940Z
Learnt from: ThomasDevoogdt
Repo: fluent/fluent-bit PR: 9277
File: .github/workflows/pr-compile-check.yaml:147-151
Timestamp: 2025-08-31T12:46:11.940Z
Learning: In fluent-bit CMakeLists.txt, the system library preference flags are defined as FLB_PREFER_SYSTEM_LIB_ZSTD and FLB_PREFER_SYSTEM_LIB_KAFKA with the FLB_ prefix.

Applied to files:

  • src/aws/flb_aws_util.c
📚 Learning: 2025-08-29T06:25:02.561Z
Learnt from: shadowshot-x
Repo: fluent/fluent-bit PR: 10794
File: tests/internal/aws_compress.c:7-7
Timestamp: 2025-08-29T06:25:02.561Z
Learning: In Fluent Bit, ZSTD (zstandard) compression library is bundled directly in the source tree at `lib/zstd-1.5.7` and is built unconditionally as a static library. Unlike optional external dependencies, ZSTD does not use conditional compilation guards like `FLB_HAVE_ZSTD` and is always available. Headers like `<fluent-bit/flb_zstd.h>` can be included directly without guards.

Applied to files:

  • src/aws/flb_aws_util.c
📚 Learning: 2025-08-31T12:46:11.940Z
Learnt from: ThomasDevoogdt
Repo: fluent/fluent-bit PR: 9277
File: .github/workflows/pr-compile-check.yaml:147-151
Timestamp: 2025-08-31T12:46:11.940Z
Learning: In fluent-bit, the correct CMake flag for using system librdkafka is `FLB_PREFER_SYSTEM_LIB_KAFKA=ON`.

Applied to files:

  • src/aws/flb_aws_util.c
📚 Learning: 2025-11-21T06:23:29.750Z
Learnt from: cosmo0920
Repo: fluent/fluent-bit PR: 11171
File: include/fluent-bit/flb_lib.h:52-53
Timestamp: 2025-11-21T06:23:29.750Z
Learning: In Fluent Bit core (fluent/fluent-bit repository), function descriptions/documentation are not required for newly added functions in header files.

Applied to files:

  • src/aws/flb_aws_util.c
📚 Learning: 2025-08-29T06:25:27.250Z
Learnt from: shadowshot-x
Repo: fluent/fluent-bit PR: 10794
File: tests/internal/aws_compress.c:93-107
Timestamp: 2025-08-29T06:25:27.250Z
Learning: In Fluent Bit, ZSTD compression is enabled by default and is treated as a core dependency, not requiring conditional compilation guards like `#ifdef FLB_HAVE_ZSTD`. Unlike some other optional components such as ARROW/PARQUET (which use `#ifdef FLB_HAVE_ARROW` guards), ZSTD support is always available and doesn't need build-time conditionals. ZSTD headers are included directly without guards across multiple plugins and core components.

Applied to files:

  • src/aws/flb_aws_util.c
📚 Learning: 2025-08-29T06:24:44.797Z
Learnt from: shadowshot-x
Repo: fluent/fluent-bit PR: 10794
File: src/aws/flb_aws_compress.c:26-26
Timestamp: 2025-08-29T06:24:44.797Z
Learning: In Fluent Bit, ZSTD support is always available and enabled by default. The build system automatically detects and uses either the system libzstd library or builds the bundled ZSTD version. Unlike other optional dependencies like Arrow which use conditional compilation guards (e.g., FLB_HAVE_ARROW), ZSTD does not require conditional includes or build flags.

Applied to files:

  • src/aws/flb_aws_util.c
📚 Learning: 2025-08-29T06:24:55.855Z
Learnt from: shadowshot-x
Repo: fluent/fluent-bit PR: 10794
File: src/aws/flb_aws_compress.c:52-56
Timestamp: 2025-08-29T06:24:55.855Z
Learning: ZSTD compression is always available in Fluent Bit and does not require conditional compilation guards. Unlike Arrow/Parquet which use #ifdef FLB_HAVE_ARROW guards, ZSTD is built unconditionally with flb_zstd.c included directly in src/CMakeLists.txt and a bundled ZSTD library at lib/zstd-1.5.7/.

Applied to files:

  • src/aws/flb_aws_util.c
📚 Learning: 2025-08-29T06:24:26.170Z
Learnt from: shadowshot-x
Repo: fluent/fluent-bit PR: 10794
File: tests/internal/aws_compress.c:39-42
Timestamp: 2025-08-29T06:24:26.170Z
Learning: In Fluent Bit, ZSTD compression support is enabled by default and does not require conditional compilation guards (like #ifdef FLB_HAVE_ZSTD) around ZSTD-related code declarations and implementations.

Applied to files:

  • src/aws/flb_aws_util.c
📚 Learning: 2025-08-29T06:25:27.250Z
Learnt from: shadowshot-x
Repo: fluent/fluent-bit PR: 10794
File: tests/internal/aws_compress.c:93-107
Timestamp: 2025-08-29T06:25:27.250Z
Learning: In Fluent Bit, ZSTD compression is enabled by default and is treated as a core dependency, not requiring conditional compilation guards like `#ifdef FLB_HAVE_ZSTD`. Unlike some other optional components, ZSTD support is always available and doesn't need build-time conditionals.

Applied to files:

  • src/aws/flb_aws_util.c
🧬 Code graph analysis (1)
src/aws/flb_aws_util.c (1)
include/fluent-bit/flb_mem.h (1)
  • flb_free (126-128)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (31)
  • GitHub Check: pr-windows-build / call-build-windows-package (Windows 64bit, x64, x64-windows-static, 3.31.6)
  • GitHub Check: pr-windows-build / call-build-windows-package (Windows 64bit (Arm64), amd64_arm64, -DCMAKE_SYSTEM_NAME=Windows -DCMA...
  • GitHub Check: pr-windows-build / call-build-windows-package (Windows 32bit, x86, x86-windows-static, 3.31.6)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_COMPILER_STRICT_POINTER_TYPES=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_ARROW=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_THREAD=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=Off, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=Off, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_THREAD=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_ADDRESS=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=On, 3.31.6, gcc, g++)
  • GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, gcc, g++, ubuntu-22.04, clang-12)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_COVERAGE=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_UNDEFINED=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SMALL=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_MEMORY=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=Off, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_UNDEFINED=On, 3.31.6, gcc, g++)
  • GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, gcc, g++, ubuntu-24.04, clang-14)
  • GitHub Check: pr-compile-without-cxx (3.31.6)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_MEMORY=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_ADDRESS=On, 3.31.6, clang, clang++)
  • GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, clang, clang++, ubuntu-24.04, clang-14)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SMALL=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=On, 3.31.6, clang, clang++)
  • GitHub Check: pr-compile-centos-7
  • GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, clang, clang++, ubuntu-22.04, clang-12)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=Off, 3.31.6, gcc, g++)
  • GitHub Check: PR - fuzzing test
🔇 Additional comments (2)
src/aws/flb_aws_util.c (2)

23-23: LGTM!

The include directive for HTTP client debugging support is correctly placed.


399-401: LGTM!

The HTTP client debugging is correctly enabled with the callback context, properly wrapped in a feature guard to prevent compilation errors when the feature is not available.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 73d6a85 and 81ac146.

📒 Files selected for processing (2)
  • include/fluent-bit/flb_aws_util.h (1 hunks)
  • src/aws/flb_aws_util.c (4 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/aws/flb_aws_util.c
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (31)
  • GitHub Check: pr-windows-build / call-build-windows-package (Windows 64bit (Arm64), amd64_arm64, -DCMAKE_SYSTEM_NAME=Windows -DCMA...
  • GitHub Check: pr-windows-build / call-build-windows-package (Windows 64bit, x64, x64-windows-static, 3.31.6)
  • GitHub Check: pr-windows-build / call-build-windows-package (Windows 32bit, x86, x86-windows-static, 3.31.6)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=Off, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_COMPILER_STRICT_POINTER_TYPES=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_UNDEFINED=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_ARROW=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_MEMORY=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=Off, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_MEMORY=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_THREAD=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_UNDEFINED=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SIMD=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_ADDRESS=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SANITIZE_THREAD=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_COVERAGE=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SMALL=On, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=Off, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=Off, 3.31.6, gcc, g++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_SMALL=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DSANITIZE_ADDRESS=On, 3.31.6, clang, clang++)
  • GitHub Check: run-ubuntu-unit-tests (-DFLB_JEMALLOC=On, 3.31.6, gcc, g++)
  • GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, gcc, g++, ubuntu-24.04, clang-14)
  • GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, gcc, g++, ubuntu-22.04, clang-12)
  • GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, clang, clang++, ubuntu-24.04, clang-14)
  • GitHub Check: pr-compile-system-libs (-DFLB_PREFER_SYSTEM_LIBS=On, 3.31.6, clang, clang++, ubuntu-22.04, clang-12)
  • GitHub Check: pr-compile-without-cxx (3.31.6)
  • GitHub Check: pr-compile-centos-7
  • GitHub Check: PR - fuzzing test
🔇 Additional comments (2)
include/fluent-bit/flb_aws_util.h (2)

107-107: Original review comment is incorrect.

The struct flb_callback type is properly declared and available. The file flb_aws_util.h includes flb_output.h, which in turn includes flb_callback.h (line 41). This makes struct flb_callback available to any code that includes flb_aws_util.h, so the field declaration on line 107 is correctly typed and will compile without issues.


105-107: This review comment is based on incorrect assumptions.

The http_cb_ctx field already exists in the struct flb_aws_client definition in the public header. It is not a newly added field—it is already initialized in production code (src/aws/flb_aws_util.c:272) and actively used by plugin code (plugins/out_kinesis_firehose/firehose.c:293). The struct layout and ABI are unchanged by this code.

Likely an incorrect or invalid review comment.

Signed-off-by: Ryan Underwood <ryan.underwood@mongodb.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants