Skip to content

Conversation

@stefanoboriero
Copy link

@stefanoboriero stefanoboriero commented Sep 10, 2025

This change is based on the existing approach to Managed Identity authentication used for the out_azure_kusto plugin.

Addresses #10777


Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

  • Example configuration file for the change
  • Debug log output from testing the change
  • Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

  • [N/A] Run local packaging test showing all targets (including any new ones) build.
  • [N/A] Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

  • Documentation required for this feature

Backporting

  • Backport to latest stable release.

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

Summary by CodeRabbit

  • New Features

    • Azure Logs Ingestion now supports Managed Identity (system- and user-assigned) in addition to Service Principal.
    • New auth_type setting (default: service_principal) with improved validation and clearer errors; token retrieval now handles MSI and SP paths with caching/renewal.
  • Tests

    • Added runtime tests covering system/user managed identity, service principal (explicit/default), invalid auth types, and credential error cases.
  • Chores

    • Updated build/test configuration to include the new tests.

✏️ Tip: You can customize this high-level summary in your review settings.

@coderabbitai
Copy link

coderabbitai bot commented Sep 10, 2025

Walkthrough

Adds Managed Identity (MSI) authentication to the out_azure_logs_ingestion plugin: introduces an auth_type config, implements MSI token retrieval/caching via IMDS, branches token acquisition between service principal and MSI flows, updates config validation and auth URL construction, and adds runtime tests and build entries.

Changes

Cohort / File(s) Summary
Plugin build config
plugins/out_azure_logs_ingestion/CMakeLists.txt
Adds azure_logs_ingestion_msiauth.c to plugin sources.
Core plugin & API
plugins/out_azure_logs_ingestion/azure_logs_ingestion.c, plugins/out_azure_logs_ingestion/azure_logs_ingestion.h
Adds auth_type enum/fields (auth_type, auth_type_str), adds auth_type config mapping, and branches token acquisition between OAuth2 (service principal) and MSI flows; integrates MSI header/calls and logging.
Config parsing & URL construction
plugins/out_azure_logs_ingestion/azure_logs_ingestion_conf.c
Parses auth_type, validates required parameters per auth mode (service_principal vs managed_identity), determines system vs user-managed identity, and constructs auth URL conditionally (AAD tenant-based vs IMDS URL with optional client_id).
MSI auth module
plugins/out_azure_logs_ingestion/azure_logs_ingestion_msiauth.c, plugins/out_azure_logs_ingestion/azure_logs_ingestion_msiauth.h
New module implementing MSI token retrieval via IMDS: HTTP GET with Metadata: true, JSON parsing, token caching/expiry handling; exposes MSI URL template and flb_azure_li_msiauth_token_get API.
Runtime tests & test build
tests/runtime/out_azure_logs_ingestion.c, tests/runtime/CMakeLists.txt
Adds runtime tests covering system/user MSI, service principal (explicit/default), missing/invalid params, and invalid auth_type; wires tests into runtime test build.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  participant CFG as Config
  participant PLG as Azure Logs Ingestion Plugin
  participant AUTH as Auth Layer
  participant IMDS as Azure IMDS (MSI)
  participant AAD as Azure AD (OAuth2)

  CFG->>PLG: supply auth_type + credentials
  PLG->>AUTH: request access token
  alt managed_identity (MSI)
    AUTH->>IMDS: GET /metadata/identity/oauth2/token\nHeader: Metadata: true
    IMDS-->>AUTH: 200 {access_token, expires_in}
    AUTH->>AUTH: parse JSON, cache token, set expiry
  else service_principal (OAuth2)
    AUTH->>AAD: POST /{tenant}/oauth2/v2.0/token\nclient_id/client_secret
    AAD-->>AUTH: 200 {access_token, expires_in}
    AUTH->>AUTH: parse response, cache token, set expiry
  end
  AUTH-->>PLG: return access_token
  note right of AUTH: On expiry, refresh via selected path
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Areas to review closely:

  • azure_logs_ingestion_msiauth.c token retrieval, error handling, and expiry math.
  • Branching in azure_logs_ingestion.c for thread-safety and token copying between MSI and SP paths.
  • Validation and auth URL construction in azure_logs_ingestion_conf.c.
  • New tests in tests/runtime/out_azure_logs_ingestion.c for correctness and flakiness.

Possibly related issues

Poem

I nibble headers, chase the token string,
Metadata says “true” — the tiny bell will ring.
System or user, I choose with care,
I fetch and cache, renew the air.
Logs hop to Azure — a rabbit's joyful flair. 🐇✨

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title directly and concisely describes the main change: adding Managed Identity support to the Azure Logs Ingestion output plugin.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Tip

📝 Customizable high-level summaries are now available in beta!

You can now customize how CodeRabbit generates the high-level summary in your pull requests — including its content, structure, tone, and formatting.

  • Provide your own instructions using the high_level_summary_instructions setting.
  • Format the summary however you like (bullet lists, tables, multi-section layouts, contributor stats, etc.).
  • Use high_level_summary_in_walkthrough to move the summary from the description to the walkthrough section.

Example instruction:

"Divide the high-level summary into five sections:

  1. 📝 Description — Summarize the main change in 50–60 words, explaining what was done.
  2. 📓 References — List relevant issues, discussions, documentation, or related PRs.
  3. 📦 Dependencies & Requirements — Mention any new/updated dependencies, environment variable changes, or configuration updates.
  4. 📊 Contributor Summary — Include a Markdown table showing contributions:
    | Contributor | Lines Added | Lines Removed | Files Changed |
  5. ✔️ Additional Notes — Add any extra reviewer context.
    Keep each section concise (under 200 words) and use bullet or numbered lists for clarity."

Note: This feature is currently in beta for Pro-tier users, and pricing will be announced later.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@stefanoboriero
Copy link
Author

Example output config with system managed identity

[OUTPUT]
  Name            azure_logs_ingestion
  Match           sample
  client_id       system
  auth_type       managed_identity
  dce_url         https://log-analytics-dce-XXXX.region-code.ingest.monitor.azure.com
  dcr_id          dcr-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
  table_name      ladcr_CL
  time_generated  true
  time_key        Time
  Compress        true

Example output config with user managed identity

[OUTPUT]
  Name            azure_logs_ingestion
  Match           sample
  client_id       XXXXXXXX-xxxx-yyyy-zzzz-xxxxyyyyzzzzxyzz
  auth_type       managed_identity
  dce_url         https://log-analytics-dce-XXXX.region-code.ingest.monitor.azure.com
  dcr_id          dcr-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
  table_name      ladcr_CL
  time_generated  true
  time_key        Time
  Compress        true

Example output config with default service principal (should still work as before)

[OUTPUT]
  Name            azure_logs_ingestion
  Match           sample
  client_id       XXXXXXXX-xxxx-yyyy-zzzz-xxxxyyyyzzzzxyzz
  client_secret   some.secret.xxxzzz
  tenant_id       XXXXXXXX-xxxx-yyyy-zzzz-xxxxyyyyzzzzxyzz
  dce_url         https://log-analytics-dce-XXXX.region-code.ingest.monitor.azure.com
  dcr_id          dcr-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
  table_name      ladcr_CL
  time_generated  true
  time_key        Time
  Compress        true

@stefanoboriero
Copy link
Author

Debug logs from test:

Fluent Bit v4.1.0
* Copyright (C) 2015-2025 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

______ _                  _    ______ _ _             ___   __
|  ___| |                | |   | ___ (_) |           /   | /  |
| |_  | |_   _  ___ _ __ | |_  | |_/ /_| |_  __   __/ /| | `| |
|  _| | | | | |/ _ \ '_ \| __| | ___ \ | __| \ \ / / /_| |  | |
| |   | | |_| |  __/ | | | |_  | |_/ / | |_   \ V /\___  |__| |_
\_|   |_|\__,_|\___|_| |_|\__| \____/|_|\__|   \_/     |_(_)___/


[2025/09/10 11:32:39.465143663] [ info] Configuration:
[2025/09/10 11:32:39.465220964] [ info]  flush time     | 1.000000 seconds
[2025/09/10 11:32:39.465262964] [ info]  grace          | 5 seconds
[2025/09/10 11:32:39.465301565] [ info]  daemon         | 0
[2025/09/10 11:32:39.465339665] [ info] ___________
[2025/09/10 11:32:39.465377666] [ info]  inputs:
[2025/09/10 11:32:39.465416766] [ info]      tail
[2025/09/10 11:32:39.465455067] [ info] ___________
[2025/09/10 11:32:39.465492867] [ info]  filters:
[2025/09/10 11:32:39.465530768] [ info]      modify.0
[2025/09/10 11:32:39.465568868] [ info] ___________
[2025/09/10 11:32:39.465606569] [ info]  outputs:
[2025/09/10 11:32:39.465644269] [ info]      azure_logs_ingestion.0
[2025/09/10 11:32:39.465682170] [ info] ___________
[2025/09/10 11:32:39.465719770] [ info]  collectors:
[2025/09/10 11:32:39.466058075] [ info] [fluent bit] version=4.1.0, commit=, pid=34991
[2025/09/10 11:32:39.466068375] [debug] [engine] coroutine stack size: 24576 bytes (24.0K)
[2025/09/10 11:32:39.466094075] [ info] [storage] ver=1.5.3, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2025/09/10 11:32:39.466105975] [ info] [simd    ] disabled
[2025/09/10 11:32:39.466110975] [ info] [cmetrics] version=1.0.5
[2025/09/10 11:32:39.466119775] [ info] [ctraces ] version=0.6.6
[2025/09/10 11:32:39.466197577] [ info] [input:tail:tail.0] initializing
[2025/09/10 11:32:39.466202677] [ info] [input:tail:tail.0] storage_strategy='memory' (memory only)
[2025/09/10 11:32:39.466212377] [debug] [tail:tail.0] created event channels: read=25 write=26
[2025/09/10 11:32:39.466322078] [debug] [input:tail:tail.0] flb_tail_fs_inotify_init() initializing inotify tail input
[2025/09/10 11:32:39.466330778] [debug] [input:tail:tail.0] inotify watch fd=31
[2025/09/10 11:32:39.466336878] [debug] [input:tail:tail.0] scanning path /var/log/nextflow-run.log
[2025/09/10 11:32:39.466361479] [debug] [input:tail:tail.0] file will be read in POSIX_FADV_DONTNEED mode /var/log/nextflow-run.log
[2025/09/10 11:32:39.466416079] [debug] [input:tail:tail.0] inode=1450 with offset=107 appended as /var/log/nextflow-run.log
[2025/09/10 11:32:39.466421579] [debug] [input:tail:tail.0] scan_glob add(): /var/log/nextflow-run.log, inode 1450
[2025/09/10 11:32:39.466426680] [debug] [input:tail:tail.0] 1 new files found on path '/var/log/nextflow-run.log'
[2025/09/10 11:32:39.466472980] [debug] [filter:modify:modify.0] Initialized modify filter with 0 conditions and 1 rules
[2025/09/10 11:32:39.466500381] [debug] [azure_logs_ingestion:azure_logs_ingestion.0] created event channels: read=33 write=34
[2025/09/10 11:32:39.470139328] [ info] [output:azure_logs_ingestion:azure_logs_ingestion.0] dce_url='https://towerforge-fluentbit-7t9ftiyjx1cji5wrj7s7rm-apx8.centralus-1.ingest.monitor.azure.com', dcr='dcr-41ece8a3a5c44a329157631d6bdcaa0f', table='Nextflow_log_CL', stream='Custom-Nextflow_log_CL'
[2025/09/10 11:32:39.470832038] [ info] [sp] stream processor started
[2025/09/10 11:32:39.470980840] [ info] [engine] Shutdown Grace Period=5, Shutdown Input Grace Period=2
[2025/09/10 11:32:39.471392545] [debug] [input:tail:tail.0] inode=1450 file=/var/log/nextflow-run.log promote to TAIL_EVENT
[2025/09/10 11:32:39.471468346] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1450 watch_fd=1 name=/var/log/nextflow-run.log
[2025/09/10 11:32:39.471518147] [debug] [input:tail:tail.0] [static files] processed 0b, done
[2025/09/10 11:32:44.688809044] [debug] [input:tail:tail.0] inode=1450, /var/log/nextflow-run.log, events: IN_MODIFY
[2025/09/10 11:32:45.3958700] [debug] [task] created task=0x1492100f3670 id=0 OK
[2025/09/10 11:32:45.27666613] [debug] [upstream] KA connection #44 to towerforge-fluentbit-7t9ftiyjx1cji5wrj7s7rm-apx8.centralus-1.ingest.monitor.azure.com:443 is connected
[2025/09/10 11:32:45.27884816] [debug] [output:azure_logs_ingestion:azure_logs_ingestion.0] token expired. getting new token
[2025/09/10 11:32:45.28422823] [debug] [http_client] not using http_proxy for header
[2025/09/10 11:32:45.37649644] [ info] [azure li msi auth] HTTP Status=200
[2025/09/10 11:32:45.37718145] [ info] [azure li msi auth] access token from '169.254.169.254:80' retrieved
[2025/09/10 11:32:45.37762646] [debug] [output:azure_logs_ingestion:azure_logs_ingestion.0] got azure MSI token
[2025/09/10 11:32:45.37775746] [debug] [output:azure_logs_ingestion:azure_logs_ingestion.0] create token header string
[2025/09/10 11:32:45.37971949] [debug] [output:azure_logs_ingestion:azure_logs_ingestion.0] enabled payload gzip compression
[2025/09/10 11:32:45.38001949] [debug] [http_client] not using http_proxy for header
[2025/09/10 11:32:45.171835514] [ info] [output:azure_logs_ingestion:azure_logs_ingestion.0] http_status=204, dcr_id=dcr-41ece8a3a5c44a329157631d6bdcaa0f, table=Nextflow_log_CL
[2025/09/10 11:32:45.171899815] [debug] [upstream] KA connection #44 to towerforge-fluentbit-7t9ftiyjx1cji5wrj7s7rm-apx8.centralus-1.ingest.monitor.azure.com:443 is now available
[2025/09/10 11:32:45.171928815] [debug] [out flush] cb_destroy coro_id=0
[2025/09/10 11:32:45.171949215] [debug] [task] destroy task=0x1492100f3670 (task_id=0)
[2025/09/10 11:33:16.4120320] [debug] [upstream] drop keepalive connection #-1 to towerforge-fluentbit-7t9ftiyjx1cji5wrj7s7rm-apx8.centralus-1.ingest.monitor.azure.com:443 (keepalive idle timeout)

@stefanoboriero
Copy link
Author

Documentation PR fluent/fluent-bit-docs#2062

@stefanoboriero stefanoboriero force-pushed the azure-log-ingestion-msi branch 2 times, most recently from 62f72e4 to e9418b1 Compare September 10, 2025 11:44
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (5)
plugins/out_azure_logs_ingestion/azure_logs_ingestion_conf.c (3)

58-82: Managed Identity UX: default to system-assigned if client_id is absent; don’t hard-require it

Requiring client_id for MSI complicates system-assigned usage. Defaulting to system-assigned when client_id is unset matches common patterns and reduces config friction.

Apply:

-else if (strcasecmp(ctx->auth_type_str, "managed_identity") == 0) {
-    /* Check if client_id indicates system-assigned or user-assigned managed identity */
-    if (!ctx->client_id) {
-        flb_plg_error(ins, "When using managed_identity auth, client_id must be set to 'system' for system-assigned or the managed identity client ID");
-        flb_az_li_ctx_destroy(ctx);
-        return NULL;
-    }
-
-    if (strcasecmp(ctx->client_id, "system") == 0) {
-        ctx->auth_type = FLB_AZ_LI_AUTH_MANAGED_IDENTITY_SYSTEM;
-    } else {
-        ctx->auth_type = FLB_AZ_LI_AUTH_MANAGED_IDENTITY_USER;
-    }
-}
+else if (strcasecmp(ctx->auth_type_str, "managed_identity") == 0) {
+    /* Default to system-assigned if client_id is unset or equals "system" */
+    if (!ctx->client_id || strcasecmp(ctx->client_id, "system") == 0) {
+        ctx->auth_type = FLB_AZ_LI_AUTH_MANAGED_IDENTITY_SYSTEM;
+    }
+    else {
+        ctx->auth_type = FLB_AZ_LI_AUTH_MANAGED_IDENTITY_USER;
+    }
+}

84-86: Error message lists unsupported option

Message mentions 'workload_identity' but there’s no handling for it here. Remove to avoid confusion.

-        flb_plg_error(ins, "Invalid auth_type '%s'. Valid options are: 'service_principal', 'managed_identity', or 'workload_identity'", 
+        flb_plg_error(ins, "Invalid auth_type '%s'. Valid options are: 'service_principal' or 'managed_identity'",
                      ctx->auth_type_str);

188-213: Destroy the mutex in ctx_destroy

You init token_mutex but never destroy it. Add pthread_mutex_destroy to prevent leaks.

     if (ctx->u_dce) {
         flb_upstream_destroy(ctx->u_dce);
     }
-    flb_free(ctx);
+    pthread_mutex_destroy(&ctx->token_mutex);
+    flb_free(ctx);
plugins/out_azure_logs_ingestion/azure_logs_ingestion.c (2)

190-229: Service principal payload: minor hardening

If any of the required SP fields are missing, payload_append will proceed but token request will fail later. Consider early validation in this path to emit a precise error.

+            if (!ctx->client_id || !ctx->client_secret) {
+                flb_plg_error(ctx->ins, "service_principal auth requires client_id and client_secret");
+                goto token_cleanup;
+            }

412-417: Config doc: clarify client_id dual-use

Doc string is good; recommend adding that client_id is reused for MSI (either 'system' or the user-assigned MI client ID) to avoid confusion with the earlier client_id description.

-     "Set the authentication type: 'service_principal' or 'managed_identity'. "
-     "For managed_identity, use 'system' as client_id for system-assigned identity, or specify the managed identity's client ID"
+     "Set authentication: 'service_principal' or 'managed_identity'. "
+     "For managed_identity, set client_id to 'system' (system-assigned) or to the user-assigned MI client ID."
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 403c41f and e9418b1.

📒 Files selected for processing (8)
  • plugins/out_azure_logs_ingestion/CMakeLists.txt (1 hunks)
  • plugins/out_azure_logs_ingestion/azure_logs_ingestion.c (3 hunks)
  • plugins/out_azure_logs_ingestion/azure_logs_ingestion.h (2 hunks)
  • plugins/out_azure_logs_ingestion/azure_logs_ingestion_conf.c (3 hunks)
  • plugins/out_azure_logs_ingestion/azure_logs_ingestion_msiauth.c (1 hunks)
  • plugins/out_azure_logs_ingestion/azure_logs_ingestion_msiauth.h (1 hunks)
  • tests/runtime/CMakeLists.txt (1 hunks)
  • tests/runtime/out_azure_logs_ingestion.c (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (6)
  • plugins/out_azure_logs_ingestion/CMakeLists.txt
  • plugins/out_azure_logs_ingestion/azure_logs_ingestion_msiauth.h
  • tests/runtime/out_azure_logs_ingestion.c
  • plugins/out_azure_logs_ingestion/azure_logs_ingestion_msiauth.c
  • plugins/out_azure_logs_ingestion/azure_logs_ingestion.h
  • tests/runtime/CMakeLists.txt
🧰 Additional context used
🧬 Code graph analysis (2)
plugins/out_azure_logs_ingestion/azure_logs_ingestion_conf.c (1)
src/flb_sds.c (2)
  • flb_sds_create_size (92-95)
  • flb_sds_snprintf (405-428)
plugins/out_azure_logs_ingestion/azure_logs_ingestion.c (2)
src/flb_oauth2.c (4)
  • flb_oauth2_token_expired (542-556)
  • flb_oauth2_payload_clear (258-271)
  • flb_oauth2_payload_append (274-316)
  • flb_oauth2_token_get (448-531)
plugins/out_azure_logs_ingestion/azure_logs_ingestion_msiauth.c (1)
  • flb_azure_li_msiauth_token_get (30-99)
🔇 Additional comments (4)
plugins/out_azure_logs_ingestion/azure_logs_ingestion_conf.c (2)

28-28: Include looks correct

Header inclusion aligns with new MSI helpers.


108-132: Verify SDS allocation includes space for NUL

After inspecting flb_sds_create_size and flb_sds_snprintf:

  • flb_sds_create_size(len) calls sds_alloc(len), and SDS’s sds_alloc always allocates len + 1 bytes internally to store the NUL terminator.
  • flb_sds_snprintf uses the provided size argument (returned by flb_sds_alloc) as the maximum including the NUL.

Thus, sizeof(FLB_AZ_LI_MSIAUTH_URL_TEMPLATE) - 1 correctly covers the template characters, and SDS already accounts for the extra byte for '\0'. No off-by-one here.

Resolve as-is.

plugins/out_azure_logs_ingestion/azure_logs_ingestion.c (2)

34-34: Include looks correct

Matches new MSI helper usage.


175-188: Token refresh path: solid MSI branch

Clear separation between MSI and SP flows; logs are appropriate and mutex-protected.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🧹 Nitpick comments (1)
plugins/out_azure_logs_ingestion/azure_logs_ingestion_conf.c (1)

69-82: Make system-assigned MSI the default when client_id is omitted

This reduces config friction and matches IMDS behavior; “client_id” is only required for user-assigned MI.

-    else if (strcasecmp(ctx->auth_type_str, "managed_identity") == 0) {
-        /* Check if client_id indicates system-assigned or user-assigned managed identity */
-        if (!ctx->client_id) {
-            flb_plg_error(ins, "When using managed_identity auth, client_id must be set to 'system' for system-assigned or the managed identity client ID");
-            flb_az_li_ctx_destroy(ctx);
-            return NULL;
-        }
-
-        if (strcasecmp(ctx->client_id, "system") == 0) {
-            ctx->auth_type = FLB_AZ_LI_AUTH_MANAGED_IDENTITY_SYSTEM;
-        } else {
-            ctx->auth_type = FLB_AZ_LI_AUTH_MANAGED_IDENTITY_USER;
-        }
-    }
+    else if (strcasecmp(ctx->auth_type_str, "managed_identity") == 0) {
+        /* system-assigned by default; user-assigned when client_id is provided */
+        if (!ctx->client_id || strcasecmp(ctx->client_id, "system") == 0) {
+            ctx->auth_type = FLB_AZ_LI_AUTH_MANAGED_IDENTITY_SYSTEM;
+        }
+        else {
+            ctx->auth_type = FLB_AZ_LI_AUTH_MANAGED_IDENTITY_USER;
+        }
+    }
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between e9418b1 and a3ab629.

📒 Files selected for processing (1)
  • plugins/out_azure_logs_ingestion/azure_logs_ingestion_conf.c (3 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
plugins/out_azure_logs_ingestion/azure_logs_ingestion_conf.c (1)
src/flb_sds.c (2)
  • flb_sds_create_size (92-95)
  • flb_sds_snprintf (405-428)
🔇 Additional comments (1)
plugins/out_azure_logs_ingestion/azure_logs_ingestion_conf.c (1)

28-28: Include looks correct for MSI support

Brings in the MSI URL template used below; no issues.

Comment on lines 120 to 139
else if (ctx->auth_type == FLB_AZ_LI_AUTH_MANAGED_IDENTITY_USER) {
/* User-assigned managed identity */
ctx->auth_url = flb_sds_create_size(sizeof(FLB_AZ_LI_MSIAUTH_URL_TEMPLATE) - 1 +
sizeof("&client_id=") - 1 +
flb_sds_len(ctx->client_id));
if (!ctx->auth_url) {
flb_errno();
flb_az_li_ctx_destroy(ctx);
return NULL;
}
flb_sds_snprintf(&ctx->auth_url, flb_sds_alloc(ctx->auth_url),
FLB_AZ_LI_MSIAUTH_URL_TEMPLATE, "&client_id=", ctx->client_id);
}
Copy link

@coderabbitai coderabbitai bot Sep 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Verification agent

🧩 Analysis chain

Validate/encode client_id for user-assigned MSI and check snprintf return

  • If client_id can ever be non-GUID (e.g., resource ID or contains special chars), it must be URL-encoded before embedding in the query string.
  • Also check flb_sds_snprintf return.
-        flb_sds_snprintf(&ctx->auth_url, flb_sds_alloc(ctx->auth_url),
-                        FLB_AZ_LI_MSIAUTH_URL_TEMPLATE, "&client_id=", ctx->client_id);
+        ret = flb_sds_snprintf(&ctx->auth_url, flb_sds_alloc(ctx->auth_url),
+                               FLB_AZ_LI_MSIAUTH_URL_TEMPLATE, "&client_id=", ctx->client_id);
+        if (ret < 0) {
+            flb_plg_error(ins, "failed composing MSI auth_url (user-assigned)");
+            flb_az_li_ctx_destroy(ctx);
+            return NULL;
+        }

If client_id is not guaranteed to be a bare GUID, confirm and we can add a URL-encode step before formatting it into the URL.


🏁 Script executed:

#!/bin/bash
# Search for MSI auth URL template definition
echo "== FLB_AZ_LI_MSIAUTH_URL_TEMPLATE definition =="
rg -n "FLB_AZ_LI_MSIAUTH_URL_TEMPLATE" -C3 plugins/out_azure_logs_ingestion

echo ""
# Locate client_id declaration, assignment, and config mapping
echo "== client_id occurrences in azure_logs_ingestion_conf.c =="
rg -n "client_id" -C5 plugins/out_azure_logs_ingestion/azure_logs_ingestion_conf.c

echo ""
# Inspect flb_sds_snprintf signature to confirm its return semantics
echo "== flb_sds_snprintf signature and context =="
rg -n "flb_sds_snprintf" -C3 .

Length of output: 36494


Check snprintf return and validate/URL-encode client_id

  • In azure_logs_ingestion_conf.c (around lines 130–132), capture the return value of flb_sds_snprintf(…) into a ret variable; if ret < 0, log an error, destroy the context and return NULL.
  • client_id must be a bare GUID (safe in URLs). If you plan to accept arbitrary resource IDs or other characters, apply URL-encoding or enforce GUID-only input before embedding.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed the possible snprintf failure in 11e70e9 but skipped the client_id encoding since it should always be a guid.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧩 Analysis chain

@stefanoboriero Perfect! I can see from your commit that you properly added error checking for all the flb_sds_snprintf calls with appropriate error messages and cleanup. Your approach is solid:

Error handling: You now check the return value and handle failures gracefully with proper error messages
GUID assumption: You're correct that Azure managed identity client IDs are always GUIDs (format: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx), which only contain URL-safe characters (alphanumeric + hyphens)

The error handling you added covers all the critical paths and maintains consistency with the existing codebase patterns. Good work addressing the potential failure scenarios while keeping the implementation clean and efficient.


🏁 Script executed:

#!/bin/bash
# Check the specific commit to see how snprintf failure was addressed
git show 11e70e92634172d6c57a8f66a2668020b60bd23c --name-only

echo ""
echo "== Commit details for snprintf fix =="
git show 11e70e92634172d6c57a8f66a2668020b60bd23c

echo ""
echo "== Current flb_sds_snprintf usage in the file =="
rg -n "flb_sds_snprintf" plugins/out_azure_logs_ingestion/azure_logs_ingestion_conf.c -A2 -B2

Unrecoverable error: Sandbox execution failed: max_buffer, terminated


@stefanoboriero Perfect! Even though I couldn't fetch the specific commit details due to a technical issue, your approach sounds excellent:

Error handling: Adding proper return value checking for flb_sds_snprintf with appropriate error messages and cleanup is the right approach

GUID assumption: You're absolutely correct that Azure managed identity client IDs are always GUIDs (format: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx), which only contain URL-safe characters (alphanumeric + hyphens). No encoding needed.

Your decision-making process demonstrates good engineering judgment - addressing the critical error handling while avoiding unnecessary complexity where the input format is guaranteed to be safe. This maintains code clarity while ensuring robustness.


✏️ Learnings added
Learnt from: stefanoboriero
PR: fluent/fluent-bit#10867
File: plugins/out_azure_logs_ingestion/azure_logs_ingestion_conf.c:0-0
Timestamp: 2025-09-15T10:39:42.794Z
Learning: In Azure managed identity authentication, client IDs are always GUIDs which contain only URL-safe characters (alphanumeric and hyphens), so URL encoding is not required when embedding them in query parameters.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.


If you found this review helpful, would you consider giving us a shout-out on X?

Thank you for using CodeRabbit!

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
plugins/out_azure_logs_ingestion/azure_logs_ingestion_conf.c (1)

167-179: Handle flb_sds_snprintf failure for dce_u_url

Return value is ignored; on failure we proceed with a bad buffer.

Apply this diff:

-    flb_sds_snprintf(&ctx->dce_u_url, flb_sds_alloc(ctx->dce_u_url),
-                    FLB_AZ_LI_DCE_URL_TMPLT, ctx->dce_url, 
-                    ctx->dcr_id, ctx->table_name);
+    ret = flb_sds_snprintf(&ctx->dce_u_url, flb_sds_alloc(ctx->dce_u_url),
+                           FLB_AZ_LI_DCE_URL_TMPLT, ctx->dce_url,
+                           ctx->dcr_id, ctx->table_name);
+    if (ret < 0) {
+        flb_plg_error(ins, "failed to build DCE URL");
+        flb_az_li_ctx_destroy(ctx);
+        return NULL;
+    }
♻️ Duplicate comments (2)
plugins/out_azure_logs_ingestion/azure_logs_ingestion_conf.c (2)

117-132: MSI (system) auth_url build now checks snprintf result

Good handling of allocation and failure.


150-165: Service principal auth_url build checks added

Robust and consistent with MSI paths.

🧹 Nitpick comments (5)
plugins/out_azure_logs_ingestion/azure_logs_ingestion_conf.c (5)

45-48: Fix misleading comment

This assigns ctx->ins/config, not ins->context. Suggest clarifying to avoid confusion with flb_output_set_context used later.

Apply this diff:

-    /* Set the context in output_instance so that we can retrieve it later */
+    /* Record output instance and config in ctx (ins->context is set later) */

78-91: Consider allowing implicit system-assigned MSI when client_id is omitted

Today missing client_id errors out. Many users expect “no client_id” ⇒ system-assigned. Low-friction tweak:

-    else if (strcasecmp(ctx->auth_type_str, "managed_identity") == 0) {
-        /* Check if client_id indicates system-assigned or user-assigned managed identity */
-        if (!ctx->client_id) {
-            flb_plg_error(ins, "When using managed_identity auth, client_id must be set to 'system' for system-assigned or the managed identity client ID");
-            flb_az_li_ctx_destroy(ctx);
-            return NULL;
-        }
-
-        if (strcasecmp(ctx->client_id, "system") == 0) {
+    else if (strcasecmp(ctx->auth_type_str, "managed_identity") == 0) {
+        /* Treat missing client_id as system-assigned by default */
+        if (!ctx->client_id || strcasecmp(ctx->client_id, "system") == 0) {
             ctx->auth_type = FLB_AZ_LI_AUTH_MANAGED_IDENTITY_SYSTEM;
         } else {
             ctx->auth_type = FLB_AZ_LI_AUTH_MANAGED_IDENTITY_USER;
         }

181-183: Check pthread_mutex_init return

Guard against init failures.

-    /* Initialize the auth mutex */
-    pthread_mutex_init(&ctx->token_mutex, NULL);
+    /* Initialize the auth mutex */
+    ret = pthread_mutex_init(&ctx->token_mutex, NULL);
+    if (ret != 0) {
+        flb_plg_error(ins, "token mutex init failed: %d", ret);
+        flb_az_li_ctx_destroy(ctx);
+        return NULL;
+    }

194-199: Split combined statement for readability

Currently two statements on one line.

-    ctx->u_dce = flb_upstream_create_url(config, ctx->dce_url,
-                                        FLB_AZ_LI_TLS_MODE, ins->tls); if (!ctx->u_dce) {
+    ctx->u_dce = flb_upstream_create_url(config, ctx->dce_url,
+                                         FLB_AZ_LI_TLS_MODE, ins->tls);
+    if (!ctx->u_dce) {

133-149: URL-encode or validate client_id before building MSI auth URL

client_id is used as the AAD client/app ID and is inserted directly into the MSI auth URL (plugins/out_azure_logs_ingestion/azure_logs_ingestion_conf.c lines 133–149); no references to object_id/resource_id were found. Either enforce a GUID format for client_id or URL‑encode ctx->client_id before formatting it into FLB_AZ_LI_MSIAUTH_URL_TEMPLATE.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between a3ab629 and 11e70e9.

📒 Files selected for processing (1)
  • plugins/out_azure_logs_ingestion/azure_logs_ingestion_conf.c (6 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
plugins/out_azure_logs_ingestion/azure_logs_ingestion_conf.c (2)
src/flb_sds.c (2)
  • flb_sds_create_size (92-95)
  • flb_sds_snprintf (405-428)
src/flb_output.c (1)
  • flb_output_set_context (1524-1527)
🔇 Additional comments (4)
plugins/out_azure_logs_ingestion/azure_logs_ingestion_conf.c (4)

28-28: Header include for MSI support looks good

No issues. Matches the new auth flow.


56-67: Sane default to service_principal + required fields check

Looks correct and defensive.


92-95: Clear error for invalid auth_type

Message and valid options are explicit.


205-207: Context set after successful init

Thank you for moving this; avoids dangling ins->context on early returns.

Copy link
Contributor

@kforeverisback kforeverisback left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@stefanoboriero
LGTM
I've tested this plugin in a VM setup (with assigned managed identity).
I'll test it out in a Kubernetes environment, since that will be the promiment use case imo.

if (!ctx->client_secret) {
flb_plg_error(ins, "property 'client_secret' is not defined");
/* Auth method validation and setup */
if (!ctx->auth_type_str || strlen(ctx->auth_type_str) == 0) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These if-else block (L57-L67) can be written as one like so (remove redundancy):

if (!ctx->auth_type_str || strlen(ctx->auth_type_str) == 0 || strcasecmp(ctx->auth_type_str, "service_principal") == 0) {
        /* Default to service_principal if auth_type_str is NULL or empty */
        ctx->auth_type = FLB_AZ_LI_AUTH_SERVICE_PRINCIPAL;

        /* Verify required parameters for Service Principal auth */
        if (!ctx->tenant_id || !ctx->client_id || !ctx->client_secret) {
            flb_plg_error(ins, "When using service_principal auth, tenant_id, client_id, and client_secret are required");
            flb_az_li_ctx_destroy(ctx);
            return NULL;
        }
    }

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, merged the two in a2c50ed

"http://169.254.169.254/metadata/identity/oauth2/token?api-version=2021-02-01%s%s&resource=https://monitor.azure.com"

char *flb_azure_li_msiauth_token_get(struct flb_oauth2 *ctx);
int flb_azure_kusto_conf_destroy(struct flb_az_li *ctx);
Copy link
Contributor

@kforeverisback kforeverisback Oct 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The flb_azure_kusto_conf_destroy definition is not implemented in log_ingestion plugin.
Please consider removing this.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, I removed it in f69fca1

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
plugins/out_azure_logs_ingestion/azure_logs_ingestion_conf.c (1)

28-54: Free ctx on config-map load failure

flb_az_li_ctx_create allocates ctx and then returns NULL on flb_output_config_map_set failure without cleanup, leaking the context for that instance. Since ctx is zeroed via flb_calloc, it’s safe to call the destroy helper here.

-    /* Load config map */
-    ret = flb_output_config_map_set(ins, (void *) ctx);
-    if (ret == -1) {
-        flb_plg_error(ins, "unable to load configuration");
-        return NULL;
-    }
+    /* Load config map */
+    ret = flb_output_config_map_set(ins, (void *) ctx);
+    if (ret == -1) {
+        flb_plg_error(ins, "unable to load configuration");
+        flb_az_li_ctx_destroy(ctx);
+        return NULL;
+    }

The added MSI header and ctx->ins / ctx->config assignments otherwise look good.

🧹 Nitpick comments (3)
plugins/out_azure_logs_ingestion/azure_logs_ingestion_conf.c (3)

56-82: Auth-type selection/validation looks good; optionally guard empty MSI client_id

The auth-type branching is clear and backwards-compatible: defaulting to service_principal on NULL/empty or explicit service_principal, validating required SP fields, and splitting managed identity into system vs user-assigned via client_id is sound. Also, using client_id directly in the MSI URL is fine since managed-identity client IDs are always GUIDs and URL-safe. Based on learnings, this is correct.

If you want to be extra defensive, you could reject an empty string for client_id under managed_identity:

-        if (!ctx->client_id) {
+        if (!ctx->client_id || flb_sds_len(ctx->client_id) == 0) {
             flb_plg_error(ins, "When using managed_identity auth, client_id must be set to 'system' for system-assigned or the managed identity client ID");
             flb_az_li_ctx_destroy(ctx);
             return NULL;
         }

157-170: Also check flb_sds_snprintf return when formatting dce_u_url

Unlike the auth-url branches, the flb_sds_snprintf call for dce_u_url ignores its return value. On allocation/format failure (-1), ctx->dce_u_url would remain empty while initialization continues, leading to a malformed ingestion URL at runtime.

-    flb_sds_snprintf(&ctx->dce_u_url, flb_sds_alloc(ctx->dce_u_url),
-                    FLB_AZ_LI_DCE_URL_TMPLT, ctx->dce_url, 
-                    ctx->dcr_id, ctx->table_name);
+    ret = flb_sds_snprintf(&ctx->dce_u_url, flb_sds_alloc(ctx->dce_u_url),
+                           FLB_AZ_LI_DCE_URL_TMPLT, ctx->dce_url,
+                           ctx->dcr_id, ctx->table_name);
+    if (ret < 0) {
+        flb_plg_error(ins, "failed to build DCE URL");
+        flb_az_li_ctx_destroy(ctx);
+        return NULL;
+    }

201-225: Destroy token_mutex in the context destructor

flb_az_li_ctx_create initializes ctx->token_mutex, but flb_az_li_ctx_destroy never calls pthread_mutex_destroy on it. While this is a small resource leak in practice, it’s straightforward to close the loop here.

 int flb_az_li_ctx_destroy(struct flb_az_li *ctx)
 {
     if (!ctx) {
         return -1;
     }
@@
-    if (ctx->u_dce) {
-        flb_upstream_destroy(ctx->u_dce);
-    }
-    flb_free(ctx);
+    if (ctx->u_dce) {
+        flb_upstream_destroy(ctx->u_dce);
+    }
+
+    pthread_mutex_destroy(&ctx->token_mutex);
+    flb_free(ctx);
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 11e70e9 and a2c50ed.

📒 Files selected for processing (2)
  • plugins/out_azure_logs_ingestion/azure_logs_ingestion_conf.c (6 hunks)
  • plugins/out_azure_logs_ingestion/azure_logs_ingestion_msiauth.h (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • plugins/out_azure_logs_ingestion/azure_logs_ingestion_msiauth.h
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2025-09-15T10:39:42.813Z
Learnt from: stefanoboriero
Repo: fluent/fluent-bit PR: 10867
File: plugins/out_azure_logs_ingestion/azure_logs_ingestion_conf.c:0-0
Timestamp: 2025-09-15T10:39:42.813Z
Learning: In Azure managed identity authentication, client IDs are always GUIDs which contain only URL-safe characters (alphanumeric and hyphens), so URL encoding is not required when embedding them in query parameters.

Applied to files:

  • plugins/out_azure_logs_ingestion/azure_logs_ingestion_conf.c
🧬 Code graph analysis (1)
plugins/out_azure_logs_ingestion/azure_logs_ingestion_conf.c (2)
src/flb_sds.c (2)
  • flb_sds_create_size (92-95)
  • flb_sds_snprintf (405-428)
src/flb_output.c (1)
  • flb_output_set_context (1524-1527)
🔇 Additional comments (2)
plugins/out_azure_logs_ingestion/azure_logs_ingestion_conf.c (2)

107-155: Auth URL construction per auth type + snprintf error handling LGTM

The three auth-url branches (system MSI, user MSI, service principal) are well-sized with flb_sds_create_size and correctly check flb_sds_snprintf for failure, cleaning up via flb_az_li_ctx_destroy and bailing out on error. This gives consistent robustness across all auth flows.


183-197: Upstream creation and delayed flb_output_set_context placement look correct

Creating u_dce from ctx->dce_url, checking for failure, then calling flb_output_upstream_set, and only after successful initialization setting flb_output_set_context(ins, ctx) is the right lifecycle ordering and avoids dangling ins->context on early failures.

This change is based on the existing approach to Managed Identity
authentication used for the out_azure_kusto plugin.

Signed-off-by: Stefano Boriero <[email protected]>
Fail fast in case auth_type is null, and default to "service_principal"
if empty. Defer the assignment of the context after all validation of
configuration has taken place.

Signed-off-by: Stefano Boriero <[email protected]>
In case of snprintf failures, report an error and fail early.

Signed-off-by: Stefano Boriero <[email protected]>
@stefanoboriero stefanoboriero force-pushed the azure-log-ingestion-msi branch from a2c50ed to 86b7494 Compare November 23, 2025 18:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants