Fix ADF pipeline dependency logic for scope array handling

[Copilot]

• Review and understand the ADF pipeline dependency issue
• Identify all instances of the incorrect dependency pattern (3 pipelines)
• Fix dependency conditions in config_RunBackfillJob pipeline
• Fix dependency conditions in config_StartExportProcess pipeline
• Fix dependency conditions in config_ConfigureExports pipeline
• Update changelog with the fix
• Revert docs/deploy files (not part of the fix - generated files)
• Verify no other similar patterns exist
• Run code review (no issues found)
• Security review completed (low risk configuration change)

Summary

Fixed the ADF pipeline dependency logic in the source Bicep file. The docs/deploy files are generated during the build/release process and should not be manually modified.

Changes Made

• Source file: src/templates/finops-hub/modules/Microsoft.CostManagement/ManagedExports/app.bicep - Added 'Failed' to dependency conditions in 3 pipelines
• Changelog: docs-mslearn/toolkit/changelog.md - Documented the fix under FinOps hubs v13

The fix allows the "Filter Invalid Scopes" activity to proceed when "Save/Set Scopes" fails (expected when scopes is a single object instead of an array).

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Manifest file not generated after successful FinOps Hub V12 deployment – ADF pipeline not triggering</issue_title>
<issue_description>### Issue Description

I’ve successfully deployed FinOps Hub V12 into my Azure Resource Group.
Post-deployment, I performed the following steps:

• Registered the Microsoft.EventGrid and Microsoft.CostManagementExports resource providers on the target subscription.
• Granted the Cost Management Contributor RBAC role to the ADF Managed Identity on the target subscription.
• Added the subscription to the scopes section in the settings.json file within the config container.

Observed Behavior

After some investigation, I noticed that:

• No export is created on the Cost Management page.
• As a result, the manifest file is not being created in the msexports container.
• Consequently, the msexports_ExecuteETL ADF pipeline is not triggered, and no cost data is ingested into the ingestion container.

Investigation Details

I looked into the config_ConfigureExports pipeline, which is expected to automatically configure exports.
The Filter Invalid Scopes activity seems to have incorrect dependency logic.

• The Save Scope activity keeps failing.
• Even if Save Scopes as Array succeeds, the pipeline does not proceed to Filter Invalid Scopes because of the dependsOn configuration.

Current logic requires both conditions to be true:

• Save Scopes as Array → Succeeded or Skipped
• Save Scopes → Succeeded

Here’s the current configuration:

{ // Filter Invalid Scopes
  "name": "Filter Invalid Scopes",
  "type": "Filter",
  "dependsOn": [
    {
      "activity": "Save Scopes",
      "dependencyConditions": [ "Succeeded" ]
    },
    {
      "activity": "Save Scopes as Array",
      "dependencyConditions": [ "Skipped", "Succeeded" ]
    }
  ],
  "userProperties": [],
  "typeProperties": {
    "items": {
      "value": "@variables('scopesArray')",
      "type": "Expression"
    },
    "condition": {
      "value": "@and(not(empty(item().scope)), not(equals(item().scope, '/')))",
      "type": "Expression"
    }
  }
}

Analysis

This logic seems incorrect because it enforces an AND condition.
In my case, Save Scopes fails, but Save Scopes as Array succeeds, so the filter never runs.

Workaround

As a temporary fix, I changed the dependency for Save Scopes to Completed (i.e., any terminal state: Succeeded, Failed, or Skipped).
This allows the pipeline to continue even if Save Scopes fails, which works for my scenario.

{
  "name": "Filter Invalid Scopes",
  "type": "Filter",
  "dependsOn": [
    {
      "activity": "Save scopes as array",
      "dependencyConditions": [ "Succeeded", "Skipped" ]
    },
    {
      "activity": "Save scopes",
      "dependencyConditions": [ "Completed" ]
    }
  ]
}

So my question:

Could you please advise me on how to handle this issue better? Because if my observation is correct, many user could face the same issue.</issue_description>

Comments on the Issue (you are @copilot in this section)

@RolandKrummenacher After investigation, I agree that the dependsOn is a logical AND. Therefore, the pipeline never gets executed when "Save scopes" has not "Completed". We need to add "Failed" to the conditions for "Save scopes". There are other pipelines that are using the same patterns (I've counted 3 places).

• Fixes Manifest file not generated after successful FinOps Hub V12 deployment – ADF pipeline not triggering #1873

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

[RolandKrummenacher]

@copilot Please don't touch any files in docs/deploy. They have nothing to do with this issue.

[RolandKrummenacher]

@copilot try again

[RolandKrummenacher]

[RolandKrummenacher]

@copilot try again

[Copilot]

@copilot try again

Reverted the docs/deploy files in commit a824d5d. Now only the source Bicep file and changelog are modified:

• src/templates/finops-hub/modules/Microsoft.CostManagement/ManagedExports/app.bicep (3 lines added)
• docs-mslearn/toolkit/changelog.md (1 line added)

The docs/deploy files will be regenerated during the build/release process.