Fix config_InitializeHub failure caused by HTTP redirects in Azure Data Explorer externaldata commands

[Copilot]

🛠️ Description

GitHub releases download URLs now return HTTP 302 redirects. Azure Data Explorer's externaldata() command rejects redirects, causing config_InitializeHub pipeline failures when loading open data CSV files (PricingUnits, Regions, ResourceTypes, Services).

Changed URL pattern:

// Before: GitHub releases (redirects to storage)
'https://github.com/microsoft/finops-toolkit/releases/download/v{version}/PricingUnits.csv'

// After: Raw content (direct access, no redirect)
'https://raw.githubusercontent.com/microsoft/finops-toolkit/refs/tags/v{version}/src/open-data/PricingUnits.csv'

Files modified:

• Updated ftkReleaseUri variable in src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/app.bicep
• Updated changelog in docs-mslearn/toolkit/changelog.md

Note: The compiled deployment templates in docs/deploy/ will be updated during the official release build process.

Affects all FinOps Hub deployments using Azure Data Explorer or Microsoft Fabric.

Related to #1885

📷 Screenshots

📋 Checklist

🔬 How did you test this change?

• 🤏 Lint tests
• 🤞 PS -WhatIf / az validate
• 👍 Manually deployed + verified
• 💪 Unit tests
• 🙌 Integration tests

🙋‍♀️ Do any of the following that apply?

• 🚨 This is a breaking change.
• 🤏 The change is less than 20 lines of code.

📑 Did you update docs/changelog.md?

• ✅ Updated changelog (required for dev PRs)
• ➡️ Will add log in a future PR (feature branch PRs only)
• ❎ Log not needed (small/internal change)

📖 Did you update documentation?

• ✅ Public docs in docs (required for dev)
• ✅ Internal dev docs in src (required for dev)
• ➡️ Will add docs in a future PR (feature branch PRs only)
• ❎ Docs not needed (small/internal change)

Original prompt

---

This section details on the original issue you should resolve

<issue_title>[ADF] config_InitializeHub is failing: "HTTP redirects are not allowed" when loading open data from github release v12 via KQL "externaldata" command </issue_title>
<issue_description>## 🐛 Problem
When initializing the FinOps Hub the step that populates the PricingUnits table fails.
The KQL command uses externaldata to read PricingUnits.csv from a GitHub releases URL and returns

HTTP client error: redirects are not allowed

so the PricingUnits table is not created or updated.

The failing command:

.set-or-replace PricingUnits <|
    externaldata(x_PricingUnitDescription: string, AccountTypes: string, x_PricingBlockSize: real, PricingUnit: string)
    [@"https://github.com/microsoft/finops-toolkit/releases/download/v12/PricingUnits.csv"]
    with (format = "csv", ignoreFirstRecord = true)
    | project-away AccountTypes

This looks like the GitHub releases download URL now returns a redirect and Azure Data Explorer does not follow redirects for externaldata().

👣 Repro steps

1. Deploy the FinOps Hub (v12) with Azure Data Explorer as the hub database.
2. In Data Factory run pipeline config_InitializeHub.
3. Inside the pipeline the Azure Data Explorer command activity Upserts PricingUnits runs the command above.
4. The activity fails with a partial query failure and the error message HTTP client error: redirects are not allowed.

Optional confirmation:
5. Run the same .set-or-replace PricingUnits command directly in the Kusto Web UI against the hub database.
6. Observe the same error.

🤔 Expected

The pipeline config_InitializeHub should complete successfully and the PricingUnits table should be created or updated from PricingUnits.csv without errors.

📷 Screenshots

1. Kusto Web UI showing the failing query and error message.

2. Data Factory pipeline config_InitializeHub with the Azure Data Explorer command activity that runs the same statement.

ℹ️ Additional context

• Environment: FinOps Hub v12, Azure Data Explorer as hub, Data Factory based deployment.

• The error text in Kusto is:

  Partial query failure: Unspecified error (message: HTTP client error: redirects are not allowed, details: StgError { kind: Generic("HTTP client error: redirects are not allowed"), source: None })

• This started to happen when using the GitHub releases URL
  https://github.com/microsoft/finops-toolkit/releases/download/v12/PricingUnits.csv
  which appears to return a redirect to another host.

• As a workaround the CSV can be copied to a storage account and referenced from there with a direct URL that does not redirect.

• Same issue with fabric based deployments

• Same issue with Region.csv, ResourceTypes.csv, Services.csv

Suggested fix ideas:

• Host PricingUnits.csv at a non redirecting URL (for example under raw.githubusercontent.com or in the repo itself) and update the KQL script.
• Or include the pricing units as an inline datatable during initialization.

🙋‍♀️ Ask for the community

We could use your help:

1. Please vote this issue up (👍) if you are also seeing the same failure in your FinOps Hub deployment.
2. Add comments if you have confirmation on which URLs work correctly with externaldata or if you have a better suggestion for hosting PricingUnits.csv.

</issue_description>

## Comments on the Issue (you are @copilot in this section)

<comments>
<comment_new><author>@RolandKrummenacher</author><body>
Related to microsoft/finops-toolkit#1885 </body></comment_new>
<comment_new><author>@RolandKrummenacher</author><body>
We could use this base url for PriceingUnits, Regions, ResourceTypes and Services Open Data: https://raw.githubusercontent.com/microsoft/finops-toolkit/refs/tags/v12/src/open-data/PricingUnits.csv</body></comment_new>
</comments>

• Fixes [ADF] config_InitializeHub is failing: "HTTP redirects are not allowed" when loading open data from github release v12 via KQL "externaldata" command  #1892

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[RolandKrummenacher]

@copilot Why have you made changes in docs/deploy? The issue is not related to remoteHub configuration. Keep the change log and the changes in app.bicep.

[Copilot]

@copilot Why have you made changes in docs/deploy? The issue is not related to remoteHub configuration. Keep the change log and the changes in app.bicep.

You're right - the docs/deploy changes included unrelated remoteHub configuration that was already in the source but hadn't been compiled yet. I've reverted those files in commit 6aa9627, keeping only the app.bicep and changelog changes. The docs/deploy JSON files will be properly updated during the official release build process.

[RolandKrummenacher]

@MSBrett , @flanakin Is this a proper fix for the problem?