Fix backward compatibility for pre-FOCUS 1.2 cost exports in Costs_transform_v1_2()

[Copilot]

🛠️ Description

Pre-FOCUS 1.2 exports store values in legacy x_ columns (x_PricingCurrency, x_SkuMeterName, x_SkuDetails). When ingested into v1.2 hubs, root-level fields remained null.

Changes:

Added coalesce fallbacks in Costs_transform_v1_2() project clause:

PricingCurrency = coalesce(PricingCurrency, x_PricingCurrency),
SkuMeter = coalesce(SkuMeter, x_SkuMeterName),
SkuPriceDetails = coalesce(SkuPriceDetails, x_SkuDetails),

Files modified:

• src/templates/finops-hub/.../IngestionSetup_v1_2.kql - Added fallback mappings (3 lines)
• docs-mslearn/toolkit/changelog.md - Documented fix

📷 Screenshots

N/A

📋 Checklist

🔬 How did you test this change?

• 🤏 Lint tests
• 🤞 PS -WhatIf / az validate
• 👍 Manually deployed + verified
• 💪 Unit tests
• 🙌 Integration tests

🙋‍♀️ Do any of the following that apply?

• 🚨 This is a breaking change.
• 🤏 The change is less than 20 lines of code.

📑 Did you update docs/changelog.md?

• ✅ Updated changelog (required for dev PRs)
• ➡️ Will add log in a future PR (feature branch PRs only)
• ❎ Log not needed (small/internal change)

📖 Did you update documentation?

• ✅ Public docs in docs (required for dev)
• ✅ Internal dev docs in src (required for dev)
• ➡️ Will add docs in a future PR (feature branch PRs only)
• ❎ Docs not needed (small/internal change)

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Costs_transform_v1_2() does not correctly support ingesting Cost Management exports that predate FOCUS 1.2.</issue_title>
<issue_description>## 🐛 Problem

Costs_transform_v1_2() does not correctly support ingesting Cost Management exports that predate FOCUS 1.2.

Older exports store values in legacy x_ columns (for example x_PricingCurrency, x_SkuMeterName, x_SkuDetails). When these exports are ingested into a v1_2 hub, the corresponding root-level fields (PricingCurrency, SkuMeter, SkuPriceDetails) remain null because the transform does not consistently fall back to the legacy fields.

This breaks backward compatibility when customers ingest historical data that was exported before FOCUS 1.2.

The attached snippet shows the exact fix: add coalesce(...) fallback projections for PricingCurrency, SkuMeter, and SkuPriceDetails.

Additionally, Prices_transform_v1_0() should be checked for the inverse mismatch to ensure pricing enrichment remains compatible when older exports are used.

---

👣 Repro steps

1. Deploy FinOps Hub (v1_2 ingestion setup).
2. Ingest Cost Management exports created before FOCUS 1.2 (older export schema with legacy x_ fields).
3. Query the transformed data (e.g. Costs_final_v1_2 or the output of Costs_transform_v1_2()).
4. Observe that:
   • PricingCurrency is null even though x_PricingCurrency has a value
   • SkuMeter is null even though x_SkuMeterName has a value
   • SkuPriceDetails is null even though x_SkuDetails has a value

---

🤔 Expected

Update the project clause in Costs_transform_v1_2() to include the following fallback mappings so that older (pre 1.2) exports are supported:

• PricingCurrency = coalesce(PricingCurrency, x_PricingCurrency),
• SkuMeter = coalesce(SkuMeter, x_SkuMeterName),
• SkuPriceDetails = coalesce(SkuPriceDetails, x_SkuDetails),

Acceptance criteria:

• For rows ingested from pre 1.2 exports, if the root-level field is null and the legacy x_ field is non-null, the output must equal the legacy value for all three fields above.

Also validate Costs_transform_v1_0() for the inverse mismatch and update only if necessary for compatibility.

---

📷 Screenshots

See attached snippet showing the exact projection changes required.

---

ℹ️ Additional context

File:

• src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts/IngestionSetup_v1_2.kql

Functions:

• Implement change: Costs_transform_v1_2()
• Validate for inverse mismatch: Costs_transform_v1_0()

Scope:

• This issue is intentionally limited to backward compatibility for pre 1.2 cost exports and the three fallback mappings above.

---

🙋‍♀️ Ask for the community

We could use your help:

1. Please vote this issue up (👍) to prioritize it.
2. Comment if you are ingesting historical cost exports (pre FOCUS 1.2) and see missing PricingCurrency, SkuMeter, or SkuPriceDetails.
   </issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes Costs_transform_v1_2() does not correctly support ingesting Cost Management exports that predate FOCUS 1.2. #1905

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[RolandKrummenacher]

@copilot I've manually deployed the change and verified.