Dependency Conflict: structured-merge-diff v4 vs v6 Incompatibility

[rshade]

Summary

We're experiencing a dependency conflict due to the Kubernetes ecosystem's migration from sigs.k8s.io/structured-merge-diff/v4 to sigs.k8s.io/structured-merge-diff/v6. This is causing incompatibility issues between upstream Kubernetes tooling and Pulumi's current dependency versions.

Problem Description

The issue stems from a bifurcation in the Kubernetes ecosystem:

• Upstream K8s tooling: Now using structured-merge-diff/v6
• Pulumi & pulumi-kubernetes: Still using structured-merge-diff/v4

This incompatibility was triggered by a recent commit in kube-openapi that bumped to v6:
kubernetes/kube-openapi@9bd5c66

Current Status

We've had to pin to the parent commit of the above change to unblock development temporarily.

Affected Dependencies

Based on our current go.mod (provider/go.mod:251):

sigs.k8s.io/structured-merge-diff/v4 v4.4.3

Related dependencies that may be affected:

• k8s.io/kube-openapi v0.0.0-20241127205056-99599406b04f
• Various k8s.io/* packages at v0.32.0

Impact

• Immediate: Blocking SDK updates and dependency management
• Long-term: Potential incompatibility with newer Kubernetes tooling and libraries
• Workaround: Currently pinning to older kube-openapi commit

Proposed Solution

1. Coordinate with upstream Pulumi SDK: Ensure both pulumi/pulumi and pulumi/pulumi-kubernetes migrate to structured-merge-diff/v6 together
2. Comprehensive dependency update: Update all related k8s.io packages to versions compatible with v6
3. Testing: Thorough testing across all supported Kubernetes versions
4. Timeline: Coordinate this change with the next major dependency update cycle

Dependencies to Update

The migration likely requires updating these package families:

• sigs.k8s.io/structured-merge-diff/v4 → sigs.k8s.io/structured-merge-diff/v6
• k8s.io/kube-openapi (to latest compatible version)
• Potentially other k8s.io/* and sigs.k8s.io/* packages

Testing Requirements

• Verify compatibility with all supported Kubernetes versions
• Test SDK generation and functionality
• Validate provider operations
• Check integration tests across all language SDKs

Related Issues

• Upstream change: kubernetes/kube-openapi@9bd5c66
• This affects both pulumi/pulumi and pulumi/pulumi-kubernetes repositories

[blampe]

Different major versions should be able to coexist by virtue of the version being part of the import path. I don't understand why this is manifesting as a blocker, because the code below works fine:

package main

import (
	_ "sigs.k8s.io/structured-merge-diff/v4/merge"
	_ "sigs.k8s.io/structured-merge-diff/v6/merge"
)

func main() {}

module structured-merge-diff

go 1.24.2

require (
	sigs.k8s.io/structured-merge-diff/v4 v4.7.0
	sigs.k8s.io/structured-merge-diff/v6 v6.2.0
)

require (
	github.com/json-iterator/go v1.1.12 // indirect
	github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
	github.com/modern-go/reflect2 v1.0.2 // indirect
	go.yaml.in/yaml/v2 v2.4.2 // indirect
	sigs.k8s.io/yaml v1.4.0 // indirect
)

[EronWright]

Which pulumi libraries are specifically being linked to, and why? Is this the dependency we seek to update (along with the rest of the k8s libraries)?

pulumi-kubernetes/provider/go.mod

Line 243 in 04cc31b

sigs.k8s.io/structured-merge-diff/v4 v4.6.0