OCPBUGS-64822: Implement upgrade blocking for conflicting ClusterImagePolicy named "openshift" #5397

QiWang19 · 2025-11-07T17:01:37Z

- What I did

Added logic to check if the resource openshift is customer-created and update the cluster operator status Upgradeable=False accordingly. The openshift CIP is cluster-managed reserved for release payload verification. This prevents upgrades when a conflicting policy is detected.

This check needs to be backported to 4.20.z as we plan to GA openshift ClusterImagePolicy (openshift/cluster-update-keys#85) in 4.21.

- How to verify it

launch cluster with this patch

4.21.0-0.nightly-2025-11-05-234508, openshift/machine-config-operator#5395 (gcp)

apply a ClusterImagePolicy name: openshift

oc create -f clusterimgpolicycr.yaml

# clusterimgpolicycr.yaml

apiVersion: config.openshift.io/v1
kind: ClusterImagePolicy
metadata:
  name: openshift
spec:
  scopes:
  - "example.com/test"
  policy:
    rootOfTrust:
      policyType: PublicKey
      publicKey:
        keyData: LS0tLS1CRUdJTiBQVUJMSUMgS0VZLS0tLS0KTUZrd0V3WUhLb1pJemowQ0FRWUlLb1pJemowREFRY0RRZ0FFVW9GVW9ZQVJlS1hHeTU5eGU1U1FPazJhSjhvKwoyL1l6NVk4R2NOM3pGRTZWaUl2a0duSGhNbEFoWGFYL2JvME05UjYyczAvNnErK1Q3dXdORnVPZzhBPT0KLS0tLS1FTkQgUFVCTElDIEtFWS0tLS0t
    signedIdentity:
      matchPolicy: RemapIdentity
      remapIdentity:
        prefix: example.com
        signedPrefix: mirror.com

$ oc get clusterimagepolicy
NAME        AGE
openshift   3m11s

Check the upgrade status has Upgradeable=False

$ oc adm upgrade
Cluster version is 4.21.0-0-2025-11-07-142257-test-ci-ln-6zj1wdt-latest

Upgradeable=False

  Reason: ConflictingClusterImagePolicy
  Message: Cluster operator machine-config should not be upgraded between minor versions: ClusterImagePolicy resource named 'openshift' conflicts with the cluster default ClusterImagePolicy object and blocks upgrades. Please delete the 'openshift' ClusterImagePolicy resource and reapply it with a different name if needed

warning: Cannot display available updates:
  Reason: NoChannel
  Message: The update channel has not been configured.

oc describe co

Name:         machine-config
Namespace:    
Labels:       <none>
Annotations:  exclude.release.openshift.io/internal-openshift-hosted: true
              include.release.openshift.io/self-managed-high-availability: true
              include.release.openshift.io/single-node-developer: true
API Version:  config.openshift.io/v1
Kind:         ClusterOperator
Metadata:
  Creation Timestamp:  2025-11-07T14:38:17Z
  Generation:          1
  Owner References:
    API Version:     config.openshift.io/v1
    Controller:      true
    Kind:            ClusterVersion
    Name:            version
    UID:             ca29e303-4d5f-4199-b400-5af02af7c412
  Resource Version:  36347
  UID:               b83c6e38-f19f-452a-a5cb-711c91b056ae
Spec:
Status:
  Conditions:
    ...
    Last Transition Time:  2025-11-07T15:20:35Z
    Message:               ClusterImagePolicy resource named 'openshift' conflicts with the cluster default ClusterImagePolicy object and blocks upgrades. Please delete the 'openshift' ClusterImagePolicy resource and reapply it with a different name if needed
    Reason:                ConflictingClusterImagePolicy
    Status:                False
    Type:                  Upgradeable
    ...

- Description for the changelog

Implement upgrade blocking for conflicting ClusterImagePolicy named "openshift"

…openshift" Added logic to check if the resource is customer-created and update the cluster operator status Upgradeable=False accordingly. This prevents upgrades when a conflicting policy is detected. This check needs to be backported to 4.20.z as we plan to GA `openshift` ClusterImagePolicy in 4.21. Signed-off-by: Qi Wang <[email protected]>

QiWang19 · 2025-11-07T17:23:44Z

/test unit

wking · 2025-11-07T17:55:19Z

pkg/operator/status.go

 	if configNode.Spec.CgroupMode == configv1.CgroupModeV1 {
 		coStatusCondition.Status = configv1.ConditionFalse
 		coStatusCondition.Reason = "ClusterOnCgroupV1"
 		coStatusCondition.Message = "Cluster is using deprecated cgroup v1 and is not upgradable. Please update the `CgroupMode` in the `nodes.config.openshift.io` object to 'v2'. Once upgraded, the cluster cannot be changed back to cgroup v1"


cgroup v1 was removed in 4.19. I dunno how this block still exists in the 4.20 controller? I'm also surprised that the API repo's dev branch still includes CgroupModeV1. Not something you need to deal with; I was just looking at this block while wondering how concerned I was about this function's "last complaint wins" competition over a single coStatusCondition.

@sairameshv FYI

Yes, it totally makes sense to remove this dead code from the clusters > 4.19
Created a PR here for the same
This can be back-ported to release-4.20 branch as well

wking

Seems like MCO maintainers may want to revisit how they maintain this function to prune obsolete content. But the code you're adding is just in the 4.20 branch, so there will be no dev-branch addition that needs future pruning. And the last-complaint-wins approach you're using follows existing precedent for DegradedPool and InterruptedBuild that will continue on in the dev branch. And the guard logic itself looks good to me. So:

/lgtm

openshift-ci-robot · 2025-11-07T18:02:13Z

@QiWang19: This pull request references Jira Issue OCPBUGS-64822, which is invalid:

expected the bug to target the "4.20.z" version, but no target version was set
release note text must be set and not match the template OR release note type must be set to "Release Note Not Required". For more information you can reference the OpenShift Bug Process.
expected dependent Jira Issue OCPBUGS-64823 to be in one of the following states: VERIFIED, RELEASE PENDING, CLOSED (ERRATA), CLOSED (CURRENT RELEASE), CLOSED (DONE), CLOSED (DONE-ERRATA), but it is Closed (Won't Do) instead
expected dependent Jira Issue OCPBUGS-64823 to target a version in 4.21.0, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

- What I did

Added logic to check if the resource openshift is customer-created and update the cluster operator status Upgradeable=False accordingly. The openshift CIP is cluster-managed reserved for release payload verification. This prevents upgrades when a conflicting policy is detected.

This check needs to be backported to 4.20.z as we plan to GA openshift ClusterImagePolicy (openshift/cluster-update-keys#85) in 4.21.

- How to verify it

launch cluster with this patch

4.21.0-0.nightly-2025-11-05-234508, openshift/machine-config-operator#5395 (gcp)

apply a ClusterImagePolicy name: openshift

oc create -f clusterimgpolicycr.yaml

# clusterimgpolicycr.yaml

apiVersion: config.openshift.io/v1
kind: ClusterImagePolicy
metadata:
 name: openshift
spec:
 scopes:
 - "example.com/test"
 policy:
   rootOfTrust:
     policyType: PublicKey
     publicKey:
       keyData: LS0tLS1CRUdJTiBQVUJMSUMgS0VZLS0tLS0KTUZrd0V3WUhLb1pJemowQ0FRWUlLb1pJemowREFRY0RRZ0FFVW9GVW9ZQVJlS1hHeTU5eGU1U1FPazJhSjhvKwoyL1l6NVk4R2NOM3pGRTZWaUl2a0duSGhNbEFoWGFYL2JvME05UjYyczAvNnErK1Q3dXdORnVPZzhBPT0KLS0tLS1FTkQgUFVCTElDIEtFWS0tLS0t
   signedIdentity:
     matchPolicy: RemapIdentity
     remapIdentity:
       prefix: example.com
       signedPrefix: mirror.com

$ oc get clusterimagepolicy
NAME        AGE
openshift   3m11s

Check the upgrade status has Upgradeable=False

$ oc adm upgrade
Cluster version is 4.21.0-0-2025-11-07-142257-test-ci-ln-6zj1wdt-latest

Upgradeable=False

 Reason: ConflictingClusterImagePolicy
 Message: Cluster operator machine-config should not be upgraded between minor versions: ClusterImagePolicy resource named 'openshift' conflicts with the cluster default ClusterImagePolicy object and blocks upgrades. Please delete the 'openshift' ClusterImagePolicy resource and reapply it with a different name if needed

warning: Cannot display available updates:
 Reason: NoChannel
 Message: The update channel has not been configured.

oc describe co

Name:         machine-config
Namespace:    
Labels:       <none>
Annotations:  exclude.release.openshift.io/internal-openshift-hosted: true
             include.release.openshift.io/self-managed-high-availability: true
             include.release.openshift.io/single-node-developer: true
API Version:  config.openshift.io/v1
Kind:         ClusterOperator
Metadata:
 Creation Timestamp:  2025-11-07T14:38:17Z
 Generation:          1
 Owner References:
   API Version:     config.openshift.io/v1
   Controller:      true
   Kind:            ClusterVersion
   Name:            version
   UID:             ca29e303-4d5f-4199-b400-5af02af7c412
 Resource Version:  36347
 UID:               b83c6e38-f19f-452a-a5cb-711c91b056ae
Spec:
Status:
 Conditions:
   ...
   Last Transition Time:  2025-11-07T15:20:35Z
   Message:               ClusterImagePolicy resource named 'openshift' conflicts with the cluster default ClusterImagePolicy object and blocks upgrades. Please delete the 'openshift' ClusterImagePolicy resource and reapply it with a different name if needed
   Reason:                ConflictingClusterImagePolicy
   Status:                False
   Type:                  Upgradeable
   ...

- Description for the changelog

Implement upgrade blocking for conflicting ClusterImagePolicy named "openshift"

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

QiWang19 · 2025-11-07T18:10:29Z

/jira refresh

openshift-ci-robot · 2025-11-07T18:10:37Z

@QiWang19: This pull request references Jira Issue OCPBUGS-64822, which is valid. The bug has been moved to the POST state.

7 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target version (4.20.z) matches configured target version for branch (4.20.z)
bug is in the state ASSIGNED, which is one of the valid states (NEW, ASSIGNED, POST)
release note text is set and does not match the template
dependent bug Jira Issue OCPBUGS-64823 is in the state Closed (Done), which is one of the valid states (VERIFIED, RELEASE PENDING, CLOSED (ERRATA), CLOSED (CURRENT RELEASE), CLOSED (DONE), CLOSED (DONE-ERRATA))
dependent Jira Issue OCPBUGS-64823 targets the "4.21.0" version, which is one of the valid target versions: 4.21.0
bug has dependents

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

QiWang19 · 2025-11-07T19:16:23Z

/verified by @QiWang19

on cluster 4.20.0-0.nightly-2025-11-06-175730, openshift/machine-config-operator#5397 (gcp)

openshift-ci-robot · 2025-11-07T19:16:36Z

@QiWang19: This PR has been marked as verified by @QiWang19.

In response to this:

/verified by @QiWang19

on cluster 4.20.0-0.nightly-2025-11-06-175730, openshift/machine-config-operator#5397 (gcp)

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

QiWang19 · 2025-11-10T16:04:47Z

/assign @isabella-janssen

isabella-janssen

/approve

These changes look fair to me & have been verified.

openshift-ci · 2025-11-11T03:16:41Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: isabella-janssen, QiWang19, wking

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [isabella-janssen]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

isabella-janssen · 2025-11-11T03:17:33Z

/label backport-risk-assessed

This change should be a safe backport.

openshift-ci-robot · 2025-11-11T03:40:41Z

/retest-required

Remaining retests: 0 against base HEAD 0c58628 and 2 for PR HEAD 87bdf1f in total

openshift-ci · 2025-11-11T06:53:45Z

@QiWang19: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/bootstrap-unit	`87bdf1f`	link	false	`/test bootstrap-unit`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

isabella-janssen · 2025-11-11T14:34:59Z

/retest-required

openshift-ci-robot · 2025-11-11T17:29:43Z

@QiWang19: Jira Issue Verification Checks: Jira Issue OCPBUGS-64822
✔️ This pull request was pre-merge verified.
✔️ All associated pull requests have merged.
✔️ All associated, merged pull requests were pre-merge verified.

Jira Issue OCPBUGS-64822 has been moved to the MODIFIED state and will move to the VERIFIED state when the change is available in an accepted nightly payload. 🕓

In response to this:

- What I did

Added logic to check if the resource openshift is customer-created and update the cluster operator status Upgradeable=False accordingly. The openshift CIP is cluster-managed reserved for release payload verification. This prevents upgrades when a conflicting policy is detected.

This check needs to be backported to 4.20.z as we plan to GA openshift ClusterImagePolicy (openshift/cluster-update-keys#85) in 4.21.

- How to verify it

launch cluster with this patch

4.21.0-0.nightly-2025-11-05-234508, openshift/machine-config-operator#5395 (gcp)

apply a ClusterImagePolicy name: openshift

oc create -f clusterimgpolicycr.yaml

# clusterimgpolicycr.yaml

apiVersion: config.openshift.io/v1
kind: ClusterImagePolicy
metadata:
 name: openshift
spec:
 scopes:
 - "example.com/test"
 policy:
   rootOfTrust:
     policyType: PublicKey
     publicKey:
       keyData: LS0tLS1CRUdJTiBQVUJMSUMgS0VZLS0tLS0KTUZrd0V3WUhLb1pJemowQ0FRWUlLb1pJemowREFRY0RRZ0FFVW9GVW9ZQVJlS1hHeTU5eGU1U1FPazJhSjhvKwoyL1l6NVk4R2NOM3pGRTZWaUl2a0duSGhNbEFoWGFYL2JvME05UjYyczAvNnErK1Q3dXdORnVPZzhBPT0KLS0tLS1FTkQgUFVCTElDIEtFWS0tLS0t
   signedIdentity:
     matchPolicy: RemapIdentity
     remapIdentity:
       prefix: example.com
       signedPrefix: mirror.com

$ oc get clusterimagepolicy
NAME        AGE
openshift   3m11s

Check the upgrade status has Upgradeable=False

$ oc adm upgrade
Cluster version is 4.21.0-0-2025-11-07-142257-test-ci-ln-6zj1wdt-latest

Upgradeable=False

 Reason: ConflictingClusterImagePolicy
 Message: Cluster operator machine-config should not be upgraded between minor versions: ClusterImagePolicy resource named 'openshift' conflicts with the cluster default ClusterImagePolicy object and blocks upgrades. Please delete the 'openshift' ClusterImagePolicy resource and reapply it with a different name if needed

warning: Cannot display available updates:
 Reason: NoChannel
 Message: The update channel has not been configured.

oc describe co

Name:         machine-config
Namespace:    
Labels:       <none>
Annotations:  exclude.release.openshift.io/internal-openshift-hosted: true
             include.release.openshift.io/self-managed-high-availability: true
             include.release.openshift.io/single-node-developer: true
API Version:  config.openshift.io/v1
Kind:         ClusterOperator
Metadata:
 Creation Timestamp:  2025-11-07T14:38:17Z
 Generation:          1
 Owner References:
   API Version:     config.openshift.io/v1
   Controller:      true
   Kind:            ClusterVersion
   Name:            version
   UID:             ca29e303-4d5f-4199-b400-5af02af7c412
 Resource Version:  36347
 UID:               b83c6e38-f19f-452a-a5cb-711c91b056ae
Spec:
Status:
 Conditions:
   ...
   Last Transition Time:  2025-11-07T15:20:35Z
   Message:               ClusterImagePolicy resource named 'openshift' conflicts with the cluster default ClusterImagePolicy object and blocks upgrades. Please delete the 'openshift' ClusterImagePolicy resource and reapply it with a different name if needed
   Reason:                ConflictingClusterImagePolicy
   Status:                False
   Type:                  Upgradeable
   ...

- Description for the changelog

Implement upgrade blocking for conflicting ClusterImagePolicy named "openshift"

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci bot requested review from dkhater-redhat and pablintino November 7, 2025 17:02

QiWang19 mentioned this pull request Nov 7, 2025

OCPNODE-3769: Implement upgrade blocking for conflicting ClusterImagePolicy named "openshift" #5395

Closed

QiWang19 force-pushed the cip-noupgrade-420 branch from 793ffbe to 87bdf1f Compare November 7, 2025 17:08

wking reviewed Nov 7, 2025

View reviewed changes

wking approved these changes Nov 7, 2025

View reviewed changes

openshift-ci bot assigned wking Nov 7, 2025

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Nov 7, 2025

QiWang19 changed the title ~~Implement upgrade blocking for conflicting ClusterImagePolicy named "openshift"~~ OCPBUGS-64822: Implement upgrade blocking for conflicting ClusterImagePolicy named "openshift" Nov 7, 2025

openshift-ci-robot added jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Nov 7, 2025

openshift-ci-robot added jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. and removed jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Nov 7, 2025

openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label Nov 7, 2025

sairameshv mentioned this pull request Nov 10, 2025

OCPNODE-3874: Remove the dead code of setting the operator status for cgroupv1 based clusters #5399

Merged

openshift-ci bot assigned isabella-janssen Nov 10, 2025

isabella-janssen reviewed Nov 11, 2025

View reviewed changes

openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 11, 2025

openshift-ci bot added the backport-risk-assessed Indicates a PR to a release branch has been evaluated and considered safe to accept. label Nov 11, 2025

openshift-ci bot assigned anuragthehatter Nov 11, 2025

openshift-ci bot assigned eurijon, imatza-rh, lyman9966, mhanss, ptalgulk01, rbbratta and rioliu-rh Nov 11, 2025

openshift-merge-bot bot merged commit f353bfb into openshift:release-4.20 Nov 11, 2025
15 of 16 checks passed

pperiyasamy mentioned this pull request Nov 13, 2025

[release-4.20] OCPBUGS-63686: Fix stale EIP assignments during failover and controller restart openshift/ovn-kubernetes#2835

Merged

QiWang19 mentioned this pull request Nov 13, 2025

OCPBUGS-64822: remove check for conflicting ClusterImagePolicy in syncUpgradeableStatus #5413

Merged

sairameshv mentioned this pull request Nov 18, 2025

OCPNODE-3874: Remove the unused constant CgroupModeV1 openshift/api#2579

Merged

OCPBUGS-64822: Implement upgrade blocking for conflicting ClusterImagePolicy named "openshift" #5397

OCPBUGS-64822: Implement upgrade blocking for conflicting ClusterImagePolicy named "openshift" #5397

Conversation

QiWang19 commented Nov 7, 2025

Uh oh!

QiWang19 commented Nov 7, 2025

Uh oh!

wking Nov 7, 2025

Choose a reason for hiding this comment

Uh oh!

QiWang19 Nov 7, 2025

Choose a reason for hiding this comment

Uh oh!

sairameshv Nov 10, 2025

Choose a reason for hiding this comment

Uh oh!

wking left a comment

Choose a reason for hiding this comment

Uh oh!

openshift-ci-robot commented Nov 7, 2025

Uh oh!

QiWang19 commented Nov 7, 2025

Uh oh!

openshift-ci-robot commented Nov 7, 2025

Uh oh!

QiWang19 commented Nov 7, 2025

Uh oh!

openshift-ci-robot commented Nov 7, 2025

Uh oh!

QiWang19 commented Nov 10, 2025

Uh oh!

isabella-janssen left a comment

Choose a reason for hiding this comment

Uh oh!

openshift-ci bot commented Nov 11, 2025

Uh oh!

isabella-janssen commented Nov 11, 2025

Uh oh!

openshift-ci-robot commented Nov 11, 2025

Uh oh!

openshift-ci bot commented Nov 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

isabella-janssen commented Nov 11, 2025

Uh oh!

Uh oh!

openshift-ci-robot commented Nov 11, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

13 participants

openshift-ci bot commented Nov 11, 2025 •

edited

Loading