Skip to content

Conversation

@zetxqx
Copy link
Contributor

@zetxqx zetxqx commented Nov 5, 2025

What type of PR is this?

/kind feature

What this PR does / why we need it:

This pull request introduces the core logic for model rewriting and weighted traffic
splitting within the request control director. It also includes the reconciler logic for InferenceModelRewrite resources.

Key changes:

  • pkg/epp/requestcontrol:
    • Adds the applyWeightedModelRewrite function to handle model rewriting based on
      InferenceModelRewrite rules.
    • Implements weighted selection of target models for traffic splitting.
    • Ensures that the oldest InferenceModelRewrite resource is respected in case of
      duplicate rules
  • pkg/epp/controller:
    • Implements the read-only reconciler logic for InferenceModelRewrite resources

Which issue(s) this PR fixes:

Fixes partially #1811

Does this PR introduce a user-facing change?:

Users can now configure `InferenceModelRewrite` resources to automatically redirect 
incoming model requests to different target models. This feature also supports weighted 
traffic splitting, allowing you to distribute requests across multiple target models based
 on defined percentages.

@k8s-ci-robot k8s-ci-robot added the kind/feature Categorizes issue or PR as related to a new feature. label Nov 5, 2025
@netlify
Copy link

netlify bot commented Nov 5, 2025

Deploy Preview for gateway-api-inference-extension ready!

Name Link
🔨 Latest commit 48f4afa
🔍 Latest deploy log https://app.netlify.com/projects/gateway-api-inference-extension/deploys/690b9494ba7b5000089f0dfa
😎 Deploy Preview https://deploy-preview-1820--gateway-api-inference-extension.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: zetxqx
Once this PR has been reviewed and has the lgtm label, please assign nirrozenbaum for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Nov 5, 2025
@zetxqx
Copy link
Contributor Author

zetxqx commented Nov 5, 2025

/retest

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/feature Categorizes issue or PR as related to a new feature. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants