Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 7 additions & 3 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,8 @@ PLUGINS := \
$(BIN_PATH)/template \
$(BIN_PATH)/wasm \
$(BIN_PATH)/network-device-injector \
$(BIN_PATH)/network-logger
$(BIN_PATH)/network-logger \
$(BIN_PATH)/rdt

ifneq ($(V),1)
Q := @
Expand Down Expand Up @@ -122,9 +123,9 @@ $(BIN_PATH)/wasm build/bin/wasm: FORCE
# test targets
#

test-gopkgs: ginkgo-tests test-ulimits
test-gopkgs: ginkgo-tests test-ulimits test-rdt

SKIPPED_PKGS="ulimit-adjuster,device-injector"
SKIPPED_PKGS="ulimit-adjuster,device-injector,rdt"

ginkgo-tests:
$(Q)$(GINKGO) run \
Expand All @@ -146,6 +147,9 @@ test-ulimits:
test-device-injector:
$(Q)cd ./plugins/device-injector && $(GO_TEST) -v

test-rdt:
$(Q)cd ./plugins/rdt && $(GO_TEST) -v

codecov: SHELL := $(shell which bash)
codecov:
bash <(curl -s https://codecov.io/bash) -f $(COVERAGE_PATH)/coverprofile
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -419,6 +419,7 @@ The following sample plugins exist for NRI:
- [ulimit adjuster](plugins/ulimit-adjuster)
- [NRI v0.1.0 plugin adapter](plugins/v010-adapter)
- [WebAssembly plugin](plugins/wasm)
- [RDT](plugins/rdt)
- [template](plugins/template)

Please see the documentation of these plugins for further details
Expand Down
30 changes: 30 additions & 0 deletions contrib/kustomize/rdt/base/daemonset.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: nri-plugin-rdt
spec:
template:
spec:
containers:
- name: plugin
image: plugin:latest
args:
- "-idx"
- "10"
resources:
requests:
cpu: "2m"
memory: "5Mi"
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
volumeMounts:
- name: nri-socket
mountPath: /var/run/nri/nri.sock
volumes:
- name: nri-socket
hostPath:
path: /var/run/nri/nri.sock
type: Socket
12 changes: 12 additions & 0 deletions contrib/kustomize/rdt/base/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: kube-system
resources:
- daemonset.yaml
images:
- name: plugin
newName: ghcr.io/containerd/nri/plugins/rdt
labels:
- includeSelectors: true
pairs:
app.kubernetes.io/name: nri-plugin-rdt
6 changes: 6 additions & 0 deletions contrib/kustomize/rdt/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- base/
components:
- ../components/image-stable
6 changes: 6 additions & 0 deletions contrib/kustomize/rdt/unstable/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../base/
components:
- ../../components/image-unstable
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ require (
github.com/moby/sys/mountinfo v0.6.2
github.com/onsi/ginkgo/v2 v2.19.1
github.com/onsi/gomega v1.34.0
github.com/opencontainers/runtime-spec v1.1.0
github.com/opencontainers/runtime-spec v1.2.2-0.20250818071321-383cadbf08c0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@marquiz Do we know what is the expected timeframe for cutting the first such runtime-spec release which would put 383cadbf08c0 commit behind a tag ?

I am asking because we have been reluctant lately to pull in untagged versions of the runtime spec, because that would cause extra hassle/work if we wanted to or were forced to cut a new release and bump containerd and cri-o to pull in that new version. For instance #157 and #166 are current drafts exactly for this reason: the runtime-spec they require is not available yet in a tagged release.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, sure. I don't think we need to hurry with this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@marquiz @mikebrow @chrishenzie In principle there is one way how we could get this partially in already now, to avoid having to keep rebasing this with a guaranteed conflict whenever we add something to LinuxContainer or LinuxContainerAdjustment.

It would be to split this up to 2 stacked PRs:

  • PR#1: The bulk of this PR, without the OCI Spec bump + a single change to Generator.Adjust(): return an error if adjust.GetLinux().GetRdt() != nil
  • draft PR#2: All the commits in PR#1 + the OCI Spec bump (eventually to a tagged version) + a Generator.Adjust() updated to do RDT adjustment if requested

Then we could merge #PR1 (and rebase PR#2 when that happens), and keep the OCI Spec-dependent bits pending until the necessary bits get tagged. We could apply the same to #157 and #166...

Maybe not worth the effort though, and we can just mark this a draft until we get a tagged OCI Spec.

github.com/opencontainers/runtime-tools v0.9.0
github.com/sirupsen/logrus v1.9.3
github.com/stretchr/testify v1.8.4
Expand Down
4 changes: 2 additions & 2 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -43,8 +43,8 @@ github.com/onsi/ginkgo/v2 v2.19.1/go.mod h1:O3DtEWQkPa/F7fBMgmZQKKsluAy8pd3rEQdr
github.com/onsi/gomega v1.34.0 h1:eSSPsPNp6ZpsG8X1OVmOTxig+CblTc4AxpPBykhe2Os=
github.com/onsi/gomega v1.34.0/go.mod h1:MIKI8c+f+QLWk+hxbePD4i0LMJSExPaZOVfkoex4cAo=
github.com/opencontainers/runtime-spec v1.0.3-0.20220825212826-86290f6a00fb/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/C5hL+LL3ko9hs6T5lQ0=
github.com/opencontainers/runtime-spec v1.1.0 h1:HHUyrt9mwHUjtasSbXSMvs4cyFxh+Bll4AjJ9odEGpg=
github.com/opencontainers/runtime-spec v1.1.0/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/C5hL+LL3ko9hs6T5lQ0=
github.com/opencontainers/runtime-spec v1.2.2-0.20250818071321-383cadbf08c0 h1:RLn0YfUWkiqPGtgUANvJrcjIkCHGRl3jcz/c557M28M=
github.com/opencontainers/runtime-spec v1.2.2-0.20250818071321-383cadbf08c0/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/C5hL+LL3ko9hs6T5lQ0=
github.com/opencontainers/runtime-tools v0.0.0-20221026201742-946c877fa809 h1:WSwkWIIS4s+E/dPF6HuVZ/hnq1WfXN371eESjREnU8k=
github.com/opencontainers/runtime-tools v0.0.0-20221026201742-946c877fa809/go.mod h1:BRHJJd0E+cx42OybVYSgUvZmU0B8P9gZuRXlZUP7TKI=
github.com/opencontainers/selinux v1.9.1 h1:b4VPEF3O5JLZgdTDBmGepaaIbAo0GqoF6EBRq5f/g3Y=
Expand Down
30 changes: 30 additions & 0 deletions pkg/adaptation/adaptation_suite_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -583,6 +583,13 @@ var _ = Describe("Plugin container creation adjustments", func() {
return api.FromOCILinuxSeccomp(&seccomp)
}(),
)
case "rdt":
if overwrite {
a.RemoveLinuxRDT()
}
a.SetLinuxRDTClosID(p.name)
a.SetLinuxRDTSchemata([]string{"L3:0=ff", "MB:0=50"})
a.SetLinuxRDTEnableMonitoring(true)
}

return a, nil, nil
Expand Down Expand Up @@ -868,6 +875,17 @@ var _ = Describe("Plugin container creation adjustments", func() {
},
},
),
Entry("adjust RDT", "rdt",
&api.ContainerAdjustment{
Linux: &api.LinuxContainerAdjustment{
Rdt: &api.LinuxRdt{
ClosId: api.String("test"),
Schemata: api.RepeatedString([]string{"L3:0=ff", "MB:0=50"}),
EnableMonitoring: api.Bool(true),
},
},
},
),
)
})

Expand Down Expand Up @@ -1008,6 +1026,18 @@ var _ = Describe("Plugin container creation adjustments", func() {
),
Entry("adjust resources", "resources/classes", false, true, nil),
Entry("adjust I/O priority (conflicts)", "I/O priority", false, true, nil),
Entry("adjust RDT (conflicts)", "rdt", false, true, nil),
Entry("adjust RDT", "rdt", true, false,
&api.ContainerAdjustment{
Linux: &api.LinuxContainerAdjustment{
Rdt: &api.LinuxRdt{
ClosId: api.String("foo"),
Schemata: api.RepeatedString([]string{"L3:0=ff", "MB:0=50"}),
EnableMonitoring: api.Bool(true),
},
},
},
),
)
})

Expand Down
18 changes: 10 additions & 8 deletions pkg/adaptation/api.go
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,7 @@ type (
LinuxDeviceCgroup = api.LinuxDeviceCgroup
LinuxIOPriority = api.LinuxIOPriority
LinuxSeccomp = api.LinuxSeccomp
LinuxRdt = api.LinuxRdt
CDIDevice = api.CDIDevice
HugepageLimit = api.HugepageLimit
Hooks = api.Hooks
Expand Down Expand Up @@ -149,14 +150,15 @@ type (
// Aliased functions for api/optional.go.
// nolint
var (
String = api.String
Int = api.Int
Int32 = api.Int32
UInt32 = api.UInt32
Int64 = api.Int64
UInt64 = api.UInt64
Bool = api.Bool
FileMode = api.FileMode
String = api.String
RepeatedString = api.RepeatedString
Int = api.Int
Int32 = api.Int32
UInt32 = api.UInt32
Int64 = api.Int64
UInt64 = api.UInt64
Bool = api.Bool
FileMode = api.FileMode
)

// Aliased functions for api/types.go.
Expand Down
60 changes: 60 additions & 0 deletions pkg/adaptation/result.go
Original file line number Diff line number Diff line change
Expand Up @@ -235,7 +235,11 @@ func (r *result) adjust(rpl *ContainerAdjustment, plugin string) error {
if err := r.adjustNamespaces(rpl.Linux.Namespaces, plugin); err != nil {
return err
}
if err := r.adjustRdt(rpl.Linux.Rdt, plugin); err != nil {
return err
}
}

if err := r.adjustRlimits(rpl.Rlimits, plugin); err != nil {
return err
}
Expand Down Expand Up @@ -451,6 +455,42 @@ func (r *result) adjustNamespaces(namespaces []*LinuxNamespace, plugin string) e
return nil
}

func (r *result) adjustRdt(rdt *LinuxRdt, plugin string) error {
if r == nil {
return nil
}

r.initAdjustRdt()

id := r.request.create.Container.Id

if rdt.GetRemove() {
r.owners.ClearRdt(id, plugin)
r.reply.adjust.Linux.Rdt = &LinuxRdt{}
}

if v := rdt.GetClosId(); v != nil {
if err := r.owners.ClaimRdtClosID(id, plugin); err != nil {
return err
}
r.reply.adjust.Linux.Rdt.ClosId = String(v.GetValue())
}
if v := rdt.GetSchemata(); v != nil {
if err := r.owners.ClaimRdtSchemata(id, plugin); err != nil {
return err
}
r.reply.adjust.Linux.Rdt.Schemata = RepeatedString(v.GetValue())
}
if v := rdt.GetEnableMonitoring(); v != nil {
if err := r.owners.ClaimRdtEnableMonitoring(id, plugin); err != nil {
return err
}
r.reply.adjust.Linux.Rdt.EnableMonitoring = Bool(v.GetValue())
}

return nil
}

func (r *result) adjustCDIDevices(devices []*CDIDevice, plugin string) error {
if len(devices) == 0 {
return nil
Expand Down Expand Up @@ -1067,3 +1107,23 @@ func (r *result) getContainerUpdate(u *ContainerUpdate, plugin string) (*Contain

return update, nil
}

func (r *result) initAdjust() {
if r.reply.adjust == nil {
r.reply.adjust = &ContainerAdjustment{}
}
}

func (r *result) initAdjustLinux() {
r.initAdjust()
if r.reply.adjust.Linux == nil {
r.reply.adjust.Linux = &LinuxContainerAdjustment{}
}
}

func (r *result) initAdjustRdt() {
r.initAdjustLinux()
if r.reply.adjust.Linux.Rdt == nil {
r.reply.adjust.Linux.Rdt = &LinuxRdt{}
}
}
30 changes: 30 additions & 0 deletions pkg/api/adjustment.go
Original file line number Diff line number Diff line change
Expand Up @@ -279,6 +279,30 @@ func (a *ContainerAdjustment) SetLinuxRDTClass(value string) {
a.Linux.Resources.RdtClass = String(value)
}

// SetLinuxRDTClosID records setting the RDT CLOS id for a container.
func (a *ContainerAdjustment) SetLinuxRDTClosID(value string) {
a.initLinuxRdt()
a.Linux.Rdt.ClosId = String(value)
}

// SetLinuxRDTSchemata records setting the RDT schemata for a container.
func (a *ContainerAdjustment) SetLinuxRDTSchemata(value []string) {
a.initLinuxRdt()
a.Linux.Rdt.Schemata = RepeatedString(value)
}

// SetLinuxRDTEnableMonitoring records enabling RDT monitoring for a container.
func (a *ContainerAdjustment) SetLinuxRDTEnableMonitoring(value bool) {
a.initLinuxRdt()
a.Linux.Rdt.EnableMonitoring = Bool(value)
}

// RemoveLinuxRdt records the removal of the RDT configuration.
func (a *ContainerAdjustment) RemoveLinuxRDT() {
a.initLinuxRdt()
a.Linux.Rdt.Remove = true
}

// AddLinuxUnified sets a cgroupv2 unified resource.
func (a *ContainerAdjustment) AddLinuxUnified(key, value string) {
a.initLinuxResourcesUnified()
Expand Down Expand Up @@ -378,3 +402,9 @@ func (a *ContainerAdjustment) initLinuxResourcesUnified() {
a.Linux.Resources.Unified = make(map[string]string)
}
}
func (a *ContainerAdjustment) initLinuxRdt() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a note: #118 has a bunch of queued similar changes, updating the result-handling code so that there is no need to always a priory create a fully constructed but empty adjustment. It would be nice to get that finally in, so I'll rebase it that on the latest main/HEAD then update with any missing init*() bits necessary.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 I'll review when that's updated. And then rebase when that's merged

a.initLinux()
if a.Linux.Rdt == nil {
a.Linux.Rdt = &LinuxRdt{}
}
}
Loading
Loading