Skip to content

ggcr: Pulling an image spawns separate goroutines and TCP connections for all layers simultaneously - WithJobs is not respected #2140

@maxb

Description

@maxb

Describe the bug

No limits on reasonable numbers of layers to pull simultaneously are applied. Every image layer is handled in parallel, even to the point where failures result because the registry isn't prepared to let one client make large numbers of concurrent requests

To Reproduce

Execute code such as

	desc, err := remote.Get(ref, remote.WithJobs(4))
	if err != nil {
		return err
	}

	image, err := desc.Image()
	if err != nil {
		return err
	}

	cachePath := "/var/tmp/oci-layout"
	ociLayoutDir, err := layout.FromPath(cachePath)
	if errors.Is(err, fs.ErrNotExist) {
		ociLayoutDir, err = layout.Write(cachePath, empty.Index)
	}
	if err != nil {
		return err
	}

	err = ociLayoutDir.ReplaceImage(image, func(desc containerregistry.Descriptor) bool { return true })
	if err != nil {
		return err
	}

whilst monitoring, e.g. with watch -n 0.5 lsof -Pi tcp, how many tcp connections are opened.

Expected behavior

remote.WithJobs(4) should have an effect, bounding how many layers are processed simultaneously.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions