Skip to content

feat: optimize image build pipeline #342

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Oct 16, 2025
Merged

feat: optimize image build pipeline #342

merged 1 commit into from
Oct 16, 2025

Conversation

@michaelawyu
Copy link
Collaborator

@michaelawyu michaelawyu commented Oct 16, 2025 •
edited
Loading

What type of PR is this?

/kind feature

What this PR does / why we need it:

This PR applies various build pipeline optimizations to improve the build success rates. Specifically, it

  • drops the vendoring step
  • set up builders to re-use caches so that deps will only need to be downloaded once for each build run across 4 images.

Which issue(s) this PR fixes:

N/A

Requirements:

How has this code been tested

N/A

Special notes for your reviewer

N/A

@michaelawyu
Applied various changes
00aacd0 
Signed-off-by: michaelawyu <[email protected]>
@codecov
Copy link

codecov bot commented Oct 16, 2025 •
edited
Loading

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 67.31%. Comparing base (bd78794) to head (00aacd0).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files 
@@            Coverage Diff             @@
##             main     #342      +/-   ##
==========================================
- Coverage   67.56%   67.31%   -0.26%     
==========================================
  Files          34       34              
  Lines        5060     5060              
==========================================
- Hits         3419     3406      -13     
- Misses       1454     1465      +11     
- Partials      187      189       +2

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@michaelawyu
Copy link
Collaborator Author

michaelawyu commented Oct 16, 2025 •
edited
Loading

Added the local build outputs for reference:

The local builds runs w/ all cache cleared. Note that only the first go mod download run takes time; the 3 subsequent runs re-use the mod cache from the first one and thus take virtually no additional time.

make docker-build-hub-net-controller-manager docker-build-member-net-controller-manager docker-build-mcs-controller-manager docker-build-net-crd-installer
Auto-detected system architecture: arm64

Cannot load builder default: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
img-builder*        docker-container                         
 \_ img-builder0     \_ desktop-linux   running   0.18.1     linux/amd64 (+2), linux/arm64, linux/arm (+2), linux/ppc64le, (3 more)
go mod tidy
docker buildx build \
                --file docker/hub-net-controller-manager.Dockerfile \
                --output=type=docker \
                --platform=linux/arm64 \
                --pull \
                --tag ghcr.io/azure/hub-net-controller-manager:00aacd0 \
                --progress=auto \
                --build-arg GOARCH=arm64 \
                --build-arg GOOS=linux .
[+] Building 77.4s (19/19) FINISHED                                                                                     docker-container:img-builder
 => [internal] load build definition from hub-net-controller-manager.Dockerfile                                                                 0.0s
 => => transferring dockerfile: 1.32kB                                                                                                          0.0s
 => [internal] load metadata for mcr.microsoft.com/azurelinux/distroless/base:3.0                                                               1.3s
 => [internal] load metadata for mcr.microsoft.com/oss/go/microsoft/golang:1.24.6                                                               1.3s
 => [internal] load .dockerignore                                                                                                               0.0s
 => => transferring context: 2B                                                                                                                 0.0s
 => [builder  1/10] FROM mcr.microsoft.com/oss/go/microsoft/golang:1.24.6@sha256:bc80e596d83c1506dae76301f6b5e11218fbbd3c0ed3d6f14c8c0dbd7a18  17.5s
 => => resolve mcr.microsoft.com/oss/go/microsoft/golang:1.24.6@sha256:bc80e596d83c1506dae76301f6b5e11218fbbd3c0ed3d6f14c8c0dbd7a18b7d7         0.0s
 => => sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1 32B / 32B                                                        0.1s
 => => sha256:71b8d5f6e7e7bec68f3d74e058459a3150780c381c5ade725c8f7c0b37cfac9b 147B / 147B                                                      0.7s
 => => sha256:911e48dd4b1972cf0bc0d864069c4fcb1230c96616f073e04b36f59c57e5a423 75.42MB / 75.42MB                                               13.1s
 => => sha256:451517eab32570c9b5f672dba6a7e566731d80624f305a7ef82352752c2a2bab 86.65MB / 86.65MB                                                9.7s
 => => sha256:10c5c3408e4b8a0e96bc9c358a5e7e303941bcf2a2140cd8c7d5e1be41762c79 321.26kB / 321.26kB                                              1.2s
 => => sha256:c4910ed05e8b3022bc1c6adfffae5e35b0d2b4c6d756ee21311b48b509147a1a 64.37MB / 64.37MB                                                4.0s
 => => sha256:8cff9c97e1a1ee42786188e1d1b57f6a2035d65b648178ac0262d0eba0c5c86d 23.57MB / 23.57MB                                                2.7s
 => => sha256:f91a155d47f89918d11d82553052c77509abd21f05e510df76ac18885f9985b9 49.63MB / 49.63MB                                                6.9s
 => => extracting sha256:f91a155d47f89918d11d82553052c77509abd21f05e510df76ac18885f9985b9                                                       1.3s
 => => extracting sha256:8cff9c97e1a1ee42786188e1d1b57f6a2035d65b648178ac0262d0eba0c5c86d                                                       0.3s
 => => extracting sha256:c4910ed05e8b3022bc1c6adfffae5e35b0d2b4c6d756ee21311b48b509147a1a                                                       1.2s
 => => extracting sha256:10c5c3408e4b8a0e96bc9c358a5e7e303941bcf2a2140cd8c7d5e1be41762c79                                                       0.0s
 => => extracting sha256:451517eab32570c9b5f672dba6a7e566731d80624f305a7ef82352752c2a2bab                                                       1.1s
 => => extracting sha256:911e48dd4b1972cf0bc0d864069c4fcb1230c96616f073e04b36f59c57e5a423                                                       2.2s
 => => extracting sha256:71b8d5f6e7e7bec68f3d74e058459a3150780c381c5ade725c8f7c0b37cfac9b                                                       0.0s
 => => extracting sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1                                                       0.0s
 => [internal] load build context                                                                                                               0.1s
 => => transferring context: 1.34MB                                                                                                             0.1s
 => [stage-1 1/3] FROM mcr.microsoft.com/azurelinux/distroless/base:3.0@sha256:1e77d97e1e39f22ed9c52f49b3508b4c1044cec23743df9098ac44e025f654f  1.0s
 => => resolve mcr.microsoft.com/azurelinux/distroless/base:3.0@sha256:1e77d97e1e39f22ed9c52f49b3508b4c1044cec23743df9098ac44e025f654f2         0.0s
 => => sha256:a83e253bd74e2da35fc330e8cc5cea4f2bcae729e467c7db3e064948144468a8 13.95MB / 13.95MB                                                0.4s
 => => extracting sha256:a83e253bd74e2da35fc330e8cc5cea4f2bcae729e467c7db3e064948144468a8                                                       0.5s
 => [builder  2/10] WORKDIR /workspace                                                                                                          0.4s
 => [builder  3/10] COPY go.mod go.mod                                                                                                          0.0s
 => [builder  4/10] COPY go.sum go.sum                                                                                                          0.0s
 => [builder  5/10] RUN --mount=type=cache,target=/go/pkg/mod go mod download                                                                  19.0s
 => [builder  6/10] COPY cmd/hub-net-controller-manager/main.go main.go                                                                         0.0s
 => [builder  7/10] COPY api/ api/                                                                                                              0.0s
 => [builder  8/10] COPY pkg/ pkg/                                                                                                              0.0s
 => [builder  9/10] RUN echo "Building images with GOOS=linux GOARCH=arm64"                                                                     0.1s
 => [builder 10/10] RUN --mount=type=cache,target=/go/pkg/mod CGO_ENABLED=1 GOOS=linux GOARCH=arm64 GOEXPERIMENT=systemcrypto GO111MODULE=on   36.7s
 => [stage-1 2/3] COPY --from=builder /workspace/hub-net-controller-manager .                                                                   0.1s
 => exporting to docker image format                                                                                                            2.2s
 => => exporting layers                                                                                                                         1.3s
 => => exporting manifest sha256:0038f1599b37d28709d8f51b82651fbab6bf39fd52fac9856b3447c0f8626ba2                                               0.0s
 => => exporting config sha256:de7d2e69de08f5becdec31327417022552f30a6974c54355dd4e3c2c728c7a9e                                                 0.0s
 => => sending tarball                                                                                                                          0.8s
 => importing to docker                                                                                                                         0.3s
 => => loading layer d75d33a915a2 26.48MB / 26.48MB                                                                                             0.3s
docker buildx build \
                --file docker/member-net-controller-manager.Dockerfile \
                --output=type=docker \
                --platform=linux/arm64 \
                --pull \
                --tag ghcr.io/azure/member-net-controller-manager:00aacd0 \
                --progress=auto \
                --build-arg GOARCH=arm64 \
                --build-arg GOOS=linux .
[+] Building 38.7s (19/19) FINISHED                                                                                     docker-container:img-builder
 => [internal] load build definition from member-net-controller-manager.Dockerfile                                                              0.0s
 => => transferring dockerfile: 1.34kB                                                                                                          0.0s
 => [internal] load metadata for mcr.microsoft.com/azurelinux/distroless/base:3.0                                                               0.2s
 => [internal] load metadata for mcr.microsoft.com/oss/go/microsoft/golang:1.24.6                                                               0.2s
 => [internal] load .dockerignore                                                                                                               0.0s
 => => transferring context: 2B                                                                                                                 0.0s
 => [builder  1/10] FROM mcr.microsoft.com/oss/go/microsoft/golang:1.24.6@sha256:bc80e596d83c1506dae76301f6b5e11218fbbd3c0ed3d6f14c8c0dbd7a18b  0.0s
 => => resolve mcr.microsoft.com/oss/go/microsoft/golang:1.24.6@sha256:bc80e596d83c1506dae76301f6b5e11218fbbd3c0ed3d6f14c8c0dbd7a18b7d7         0.0s
 => [internal] load build context                                                                                                               0.0s
 => => transferring context: 26.02kB                                                                                                            0.0s
 => CACHED [stage-1 1/3] FROM mcr.microsoft.com/azurelinux/distroless/base:3.0@sha256:1e77d97e1e39f22ed9c52f49b3508b4c1044cec23743df9098ac44e0  0.0s
 => => resolve mcr.microsoft.com/azurelinux/distroless/base:3.0@sha256:1e77d97e1e39f22ed9c52f49b3508b4c1044cec23743df9098ac44e025f654f2         0.0s
 => CACHED [builder  2/10] WORKDIR /workspace                                                                                                   0.0s
 => CACHED [builder  3/10] COPY go.mod go.mod                                                                                                   0.0s
 => CACHED [builder  4/10] COPY go.sum go.sum                                                                                                   0.0s
 => CACHED [builder  5/10] RUN --mount=type=cache,target=/go/pkg/mod go mod download                                                            0.0s
 => [builder  6/10] COPY cmd/member-net-controller-manager/main.go main.go                                                                      0.0s
 => [builder  7/10] COPY api/ api/                                                                                                              0.0s
 => [builder  8/10] COPY pkg/ pkg/                                                                                                              0.0s
 => [builder  9/10] RUN echo "Building images with GOOS=linux GOARCH=arm64"                                                                     0.0s
 => [builder 10/10] RUN --mount=type=cache,target=/go/pkg/mod CGO_ENABLED=1 GOOS=linux GOARCH=arm64 GOEXPERIMENT=systemcrypto GO111MODULE=on   36.0s
 => [stage-1 2/3] COPY --from=builder /workspace/member-net-controller-manager .                                                                0.1s
 => exporting to docker image format                                                                                                            2.0s
 => => exporting layers                                                                                                                         1.3s
 => => exporting manifest sha256:0ec40035a0c2f2908d6141881751b693cb243fb016c24ca7404bb09eece1932f                                               0.0s
 => => exporting config sha256:4b1edb7505b41316fd9b847daa935c723207e34c94695973832d6acce33fb414                                                 0.0s
 => => sending tarball                                                                                                                          0.7s
 => importing to docker                                                                                                                         0.3s
 => => loading layer 0c7d5df873b8 26.42MB / 26.42MB                                                                                             0.3s
docker buildx build \
                --file docker/mcs-controller-manager.Dockerfile \
                --output=type=docker \
                --platform=linux/arm64 \
                --pull \
                --tag ghcr.io/azure/mcs-controller-manager:00aacd0 \
                --progress=auto \
                --build-arg GOARCH=arm64 \
                --build-arg GOOS=linux .
[+] Building 28.4s (19/19) FINISHED                                                                                     docker-container:img-builder
 => [internal] load build definition from mcs-controller-manager.Dockerfile                                                                     0.0s
 => => transferring dockerfile: 1.30kB                                                                                                          0.0s
 => [internal] load metadata for mcr.microsoft.com/azurelinux/distroless/base:3.0                                                               0.1s
 => [internal] load metadata for mcr.microsoft.com/oss/go/microsoft/golang:1.24.6                                                               0.1s
 => [internal] load .dockerignore                                                                                                               0.0s
 => => transferring context: 2B                                                                                                                 0.0s
 => [builder  1/10] FROM mcr.microsoft.com/oss/go/microsoft/golang:1.24.6@sha256:bc80e596d83c1506dae76301f6b5e11218fbbd3c0ed3d6f14c8c0dbd7a18b  0.0s
 => => resolve mcr.microsoft.com/oss/go/microsoft/golang:1.24.6@sha256:bc80e596d83c1506dae76301f6b5e11218fbbd3c0ed3d6f14c8c0dbd7a18b7d7         0.0s
 => [internal] load build context                                                                                                               0.0s
 => => transferring context: 20.24kB                                                                                                            0.0s
 => CACHED [stage-1 1/3] FROM mcr.microsoft.com/azurelinux/distroless/base:3.0@sha256:1e77d97e1e39f22ed9c52f49b3508b4c1044cec23743df9098ac44e0  0.0s
 => => resolve mcr.microsoft.com/azurelinux/distroless/base:3.0@sha256:1e77d97e1e39f22ed9c52f49b3508b4c1044cec23743df9098ac44e025f654f2         0.0s
 => CACHED [builder  2/10] WORKDIR /workspace                                                                                                   0.0s
 => CACHED [builder  3/10] COPY go.mod go.mod                                                                                                   0.0s
 => CACHED [builder  4/10] COPY go.sum go.sum                                                                                                   0.0s
 => CACHED [builder  5/10] RUN --mount=type=cache,target=/go/pkg/mod go mod download                                                            0.0s
 => [builder  6/10] COPY cmd/mcs-controller-manager/main.go main.go                                                                             0.0s
 => [builder  7/10] COPY api/ api/                                                                                                              0.0s
 => [builder  8/10] COPY pkg/ pkg/                                                                                                              0.0s
 => [builder  9/10] RUN echo "Building images with GOOS=linux GOARCH=arm64"                                                                     0.1s
 => [builder 10/10] RUN --mount=type=cache,target=/go/pkg/mod CGO_ENABLED=1 GOOS=linux GOARCH=arm64 GOEXPERIMENT=systemcrypto GO111MODULE=on   26.0s
 => [stage-1 2/3] COPY --from=builder /workspace/mcs-controller-manager .                                                                       0.1s
 => exporting to docker image format                                                                                                            1.9s
 => => exporting layers                                                                                                                         1.2s
 => => exporting manifest sha256:72fbeb66f636f40fd698ff560be14ba3af5f7b1ddb34f10fb198726debbd7fa1                                               0.0s
 => => exporting config sha256:79e12977ca5430f945d1f93b1a2d8295d32395d42ec99c6c4c1729342642b858                                                 0.0s
 => => sending tarball                                                                                                                          0.7s
 => importing to docker                                                                                                                         0.3s
 => => loading layer 02b6104bb0bd 24.78MB / 24.78MB                                                                                             0.3s
docker buildx build \
                --file docker/net-crd-installer.Dockerfile \
                --output=type=docker \
                --platform=linux/arm64 \
                --pull \
                --tag ghcr.io/azure/net-crd-installer:00aacd0 \
                --progress=auto \
                --build-arg GOARCH=arm64 \
                --build-arg GOOS=linux .
[+] Building 28.9s (18/18) FINISHED                                                                                     docker-container:img-builder
 => [internal] load build definition from net-crd-installer.Dockerfile                                                                          0.0s
 => => transferring dockerfile: 1.31kB                                                                                                          0.0s
 => [internal] load metadata for mcr.microsoft.com/azurelinux/distroless/base:3.0                                                               0.2s
 => [internal] load metadata for mcr.microsoft.com/oss/go/microsoft/golang:1.24.6                                                               0.2s
 => [internal] load .dockerignore                                                                                                               0.0s
 => => transferring context: 2B                                                                                                                 0.0s
 => [builder 1/8] FROM mcr.microsoft.com/oss/go/microsoft/golang:1.24.6@sha256:bc80e596d83c1506dae76301f6b5e11218fbbd3c0ed3d6f14c8c0dbd7a18b7d  0.0s
 => => resolve mcr.microsoft.com/oss/go/microsoft/golang:1.24.6@sha256:bc80e596d83c1506dae76301f6b5e11218fbbd3c0ed3d6f14c8c0dbd7a18b7d7         0.0s
 => CACHED [stage-1 1/4] FROM mcr.microsoft.com/azurelinux/distroless/base:3.0@sha256:1e77d97e1e39f22ed9c52f49b3508b4c1044cec23743df9098ac44e0  0.0s
 => => resolve mcr.microsoft.com/azurelinux/distroless/base:3.0@sha256:1e77d97e1e39f22ed9c52f49b3508b4c1044cec23743df9098ac44e025f654f2         0.0s
 => [internal] load build context                                                                                                               0.0s
 => => transferring context: 145.96kB                                                                                                           0.0s
 => CACHED [builder 2/8] WORKDIR /workspace                                                                                                     0.0s
 => CACHED [builder 3/8] COPY go.mod go.mod                                                                                                     0.0s
 => CACHED [builder 4/8] COPY go.sum go.sum                                                                                                     0.0s
 => CACHED [builder 5/8] RUN --mount=type=cache,target=/go/pkg/mod go mod download                                                              0.0s
 => [builder 6/8] COPY cmd/net-crd-installer/ cmd/net-crd-installer/                                                                            0.0s
 => [builder 7/8] RUN echo "Building images with GOOS=linux GOARCH=arm64"                                                                       0.0s
 => [builder 8/8] RUN --mount=type=cache,target=/go/pkg/mod CGO_ENABLED=1 GOOS=linux GOARCH=arm64 GOEXPERIMENT=systemcrypto GO111MODULE=on go  26.5s
 => [stage-1 2/4] COPY --from=builder /workspace/net-crd-installer .                                                                            0.1s
 => [stage-1 3/4] COPY config/crd/bases/ /workspace/config/crd/bases/                                                                           0.0s
 => exporting to docker image format                                                                                                            1.9s
 => => exporting layers                                                                                                                         1.2s
 => => exporting manifest sha256:6d564dbea9f6ef1e3937e305823fe6e7ffc848de3fc3a592d97319770dd2c24d                                               0.0s
 => => exporting config sha256:b71a6c9fe04c11c21695aa6b18a688c10481b13c5bf823b542acbc1e7b5c72f1                                                 0.0s
 => => sending tarball                                                                                                                          0.7s
 => importing to docker                                                                                                                         0.3s
 => => loading layer 83dc36fd56f1 22.62MB / 22.62MB                                                                                             0.3s
 => => loading layer e365bf0eda72 9.84kB / 9.84kB                                                                                               0.0s

@michaelawyu michaelawyu merged commit 677dc3b into main Oct 16, 2025
15 of 20 checks passed
ryanzhang-oss
ryanzhang-oss reviewed Oct 16, 2025
View reviewed changes
docker/member-net-controller-manager.Dockerfile
@@ -1,5 +1,5 @@
# Build the member-net-controller-manager binary
FROM mcr.microsoft.com/oss/go/microsoft/golang:1.24.6 as builder
FROM mcr.microsoft.com/oss/go/microsoft/golang:1.24.6 AS builder
Copy link
Contributor

@ryanzhang-oss ryanzhang-oss Oct 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it case sensitive?

@zhiying-lin zhiying-lin deleted the feat/optimize-build-pipeline branch October 17, 2025 02:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ryanzhang-oss ryanzhang-oss ryanzhang-oss left review comments

@zhiying-lin zhiying-lin zhiying-lin approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@michaelawyu @zhiying-lin @ryanzhang-oss