supergreen ~ Faster Rust builds!

cargo-green: a cached & remote-ready Rust projects builder.

cargo-green is

• a cargo plugin that sets a $RUSTC_WRAPPER then calls cargo.
• a RUSTC_WRAPPER that builds Dockerfiles
• by forwarding rustc calls to BuildKit builders

• Installation
• Usage
• Remote execution
• Caching
• Configuration
  • $CARGOGREEN_LOG_PATH
  • $CARGOGREEN_LOG
  • $CARGOGREEN_LOG_STYLE
  • $CARGOGREEN_RUNNER
  • $BUILDX_BUILDER
  • $CARGOGREEN_BUILDER_IMAGE
  • $CARGOGREEN_SYNTAX_IMAGE
  • $CARGOGREEN_REGISTRY_MIRRORS
  • $CARGOGREEN_CACHE_IMAGES
  • $CARGOGREEN_CACHE_FROM_IMAGES
  • $CARGOGREEN_CACHE_TO_IMAGES
  • $CARGOGREEN_FINAL_PATH
  • $CARGOGREEN_BASE_IMAGE
  • $CARGOGREEN_SET_ENVS
  • $CARGOGREEN_BASE_IMAGE_INLINE
  • $CARGOGREEN_WITH_NETWORK
  • $CARGOGREEN_ADD_APT
  • $CARGOGREEN_ADD_APT_GET
  • $CARGOGREEN_ADD_APK
  • $CARGOGREEN_EXPERIMENT
• Alternatives
• Origins
• Hacking
• Goals
• Upstream issues & patches
• En vrac

Installation

cargo install cargo-green
cargo install --locked --force --git https://github.com/fenollp/supergreen.git cargo-green

# Make sure $CARGO_HOME/bin is in your $PATH
which cargo-green

When building locally, ensure either a docker or podman client is installed.

Minimum requirements:

• Ubuntu 22.04
• buildkit 0.12.0
• github.com/docker/buildx v0.11.2
• rust 1.73

Usage

# Usage:
  cargo green supergreen env [ENV ...]                           Show used values
  cargo green supergreen doc [ENV ...]                           Documentation of said values
  cargo green fetch                                              Pulls images and crates
  cargo green supergreen sync                                    Pulls everything, for offline usage
  cargo green supergreen push                                    Push cache image (all tags)
  cargo green supergreen builder [ { recreate | rm } --clean ]   Manage local/remote builder
  cargo green supergreen -h | --help
  cargo green supergreen -V | --version
  cargo green ...any cargo subcommand...

# Try:
  cargo clean # Start from a clean slate
  cargo green build
  cargo supergreen env CARGOGREEN_BASE_IMAGE 2>/dev/null
  cargo supergreen help

# Suggestion:
  alias cargo='cargo green'
  # Now try, within your project:
  cargo fetch
  cargo test

Remote execution

Say you have a bigger machine in your ~/.ssh/config called extra-oomph:

DOCKER_HOST=ssh://extra-oomph cargo green test

This will compile the tests on the remote machine and run then locally. TODO: also run tests remotely, like cross does: https://github.com/cross-rs/cross/blob/49cd054de9b832dfc11a4895c72b0aef533b5c6a/README.md#supported-targets

For more knobs to tune, see also:

• $BUILDX_BUILDER
• $CARGOGREEN_BUILDER_IMAGE

Caching

Share your build cache with your team and CI, never feel cold starts!

[package.metadata.green]
cache-images = [ "docker-image://my.org/team/my-project", "docker-image://ghcr.io/me/fork" ]
cache-from-images = [ "docker-image://some.org/global/cache" ]

Configuration

Tune the behavior of cargo-green either through environment variables or via the package's green metadata section.

[package]
name = "my-crate"
# ...

[package.metadata.green]
registry-mirrors = [ "mirror.gcr.io", "public.ecr.aws/docker" ] # Default values

Environment variables that are prefixed with $CARGOGREEN_ override TOML settings.

export CARGOGREEN_REGISTRY_MIRRORS=mirror.gcr.io
cargo green build

$CARGOGREEN_LOG_PATH

Path to a text file to write logs.

Use by setting this environment variable (no Cargo.toml setting):

export CARGOGREEN_LOG_PATH="/tmp/my-logs.txt"
# This needs to be set: nothing is logged by default
export CARGOGREEN_LOG="info"

$CARGOGREEN_LOG

Filter logs. Equivalent to $RUST_LOG (and doesn't conflict with cargo's).

By default, writes logs under /tmp.

See https://docs.rs/env_logger/#enabling-logging

Use by setting this environment variable (no Cargo.toml setting):

export CARGOGREEN_LOG="trace,cargo_green::build=info"

$CARGOGREEN_LOG_STYLE

Style logs. Equivalent to $RUST_LOG_STYLE.

See https://docs.rs/env_logger/#disabling-colors

Use by setting this environment variable (no Cargo.toml setting):

export CARGOGREEN_LOG_STYLE="never"

$CARGOGREEN_RUNNER

Pick which executor to use: "docker" (default), "podman" or "none".

The runner gets forwarded these environment variables:

• $BUILDKIT_COLORS
• $BUILDKIT_HOST
• $BUILDKIT_PROGRESS
• $BUILDKIT_TTY_LOG_LINES
• $BUILDX_BAKE_GIT_AUTH_HEADER
• $BUILDX_BAKE_GIT_AUTH_TOKEN
• $BUILDX_BAKE_GIT_SSH
• $BUILDX_BUILDER
• $DOCKER_BUILDKIT
• $BUILDX_CONFIG
• $BUILDX_CPU_PROFILE
• $BUILDX_EXPERIMENTAL
• $BUILDX_GIT_CHECK_DIRTY
• $BUILDX_GIT_INFO
• $BUILDX_GIT_LABELS
• $BUILDX_MEM_PROFILE
• $BUILDX_METADATA_PROVENANCE
• $BUILDX_METADATA_WARNINGS
• $BUILDX_NO_DEFAULT_ATTESTATIONS
• $BUILDX_NO_DEFAULT_LOAD
• $DOCKER_API_VERSION
• $DOCKER_CERT_PATH
• $DOCKER_CONFIG
• $DOCKER_CONTENT_TRUST
• $DOCKER_CONTENT_TRUST_SERVER
• $DOCKER_CONTEXT
• $DOCKER_DEFAULT_PLATFORM
• $DOCKER_HIDE_LEGACY_COMMANDS
• $DOCKER_HOST
• $DOCKER_TLS
• $DOCKER_TLS_VERIFY
• $EXPERIMENTAL_BUILDKIT_SOURCE_POLICY
• $HTTP_PROXY
• $HTTPS_PROXY
• $NO_PROXY

Use by setting this environment variable (no Cargo.toml setting):

export CARGOGREEN_RUNNER="docker"

$BUILDX_BUILDER

Sets which BuildKit builder to use, through $BUILDX_BUILDER.

See https://docs.docker.com/build/building/variables/#buildx_builder

• Unset: creates & handles a builder named "supergreen". Upgrades it if too old, while trying to keep old cached data
• Set to "": skips using a builder
• Set to "supergreen": uses existing and just warns if too old
• Set: use that as builder, no questions asked

See also

• $DOCKER_HOST: https://docs.docker.com/engine/reference/commandline/cli/#environment-variables
• $DOCKER_CONTEXT: https://docs.docker.com/reference/cli/docker/#environment-variables
• $BUILDKIT_HOST: https://docs.docker.com/build/building/variables/#buildkit_host

$CARGOGREEN_BUILDER_IMAGE

Sets which BuildKit builder version to use.

See https://docs.docker.com/build/builders/

Use by setting this environment variable (no Cargo.toml setting):

export CARGOGREEN_BUILDER_IMAGE="docker-image://docker.io/moby/buildkit:latest"

$CARGOGREEN_SYNTAX_IMAGE

Sets which BuildKit frontend syntax to use.

See https://docs.docker.com/build/buildkit/frontend/#stable-channel

Use by setting this environment variable (no Cargo.toml setting):

export CARGOGREEN_SYNTAX_IMAGE="docker-image://docker.io/docker/dockerfile:1"

$CARGOGREEN_REGISTRY_MIRRORS

Registry mirrors for Docker Hub (docker.io). Defaults to GCP & AWS mirrors.

See https://docs.docker.com/build/buildkit/configure/#registry-mirror

Namely hosts with maybe a port and a path:

• dockerhub.timeweb.cloud
• dockerhub1.beget.com
• localhost:5000
• mirror.gcr.io
• public.ecr.aws/docker

registry-mirrors = [ "mirror.gcr.io", "public.ecr.aws/docker" ]

This environment variable takes precedence over any Cargo.toml settings:

# Note: values here are comma-separated.
export CARGOGREEN_REGISTRY_MIRRORS="mirror.gcr.io,public.ecr.aws/docker"

$CARGOGREEN_CACHE_IMAGES

Both read and write cached data to and from image registries

Exactly a combination of [Green::cache_from_images] and [Green::cache_to_images].

See

• type=registry at https://docs.docker.com/build/cache/backends/
• and https://docs.docker.com/build/cache/backends/registry/

cache-images = [ "docker-image://my.org/team/my-project", "docker-image://some.org/global/cache" ]

This environment variable takes precedence over any Cargo.toml settings:

# Note: values here are comma-separated.
export CARGOGREEN_CACHE_IMAGES="docker-image://my.org/team/my-project,docker-image://some.org/global/cache"

$CARGOGREEN_CACHE_FROM_IMAGES

Read cached data from image registries

See also [Green::cache_images] and [Green::cache_to_images].

cache-from-images = [ "docker-image://my.org/team/my-project-in-ci", "docker-image://some.org/global/cache" ]

This environment variable takes precedence over any Cargo.toml settings:

# Note: values here are comma-separated.
export CARGOGREEN_CACHE_FROM_IMAGES="docker-image://my.org/team/my-project-in-ci,docker-image://some.org/global/cache"

$CARGOGREEN_CACHE_TO_IMAGES

Write cached data to image registries

Note that errors caused by failed cache exports are not ignored.

See also [Green::cache_images] and [Green::cache_from_images].

cache-to-images = [ "docker-image://my.org/team/my-fork" ]

This environment variable takes precedence over any Cargo.toml settings:

# Note: values here are comma-separated.
export CARGOGREEN_CACHE_TO_IMAGES="docker-image://my.org/team/my-fork"

$CARGOGREEN_FINAL_PATH

Write final containerfile to given path.

Helps e.g. create a containerfile of e.g. a binary to use for best caching of dependencies.

Use by setting this environment variable (no Cargo.toml setting):

export CARGOGREEN_FINAL_PATH="$PWD/[email protected]"

$CARGOGREEN_BASE_IMAGE

Sets the base Rust image, as an image URL (or any build context, actually).

If needing additional envs to be passed to rustc or build script, set them in the base image.

This can be done in that same config file with base-image-inline.

See also:

• also-run
• base-image-inline
• additional-build-arguments

For remote builds: make sure this is accessible non-locally.

base-image = "docker-image://docker.io/library/rust:1-slim"

The value must start with docker-image:// and image must be available on the $DOCKER_HOST, eg:

export CARGOGREEN_BASE_IMAGE=docker-image://rustc_with_libs
DOCKER_HOST=ssh://my-remote-builder docker buildx build -t rustc_with_libs - <<EOF
FROM docker.io/library/rust:1.69.0-slim-bookworm@sha256:8bdd28ef184d85c9b4932586af6280732780e806e5f452065420f2e783323ca3
RUN set -eux && apt update && apt install -y libpq-dev libssl3
ENV KEY=value
EOF

This environment variable takes precedence over any Cargo.toml settings:

export CARGOGREEN_BASE_IMAGE="docker-image://docker.io/library/rust:1-slim"

$CARGOGREEN_SET_ENVS

Pass environment variables through to build runner.

May be useful if a build script exported some vars that a package then reads. See also:

• packages

See about $GIT_AUTH_TOKEN: https://docs.docker.com/build/building/secrets/#git-authentication-for-remote-contexts

NOTE: this doesn't (yet) accumulate dependencies' set-envs values! Meaning only the top-level crate's setting is used, for all crates/dependencies.

set-envs = [ "GIT_AUTH_TOKEN", "TYPENUM_BUILD_CONSTS", "TYPENUM_BUILD_OP" ]

This environment variable takes precedence over any Cargo.toml settings:

# Note: values here are comma-separated.
export CARGOGREEN_SET_ENVS="GIT_AUTH_TOKEN,TYPENUM_BUILD_CONSTS,TYPENUM_BUILD_OP"

$CARGOGREEN_BASE_IMAGE_INLINE

Sets the base Rust image for root package and all dependencies, unless themselves being configured differently.

See also:

• with-network
• additional-build-arguments

In order to avoid unexpected changes, you may want to pin the image using an immutable digest.

Note that carefully crafting crossplatform stages can be non-trivial.

base-image-inline = """
FROM --platform=$BUILDPLATFORM rust:1 AS rust-base
RUN --mount=from=some-context,dst=/tmp/some-context cp -r /tmp/some-context ./
RUN --mount=type=secret,id=aws
"""

# This must also be set so digest gets pinned automatically.
base-image = "docker-image://rust:1"

This environment variable takes precedence over any Cargo.toml settings:

IFS='' read -r -d '' CARGOGREEN_BASE_IMAGE_INLINE <<"EOF"
FROM rust:1 AS rust-base
RUN --mount=from=some-context,dst=/tmp/some-context cp -r /tmp/some-context ./
RUN --mount=type=secret,id=aws
EOF
echo "$CARGOGREEN_BASE_IMAGE_INLINE" # (with quotes to preserve newlines)
export CARGOGREEN_BASE_IMAGE_INLINE
export CARGOGREEN_BASE_IMAGE=docker-image://rust:1

$CARGOGREEN_WITH_NETWORK

Controls runner's --network none (default) | default | host setting.

Set this to "default" if e.g. your base-image-inline calls curl or wget or installs some packages.

If adding system packages with add, this gets automatically set to "default".

It turns out --network is part of BuildKit's cache key, so an initial online build won't cache-hit on later offline builds.

Set to none when in $CARGO_NET_OFFLINE mode. See

• https://doc.rust-lang.org/cargo/reference/config.html#netoffline
• rust-lang/rustup#4289

This environment variable takes precedence over any Cargo.toml settings:

export CARGOGREEN_WITH_NETWORK="none"

$CARGOGREEN_ADD_APT

Adds OS packages to the base image with apt install.

See also:

• add.apk
• add.apt-get
• base-image

[package.metadata.green]
add.apt = [ "libpq-dev", "pkg-config" ]

This environment variable takes precedence over any Cargo.toml settings:

# Note: values here are comma-separated.
export CARGOGREEN_ADD_APT="libpq-dev,pkg-config"

# Inspect the resulting base image with:
cargo green supergreen env CARGOGREEN_BASE_IMAGE_INLINE

$CARGOGREEN_ADD_APT_GET

Adds OS packages to the base image with apt-get install.

See also:

• add.apk
• add.apt
• base-image

add.apt-get = [ "libpq-dev", "pkg-config" ]

This environment variable takes precedence over any Cargo.toml settings:

# Note: values here are comma-separated.
export CARGOGREEN_ADD_APT_GET="libpq-dev,pkg-config"

# Inspect the resulting base image with:
cargo green supergreen env CARGOGREEN_BASE_IMAGE_INLINE

$CARGOGREEN_ADD_APK

Adds OS packages to the base image with apk add.

See also:

• add.apt
• add.apt-get
• base-image

add.apk = [ "libpq-dev", "pkgconf" ]

This environment variable takes precedence over any Cargo.toml settings:

# Note: values here are comma-separated.
export CARGOGREEN_ADD_APK="libpq-dev,pkg-conf"

# Inspect the resulting base image with:
cargo green supergreen env CARGOGREEN_BASE_IMAGE_INLINE

$CARGOGREEN_EXPERIMENT

A comma-separated list of names of features to activate.

A name that does not match exactly is an error.

• finalpathnonprimary:

  • Write final containerfile on every rustc call.
  • Helps e.g. debug builds failing too early.

• incremental:

  • Also wrap incremental compilation.
  • See https://doc.rust-lang.org/cargo/reference/config.html#buildincremental

• repro:

  • Try and test for builds hermeticity and reproducibility.
  • See https://docs.docker.com/reference/cli/docker/buildx/build/#no-cache-filter

Use by setting this environment variable (no Cargo.toml setting):

export CARGOGREEN_EXPERIMENT="finalpathnonprimary,repro"

Alternatives

In no particular order:

• ipetkov's crane: A Nix library for building cargo projects. Never build twice thanks to incremental artifact caching.
  • crane.dev
  • => Very complete! Relies on nix tools and language.
• Mozilla's sccache: sccache - Shared Compilation Cache
  • Cargo Book ~ Build cache ~ Shared cache
  • => Relies on everyone having the same paths and doesn't cache all crate types.
• garentyler's distrustc: A Rust-compatible distcc implementation
  • distcc's manpage
• LukeMathWalker's cargo-chef: A cargo-subcommand to speed up Rust Docker builds using Docker layer caching.
  • 5x Faster Rust Docker Builds with cargo-chef
  • => Relies on everyone having the same paths + cache isn't crate-granular.
• Bazel's rules_rust: Rules Rust
  • Building a Rust workspace with Bazel
  • Rust rules for Bazel
  • => Replaces cargo with bazel.
• sgeisler's cargo-remote: cargo subcommand to compile rust projects remotely
  • Building on a remote server
  • => Uses rsync and ssh; seems unmaintained.
• Swatinem's rust-cache: A GitHub Action that implements smart caching for rust/cargo projects
  • => A GitHub Action.
• cross-rs's cargo-cross: “Zero setup” cross compilation and “cross testing” of Rust crates
  • Look at all these (compilation + testing) Supported targets!
  • Remote building and caching with Data Volumes
  • => seldomly maintained but a lifesaver for cross compilation. See also this article on what cargo-green does (perfect layering):
• Better support of Docker layer caching in Cargo
  • Exploring the problem of faster Cargo Docker builds
  • Another reddit discussion

Origins

• PoC originally written in Bash: https://github.com/fenollp/buildxargs/blob/buildx/tryin.sh
• Initial blog post https://fenollp.github.io/faster-rust-builds-docker_host

Hacking

./hack/cli.sh ...

# Usage:           $0                              #=> generate CI
#
# Usage:           $0 ( <name@version> | <name> )  #=> cargo install name@version
# Usage:           $0   ok                         #=> cargo install all working bins
#
# Usage:           $0 ( build | test )             #=> cargo build ./cargo-green
#
# Usage:    jobs=1 $0 ..                           #=> cargo --jobs=$jobs
# Usage: offline=1 $0 ..                           #=> cargo --frozen (defaults to just: --locked)
# Usage:    rmrf=1 $0 ..                           #=> rm -rf $CARGO_TARGET_DIR/*; cargo ...
# Usage:   reset=1 $0 ..                           #=> docker buildx rm $BUILDX_BUILDER; cargo ...
# Usage:   clean=1 $0 ..                           #=> Both reset=1 + rmrf=1
# Usage:   final=0 $0 ..                           #=> Don't generate final Containerfile
#
# Usage:    DOCKER_HOST=.. $0 ..                   #=> Overrides machine
# Usage: BUILDX_BUILDER=.. $0 ..                   #=> Overrides builder (set to "empty" to set BUILDX_BUILDER='')

./hack/recipes.sh

Syncs ./recipes/*.Dockerfile files.

./hack/caching.sh

Verifies properties about caching crates & granularity.

docker buildx prune --all --force

./hack/hit.sh

Estimate of amount of crates reused through compilation of ./recipes/ --> ~5%! Expecting more with larger/more representative corpus + smart locking of transitive deps.

recipes/[email protected]
8< 8< 8<
3: dep-l-utf8parse-0.2.1-522ff71b25340e24
5: dep-l-bitflags-1.3.2-70ce9f1f2fa253bc
5: dep-l-strsim-0.10.0-fd42a4ea370e31ec
5: dep-l-unicode-ident-1.0.12-4c1dc76c11b3deb8
6: dep-l-cfg-if-1.0.0-da34da6838abd7f1

Total recipes: 15
Total stages: 1065
Stages in common: 58
5.44%

When git-bisecting

all:
	cargo +nightly fmt --all
	./hack/clis.sh | tee .github/workflows/clis.yml
	./hack/self.sh | tee .github/workflows/self.yml
	CARGO_TARGET_DIR=$${CARGO_TARGET_DIR:-target/clippy} cargo clippy --locked --frozen --offline --all-targets --all-features -- --no-deps -W clippy::cast_lossless -W clippy::redundant_closure_for_method_calls -W clippy::str_to_string -W clippy::unnecessary_wraps
	RUST_MIN_STACK=8000000 cargo nextest run --all-targets --all-features --locked --frozen --offline --no-fail-fast
	git --no-pager diff --exit-code

Goals

• seamlessly build on another machine (with more cores, more cache)
  • support remote builds by setting env DOCKER_HOST= with e.g. ssh://[email protected]
    • Build cache is saved remotely, artifacts are saved locally
    • Tests building happens on remote machine, execution happens on local machine
• seamlessly integrate with normal cargo usage
  • only pull sources from local filesystem
  • produce the same intermediary artefacts as local cargo does
  • fallback to normal, local rustc anytime
    • switching from this wrapper back to local rustc does necessitate a fresh build
• wrap rustc calls in buildkit-like calls (docker, podman)
  • docker
  • podman TODO: test in CI
  • deps compatibility
    • handle Rust-only deps
    • handle all the other deps (expand this list) (use crater)
      • C deps
      • ...
  • runner compatibility
    • set .dockerignores (to be authoritative on srcs)
  • trace these outputs (STDOUT/STDERR) for debugging
• available as a rustc wrapper through $RUSTC_WRAPPER
• available as a cargo subcommand
  • configuration profiles (user, team, per-workspace, per-crate, CI, ...)
  • seamlessly use current/local rustc version
    • support overriding rustc base image
  • seamlessly use current/local tools (mold, ...)
    • config expressions on top of base image config?
    • just suggest an inline Dockerfile stage?
  • support CRUD-ish operations on local/remotes cache
  • [SEC] support building a crate without it having network access
• integrate with shipping OCI images
• share cache with the World cf. user-wide-cache
  • never rebuild a dep (for a given version of rustc, ...)
    • ensure finest cache granularity (crate-level)
    • free users from cache key built from Cargo.lock (changes on every release cut!)
  • share cache with other projects on local machine
    • fix WORKDIRs + rewrite paths with remap-path-prefix
  • share cache with CI and team
    • share cache with CI (at least for a single user)
  • [SEC] ensure private deps don't leak through/to cache
  • CLI gives the Dockerfile that cargo install's any crate
• suggest a global cache -faciliting configuration profile
• integrate with cross
  • build for a non-local target
  • run/test for a non-local target (with cross's same caveats ie. QEMU)

Upstream issues & patches

• rust: Compile a crate from its source archive directly
• cargo: Tell rustc wrappers which envs to pass through to allow env sandboxing
• buildkit: docker/dockerfile image tags are late
• gRPC buffers too small
  • buildkit: Build function: ResourceExhausted: grpc: received message larger than max (_ vs. 4194304)
  • buildx: ResourceExhausted: grpc: received message larger than max (_ vs. 4194304)
  • buildkit: remote docker buildx build with large dockerfile gives trying to send message larger than max (22482550 vs. 16777216) error
• buildkit: Support extracting ADD --checksum=.. https://.. ..
• buildkit: RUN --no-cache to skip reading & writing a RUN layer to cache
• buildkit: FR: an option to delay --cache-to pushes
• buildkit: Looking for a consistently "latest" BuildKit image tag
• buildkit: docker/dockerfile image tags are late
• buildx: Support --format=json for buildx du --verbose
• TODO
  1. buildkit flag to disable dockerignore and save disk read
  • --ignore-file (closed) Add support for specifying .dockerignore file with -i/--ignore
  • --no-ignore-file
  1. docker build support multiple input files
  • --file-part

  • =1 stage per Dockerfile part file

    • order doesn't matter: order is fixed when consolidation happens (internally)
  1. cargo + docker
  • cargo build --dependencies-only
• Getting an image's digest fast, within a docker-container builder
  • Inspect image manifest without pushing to registry or load to local docker daemon
  • Proposal: introduce enhanced image resolution gateway API
• buildkit: allow exporting cache layers in parallel to the remote registry
• buildkit: remote docker buildx build with large dockerfile gives trying to send message larger than max (22482550 vs. 16777216) error
• buildx: --cache-from takes longer than actual (cached) build
• buildx: The cache export step hangs
• buildkit: COPY --rewrite-timestamp ... to apply SOURCE_DATE_EPOCH build arg value to the timestamps of the files
• buildkit: Dockerfile frontend: ADD --checksum=.. https://.. hides HTTP error

En vrac

• Proposal: c8d: expose contentstore API #44369 moby/moby#44369

• Incremental export transfer #1224 moby/buildkit#1224

• "sending tarball" takes a long time even when the image already exists #107 docker/buildx#107

• Build drivers https://docs.docker.com/build/drivers/

• https://docs.docker.com/build/ci/github-actions/configure-builder/#max-parallelism

• https://docs.docker.com/engine/reference/builder/#buildkit-built-in-build-args

• https://github.com/moby/buildkit#export-cache

• tunnel tty into a docker build through http

• docker build remote driver https://docs.docker.com/build/drivers/remote

• rootless k8s driver https://docs.docker.com/build/drivers/kubernetes/#rootless-mode

• tune many options https://docs.docker.com/build/drivers/docker-container/

  • https://docs.docker.com/config/containers/resource_constraints/
  • https://hub.docker.com/r/moby/buildkit
    • https://github.com/moby/buildkit/releases

• https://docs.docker.com/build/attestations/sbom/

  • https://github.com/moby/buildkit/blob/647a997b389757068760410053873745acabfc80/docs/attestations/sbom.md?plain=1#L48
  • BUILDKIT_SBOM_SCAN_CONTEXT and BUILDKIT_SBOM_SCAN_STAGE

• Support extracting ADD --checksum=.. https://.. .. #4907

• docker image ls --filter=reference=docker.io/$MY/$IMG != docker image ls --filter=reference=$MY/$IMG #47809

• Proposal: csv syntax for git repos #4905

• prune: filtering out ADD --checksum=... https://... entries #2448

• -o=.: open $HOME/.local/share/docker/overlay2/066f6../work/work: permission denied #2219

• cargo restrict targets of crate

• Target configuration for binaries #9208 rust-lang/cargo#9208

• Unsafe fields #3458 rust-lang/rfcs#3458

• Warning when large binary files are included into the bundle #9058 rust-lang/cargo#9058

• Hermetic build mode #9506 rust-lang/cargo#9506

• Feature Request static asserts #2790 rust-lang/rfcs#2790

• greater supply chain attack risk due to large dependency trees? https://www.reddit.com/r/rust/comments/102yz60/greater_supply_chain_attack_risk_due_to_large/

  • https://github.com/rust-secure-code/cargo-supply-chain

• https://doc.rust-lang.org/rustc/command-line-arguments.html#option-emit

• https://rust-lang.github.io/rustup/overrides.html

• https://docs.rs/rustflags/0.1.4/rustflags/index.html

• Provide better diagnostics for why crates are rebuilt

• [build] rustflags = ["--remap-path-prefix"

  • RFC: trim-paths
  • RFC: -Zremap-cwd-prefix=.

• crater: Run experiments across parts of the Rust ecosystem!

• cargo-options Clap parser

• /r/Rust scare Serde has started shipping precompiled binaries with no way to opt out

• assertion failed: edges.remove(&key) #13889

• How we rescued our build process from 24+ hour nightmares

• Partially sandbox your Rust builds

• Is just me or Rust is too heavy for my computer to handle?

• Doesn't detect Docker Rootless #4

• Using S3 as a container registry

• What's the best practice for caching compilation of Rust dependencies?

• Enable Fast Compiles

  • Compiling is slow...

• Everytime I try to use Tauri for Android... Why?

  • on size of compilation artifacts

  • ultimately the "real" solution has got to be a complete overhaul of the entire compilation system to be entirely on-demand in a granular basis, rather than "compile every crate in the dependency tree wholesale".

  • suggestion to use a shared target folder