Skip to content

[CI/Build] Moves to cuda-base runtime image while retaining minimal JIT dependencies #29270

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
vllm-bot merged 4 commits into from
Nov 24, 2025
Merged

[CI/Build] Moves to cuda-base runtime image while retaining minimal JIT dependencies #29270

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
9ec0bb4
Fixes
bbartels Nov 23, 2025
60d6114
Fixes dockerfile dependency graph
bbartels Nov 23, 2025
a53e398
Update docker/Dockerfile
bbartels Nov 23, 2025
ddadf49
Fixes missing cuda-cuobjdump
bbartels Nov 23, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 14 additions & 2 deletions docker/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@ ARG PYTHON_VERSION=3.12
# glibc version is baked into the distro, and binaries built with one glibc
# version are not backwards compatible with OSes that use an earlier version.
ARG BUILD_BASE_IMAGE=nvidia/cuda:${CUDA_VERSION}-devel-ubuntu20.04
# TODO: Restore to base image after FlashInfer AOT wheel fixed
ARG FINAL_BASE_IMAGE=nvidia/cuda:${CUDA_VERSION}-devel-ubuntu22.04
# Using cuda base image with minimal dependencies necessary for JIT compilation (FlashInfer, DeepGEMM, EP kernels)
ARG FINAL_BASE_IMAGE=nvidia/cuda:${CUDA_VERSION}-base-ubuntu22.04
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot Nov 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

The change to use nvidia/cuda:${CUDA_VERSION}-base-ubuntu22.04 as the FINAL_BASE_IMAGE will cause the Docker build to fail with the default CUDA_VERSION=12.9.1. The image tag nvidia/cuda:12.9.1-base-ubuntu22.04 does not exist on Docker Hub. This will break the build at the FROM ${FINAL_BASE_IMAGE} instruction. While CUDA_VERSION can be overridden at build time, the default value should correspond to an existing image to ensure the Dockerfile is buildable out-of-the-box.

Copy link
Contributor

@wzshiming wzshiming Nov 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked, the image exists.

https://hub.docker.com/layers/nvidia/cuda/12.9.1-base-ubuntu22.04


# By parameterizing the Deadsnakes repository URL, we allow third-party to use
# their own mirror. When doing so, we don't benefit from the transparent
Expand Down Expand Up @@ -328,6 +328,18 @@ RUN echo 'tzdata tzdata/Areas select America' | debconf-set-selections \
&& curl -sS ${GET_PIP_URL} | python${PYTHON_VERSION} \
&& python3 --version && python3 -m pip --version

# Install CUDA development tools and build essentials for runtime JIT compilation
# (FlashInfer, DeepGEMM, EP kernels all require compilation at runtime)
RUN CUDA_VERSION_DASH=$(echo $CUDA_VERSION | cut -d. -f1,2 | tr '.' '-') && \
Copy link
Contributor

@wzshiming wzshiming Nov 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The previous step was missing rm -rf /var/lib/apt/lists/*, I think this can be merged with the previous step to reduce duplication of updates and reduce the volume a little more

Copy link
Contributor Author

@bbartels bbartels Nov 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is another PR that's open that addresses the previous rm -rf /var/lib/apt/lists/* being missing. #29060 I didn't want to cause any conflicts there. But happy to address it here instead if you prefer :)

apt-get update -y && \
apt-get install -y --no-install-recommends \
cuda-nvcc-${CUDA_VERSION_DASH} \
cuda-cudart-${CUDA_VERSION_DASH} \
cuda-nvrtc-${CUDA_VERSION_DASH} \
cuda-cuobjdump-${CUDA_VERSION_DASH} \
libcublas-${CUDA_VERSION_DASH} && \
rm -rf /var/lib/apt/lists/*

ARG PIP_INDEX_URL UV_INDEX_URL
ARG PIP_EXTRA_INDEX_URL UV_EXTRA_INDEX_URL
ARG PYTORCH_CUDA_INDEX_BASE_URL
Expand Down
Binary file modified docs/assets/contributing/dockerfile-stages-dependency.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.