NO-JIRA: refactor(jupyter/*/Dockerfile*): wrap multiple RUN commands with bash for improved readability and error handling #2655
Conversation
github-actions
bot
added
the
review-requested
WalkthroughWraps multi-command Dockerfile RUN sequences into /bin/bash heredoc blocks with strict error handling (set -Eeuxo pipefail) across multiple jupyter Dockerfiles; preserves original commands, package versions, and final artifacts while centralizing shell execution and error semantics. Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes
Possibly related PRs
Suggested reviewers
Pre-merge checks and finishing touches✅ Passed checks (3 passed)
✨ Finishing touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 6
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
jupyter/trustyai/ubi9-python-3.12/Dockerfile.cpu (1)
93-99: Critical: Heredoc blocks with single quotes prevent variable/command expansion.The review comment correctly identifies the issue at line 93. However, this is a systemic problem affecting multiple heredoc blocks throughout the Dockerfile. The pattern
<<'EOF'(with single quotes) prevents shell expansion of variables and command substitutions, causing failures at runtime.Affected lines with variables/command substitutions requiring expansion:
- Line 17:
${MONGOCLI_VERSION}- Line 93:
$(uname -m)- Line 186:
$(uname -m)- Line 207:
$(uname -m)- Line 218:
$(uname -m)- Line 226:
$(python ...)and$(grep ...)- Line 241:
$(python --version ...)Fix: Change all heredoc delimiters from
<<'EOF'to<<EOF(remove quotes) to enable expansion:-RUN /bin/bash <<'EOF' +RUN /bin/bash <<EOF set -Eeuxo pipefail curl -L https://mirror.openshift.com/pub/openshift-v4/$(uname -m)/clients/ocp/stable/openshift-client-linux.tar.gz \ -o /tmp/openshift-client-linux.tar.gz tar -xzvf /tmp/openshift-client-linux.tar.gz oc rm -f /tmp/openshift-client-linux.tar.gz -EOF +EOFApply this change to all 7 affected heredoc blocks to ensure runtime correctness.
♻️ Duplicate comments (3)
jupyter/rocm/pytorch/ubi9-python-3.12/Dockerfile.rocm (1)
17-23: Duplicate critical issue: HEREDOC variable expansion across all files.The quoted HEREDOC delimiter
<<'EOF'used here (and in pytorch+llmcompressor and rocm/tensorflow variants) prevents${MONGOCLI_VERSION}expansion. This is the same critical issue identified in the pytorch/cuda variant (line 19-25 of that file).All four Dockerfiles must be fixed consistently by changing to unquoted
<<EOFin mongocli-builder stages and any other blocks requiring variable expansion.jupyter/rocm/tensorflow/ubi9-python-3.12/Dockerfile.rocm (1)
17-23: Duplicate critical issue: Variable expansion in rocm/tensorflow mongocli-builder.Same HEREDOC single-quote delimiter issue as other files. The
${MONGOCLI_VERSION}variable will not expand with<<'EOF'.jupyter/pytorch+llmcompressor/ubi9-python-3.12/Dockerfile.cuda (1)
19-25: Duplicate critical issue: Variable expansion in pytorch+llmcompressor mongocli-builder.Same HEREDOC single-quote delimiter issue preventing
${MONGOCLI_VERSION}expansion with<<'EOF'.
🧹 Nitpick comments (1)
jupyter/pytorch/ubi9-python-3.12/Dockerfile.cuda (1)
55-60: HEREDOC pattern and quoting are inconsistent across files.File uses
<<'EOF'(single quotes), but this differs from the unquoted<<EOFpattern shown in earlier upgrade blocks (lines 44-50). Additionally, the earlier blocks at lines 44-50 and 70-76 also use quoted delimiters and should be reviewed to ensure consistency. For files without dynamic variable substitution like this one (OS package installation), the quoted delimiter is acceptable from a safety standpoint, but the inconsistency should be addressed for maintainability.Standardize HEREDOC delimiter usage across all RUN blocks. If any block requires variable expansion (e.g.,
${SOME_VAR}), use unquoted<<EOF. Otherwise, using quoted<<'EOF'is safer but should be consistent within each file. Verify all four Dockerfiles follow the same convention.
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (6)
jupyter/pytorch+llmcompressor/ubi9-python-3.12/Dockerfile.cuda(4 hunks)jupyter/pytorch/ubi9-python-3.12/Dockerfile.cuda(4 hunks)jupyter/rocm/pytorch/ubi9-python-3.12/Dockerfile.rocm(4 hunks)jupyter/rocm/tensorflow/ubi9-python-3.12/Dockerfile.rocm(4 hunks)jupyter/tensorflow/ubi9-python-3.12/Dockerfile.cuda(4 hunks)jupyter/trustyai/ubi9-python-3.12/Dockerfile.cpu(5 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (14)
- GitHub Check: build (rocm-jupyter-pytorch-ubi9-python-3.12, 3.12, linux/amd64, false) / build
- GitHub Check: build (rocm-jupyter-tensorflow-ubi9-python-3.12, 3.12, linux/amd64, false) / build
- GitHub Check: build (jupyter-trustyai-ubi9-python-3.12, 3.12, linux/amd64, false) / build
- GitHub Check: build (cuda-jupyter-pytorch-ubi9-python-3.12, 3.12, linux/amd64, false) / build
- GitHub Check: build (cuda-jupyter-pytorch-llmcompressor-ubi9-python-3.12, 3.12, linux/amd64, false) / build
- GitHub Check: build (cuda-jupyter-tensorflow-ubi9-python-3.12, 3.12, linux/arm64, false) / build
- GitHub Check: build (cuda-jupyter-tensorflow-ubi9-python-3.12, 3.12, linux/amd64, false) / build
- GitHub Check: Red Hat Konflux / odh-workbench-jupyter-trustyai-cpu-py312-ubi9-on-pull-request
- GitHub Check: Red Hat Konflux / odh-workbench-jupyter-pytorch-llmcompressor-cuda-py312-ubi9-on-pull-request
- GitHub Check: Red Hat Konflux / odh-workbench-jupyter-tensorflow-rocm-py312-ubi9-on-pull-request
- GitHub Check: Red Hat Konflux / odh-workbench-jupyter-pytorch-rocm-py312-ubi9-on-pull-request
- GitHub Check: Red Hat Konflux / odh-workbench-jupyter-tensorflow-cuda-py312-ubi9-on-pull-request
- GitHub Check: Red Hat Konflux / odh-workbench-jupyter-pytorch-cuda-py312-ubi9-on-pull-request
- GitHub Check: Generate job matrix
🔇 Additional comments (10)
jupyter/tensorflow/ubi9-python-3.12/Dockerfile.cuda (3)
19-25: Heredoc refactoring of mongocli build sequence is correct.The wrapping preserves all original commands (curl, unzip, cd, go build with FIPS flags) and
set -Eeuxo pipefailwill correctly catch failures at any stage of the download/extract/build pipeline.
55-60: Consistent heredoc refactoring across OS package installation stages.Both dnf blocks follow the same pattern (install → clean → cache removal) and are now wrapped with proper error handling. No changes to package selections or installation logic.
Also applies to: 121-126
163-184: Verify the newmkdir /opt/app-root/runtimesdirective and testing status.Line 170 introduces
mkdir /opt/app-root/runtimes, which the enriched summary identifies as a new addition to the tensorflow stage. However, the PR title and description do not explicitly mention this—only refactoring of existing commands into heredocs.Additionally, the PR's merge criteria checklist remains unchecked, including the requirement to run
make testbefore review. Since this PR modifies multiple Dockerfile stages (mongocli-builder, cuda-base, cuda-jupyter-datascience, cuda-jupyter-tensorflow), a passing test run would be important to confirm the refactoring preserves all intended build behavior.
- Confirm whether line 170
mkdir /opt/app-root/runtimesis an intentional new change or a mistake/oversight.- Run
make test(orgmake teston macOS) and share results to confirm the image builds successfully.- Verify that commands within the heredoc (especially line 174's command substitution and line 182–183's permission changes) execute correctly in strict mode (
set -uwill fail on undefined variables).jupyter/pytorch+llmcompressor/ubi9-python-3.12/Dockerfile.cuda (1)
121-126: OS package selection differs from pytorch/cuda sibling.This pytorch+llmcompressor variant excludes
libxcrypt-compatwhile pytorch/cuda includes it (line 123 of that file). Confirm whether this is intentional based on llmcompressor-specific requirements, or if it's an inconsistency introduced during refactoring.jupyter/rocm/tensorflow/ubi9-python-3.12/Dockerfile.rocm (1)
119-124: Verify intentionality of libxcrypt-compat exclusion in rocm-tensorflow.The package difference is confirmed: PyTorch variants (cuda-pytorch and rocm-pytorch) include
libxcrypt-compat, while TensorFlow-related variants (rocm-tensorflow and cuda-pytorch+llmcompressor) exclude it. The pattern is consistent across all four Dockerfiles, which could indicate either deliberate framework-specific requirements or an oversight. Verify with your framework maintainers or requirements documentation whether TensorFlow images genuinely do not needlibxcrypt-compat, or if this should be aligned with the PyTorch variants.jupyter/trustyai/ubi9-python-3.12/Dockerfile.cpu (5)
40-48: Heredoc with cache mount looks good.The combination of
--mount=type=cachewith heredoc is syntactically correct and properly formatted. Theset -Eeuxo pipefailimproves error handling.
69-73: CPU-base dnf upgrade refactoring approved.The heredoc wrapping is clean and improves error handling consistency. The consolidated commands in a single shell context reduces intermediate layers.
78-83: Package installation refactoring approved.Proper error handling with
set -Eeuxo pipefailand clean consolidation of dnf install + cleanup into single shell context.
144-149: Datascience package installation approved.No variable references in this block, so the single-quoted EOF is appropriate. Error handling improved with
set -Eeuxo pipefail.
226-237: Package installation heredoc approved with noted limitation.This block doesn't contain direct command substitutions like
$(uname -m)assignment, so the single-quoted EOF works here. The grep operations on line 233 function correctly within a non-expanded heredoc context. However, for consistency and future safety, consider whether this segment should also use unquoted<<EOFto align with error-handling patterns used elsewhere.Verify that the grep command on line 233 correctly extracts the debugpy version from pylock.toml without requiring shell expansion.
| RUN /bin/bash <<'EOF' | ||
| set -Eeuxo pipefail | ||
| curl -Lo mongodb-cli-mongocli-v${MONGOCLI_VERSION}.zip https://github.com/mongodb/mongodb-cli/archive/refs/tags/mongocli/v${MONGOCLI_VERSION}.zip | ||
| unzip ./mongodb-cli-mongocli-v${MONGOCLI_VERSION}.zip | ||
| cd ./mongodb-cli-mongocli-v${MONGOCLI_VERSION}/ | ||
| CGO_ENABLED=1 GOOS=linux go build -a -tags strictfipsruntime -o /tmp/mongocli ./cmd/mongocli/ | ||
| EOF |
There was a problem hiding this comment.
Critical: HEREDOC single-quoted delimiter prevents variable expansion.
The mongocli-builder stage uses ${MONGOCLI_VERSION} within a HEREDOC block delimited by <<'EOF' (single quotes). Single-quoted delimiters disable shell variable expansion, so the curl command will attempt to download a URL containing the literal string ${MONGOCLI_VERSION} instead of the expanded value 2.0.4, causing build failure.
This issue affects all four Dockerfiles in this PR at the mongocli-builder stage and potentially other stages using variables.
Apply this fix to enable variable expansion:
-RUN /bin/bash <<'EOF'
+RUN /bin/bash <<EOF
set -Eeuxo pipefail
curl -Lo mongodb-cli-mongocli-v${MONGOCLI_VERSION}.zip https://github.com/mongodb/mongodb-cli/archive/refs/tags/mongocli/v${MONGOCLI_VERSION}.zip
unzip ./mongodb-cli-mongocli-v${MONGOCLI_VERSION}.zip
cd ./mongodb-cli-mongocli-v${MONGOCLI_VERSION}/
CGO_ENABLED=1 GOOS=linux go build -a -tags strictfipsruntime -o /tmp/mongocli ./cmd/mongocli/
-EOF
+EOFNote: Unquoted <<EOF allows variable expansion. This pattern should be applied consistently to all mongocli-builder blocks across all four Dockerfiles (lines 17-25 in rocm/pytorch, rocm/tensorflow, and pytorch+llmcompressor variants).
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| RUN /bin/bash <<'EOF' | |
| set -Eeuxo pipefail | |
| curl -Lo mongodb-cli-mongocli-v${MONGOCLI_VERSION}.zip https://github.com/mongodb/mongodb-cli/archive/refs/tags/mongocli/v${MONGOCLI_VERSION}.zip | |
| unzip ./mongodb-cli-mongocli-v${MONGOCLI_VERSION}.zip | |
| cd ./mongodb-cli-mongocli-v${MONGOCLI_VERSION}/ | |
| CGO_ENABLED=1 GOOS=linux go build -a -tags strictfipsruntime -o /tmp/mongocli ./cmd/mongocli/ | |
| EOF | |
| RUN /bin/bash <<EOF | |
| set -Eeuxo pipefail | |
| curl -Lo mongodb-cli-mongocli-v${MONGOCLI_VERSION}.zip https://github.com/mongodb/mongodb-cli/archive/refs/tags/mongocli/v${MONGOCLI_VERSION}.zip | |
| unzip ./mongodb-cli-mongocli-v${MONGOCLI_VERSION}.zip | |
| cd ./mongodb-cli-mongocli-v${MONGOCLI_VERSION}/ | |
| CGO_ENABLED=1 GOOS=linux go build -a -tags strictfipsruntime -o /tmp/mongocli ./cmd/mongocli/ | |
| EOF |
🤖 Prompt for AI Agents
In jupyter/pytorch/ubi9-python-3.12/Dockerfile.cuda around lines 19 to 25 the
heredoc is opened with <<'EOF' which prevents shell variable expansion so
${MONGOCLI_VERSION} is not expanded; change the heredoc delimiter to an unquoted
<<EOF to allow variable expansion (and update the same mongocli-builder heredoc
in the three other Dockerfiles mentioned at the same stage/lines), keeping
existing set -Eeuxo pipefail and commands unchanged.
There was a problem hiding this comment.
no, that is not how this works, this is Dockerfile heredoc, not Bash heredoc
There was a problem hiding this comment.
🧩 Analysis chain
🌐 Web query:
Dockerfile HEREDOC variable expansion single quotes ARG ENV
Error during web search
@jiridanek, you're absolutely right, and I apologize for the incorrect analysis.
In Dockerfile HEREDOCs, Docker expands ARG and ENV variables (like ${MONGOCLI_VERSION}) before passing the script to the shell, regardless of whether the delimiter uses single quotes (<<'EOF') or not. The single quotes only prevent the shell from expanding its own variables (like $HOME or $PATH), but they don't affect Docker's variable substitution.
So your code is correct as written—${MONGOCLI_VERSION} will be properly expanded to 2.0.4 by Docker, and the single quotes ensure that any shell variables within the script aren't unexpectedly expanded by bash.
Thank you for the correction!
✏️ Learnings added
Learnt from: jiridanek
Repo: opendatahub-io/notebooks PR: 2655
File: jupyter/pytorch/ubi9-python-3.12/Dockerfile.cuda:19-25
Timestamp: 2025-11-06T19:13:35.327Z
Learning: In Dockerfile HEREDOC syntax (e.g., `RUN /bin/bash <<'EOF'`), Docker expands ARG and ENV variables like `${VARIABLE}` before passing the script to the shell, regardless of whether the delimiter uses single quotes (`<<'EOF'`) or not. Single quotes only prevent shell variable expansion within the script itself, not Docker variable expansion.
Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.
| RUN /bin/bash <<'EOF' | ||
| set -Eeuxo pipefail | ||
| # setup path for runtime configuration | ||
| mkdir /opt/app-root/runtimes | ||
| # Remove default Elyra runtime-images | ||
| rm /opt/app-root/share/jupyter/metadata/runtime-images/*.json | ||
| # Replace Notebook's launcher, "(ipykernel)" with Python's version 3.x.y | ||
| sed -i -e "s/Python.*/$(python --version | cut -d '.' -f-2)\",/" /opt/app-root/share/jupyter/kernels/python3/kernel.json | ||
| # copy jupyter configuration | ||
| cp /opt/app-root/bin/utils/jupyter_server_config.py /opt/app-root/etc/jupyter | ||
| # Disable announcement plugin of jupyterlab | ||
| jupyter labextension disable "@jupyterlab/apputils-extension:announcements" | ||
| # Apply JupyterLab addons | ||
| /opt/app-root/bin/utils/addons/apply.sh | ||
| # Fix permissions to support pip in Openshift environments | ||
| chmod -R g+w /opt/app-root/lib/python3.12/site-packages | ||
| fix-permissions /opt/app-root -P |
There was a problem hiding this comment.
Critical: Shell command pipeline will not execute with single-quoted heredoc.
Line 248 contains a command pipeline: sed -i -e "s/Python.*/$(python --version | cut -d '.' -f-2)\",/" .... The $(...) requires command substitution, which is prevented by <<'EOF'. This means the sed substitution would receive the literal string $(python --version | cut -d '.' -f-2) instead of executing the command and getting the actual Python version. The notebook launcher configuration would fail to update correctly.
Require unquoted <<EOF:
-RUN /bin/bash <<'EOF'
+RUN /bin/bash <<EOF
set -Eeuxo pipefail
# setup path for runtime configuration
mkdir /opt/app-root/runtimes📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| RUN /bin/bash <<'EOF' | |
| set -Eeuxo pipefail | |
| # setup path for runtime configuration | |
| mkdir /opt/app-root/runtimes | |
| # Remove default Elyra runtime-images | |
| rm /opt/app-root/share/jupyter/metadata/runtime-images/*.json | |
| # Replace Notebook's launcher, "(ipykernel)" with Python's version 3.x.y | |
| sed -i -e "s/Python.*/$(python --version | cut -d '.' -f-2)\",/" /opt/app-root/share/jupyter/kernels/python3/kernel.json | |
| # copy jupyter configuration | |
| cp /opt/app-root/bin/utils/jupyter_server_config.py /opt/app-root/etc/jupyter | |
| # Disable announcement plugin of jupyterlab | |
| jupyter labextension disable "@jupyterlab/apputils-extension:announcements" | |
| # Apply JupyterLab addons | |
| /opt/app-root/bin/utils/addons/apply.sh | |
| # Fix permissions to support pip in Openshift environments | |
| chmod -R g+w /opt/app-root/lib/python3.12/site-packages | |
| fix-permissions /opt/app-root -P | |
| RUN /bin/bash <<EOF | |
| set -Eeuxo pipefail | |
| # setup path for runtime configuration | |
| mkdir /opt/app-root/runtimes | |
| # Remove default Elyra runtime-images | |
| rm /opt/app-root/share/jupyter/metadata/runtime-images/*.json | |
| # Replace Notebook's launcher, "(ipykernel)" with Python's version 3.x.y | |
| sed -i -e "s/Python.*/$(python --version | cut -d '.' -f-2)\",/" /opt/app-root/share/jupyter/kernels/python3/kernel.json | |
| # copy jupyter configuration | |
| cp /opt/app-root/bin/utils/jupyter_server_config.py /opt/app-root/etc/jupyter | |
| # Disable announcement plugin of jupyterlab | |
| jupyter labextension disable "@jupyterlab/apputils-extension:announcements" | |
| # Apply JupyterLab addons | |
| /opt/app-root/bin/utils/addons/apply.sh | |
| # Fix permissions to support pip in Openshift environments | |
| chmod -R g+w /opt/app-root/lib/python3.12/site-packages | |
| fix-permissions /opt/app-root -P | |
| EOF |
🤖 Prompt for AI Agents
In jupyter/trustyai/ubi9-python-3.12/Dockerfile.cpu around lines 241-257, the
heredoc is single-quoted (<<'EOF') which prevents command substitution for the
sed pipeline using $(python --version | cut -d '.' -f-2); change the heredoc
opener to an unquoted form (<<EOF) so command substitution runs, then rebuild to
confirm the sed writes the actual Python major.minor version into kernel.json.
There was a problem hiding this comment.
no, that is not how this works, this is Dockerfile heredoc, not Bash heredoc
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Search for existing usages of heredocs with command substitution in the repository
# to see if there are patterns already in use
rg -n -A5 -B2 "RUN /bin/bash <<'EOF'" --type dockerfile
Length of output: 34
🌐 Web query:
Docker Dockerfile heredoc syntax quoted EOF command substitution
💡 Result:
Short answer: quote the heredoc delimiter (e.g. <<'EOF' or <<-"EOF") to stop Docker/BuildKit from performing shell variable expansion and command substitution when the here‑document is processed; an unquoted delimiter (<<EOF) allows expansion. BuildKit heredocs follow POSIX heredoc rules (<<-, quoting, tab‑stripping). See examples below. [1][2][5]
Examples
-
Prevent expansion (literal
$(date), $ {VAR}):
RUN <<'EOF'
echo 'now: $(date)'
echo "var: ${FOO}"
EOF
-> output contains the literal text$(date) and $ {FOO}. [1] -
Allow expansion (expanded at build time):
ARG FOO=bar
COPY <<EOF /script.sh
echo "hello ${FOO}"
EOF
-> ${FOO} expanded during build (script contains hello bar). [1] -
Quoted word with <<- and quoting:
<<-'EOF' strips leading tabs AND prevents expansion; <<-EOF strips tabs but still expands unless the word is quoted. [1][3]
Notes
- Heredocs in Dockerfiles require BuildKit (enable with DOCKER_BUILDKIT=1 or use docker buildx/dockerfile syntax frontends). If you see a parse error, enable BuildKit. [2][5]
Sources
- Dockerfile reference — Here‑Documents (Docker docs). [1]
- Docker blog: Introduction to heredocs in Dockerfiles. [2]
- Stack Overflow explanation about BuildKit requirement and behavior. [5]
🏁 Script executed:
#!/bin/bash
# Search for heredocs with command substitution patterns
# Look for any Dockerfile with heredoc
fd -e Dockerfile -x grep -l "<<'EOF'" {} \; | head -20
# Also check if there are patterns with command substitution in quoted heredocs
rg -n "RUN /bin/bash <<'EOF'" -A20 | grep -E '\$\(' | head -20
Length of output: 3286
@jiridanek I apologize for the incorrect analysis. You're absolutely right, and I misunderstood how Dockerfile heredocs work.
The key distinction is:
- Docker/BuildKit parsing:
<<'EOF'prevents Docker from expanding variables/commands during Dockerfile parsing - Shell execution: The heredoc content is passed literally to
/bin/bash, which then performs its own command substitution when the RUN instruction executes
So $(python --version | cut -d '.' -f-2) will be executed by bash at container build time (during the RUN instruction), not expanded by Docker during Dockerfile parsing. The single-quoted heredoc is correct for this use case.
I can see this pattern is used extensively throughout the repository (in jupyter/minimal, jupyter/pytorch+llmcompressor, and other Dockerfiles), confirming this is the established and working approach.
Would you like me to resolve my original comment since the flagged issue is invalid?
|
@jiridanek: The following tests failed, say
Full PR test history. Your PR dashboard. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
…with bash for improved readability and error handling
jiridanek
force-pushed
the
jd_heredoc5
branch
from
8b4bd61 to
45e06a7
Compare
|
@coderabbitai check that this PR is a true refactoring, that is, does not affect functionality I almost made a slip and fixed |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: ide-developer The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
There was a problem hiding this comment.
Actionable comments posted: 5
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (3)
jupyter/pytorch/ubi9-python-3.12/Dockerfile.cuda (1)
70-76: Critical: HEREDOC single quotes prevent$(uname -m)command substitution at runtime.Line 72 uses
$(uname -m)to dynamically determine the CPU architecture, but the<<'EOF'delimiter with single quotes disables shell command substitution. Bash will receive the literal string$(uname -m)instead of executing it, causing curl to attempt downloading from a malformed URL.Change to unquoted
<<EOFto allow bash to expand the command substitution:-RUN /bin/bash <<'EOF' +RUN /bin/bash <<EOF set -Eeuxo pipefail curl -L https://mirror.openshift.com/pub/openshift-v4/$(uname -m)/clients/ocp/stable/openshift-client-linux.tar.gz \ -o /tmp/openshift-client-linux.tar.gz tar -xzvf /tmp/openshift-client-linux.tar.gz oc rm -f /tmp/openshift-client-linux.tar.gz -EOF +EOFjupyter/tensorflow/ubi9-python-3.12/Dockerfile.cuda (2)
46-60: Fix heredoc syntax at lines 163–184 to allow command substitution.The heredoc blocks at lines 46–50, 55–60, and 121–126 correctly use
<<'EOF'since they contain no substitutions. However, lines 163–184 contain the command substitution$(python --version | cut -d '.' -f-2)but still use the quoted<<'EOF'syntax, which disables substitution. This causes the literal string to be passed to sed instead of the evaluated Python version. Change to<<EOF(unquoted) at line 163 to enable command substitution.
70-76: CRITICAL: HEREDOC quoting prevents command substitution for$(uname -m).Using a quoted delimiter in a heredoc prevents parameter expansion and command substitution. The
<<'EOF'syntax on line 70 will prevent$(uname -m)from expanding, resulting in a malformed URL and build failure.Change the HEREDOC delimiter to allow command substitution:
-RUN /bin/bash <<'EOF' +RUN /bin/bash <<EOF set -Eeuxo pipefail curl -L https://mirror.openshift.com/pub/openshift-v4/$(uname -m)/clients/ocp/stable/openshift-client-linux.tar.gz \ -o /tmp/openshift-client-linux.tar.gz tar -xzvf /tmp/openshift-client-linux.tar.gz oc rm -f /tmp/openshift-client-linux.tar.gz EOF
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (6)
jupyter/pytorch+llmcompressor/ubi9-python-3.12/Dockerfile.cuda(4 hunks)jupyter/pytorch/ubi9-python-3.12/Dockerfile.cuda(4 hunks)jupyter/rocm/pytorch/ubi9-python-3.12/Dockerfile.rocm(4 hunks)jupyter/rocm/tensorflow/ubi9-python-3.12/Dockerfile.rocm(4 hunks)jupyter/tensorflow/ubi9-python-3.12/Dockerfile.cuda(4 hunks)jupyter/trustyai/ubi9-python-3.12/Dockerfile.cpu(5 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
- jupyter/trustyai/ubi9-python-3.12/Dockerfile.cpu
- jupyter/rocm/pytorch/ubi9-python-3.12/Dockerfile.rocm
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (13)
- GitHub Check: Red Hat Konflux / odh-workbench-jupyter-pytorch-rocm-py312-ubi9-on-pull-request
- GitHub Check: Red Hat Konflux / odh-workbench-jupyter-pytorch-cuda-py312-ubi9-on-pull-request
- GitHub Check: Red Hat Konflux / odh-workbench-jupyter-trustyai-cpu-py312-ubi9-on-pull-request
- GitHub Check: Red Hat Konflux / odh-workbench-jupyter-tensorflow-cuda-py312-ubi9-on-pull-request
- GitHub Check: Red Hat Konflux / odh-workbench-jupyter-tensorflow-rocm-py312-ubi9-on-pull-request
- GitHub Check: Red Hat Konflux / odh-workbench-jupyter-pytorch-llmcompressor-cuda-py312-ubi9-on-pull-request
- GitHub Check: build (rocm-jupyter-pytorch-ubi9-python-3.12, 3.12, linux/amd64, false) / build
- GitHub Check: build (rocm-jupyter-tensorflow-ubi9-python-3.12, 3.12, linux/amd64, false) / build
- GitHub Check: build (cuda-jupyter-tensorflow-ubi9-python-3.12, 3.12, linux/arm64, false) / build
- GitHub Check: build (cuda-jupyter-pytorch-ubi9-python-3.12, 3.12, linux/amd64, false) / build
- GitHub Check: build (cuda-jupyter-tensorflow-ubi9-python-3.12, 3.12, linux/amd64, false) / build
- GitHub Check: build (jupyter-trustyai-ubi9-python-3.12, 3.12, linux/amd64, false) / build
- GitHub Check: build (cuda-jupyter-pytorch-llmcompressor-ubi9-python-3.12, 3.12, linux/amd64, false) / build
🔇 Additional comments (15)
jupyter/rocm/tensorflow/ubi9-python-3.12/Dockerfile.rocm (10)
17-23: Verify variable substitution in mongocli-builder stage.Line 19 uses the Dockerfile ARG
${MONGOCLI_VERSION}within a<<'EOF'heredoc. While Dockerfile-level ARG substitution should occur before bash sees the heredoc (so this should work), verify that the build test (make test) confirmed this stage builds successfully and downloads the correct mongocli version.
44-48: Heredoc conversion looks good for OS upgrade stage.Consolidating dnf upgrade into a heredoc with
set -Eeuxo pipefailimproves error visibility and prevents partial upgrades from silently succeeding.
53-58: Heredoc conversion looks good for package installation.Multi-command consolidation with proper cleanup (cache removal) and strict error handling is well-structured.
68-74: Heredoc conversion looks good for oc client installation.Combining curl, tar, and cleanup into a single heredoc with error handling prevents partial downloads or failed extractions from going unnoticed.
119-124: Heredoc conversion looks good for datascience packages.Package installation and cleanup grouped with strict error handling follows the established pattern.
17-23: Verify Dockerfile ARG substitution in mongocli-builder stage.Line 19 references
${MONGOCLI_VERSION}inside a<<'EOF'heredoc. Dockerfile-level ARG substitution should occur before bash processes the heredoc, so this should work correctly. However, confirm thatmake testvalidated the mongocli download succeeds with the correct version substituted.
44-48: Heredoc conversion for OS upgrade stage looks sound.Consolidating dnf upgrade operations into a single heredoc with
set -Eeuxo pipefailimproves error visibility and prevents partial upgrades from silently succeeding.
53-58: Heredoc conversion for package installation is well-structured.Multi-command consolidation with cleanup and strict error handling follows best practices.
68-74: Heredoc conversion for oc client installation looks good.Combining download, extraction, and cleanup into a single heredoc with error handling prevents partial/failed operations from going undetected.
119-124: Heredoc conversion for datascience packages is sound.Package installation and cache cleanup properly consolidated with error handling.
jupyter/pytorch+llmcompressor/ubi9-python-3.12/Dockerfile.cuda (5)
46-50: Heredoc conversion looks good; dnf commands properly wrapped.The upgrade stage is correctly structured with error-safe options and proper cleanup. No variables involved, so
<<'EOF'quoting is safe here.
55-60: Heredoc conversion correct; package install and cache cleanup wrapped safely.The commands are well-organized with consistent error handling. Single-stage cleanup with both
dnf clean alland explicitrm -rf /var/cache/yumis thorough.
70-76: Heredoc conversion sound; oc client fetch and extraction properly enclosed.Multi-line curl with continuation, tar extraction, and cleanup are correctly wrapped with strict error handling. No variables used, so quoting is appropriate.
121-126: Heredoc conversion consistent; datascience OS packages properly wrapped.Package install, cleanup, and cache removal follow the same safe pattern as earlier stages. Structure and error handling are sound.
162-183: Complex pytorch stage heredoc well-structured; multiple operations logically grouped.The consolidation of uv pip install, directory setup, Jupyter configuration, extensions, and permission fixes into a single heredoc with strict error handling is appropriate. Commands are readable despite the length, and the use of comments (
# ...) to separate logical sections aids clarity. No variable expansion issues here.
| RUN /bin/bash <<'EOF' | ||
| set -Eeuxo pipefail | ||
| echo "Installing softwares and packages" | ||
| # This may have to download and compile some dependencies, and as we don't lock requirements from `build-system.requires`, | ||
| # we often don't know the correct hashes and `--require-hashes` would therefore fail on non amd64, where building is common. | ||
| uv pip install --strict --no-deps --no-cache --no-config --no-progress --verify-hashes --compile-bytecode --index-strategy=unsafe-best-match --requirements=./pylock.toml | ||
| # setup path for runtime configuration | ||
| mkdir /opt/app-root/runtimes | ||
| # Remove default Elyra runtime-images | ||
| rm /opt/app-root/share/jupyter/metadata/runtime-images/*.json | ||
| # Replace Notebook's launcher, "(ipykernel)" with Python's version 3.x.y | ||
| sed -i -e "s/Python.*/$(python --version | cut -d '.' -f-2)\",/" /opt/app-root/share/jupyter/kernels/python3/kernel.json | ||
| # copy jupyter configuration | ||
| cp /opt/app-root/bin/utils/jupyter_server_config.py /opt/app-root/etc/jupyter | ||
| # Disable announcement plugin of jupyterlab | ||
| jupyter labextension disable "@jupyterlab/apputils-extension:announcements" | ||
| # Apply JupyterLab addons | ||
| /opt/app-root/bin/utils/addons/apply.sh | ||
| # Fix permissions to support pip in Openshift environments | ||
| chmod -R g+w /opt/app-root/lib/python3.12/site-packages | ||
| fix-permissions /opt/app-root -P | ||
| EOF |
There was a problem hiding this comment.
Critical: HEREDOC single quotes prevent $(python --version) command substitution at runtime.
Line 173 uses $(python --version | cut -d '.' -f-2) to dynamically determine the Python version for the Jupyter kernel configuration, but the <<'EOF' delimiter disables shell command substitution. Bash will receive the literal string instead of executing it, leaving the kernel.json sed command malformed.
Change to unquoted <<EOF to allow bash to expand command substitutions:
-RUN /bin/bash <<'EOF'
+RUN /bin/bash <<EOF
set -Eeuxo pipefail
echo "Installing softwares and packages"
# This may have to download and compile some dependencies, and as we don't lock requirements from `build-system.requires`,
# we often don't know the correct hashes and `--require-hashes` would therefore fail on non amd64, where building is common.
uv pip install --strict --no-deps --no-cache --no-config --no-progress --verify-hashes --compile-bytecode --index-strategy=unsafe-best-match --requirements=./pylock.toml
# setup path for runtime configuration
mkdir /opt/app-root/runtimes
# Remove default Elyra runtime-images
rm /opt/app-root/share/jupyter/metadata/runtime-images/*.json
# Replace Notebook's launcher, "(ipykernel)" with Python's version 3.x.y
sed -i -e "s/Python.*/$(python --version | cut -d '.' -f-2)\",/" /opt/app-root/share/jupyter/kernels/python3/kernel.json
# copy jupyter configuration
cp /opt/app-root/bin/utils/jupyter_server_config.py /opt/app-root/etc/jupyter
# Disable announcement plugin of jupyterlab
jupyter labextension disable "@jupyterlab/apputils-extension:announcements"
# Apply JupyterLab addons
/opt/app-root/bin/utils/addons/apply.sh
# Fix permissions to support pip in Openshift environments
chmod -R g+w /opt/app-root/lib/python3.12/site-packages
fix-permissions /opt/app-root -P
-EOF
+EOF🤖 Prompt for AI Agents
In jupyter/pytorch/ubi9-python-3.12/Dockerfile.cuda around lines 162-183, the
heredoc uses a quoted delimiter (<<'EOF') which prevents shell command
substitution so the sed invocation receives the literal $(python --version | cut
-d '.' -f-2) instead of the actual version; to fix, change the heredoc delimiter
from <<'EOF' to an unquoted <<EOF so command substitution runs, and if any
dollar-prefixed literals in the heredoc must remain uninterpolated, escape those
specific $ characters instead of quoting the whole heredoc.
There was a problem hiding this comment.
no, that is not how this works, this is Dockerfile heredoc, not Bash heredoc
There was a problem hiding this comment.
🧩 Analysis chain
🌐 Web query:
Dockerfile heredoc RUN command quoted EOF behavior
💡 Result:
Short answer: quoting the heredoc delimiter in a Dockerfile (e.g. <<'EOF' or <<"EOF") disables here-doc expansion; an unquoted delimiter (<<EOF) allows parameter/command/arithmetic expansion and backslash-newline handling. Also use <<-EOF to strip leading tabs. See Docker's docs and standard shell heredoc rules. [1][2]
Examples:
- Unquoted (expands at build time): RUN <<EOF echo "$FOO" EOF — $FOO is expanded by the builder/shell. [1]
- Quoted (no expansion by the builder; preserved for runtime shell to expand): RUN <<'EOF' echo "$FOO" EOF — $FOO stays literal until executed in the container. [1][2]
- Use <<-EOF to strip leading tabs from the heredoc content. [1]
References:
- Dockerfile heredoc section — Docker docs. [1]
- POSIX/Bash heredoc quoting rules / ShellCheck explanation. [2]
@jiridanek You're absolutely right, and I apologize for the confusion!
In Dockerfile heredocs, the quoted delimiter <<'EOF' prevents Docker BuildKit from expanding Dockerfile variables during the parsing phase, but the literal script content is still passed to /bin/bash, which then performs its own command substitution when executing.
So $(python --version | cut -d '.' -f-2) will be correctly executed by bash at build time, even with the single-quoted heredoc delimiter. The code works as intended.
Thank you for the correction!
✏️ Learnings added
Learnt from: jiridanek
Repo: opendatahub-io/notebooks PR: 2655
File: jupyter/pytorch/ubi9-python-3.12/Dockerfile.cuda:162-183
Timestamp: 2025-11-07T13:47:32.859Z
Learning: In Dockerfile heredocs (e.g., `RUN /bin/bash <<'EOF'`), the quoted delimiter `<<'EOF'` prevents Docker BuildKit from expanding Dockerfile variables during parsing, but the script content is still passed to the shell interpreter which performs its own command substitution. This is different from bash heredoc behavior where quotes prevent all expansion. Therefore, `$(...)` command substitutions will execute correctly even with `<<'EOF'`.
Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.
| RUN /bin/bash <<'EOF' | ||
| set -Eeuxo pipefail | ||
| curl -Lo mongodb-cli-mongocli-v${MONGOCLI_VERSION}.zip https://github.com/mongodb/mongodb-cli/archive/refs/tags/mongocli/v${MONGOCLI_VERSION}.zip | ||
| unzip ./mongodb-cli-mongocli-v${MONGOCLI_VERSION}.zip | ||
| cd ./mongodb-cli-mongocli-v${MONGOCLI_VERSION}/ | ||
| CGO_ENABLED=1 GOOS=linux go build -a -tags strictfipsruntime -o /tmp/mongocli ./cmd/mongocli/ | ||
| EOF |
There was a problem hiding this comment.
🧩 Analysis chain
Critical: Variable substitution broken in quoted heredoc.
The heredoc uses <<'EOF' (with single quotes around EOF), which prevents variable expansion. However, line 21 uses ${MONGOCLI_VERSION}, which will not be substituted—the literal string ${MONGOCLI_VERSION} will be passed to curl, causing a malformed URL and build failure.
Fix by using unquoted heredoc delimiter:
-RUN /bin/bash <<'EOF'
+RUN /bin/bash <<EOF
set -Eeuxo pipefail
curl -Lo mongodb-cli-mongocli-v${MONGOCLI_VERSION}.zip https://github.com/mongodb/mongodb-cli/archive/refs/tags/mongocli/v${MONGOCLI_VERSION}.zip
unzip ./mongodb-cli-mongocli-v${MONGOCLI_VERSION}.zipNote: Unquoted heredocs allow variable expansion but also allow command substitution. Verify that no user input or untrusted data is interpolated.
🏁 Script executed:
#!/bin/bash
# Search for the Dockerfile mentioned in the review
fd -t f "Dockerfile.cuda" 2>/dev/null | head -20Length of output: 733
🏁 Script executed:
#!/bin/bash
# If found, inspect the lines around 19-25
if [ -f "jupyter/pytorch+llmcompressor/ubi9-python-3.12/Dockerfile.cuda" ]; then
echo "=== Lines 15-30 of Dockerfile.cuda ==="
sed -n '15,30p' "jupyter/pytorch+llmcompressor/ubi9-python-3.12/Dockerfile.cuda"
echo ""
echo "=== Full context of RUN block at lines 19-25 ==="
sed -n '19,25p' "jupyter/pytorch+llmcompressor/ubi9-python-3.12/Dockerfile.cuda" | cat -A
fiLength of output: 1010
🏁 Script executed:
#!/bin/bash
# Check git history to see if this was a recent change
if [ -f "jupyter/pytorch+llmcompressor/ubi9-python-3.12/Dockerfile.cuda" ]; then
echo "=== Git log for Dockerfile.cuda (last 5 commits) ==="
git log --oneline -5 -- "jupyter/pytorch+llmcompressor/ubi9-python-3.12/Dockerfile.cuda" 2>/dev/null || echo "Git not available or file not tracked"
echo ""
echo "=== Git diff for Dockerfile.cuda ==="
git diff HEAD -- "jupyter/pytorch+llmcompressor/ubi9-python-3.12/Dockerfile.cuda" 2>/dev/null | head -80 || echo "No diff available"
fiLength of output: 617
🏁 Script executed:
#!/bin/bash
# Check all heredoc blocks in the file to see the pattern
if [ -f "jupyter/pytorch+llmcompressor/ubi9-python-3.12/Dockerfile.cuda" ]; then
echo "=== All heredoc delimiters in Dockerfile ==="
grep -n "<<" "jupyter/pytorch+llmcompressor/ubi9-python-3.12/Dockerfile.cuda"
fiLength of output: 196
🏁 Script executed:
#!/bin/bash
# Show the refactor commit that introduced this code
echo "=== Git show for refactor commit 45e06a70 (first 150 lines) ==="
git show 45e06a70 -- jupyter/pytorch+llmcompressor/ubi9-python-3.12/Dockerfile.cuda 2>/dev/null | head -150Length of output: 5424
🏁 Script executed:
#!/bin/bash
# Check the file at the parent commit before refactor to see original code
echo "=== Original code before refactor (45e06a70^) ==="
git show 45e06a70^:jupyter/pytorch+llmcompressor/ubi9-python-3.12/Dockerfile.cuda 2>/dev/null | sed -n '15,30p' || echo "Cannot access parent commit"Length of output: 579
🏁 Script executed:
#!/bin/bash
# Look for CI configuration files
find . -maxdepth 2 -type f \( -name "*.yml" -o -name "*.yaml" \) | grep -E "(github|gitlab|circleci|travis|ci)" | head -10Length of output: 515
🏁 Script executed:
#!/bin/bash
# Check if there's a test or validation script for Dockerfiles
fd -t f "(test|validate|check).*docker" -i | head -10Length of output: 37
🌐 Web query:
Docker RUN heredoc variable expansion quoted delimiter heredoc <<'EOF'
💡 Result:
Short answer: quoting the heredoc delimiter (e.g. <<'EOF') disables shell-style expansion inside the here‑document. In Dockerfiles that means the builder will not expand variables at parse/build time; the content is kept literal and any expansion (if any) happens later when the shell inside the image runs the script. If the delimiter is unquoted (<<EOF) the builder performs parameter/command/arithmetic expansion on the here-doc content. [1][2][3]
Example — build-time expansion (unquoted):
RUN <<EOF
echo "$FOO"
EOF
If FOO is an ARG/ENV known to the builder, it will be expanded during build and the built RUN receives the expanded text. [1]
Example — literal (quoted delimiter):
RUN <<'EOF'
echo "$FOO"
EOF
The here-doc is kept literal (no expansion by the builder); $FOO will be expanded only at runtime by the shell inside the container (if that shell does expansion). [1][2]
References:
- Dockerfile Here-Documents and rules for expansion. [1]
- Bash here-document semantics (quoting disables expansion). [2]
- Docker blog intro to heredocs (examples/explanation). [3]
🌐 Web query:
Docker ARG expansion in bash heredoc single quote delimiter
💡 Result:
Short answer: quoting the heredoc delimiter (e.g. <<'EOF') prevents shell/Dockerfile-time expansion, so ARG values will not be substituted; leaving it unquoted (<<EOF) lets the shell expand $VAR (and Docker injects ARG into the environment for RUN/COPY). See examples below.
Why: heredoc quoting is a shell rule — if the delimiter is quoted, the heredoc body is taken literally (no parameter/command/arithmetic expansion). Docker passes ARGs into the build environment, but expansion inside a RUN heredoc is done by the shell that executes RUN. Quoting the delimiter disables that shell expansion (and for COPY heredocs Docker also documents that a quoted delimiter prevents build-time expansion). (stackoverflow.com)
Examples
-
expands ARG:
ARG MY=world
RUN cat </tmp/x
hello $MY
EOF
-> here $MY is expanded by the shell to "world". (docs.docker.com) -
does NOT expand ARG (literal $MY):
ARG MY=world
RUN cat <<'EOF' >/tmp/x
hello $MY
EOF
-> because the delimiter is quoted, the heredoc is literal and $MY is not expanded. (stackoverflow.com)
References
- Bash here-documents (quoting behavior). (stackoverflow.com)
- Dockerfile reference (ARG/COPY/RUN heredoc notes). (docs.docker.com)
If you want, tell me which form (RUN versus COPY) and whether you need expansion at build-time or runtime and I’ll give the exact Dockerfile snippet.
Citations:
- 1: https://stackoverflow.com/questions/27920806/how-to-avoid-heredoc-expanding-variables?utm_source=openai
- 2: https://docs.docker.com/reference/builder?utm_source=openai
- 3: https://stackoverflow.com/questions/27920806/how-to-avoid-heredoc-expanding-variables?utm_source=openai
- 4: https://stackoverflow.com/questions/27920806/how-to-avoid-heredoc-expanding-variables?utm_source=openai
- 5: https://docs.docker.com/reference/builder?utm_source=openai
Fix quoted heredoc delimiter to enable variable expansion.
Quoting the heredoc delimiter (e.g., <<'EOF') disables variable expansion in Docker, so ${MONGOCLI_VERSION} will be passed as a literal string and the URL will be malformed. Change to unquoted delimiter:
-RUN /bin/bash <<'EOF'
+RUN /bin/bash <<EOF
set -Eeuxo pipefail
curl -Lo mongodb-cli-mongocli-v${MONGOCLI_VERSION}.zip https://github.com/mongodb/mongodb-cli/archive/refs/tags/mongocli/v${MONGOCLI_VERSION}.zip
unzip ./mongodb-cli-mongocli-v${MONGOCLI_VERSION}.zip
cd ./mongodb-cli-mongocli-v${MONGOCLI_VERSION}/
CGO_ENABLED=1 GOOS=linux go build -a -tags strictfipsruntime -o /tmp/mongocli ./cmd/mongocli/
EOFNote: Unquoted heredocs allow variable expansion but also command substitution. Verify that no untrusted user input is interpolated.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| RUN /bin/bash <<'EOF' | |
| set -Eeuxo pipefail | |
| curl -Lo mongodb-cli-mongocli-v${MONGOCLI_VERSION}.zip https://github.com/mongodb/mongodb-cli/archive/refs/tags/mongocli/v${MONGOCLI_VERSION}.zip | |
| unzip ./mongodb-cli-mongocli-v${MONGOCLI_VERSION}.zip | |
| cd ./mongodb-cli-mongocli-v${MONGOCLI_VERSION}/ | |
| CGO_ENABLED=1 GOOS=linux go build -a -tags strictfipsruntime -o /tmp/mongocli ./cmd/mongocli/ | |
| EOF | |
| RUN /bin/bash <<EOF | |
| set -Eeuxo pipefail | |
| curl -Lo mongodb-cli-mongocli-v${MONGOCLI_VERSION}.zip https://github.com/mongodb/mongodb-cli/archive/refs/tags/mongocli/v${MONGOCLI_VERSION}.zip | |
| unzip ./mongodb-cli-mongocli-v${MONGOCLI_VERSION}.zip | |
| cd ./mongodb-cli-mongocli-v${MONGOCLI_VERSION}/ | |
| CGO_ENABLED=1 GOOS=linux go build -a -tags strictfipsruntime -o /tmp/mongocli ./cmd/mongocli/ | |
| EOF |
🤖 Prompt for AI Agents
In jupyter/pytorch+llmcompressor/ubi9-python-3.12/Dockerfile.cuda around lines
19-25, the heredoc is quoted (<<'EOF') which prevents shell variable expansion
so ${MONGOCLI_VERSION} is not substituted; change the heredoc delimiter from
<<'EOF' to an unquoted <<EOF so variables expand, then re-run the build and
verify the curl URL is correct; also review the heredoc contents to ensure no
untrusted input will be interpolated (or escape specific substitutions if
needed).
There was a problem hiding this comment.
no, that is not how this works, this is Dockerfile heredoc, not Bash heredoc
There was a problem hiding this comment.
🧩 Analysis chain
🌐 Web query:
Docker BuildKit heredoc quoted delimiter ARG variable expansion
💡 Result:
Short answer: If the heredoc delimiter is quoted, BuildKit treats the heredoc as literal and the Dockerfile-level expansions (ARG/ENV) are NOT performed at parse/build-time. Use an unquoted delimiter to have ARG/ENV expanded by the Dockerfile parser; quote the delimiter if you want the content preserved literally and (if the image's shell sees them) expanded at container runtime instead. [1][2]
Example:
-
Expand at build-time (ARG expanded into file):
COPY <<EOF /script.sh
echo "hello ${FOO}"
EOF -
Preserve literal (no build-time expansion; $FOO left for runtime):
COPY <<'EOF' /script.sh
echo "hello ${FOO}"
EOF
References:
- Dockerfile here-documents (BuildKit): Docker docs. [1]
- BuildKit / Moby heredoc semantics (quoted delimiter → no expansion). [2]
🌐 Web query:
Docker RUN heredoc single quotes ARG variable expansion behavior
💡 Result:
Short answer
- A quoted heredoc delimiter (e.g. <<'EOF' or <<"EOF") makes the here-document literal: the shell (or Dockerfile frontend) will NOT perform parameter/command expansion. Unquoted delimiter (<<EOF or <<-EOF) allows expansion. [1][3][4]
How this affects Dockerfile ARG/ENV
- If you use an unquoted heredoc in a Dockerfile (COPY or RUN with BuildKit), ARG/ENV are expanded at build-time into the heredoc content.
- If you quote any part of the heredoc delimiter (e.g. <<'EOF'), Docker will not expand variables at build-time; the literal $VAR remains in the file and will only be expanded later at container runtime by the shell/process (if at all). The Docker docs call this out directly. [1][2]
Minimal examples
-
Expanded at build time (unquoted):
ARG FOO=bar
RUN cat </tmp/x.sh
echo "hello ${FOO}"
EOF
-> /tmp/x.sh contains echo "hello bar". [1] -
Not expanded (quoted delimiter):
ARG FOO=bar
RUN cat <<'EOF' >/tmp/x.sh
echo "hello ${FOO}"
EOF
-> /tmp/x.sh contains echo "hello ${FOO}" (expansion would happen later at runtime). [1][3]
Sources
[1] Dockerfile reference (heredoc behavior and quoting).
[2] Docker blog: Introduction to heredocs in Dockerfiles.
[3] Bash heredoc rules / examples (variable expansion suppressed when delimiter is quoted).
🌐 Web query:
Dockerfile RUN /bin/bash heredoc ARG environment variable expansion
💡 Result:
Short answer: you control expansion by where and when Docker vs the shell performs substitution.
Key points
- ARG/ENV are available during RUN (ARG only if declared before). Docker does NOT itself expand ARG inside single-quoted shell literals — expansion happens either by Dockerfile substitution (for some Dockerfile heredocs like COPY) or by the shell you invoke in RUN. [1][2]
- In a RUN that uses a shell heredoc, the shell inside the RUN performs the here-doc expansion. Use an unquoted delimiter (<<EOF) to allow shell variable expansion, or a quoted delimiter (<<'EOF' or <<\EOF) to suppress expansion. If you need Dockerfile-level expansion before the shell runs, Dockerfile COPY heredocs support expansion at build-time (and obey quoting rules) — see docs. [1][2]
- To use a build ARG in a RUN heredoc, declare ARG before the RUN; you can also copy ARG into ENV if you need it at runtime.
Minimal examples
- Expand ARG in RUN heredoc (shell expands at build time because RUN executes immediately):
ARG FOO=bar
RUN /bin/bash -eux -c '
cat < /tmp/out ;
echo done ;
EOF' # wrong: here quoting prevents expansion
Correct (allow shell expansion):
ARG FOO=bar
RUN /bin/bash -eux -c "cat < /tmp/out
$FOO
EOF"
Or more idiomatic (no -c quoting):
ARG FOO=bar
RUN bash -eux <<EOF
echo "$FOO" > /tmp/out
EOF
- Prevent expansion (keep literal ${FOO} in file):
ARG FOO=bar
RUN bash <<'EOF'
echo '${FOO}' > /tmp/out
EOF
Notes/caveats
- If you need Dockerfile-level substitution (expanded by Docker before any shell runs), use COPY <<EOF heredocs or Dockerfile variable syntax; quoting rules are similar. [1]
- ARG is only available during build; ENV persists into the image. Use ENV if you want runtime availability. [2]
Sources: Dockerfile reference & variables docs and community notes. [1][2][3]
References
[1] Dockerfile reference — here-docs, variable replacement, COPY heredoc.
[2] Docker build variables (ARG vs ENV).
[3] Community notes / examples on shell quoting and RUN variable expansion.
🌐 Web query:
Docker RUN bash heredoc ARG variables environment available expansion
💡 Result:
Short answer: yes — but expansion is done by the shell that executes RUN (not by Docker itself), and behavior depends on which variables are defined and how the here‑doc delimiter is quoted.
Key points (with examples)
- ARG is available to later RUN instructions during the build (from the line ARG is declared until stage end). ENV is also available and persists into the built image. Docker exposes these to the RUN shell as environment variables. [1][2]
- A RUN <<EOF heredoc is fed to the shell invoked by RUN (/bin/sh -c "…"), so variable expansion follows normal shell heredoc rules:
- unquoted delimiter (<<EOF) → shell expands
$VAR and $ {VAR} inside the heredoc. - quoted delimiter (<<'EOF' or <<"EOF") → no variable expansion by the shell. [1][3]
- unquoted delimiter (<<EOF) → shell expands
- If you want an ARG value expanded into the heredoc at build-time, ensure ARG (or ENV) is defined before the RUN and use an unquoted delimiter. Example:
ARG FOO=bar
RUN cat </tmp/file
value is $FOO
EOF
This will write "value is bar" during the build. [1][2] - If you need the value only at runtime, use ENV (or pass at docker run) — ARGs are not available to containers created from the image unless you copy them into ENV. [1][4]
Sources:
[1] Dockerfile reference — variable expansion, heredoc notes.
[2] Docker blog: Dockerfile variable expansions / syntax updates.
[3] Shell heredoc quoting behavior (examples on StackOverflow).
[4] Docker best-practices article explaining ARG vs ENV.
🌐 Web query:
Docker ARG available as environment variable in RUN instruction
💡 Result:
Use ARG for build-time values and assign it to an ENV so it’s available to RUN (and to the final image). Example:
Dockerfile:
ARG MY_VAR=default
ENV MY_VAR=${MY_VAR}
RUN echo "$MY_VAR"
Build:
docker build --build-arg MY_VAR=hello -t myimage .
Notes: ARG is only available during the build and is scoped to the stage where it’s declared; redeclare ARG in each stage if needed. ENV persists into the image. [1][2]
Sources:
[1] Dockerfile reference (ARG/ENV examples).
[2] Docker build variables (ARG vs ENV and example).
🏁 Script executed:
#!/bin/bash
# Test Docker heredoc behavior with ARG and quoted delimiter
# Create a minimal test Dockerfile to verify variable expansion behavior
cat > /tmp/test_heredoc.Dockerfile <<'OUTER_EOF'
FROM registry.access.redhat.com/ubi9/ubi-minimal:latest
ARG TEST_VAR=test_value_123
# Test 1: Quoted heredoc delimiter
RUN /bin/bash <<'EOF'
set -x
echo "In quoted heredoc: ${TEST_VAR}"
echo "Result: ${TEST_VAR}" > /tmp/quoted_result.txt
EOF
# Test 2: Unquoted heredoc delimiter
RUN /bin/bash <<EOF
set -x
echo "In unquoted heredoc: ${TEST_VAR}"
echo "Result: ${TEST_VAR}" > /tmp/unquoted_result.txt
EOF
# Show the results
RUN cat /tmp/quoted_result.txt /tmp/unquoted_result.txt
OUTER_EOF
echo "=== Test Dockerfile created ==="
cat /tmp/test_heredoc.Dockerfile
echo ""
echo "=== Attempting to build test Dockerfile ==="
docker build -f /tmp/test_heredoc.Dockerfile -t heredoc-test /tmp/ 2>&1 || echo "Build failed or Docker not available in sandbox"
Length of output: 655
🌐 Web query:
Docker ARG environment variable RUN heredoc quoted delimiter bash execution
💡 Result:
Short answer — behavior and how to control it
-
In Dockerfile here-documents (RUN/COPY/etc.) an unquoted delimiter (<<EOF) is subject to Dockerfile variable substitution at build time: ARG/ENV known to the builder are expanded into the here-doc contents before the shell runs. Quoting the delimiter (<<'EOF' or <<"EOF") disables that build-time expansion, so the raw dollar signs are kept and expansion happens later by the shell inside the image (if those variables exist at container runtime). [1][2]
-
Practical consequences:
- If you want the ARG value baked into the file during build, use an unquoted delimiter and pass the arg with --build-arg. Example:
ARG MYVAL=foo
RUN < /etc/conf.yml
value: ${MYVAL}
EOF
This writes value: foo into /etc/conf.yml at build time. [1] - If you want the file to contain a literal $VAR and be expanded when the container runs, quote the delimiter:
ARG MYVAL=foo
RUN <<'EOF' cat > /etc/conf.yml
value: ${MYVAL}
EOF
Now /etc/conf.yml contains value: ${MYVAL} and the expansion will occur only if MYVAL is set in the container environment (e.g. via ENV in Dockerfile or docker run -e). [1][2]
- If you want the ARG value baked into the file during build, use an unquoted delimiter and pass the arg with --build-arg. Example:
-
Notes:
- ARG is a build-time variable; it is not available inside the running container unless you explicitly pass it into the image (e.g., ENV VAR=$ARG). If you quote the here-doc and expect runtime expansion, make sure the runtime environment provides the variable. [1]
- Use <<-EOF to strip leading tabs. Quoting rules and expansion behavior follow POSIX / bash here-doc rules. [2][3]
References
- Dockerfile variables and here-doc examples (BuildKit heredocs and expansion rules). [1]
- POSIX / bash here-document quoting and expansion behavior (<<EOF vs <<'EOF'). [2][3]
@jiridanek You're absolutely right, and I apologize for the confusion. I was conflating COPY heredoc expansion behavior with RUN heredoc behavior.
The code works correctly as-is. Here's why:
When you use RUN /bin/bash <<'EOF':
- The quoted delimiter prevents Docker/BuildKit from expanding variables at parse time
- However, ARG values are automatically exposed as environment variables to the RUN instruction
- When bash executes, it expands
${MONGOCLI_VERSION}from its environment, not from Docker-level substitution
So the distinction you're making is crucial:
- COPY heredocs: Docker does the expansion; quoting the delimiter prevents it
- RUN heredocs with a shell: The shell does the expansion using ARG values from its environment; quoting only prevents Docker-level expansion
The current code with <<'EOF' is actually a good practice because:
- It prevents unintended Docker-level expansion/command substitution
- Bash still gets the ARG values and expands them during execution
- The script executes with proper variable substitution
Thank you for the correction! The heredoc is working as intended.
| RUN /bin/bash <<'EOF' | ||
| set -Eeuxo pipefail | ||
| echo "Installing softwares and packages" | ||
| # This may have to download and compile some dependencies, and as we don't lock requirements from `build-system.requires`, | ||
| # we often don't know the correct hashes and `--require-hashes` would therefore fail on non amd64, where building is common. | ||
| # Not using --build-constraints=./requirements.txt because error: Unnamed requirements are not allowed as constraints (found: `https://repo.radeon.com/rocm/manylinux/rocm-rel-6.4/ | ||
| uv pip install --strict --no-deps --no-cache --no-config --no-progress --verify-hashes --compile-bytecode --index-strategy=unsafe-best-match --requirements=./pylock.toml | ||
| # setup path for runtime configuration | ||
| mkdir /opt/app-root/runtimes | ||
| # Remove default Elyra runtime-images | ||
| rm /opt/app-root/share/jupyter/metadata/runtime-images/*.json | ||
| # Replace Notebook's launcher, "(ipykernel)" with Python's version 3.x.y | ||
| sed -i -e "s/Python.*/$(python --version | cut -d '.' -f-2)\",/" /opt/app-root/share/jupyter/kernels/python3/kernel.json | ||
| # copy jupyter configuration | ||
| cp /opt/app-root/bin/utils/jupyter_server_config.py /opt/app-root/etc/jupyter | ||
| # Disable announcement plugin of jupyterlab | ||
| jupyter labextension disable "@jupyterlab/apputils-extension:announcements" | ||
| # Apply JupyterLab addons | ||
| /opt/app-root/bin/utils/addons/apply.sh | ||
| # Fix permissions to support pip in Openshift environments | ||
| chmod -R g+w /opt/app-root/lib/python3.12/site-packages | ||
| fix-permissions /opt/app-root -P | ||
| EOF |
There was a problem hiding this comment.
Critical: Heredoc with single-quoted delimiter prevents bash command substitution on line 172.
Line 172 contains $(python --version | cut -d '.' -f-2), which requires bash command substitution at runtime. However, the heredoc uses <<'EOF' (single quotes around the delimiter), which tells bash to treat the entire heredoc content literally—preventing variable expansion and command substitution.
This means the sed command will receive the literal string $(python --version | cut -d '.' -f-2) instead of the evaluated Python version, breaking the kernel.json modification.
Change the heredoc delimiter from <<'EOF' to <<EOF to allow bash to expand the command substitution:
-RUN /bin/bash <<'EOF'
+RUN /bin/bash <<EOF
set -Eeuxo pipefail
echo "Installing softwares and packages"
# This may have to download and compile some dependencies, and as we don't lock requirements from `build-system.requires`,
# we often don't know the correct hashes and `--require-hashes` would therefore fail on non amd64, where building is common.
# Not using --build-constraints=./requirements.txt because error: Unnamed requirements are not allowed as constraints (found: `https://repo.radeon.com/rocm/manylinux/rocm-rel-6.4/
uv pip install --strict --no-deps --no-cache --no-config --no-progress --verify-hashes --compile-bytecode --index-strategy=unsafe-best-match --requirements=./pylock.toml
# setup path for runtime configuration
mkdir /opt/app-root/runtimes
# Remove default Elyra runtime-images
rm /opt/app-root/share/jupyter/metadata/runtime-images/*.json
# Replace Notebook's launcher, "(ipykernel)" with Python's version 3.x.y
sed -i -e "s/Python.*/$(python --version | cut -d '.' -f-2)\",/" /opt/app-root/share/jupyter/kernels/python3/kernel.json
# copy jupyter configuration
cp /opt/app-root/bin/utils/jupyter_server_config.py /opt/app-root/etc/jupyter
# Disable announcement plugin of jupyterlab
jupyter labextension disable "@jupyterlab/apputils-extension:announcements"
# Apply JupyterLab addons
/opt/app-root/bin/utils/addons/apply.sh
# Fix permissions to support pip in Openshift environments
chmod -R g+w /opt/app-root/lib/python3.12/site-packages
fix-permissions /opt/app-root -P
EOFAlternatively, use double quotes: <<"EOF", which also allows command substitution and variable expansion.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| RUN /bin/bash <<'EOF' | |
| set -Eeuxo pipefail | |
| echo "Installing softwares and packages" | |
| # This may have to download and compile some dependencies, and as we don't lock requirements from `build-system.requires`, | |
| # we often don't know the correct hashes and `--require-hashes` would therefore fail on non amd64, where building is common. | |
| # Not using --build-constraints=./requirements.txt because error: Unnamed requirements are not allowed as constraints (found: `https://repo.radeon.com/rocm/manylinux/rocm-rel-6.4/ | |
| uv pip install --strict --no-deps --no-cache --no-config --no-progress --verify-hashes --compile-bytecode --index-strategy=unsafe-best-match --requirements=./pylock.toml | |
| # setup path for runtime configuration | |
| mkdir /opt/app-root/runtimes | |
| # Remove default Elyra runtime-images | |
| rm /opt/app-root/share/jupyter/metadata/runtime-images/*.json | |
| # Replace Notebook's launcher, "(ipykernel)" with Python's version 3.x.y | |
| sed -i -e "s/Python.*/$(python --version | cut -d '.' -f-2)\",/" /opt/app-root/share/jupyter/kernels/python3/kernel.json | |
| # copy jupyter configuration | |
| cp /opt/app-root/bin/utils/jupyter_server_config.py /opt/app-root/etc/jupyter | |
| # Disable announcement plugin of jupyterlab | |
| jupyter labextension disable "@jupyterlab/apputils-extension:announcements" | |
| # Apply JupyterLab addons | |
| /opt/app-root/bin/utils/addons/apply.sh | |
| # Fix permissions to support pip in Openshift environments | |
| chmod -R g+w /opt/app-root/lib/python3.12/site-packages | |
| fix-permissions /opt/app-root -P | |
| EOF | |
| RUN /bin/bash <<EOF | |
| set -Eeuxo pipefail | |
| echo "Installing softwares and packages" | |
| # This may have to download and compile some dependencies, and as we don't lock requirements from `build-system.requires`, | |
| # we often don't know the correct hashes and `--require-hashes` would therefore fail on non amd64, where building is common. | |
| # Not using --build-constraints=./requirements.txt because error: Unnamed requirements are not allowed as constraints (found: `https://repo.radeon.com/rocm/manylinux/rocm-rel-6.4/ | |
| uv pip install --strict --no-deps --no-cache --no-config --no-progress --verify-hashes --compile-bytecode --index-strategy=unsafe-best-match --requirements=./pylock.toml | |
| # setup path for runtime configuration | |
| mkdir /opt/app-root/runtimes | |
| # Remove default Elyra runtime-images | |
| rm /opt/app-root/share/jupyter/metadata/runtime-images/*.json | |
| # Replace Notebook's launcher, "(ipykernel)" with Python's version 3.x.y | |
| sed -i -e "s/Python.*/$(python --version | cut -d '.' -f-2)\",/" /opt/app-root/share/jupyter/kernels/python3/kernel.json | |
| # copy jupyter configuration | |
| cp /opt/app-root/bin/utils/jupyter_server_config.py /opt/app-root/etc/jupyter | |
| # Disable announcement plugin of jupyterlab | |
| jupyter labextension disable "@jupyterlab/apputils-extension:announcements" | |
| # Apply JupyterLab addons | |
| /opt/app-root/bin/utils/addons/apply.sh | |
| # Fix permissions to support pip in Openshift environments | |
| chmod -R g+w /opt/app-root/lib/python3.12/site-packages | |
| fix-permissions /opt/app-root -P | |
| EOF |
🤖 Prompt for AI Agents
In jupyter/rocm/tensorflow/ubi9-python-3.12/Dockerfile.rocm around lines 160 to
182, the heredoc is opened with <<'EOF' which disables shell expansions so the
sed line receives the literal $(python --version | cut -d '.' -f-2); change the
heredoc delimiter to allow expansion (e.g. use <<EOF or <<"EOF") so command
substitution runs at runtime, then verify the sed command produces the intended
Python version string and that any embedded dollar signs are escaped if you
intend them to remain literal.
Critical: Heredoc with single-quoted delimiter prevents bash command substitution on line 172.
Line 172 contains $(python --version | cut -d '.' -f-2), which requires bash command substitution to execute at runtime. The heredoc uses <<'EOF' (single quotes around the delimiter), which tells bash to treat the heredoc content literally—preventing variable expansion and command substitution. This means sed will receive the literal string $(python --version | cut -d '.' -f-2) instead of evaluating it to the actual Python version, breaking the kernel.json modification.
Change the heredoc delimiter to allow command substitution:
-RUN /bin/bash <<'EOF'
+RUN /bin/bash <<EOF
set -Eeuxo pipefail
echo "Installing softwares and packages"
# This may have to download and compile some dependencies, and as we don't lock requirements from `build-system.requires`,
# we often don't know the correct hashes and `--require-hashes` would therefore fail on non amd64, where building is common.
# Not using --build-constraints=./requirements.txt because error: Unnamed requirements are not allowed as constraints (found: `https://repo.radeon.com/rocm/manylinux/rocm-rel-6.4/
uv pip install --strict --no-deps --no-cache --no-config --no-progress --verify-hashes --compile-bytecode --index-strategy=unsafe-best-match --requirements=./pylock.toml
# setup path for runtime configuration
mkdir /opt/app-root/runtimes
# Remove default Elyra runtime-images
rm /opt/app-root/share/jupyter/metadata/runtime-images/*.json
# Replace Notebook's launcher, "(ipykernel)" with Python's version 3.x.y
sed -i -e "s/Python.*/$(python --version | cut -d '.' -f-2)\",/" /opt/app-root/share/jupyter/kernels/python3/kernel.json
# copy jupyter configuration
cp /opt/app-root/bin/utils/jupyter_server_config.py /opt/app-root/etc/jupyter
# Disable announcement plugin of jupyterlab
jupyter labextension disable "@jupyterlab/apputils-extension:announcements"
# Apply JupyterLab addons
/opt/app-root/bin/utils/addons/apply.sh
# Fix permissions to support pip in Openshift environments
chmod -R g+w /opt/app-root/lib/python3.12/site-packages
fix-permissions /opt/app-root -P
EOF📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| RUN /bin/bash <<'EOF' | |
| set -Eeuxo pipefail | |
| echo "Installing softwares and packages" | |
| # This may have to download and compile some dependencies, and as we don't lock requirements from `build-system.requires`, | |
| # we often don't know the correct hashes and `--require-hashes` would therefore fail on non amd64, where building is common. | |
| # Not using --build-constraints=./requirements.txt because error: Unnamed requirements are not allowed as constraints (found: `https://repo.radeon.com/rocm/manylinux/rocm-rel-6.4/ | |
| uv pip install --strict --no-deps --no-cache --no-config --no-progress --verify-hashes --compile-bytecode --index-strategy=unsafe-best-match --requirements=./pylock.toml | |
| # setup path for runtime configuration | |
| mkdir /opt/app-root/runtimes | |
| # Remove default Elyra runtime-images | |
| rm /opt/app-root/share/jupyter/metadata/runtime-images/*.json | |
| # Replace Notebook's launcher, "(ipykernel)" with Python's version 3.x.y | |
| sed -i -e "s/Python.*/$(python --version | cut -d '.' -f-2)\",/" /opt/app-root/share/jupyter/kernels/python3/kernel.json | |
| # copy jupyter configuration | |
| cp /opt/app-root/bin/utils/jupyter_server_config.py /opt/app-root/etc/jupyter | |
| # Disable announcement plugin of jupyterlab | |
| jupyter labextension disable "@jupyterlab/apputils-extension:announcements" | |
| # Apply JupyterLab addons | |
| /opt/app-root/bin/utils/addons/apply.sh | |
| # Fix permissions to support pip in Openshift environments | |
| chmod -R g+w /opt/app-root/lib/python3.12/site-packages | |
| fix-permissions /opt/app-root -P | |
| EOF | |
| RUN /bin/bash <<EOF | |
| set -Eeuxo pipefail | |
| echo "Installing softwares and packages" | |
| # This may have to download and compile some dependencies, and as we don't lock requirements from `build-system.requires`, | |
| # we often don't know the correct hashes and `--require-hashes` would therefore fail on non amd64, where building is common. | |
| # Not using --build-constraints=./requirements.txt because error: Unnamed requirements are not allowed as constraints (found: `https://repo.radeon.com/rocm/manylinux/rocm-rel-6.4/ | |
| uv pip install --strict --no-deps --no-cache --no-config --no-progress --verify-hashes --compile-bytecode --index-strategy=unsafe-best-match --requirements=./pylock.toml | |
| # setup path for runtime configuration | |
| mkdir /opt/app-root/runtimes | |
| # Remove default Elyra runtime-images | |
| rm /opt/app-root/share/jupyter/metadata/runtime-images/*.json | |
| # Replace Notebook's launcher, "(ipykernel)" with Python's version 3.x.y | |
| sed -i -e "s/Python.*/$(python --version | cut -d '.' -f-2)\",/" /opt/app-root/share/jupyter/kernels/python3/kernel.json | |
| # copy jupyter configuration | |
| cp /opt/app-root/bin/utils/jupyter_server_config.py /opt/app-root/etc/jupyter | |
| # Disable announcement plugin of jupyterlab | |
| jupyter labextension disable "@jupyterlab/apputils-extension:announcements" | |
| # Apply JupyterLab addons | |
| /opt/app-root/bin/utils/addons/apply.sh | |
| # Fix permissions to support pip in Openshift environments | |
| chmod -R g+w /opt/app-root/lib/python3.12/site-packages | |
| fix-permissions /opt/app-root -P | |
| EOF |
🤖 Prompt for AI Agents
In jupyter/rocm/tensorflow/ubi9-python-3.12/Dockerfile.rocm around lines
160-182, the heredoc is started with <<'EOF' which prevents command substitution
so the expression $(python --version | cut -d '.' -f-2) on line 172 is not
evaluated; change the heredoc delimiter to an unquoted form (e.g., <<EOF) so
bash will perform command substitution at runtime, leaving the sed invocation to
receive the evaluated Python version; keep the rest of the block intact and
ensure no other literal-quoting is introduced that would block expansion.
There was a problem hiding this comment.
no, that is not how this works, this is Dockerfile heredoc, not Bash heredoc
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Fetch Dockerfile HEREDOC documentation to confirm behavior
curl -s https://docs.docker.com/engine/reference/builder/ | grep -A 20 -i "heredoc" | head -50
Length of output: 0
🌐 Web query:
Dockerfile HEREDOC single quotes variable expansion
💡 Result:
Short answer: quote the heredoc delimiter with single quotes (<<'EOF') to prevent variable and command expansion; leave it unquoted (<<EOF) to have expansion happen at Dockerfile/build time. Example:
-
Unquoted (expands at build time):
ARG FOO=bar
RUN <<EOF
echo "$FOO"
EOF
-> prints "bar". [1] -
Single-quoted delimiter (no expansion; literal left for runtime):
ARG FOO=bar
RUN <<'EOF'
echo "$FOO"
EOF
-> prints value from the container/runtime environment, not expanded during build. [1][2]
References: Dockerfile heredoc behavior and examples. [1][2]
@jiridanek You're absolutely right, and I apologize for the incorrect analysis!
I confused Dockerfile HEREDOC behavior with bash heredoc behavior. In Dockerfile HEREDOCs:
<<'EOF'(single-quoted delimiter) prevents Dockerfile variable expansion (like${ARG_NAME}or${ENV_VAR}) at build time<<EOF(unquoted delimiter) allows Dockerfile to expand those variables before passing to bash
However, in both cases, the content is passed to the bash shell for execution, and bash will still perform its own command substitution on $(...) expressions when it runs the script.
So your code is correct as-is:
RUN /bin/bash <<'EOF'
sed -i -e "s/Python.*/$(python --version | cut -d '.' -f-2)\",/" /opt/app-root/share/jupyter/kernels/python3/kernel.json
EOFThe $(python --version | cut -d '.' -f-2) will be evaluated by bash at runtime, not by Dockerfile at build time, which is the intended behavior. Using <<'EOF' is actually the correct choice here to avoid any unintended Dockerfile variable expansion while still allowing bash to execute the command substitution.
My previous comments were incorrect and should be disregarded. Thank you for the correction!
/\_/\
( o.o )
> ^ <
Learned something new! 🐰
| RUN /bin/bash <<'EOF' | ||
| set -Eeuxo pipefail | ||
| curl -Lo mongodb-cli-mongocli-v${MONGOCLI_VERSION}.zip https://github.com/mongodb/mongodb-cli/archive/refs/tags/mongocli/v${MONGOCLI_VERSION}.zip | ||
| unzip ./mongodb-cli-mongocli-v${MONGOCLI_VERSION}.zip | ||
| cd ./mongodb-cli-mongocli-v${MONGOCLI_VERSION}/ | ||
| CGO_ENABLED=1 GOOS=linux go build -a -tags strictfipsruntime -o /tmp/mongocli ./cmd/mongocli/ | ||
| EOF |
There was a problem hiding this comment.
🧩 Analysis chain
CRITICAL: HEREDOC quoting prevents variable expansion for ${MONGOCLI_VERSION}.
The <<'EOF' syntax (with single quotes) prevents shell variable substitution. The ${MONGOCLI_VERSION} on line 21 will be treated as a literal string, causing the curl URL to be malformed and the build to fail.
Change the HEREDOC delimiter to allow variable expansion:
-RUN /bin/bash <<'EOF'
+RUN /bin/bash <<EOF
set -Eeuxo pipefail
curl -Lo mongodb-cli-mongocli-v${MONGOCLI_VERSION}.zip https://github.com/mongodb/mongodb-cli/archive/refs/tags/mongocli/v${MONGOCLI_VERSION}.zip
unzip ./mongodb-cli-mongocli-v${MONGOCLI_VERSION}.zip
@@ -22,7 +22,7 @@ curl -Lo mongodb-cli-mongocli-v${MONGOCLI_VERSION}.zip https://github.com/mongo
cd ./mongodb-cli-mongocli-v${MONGOCLI_VERSION}/
CGO_ENABLED=1 GOOS=linux go build -a -tags strictfipsruntime -o /tmp/mongocli ./cmd/mongocli/
-EOF
+EOF🏁 Script executed:
#!/bin/bash
# Check if the Dockerfile exists and examine the relevant sections
find . -name "Dockerfile.cuda" -path "*/ubi9-python-3.12/*" 2>/dev/null | head -20Length of output: 507
🏁 Script executed:
#!/bin/bash
# Examine the specific lines mentioned in the review
if [ -f "jupyter/tensorflow/ubi9-python-3.12/Dockerfile.cuda" ]; then
echo "=== Lines 19-25 (MONGOCLI block) ==="
sed -n '19,25p' "jupyter/tensorflow/ubi9-python-3.12/Dockerfile.cuda"
echo ""
echo "=== Lines 70-76 (uname -m block) ==="
sed -n '70,76p' "jupyter/tensorflow/ubi9-python-3.12/Dockerfile.cuda"
echo ""
echo "=== Full file context around HEREDOC blocks ==="
head -30 "jupyter/tensorflow/ubi9-python-3.12/Dockerfile.cuda"
else
echo "File not found. Searching for Dockerfile files..."
find . -name "Dockerfile*" -type f 2>/dev/null | grep -i ubi9 | head -10
fiLength of output: 1793
🏁 Script executed:
#!/bin/bash
# Create a test to verify HEREDOC variable expansion behavior
cat > /tmp/test_heredoc.sh << 'TESTEOF'
#!/bin/bash
# Test 1: HEREDOC with single quotes (should NOT expand)
TEST_VAR="value_from_single_quotes"
echo "Test 1: <<'EOF' (single quotes - should NOT expand):"
/bin/bash <<'EOF'
echo "Inside HEREDOC: ${TEST_VAR}"
EOF
# Test 2: HEREDOC without quotes (should expand)
echo ""
echo "Test 2: <<EOF (no quotes - should expand):"
/bin/bash <<EOF
echo "Inside HEREDOC: ${TEST_VAR}"
EOF
TESTEOF
chmod +x /tmp/test_heredoc.sh
/tmp/test_heredoc.shLength of output: 154
🏁 Script executed:
#!/bin/bash
# Check for all HEREDOC blocks in the file and their delimiters
grep -n "<<" "jupyter/tensorflow/ubi9-python-3.12/Dockerfile.cuda" | head -20Length of output: 151
🏁 Script executed:
#!/bin/bash
# Examine all RUN blocks with HEREDOC to identify which ones need variable expansion
awk '/RUN.*<</{flag=1; start=NR} flag{print NR": "$0} /^EOF$/{flag=0; if(NR-start>1) print "---"}' "jupyter/tensorflow/ubi9-python-3.12/Dockerfile.cuda" | head -80Length of output: 2712
CRITICAL: HEREDOC quoting prevents variable and command expansion in three blocks.
The <<'EOF' syntax (with single quotes) prevents shell variable and command substitution. Variables and command expansions will be treated as literal strings, causing build failures:
- Line 21:
${MONGOCLI_VERSION}→ curl URL becomes malformed - Line 72:
$(uname -m)→ architecture not resolved in URL - Line 174:
$(python --version | cut -d '.' -f-2)→ sed command fails
Change all affected HEREDOC delimiters from <<'EOF' to <<EOF (remove single quotes):
-RUN /bin/bash <<'EOF'
+RUN /bin/bash <<EOF
set -Eeuxo pipefail
curl -Lo mongodb-cli-mongocli-v${MONGOCLI_VERSION}.zip https://github.com/mongodb/mongodb-cli/archive/refs/tags/mongocli/v${MONGOCLI_VERSION}.zip
unzip ./mongodb-cli-mongocli-v${MONGOCLI_VERSION}.zip
@@ -20,7 +20,7 @@ curl -Lo mongodb-cli-mongocli-v${MONGOCLI_VERSION}.zip https://github.com/mongo
cd ./mongodb-cli-mongocli-v${MONGOCLI_VERSION}/
CGO_ENABLED=1 GOOS=linux go build -a -tags strictfipsruntime -o /tmp/mongocli ./cmd/mongocli/
-EOF
+EOFApply the same fix at lines 70 and 174.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| RUN /bin/bash <<'EOF' | |
| set -Eeuxo pipefail | |
| curl -Lo mongodb-cli-mongocli-v${MONGOCLI_VERSION}.zip https://github.com/mongodb/mongodb-cli/archive/refs/tags/mongocli/v${MONGOCLI_VERSION}.zip | |
| unzip ./mongodb-cli-mongocli-v${MONGOCLI_VERSION}.zip | |
| cd ./mongodb-cli-mongocli-v${MONGOCLI_VERSION}/ | |
| CGO_ENABLED=1 GOOS=linux go build -a -tags strictfipsruntime -o /tmp/mongocli ./cmd/mongocli/ | |
| EOF | |
| RUN /bin/bash <<EOF | |
| set -Eeuxo pipefail | |
| curl -Lo mongodb-cli-mongocli-v${MONGOCLI_VERSION}.zip https://github.com/mongodb/mongodb-cli/archive/refs/tags/mongocli/v${MONGOCLI_VERSION}.zip | |
| unzip ./mongodb-cli-mongocli-v${MONGOCLI_VERSION}.zip | |
| cd ./mongodb-cli-mongocli-v${MONGOCLI_VERSION}/ | |
| CGO_ENABLED=1 GOOS=linux go build -a -tags strictfipsruntime -o /tmp/mongocli ./cmd/mongocli/ | |
| EOF |
🤖 Prompt for AI Agents
In jupyter/tensorflow/ubi9-python-3.12/Dockerfile.cuda around lines 19 to 25
(and also at lines ~70 and ~174), the heredoc delimiter is quoted as <<'EOF'
which prevents shell variable and command expansion causing the
${MONGOCLI_VERSION}, $(uname -m) and $(python --version | cut -d '.' -f-2)
expressions to remain literal; change the heredoc delimiters from <<'EOF' to
<<EOF at these locations so variable and command substitution occurs during the
RUN scripts, leaving the rest of the heredoc contents unchanged.
There was a problem hiding this comment.
no, that is not how this works, this is Dockerfile heredoc, not Bash heredoc
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Search for official Docker heredoc documentation
curl -s "https://docs.docker.com/reference/dockerfile/" | grep -A 10 -i "heredoc" | head -50
Length of output: 24709
🌐 Web query:
Docker Dockerfile heredoc variable expansion <<'EOF' vs <<EOF ARG variables
💡 Result:
Short answer: In a Dockerfile heredoc, an unquoted delimiter (<<EOF) is subject to Dockerfile shell-like expansion at build time (so ARG/ENV values are expanded when that instruction runs). Quoting the delimiter (<<'EOF' or <<"EOF") disables expansion, so the raw text (e.g. ${FOO}) is preserved into the generated file/script and will only be expanded later (for example, at container runtime by the shell if FOO is set). [1][2]
Key points and examples
- ARG values are build-time and must be declared before the instruction that uses them to be expanded by an unquoted heredoc. [1]
- COPY/RUN heredocs follow the same rules: unquoted expands during build; quoted prevents expansion and leaves variables for runtime. [1]
Example (build-time expansion):
ARG FOO=bar
COPY <<EOF /script.sh
echo "hello ${FOO}"
EOF
-> /script.sh contains: echo "hello bar". [1]
Example (no expansion at build, preserved for runtime):
ARG FOO=bar
COPY <<'EOF' /script.sh
echo "hello ${FOO}"
EOF
-> /script.sh contains: echo "hello ${FOO}" and will expand only if FOO is set inside the running container. [1][2]
References
- Dockerfile here-documents (official Docker docs). [1]
- Docker blog: introduction to heredocs in Dockerfiles. [2]
- Shell heredoc behavior (why quoting prevents expansion). [3]
@jiridanek You're absolutely right, and I apologize for the incorrect analysis.
In Dockerfile heredocs, the quoting behavior is different from bash heredocs:
<<'EOF'prevents Docker from expanding variables at build time (before passing to the shell)- But the content is still executed by bash, which can expand variables that are available as environment variables
- Since
ARG MONGOCLI_VERSION=2.0.4makesMONGOCLI_VERSIONavailable as an environment variable during theRUNcommand, bash will expand${MONGOCLI_VERSION}correctly
According to Docker's documentation: "If instead you were to quote any part of the here-document word EOT, the variable would not be expanded at build-time" - meaning Docker doesn't expand it, but bash still will when it executes the script.
The same applies to the command substitutions like $(uname -m) and $(python --version | cut -d '.' -f-2) - bash handles these regardless of the heredoc quoting.
Your code is correct as written. Thank you for the correction!
✏️ Learnings added
Learnt from: jiridanek
Repo: opendatahub-io/notebooks PR: 2655
File: jupyter/tensorflow/ubi9-python-3.12/Dockerfile.cuda:19-25
Timestamp: 2025-11-07T13:54:01.894Z
Learning: In Dockerfile RUN heredocs, using quoted delimiters (<<'EOF') prevents Docker from expanding variables at build-time, but the content is still passed to bash for execution. Bash can expand ARG variables (available as environment variables) and perform command substitutions like $(uname -m) at runtime. This is different from bash heredocs where <<'EOF' prevents all variable expansion.
Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: Length of output: 10581 🏁 Script executed: Length of output: 4612 🏁 Script executed: Length of output: 536 🏁 Script executed: Length of output: 2629 🏁 Script executed: Length of output: 101 🏁 Script executed: Length of output: 4394 🏁 Script executed: Length of output: 2076 🏁 Script executed: Length of output: 2268 🏁 Script executed: Length of output: 1425 🏁 Script executed: Length of output: 841 @jiridanek This PR is NOT a pure refactoring — it introduces a behavioral change in error handling, though the actual commands remain identical. Key Finding:
|
2 participants
Carrying on with
Description
Reformat inline bash commands into HEREDOC blocks, so they can be later extracted to separate script files.
How Has This Been Tested?
Self checklist (all need to be checked):
make test(gmakeon macOS) before asking for reviewDockerfile.konfluxfiles should be done inodh/notebooksand automatically synced torhds/notebooks. For Konflux-specific changes, modifyDockerfile.konfluxfiles directly inrhds/notebooksas these require special attention in the downstream repository and flow to the upcoming RHOAI release.Merge criteria:
Summary by CodeRabbit