From ae15bc64ebc9f099eeb432e860ab84321edd9403 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jiri=20Dan=C4=9Bk?= Date: Tue, 4 Nov 2025 16:35:19 +0100 Subject: [PATCH 1/3] NO-JIRA: chore(docs/adr): document decision to implement Dockerfile templating to reduce duplication --- ...erfile-templating-to-remove-duplication.md | 82 +++++++++++++++++++ 1 file changed, 82 insertions(+) create mode 100644 docs/architecture/decisions/0005-implement-dockerfile-templating-to-remove-duplication.md diff --git a/docs/architecture/decisions/0005-implement-dockerfile-templating-to-remove-duplication.md b/docs/architecture/decisions/0005-implement-dockerfile-templating-to-remove-duplication.md new file mode 100644 index 000000000..3805162c0 --- /dev/null +++ b/docs/architecture/decisions/0005-implement-dockerfile-templating-to-remove-duplication.md @@ -0,0 +1,82 @@ +# 5. Implement Dockerfile templating to remove duplication + +Date: 2025-11-04 + +## Status + +Accepted + +## Context +The project contains multiple Dockerfiles/Containerfiles with significant duplication across different variants (CPU/GPU, different Python versions, different base distributions). +Maintaining these duplicated files is error-prone, as changes need to be manually propagated across multiple files, increasing the risk of inconsistencies, merge conflicts, and bugs. + +We agreed to implement a templating solution to reduce duplication. + +We are aware that introducing abstractions may make the project harder to understand for newcomers, and inappropriate abstractions would make it more difficult to make changes that used to be easy. +Therefore, we're committed to maintaining clarity and debuggability of the build process and to keeping the abstraction maleable so that it can be adjusted to future needs as the project evolves. + +## Decision +We are already using the standard `${VARIABLE}` substitution via `podman build --build-arg-file`. + +Implement Dockerfile/Containerfile templating using a phased approach: + +### Phase 1: Containerfile.in with Podman/CPP + +Begin using Podman's native `Containerfile.in` preprocessing feature (available since Podman 2.2.1). + +Containerfiles with `.in` suffix are automatically preprocessed via CPP(1), enabling: +- **C preprocessor directives**: `#include`, `#define`, `#ifdef`, `#if`, `#else`, `#endif`, etc. +- **File decomposition**: Break Containerfiles into reusable parts via `#include` directive +- **Manual preprocessing**: Files can be preprocessed outside Podman via `cpp -E Containerfile.in` +- **Already in use**: We currently use this feature in the project + +Documentation: https://docs.podman.io/en/v2.2.1/markdown/podman-build.1.html + +Note: Historical issues with comment handling (https://github.com/containers/buildah/issues/3229) were resolved in https://github.com/containers/buildah/pull/3241 + +### Phase 2: Python-based Templating +When Containerfile.in proves insufficient for complex templating needs, adopt a Python-based solution: + +Options evaluated: +1. **Python f-strings**: Simple, native, no dependencies, good for basic interpolation +2. **stencils** (https://github.com/devcoons/stencils): Lightweight alternative, less mature +3. **Jinja2**: Industry-standard, powerful control structures (loops, conditionals), extensive ecosystem + +Recommendation: Jinja2 for Phase 2 due to widespread adoption, maturity, and powerful templating capabilities. + +### Generated Code Management Decision +Must decide whether to: +- **Option A: Commit generated Containerfiles to Git** + - Pros: Reproducible builds, easier debugging, clear history of what was built + - Cons: Potential for drift between templates and committed files, larger repo + +- **Option B: Generate on-the-fly during build** + - Pros: Single source of truth, impossible for drift, cleaner repo + - Cons: Requires templating tooling in CI/CD, harder to debug build failures + +Initial recommendation: Commit generated files (Option A) for traceability and easier debugging, with CI checks to ensure templates and generated files stay in sync. + +## Consequences + +### Positive +- Reduced duplication across Containerfiles +- Easier maintenance: changes to common patterns apply across all variants +- Lower risk of inconsistencies between variants +- Improved consistency in build configuration +- Clearer separation between variant-specific and common configuration + +### Negative/Risks +- Additional complexity in the build process +- Learning curve for contributors unfamiliar with the templating system +- Risk of template bugs affecting multiple Containerfiles simultaneously +- Need to maintain tooling/scripts for template generation +- If committing generated files: need CI validation to catch drift +- If generating on-the-fly: build failures harder to debug without seeing the final Containerfile + +### Mitigations +- Start with the simplest solution (Containerfile.in) and only move to more complex templating as needed +- Document templating approach clearly in the repository +- Implement CI checks to validate generated Containerfiles +- Provide clear error messages in templating scripts +- Keep templates as simple and readable as possible +- Consider a hybrid approach: commit generated files but validate against templates in CI From e3425bfc54fd8e0623867f2cd1de75f9e7fc14cd Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jiri=20Dan=C4=9Bk?= Date: Tue, 4 Nov 2025 16:49:25 +0100 Subject: [PATCH 2/3] expand ADR to include references to prior deduplication efforts and relevant work --- ...t-dockerfile-templating-to-remove-duplication.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/docs/architecture/decisions/0005-implement-dockerfile-templating-to-remove-duplication.md b/docs/architecture/decisions/0005-implement-dockerfile-templating-to-remove-duplication.md index 3805162c0..569f68b76 100644 --- a/docs/architecture/decisions/0005-implement-dockerfile-templating-to-remove-duplication.md +++ b/docs/architecture/decisions/0005-implement-dockerfile-templating-to-remove-duplication.md @@ -80,3 +80,16 @@ Initial recommendation: Commit generated files (Option A) for traceability and e - Provide clear error messages in templating scripts - Keep templates as simple and readable as possible - Consider a hybrid approach: commit generated files but validate against templates in CI + +### Previous work + +- +- [RHOAIENG-16969 Remove specific instances of code duplication in odh/notebooks](https://issues.redhat.com/browse/RHOAIENG-16969) +- [RHOAIENG-19047 Deduplicate files that get included in workbench images](https://issues.redhat.com/browse/RHOAIENG-19047) +- [RHOAIENG-19046 Remove reliance on "chained builds" in notebooks repo](https://issues.redhat.com/browse/RHOAIENG-19046) +- [fix(makefile): standardized image targets #1015](https://github.com/opendatahub-io/notebooks/pull/1015) + - [RHOAIENG-16587: fix(test): ensure papermill tests run successfully for all supported notebooks #834](https://github.com/opendatahub-io/notebooks/pull/834) + - +- [📦 Consolidate duplicate bootstrapper implementations across Python 3.12 runtime environments #1349](https://github.com/opendatahub-io/notebooks/issues/1349) +- [improve and simplify docker multistage build in jupyter/datascience/ubi9-python-3.12/Dockerfile.cpu #2467](https://github.com/opendatahub-io/notebooks/issues/2467) +- [Refactor notebooks-release workflow to eliminate code duplication using reusable workflows #1185](https://github.com/opendatahub-io/notebooks/issues/1185) From 796ad353071ca650ddbd82af07d1294f3c20129e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jiri=20Dan=C4=9Bk?= Date: Tue, 4 Nov 2025 16:54:11 +0100 Subject: [PATCH 3/3] Update docs/architecture/decisions/0005-implement-dockerfile-templating-to-remove-duplication.md Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> --- ...005-implement-dockerfile-templating-to-remove-duplication.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/architecture/decisions/0005-implement-dockerfile-templating-to-remove-duplication.md b/docs/architecture/decisions/0005-implement-dockerfile-templating-to-remove-duplication.md index 569f68b76..7622f7a30 100644 --- a/docs/architecture/decisions/0005-implement-dockerfile-templating-to-remove-duplication.md +++ b/docs/architecture/decisions/0005-implement-dockerfile-templating-to-remove-duplication.md @@ -13,7 +13,7 @@ Maintaining these duplicated files is error-prone, as changes need to be manuall We agreed to implement a templating solution to reduce duplication. We are aware that introducing abstractions may make the project harder to understand for newcomers, and inappropriate abstractions would make it more difficult to make changes that used to be easy. -Therefore, we're committed to maintaining clarity and debuggability of the build process and to keeping the abstraction maleable so that it can be adjusted to future needs as the project evolves. +Therefore, we're committed to maintaining clarity and debuggability of the build process and to keeping the abstraction malleable so that it can be adjusted to future needs as the project evolves. ## Decision We are already using the standard `${VARIABLE}` substitution via `podman build --build-arg-file`.