Stabilize and automate the dev container build

[brcarp]

Summary

This pull request overhauls the project's .devcontainer environment to create a stable, fully automated, and multi-platform development experience. While the initial motivation was to upgrade the base image from Debian Bullseye to Bookworm, the process revealed several underlying issues with the existing setup that required manual intervention and failed completely on arm64 (Apple Silicon) architectures.

These changes ensure that any developer on a modern arm64 or x86_64 machine can clone the repository, run devcontainer up, and have a working, buildable, and testable environment with zero manual configuration.

Key Problems Addressed

1. Brittle Build Process: The original Dockerfile was inefficient, had out-of-date package names, and suffered from rustup permission errors that prevented a clean build.
2. Lack of Architecture Portability: The dev container failed to build and run on arm64 hosts due to a series of exec format error issues. The build and test scripts also had hardcoded defaults and inconsistent logic that broke the arm64 test workflow.
3. Outdated Container Startup Logic: The postCreateCommand script used a legacy pattern to configure multi-architecture support that was the root cause of the startup failures on arm64 machines.

Solutions and Technical Details

This PR implements a holistic solution by modernizing the Dockerfile, making the build scripts intelligent, and refactoring the container startup process.

1. Dockerfile Stabilization

The Dockerfile was completely refactored to be more efficient and robust:

• Upgraded to Bookworm: The base image is now mcr.microsoft.com/devcontainers/rust:2-1-bookworm.
• Consolidated Dependencies: All apt-get packages are installed in a single, correctly ordered layer. This includes adding gcc-x86-64-linux-gnu and libc6-dev-amd64-cross to fully support cross-compilation of C-based crates like ring.
• Resolved rustup Permissions: The Rust toolchain is now installed as root during the image build, and ownership of the /usr/local/rustup and /usr/local/cargo directories is then granted to the vscode user. This provides a clean and reliable toolchain setup.
• pip Compatibility: Added the --break-system-packages flag to pip install to comply with Debian Bookworm's new policies on externally managed environments.

2. Architecture-Aware Build and Test Scripts

The build and test scripts are now portable:

• The bin/build and bin/test scripts now auto-detect the container's architecture using uname -m.
• This ensures that CRYPTEIA_BUILD_TARGET and CRYPTEIA_BUILD_SUFFIX are set correctly, whether the container is running on an arm64 or x86_64 host.
• This fixed a critical bug where the .so file was built with one name but the test suite looked for another, causing LD_PRELOAD failures.

3. Modernizing the QEMU and binfmt Setup

This was the most critical fix for arm64 stability. The original setup used an outdated pattern that is not reliable in modern container environments.

• The "Old Pattern" (in postCreate): The postCreate script previously ran docker run --privileged multiarch/qemu-user-static. This was a common workaround to register QEMU's handlers with the host kernel's binfmt_misc service. The logic was that a Dockerfile build is sandboxed and cannot interact with the kernel, so a privileged, running container had to do it after startup. The fatal flaw in this pattern is that this secondary container did not inherit the parent's architecture settings, causing it to pull an amd64 image on an arm64 host and fail.

• The "New Pattern" (in Dockerfile): We now handle this entirely within the Dockerfile. By installing the qemu-user-static and binfmt-support packages and running update-binfmts, we prepare the image with all the necessary tools. Modern container runtimes like Docker Desktop and OrbStack are smart enough to use these tools to correctly configure the host kernel when the container starts. This approach is more reliable, self-contained, and completely avoids the architecture mismatch that was causing the startup failure.

As a result, we were able to disable the conflicting QEMU setup in the docker-in-docker feature ("install-qemu": false) and remove the legacy postCreate command, leading to a much cleaner and more robust startup sequence.

[jeremiahlukus]

Once you get tests passing, I’ll look into it this. It’s going to take a bit to test since you are updating rust I want to make sure it doesn’t break anything.

[jeremiahlukus]

This also seems like a lot of code for not too much gain. If you want to update rust then have that as a PR. If you want to add Amazon Linux 2023 have that as a new PR. Seems like you just let AI go wild and told it to update all the things. Separating these features out makes it easier to test and also review.

[jeremiahlukus]

Closing for now, I appreciate the effort but it’s doing a bit too much at once. I don’t want to get kicked from the free GitHub actions tier running tests this frequently.

Feel free to open smaller, targeted PRs. Again thank you for the effort.