Deprecate the Docker container runtime and merge with containerd

[afbjorklund]

The "docker" container runtime does not have full support for new Kubernetes features.

The dockershim successor since v1.24, cri-dockerd, is not fully maintained and supported.

But the "containerd" container runtime does not have support for the Docker REST API.

Instead there is a small middleware, nerdctld, that is translating API calls to the nerdctl CLI.

---

The best would be if the supported container runtime could run the supported daemon.

Then you wouldn't need to have cri-dockerd and nerdctld, but only containerd and docker.

The dockerd (daemon) would only be started on-demand (socket activated), not by default.

When using minikube image, there is no need for docker - it only needs containerd/buildkit.

---

The docker snapshotter is replaced with containerd, and docker build is replaced with buildkit.

This change has already happened in Docker Desktop, but it is still optional in Docker Engine.

https://docs.docker.com/engine/storage/containerd/ (overlay2 -> io.containerd.snapshotter.v1)

The "docker buildx" plugin uses BuildKit, and it has replaced the legacy "docker build" builder.

---

By configuring Docker to use the "k8s.io" namespace instead of "moby", it sees the same images.

This is similar to how Podman sees the same images as cri-o, since it only has a single namespace.

This way, the feature of minikube docker-env building images for Kubernetes is still preserved...

{
  "features": {
    "containerd-snapshotter": true
  },
  "containerd-namespace": "k8s.io"
}

NOTE: This changes the storage directory used, from /var/lib/docker to /var/lib/containerd

[medyagh]

that sounds like a great plan ! and we should have that in our road map

[afbjorklund]

The suggestion is to get rid of the docker runtime eventually (alias to containerd?), since it is used for preloads and so on.

Note also that this issue doesn't really affect the docker driver, it is about the container runtime/engine inside the cluster...

Currently minikube uses the containerd version from the docker installation, with this change it would be the opposite.

The containerd runtime can move to a newer version (like: 2.x), and then dockerd can just reuse the runc and containerd.

BEFORE

CLI	Daemon	API	Runtime	CRI
docker	dockerd	docker.sock	cri-dockerd	cri-dockerd.sock
nerdctl	nerdctld	nerdctl.sock	cri-containerd (plugin)	containerd.sock
podman	podman.socket	podman.sock	cri-o	crio.sock

AFTER

CLI	Daemon	API	Runtime	CRI
docker	docker.socket	docker.sock	cri-containerd (plugin)	containerd.sock
podman	podman.socket	podman.sock	cri-o	crio.sock

Note: you can still use nerdctl (and ctr) locally, it is just the middleware that is removed...

Just like it is possible to use podman locally, without going through the podman.sock remote.

• https://kubernetes.io/blog/2017/11/containerd-container-runtime-options-kubernetes/#cri-containerd

• https://kubernetes.io/blog/2018/05/24/kubernetes-containerd-integration-goes-ga/#architecture-improvements

[afbjorklund]

https://www.datadoghq.com/container-report/#9

• • https://kubernetes.io/blog/2022/02/17/dockershim-faq/#are-there-examples-of-folks-using-other-runtimes-in-production-today

[nirs]

@afbjorklund Can you explain this proposal in more details?

• What will be the difference between the existing containerd runtime and the new "merged" docker runtime?
• Why do we need a docker runtime if docker is not used by kubernetes?

[afbjorklund]

• What will be the difference between the existing containerd runtime and the new "merged" docker runtime?

The main difference is that there will be a dockerd, so that minikube docker-env works for accessing the k8s.io images.
That is, you can now build and load images using docker too... The containers, however, live in a different namespace

Compared to the current nerdctld middleware, it has full support for buildkit grpc so no need for DOCKER_BUILDKIT=0.
And there is full support for all the docker commands (if you need them for something), and just not load and build

• Why do we need a docker runtime if docker is not used by kubernetes?

Many users of minikube are used to accessing their cluster runtime directly, with the minikube docker-env command.
For some reason, they have been unable to adopt to only using containerd and give up on the usual docker commands.

Maybe they need docker for something, and don't want to run two VMs. Or maybe "KIC" is not enough for their needs.
So they want to continue using docker, but still have a supported container runtime for Kubernetes (and docker is not)

[afbjorklund]

But you don't need a minikube docker-env (Docker API access), if minikube image is enough for your needs...
You can also still use minikube ssh to log in to the nodes, and run the runtime there directly (without daemon)

There are some quirks left to sort out, with the containerd snapshotter (like each image being duplicated 2-3 times)
And the proposal is to drop the tcp:// access and only go for ssh://, even if that is not strictly necessary to do.

---

There are extra entries, with "sha256" and a different ID.

docker@minikube:~$ docker images
REPOSITORY                                TAG                                                                IMAGE ID       CREATED        SIZE
kindest/kindnetd                          v20250512-df8de77b                                                 07a4b3fe0077   55 years ago   2.37kB
sha256                                    409467f978b4a30fe717012736557d637f66371452c3b279c02b943b367a141c   07a4b3fe0077   55 years ago   2.37kB
registry.k8s.io/kube-proxy                v1.34.0                                                            364da8a25c74   55 years ago   1.38kB
sha256                                    df0860106674df871eebbd01fede90c764bf472f5b97eca7e945761292e9b0ce   364da8a25c74   55 years ago   1.38kB
gcr.io/k8s-minikube/storage-provisioner   v5                                                                 18eb69d1418e   55 years ago   1.32kB
sha256                                    6e38f40d628db3002f5617342c8872c935de530d867d0f709a2fbda1a302a562   18eb69d1418e   55 years ago   1.32kB
sha256                                    cd073f4c5f6a8e9dc6f3125ba00cf60819cae95c1ec84a1f146ee4a9cf9e803f   278fb9dbcca9   55 years ago   1.39kB
registry.k8s.io/pause                     3.10.1                                                             278fb9dbcca9   55 years ago   1.39kB
registry.k8s.io/kube-controller-manager   v1.34.0                                                            f8ba6c082136   55 years ago   5.99kB
sha256                                    a0af72f2ec6d628152b015a46d4074df8f77d5b686978987c70f48b8c7660634   f8ba6c082136   55 years ago   5.99kB
sha256                                    90550c43ad2bcfd11fcd5fd27d2eac5a7ca823be1308884b33dd816ec169be90   fe86fe2f6402   55 years ago   6.14kB
registry.k8s.io/kube-apiserver            v1.34.0                                                            fe86fe2f6402   55 years ago   6.14kB
registry.k8s.io/kube-scheduler            v1.34.0                                                            8fbe6d18415c   55 years ago   5.99kB
sha256                                    46169d968e9203e8b10debaf898210fe11c94b5864c351ea0f6fcf621f659bdc   8fbe6d18415c   55 years ago   5.99kB
registry.k8s.io/coredns/coredns           v1.12.1                                                            e8c262566636   55 years ago   5.59kB
sha256                                    52546a367cc9e0d924aa3b190596a9167fa6e53245023b5b5baf0f07e5443969   e8c262566636   55 years ago   5.59kB
registry.k8s.io/etcd                      3.6.4-0                                                            e36c08168342   55 years ago   5.55kB
sha256                                    5f1f5298c888daa46c4409ff4cefe5ca9d16e479419f94cdb5f5d5563dac0115   e36c08168342   55 years ago   5.55kB
docker@minikube:~$ sudo crictl images --digests              
IMAGE                                     TAG                  DIGEST              IMAGE ID            SIZE
docker.io/kindest/kindnetd                v20250512-df8de77b   07a4b3fe0077a       409467f978b4a       44.4MB
gcr.io/k8s-minikube/storage-provisioner   v5                   18eb69d1418e8       6e38f40d628db       9.06MB
registry.k8s.io/coredns/coredns           v1.12.1              e8c262566636e       52546a367cc9e       22.4MB
registry.k8s.io/etcd                      3.6.4-0              e36c081683425       5f1f5298c888d       74.3MB
registry.k8s.io/kube-apiserver            v1.34.0              fe86fe2f64021       90550c43ad2bc       27.1MB
registry.k8s.io/kube-controller-manager   v1.34.0              f8ba6c082136e       a0af72f2ec6d6       22.8MB
registry.k8s.io/kube-proxy                v1.34.0              364da8a25c742       df0860106674d       26MB
registry.k8s.io/kube-scheduler            v1.34.0              8fbe6d18415c8       46169d968e920       17.4MB
registry.k8s.io/pause                     3.10.1               278fb9dbcca95       cd073f4c5f6a8       320kB

[afbjorklund]

One big change here is that it is only the images that are shared, moving from /var/lib/docker to /var/lib/containerd

The containers are not shared, the way they were before. So it is not supported to use docker to manipulate pods.
It is possible to use nerdctl (or ctr) for containerd, and it is possible the debugging tool crictl from Kubernetes.

But any containers that are started with docker are not visible to the cluster, and vice-versa. Only the images are...
Normally there are no such containers started, only temporary ones and the ones started during building of images.

This is the same way that it works with podman and cri-o, where only images are shared - but not the containers.

[medyagh]

I dont think we need to Depricate the Docker Container Runtime per say ! we could just change the default container runtime.
yes the cir-dockerd might not keep up with the latest features but that doesnt mean it is not enough for some users or might even be the prefered runtime for some users for specific use cases such as local development.

[afbjorklund]

You could change to the containerd snapshotter either way, just like the builder (build) was changed to buildkit (buildx).

1. https://www.docker.com/blog/extending-docker-integration-with-containerd/

2. https://www.docker.com/blog/faster-builds-in-compose-thanks-to-buildkit-support/

Even if you do decide to keep dockershim around, and hope that Mirantis gets cri-dockerd back on track for cgroups v2 etc.

Then you would keep the original "moby" namespace:

{
  "features": {
    "containerd-snapshotter": true
  }
}