Skip to content

Commit ed0d352

Browse files
galal-husseingalal-hussein
authored
K3S Sysbox Blog Post (#478)
* K3S Sysbox Blog Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>
1 parent ed19f7b commit ed0d352

File tree

2 files changed

+152
-1
lines changed

2 files changed

+152
-1
lines changed

‎blog/2025-09-27-k3s-sysbox.md‎

Lines changed: 145 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,145 @@
1+
---
2+
title: Sysbox Runtime With K3s
3+
description: Integrating sysbox runtime with k3s' containerd
4+
authors: husseingalal
5+
tags: ["runc","k3s","sysbox"]
6+
hide_table_of_contents: true
7+
---
8+
9+
The K3s binary bundles all the components needed to run a production-ready, CNCF-conformant Kubernetes cluster including containerd, runc, kubelet, and more. In this post we will discuss how containerd communicates with OCI runtimes and will discuss adding another container runtime (Sysbox) to K3s and how it can be used to run system pods in your environment.
10+
11+
## Containerd and Runc
12+
13+
First we need to talk briefly about how containerd works with runc. Containerd is a long running daemon that is responsible for:
14+
15+
- **Image Management**: Pulls and stores images from registries.
16+
- **Container Management**: Manages the lifecycle of containers (create, start, stop, delete).
17+
- **Snapshot Management**: Uses snapshotters to manage the filesystem layers for containers.
18+
- **Runtime Management**: Delegates the creation of containers to OCI-compatible runtimes like runc.
19+
20+
When you create a pod in Kubernetes, kubelet uses the CRI plugin implemented in containerd to request a **pod sandbox** (`RunPodSandbox`) and then container creation (`CreateContainer`). Containerd then calls a `shim` process that acts as a middleman between `containerd` and the OCI runtime (for example `runc`). This shim process allows the container to keep running even if the containerd daemon crashes or restarts.
21+
22+
The shim generates the OCI runtime bundle (`config.json` and `rootfs` path) and then executes the runc binary. Runc reads the `config.json`, sets up the container’s namespaces and cgroups, and then launches the container process.
23+
24+
Runc is the component that directly interfaces with the Linux kernel — configuring cgroups, namespaces, seccomp, capabilities, and mounts. After runc finishes creating the container, it **exits**, leaving the shim to manage the lifecycle and I/O of the container.
25+
26+
```mermaid
27+
graph TD
28+
Kubelet["Kubelet<br/>(CRI)"]
29+
Containerd["Containerd"]
30+
Shim["Shim"]
31+
Runtime["OCI Runtime<br/>(runc)"]
32+
Container["Container"]
33+
34+
Kubelet -->|"RunPodSandbox()
35+
CreateContainer()"| Containerd
36+
Containerd -->|"Start/Stop Task"| Shim
37+
Shim -->|"Executes Container"| Runtime
38+
Runtime -->|"clone()
39+
cgroupns()
40+
setns()"| Container
41+
```
42+
43+
## Sysbox Runtime
44+
45+
[Sysbox](https://github.com/nestybox/sysbox) is an open-source, next-generation container runtime created by Nestybox. Unlike traditional runtimes (such as runc), Sysbox is designed to let you run "system containers". It primarily leverages **Linux user namespaces** and other features to provide containers that behave more like lightweight virtual machines.
46+
47+
This means you can run workloads like Docker, Systemd, containerd, or even K3s inside your pods — all without requiring privileged mode.
48+
49+
In short, Sysbox bridges the gap between application containers and virtual machines, enabling use cases like running Kubernetes-in-Kubernetes (K8s-in-K8s), CI/CD pipelines that need full OS-like environments, or development sandboxes with VM-level isolation but container speed.
50+
51+
:::info
52+
Currently, Sysbox officially supports **CRI-O** only. CRI-O has native support for Linux user namespaces, which Sysbox relies on. While containerd added user namespace support starting in version v2.0, there was a [bug](https://github.com/nestybox/sysbox/issues/958) in sysbox-runc that prevented it from working properly with Sysbox.
53+
:::
54+
55+
## Sysbox-runc Containerd Integration
56+
57+
After investigating this issue, I was able to locate the root cause, as explained in this [PR](https://github.com/nestybox/sysbox-runc/pull/106). Containerd was failing to run a specific subcommand for sysbox-runc called `features`, which led to the following error:
58+
59+
```
60+
level=debug msg="failed to introspect features of runtime \"sysbox-runc\"" error="failed to unmarshal Features (*anypb.Any): type with url : not found"
61+
```
62+
63+
Because of this, containerd instructed sysbox-runc to run containers **without user namespaces**, causing container creation to fail. The fix for this bug was recently merged in the `sysbox-runc` repo, enabling containerd to work with sysbox-runc.
64+
65+
# Running Sysbox-runc With K3S
66+
67+
In order to run `sysbox-runc` with K3s you need to have a running K3s cluster, and then you can proceed to install the latest version of sysbox, However, since the fix for containerd support hasn't yet been integrated to sysbox main repo only in `sysbox-runc`, we need to build the binaries from source to get the latest updates.
68+
69+
1. Install docker in your system.
70+
71+
2. Clone the repo and prepare the code
72+
73+
```
74+
git clone --recursive https://github.com/nestybox/sysbox.git
75+
76+
cd sysbox/sysbox-runc
77+
78+
git pull origin main
79+
80+
cd ..
81+
82+
make IMAGE_BASE_DISTRO=ubuntu IMAGE_BASE_RELEASE=jammy sysbox-static
83+
```
84+
85+
You can then copy the binaries built to `/usr/bin` or if you are building on the same machine that you will run containerd you can just run:
86+
87+
```
88+
make install
89+
```
90+
91+
3. Run sysbox binary
92+
93+
```
94+
sysbox
95+
```
96+
97+
4. Create sysbox runc runtime class
98+
99+
```
100+
apiVersion: node.k8s.io/v1
101+
handler: sysbox-runc
102+
kind: RuntimeClass
103+
metadata:
104+
name: sysbox-runc
105+
```
106+
107+
5. Add sysbox runc to containerd configuration, you can do that by creating `/var/lib/rancher/k3s/agent/etc/containerd/config.toml.tmpl`:
108+
109+
```
110+
[plugins.'io.containerd.cri.v1.runtime'.containerd.runtimes.sysbox-runc]
111+
runtime_type = "io.containerd.runc.v2"
112+
113+
[plugins.'io.containerd.cri.v1.runtime'.containerd.runtimes.sysbox-runc.options]
114+
SystemdCgroup = false
115+
BinaryName="/usr/bin/sysbox-runc"
116+
```
117+
118+
6. Finally you can create pod running with the runtime class for sysbox-runc and `hostUsers: false`:
119+
120+
```
121+
apiVersion: v1
122+
kind: Pod
123+
metadata:
124+
name: ubuntu
125+
spec:
126+
runtimeClassName: sysbox-runc
127+
hostUsers: false
128+
containers:
129+
- name: ubuntu2204
130+
image: ubuntu:22.04
131+
command: ["sleep", "40000000000"]
132+
restartPolicy: Never
133+
```
134+
135+
## Conclusion
136+
137+
Sysbox brings a powerful capability to Kubernetes: the ability to run system-level workloads inside containers with strong isolation, without requiring privileged mode. When combined with K3s, this opens the door to new use cases such as:
138+
139+
- Running Kubernetes-in-Kubernetes clusters for virtual clusters ([k3k](https://github.com/rancher/k3k)).
140+
- Creating secure developer sandboxes that behave like lightweight VMs.
141+
- Running system daemons or nested container engines inside pods.
142+
143+
While Sysbox is officially supported with CRI-O today, the recent fixes in `sysbox-runc` allow it to run on containerd as well — making it possible to integrate with K3s. The integration is still evolving, but it shows how the container ecosystem is moving beyond traditional app containers toward more flexible "system containers."
144+
145+
If you’re experimenting with K3s and want to explore system workloads inside pods, Sysbox provides a compelling way to do so while maintaining Kubernetes-native workflows

‎blog/authors.yml‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,4 +9,10 @@ manuelbuil:
99
title: K3s maintainer
1010
image_url: https://github.com/manuelbuil.png
1111
url: https://github.com/manuelbuil
12-
name: Manuel Buil
12+
name: Manuel Buil
13+
14+
husseingalal:
15+
title: K3s maintainer
16+
image_url: https://github.com/galal-hussein.png
17+
url: https://github.com/galal-hussein
18+
name: Hussein Galal

0 commit comments

Comments
 (0)