Addons: helm based addons #21847

volatilemolotov · 2025-11-04T22:11:40Z

This PR adds support for addons based on helm.

Adds the following:

helmChart struct to addon. If this is defined code will go to helmCommand . If set to nil regular kubectl addon install happens
helmInstall is triggered which installs the helm binary based on arch. This was done to not add bloat to the binary/iso
uses values and value files to set --set and --values respectively. This way one can ship a value file to configure via embed.FS

Example impelentation:

In addons list in pkg/minikube/assets/addons.go

		"testaddon": NewAddon([]*BinAsset{
   	MustBinAsset(addons.TestAddonAssets, "test-addon/test-addon.yaml", vmpath.GuestAddonsDir, "test-addon-values.yaml", "0640"),
   	}, false, "testaddon", "3rd party (akvelon.com)", "volatilemolotov", "", nil, nil,
   	&HelmChart{
   		Name:      "testaddon",
   		Repo:      "oci://linktooci/chart",
   		Namespace: "testaddon",
   		Values:    []string{"service.type=NodePort"},
   		ValueFiles: []string{"/etc/kubernetes/addons/test-addon-values.yaml",},
   	},

Provides foundation for #21257

k8s-ci-robot · 2025-11-04T22:11:50Z

Hi @volatilemolotov. Thanks for your PR.

I'm waiting for a github.com member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

minikube-bot · 2025-11-04T22:12:03Z

Can one of the admins verify this patch?

medyagh · 2025-11-05T20:05:52Z

pkg/addons/helm.go

+	"k8s.io/minikube/pkg/minikube/vmpath"
+)
+
+func helmCommand(ctx context.Context, chart *assets.HelmChart, enable bool) *exec.Cmd {


add a comment explain in short what this func does?

Yeah, short description added

how about make this a Generic func, and have two other helpers
called
insatallHelmChart
uninstallHelmChart

so we dont have to pass the enable to it

Added installHelmChart and uninstallHelmChart along with helmUninstallOrInstall

pkg/addons/helm.go

medyagh · 2025-11-05T20:07:24Z

pkg/addons/helm.go

+		default:
+			return fmt.Errorf("failure to detect architecture or unsupported architecture: %s", arch)
+	}
+	helmURL := fmt.Sprintf("https://get.helm.sh/helm-v3.19.0-linux-%s.tar.gz", helmArch)


is there a "latest" helm url that we could use ?

Not aware of any available latest target.

changed this to get_helm.sh

medyagh · 2025-11-05T20:08:29Z

pkg/addons/helm.go

+func helmInstall(addon *assets.Addon, runner command.Runner) error {
+	_, err := runner.RunCmd(exec.Command("test", "-f", "/usr/bin/helm"))
+	if err != nil {
+		// If not, install it


how about installing it through their Script ?
https://helm.sh/docs/intro/install#from-script

this will install latest by script and no need for us to parse archs

Did not want to shoehorn a whole script into this but we could do that. It would require a chmod call and similar but doable. Ill see if its works

Actually not possible without changing either ISO or the script. Script will try to put helm into /usr/local/bin which the VM does not have. Issue encountered on macos with krunkit driver. Same happens on linux with kvm driver

Output from minikube ssh and $ ls /usr/

bin lib lib64 libexec sbin share

Ill revisit this one. Probably we can make the dir and copy the binary where it needs to be and satisfy both environments

Decided to go with checking if the folder exists and create it if it does not. Then copy helm binary to the PATH.

medyagh · 2025-11-05T21:35:01Z

pkg/addons/addons.go

+
+			ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
+			defer cancel()
+			cmd := helmCommand(ctx, addon.HelmChart, enable)


would this ever be called with disable?

pkg/addons/addons.go

medyagh · 2025-11-17T20:12:42Z

pkg/addons/helm.go

+	"k8s.io/minikube/pkg/minikube/vmpath"
+)
+
+// runs a helm install based on the contents of chart *assets.HelmChart


clarify in the comment that this runs inside minikube vm/container

medyagh · 2025-11-17T20:15:10Z

pkg/addons/helm.go

+
+		installCmd := fmt.Sprint("curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 && chmod 700 get_helm.sh && ./get_helm.sh")
+		_, err = runner.RunCmd(exec.Command("sudo", "bash", "-c", installCmd))
+		_, err = runner.RunCmd(exec.Command("sudo", "mv", "/usr/local/bin/helm", "/usr/bin/helm"))


add comment explain the reason we do this is /usr/loca/bin/helm is not in PATH on both iso/kicbase

site/content/en/docs/contrib/helm-addons.md

medyagh · 2025-11-18T21:56:59Z

/ok-to-test

medyagh · 2025-11-19T18:24:23Z

/ok-to-test

…inary

…entation

k8s-ci-robot · 2025-11-21T13:33:48Z

@volatilemolotov: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
integration-kvm-crio-linux-x86-64	`fa4d670`	link	true	`/test integration-kvm-crio-linux-x86-64`
integration-kvm-containerd-linux-x86-64	`fa4d670`	link	true	`/test integration-kvm-containerd-linux-x86-64`
integration-none-docker-linux-x86-64	`fa4d670`	link	true	`/test integration-none-docker-linux-x86-64`
integration-docker-crio-linux-x86-64	`fa4d670`	link	true	`/test integration-docker-crio-linux-x86-64`
integration-docker-containerd-linux-x86-64	`fa4d670`	link	true	`/test integration-docker-containerd-linux-x86-64`

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

minikube-pr-bot · 2025-11-21T14:22:23Z

kvm2 driver with docker runtime

┌────────────────┬──────────┬────────────────────────┐
│    COMMAND     │ MINIKUBE │ MINIKUBE  ( PR 21847 ) │
├────────────────┼──────────┼────────────────────────┤
│ minikube start │ 42.4s    │ 41.6s                  │
│ enable ingress │ 15.6s    │ 15.7s                  │
└────────────────┴──────────┴────────────────────────┘

Times for minikube ingress: 15.7s 15.7s 15.7s 15.2s 15.7s
Times for minikube (PR 21847) ingress: 16.2s 15.7s 15.7s 15.2s 15.8s

Times for minikube start: 42.7s 40.1s 43.7s 40.8s 44.7s
Times for minikube (PR 21847) start: 41.6s 43.3s 41.1s 41.0s 40.8s

docker driver with docker runtime

┌────────────────┬──────────┬────────────────────────┐
│    COMMAND     │ MINIKUBE │ MINIKUBE  ( PR 21847 ) │
├────────────────┼──────────┼────────────────────────┤
│ minikube start │ 21.9s    │ 23.3s                  │
│ enable ingress │ 11.4s    │ 11.4s                  │
└────────────────┴──────────┴────────────────────────┘

Times for minikube start: 21.1s 21.9s 22.2s 21.6s 22.5s
Times for minikube (PR 21847) start: 24.7s 22.9s 24.6s 22.6s 21.9s

Times for minikube (PR 21847) ingress: 12.6s 10.6s 12.6s 10.6s 10.6s
Times for minikube ingress: 10.6s 11.6s 11.6s 10.6s 12.6s

docker driver with containerd runtime

┌────────────────┬──────────┬────────────────────────┐
│    COMMAND     │ MINIKUBE │ MINIKUBE  ( PR 21847 ) │
├────────────────┼──────────┼────────────────────────┤
│ minikube start │ 21.0s    │ 20.8s                  │
│ enable ingress │ 20.3s    │ 21.5s                  │
└────────────────┴──────────┴────────────────────────┘

Times for minikube start: 19.9s 21.9s 23.1s 20.8s 19.4s
Times for minikube (PR 21847) start: 21.4s 19.8s 22.3s 20.2s 20.1s

Times for minikube ingress: 20.1s 20.1s 20.1s 21.1s 20.1s
Times for minikube (PR 21847) ingress: 20.1s 20.1s 22.1s 20.1s 25.1s

minikube-pr-bot · 2025-11-21T16:47:46Z

Here are the number of top 10 failed tests in each environments with lowest flake rate.

Environment	Test Name	Flake Rate
KVM_Linux_crio (1 failed)	TestAddons/parallel/CSI(gopogh)	14.00% (chart)
Docker_Windows (1 failed)	TestErrorSpam/setup(gopogh)	Unknown
KVM_Linux (9 failed)	TestISOImage/PersistentMounts//data(gopogh)	Unknown
KVM_Linux (9 failed)	TestISOImage/PersistentMounts//var/lib/docker(gopogh)	Unknown
KVM_Linux (9 failed)	TestISOImage/PersistentMounts//var/lib/cni(gopogh)	Unknown
KVM_Linux (9 failed)	TestISOImage/PersistentMounts//var/lib/kubelet(gopogh)	Unknown
KVM_Linux (9 failed)	TestISOImage/PersistentMounts//var/lib/minikube(gopogh)	Unknown
KVM_Linux (9 failed)	TestISOImage/PersistentMounts//var/lib/toolbox(gopogh)	Unknown
KVM_Linux (9 failed)	TestISOImage/PersistentMounts//var/lib/boot2docker(gopogh)	Unknown
KVM_Linux (9 failed)	TestISOImage/VersionJSON(gopogh)	Unknown
KVM_Linux (9 failed)	TestISOImage/eBPFSupport(gopogh)	Unknown
Docker_Linux_containerd (4 failed)	TestStartStop/group/old-k8s-version/serial/DeployApp(gopogh)	0.00% (chart)
Docker_Linux_containerd (4 failed)	TestStartStop/group/no-preload/serial/DeployApp(gopogh)	0.00% (chart)
Docker_Linux_containerd (4 failed)	TestStartStop/group/embed-certs/serial/DeployApp(gopogh)	0.00% (chart)
Docker_Linux_containerd (4 failed)	TestStartStop/group/default-k8s-diff-port/serial/DeployApp(gopogh)	2.00% (chart)
Docker_Linux_containerd_arm64 (4 failed)	TestStartStop/group/old-k8s-version/serial/DeployApp(gopogh)	0.00% (chart)
Docker_Linux_containerd_arm64 (4 failed)	TestStartStop/group/no-preload/serial/DeployApp(gopogh)	0.00% (chart)
Docker_Linux_containerd_arm64 (4 failed)	TestStartStop/group/embed-certs/serial/DeployApp(gopogh)	0.00% (chart)
Docker_Linux_containerd_arm64 (4 failed)	TestStartStop/group/default-k8s-diff-port/serial/DeployApp(gopogh)	0.00% (chart)

Besides the following environments also have failed tests:

Docker_Linux_crio: 38 failed (gopogh)
Docker_Linux_crio_arm64: 36 failed (gopogh)

To see the flake rates of all tests by environment, click here.

medyagh · 2025-11-21T19:43:47Z

/lgtm

k8s-ci-robot · 2025-11-21T19:43:56Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: medyagh, volatilemolotov

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [medyagh]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Nov 4, 2025

k8s-ci-robot requested review from ComradeProgrammer and nirs November 4, 2025 22:11

k8s-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Nov 4, 2025

k8s-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Nov 4, 2025

medyagh reviewed Nov 5, 2025

View reviewed changes

pkg/addons/helm.go Outdated Show resolved Hide resolved

medyagh reviewed Nov 5, 2025

View reviewed changes

medyagh requested changes Nov 5, 2025

View reviewed changes

medyagh reviewed Nov 5, 2025

View reviewed changes

medyagh reviewed Nov 7, 2025

View reviewed changes

pkg/addons/addons.go Show resolved Hide resolved

medyagh reviewed Nov 17, 2025

View reviewed changes

medyagh requested changes Nov 17, 2025

View reviewed changes

k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Nov 18, 2025

This comment has been minimized.

Sign in to view

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 20, 2025

This comment has been minimized.

Sign in to view

volatilemolotov added 6 commits November 21, 2025 14:32

add support for helm based addons

59ecb15

refactor helm install

1fe32a0

add description to helmcommand and rename helminstall to helminstallb…

3d9e4a2

…inary

refactor helm functions

4779d97

refactor helm functions - fix install and tests

c3caac3

add documentation for the helm based addons

dd0222e

volatilemolotov added 4 commits November 21, 2025 14:32

add clarification comments on some implementation details, link docum…

0ea2f09

…entation

add proper boilerplate

00a2be2

replace link with relref

00cfd2b

run make fmt

fa4d670

volatilemolotov force-pushed the addons/helm_addon branch from fa0bca2 to fa4d670 Compare November 21, 2025 13:32

k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 21, 2025

medyagh approved these changes Nov 21, 2025

View reviewed changes

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 21, 2025

medyagh merged commit 7fcc6ec into kubernetes:master Nov 21, 2025
31 of 51 checks passed

k8s-ci-robot assigned medyagh Nov 21, 2025

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Nov 21, 2025

Addons: helm based addons #21847

Addons: helm based addons #21847

Conversation

volatilemolotov commented Nov 4, 2025

Uh oh!

k8s-ci-robot commented Nov 4, 2025

Uh oh!

minikube-bot commented Nov 4, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

medyagh commented Nov 18, 2025

Uh oh!

medyagh commented Nov 19, 2025

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

k8s-ci-robot commented Nov 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

minikube-pr-bot commented Nov 21, 2025

Uh oh!

minikube-pr-bot commented Nov 21, 2025

Uh oh!

medyagh commented Nov 21, 2025

Uh oh!

Uh oh!

k8s-ci-robot commented Nov 21, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

k8s-ci-robot commented Nov 21, 2025 •

edited

Loading