Merge pull request #5525 from adrianmoisey/5495-deprecate-ipvs-mode-in-kube-proxy

KEP-5495: Deprecate IPVS mode in kube-proxy

Author: k8s-ci-robot

keps/prod-readiness/sig-network/5495.yaml

@@ -0,0 +1,3 @@
+kep-number: 5495
+deprecated:
+  approver: "@soltysh"

keps/sig-network/5495-deprecate-ipvs-mode-in-kube-proxy/README.md

@@ -0,0 +1,334 @@
+<!--
+**Note:** When your KEP is complete, all of these comment blocks should be removed.
+
+To get started with this template:
+
+- [X] **Pick a hosting SIG.**
+  Make sure that the problem space is something the SIG is interested in taking
+  up. KEPs should not be checked in without a sponsoring SIG.
+- [X] **Create an issue in kubernetes/enhancements**
+  When filing an enhancement tracking issue, please make sure to complete all
+  fields in that template. One of the fields asks for a link to the KEP. You
+  can leave that blank until this KEP is filed, and then go back to the
+  enhancement and add the link.
+- [ ] **Make a copy of this template directory.**
+  Copy this template into the owning SIG's directory and name it
+  `NNNN-short-descriptive-title`, where `NNNN` is the issue number (with no
+  leading-zero padding) assigned to your enhancement above.
+- [ ] **Fill out as much of the kep.yaml file as you can.**
+  At minimum, you should fill in the "Title", "Authors", "Owning-sig",
+  "Status", and date-related fields.
+- [ ] **Fill out this file as best you can.**
+  At minimum, you should fill in the "Summary" and "Motivation" sections.
+  These should be easy if you've preflighted the idea of the KEP with the
+  appropriate SIG(s).
+- [ ] **Create a PR for this KEP.**
+  Assign it to people in the SIG who are sponsoring this process.
+- [ ] **Merge early and iterate.**
+  Avoid getting hung up on specific details and instead aim to get the goals of
+  the KEP clarified and merged quickly. The best way to do this is to just
+  start with the high-level sections and fill out details incrementally in
+  subsequent PRs.
+
+Just because a KEP is merged does not mean it is complete or approved. Any KEP
+marked as `provisional` is a working document and subject to change. You can
+denote sections that are under active debate as follows:
+
+```
+<<[UNRESOLVED optional short context or usernames ]>>
+Stuff that is being argued.
+<<[/UNRESOLVED]>>
+```
+
+When editing KEPS, aim for tightly-scoped, single-topic PRs to keep discussions
+focused. If you disagree with what is already in a document, open a new PR
+with suggested changes.
+
+One KEP corresponds to one "feature" or "enhancement" for its whole lifecycle.
+You do not need a new KEP to move from beta to GA, for example. If
+new details emerge that belong in the KEP, edit the KEP. Once a feature has become
+"implemented", major changes should get new KEPs.
+
+The canonical place for the latest set of instructions (and the likely source
+of this file) is [here](/keps/NNNN-kep-template/README.md).
+
+**Note:** Any PRs to move a KEP to `implementable`, or significant changes once
+it is marked `implementable`, must be approved by each of the KEP approvers.
+If none of those approvers are still appropriate, then changes to that list
+should be approved by the remaining approvers and/or the owning SIG (or
+SIG Architecture for cross-cutting KEPs).
+-->
+# KEP-5495: Deprecate IPVS mode in kube-proxy
+
+<!-- toc -->
+- [Release Signoff Checklist](#release-signoff-checklist)
+- [Summary](#summary)
+- [Motivation](#motivation)
+  - [Goals](#goals)
+  - [Non-Goals](#non-goals)
+- [Proposal](#proposal)
+  - [Notes/Constraints/Caveats (Optional)](#notesconstraintscaveats-optional)
+  - [Risks and Mitigations](#risks-and-mitigations)
+- [Design Details](#design-details)
+  - [Test Plan](#test-plan)
+      - [Prerequisite testing updates](#prerequisite-testing-updates)
+      - [Unit tests](#unit-tests)
+      - [Integration tests](#integration-tests)
+      - [e2e tests](#e2e-tests)
+  - [Graduation Criteria](#graduation-criteria)
+    - [Deprecation](#deprecation)
+  - [Upgrade / Downgrade Strategy](#upgrade--downgrade-strategy)
+  - [Version Skew Strategy](#version-skew-strategy)
+- [Production Readiness Review Questionnaire](#production-readiness-review-questionnaire)
+  - [Feature Enablement and Rollback](#feature-enablement-and-rollback)
+  - [Rollout, Upgrade and Rollback Planning](#rollout-upgrade-and-rollback-planning)
+  - [Monitoring Requirements](#monitoring-requirements)
+  - [Dependencies](#dependencies)
+  - [Scalability](#scalability)
+  - [Troubleshooting](#troubleshooting)
+- [Implementation History](#implementation-history)
+- [Drawbacks](#drawbacks)
+- [Alternatives](#alternatives)
+- [Infrastructure Needed (Optional)](#infrastructure-needed-optional)
+<!-- /toc -->
+
+## Release Signoff Checklist
+
+Items marked with (R) are required *prior to targeting to a milestone / release*.
+
+- [ ] (R) Enhancement issue in release milestone, which links to KEP dir in [kubernetes/enhancements] (not the initial KEP PR)
+- [ ] (R) KEP approvers have approved the KEP status as `implementable`
+- [ ] (R) Design details are appropriately documented
+- [ ] (R) Test plan is in place, giving consideration to SIG Architecture and SIG Testing input (including test refactors)
+  - [ ] e2e Tests for all Beta API Operations (endpoints)
+  - [ ] (R) Ensure GA e2e tests meet requirements for [Conformance Tests](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/conformance-tests.md) 
+  - [ ] (R) Minimum Two Week Window for GA e2e tests to prove flake free
+- [ ] (R) Graduation criteria is in place
+  - [ ] (R) [all GA Endpoints](https://github.com/kubernetes/community/pull/1806) must be hit by [Conformance Tests](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/conformance-tests.md) 
+- [ ] (R) Production readiness review completed
+- [ ] (R) Production readiness review approved
+- [ ] "Implementation History" section is up-to-date for milestone
+- [ ] User-facing documentation has been created in [kubernetes/website], for publication to [kubernetes.io]
+- [ ] Supporting documentation—e.g., additional design documents, links to mailing list discussions/SIG meetings, relevant PRs/issues, release notes
+
+<!--
+**Note:** This checklist is iterative and should be reviewed and updated every time this enhancement is being considered for a milestone.
+-->
+
+[kubernetes.io]: https://kubernetes.io/
+[kubernetes/enhancements]: https://git.k8s.io/enhancements
+[kubernetes/kubernetes]: https://git.k8s.io/kubernetes
+[kubernetes/website]: https://git.k8s.io/website
+
+## Summary
+
+This KEP proposes deprecation of `ipvs` in kube-proxy.
+
+## Motivation
+
+At time of writing, kube-proxy has four supported backends (`iptables`, `ipvs`, `nftables`, and `winkernel`).
+
+sig-network currently lacks maintainers who are familar with the `ipvs` backend code, and as such, has been encouraging
+users who report `ipvs` bugs to move to using the `nftables` backend mode, where they can. (ie: [first example], [second example])
+
+`ipvs` was introduced in [KEP-265] to solve performance problems in large clusters.
+[KEP-3866] was created to introduce a new `nftables` mode to kube-proxy. The goal of this new backend mode
+has always been to eventually replace `ipvs` and `iptables`[^1], as it solve the performance issues of iptables
+and already solves many of the bugs present in the `ipvs` mode.
+
+[First example]: https://github.com/kubernetes/kubernetes/issues/132689#issuecomment-3031585314
+[second example]: https://github.com/kubernetes/kubernetes/issues/132068#issuecomment-2945169346
+[^1]: See https://github.com/kubernetes/enhancements/blob/master/keps/sig-network/3866-nftables-proxy/README.md#we-will-hopefully-be-able-to-trade-2-supported-backends-for-1
+
+[KEP-265]: https://github.com/kubernetes/enhancements/blob/master/keps/sig-network/265-ipvs-based-load-balancing/README.md
+[KEP-3866]: https://github.com/kubernetes/enhancements/blob/master/keps/sig-network/3866-nftables-proxy/README.md
+
+### Goals
+
+- Deprecate the `ipvs` mode of kube-proxy
+
+### Non-Goals
+
+- Removal of the `ipvs` mode of kube-proxy - The specifics of removal of `ipvs` are being handled in [KEP-5343]
+
+[KEP-5343]: https://github.com/kubernetes/enhancements/pull/5344
+
+## Proposal
+
+### Notes/Constraints/Caveats (Optional)
+
+The specifics of removal of `ipvs` mode will be handled in [KEP-5343]
+
+### Risks and Mitigations
+
+N/A
+
+## Design Details
+
+### Test Plan
+
+[X] I/we understand the owners of the involved components may require updates to
+existing tests to make this code solid enough prior to committing the changes necessary
+to implement this enhancement.
+
+##### Prerequisite testing updates
+
+##### Unit tests
+
+N/A - This KEP is only adding a deprecation warning and updating necessary parts of the website
+
+##### Integration tests
+
+N/A - This KEP is only adding a deprecation warning and updating nessesary parts of the website
+
+##### e2e tests
+
+N/A - This KEP is only adding a deprecation warning and updating nessesary parts of the website
+
+### Graduation Criteria
+
+#### Deprecation
+
+- The Kubernetes web site has been updated with deprecation notices for the `ipvs` mode of kube-proxy
+- Kube-proxy prints a warning when a user starts kube-proxy in `ipvs` mode
+- All nftables-mode bugfixes have been backported to 1.34 and 1.33, to ensure that `ipvs` users on older releases can still migrate to `nftables`.
+
+### Upgrade / Downgrade Strategy
+
+N/A - The only code change is a warning notice
+
+### Version Skew Strategy
+
+N/A - The only code change is a warning notice
+
+## Production Readiness Review Questionnaire
+
+### Feature Enablement and Rollback
+
+###### How can this feature be enabled / disabled in a live cluster?
+
+N/A
+
+###### Does enabling the feature change any default behavior?
+
+N/A
+
+###### Can the feature be disabled once it has been enabled (i.e. can we roll back the enablement)?
+
+N/A
+
+###### What happens if we reenable the feature if it was previously rolled back?
+
+###### Are there any tests for feature enablement/disablement?
+
+N/A
+
+### Rollout, Upgrade and Rollback Planning
+
+N/A
+
+###### How can a rollout or rollback fail? Can it impact already running workloads?
+
+N/A
+
+###### What specific metrics should inform a rollback?
+
+N/A
+
+###### Were upgrade and rollback tested? Was the upgrade->downgrade->upgrade path tested?
+
+N/A
+
+###### Is the rollout accompanied by any deprecations and/or removals of features, APIs, fields of API types, flags, etc.?
+
+N/A
+
+### Monitoring Requirements
+
+N/A
+
+###### How can an operator determine if the feature is in use by workloads?
+
+N/A
+
+###### How can someone using this feature know that it is working for their instance?
+
+N/A
+
+###### What are the reasonable SLOs (Service Level Objectives) for the enhancement?
+
+N/A
+
+###### What are the SLIs (Service Level Indicators) an operator can use to determine the health of the service?
+
+N/A
+
+###### Are there any missing metrics that would be useful to have to improve observability of this feature?
+
+N/A
+
+### Dependencies
+
+###### Does this feature depend on any specific services running in the cluster?
+
+N/A
+
+### Scalability
+
+N/A
+
+###### Will enabling / using this feature result in any new API calls?
+
+N/A
+
+###### Will enabling / using this feature result in introducing new API types?
+
+N/A
+
+###### Will enabling / using this feature result in any new calls to the cloud provider?
+
+N/A
+
+###### Will enabling / using this feature result in increasing size or count of the existing API objects?
+
+N/A
+
+###### Will enabling / using this feature result in increasing time taken by any operations covered by existing SLIs/SLOs?
+
+N/A
+
+###### Will enabling / using this feature result in non-negligible increase of resource usage (CPU, RAM, disk, IO, ...) in any components?
+
+N/A
+
+###### Can enabling / using this feature result in resource exhaustion of some node resources (PIDs, sockets, inodes, etc.)?
+
+N/A
+
+### Troubleshooting
+
+###### How does this feature react if the API server and/or etcd is unavailable?
+
+###### What are other known failure modes?
+
+N/A
+
+###### What steps should be taken if SLOs are not being met to determine the problem?
+
+## Implementation History
+
+N/A
+
+## Drawbacks
+
+Users may be running version of the kernel which isn't new enough to support the `nftables` backend mode in kube-proxy.
+
+## Alternatives
+
+Getting active maintainers for `ipvs` may be a short term alternative, see [The ipvs mode of kube-proxy will not save us] (from [KEP-3866]) for details
+
+[The ipvs mode of kube-proxy will not save us]: https://github.com/kubernetes/enhancements/blob/master/keps/sig-network/3866-nftables-proxy/README.md#the-ipvs-mode-of-kube-proxy-will-not-save-us
+
+## Infrastructure Needed (Optional)
+
+N/A

keps/sig-network/5495-deprecate-ipvs-mode-in-kube-proxy/kep.yaml

@@ -0,0 +1,37 @@
+title: Deprecate ipvs mode in kube-proxy
+kep-number: 5495
+authors:
+  - "@adrianmoisey"
+owning-sig: sig-network
+participating-sigs: []
+status: implementable
+creation-date: 2025-08-25
+reviewers:
+  - "@thockin"
+  - "@danwinship"
+  - "@aojea"
+approvers:
+  - "@thockin"
+  - "@danwinship"
+  - "@aojea"
+
+see-also:
+  - "/keps/sig-network/265-ipvs-based-load-balancing/README.md"
+  - "/keps/sig-network/3866-nftables-proxy/README.md"
+replaces: []
+
+stage: deprecated
+
+latest-milestone: "v1.35"
+
+# The milestone at which this feature was, or is targeted to be, at each stage.
+milestone:
+  deprecated: "v1.35"
+
+# The following PRR answers are required at alpha release
+# List the feature gate name and the components for which it must be enabled
+feature-gates: []
+disable-supported: false
+
+# The following PRR answers are required at beta release
+metrics: []