Skip to content

Commit 579af1b

Browse files
committed
Add CA compatability section
1 parent 9db91ae commit 579af1b

File tree

1 file changed

+20
-4
lines changed
  • keps/sig-node/3953-node-resource-hot-plug

1 file changed

+20
-4
lines changed

keps/sig-node/3953-node-resource-hot-plug/README.md

Lines changed: 20 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,7 @@ tags, and then generate with `hack/update-toc.sh`.
2929
- [Design Details](#design-details)
3030
- [Handling hotplug events](#handling-hotplug-events)
3131
- [Flow Control for updating swap limit for containers](#flow-control-for-updating-swap-limit-for-containers)
32+
- [Compatability with Cluster Autoscaler](#compatability-with-cluster-autoscaler)
3233
- [Handling HotUnplug Events](#handling-hotunplug-events)
3334
- [Flow Control](#flow-control)
3435
- [Test Plan](#test-plan)
@@ -135,15 +136,15 @@ Implementing this KEP will empower nodes to recognize and adapt to changes in th
135136

136137
### Goals
137138

138-
* Achieve seamless node capacity expansion through hot plugging resources.
139+
* Achieve seamless node capacity expansion through resource hotplug.
139140
* Enable the re-initialization of resource managers (CPU manager, memory manager) and kube runtime manager without reset to accommodate alterations in the node's resource allocation.
140141
* Recalculating and updating the OOMScoreAdj and swap memory limit for existing pods.
141142

142143
### Non-Goals
143144

144145
* Dynamically adjust system reserved and kube reserved values.
145146
* Hot unplug of node resources.
146-
* Update the autoscaler to utilize resource hot plugging.
147+
* Update the autoscaler to utilize resource hotplug.
147148
* Re-balance workloads across the nodes.
148149
* Update runtime/NRI plugins with host resource changes.
149150

@@ -278,9 +279,9 @@ With increase in cluster resources the following components will be updated:
278279
Once the capacity of the node is altered, the following are the sequence of events that occur in the kubelet. If any errors are
279280
observed in any of the steps, operation is retried from step 1 along with a `FailedNodeResize` event under the node object.
280281
1. Resizing existing containers:
281-
a.With the increased memory capacity of the nodes, the kubelet proceeds to update fields that are directly related to
282+
a. With the increased memory capacity of the nodes, the kubelet proceeds to update fields that are directly related to
282283
the available memory on the host. This would lead to recalculation of oom_score_adj and swap_limits.
283-
b.This is achieved by invoking the CRI API - UpdateContainerResources.
284+
b. This is achieved by invoking the CRI API - UpdateContainerResources.
284285

285286
2. Reinitialise Resource Manager:
286287
a. Resource managers such as CPU,Memory are updated with the latest available capacities on the host. This posts the latest
@@ -318,6 +319,21 @@ T=1: Resize Instance to Hotplug Memory:
318319

319320
Similar flow is applicable for updating oom_score_adj.
320321

322+
#### Compatability with Cluster Autoscaler
323+
324+
The Cluster Autoscaler (CA) presently anticipates uniform allocatable values among nodes within the same NodeGroup, using existing Nodes as templates for
325+
newly provisioned Nodes from the same NodeGroup. However, with the introduction of NodeResourceHotplug, this assumption may no longer hold true.
326+
If not appropriately addressed, this could cause the Cluster Autoscaler to randomly select a Node from the group and assume identical allocatable values for all upcoming Nodes.
327+
This could lead to suboptimal decisions, such as repeatedly attempting to provision Nodes for pending Pods that are incompatible, or overlooking potential Nodes that could accommodate such Pods.
328+
329+
To ensure the Cluster Autoscaler acknowledges resource hotplug, the following approaches have been proposed by the Cluster Autoscaler team:
330+
1. Capture Node's Initial Allocatable Values:
331+
* Introduce a new field within the Node object to record initial node allocatable values, which remain unchanged during resource hotplug.
332+
* The Cluster Autoscaler can leverage this field to anticipate potential hotplug of resources, using it as a template for configuring new Nodes.
333+
334+
2. Identify Nodes Affected by Hotplug:
335+
* By flagging a Node as being impacted by hotplug, the Cluster Autoscaler could revert to a less reliable but more conservative "scale from 0 nodes" logic.
336+
321337
### Handling HotUnplug Events
322338

323339
Though this KEP focuses only on resource hotplug, It will enable the kubelet to capture the current available capacity of the node (Regardless of whether it was a hotplug or hotunplug of resources.)

0 commit comments

Comments
 (0)