Configuration for scaling down only empty nodes not working #8707
Description
Which component are you using?:
/area cluster-autoscaler
What version of the component are you using?:
Component version: v1.32.1
What k8s version are you using (kubectl version)?:
1.33.3
What environment is this in?:
AKS cluster
What did you expect to happen?:
I want to configure the cluster autoscaler to only scale down empty nodes, because some applications hosted on the cluster are not replicated and cannot be interrupted.
According to this documentation in your FAQ, I tried setting the --scale-down-utilization-threshold parameter to 0 and --ignore-daemonsets-utilization to true (I don't have any mirror pods running on my nodes). By doing this, I expect that a node with some pods running on it (excluding daemonsets) is never a candidate for scale down, while an empty node (with only daemonsets running on it) can be scaled down normally.
What happened instead?:
When I set the threshold to 0 and the ignore-daemonsets parameter, the autoscaler is not performing any scale down operation anymore. Even when a node is empty (containing only daemonsets), it is not scaled down.
How to reproduce it (as minimally and precisely as possible):
Configure the cluster autoscaler with --scale-down-utilization-threshold to 0 and --ignore-daemonsets-utilization to true.
You can also decrease scale-down-unneeded-time and scale-down-delay-after-add values to make it easier to test.
Trigger a scale up of a new node in the cluster, either forcing it manually, or by creating a new pod with an appropriate request. Then delete your test pod, leaving the new node empty. Wait for the delay configured above to see if a scale down is triggered. You can also look at the logs of the cluster autoscaler.
Anything else we need to know?:
I have the impression that issue might come from these lines of code verifying the scale down eligibility of a node. Indeed, all nodes with utilization that is above or equal to the threshold are not marked as scale down candidates. As such, if threshold is equal to 0, all nodes will meet the condition "utilization >= 0" and no one can be scaled down. To follow the FAQ documentation and allow for scaling down only empty nodes, I think that maybe the "equal" sign should be removed from there.