Skip to content

✨ Support Node Auto Placement and Node AF/AAF #3655

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 25 commits into
base: main
Choose a base branch
from
Open

✨ Support Node Auto Placement and Node AF/AAF #3655

wants to merge 25 commits into from

Conversation

@zhanggbj
Copy link
Contributor

@zhanggbj zhanggbj commented Oct 28, 2025 •
edited
Loading

What this PR does / why we need it:
Support Node Auto Placement and Node AF/AAF

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

zhanggbj and others added 11 commits October 22, 2025 14:19
@zhanggbj
Bump VMOP and add NodeAutoPlacement Feature Gate
fa60556 
- Bump VMOP including Node AF/AAF support
- Add NodeAutoPlacement Feature Gate

(cherry picked from commit 700c8ae)
@zhanggbj
Add VMG controller of node auto placement
b52056f 
(cherry picked from commit cfeb862)
@srm09 @zhanggbj
Updates logic for VMG creation
72fad61 
Removes the extra cases for VMG creation, such that VMG is created for:
1. Multiple zones, multiple MDs with no failureDomain
2. Multiple zones, multiple MDs with failureDomain
3. Single zone, existing cluster with no failureDomain MDs

Signed-off-by: Sagar Muchhal <sagar.muchhal@broadcom.com>
@srm09 @zhanggbj
Initial impl for VSphereMachine AAF changes
124623a 
- Updates VMOP API dependency

Misc VMG fixes
- Use namingStrategy to calculate VM names
- Use MachineDeployment names for VMG placement label
- Includes all machinedeployments to generate node-pool -> zone
  mapping

Fixes VMG webhook validation error
- Adds cluster-name label to Af/AAF spec
- re-adds zone topology key back to anti-aff spec

Signed-off-by: Sagar Muchhal <sagar.muchhal@broadcom.com>
@srm09 @zhanggbj
Removes cluster-name label from affinity rules
9f261d5 
Signed-off-by: Sagar Muchhal <sagar.muchhal@broadcom.com>
@srm09 @zhanggbj
Selectively add node-pool AAF constraint
22d7db0 
Signed-off-by: Sagar Muchhal <sagar.muchhal@broadcom.com>
@zhanggbj @srm09
Refine VMG controller when generate per-MD zone labels (kubernetes-si…
8474942 
…gs#71)

* Refine VMG controller when generate per-MD zone labels

- Skip legacy already-placed VM which do not have placement info
- Skip VM which do not have zone info

* Apply suggestions from code review

---------

Co-authored-by: Sagar Muchhal <sagar.muchhal@broadcom.com>
@zhanggbj
Sync VSphereMachines in VMG controller
833cdb5 
- Sync VSphereMachines during day-2 operations in VMG controller
- Only wait for all intended VSphereMachines during initial Cluster creation
- Use annotations in VMG for per-md-zone info

Signed-off-by: Gong Zhang <gong.zhang@broadcom.com>
@zhanggbj
Add VMG reconciler unit test and bump VMOP
36c3550 
- Add VMG recociler unit test
- Bump VMOP due to API change
- Filter out VSphereMachine event except create/delete events

Signed-off-by: Gong Zhang <gong.zhang@broadcom.com>
@zhanggbj
Surface error when VM is waiting for VMG
189f6d2 
Signed-off-by: Gong Zhang <gong.zhang@broadcom.com>
@zhanggbj
Update dependency in packaging&test
8cc8e8a 
Signed-off-by: Gong Zhang <gong.zhang@broadcom.com>
@k8s-ci-robot k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Oct 28, 2025
@k8s-ci-robot k8s-ci-robot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Oct 28, 2025
@zhanggbj zhanggbj changed the title [WIP][DoNotReview]Support Node Auto Placement and Node AF/AAF [WIP][DoNotReview]✨ Support Node Auto Placement and Node AF/AAF Oct 28, 2025
@zhanggbj zhanggbj force-pushed the node_auto_placement branch 3 times, most recently from 1160ce9 to 1cd61f9 Compare October 28, 2025 10:31
@zhanggbj
Copy link
Contributor Author

zhanggbj commented Oct 31, 2025

/test ?

@k8s-ci-robot
Copy link
Contributor

k8s-ci-robot commented Oct 31, 2025

@zhanggbj: The following commands are available to trigger required jobs:

/test pull-cluster-api-provider-vsphere-e2e-govmomi-blocking-main

/test pull-cluster-api-provider-vsphere-e2e-govmomi-conformance-ci-latest-main

/test pull-cluster-api-provider-vsphere-e2e-govmomi-conformance-main

/test pull-cluster-api-provider-vsphere-e2e-govmomi-main

/test pull-cluster-api-provider-vsphere-e2e-govmomi-upgrade-1-34-1-35-main

/test pull-cluster-api-provider-vsphere-e2e-supervisor-blocking-main

/test pull-cluster-api-provider-vsphere-e2e-supervisor-conformance-ci-latest-main

/test pull-cluster-api-provider-vsphere-e2e-supervisor-conformance-main

/test pull-cluster-api-provider-vsphere-e2e-supervisor-main

/test pull-cluster-api-provider-vsphere-e2e-supervisor-upgrade-1-34-1-35-main

/test pull-cluster-api-provider-vsphere-e2e-vcsim-govmomi-main

/test pull-cluster-api-provider-vsphere-e2e-vcsim-supervisor-main

/test pull-cluster-api-provider-vsphere-test-main

/test pull-cluster-api-provider-vsphere-verify-main

The following commands are available to trigger optional jobs:

/test pull-cluster-api-provider-vsphere-apidiff-main

/test pull-cluster-api-provider-vsphere-janitor-main

Use /test all to run the following jobs that were automatically triggered:

pull-cluster-api-provider-vsphere-apidiff-main

pull-cluster-api-provider-vsphere-e2e-govmomi-blocking-main

pull-cluster-api-provider-vsphere-e2e-supervisor-blocking-main

pull-cluster-api-provider-vsphere-e2e-vcsim-govmomi-main

pull-cluster-api-provider-vsphere-e2e-vcsim-supervisor-main

pull-cluster-api-provider-vsphere-test-main

pull-cluster-api-provider-vsphere-verify-main

In response to this:

/test ?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@zhanggbj
Copy link
Contributor Author

zhanggbj commented Oct 31, 2025

/test pull-cluster-api-provider-vsphere-e2e-govmomi-blocking-main
/test pull-cluster-api-provider-vsphere-e2e-govmomi-conformance-ci-latest-main
/test pull-cluster-api-provider-vsphere-e2e-govmomi-conformance-main
/test pull-cluster-api-provider-vsphere-e2e-govmomi-main
/test pull-cluster-api-provider-vsphere-e2e-govmomi-upgrade-1-34-1-35-main
/test pull-cluster-api-provider-vsphere-e2e-supervisor-blocking-main
/test pull-cluster-api-provider-vsphere-e2e-supervisor-conformance-ci-latest-main
/test pull-cluster-api-provider-vsphere-e2e-supervisor-conformance-main
/test pull-cluster-api-provider-vsphere-e2e-supervisor-main
/test pull-cluster-api-provider-vsphere-e2e-supervisor-upgrade-1-34-1-35-main
/test pull-cluster-api-provider-vsphere-e2e-vcsim-govmomi-main
/test pull-cluster-api-provider-vsphere-e2e-vcsim-supervisor-main
/test pull-cluster-api-provider-vsphere-test-main
/test pull-cluster-api-provider-vsphere-verify-main
/test pull-cluster-api-provider-vsphere-apidiff-main
/test pull-cluster-api-provider-vsphere-janitor-main

@zhanggbj zhanggbj force-pushed the node_auto_placement branch 7 times, most recently from 22604bf to 7c7c1a7 Compare November 3, 2025 08:55
@zhanggbj
Address review comments
dc441f6 
- Refine VMG controller watch
- Handle race conditions in VMG controller by gating member update
- Refine data struct for VM Affinity config
- Refine UT
- Refine naming, logging, godoc
- Miscellaneous

Signed-off-by: Gong Zhang <gong.zhang@broadcom.com>
@zhanggbj zhanggbj force-pushed the node_auto_placement branch from ea432ed to dc441f6 Compare November 21, 2025 13:29
@zhanggbj
Fix UT failure
9cc5762 
Signed-off-by: Gong Zhang <gong.zhang@broadcom.com>
zhanggbj
zhanggbj commented Nov 23, 2025
View reviewed changes
Copy link
Contributor Author

@zhanggbj zhanggbj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All comments are addressed, including the ones missed by mistake last time.

zhanggbj

This comment was marked as spam.

Sign in to view

@zhanggbj zhanggbj force-pushed the node_auto_placement branch 2 times, most recently from 8dabe39 to 19ebccb Compare November 23, 2025 13:27
@zhanggbj
Refine godoc
b981168 
Signed-off-by: Gong Zhang <gong.zhang@broadcom.com>
@zhanggbj zhanggbj force-pushed the node_auto_placement branch from 19ebccb to b981168 Compare November 23, 2025 14:16
@zhanggbj
Copy link
Contributor Author

zhanggbj commented Nov 24, 2025

/test pull-cluster-api-provider-vsphere-e2e-vcsim-supervisor-main

@zhanggbj zhanggbj force-pushed the node_auto_placement branch 3 times, most recently from 42a6d73 to 3ed2e93 Compare November 24, 2025 14:46
@zhanggbj
Copy link
Contributor Author

zhanggbj commented Nov 24, 2025

/test pull-cluster-api-provider-vsphere-e2e-govmomi-blocking-main
/test pull-cluster-api-provider-vsphere-e2e-supervisor-blocking-main

@zhanggbj
Copy link
Contributor Author

zhanggbj commented Nov 24, 2025

Hey @fabriziopandini @sbueringer ,
All comments are addressed, including the ones missed by mistake last time. Now CI are green. Please help to take another look, really appreciate!

For the concern about race conditions, I have implemented checks to mitigate race conditions during initial and post-placement, and also have unit teststo cover the scenarios.

@zhanggbj zhanggbj force-pushed the node_auto_placement branch from 3ed2e93 to 9b1754a Compare November 25, 2025 06:44
@zhanggbj
Copy link
Contributor Author

zhanggbj commented Nov 25, 2025

Pushed a small change and checking UT failures.

@zhanggbj
Refine VMG member update to avoid race condition
2ac47d4 
Signed-off-by: Gong Zhang <gong.zhang@broadcom.com>
@zhanggbj zhanggbj force-pushed the node_auto_placement branch from 9b1754a to 2ac47d4 Compare November 25, 2025 09:25
@zhanggbj
Copy link
Contributor Author

zhanggbj commented Nov 25, 2025

/test pull-cluster-api-provider-vsphere-e2e-supervisor-blocking-main

@zhanggbj zhanggbj mentioned this pull request Nov 26, 2025
Open
@zhanggbj
Copy link
Contributor Author

zhanggbj commented Nov 26, 2025

/test pull-cluster-api-provider-vsphere-e2e-govmomi-blocking-main

@zhanggbj
Update vm-operator package to v1.9.1-0.20251003150112-9b458d311c4c
c41abeb 
Since Encryption Class requires API in this newer version, bump vm-operator package

Signed-off-by: Gong Zhang <gong.zhang@broadcom.com>
@zhanggbj zhanggbj force-pushed the node_auto_placement branch from fd72f09 to c41abeb Compare November 26, 2025 07:49
zhanggbj and others added 5 commits November 27, 2025 11:46
@zhanggbj
Added reconcile flow cases
516af84 
Signed-off-by: Gong Zhang <gong.zhang@broadcom.com>
@fabriziopandini @zhanggbj
POC AAF
ac0ec4b
@sbueringer @zhanggbj
Review comments / fixes
822f8f0
@fabriziopandini @zhanggbj
Final cleanup
db8095d
@zhanggbj
Finalize VMG controller and UT
08b8304 
Signed-off-by: Gong Zhang <gong.zhang@broadcom.com>
@zhanggbj zhanggbj force-pushed the node_auto_placement branch from b8c569e to 08b8304 Compare November 27, 2025 09:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@sbueringer sbueringer sbueringer left review comments

@fabriziopandini fabriziopandini fabriziopandini left review comments

@MaxRink MaxRink Awaiting requested review from MaxRink

@rvanderp3 rvanderp3 Awaiting requested review from rvanderp3

+2 more reviewers

@DanielXiao DanielXiao DanielXiao left review comments

@silvery1622 silvery1622 silvery1622 left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

@DanielXiao DanielXiao

@sbueringer sbueringer

@fabriziopandini fabriziopandini

Labels

cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@zhanggbj @k8s-ci-robot @DanielXiao @sbueringer @fabriziopandini @silvery1622 @srm09