Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion api/core/v1beta2/machineset_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ const (
MachineSetMoveMachinesToMachineSetAnnotation = "in-place-updates.internal.cluster.x-k8s.io/move-machines-to-machineset"

// MachineSetReceiveMachinesFromMachineSetsAnnotation is an internal annotation added by the MD controller to the newMS
// when it should receive replicas from oldMSs as a first step of an in-place upgrade operation
// when it should receive replicas from oldMSs as a first step of an in-place update operation
// The annotation value is a comma separated list of oldMSs.
// Note: This annotation is used in pair with MachineSetMoveMachinesToMachineSetAnnotation to perform a two-ways check before moving a machine from oldMS to newMS:
//
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -146,17 +146,17 @@ underlying objects like control plane and MachineDeployment act in the same way

| Changed field | Effects on Clusters |
|---------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| infrastructure.ref | Corresponding InfrastructureCluster objects are updated (in place update). |
| controlPlane.metadata | If labels/annotations are added, changed or deleted the ControlPlane objects are updated (in place update).<br /><br /> In case of KCP, corresponding controlPlane Machines, KubeadmConfigs and InfrastructureMachines are updated in-place. |
| controlPlane.ref | Corresponding ControlPlane objects are updated (in place update). <br /> If updating ControlPlane objects implies changes in the spec, the corresponding ControlPlane Machines are updated accordingly (rollout). |
| controlPlane.machineInfrastructure.ref | If the referenced template has changes only in metadata labels or annotations, the corresponding InfrastructureMachineTemplates are updated (in place update). <br /> <br />If the referenced template has changes in the spec:<br /> - Corresponding InfrastructureMachineTemplate are rotated (create new, delete old)<br /> - Corresponding ControlPlane objects are updated with the reference to the newly created template (in place update)<br /> - The corresponding controlPlane Machines are updated accordingly (rollout). |
| infrastructure.ref | Corresponding InfrastructureCluster objects are updated (in-place update). |
| controlPlane.metadata | If labels/annotations are added, changed or deleted the ControlPlane objects are updated (in-place update).<br /><br /> In case of KCP, corresponding controlPlane Machines, KubeadmConfigs and InfrastructureMachines are updated in-place. |
| controlPlane.ref | Corresponding ControlPlane objects are updated (in-place update). <br /> If updating ControlPlane objects implies changes in the spec, the corresponding ControlPlane Machines are updated accordingly (rollout). |
| controlPlane.machineInfrastructure.ref | If the referenced template has changes only in metadata labels or annotations, the corresponding InfrastructureMachineTemplates are updated (in-place update). <br /> <br />If the referenced template has changes in the spec:<br /> - Corresponding InfrastructureMachineTemplate are rotated (create new, delete old)<br /> - Corresponding ControlPlane objects are updated with the reference to the newly created template (in-place update)<br /> - The corresponding controlPlane Machines are updated accordingly (rollout). |
| controlPlane.nodeDrainTimeout | If the value is changed the ControlPlane object is updated in-place.<br/> <br/> In case of KCP, the change is propagated in-place to control plane Machines. |
| controlPlane.nodeVolumeDetachTimeout | If the value is changed the ControlPlane object is updated in-place.<br/> <br/> In case of KCP, the change is propagated in-place to control plane Machines. |
| controlPlane.nodeDeletionTimeout | If the value is changed the ControlPlane object is updated in-place.<br/> <br/> In case of KCP, the change is propagated in-place to control plane Machines. |
| workers.machineDeployments | If a new MachineDeploymentClass is added, no changes are triggered to the Clusters. <br />If an existing MachineDeploymentClass is changed, effect depends on the type of change (see below). |
| workers.machineDeployments[].template.metadata | If labels/annotations are added, changed or deleted the MachineDeployment objects are updated (in place update) and corresponding worker Machines are updated (in-place). |
| workers.machineDeployments[].template.bootstrap.ref | If the referenced template has changes only in metadata labels or annotations, the corresponding BootstrapTemplates are updated (in place update).<br /> <br />If the referenced template has changes in the spec:<br /> - Corresponding BootstrapTemplate are rotated (create new, delete old). <br /> - Corresponding MachineDeployments objects are updated with the reference to the newly created template (in place update). <br /> - The corresponding worker machines are updated accordingly (rollout) |
| workers.machineDeployments[].template.infrastructure.ref | If the referenced template has changes only in metadata labels or annotations, the corresponding InfrastructureMachineTemplates are updated (in place update). <br /> <br />If the referenced template has changes in the spec:<br /> - Corresponding InfrastructureMachineTemplate are rotated (create new, delete old).<br /> - Corresponding MachineDeployments objects are updated with the reference to the newly created template (in place update). <br /> - The corresponding worker Machines are updated accordingly (rollout) |
| workers.machineDeployments[].template.metadata | If labels/annotations are added, changed or deleted the MachineDeployment objects are updated (in-place update) and corresponding worker Machines are updated (in-place). |
| workers.machineDeployments[].template.bootstrap.ref | If the referenced template has changes only in metadata labels or annotations, the corresponding BootstrapTemplates are updated (in-place update).<br /> <br />If the referenced template has changes in the spec:<br /> - Corresponding BootstrapTemplate are rotated (create new, delete old). <br /> - Corresponding MachineDeployments objects are updated with the reference to the newly created template (in-place update). <br /> - The corresponding worker machines are updated accordingly (rollout) |
| workers.machineDeployments[].template.infrastructure.ref | If the referenced template has changes only in metadata labels or annotations, the corresponding InfrastructureMachineTemplates are updated (in-place update). <br /> <br />If the referenced template has changes in the spec:<br /> - Corresponding InfrastructureMachineTemplate are rotated (create new, delete old).<br /> - Corresponding MachineDeployments objects are updated with the reference to the newly created template (in-place update). <br /> - The corresponding worker Machines are updated accordingly (rollout) |
| workers.machineDeployments[].template.nodeDrainTimeout | If the value is changed the MachineDeployment is updated in-place.<br/> <br/> The change is propagated in-place to the MachineDeployment Machine. |
| workers.machineDeployments[].template.nodeVolumeDetachTimeout | If the value is changed the MachineDeployment is updated in-place.<br/> <br/> The change is propagated in-place to the MachineDeployment Machine. |
| workers.machineDeployments[].template.nodeDeletionTimeout | If the value is changed the MachineDeployment is updated in-place.<br/> <br/> The change is propagated in-place to the MachineDeployment Machine. |
Expand Down
2 changes: 1 addition & 1 deletion docs/book/src/user/concepts.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ A "Machine" is the declarative spec for an infrastructure component hosting a Ku

Common fields such as Kubernetes version are modeled as fields on the Machine's spec. Any information that is provider-specific is part of the `InfrastructureRef` and is not portable between different providers.

#### Machine Immutability (In-place Upgrade vs. Replace)
#### Machine Immutability (In-place Update vs. Replace)

From the perspective of Cluster API, all Machines are immutable: once they are created, they are never updated (except for labels, annotations and status), only deleted.

Expand Down
2 changes: 1 addition & 1 deletion docs/proposals/20181121-machine-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ One simplification that might be controversial in this proposal is the lack of
API control over "in-place" versus "replace" reconciliation strategies. For
instance, if a Machine's spec is updated with a different version of kubelet
than is actually running, it is up to the provider-specific controller whether
the request would best be fulfilled by performing an in-place upgrade on the
the request would best be fulfilled by performing an in-place update on the
Node, or by deleting the Node and creating a new one in its place (or reporting
an error if this particular update is not supported). One can force a Node
replacement by deleting and recreating the Machine object rather than updating
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,9 @@ into the proposal or into the user-facing documentation for this feature.
- Old MS is instructed to move machines to the newMS, and newMS is informed to receive machines from oldMS.
- MS controller manages a subset of Machines
- When scaling down the old MS, if required to move, MS controller is responsible for moving a Machine to newMS
- When reconciling the new MachineSet, the MS controller takes ownership of the moved machine and begins the actual in-place upgrade.
- When reconciling the new MachineSet, the MS controller takes ownership of the moved machine and begins the actual in-place update.

- Orchestration of in-place upgrades between MD controller, MS controller, and Machine controller is implemented using annotations.
- Orchestration of in-place updates between MD controller, MS controller, and Machine controller is implemented using annotations.
Following schemas provide an overview of how new annotations are used.

Workflow #1: MD controller detects an in-place update is possible and it informs oldMS and newMS about how to perform this operation.
Expand Down Expand Up @@ -74,7 +74,7 @@ sequenceDiagram
MD Controller->>MS2 (NewMS): Scale up to acknowledge receipt of M1<br/>Apply annotation ".../acknowledged-move": "M1"
```

Workflow #4: MS controller, when reconciling newMS, detects that a machine has been acknowledged; it cleans up annotations on the machine, allowing the in-place upgrade to begin.
Workflow #4: MS controller, when reconciling newMS, detects that a machine has been acknowledged; it cleans up annotations on the machine, allowing the in-place update to begin.

```mermaid
sequenceDiagram
Expand Down
8 changes: 4 additions & 4 deletions docs/proposals/20240807-in-place-updates.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ __In-place Update__: any change to a Machine spec, including the Kubernetes Vers

__Update Lifecycle Hook__: CAPI Lifecycle Runtime Hook to invoke external update extensions.

__Update Extension__: Runtime Extension (Implementation) is a component responsible to perform in place updates when the `External Update Lifecycle Hook` is invoked.
__Update Extension__: Runtime Extension (Implementation) is a component responsible to perform in-place updates when the `External Update Lifecycle Hook` is invoked.

## Summary

Expand Down Expand Up @@ -114,9 +114,9 @@ With the implementation of an Update extension, users can take ownership of the

### Divide and conquer

Considering the complexity of this topic, a phased approach is required to design and implement the solution for in-place upgrades.
Considering the complexity of this topic, a phased approach is required to design and implement the solution for in-place updates.

The main goal of the first iteration of this proposal is to make it possible for Cluster API users to start experimenting usage of in-place upgrades, so we can gather feedback and evolve to the next stage.
The main goal of the first iteration of this proposal is to make it possible for Cluster API users to start experimenting usage of in-place updates, so we can gather feedback and evolve to the next stage.

This iteration will focus on implementing the machinery required to interact with update extensions, while users facing changes in the API types are deferred to follow up iterations.

Expand Down Expand Up @@ -172,7 +172,7 @@ As a cluster operator, I want to perform in-place updates on my Kubernetes clust

#### Story 2

As a cluster operator, I want to seamlessly transition between rolling and in-place updates while maintaining a consistent user interface. I appreciate the option to extend the rollout process with in-place upgrade capabilities, ensuring that the update process aligns with my organization's unique needs.
As a cluster operator, I want to seamlessly transition between rolling and in-place updates while maintaining a consistent user interface. I appreciate the option to extend the rollout process with in-place update capabilities, ensuring that the update process aligns with my organization's unique needs.

#### Story 3
As a cluster operator for resource constrained environments, I want to utilize CAPI pluggable external update mechanism to implement in-place updates without requiring additional compute capacity in a single node cluster.
Expand Down
Loading