Merge pull request #98803 from dfitzmau/OCPBUGS-59964

OCPBUGS-59964: Documented configuring dual-stack networking for an Eg…

Author: dfitzmau

_topic_maps/_topic_map.yml

@@ -1627,8 +1627,6 @@ Topics:
     File: configuring-secondary-external-gateway
   - Name: Configuring an egress IP address
     File: configuring-egress-ips-ovn
-  - Name: Assigning an egress IP address
-    File: assigning-egress-ips-ovn
   - Name: Configuring an egress service
     File: configuring-egress-traffic-for-vrf-loadbalancer-services
   - Name: Considerations for the use of an egress router pod

modules/nw-egress-ips-about.adoc

@@ -2,18 +2,13 @@
 //
 // * networking/ovn_kubernetes_network_provider/configuring-egress-ips-ovn.adoc
 
-ifeval::["{context}" == "configuring-egress-ips-ovn"]
-:ovn:
-endif::[]
-
+:_mod-docs-content-type: REFERENCE
 [id="nw-egress-ips-about_{context}"]
 = Egress IP address architectural design and implementation
 
-The {product-title} egress IP address functionality allows you to ensure that the traffic from one or more pods in one or more namespaces has a consistent source IP address for services outside the cluster network.
-
-For example, you might have a pod that periodically queries a database that is hosted on a server outside of your cluster. To enforce access requirements for the server, a packet filtering device is configured to allow traffic only from specific IP addresses.
-To ensure that you can reliably allow access to the server from only that specific pod, you can configure a specific egress IP address for the pod that makes the requests to the server.
+By using the {product-title} egress IP address functionality, you can ensure that the traffic from one or more pods in one or more namespaces has a consistent source IP address for services outside the cluster network. 
 
+For example, you might have a pod that periodically queries a database that is hosted on a server outside of your cluster. To enforce access requirements for the server, a packet filtering device is configured to allow traffic only from specific IP addresses. To ensure that you can reliably allow access to the server from only that specific pod, you can configure a specific egress IP address for the pod that makes the requests to the server.
 
 An egress IP address assigned to a namespace is different from an egress router, which is used to send traffic to specific destinations.
 
@@ -72,14 +67,14 @@ ifndef::openshift-rosa[]
 [id="nw-egress-ips-public-cloud-platform-considerations_{context}"]
 == Public cloud platform considerations
 
-Typically, public cloud providers place a limit on egress IPs. This means that there is a constraint on the absolute number of assignable IP addresses per node for clusters provisioned on public cloud infrastructure. The maximum number of assignable IP addresses per node, or the _IP capacity_, can be described in the following formula:
+Typically, public cloud providers place a limit on egress IP addresses. This means that there is a constraint on the absolute number of assignable IP addresses per node for clusters provisioned on public cloud infrastructure. The maximum number of assignable IP addresses per node, or the _IP capacity_, can be described in the following formula:
 
 [source,text]
 ----
 IP capacity = public cloud default capacity - sum(current IP assignments)
 ----
 
-While the Egress IPs capability manages the IP address capacity per node, it is important to plan for this constraint in your deployments. For example, if a public cloud provider limits IP address capacity to 10 IP addresses per node, and you have 8 nodes, the total number of assignable IP addresses is only 80. To achieve a higher IP address capacity, you would need to allocate additional nodes. For example, if you needed 150 assignable IP addresses, you would need to allocate 7 additional nodes.
+While the Egress IP addresses capability manages the IP address capacity per node, it is important to plan for this constraint in your deployments. For example, if a public cloud provider limits IP address capacity to 10 IP addresses per node, and you have 8 nodes, the total number of assignable IP addresses is only 80. To achieve a higher IP address capacity, you would need to allocate additional nodes. For example, if you needed 150 assignable IP addresses, you would need to allocate 7 additional nodes.
 
 To confirm the IP capacity and subnets for any node in your public cloud environment, you can enter the `oc get node <node_name> -o yaml` command. The `cloud.network.openshift.io/egress-ipconfig` annotation includes capacity and subnet information for the node.
 
@@ -160,80 +155,8 @@ On Azure, the following capacity limits exist for IP address assignment:
 
 For more information, see link:https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/azure-subscription-service-limits?toc=/azure/virtual-network/toc.json#networking-limits[Networking limits].
 
-ifdef::ovn[]
-[id="nw-egress-ips-multi-nic-considerations_{context}"]
-== Considerations for using an egress IP on additional network interfaces
-
-In {product-title}, egress IPs provide administrators a way to control network traffic. Egress IPs can be used with a `br-ex` Open vSwitch (OVS) bridge interface and any physical interface that has IP connectivity enabled.
-
-You can inspect your network interface type by running the following command:
-
-[source,terminal]
-----
-$ ip -details link show
-----
-
-The primary network interface is assigned a node IP address which also contains a subnet mask. Information for this node IP address can be retrieved from the Kubernetes node object for each node within your cluster by inspecting the `k8s.ovn.org/node-primary-ifaddr` annotation. In an IPv4 cluster, this annotation is similar to the following example: `"k8s.ovn.org/node-primary-ifaddr: {"ipv4":"192.168.111.23/24"}"`.
-
-If the egress IP is not within the subnet of the primary network interface subnet, you can use an egress IP on another Linux network interface that is not of the primary network interface type. By doing so, {product-title} administrators are provided with a greater level of control over networking aspects such as routing, addressing, segmentation, and security policies. This feature provides users with the option to route workload traffic over specific network interfaces for purposes such as traffic segmentation or meeting specialized requirements.
-
-If the egress IP is not within the subnet of the primary network interface, then the selection of another network interface for egress traffic might occur if they are present on a node.
-
-You can determine which other network interfaces might support egress IPs by inspecting the `k8s.ovn.org/host-cidrs` Kubernetes node annotation. This annotation contains the addresses and subnet mask found for the primary network interface. It also contains additional network interface addresses and subnet mask information. These addresses and subnet masks are assigned to network interfaces that use the link:https://networklessons.com/cisco/ccna-200-301/longest-prefix-match-routing[longest prefix match routing] mechanism to determine which network interface supports the egress IP.
-
-[NOTE]
-====
-OVN-Kubernetes provides a mechanism to control and direct outbound network traffic from specific namespaces and pods. This ensures that it exits the cluster through a particular network interface and with a specific egress IP address.
-====
-
-
-[id="nw-egress-ips-multi-nic-requirements_{context}"]
-=== Requirements for assigning an egress IP to a network interface that is not the primary network interface
-
-For users who want an egress IP and traffic to be routed over a particular interface that is not the primary network interface, the following conditions must be met:
-
-* {product-title} is installed on a bare-metal cluster. This feature is disabled within a cloud or a hypervisor environment.
-
-* Your {product-title} pods are not configured as _host-networked_.
-
-* If a network interface is removed or if the IP address and subnet mask which allows the egress IP to be hosted on the interface is removed, the egress IP is reconfigured. Consequently, the egress IP could be assigned to another node and interface.
-
-* If you use an Egress IP address on a secondary network interface card (NIC), you must use the Node Tuning Operator to enable IP forwarding on the secondary NIC.
-endif::ovn[]
 endif::openshift-rosa[]
 
-ifdef::ovn[]
-[id="nw-egress-ips-considerations_{context}"]
-== Assignment of egress IPs to pods
-
-To assign one or more egress IPs to a namespace or specific pods in a namespace, the following conditions must be satisfied:
-
-- At least one node in your cluster must have the `k8s.ovn.org/egress-assignable: ""` label.
-- An `EgressIP` object exists that defines one or more egress IP addresses to use as the source IP address for traffic leaving the cluster from pods in a namespace.
-
-[IMPORTANT]
-====
-If you create `EgressIP` objects prior to labeling any nodes in your cluster for egress IP assignment, {product-title} might assign every egress IP address to the first node with the `k8s.ovn.org/egress-assignable: ""` label.
-
-To ensure that egress IP addresses are widely distributed across nodes in the cluster, always apply the label to the nodes you intent to host the egress IP addresses before creating any `EgressIP` objects.
-====
-
-[id="nw-egress-ips-node-assignment_{context}"]
-== Assignment of egress IPs to nodes
-
-When creating an `EgressIP` object, the following conditions apply to nodes that are labeled with the `k8s.ovn.org/egress-assignable: ""` label:
-
-- An egress IP address is never assigned to more than one node at a time.
-- An egress IP address is equally balanced between available nodes that can host the egress IP address.
-- If the `spec.EgressIPs` array in an `EgressIP` object specifies more than one IP address, the following conditions apply:
-* No node will ever host more than one of the specified IP addresses.
-* Traffic is balanced roughly equally between the specified IP addresses for a given namespace.
-- If a node becomes unavailable, any egress IP addresses assigned to it are automatically reassigned, subject to the previously described conditions.
-
-When a pod matches the selector for multiple `EgressIP` objects, there is no guarantee which of the egress IP addresses that are specified in the `EgressIP` objects is assigned as the egress IP address for the pod.
-
-Additionally, if an `EgressIP` object specifies multiple egress IP addresses, there is no guarantee which of the egress IP addresses might be used. For example, if a pod matches a selector for an `EgressIP` object with two egress IP addresses, `10.10.20.1` and `10.10.20.2`, either might be used for each TCP connection or UDP conversation.
-
 [id="nw-egress-ips-node-architecture_{context}"]
 == Architectural diagram of an egress IP address configuration
 
@@ -246,12 +169,7 @@ Both Node 1 and Node 3 are labeled with `k8s.ovn.org/egress-assignable: ""` and
 
 The dashed lines in the diagram depict the traffic flow from pod1, pod2, and pod3 traveling through the pod network to egress the cluster from Node 1 and Node 3. When an external service receives traffic from any of the pods selected by the example `EgressIP` object, the source IP address is either `192.168.126.10` or `192.168.126.102`. The traffic is balanced roughly equally between these two nodes.
 
-The following resources from the diagram are illustrated in detail:
-
-`Namespace` objects::
-+
---
-The namespaces are defined in the following manifest:
+Based on the diagram, the following manifest file defines namespaces:
 
 .Namespace objects
 [source,yaml]
@@ -270,12 +188,8 @@ metadata:
   labels:
     env: prod
 ----
---
 
-`EgressIP` object::
-+
---
-The following `EgressIP` object describes a configuration that selects all pods in any namespace with the `env` label set to `prod`. The egress IP addresses for the selected pods are `192.168.126.10` and `192.168.126.102`.
+Based on the diagram, the following `EgressIP` object describes a configuration that selects all pods in any namespace with the `env` label set to `prod`. The egress IP addresses for the selected pods are `192.168.126.10` and `192.168.126.102`.
 
 .`EgressIP` object
 [source,yaml]
@@ -300,9 +214,3 @@ status:
 ----
 
 For the configuration in the previous example, {product-title} assigns both egress IP addresses to the available nodes. The `status` field reflects whether and where the egress IP addresses are assigned.
---
-endif::ovn[]
-
-ifdef::ovn[]
-:!ovn:
-endif::ovn[]

modules/nw-egress-ips-assign.adoc

@@ -1,6 +1,6 @@
 // Module included in the following assemblies:
 //
-// * networking/ovn_kubernetes_network_provider/assigning-egress-ips-ovn.adoc
+// * networking/ovn_kubernetes_network_provider/configuring-egress-ips-ovn.adoc
 
 :_mod-docs-content-type: PROCEDURE
 [id="nw-egress-ips-assign_{context}"]
@@ -16,8 +16,10 @@ You can assign one or more egress IP addresses to a namespace or to specific pod
 
 .Procedure
 
-. Create an `EgressIP` object:
+. Create an `EgressIP` object.
++
 .. Create a `<egressips_name>.yaml` file where `<egressips_name>` is the name of the object.
++
 .. In the file that you created, define an `EgressIP` object, as in the following example:
 +
 [source,yaml]
@@ -39,9 +41,14 @@ spec:
 +
 [source,terminal]
 ----
-$ oc apply -f <egressips_name>.yaml <1>
+$ oc apply -f <egressips_name>.yaml
 ----
-<1> Replace `<egressips_name>` with the name of the object.
++
+--
+where:
+
+`<egressips_name>`:: Replace `<egressips_name>` with the name of the object.
+--
 +
 .Example output
 [source,terminal]
@@ -55,13 +62,18 @@ egressips.k8s.ovn.org/<egressips_name> created
 +
 [source,terminal]
 ----
-$ oc label ns <namespace> env=qa <1>
+$ oc label ns <namespace> env=qa
 ----
-<1> Replace `<namespace>` with the namespace that requires egress IP addresses.
++
+--
+where:
+
+`<namespace>`:: Replace `<namespace>` with the namespace that requires egress IP addresses.
+--
 
 .Verification
 
-* To show all egress IPs that are in use in your cluster, enter the following command:
+* To show all egress IP addresses that are in use in your cluster, enter the following command:
 +
 [source,terminal]
 ----

modules/nw-egress-ips-config-object.adoc

@@ -1,7 +1,8 @@
 // Module included in the following assemblies:
 //
-// * networking/ovn_kubernetes_network_provider/assigning-egress-ips-ovn.adoc
+// * networking/ovn_kubernetes_network_provider/configuring-egress-ips-ovn.adoc
 
+:_mod-docs-content-type: CONCEPT
 [id="nw-egress-ips-config-object_{context}"]
 = The egressIPConfig object
 
@@ -25,11 +26,17 @@ spec:
     hostPrefix: 23
   defaultNetwork:
     ovnKubernetesConfig:
-      egressIPConfig: <1>
-        reachabilityTotalTimeoutSeconds: 5 <2>
+      egressIPConfig:
+        reachabilityTotalTimeoutSeconds: 5
       gatewayConfig:
         routingViaHost: false
       genevePort: 6081
 ----
-<1> The `egressIPConfig` holds the configurations for the options of the `EgressIP` object. By changing these configurations, you can extend the `EgressIP` object.
-<2> The value for `reachabilityTotalTimeoutSeconds` accepts integer values from `0` to `60`. A value of `0` disables the reachability check of the egressIP node. Setting a value from `1` to `60` corresponds to the timeout in seconds for a probe to send the reachability check to the node.
+
+--
+where:
+
+`<egressIPConfig>`:: The `egressIPConfig` holds the configurations for the options of the `EgressIP` object. By changing these configurations, you can extend the `EgressIP` object.
+
+`<reachabilityTotalTimeoutSeconds>`:: The value for `reachabilityTotalTimeoutSeconds` accepts integer values from `0` to `60`. A value of `0` disables the reachability check of the egressIP node. Setting a value from `1` to `60` corresponds to the timeout in seconds for a probe to send the reachability check to the node.
+--

modules/nw-egress-ips-considerations.adoc

@@ -0,0 +1,32 @@
+// Module included in the following assemblies:
+//
+// * networking/ovn_kubernetes_network_provider/configuring-egress-ips-ovn.adoc
+
+:_mod-docs-content-type: REFERENCE
+[id="nw-egress-ips-considerations_{context}"]
+= Assignment of egress IPs to a namespace, nodes, and pods
+
+To assign one or more egress IPs to a namespace or specific pods in a namespace, the following conditions must be satisfied:
+
+- At least one node in your cluster must have the `k8s.ovn.org/egress-assignable: ""` label.
+- An `EgressIP` object exists that defines one or more egress IP addresses to use as the source IP address for traffic leaving the cluster from pods in a namespace.
+
+[IMPORTANT]
+====
+If you create `EgressIP` objects prior to labeling any nodes in your cluster for egress IP assignment, {product-title} might assign every egress IP address to the first node with the `k8s.ovn.org/egress-assignable: ""` label.
+
+To ensure that egress IP addresses are widely distributed across nodes in the cluster, always apply the label to the nodes you intent to host the egress IP addresses before creating any `EgressIP` objects.
+====
+
+When creating an `EgressIP` object, the following conditions apply to nodes that are labeled with the `k8s.ovn.org/egress-assignable: ""` label:
+
+- An egress IP address is never assigned to more than one node at a time.
+- An egress IP address is equally balanced between available nodes that can host the egress IP address.
+- If the `spec.EgressIPs` array in an `EgressIP` object specifies more than one IP address, the following conditions apply:
+* No node will ever host more than one of the specified IP addresses.
+* Traffic is balanced roughly equally between the specified IP addresses for a given namespace.
+- If a node becomes unavailable, any egress IP addresses assigned to it are automatically reassigned, subject to the previously described conditions.
+
+When a pod matches the selector for multiple `EgressIP` objects, there is no guarantee which of the egress IP addresses that are specified in the `EgressIP` objects is assigned as the egress IP address for the pod.
+
+Additionally, if an `EgressIP` object specifies multiple egress IP addresses, there is no guarantee which of the egress IP addresses might be used. For example, if a pod matches a selector for an `EgressIP` object with two egress IP addresses, `10.10.20.1` and `10.10.20.2`, either might be used for each TCP connection or UDP conversation.