Merge pull request #99992 from dfitzmau/OSDOCS-16309

OSDOCS-16309: Updated the IPoIB docs by including Bond Over Infiniban…

Author: dfitzmau

modules/virt-creating-infiniband-interface-on-nodes.adoc

@@ -6,7 +6,9 @@
 [id="virt-creating-infiniband-interface-on-nodes_{context}"]
 = Creating an IP over InfiniBand interface on nodes
 
-On the {product-title} web console, you can install a Red{nbsp}Hat certified third-party Operator, such as the NVIDIA Network Operator, that supports InfiniBand (IPoIB) mode. Typically, you would use the third-party Operator with other vendor infrastructure to manage resources in an {product-title} cluster. To create an IPoIB interface on nodes in your cluster, you must define an InfiniBand (IPoIB) interface in a `NodeNetworkConfigurationPolicy` (NNCP) manifest file. 
+On the {product-title} web console, you can install a Red{nbsp}Hat certified third-party Operator, such as the NVIDIA Network Operator, that supports IP over InfiniBand (IPoIB) mode. Typically, you would use the third-party Operator with other vendor infrastructure to manage resources in an {product-title} cluster. To create an IPoIB interface on nodes in your cluster, you must define an InfiniBand (IPoIB) interface in a `NodeNetworkConfigurationPolicy` (NNCP) manifest file.
+
+If you need to attach IPoIB to a bond interface, only the `active-backup` mode supports this configuration.
 
 [IMPORTANT]
 ====
@@ -24,7 +26,6 @@ For more information about the NVIDIA Operator, see link:https://docs.nvidia.com
 
 . Create or edit a `NodeNetworkConfigurationPolicy` (NNCP) manifest file, and then specify an IPoIB interface in the file.
 +
-
 [source,yaml]
 ----
 apiVersion: nmstate.io/v1
@@ -36,8 +37,8 @@ spec:
     interfaces:
     - description: ""
       infiniband:
-        mode: datagram <1>
-        pkey: "0xffff" <2>
+        mode: datagram
+        pkey: "0xffff"
       ipv4:
         address:
         - ip: 100.125.3.4
@@ -48,21 +49,35 @@ spec:
         enabled: false
       name: ibp27s0
       state: up
-      identifier: mac-address <3>
-      mac-address: 20:00:55:04:01:FE:80:00:00:00:00:00:00:00:02:C9:02:00:23:13:92 <4>
-      type: infiniband <5>
+      identifier: mac-address
+      mac-address: 20:00:55:04:01:FE:80:00:00:00:00:00:00:00:02:C9:02:00:23:13:92
+      type: infiniband
 # ...
 ----
-<1> `datagram` is the default mode for an IPoIB interface, and this mode improves optimizes performance and latency. `connected` mode is a supported mode but consider only using this mode when you need to adjust the maximum transmission unit (MTU) value to improve node connectivity with surrounding network devices.  
-<2> Supports a string or an integer value. The parameter defines the protection key, or _P-key_, for the interface for the purposes of authentication and encrypted communications with a third-party vendor, such as NVIDIA. Values `None` and `0xffff` indicate the protection key for the base interface in an InfiniBand system.
-<3> Supported values include `name`, the default value, and `mac-address`. The `name` value applies a configuration to an interface that holds a specified interface name. 
-<4> Holds the MAC address of an interface. For an IP-over-InfiniBand (IPoIB) interface, the address is a 20-byte string. 
-<5> Sets the type of interface to `infiniband`.
++
+--
+where:
+
+`<mode>`:: `datagram` is the default mode for an IPoIB interface. This mode provides improved CPU performance and low-latency capabilitities for pod-to-pod communication. `connected` mode is a supported mode but consider only using this mode when you need to adjust the maximum transmission unit (MTU) value to improve node connectivity with surrounding network devices.
+
+`<pkey>`:: Supports a string or an integer value. The parameter defines the protection key, or _P-key_, for the interface for the purposes of authentication and encrypted communications with a third-party vendor, such as NVIDIA. Values `None` and `0xffff` indicate the protection key for the base interface in an InfiniBand system.
+
+`<identifier>`:: Supported values include `name`, the default value, and `mac-address`. The `name` value applies a configuration to an interface that holds a specified interface name. 
+
+`<mac-address>`:: Holds the MAC address of an interface. For an IP-over-InfiniBand (IPoIB) interface, the address is a 20-byte string. 
+
+`<type>`:: Sets the type of interface to `infiniband`.
+--
 
 . Apply the NNCP configuration to each node in your cluster by running the following command. The Kubernetes NMState Operator can then create an IPoIB interface on each node. 
 +
 [source,yaml]
 ----
-$ oc apply -f <nncp_file_name> <1>
+$ oc apply -f <nncp_file_name>
 ----
-<1> Replace `<nncp_file_name>` with the name of your NNCP file.
++
+--
+where:
+
+`<nncp_file_name>`:: Replace `<nncp_file_name>` with the name of your NNCP file.
+--