Skip to content

Commit 2520c26

Browse files
lpettyjolpettyjo
authored
Merge pull request #98557 from lpettyjo/OSDOCS-15380
OSDOCS-15380# selinuxChangePolicy & fsGroupChangePolicy per namespace/pod level
2 parents a40835b + f7bcbb8 commit 2520c26

11 files changed

+326
-27
lines changed

‎modules/storage-persistent-storage-change-policy-overview.adoc‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * storage/understanding-persistent-storage.adoc
4+
//* microshift_storage/understanding-persistent-storage-microshift.adoc
5+
6+
:_mod-docs-content-type: CONCEPT
7+
[id="storage_persistent_storage_change_policy_overview_{context}"]
8+
= Reducing pod timeouts
9+
10+
A volume with many files can cause pod startup delays and timeouts. You can set certain parameters to improve this issue.
11+
12+
* *fsGroup*
13+
+
14+
Set `fsGroupChangePolicy` to `OnRootMismatch` to stop recursively changing ownership and permissions to match the fsGroup specified in a pod’s `securityContext`.
15+
16+
* *SELinux*
17+
+
18+
Set `seLinuxChangePolicy` to `MountOption` to avoid runtime recursively relabeling all files on a volume to match the pod’s SELinux context.

‎modules/storage-persistent-storage-fsGroup-namespace.adoc‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * storage/understanding-persistent-storage.adoc
4+
//* microshift_storage/understanding-persistent-storage-microshift.adoc
5+
6+
:_mod-docs-content-type: PROCEDURE
7+
[id="using_fsGroup_namespace_{context}"]
8+
= Changing fsGroup at the namespace level
9+
10+
After applying the desired setting for `fsGroupChangePolicy` at the namespace level, all subsequently created pods in that namespace inherit the setting. However, if desired, you can override the inherited `fsGroupChangePolicy` setting for individual pods. Setting `fsGroupChangePolicy` at the pod level overrides inheritance from the namespace level setting for that pod.
11+
12+
.Prerequisites
13+
14+
* Logged in to a running {product-title} cluster with administrator privileges.
15+
16+
* Access to the {product-title} console.
17+
18+
.Procedure
19+
20+
To set `fsGroupChangePolicy` per namespace:
21+
22+
. Select the desired namespace:
23+
24+
.. Click *Administration* > *Namespaces*.
25+
26+
.. On the *Namespaces* page, click the desired namespace. The *Namespace details* page appears.
27+
28+
. Add the `fsGroupChangePolicy` label to the namespace:
29+
30+
.. On the *Namespace details* page, next to *Labels*, click *Edit*.
31+
32+
.. In the *Edit labels* dialog, add the label `storage.openshift.io/fsgroup-change-policy` and set it equal to either:
33+
+
34+
* `OnRootMismatch`: Specifies only changing permissions and ownership if the permission and the ownership of root directory does not match with expected permissions of the volume, thus helping to avoid pod timeout problems.
35+
* `Always`: (Default) Specifies always changing permission and ownership of the volume when a volume is mounted.
36+
37+
.. Click *Save*.
38+
39+
.Verification
40+
Start up a pod in the previously edited namespace and observe that the parameter `spec.securityContext.fsGroupChangePolicy` contains the value that you set for the namespace.
41+
42+
.Example pod YAML file showing `fsGroupChangePolicy` setting
43+
[source,yaml]
44+
----
45+
securityContext:
46+
seLinuxOptions:
47+
level: 's0:c27,c24'
48+
runAsNonRoot: true
49+
fsGroup: 1000750000
50+
fsGroupChangePolicy: OnRootMismatch <1>
51+
...
52+
----
53+
<1> This value is inherited from the namespace.
54+

‎modules/storage-persistent-storage-fsGroup-pod.adoc‎

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
:_mod-docs-content-type: PROCEDURE
2+
// Module included in the following assemblies:
3+
//
4+
// * storage/understanding-persistent-storage.adoc
5+
//* microshift_storage/understanding-persistent-storage-microshift.adoc
6+
7+
[id="using_fsGroup_pod_{context}"]
8+
= Changing fsGroup at the pod level
9+
10+
You can set the set the `fsGroupChangePolicy` parameter in a new or existing deployment, and then the pods that it manages will have this parameter value. You can similarly do this for a statefulset. You cannot edit an existing pod to set `fsGroupChangePolicy`; however, you can set this parameter when creating a new pod.
11+
12+
This procedure describes how to set the `fsGroupChangePolicy` parameter in an existing deployment.
13+
14+
.Prerequisites
15+
16+
* Access to the {product-title} console.
17+
18+
.Procedure
19+
20+
To set the `fsGroupChangePolicy` parameter in an existing deployment:
21+
22+
. Click *Workloads* > *Deployments*.
23+
24+
. On the *Deployment* page, click the desired deployment.
25+
26+
. On the *Deployment details* page, click the *YAML* tab.
27+
28+
. Edit the deployment's YAML file under `spec.template.spec.securityContext` using the following example file:
29+
+
30+
.Example deployment YAML file setting `fsGroupChangePolicy`
31+
[source,yaml]
32+
----
33+
...
34+
spec:
35+
replicas: 3
36+
selector:
37+
matchLabels:
38+
app: my-app
39+
template:
40+
metadata:
41+
creationTimestamp: null
42+
labels:
43+
app: my-app
44+
spec:
45+
containers:
46+
- name: container
47+
image: 'image-registry.openshift-image-registry.svc:5000/openshift/httpd:latest'
48+
ports:
49+
- containerPort: 8080
50+
protocol: TCP
51+
resources: {}
52+
terminationMessagePath: /dev/termination-log
53+
terminationMessagePolicy: File
54+
imagePullPolicy: Always
55+
restartPolicy: Always
56+
terminationGracePeriodSeconds: 30
57+
dnsPolicy: ClusterFirst
58+
securityContext:
59+
fsGroupChangePolicy: OnRootMismatch <1>
60+
...
61+
----
62+
<1> `OnRootMismatch` specifies skipping recursive permission change, thus helping to avoid pod timeout problems. The default value is `Always`, which always changes permission and ownership of the volume when a volume is mounted.
63+
64+
. Click *Save*.

‎modules/storage-persistent-storage-fsGroup.adoc‎

Lines changed: 9 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -3,33 +3,23 @@
33
// * storage/understanding-persistent-storage.adoc
44
//* microshift_storage/understanding-persistent-storage-microshift.adoc
55

6+
:_mod-docs-content-type: CONCEPT
7+
[id="using_fsGroup_overview_{context}"]
8+
= Reducing pod timeouts using fsGroup
69

7-
[id="using_fsGroup_{context}"]
8-
= Using fsGroup to reduce pod timeouts
10+
If a storage volume contains many files (for example, a million or more), you might experience pod timeouts.
911

10-
If a storage volume contains many files (~1,000,000 or greater), you may experience pod timeouts.
11-
12-
This can occur because, by default, {product-title} recursively changes ownership and permissions for the contents of each volume to match the `fsGroup` specified in a pod's `securityContext` when that volume is mounted. For large volumes, checking and changing ownership and permissions can be time consuming, slowing pod startup. You can use the `fsGroupChangePolicy` field inside a `securityContext` to control the way that {product-title} checks and manages ownership and permissions for a volume.
12+
This can occur because, by default, {product-title} recursively changes ownership and permissions for the contents of each volume to match the `fsGroup` specified in a pod's `securityContext` when that volume is mounted. For volumes with many files, checking and changing ownership and permissions can be time consuming, slowing pod startup. You can use the `fsGroupChangePolicy` field inside a `securityContext` to control the way that {product-title} checks and manages ownership and permissions for a volume.
1313

1414
`fsGroupChangePolicy` defines behavior for changing ownership and permission of the volume before being exposed inside a pod. This field only applies to volume types that support `fsGroup`-controlled ownership and permissions. This field has two possible values:
1515

1616
* `OnRootMismatch`: Only change permissions and ownership if permission and ownership of root directory does not match with expected permissions of the volume. This can help shorten the time it takes to change ownership and permission of a volume to reduce pod timeouts.
1717
18-
* `Always`: Always change permission and ownership of the volume when a volume is mounted.
19-
20-
.`fsGroupChangePolicy` example
21-
[source,yaml]
22-
----
23-
securityContext:
24-
runAsUser: 1000
25-
runAsGroup: 3000
26-
fsGroup: 2000
27-
fsGroupChangePolicy: "OnRootMismatch" <1>
28-
...
29-
----
30-
<1> `OnRootMismatch` specifies skipping recursive permission change, thus helping to avoid pod timeout problems.
18+
* `Always`: (Default) Always change permission and ownership of the volume when a volume is mounted.
3119
3220
[NOTE]
3321
====
34-
The fsGroupChangePolicyfield has no effect on ephemeral volume types, such as secret, configMap, and emptydir.
22+
The `fsGroupChangePolicy` field has no effect on ephemeral volume types, such as secret, configMap, and emptydir.
3523
====
24+
25+
You can set `fsGroupChangePolicy` at either the namespace or pod level.

‎modules/storage-persistent-storage-pv.adoc‎

Lines changed: 1 addition & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -130,7 +130,7 @@ The following table lists the access modes:
130130
|ReadWriteOnce
131131
|`RWO`
132132
|The volume can be mounted as read-write by a single node.
133-
|ReadWriteOncePod ^[1]^
133+
|ReadWriteOncePod
134134
|`RWOP`
135135
|The volume can be mounted as read-write by a single pod on a single node.
136136
ifdef::openshift-enterprise,openshift-webscale,openshift-origin[]
@@ -142,13 +142,6 @@ ifdef::openshift-enterprise,openshift-webscale,openshift-origin[]
142142
|The volume can be mounted as read-write by many nodes.
143143
endif::[]
144144
|===
145-
--
146-
1. RWOP uses the SELinux mount feature. This feature is driver dependent, and enabled by default in AWS EBS
147-
ifndef::openshift-dedicated,openshift-rosa,openshift-rosa-hcp[]
148-
, Azure Disk, GCP PD, IBM Cloud Block Storage volume, Cinder, vSphere,
149-
endif::openshift-dedicated,openshift-rosa,openshift-rosa-hcp[]
150-
and {rh-storage-first}. For third-party drivers, contact your storage vendor.
151-
--
152145

153146
.Supported access modes for persistent volumes
154147
[cols=",^v,^v,^v,^v", width="100%",options="header"]

‎modules/storage-persistent-storage-selinuxChangePolicy-namespace.adoc‎

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * storage/understanding-persistent-storage.adoc
4+
//* microshift_storage/understanding-persistent-storage-microshift.adoc
5+
6+
:_mod-docs-content-type: PROCEDURE
7+
[id="using_selinuxChangePolicy_namespace_{context}"]
8+
= Changing seLinuxChangePolicy at the namespace level
9+
10+
After applying the desired setting for `seLinuxChangePolicy` at the namespace level, all subsequently created pods in that namespace inherit the setting. However, if desired, you can override the inherited `seLinuxChangePolicy` setting for individual pods. Setting `seLinuxChangePolicy` at the pod level overrides inheritance from the namespace level setting for that pod.
11+
12+
.Prerequisites
13+
14+
* Logged in to a running {product-title} cluster with administrator privileges.
15+
16+
* Access to the {product-title} console.
17+
18+
.Procedure
19+
20+
To set `SELinuxChangePolicy` per namespace:
21+
22+
. Select the desired namespace:
23+
24+
.. Click *Administration* > *Namespaces*.
25+
26+
.. On the *Namespaces* page, click the desired namespace. The *Namespace details* page appears.
27+
28+
. Add the `seLinuxChangePolicy` label to the namespace:
29+
30+
.. On the *Namespace details* page, next to *Labels*, click *Edit*.
31+
32+
.. In the *Edit labels* dialog, add the label `storage.openshift.io/selinux-change-policy=Recursive`.
33+
+
34+
This specifies recursively relabeling all files on pod volumes to the appropriate SELinux context.
35+
36+
.. Click *Save*.
37+
38+
.Verification
39+
Start up a pod in the previously edited namespace and observe that the parameter `spec.securityContext.seLinuxChangePolicy` is set to `Recursive`.
40+
41+
.Example pod YAML file showing `seLinuxChangePolicy` setting
42+
[source,yaml]
43+
----
44+
securityContext:
45+
seLinuxOptions:
46+
level: 's0:c27,c19'
47+
runAsNonRoot: true
48+
fsGroup: 1000740000
49+
seccompProfile:
50+
type: RuntimeDefault
51+
seLinuxChangePolicy: Recursive <1>
52+
...
53+
----
54+
<1> This value is inherited from the namespace.
55+

‎modules/storage-persistent-storage-selinuxChangePolicy-opt-out.adoc‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * storage/understanding-persistent-storage.adoc
4+
//* microshift_storage/understanding-persistent-storage-microshift.adoc
5+
6+
:_mod-docs-content-type: CONCEPT
7+
[id="using_selinuxChangePolicy_pod-opt-out_{context}"]
8+
= Opting out of the SELinux mount option default
9+
10+
If you want to opt out of the future move to mount option as default, you can affirmatively set the `seLinuxChangePolicy` parameter to `Recursive` at either the individual pod or namespace level.

‎modules/storage-persistent-storage-selinuxChangePolicy-pod.adoc‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * storage/understanding-persistent-storage.adoc
4+
//* microshift_storage/understanding-persistent-storage-microshift.adoc
5+
6+
:_mod-docs-content-type: PROCEDURE
7+
[id="using_selinuxChangePolicy_pod_{context}"]
8+
= Changing seLinuxChangePolicy at the pod level
9+
10+
You can set the set the `seLinuxChangePolicy` parameter in a new or existing Deployment, and then the pods that it manages will have this parameter value. You can similarly do this for a StatefulSet. You cannot edit an existing pod to set `seLinuxChangePolicy`; however, you can set this parameter when creating a new pod.
11+
12+
This procedure describes how to set the `seLinuxChangePolicy` parameter in an existing deployment.
13+
14+
.Prerequisites
15+
16+
* Access to the {product-title} console.
17+
18+
.Procedure
19+
20+
To set the `seLinuxChangePolicy` parameter in an existing deployment:
21+
22+
. Click *Workloads* > *Deployments*.
23+
24+
. On the *Deployment* page, click the desired deployment.
25+
26+
. On the *Deployment details* page, click the *YAML* tab.
27+
28+
. Edit the deployment's YAML file under `spec.template.spec.securityContext` as per the following example file:
29+
+
30+
.Example deployment YAML file setting `seLinuxChangePolicy`
31+
[source,yaml]
32+
----
33+
...
34+
securityContext:
35+
seLinuxChangePolicy: Recursive <1>
36+
...
37+
----
38+
<1> Specifies recursively relabeling all files on all pod volumes to the appropriate SELinux context.
39+
40+
. Click *Save*.

‎modules/storage-persistent-storage-selinuxChangePolicy-testing-mountoption-RWO-RWX.adoc‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * storage/understanding-persistent-storage.adoc
4+
//* microshift_storage/understanding-persistent-storage-microshift.adoc
5+
6+
:_mod-docs-content-type: PROCEDURE
7+
[id="using_selinuxChangePolicy_testing-mountoption-rwo-rwx_{context}"]
8+
= Testing the RWO and RWX and SELinux mount option feature
9+
10+
In {product-title} 4.20, you can evaluate the mount option feature for RWO and RWX volumes as a Technology Preview feature.
11+
12+
:FeatureName: RWO/RWX SELinux mount
13+
include::snippets/technology-preview.adoc[]
14+
15+
To evaluate the mount option feature:
16+
17+
* Enable Feature Gates. For information about enabling Feature Gates, see _Section Enabling features using feature gates_.
18+
+
19+
RWO and RWX volumes now have mount option as the default behavior.

‎modules/storage-persistent-storage-selinuxChangePolicy.adoc‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * storage/understanding-persistent-storage.adoc
4+
//* microshift_storage/understanding-persistent-storage-microshift.adoc
5+
6+
:_mod-docs-content-type: CONCEPT
7+
[id="using_selinuxChangePolicy_overview_{context}"]
8+
= Reducing pod timeouts using seLinuxChangePolicy
9+
10+
SELinux (Security-Enhanced Linux) is a security mechanism that assigns security labels (contexts) to all objects (files, processes, network ports, etc.) on a system. These labels determine what a process can access. In {product-title}, SELinux helps prevent containers from escaping and accessing the host system or other containers.
11+
12+
When a pod starts, the container runtime recursively relabels all files on a volume to match the pod's SELinux context. For volumes with many files, this can significantly increase pod startup times.
13+
14+
Mount option specifies avoiding recursive relabeling of all files by attempting to mount the volume with the correct SELinux label directly using the -o context mount option, thus helping to avoid pod timeout problems.
15+
16+
.RWOP and SELinux mount option
17+
18+
ReadWriteOncePod (RWOP) persistent volumes use the SELinux mount feature by default.
19+
20+
The mount option feature is driver dependent, and enabled by default in AWS EBS
21+
ifndef::openshift-dedicated,openshift-rosa,openshift-rosa-hcp[]
22+
, Azure Disk, GCP PD, {ibm-cloud-title} Block Storage volume, Cinder, vSphere,
23+
endif::openshift-dedicated,openshift-rosa,openshift-rosa-hcp[]
24+
and {rh-storage-first}. For third-party drivers, contact your storage vendor.
25+
26+
.RWO and RWX and SELinux mount option
27+
28+
ReadWriteOnce (RWO) and ReadWriteMany (RWX) volumes use recursive relabeling by default.
29+
30+
[IMPORTANT]
31+
====
32+
In a future {product-title} version, RWO and RWX volumes will use *mount option by default*.
33+
====

0 commit comments

Comments
 (0)