Skip to content

Commit cea652e

Browse files
pohlypohly
committed
DRA: better support for kind.yaml changes
A recent change to test/e2e/dra/kind.yaml did not start the DRA jobs, so it didn't get noticed that the change broke the jobs. We need to include YAML in the set of files which may trigger jobs. The kind.yaml change was reverted for now, but will come eventually. The canary jobs get updated to support that change without breaking current master.
1 parent 1e33308 commit cea652e

File tree

4 files changed

+92
-84
lines changed

4 files changed

+92
-84
lines changed

‎config/jobs/kubernetes/sig-node/dra-canary.yaml‎

Lines changed: 60 additions & 72 deletions
Original file line numberDiff line numberDiff line change
@@ -52,19 +52,17 @@ presubmits:
5252
5353
# Additional features are not in kind.yaml, but they can be added at the end.
5454
for feature in ${features[@]}; do echo " ${feature}: true"; done
55-
56-
# Append ClusterConfiguration which causes etcd to use /tmp
57-
# (https://github.com/kubernetes-sigs/kind/issues/845#issuecomment-1261248420).
58-
# There's no kubeadmConfigPatches in any kind.yaml, so we can append at the end.
59-
cat <<EOF
60-
kubeadmConfigPatches:
61-
- |
62-
kind: ClusterConfiguration
63-
etcd:
64-
local:
65-
dataDir: /tmp/etcd
66-
EOF
6755
) >/tmp/kind.yaml
56+
57+
# Add or extend kubeadmConfigPatches with a ClusterConfiguration which causes etcd to use /tmp
58+
# (https://github.com/kubernetes-sigs/kind/issues/845#issuecomment-1261248420).
59+
# kind.yaml may or may not have a `kubeadmConfigPatches`, so we have to be
60+
# careful.
61+
if ! grep -q '^kubeadmConfigPatches:$' /tmp/kind.yaml; then
62+
echo "kubeadmConfigPatches:" >>/tmp/kind.yaml
63+
fi
64+
sed -i -e '/^kubeadmConfigPatches:$/a\' -e '- |\n kind: ClusterConfiguration\n etcd:\n local:\n dataDir: /tmp/etcd\n' /tmp/kind.yaml
65+
6866
cat /tmp/kind.yaml
6967
kind create cluster --retain --config /tmp/kind.yaml --image dra/node:latest
7068
atexit () {
@@ -138,19 +136,17 @@ presubmits:
138136
139137
# Additional features are not in kind.yaml, but they can be added at the end.
140138
for feature in ${features[@]}; do echo " ${feature}: true"; done
141-
142-
# Append ClusterConfiguration which causes etcd to use /tmp
143-
# (https://github.com/kubernetes-sigs/kind/issues/845#issuecomment-1261248420).
144-
# There's no kubeadmConfigPatches in any kind.yaml, so we can append at the end.
145-
cat <<EOF
146-
kubeadmConfigPatches:
147-
- |
148-
kind: ClusterConfiguration
149-
etcd:
150-
local:
151-
dataDir: /tmp/etcd
152-
EOF
153139
) >/tmp/kind.yaml
140+
141+
# Add or extend kubeadmConfigPatches with a ClusterConfiguration which causes etcd to use /tmp
142+
# (https://github.com/kubernetes-sigs/kind/issues/845#issuecomment-1261248420).
143+
# kind.yaml may or may not have a `kubeadmConfigPatches`, so we have to be
144+
# careful.
145+
if ! grep -q '^kubeadmConfigPatches:$' /tmp/kind.yaml; then
146+
echo "kubeadmConfigPatches:" >>/tmp/kind.yaml
147+
fi
148+
sed -i -e '/^kubeadmConfigPatches:$/a\' -e '- |\n kind: ClusterConfiguration\n etcd:\n local:\n dataDir: /tmp/etcd\n' /tmp/kind.yaml
149+
154150
cat /tmp/kind.yaml
155151
kind create cluster --retain --config /tmp/kind.yaml --image dra/node:latest
156152
atexit () {
@@ -224,19 +220,17 @@ presubmits:
224220
225221
# Additional features are not in kind.yaml, but they can be added at the end.
226222
for feature in ${features[@]}; do echo " ${feature}: true"; done
227-
228-
# Append ClusterConfiguration which causes etcd to use /tmp
229-
# (https://github.com/kubernetes-sigs/kind/issues/845#issuecomment-1261248420).
230-
# There's no kubeadmConfigPatches in any kind.yaml, so we can append at the end.
231-
cat <<EOF
232-
kubeadmConfigPatches:
233-
- |
234-
kind: ClusterConfiguration
235-
etcd:
236-
local:
237-
dataDir: /tmp/etcd
238-
EOF
239223
) >/tmp/kind.yaml
224+
225+
# Add or extend kubeadmConfigPatches with a ClusterConfiguration which causes etcd to use /tmp
226+
# (https://github.com/kubernetes-sigs/kind/issues/845#issuecomment-1261248420).
227+
# kind.yaml may or may not have a `kubeadmConfigPatches`, so we have to be
228+
# careful.
229+
if ! grep -q '^kubeadmConfigPatches:$' /tmp/kind.yaml; then
230+
echo "kubeadmConfigPatches:" >>/tmp/kind.yaml
231+
fi
232+
sed -i -e '/^kubeadmConfigPatches:$/a\' -e '- |\n kind: ClusterConfiguration\n etcd:\n local:\n dataDir: /tmp/etcd\n' /tmp/kind.yaml
233+
240234
cat /tmp/kind.yaml
241235
kind create cluster --retain --config /tmp/kind.yaml --image dra/node:latest
242236
atexit () {
@@ -308,19 +302,17 @@ presubmits:
308302
309303
# Additional features are not in kind.yaml, but they can be added at the end.
310304
for feature in ${features[@]}; do echo " ${feature}: true"; done
311-
312-
# Append ClusterConfiguration which causes etcd to use /tmp
313-
# (https://github.com/kubernetes-sigs/kind/issues/845#issuecomment-1261248420).
314-
# There's no kubeadmConfigPatches in any kind.yaml, so we can append at the end.
315-
cat <<EOF
316-
kubeadmConfigPatches:
317-
- |
318-
kind: ClusterConfiguration
319-
etcd:
320-
local:
321-
dataDir: /tmp/etcd
322-
EOF
323305
) >/tmp/kind.yaml
306+
307+
# Add or extend kubeadmConfigPatches with a ClusterConfiguration which causes etcd to use /tmp
308+
# (https://github.com/kubernetes-sigs/kind/issues/845#issuecomment-1261248420).
309+
# kind.yaml may or may not have a `kubeadmConfigPatches`, so we have to be
310+
# careful.
311+
if ! grep -q '^kubeadmConfigPatches:$' /tmp/kind.yaml; then
312+
echo "kubeadmConfigPatches:" >>/tmp/kind.yaml
313+
fi
314+
sed -i -e '/^kubeadmConfigPatches:$/a\' -e '- |\n kind: ClusterConfiguration\n etcd:\n local:\n dataDir: /tmp/etcd\n' /tmp/kind.yaml
315+
324316
cat /tmp/kind.yaml
325317
kind create cluster --retain --config /tmp/kind.yaml --image dra/node:latest
326318
atexit () {
@@ -437,19 +429,17 @@ presubmits:
437429
438430
# Additional features are not in kind.yaml, but they can be added at the end.
439431
for feature in ${features[@]}; do echo " ${feature}: true"; done
440-
441-
# Append ClusterConfiguration which causes etcd to use /tmp
442-
# (https://github.com/kubernetes-sigs/kind/issues/845#issuecomment-1261248420).
443-
# There's no kubeadmConfigPatches in any kind.yaml, so we can append at the end.
444-
cat <<EOF
445-
kubeadmConfigPatches:
446-
- |
447-
kind: ClusterConfiguration
448-
etcd:
449-
local:
450-
dataDir: /tmp/etcd
451-
EOF
452432
) >/tmp/kind.yaml
433+
434+
# Add or extend kubeadmConfigPatches with a ClusterConfiguration which causes etcd to use /tmp
435+
# (https://github.com/kubernetes-sigs/kind/issues/845#issuecomment-1261248420).
436+
# kind.yaml may or may not have a `kubeadmConfigPatches`, so we have to be
437+
# careful.
438+
if ! grep -q '^kubeadmConfigPatches:$' /tmp/kind.yaml; then
439+
echo "kubeadmConfigPatches:" >>/tmp/kind.yaml
440+
fi
441+
sed -i -e '/^kubeadmConfigPatches:$/a\' -e '- |\n kind: ClusterConfiguration\n etcd:\n local:\n dataDir: /tmp/etcd\n' /tmp/kind.yaml
442+
453443
cat /tmp/kind.yaml
454444
kind create cluster --retain --config /tmp/kind.yaml --image dra/node:latest
455445
atexit () {
@@ -566,19 +556,17 @@ presubmits:
566556
567557
# Additional features are not in kind.yaml, but they can be added at the end.
568558
for feature in ${features[@]}; do echo " ${feature}: true"; done
569-
570-
# Append ClusterConfiguration which causes etcd to use /tmp
571-
# (https://github.com/kubernetes-sigs/kind/issues/845#issuecomment-1261248420).
572-
# There's no kubeadmConfigPatches in any kind.yaml, so we can append at the end.
573-
cat <<EOF
574-
kubeadmConfigPatches:
575-
- |
576-
kind: ClusterConfiguration
577-
etcd:
578-
local:
579-
dataDir: /tmp/etcd
580-
EOF
581559
) >/tmp/kind.yaml
560+
561+
# Add or extend kubeadmConfigPatches with a ClusterConfiguration which causes etcd to use /tmp
562+
# (https://github.com/kubernetes-sigs/kind/issues/845#issuecomment-1261248420).
563+
# kind.yaml may or may not have a `kubeadmConfigPatches`, so we have to be
564+
# careful.
565+
if ! grep -q '^kubeadmConfigPatches:$' /tmp/kind.yaml; then
566+
echo "kubeadmConfigPatches:" >>/tmp/kind.yaml
567+
fi
568+
sed -i -e '/^kubeadmConfigPatches:$/a\' -e '- |\n kind: ClusterConfiguration\n etcd:\n local:\n dataDir: /tmp/etcd\n' /tmp/kind.yaml
569+
582570
cat /tmp/kind.yaml
583571
kind create cluster --retain --config /tmp/kind.yaml --image dra/node:latest
584572
atexit () {

‎config/jobs/kubernetes/sig-node/dra-presubmit.yaml‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ presubmits:
88
skip_branches:
99
- release-\d+\.\d+ # per-release image
1010
always_run: false
11-
run_if_changed: /(dra|dynamicresources|resourceclaim|deviceclass|resourceslice|resourceclaimtemplate|dynamic-resource-allocation|pkg/apis/resource|api/resource)/.*.go
11+
run_if_changed: /(dra|dynamicresources|resourceclaim|deviceclass|resourceslice|resourceclaimtemplate|dynamic-resource-allocation|pkg/apis/resource|api/resource)/.*\.(go|yaml)
1212
optional: true
1313
labels:
1414
preset-service-account: "true"
@@ -94,7 +94,7 @@ presubmits:
9494
skip_branches:
9595
- release-\d+\.\d+ # per-release image
9696
always_run: false
97-
run_if_changed: /(dra|dynamicresources|resourceclaim|deviceclass|resourceslice|resourceclaimtemplate|dynamic-resource-allocation|pkg/apis/resource|api/resource)/.*.go
97+
run_if_changed: /(dra|dynamicresources|resourceclaim|deviceclass|resourceslice|resourceclaimtemplate|dynamic-resource-allocation|pkg/apis/resource|api/resource)/.*\.(go|yaml)
9898
optional: true
9999
labels:
100100
preset-service-account: "true"
@@ -269,7 +269,7 @@ presubmits:
269269
skip_branches:
270270
- release-\d+\.\d+ # per-release image
271271
always_run: false
272-
run_if_changed: /(dra|dynamicresources|resourceclaim|deviceclass|resourceslice|resourceclaimtemplate|dynamic-resource-allocation|pkg/apis/resource|api/resource)/.*.go
272+
run_if_changed: /(dra|dynamicresources|resourceclaim|deviceclass|resourceslice|resourceclaimtemplate|dynamic-resource-allocation|pkg/apis/resource|api/resource)/.*\.(go|yaml)
273273
optional: true
274274
labels:
275275
preset-service-account: "true"
@@ -400,7 +400,7 @@ presubmits:
400400
skip_branches:
401401
- release-\d+\.\d+ # per-release image
402402
always_run: false
403-
run_if_changed: /(dra|dynamicresources|resourceclaim|deviceclass|resourceslice|resourceclaimtemplate|dynamic-resource-allocation|pkg/apis/resource|api/resource)/.*.go
403+
run_if_changed: /(dra|dynamicresources|resourceclaim|deviceclass|resourceslice|resourceclaimtemplate|dynamic-resource-allocation|pkg/apis/resource|api/resource)/.*\.(go|yaml)
404404
optional: true
405405
labels:
406406
preset-service-account: "true"
@@ -531,7 +531,7 @@ presubmits:
531531
skip_branches:
532532
- release-\d+\.\d+ # per-release image
533533
always_run: false
534-
run_if_changed: /(dra|dynamicresources|resourceclaim|deviceclass|resourceslice|resourceclaimtemplate|dynamic-resource-allocation|pkg/apis/resource|api/resource)/.*.go
534+
run_if_changed: /(dra|dynamicresources|resourceclaim|deviceclass|resourceslice|resourceclaimtemplate|dynamic-resource-allocation|pkg/apis/resource|api/resource)/.*\.(go|yaml)
535535
optional: true
536536
labels:
537537
preset-service-account: "true"
@@ -662,7 +662,7 @@ presubmits:
662662
skip_branches:
663663
- release-\d+\.\d+ # per-release image
664664
always_run: false
665-
run_if_changed: /(dra|dynamicresources|resourceclaim|deviceclass|resourceslice|resourceclaimtemplate|dynamic-resource-allocation|pkg/apis/resource|api/resource)/.*.go
665+
run_if_changed: /(dra|dynamicresources|resourceclaim|deviceclass|resourceslice|resourceclaimtemplate|dynamic-resource-allocation|pkg/apis/resource|api/resource)/.*\.(go|yaml)
666666
optional: true
667667
labels:
668668
preset-service-account: "true"

‎config/jobs/kubernetes/sig-node/dra.generate.conf‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@ description = Runs E2E tests for Dynamic Resource Allocation beta features again
3333
job_type = e2e
3434
use_dind = true
3535
cluster = eks-prow-build-cluster
36-
run_if_changed = /(dra|dynamicresources|resourceclaim|deviceclass|resourceslice|resourceclaimtemplate|dynamic-resource-allocation|pkg/apis/resource|api/resource)/.*.go
36+
run_if_changed = /(dra|dynamicresources|resourceclaim|deviceclass|resourceslice|resourceclaimtemplate|dynamic-resource-allocation|pkg/apis/resource|api/resource)/.*\.(go|yaml)
3737
release_informing = true
3838

3939
# This jobs runs e2e.test with a focus on tests for the Dynamic Resource Allocation feature (currently alpha, soon beta)
@@ -46,7 +46,7 @@ job_type = e2e
4646
cluster = eks-prow-build-cluster
4747
all_features = true
4848
use_dind = true
49-
run_if_changed = /(dra|dynamicresources|resourceclaim|deviceclass|resourceslice|resourceclaimtemplate|dynamic-resource-allocation|pkg/apis/resource|api/resource)/.*.go
49+
run_if_changed = /(dra|dynamicresources|resourceclaim|deviceclass|resourceslice|resourceclaimtemplate|dynamic-resource-allocation|pkg/apis/resource|api/resource)/.*\.(go|yaml)
5050

5151
# Compared to ci-kind-dra-all, this one also enables slow tests.
5252
# Needs to be triggered manually.
@@ -67,7 +67,7 @@ description = Runs E2E tests for Dynamic Resource Allocation beta features again
6767
job_type = e2e
6868
use_dind = true
6969
cluster = eks-prow-build-cluster
70-
run_if_changed = /(dra|dynamicresources|resourceclaim|deviceclass|resourceslice|resourceclaimtemplate|dynamic-resource-allocation|pkg/apis/resource|api/resource)/.*.go
70+
run_if_changed = /(dra|dynamicresources|resourceclaim|deviceclass|resourceslice|resourceclaimtemplate|dynamic-resource-allocation|pkg/apis/resource|api/resource)/.*\.(go|yaml)
7171
kubelet_skew = 1
7272

7373
# This job runs the current e2e.test against a cluster where the kubelet is from the "current - 2" release.
@@ -78,7 +78,7 @@ description = Runs E2E tests for Dynamic Resource Allocation beta features again
7878
job_type = e2e
7979
use_dind = true
8080
cluster = eks-prow-build-cluster
81-
run_if_changed = /(dra|dynamicresources|resourceclaim|deviceclass|resourceslice|resourceclaimtemplate|dynamic-resource-allocation|pkg/apis/resource|api/resource)/.*.go
81+
run_if_changed = /(dra|dynamicresources|resourceclaim|deviceclass|resourceslice|resourceclaimtemplate|dynamic-resource-allocation|pkg/apis/resource|api/resource)/.*\.(go|yaml)
8282
kubelet_skew = 2
8383

8484
# This job runs the current e2e.test against a cluster where the kubelet is from the "current - 3" release.
@@ -89,7 +89,7 @@ description = Runs E2E tests for Dynamic Resource Allocation beta features again
8989
job_type = e2e
9090
use_dind = true
9191
cluster = eks-prow-build-cluster
92-
run_if_changed = /(dra|dynamicresources|resourceclaim|deviceclass|resourceslice|resourceclaimtemplate|dynamic-resource-allocation|pkg/apis/resource|api/resource)/.*.go
92+
run_if_changed = /(dra|dynamicresources|resourceclaim|deviceclass|resourceslice|resourceclaimtemplate|dynamic-resource-allocation|pkg/apis/resource|api/resource)/.*\.(go|yaml)
9393
kubelet_skew = 3
9494

9595
# This executes tests in test/e2e_dra with special requirements (local-up-cluster.sh!).
@@ -102,7 +102,7 @@ use_dind_cdi = true
102102
job_type = integration
103103
need_kubernetes_repo = true
104104
cluster = eks-prow-build-cluster
105-
run_if_changed = /(dra|dynamicresources|resourceclaim|deviceclass|resourceslice|resourceclaimtemplate|dynamic-resource-allocation|pkg/apis/resource|api/resource)/.*.go
105+
run_if_changed = /(dra|dynamicresources|resourceclaim|deviceclass|resourceslice|resourceclaimtemplate|dynamic-resource-allocation|pkg/apis/resource|api/resource)/.*\.(go|yaml)
106106

107107
# This job runs the node e2e tests for DRA with CRI-O
108108
[node-e2e-crio-dra]

‎config/jobs/kubernetes/sig-node/dra.jinja‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -172,6 +172,25 @@ presubmits:
172172
GINKGO_E2E_PID=
173173
trap 'if [ "${GINKGO_E2E_PID}" ]; then kill -TERM "${GINKGO_E2E_PID}"; fi' TERM
174174
trap 'if [ "${GINKGO_E2E_PID}" ]; then kill -INT "${GINKGO_E2E_PID}"; fi' INT
175+
{%- if canary %}
176+
# The final kind.yaml the result of getting the original kind.yaml, manipulating it with sed,
177+
# and adding something at the end.
178+
(
179+
${kind_yaml_cmd[@]}
180+
181+
# Additional features are not in kind.yaml, but they can be added at the end.
182+
for feature in ${features[@]}; do echo " ${feature}: true"; done
183+
) >/tmp/kind.yaml
184+
185+
# Add or extend kubeadmConfigPatches with a ClusterConfiguration which causes etcd to use /tmp
186+
# (https://github.com/kubernetes-sigs/kind/issues/845#issuecomment-1261248420).
187+
# kind.yaml may or may not have a `kubeadmConfigPatches`, so we have to be
188+
# careful.
189+
if ! grep -q '^kubeadmConfigPatches:$' /tmp/kind.yaml; then
190+
echo "kubeadmConfigPatches:" >>/tmp/kind.yaml
191+
fi
192+
sed -i -e '/^kubeadmConfigPatches:$/a\' -e '- |\n kind: ClusterConfiguration\n etcd:\n local:\n dataDir: /tmp/etcd\n' /tmp/kind.yaml
193+
{% else %}
175194
# The final kind.yaml the result of getting the original kind.yaml, manipulating it with sed,
176195
# and adding something at the end.
177196
(
@@ -192,6 +211,7 @@ presubmits:
192211
dataDir: /tmp/etcd
193212
EOF
194213
) >/tmp/kind.yaml
214+
{%- endif %}
195215
cat /tmp/kind.yaml
196216
kind create cluster --retain --config /tmp/kind.yaml --image dra/node:latest
197217
atexit () {

0 commit comments

Comments
 (0)