Skip to content

Separate conformance tests dependencies #4188

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
+365 −100
Open

Separate conformance tests dependencies #4188

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
c8f2c2d
Separate conformance tests dependencies
rikatz Oct 17, 2025
533bd2f
Provide instructions to generate conformance binary
rikatz Oct 17, 2025
300eba0
Bump libraries missed during rebase
rikatz Oct 20, 2025
48626b4
Add conformance build to makefile, bump tools mods
rikatz Oct 20, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ Session.vim
/site
go.work.sum
/tmp
/conformance.test

/cache
.venv/
Expand Down
11 changes: 10 additions & 1 deletion Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,11 @@ GO_TEST_FLAGS ?=
CEL_TEST_K8S_VERSION ?=
CEL_TEST_CRD_CHANNEL ?= standard

all: generate vet fmt verify test
# Compilation flags for binaries
GOARCH ?= $(shell go env GOARCH)
GOOS ?= $(shell go env GOOS)

all: generate vet fmt verify test conformance-bin

.PHONY: clean-generated
clean-generated:
Expand Down Expand Up @@ -103,6 +107,11 @@ test.crds-validation:
conformance:
go test ${GO_TEST_FLAGS} -v ./conformance -run TestConformance -args ${CONFORMANCE_FLAGS}

# Build a conformance.test binary that can be used as a standalone binary to run conformance test
.PHONY: conformance-bin
conformance-bin:
GOOS=$(GOOS) GOARCH=$(GOARCH) go test -c -v ./conformance

# Install CRD's and example resources to a preexisting cluster.
.PHONY: install
install: crd example
Expand Down
71 changes: 71 additions & 0 deletions conformance/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
module sigs.k8s.io/gateway-api/conformance

go 1.24.0

replace sigs.k8s.io/gateway-api => ../
Copy link
Contributor

@dprotaso dprotaso Oct 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Technically you don't need this given you have the go.work folder

and allow generating a binary to run conformance on a different machine

Also having the replace will prevent go install and go run from working.

Copy link
Member Author

@rikatz rikatz Oct 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's the case of having "go install" or "go run" here as we don't have a "main"?

I agree that replacement may be problematic in case we try to do some "go install", but "go run" works fine:

go run ./tools/gepstoc/main.go

the "tools" directory has its own go.mod already. Did I missed something?


require (
github.com/miekg/dns v1.1.68
github.com/stretchr/testify v1.11.1
golang.org/x/net v0.46.0
golang.org/x/sync v0.17.0
google.golang.org/grpc v1.76.0
google.golang.org/protobuf v1.36.10
k8s.io/api v0.34.1
k8s.io/apiextensions-apiserver v0.34.1
k8s.io/apimachinery v0.34.1
k8s.io/client-go v0.34.1
k8s.io/utils v0.0.0-20251002143259-bc988d571ff4
sigs.k8s.io/controller-runtime v0.22.3
sigs.k8s.io/gateway-api v0.0.0-00010101000000-000000000000
Copy link
Contributor

@dprotaso dprotaso Oct 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious what this pseudo version means exactly?

In order for the build to work I think we'll need to have automation to update this module version

eg.

  1. A commit makes a change to the sigs.k8s.io/gateway-api module
  2. Automation then needs to bump the version in this module file

Unsure if we want to do this as part of every release (more error prone) or every commit (then you could build the conformance binary at HEAD). One caveat is we'd want to make sure this doesn't trigger an infinite loop.

Copy link
Member Author

@rikatz rikatz Oct 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pseudo version is related to the replacement we are doing, mostly saying "we don't care, use the latest from the replacement"

Similar to Kubernetes main repo: https://github.com/kubernetes/kubernetes/blob/master/go.mod#L88

As we are not building binaries, but APIs, it makes sense that we keep the "go.mod" of the API as clean as possible, with the "cost" IMO of the inner modules that we don't provide to consumers with these kind of tweaks.

sigs.k8s.io/yaml v1.6.0
)

require (
github.com/Masterminds/semver/v3 v3.4.0 // indirect
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
github.com/emicklei/go-restful/v3 v3.13.0 // indirect
github.com/evanphx/json-patch/v5 v5.9.11 // indirect
github.com/fxamacker/cbor/v2 v2.9.0 // indirect
github.com/go-logr/logr v1.4.3 // indirect
github.com/go-logr/zapr v1.3.0 // indirect
github.com/go-openapi/jsonpointer v0.21.2 // indirect
github.com/go-openapi/jsonreference v0.21.0 // indirect
github.com/go-openapi/swag v0.23.1 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/google/gnostic-models v0.7.0 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/gorilla/websocket v1.5.4-0.20250319132907-e064f32e3674 // indirect
github.com/josharian/intern v1.0.0 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/mailru/easyjson v0.9.0 // indirect
github.com/moby/spdystream v0.5.0 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.3-0.20250322232337-35a7c28c31ee // indirect
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f // indirect
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
github.com/spf13/pflag v1.0.7 // indirect
github.com/x448/float16 v0.8.4 // indirect
go.uber.org/automaxprocs v1.6.0 // indirect
go.uber.org/multierr v1.11.0 // indirect
go.uber.org/zap v1.27.0 // indirect
go.yaml.in/yaml/v2 v2.4.2 // indirect
go.yaml.in/yaml/v3 v3.0.4 // indirect
golang.org/x/mod v0.28.0 // indirect
golang.org/x/oauth2 v0.30.0 // indirect
golang.org/x/sys v0.37.0 // indirect
golang.org/x/term v0.36.0 // indirect
golang.org/x/text v0.30.0 // indirect
golang.org/x/time v0.12.0 // indirect
golang.org/x/tools v0.37.0 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20250826171959-ef028d996bc1 // indirect
gopkg.in/evanphx/json-patch.v4 v4.13.0 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
k8s.io/klog/v2 v2.130.1 // indirect
k8s.io/kube-openapi v0.0.0-20250814151709-d7b6acb124c3 // indirect
sigs.k8s.io/json v0.0.0-20250730193827-2d320260d730 // indirect
sigs.k8s.io/randfill v1.0.0 // indirect
sigs.k8s.io/structured-merge-diff/v6 v6.3.0 // indirect
)
210 changes: 210 additions & 0 deletions conformance/go.sum
View file
Open in desktop

Large diffs are not rendered by default.

30 changes: 9 additions & 21 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,21 +3,13 @@ module sigs.k8s.io/gateway-api
go 1.24.0

require (
github.com/miekg/dns v1.1.68
github.com/stretchr/testify v1.11.1
golang.org/x/net v0.46.0
golang.org/x/sync v0.17.0
google.golang.org/grpc v1.76.0
google.golang.org/protobuf v1.36.10
k8s.io/api v0.34.1
k8s.io/apiextensions-apiserver v0.34.1
k8s.io/apimachinery v0.34.1
k8s.io/client-go v0.34.1
k8s.io/kube-openapi v0.0.0-20250814151709-d7b6acb124c3
k8s.io/utils v0.0.0-20250820121507-0af2bda4dd1d
sigs.k8s.io/controller-runtime v0.22.3
Copy link
Member

@robscott robscott Oct 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's left that still requires a controller-runtime dependency?

Copy link
Member Author

@rikatz rikatz Oct 19, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the pkg/test uses envtest, which is part of controller-runtime.

Probably as part of the VAP tests effort we can split it as well, and get rid once for all of controller-runtime as an API dependency

sigs.k8s.io/structured-merge-diff/v6 v6.3.0
sigs.k8s.io/yaml v1.6.0
)

require (
Expand All @@ -28,48 +20,44 @@ require (
github.com/evanphx/json-patch/v5 v5.9.11 // indirect
github.com/fxamacker/cbor/v2 v2.9.0 // indirect
github.com/go-logr/logr v1.4.3 // indirect
github.com/go-logr/zapr v1.3.0 // indirect
github.com/go-openapi/jsonpointer v0.21.2 // indirect
github.com/go-openapi/jsonreference v0.21.0 // indirect
github.com/go-openapi/swag v0.23.1 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/google/gnostic-models v0.7.0 // indirect
github.com/google/go-cmp v0.7.0 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/gorilla/websocket v1.5.4-0.20250319132907-e064f32e3674 // indirect
github.com/josharian/intern v1.0.0 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/mailru/easyjson v0.9.0 // indirect
github.com/moby/spdystream v0.5.0 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.3-0.20250322232337-35a7c28c31ee // indirect
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/mxk/go-flowrate v0.0.0-20140419014527-cca7078d478f // indirect
github.com/onsi/ginkgo/v2 v2.25.1 // indirect
github.com/onsi/gomega v1.38.1 // indirect
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
github.com/prometheus/client_golang v1.23.0 // indirect
github.com/prometheus/client_model v0.6.2 // indirect
github.com/prometheus/common v0.65.0 // indirect
github.com/prometheus/procfs v0.17.0 // indirect
github.com/spf13/pflag v1.0.7 // indirect
github.com/spf13/pflag v1.0.10 // indirect
github.com/x448/float16 v0.8.4 // indirect
go.uber.org/multierr v1.11.0 // indirect
go.uber.org/zap v1.27.0 // indirect
go.yaml.in/yaml/v2 v2.4.2 // indirect
go.yaml.in/yaml/v2 v2.4.3 // indirect
go.yaml.in/yaml/v3 v3.0.4 // indirect
golang.org/x/mod v0.29.0 // indirect
golang.org/x/oauth2 v0.30.0 // indirect
golang.org/x/net v0.44.0 // indirect
golang.org/x/oauth2 v0.32.0 // indirect
golang.org/x/sys v0.37.0 // indirect
golang.org/x/term v0.36.0 // indirect
golang.org/x/text v0.30.0 // indirect
golang.org/x/time v0.12.0 // indirect
golang.org/x/tools v0.38.0 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20250826171959-ef028d996bc1 // indirect
golang.org/x/time v0.14.0 // indirect
google.golang.org/protobuf v1.36.8 // indirect
gopkg.in/evanphx/json-patch.v4 v4.13.0 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
k8s.io/apiextensions-apiserver v0.34.1 // indirect
k8s.io/klog/v2 v2.130.1 // indirect
k8s.io/utils v0.0.0-20250820121507-0af2bda4dd1d // indirect
sigs.k8s.io/json v0.0.0-20250730193827-2d320260d730 // indirect
sigs.k8s.io/randfill v1.0.0 // indirect
sigs.k8s.io/yaml v1.6.0 // indirect
)
Loading