Skip to content

Commit 130be28

Browse files
dmartinolclaude
dmartinol
and
claude
authored
MCPRegistry e2e tests (#2017)
* Add MCPRegistry controller e2e test framework - Create Ginkgo-based test suite for operator testing - Add comprehensive test helpers for MCPRegistry operations - Include test fixtures with sample YAML manifests - Set up Kubernetes test environment with envtest support - Add namespace isolation and cleanup utilities 🤖 Generated with [Claude Code](https://claude.ai/code) Signed-off-by: Daniele Martinoli <dmartino@redhat.com> Co-authored-by: Claude <noreply@anthropic.com> * Add comprehensive Ginkgo test framework for MCPRegistry e2e testing - Create test suite with proper Kubernetes environment setup - Add specialized helper utilities for MCPRegistry operations - Implement ConfigMap test helpers for registry data validation - Add status validation helpers for phase and condition checking - Create timing utilities with proper timeout configurations - Add test data factories for generating test resources - Include builder patterns for fluent resource construction - Support both ToolHive and upstream MCP registry formats - Add comprehensive test fixtures and scenarios 🤖 Generated with [Claude Code](https://claude.ai/code) Signed-off-by: Daniele Martinoli <dmartino@redhat.com> Co-authored-by: Claude <noreply@anthropic.com> * Fix MCPRegistry e2e test timeouts and finalizer handling - Fix finalizer removal using Patch instead of Update to avoid resource conflicts - Update registry data structure to match expected schema (add required fields: tier, status, tools, image) - Add proper registry deletion waiting in cleanup to prevent namespace deletion issues - Fix lint errors by removing dot imports from non-test files - Add comprehensive MCPRegistry lifecycle test coverage - Improve error handling and logging in test helpers Signed-off-by: Daniele Martinoli <dmartino@redhat.com> Co-authored-by: Claude <noreply@anthropic.com> 🤖 Generated with [Claude Code](https://claude.ai/code) * reviewed finalization logic to avoid unnecessary attempts (and logged errors) Signed-off-by: Daniele Martinoli <dmartino@redhat.com> * extended and successful validation of "should create MCPRegistry with correct initial status" Signed-off-by: Daniele Martinoli <dmartino@redhat.com> * Enhance e2e test setup by adding support for kubebuilder assets - Introduced environment variable handling for KUBEBUILDER_ASSETS - Added warning for missing kubebuilder assets to improve test reliability - Updated test environment configuration to include BinaryAssetsDirectory This change aims to streamline the e2e testing process and provide clearer feedback on asset availability. Signed-off-by: Daniele Martinoli <dmartino@redhat.com> * - reviewed controller logic to avoid reconciliaton loops - initial draft of e2e tests Signed-off-by: Daniele Martinoli <dmartino@redhat.com> * more tests for automatic and manual sync Signed-off-by: Daniele Martinoli <dmartino@redhat.com> * filtering and git source e2e tests * controller changes to succeed e2e tests Signed-off-by: Daniele Martinoli <dmartino@redhat.com> * bump chart version and update CRD docs Signed-off-by: Daniele Martinoli <dmartino@redhat.com> * rebase issue * bump chart version * bump chart version Signed-off-by: Daniele Martinoli <dmartino@redhat.com> * removed the whole test/e2e/operator/fixtures folder Signed-off-by: Daniele Martinoli <dmartino@redhat.com> * removed bunch of unused functions and the unneded defer clause Signed-off-by: Daniele Martinoli <dmartino@redhat.com> * integrated operator e2e tests in CI and added new task to execute them Signed-off-by: Daniele Martinoli <dmartino@redhat.com> * moved automated tests to operator CI Signed-off-by: Daniele Martinoli <dmartino@redhat.com> * fix for automated tests Signed-off-by: Daniele Martinoli <dmartino@redhat.com> * installed ginkgo Signed-off-by: Daniele Martinoli <dmartino@redhat.com> * try to fix e2e test failures (git clone issue) Signed-off-by: Daniele Martinoli <dmartino@redhat.com> * restored commented tests Signed-off-by: Daniele Martinoli <dmartino@redhat.com> * - reviewed apply status logic to run a single status update per reconciliation and avoid errors due to updated resource. - reviewed overall status calculation to consider updates or (if nil) lates status Signed-off-by: Daniele Martinoli <dmartino@redhat.com> --------- Signed-off-by: Daniele Martinoli <dmartino@redhat.com> Co-authored-by: Claude <noreply@anthropic.com>
1 parent 3b0ff07 commit 130be28

33 files changed

+4742
-214
lines changed

‎.github/workflows/operator-ci.yml‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -187,3 +187,12 @@ jobs:
187187
chainsaw test --test-dir test/e2e/chainsaw/operator/single-tenancy/setup --config .chainsaw.yaml
188188
chainsaw test --test-dir test/e2e/chainsaw/operator/single-tenancy/test-scenarios --config .chainsaw.yaml
189189
chainsaw test --test-dir test/e2e/chainsaw/operator/single-tenancy/cleanup --config .chainsaw.yaml
190+
191+
- name: Install Ginkgo CLI
192+
run: |
193+
go install github.com/onsi/ginkgo/v2/ginkgo@latest
194+
195+
- name: Run Ginkgo tests
196+
run: |
197+
go install sigs.k8s.io/controller-runtime/tools/setup-envtest@latest
198+
KUBEBUILDER_ASSETS="$(setup-envtest use 1.28.0 -p path)" ginkgo --github-output test/e2e/operator

‎cmd/thv-operator/Taskfile.yml‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -198,7 +198,16 @@ tasks:
198198
- go install sigs.k8s.io/controller-runtime/tools/setup-envtest@latest
199199
- KUBEBUILDER_ASSETS="$(shell setup-envtest use 1.28.0 -p path)" go test ./cmd/thv-operator/... -coverprofile cover.out
200200

201+
operator-e2e-test-ginkgo:
202+
desc: Run E2E tests for the operator with Ginkgo
203+
cmds:
204+
- KUBEBUILDER_ASSETS="$(setup-envtest use 1.28.0 -p path)" ginkgo -v test/e2e/operator
205+
206+
# Backwards compatibility
201207
operator-e2e-test:
208+
deps: [operator-e2e-test-chainsaw]
209+
210+
operator-e2e-test-chainsaw:
202211
desc: Run E2E tests for the operator
203212
cmds:
204213
- |

‎cmd/thv-operator/api/v1alpha1/mcpregistry_types.go‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -97,7 +97,7 @@ type GitSource struct {
9797
// Repository is the Git repository URL (HTTP/HTTPS/SSH)
9898
// +kubebuilder:validation:Required
9999
// +kubebuilder:validation:MinLength=1
100-
// +kubebuilder:validation:Pattern="^(https?://|git@|ssh://|git://).*"
100+
// +kubebuilder:validation:Pattern="^(file:///|https?://|git@|ssh://|git://).*"
101101
Repository string `json:"repository"`
102102

103103
// Branch is the Git branch to use (mutually exclusive with Tag and Commit)
@@ -186,6 +186,10 @@ type MCPRegistryStatus struct {
186186
// +optional
187187
APIStatus *APIStatus `json:"apiStatus,omitempty"`
188188

189+
// LastAppliedFilterHash is the hash of the last applied filter
190+
// +optional
191+
LastAppliedFilterHash string `json:"lastAppliedFilterHash,omitempty"`
192+
189193
// StorageRef is a reference to the internal storage location
190194
// +optional
191195
StorageRef *StorageReference `json:"storageRef,omitempty"`

‎cmd/thv-operator/controllers/mcpregistry_controller.go‎

Lines changed: 99 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,9 @@ package controllers
22

33
import (
44
"context"
5+
"crypto/sha256"
6+
"encoding/hex"
7+
"encoding/json"
58
"fmt"
69
"time"
710

@@ -208,7 +211,7 @@ func (r *MCPRegistryReconciler) Reconcile(ctx context.Context, req ctrl.Request)
208211
r.deriveOverallStatus(ctx, mcpRegistry, statusManager, statusDeriver)
209212

210213
// 8. Apply all status changes in a single batch update
211-
if statusUpdateErr := statusManager.Apply(ctx, r.Client); statusUpdateErr != nil {
214+
if statusUpdateErr := r.applyStatusUpdates(ctx, r.Client, mcpRegistry, statusManager); statusUpdateErr != nil {
212215
ctxLogger.Error(statusUpdateErr, "Failed to apply batched status update")
213216
// Return the status update error only if there was no main reconciliation error
214217
if syncErr == nil {
@@ -386,20 +389,23 @@ func (*MCPRegistryReconciler) deriveOverallStatus(
386389
statusManager mcpregistrystatus.StatusManager, statusDeriver mcpregistrystatus.StatusDeriver) {
387390
ctxLogger := log.FromContext(ctx)
388391

392+
syncStatus := statusManager.Sync().Status()
393+
if syncStatus == nil {
394+
syncStatus = mcpRegistry.Status.SyncStatus
395+
}
396+
apiStatus := statusManager.API().Status()
397+
if apiStatus == nil {
398+
apiStatus = mcpRegistry.Status.APIStatus
399+
}
389400
// Use the StatusDeriver to determine the overall phase and message
390401
// based on current sync and API statuses
391-
derivedPhase, derivedMessage := statusDeriver.DeriveOverallStatus(
392-
mcpRegistry.Status.SyncStatus,
393-
mcpRegistry.Status.APIStatus,
394-
)
402+
derivedPhase, derivedMessage := statusDeriver.DeriveOverallStatus(syncStatus, apiStatus)
395403

396404
// Only update phase and message if they've changed
397405
statusManager.SetOverallStatus(derivedPhase, derivedMessage)
398-
ctxLogger.Info("Updated overall status",
399-
"oldPhase", mcpRegistry.Status.Phase,
400-
"newPhase", derivedPhase,
401-
"oldMessage", mcpRegistry.Status.Message,
402-
"newMessage", derivedMessage)
406+
ctxLogger.Info("Updated overall status", "syncStatus", syncStatus, "apiStatus", apiStatus,
407+
"oldPhase", mcpRegistry.Status.Phase, "newPhase", derivedPhase,
408+
"oldMessage", mcpRegistry.Status.Message, "newMessage", derivedMessage)
403409
}
404410

405411
// SetupWithManager sets up the controller with the Manager.
@@ -411,3 +417,86 @@ func (r *MCPRegistryReconciler) SetupWithManager(mgr ctrl.Manager) error {
411417
Owns(&corev1.ConfigMap{}).
412418
Complete(r)
413419
}
420+
421+
// Apply applies all collected status changes in a single batch update.
422+
// Only actual changes are applied to the status to avoid unnecessary reconciliations
423+
func (r *MCPRegistryReconciler) applyStatusUpdates(
424+
ctx context.Context, k8sClient client.Client,
425+
mcpRegistry *mcpv1alpha1.MCPRegistry, statusManager mcpregistrystatus.StatusManager) error {
426+
427+
ctxLogger := log.FromContext(ctx)
428+
429+
// Refetch the latest version of the resource to avoid conflicts
430+
latestRegistry := &mcpv1alpha1.MCPRegistry{}
431+
if err := k8sClient.Get(ctx, client.ObjectKeyFromObject(mcpRegistry), latestRegistry); err != nil {
432+
ctxLogger.Error(err, "Failed to fetch latest MCPRegistry version for status update")
433+
return fmt.Errorf("failed to fetch latest MCPRegistry version: %w", err)
434+
}
435+
latestRegistryStatus := latestRegistry.Status
436+
hasUpdates := false
437+
438+
// Apply manual sync trigger change if necessary
439+
if mcpRegistry.Annotations != nil {
440+
if triggerValue := mcpRegistry.Annotations[mcpregistrystatus.SyncTriggerAnnotation]; triggerValue != "" {
441+
if latestRegistryStatus.LastManualSyncTrigger != triggerValue {
442+
latestRegistryStatus.LastManualSyncTrigger = triggerValue
443+
hasUpdates = true
444+
ctxLogger.Info("Updated LastManualSyncTrigger", "trigger", triggerValue)
445+
}
446+
}
447+
}
448+
449+
// Apply filter change if necessary
450+
currentFilterJSON, err := json.Marshal(mcpRegistry.Spec.Filter)
451+
if err != nil {
452+
ctxLogger.Error(err, "Failed to marshal current filter")
453+
return fmt.Errorf("failed to marshal current filter: %w", err)
454+
}
455+
currentFilterHash := sha256.Sum256(currentFilterJSON)
456+
currentFilterHashStr := hex.EncodeToString(currentFilterHash[:])
457+
if latestRegistryStatus.LastAppliedFilterHash != currentFilterHashStr {
458+
latestRegistryStatus.LastAppliedFilterHash = currentFilterHashStr
459+
hasUpdates = true
460+
ctxLogger.Info("Updated LastAppliedFilterHash", "hash", currentFilterHashStr)
461+
}
462+
463+
// Update storage reference if necessary
464+
storageRef := r.storageManager.GetStorageReference(latestRegistry)
465+
if storageRef != nil {
466+
if latestRegistryStatus.StorageRef == nil || latestRegistryStatus.StorageRef.ConfigMapRef.Name != storageRef.ConfigMapRef.Name {
467+
latestRegistryStatus.StorageRef = storageRef
468+
hasUpdates = true
469+
ctxLogger.Info("Updated StorageRef", "storageRef", storageRef)
470+
}
471+
}
472+
473+
// Apply status changes from status manager
474+
hasUpdates = statusManager.UpdateStatus(ctx, &latestRegistryStatus) || hasUpdates
475+
476+
// Single status update using the latest version
477+
if hasUpdates {
478+
latestRegistry.Status = latestRegistryStatus
479+
if err := k8sClient.Status().Update(ctx, latestRegistry); err != nil {
480+
ctxLogger.Error(err, "Failed to apply batched status update")
481+
return fmt.Errorf("failed to apply batched status update: %w", err)
482+
}
483+
var syncPhase mcpv1alpha1.SyncPhase
484+
if latestRegistryStatus.SyncStatus != nil {
485+
syncPhase = latestRegistryStatus.SyncStatus.Phase
486+
}
487+
var apiPhase string
488+
if latestRegistryStatus.APIStatus != nil {
489+
apiPhase = string(latestRegistryStatus.APIStatus.Phase)
490+
}
491+
ctxLogger.V(1).Info("Applied batched status updates",
492+
"phase", latestRegistryStatus.Phase,
493+
"syncPhase", syncPhase,
494+
"apiPhase", apiPhase,
495+
"message", latestRegistryStatus.Message,
496+
"conditionsCount", len(latestRegistryStatus.Conditions))
497+
} else {
498+
ctxLogger.V(1).Info("No batched status updates applied")
499+
}
500+
501+
return nil
502+
}

‎cmd/thv-operator/pkg/mcpregistrystatus/collector.go‎

Lines changed: 46 additions & 50 deletions
Original file line numberDiff line numberDiff line change
@@ -3,11 +3,9 @@ package mcpregistrystatus
33

44
import (
55
"context"
6-
"fmt"
76

87
"k8s.io/apimachinery/pkg/api/meta"
98
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
10-
"sigs.k8s.io/controller-runtime/pkg/client"
119
"sigs.k8s.io/controller-runtime/pkg/log"
1210

1311
mcpv1alpha1 "github.com/stacklok/toolhive/cmd/thv-operator/api/v1alpha1"
@@ -122,58 +120,46 @@ func (s *StatusCollector) SetAPIStatus(phase mcpv1alpha1.APIPhase, message strin
122120
s.hasChanges = true
123121
}
124122

125-
// Apply applies all collected status changes in a single batch update.
126-
func (s *StatusCollector) Apply(ctx context.Context, k8sClient client.Client) error {
127-
if !s.hasChanges {
128-
return nil
129-
}
123+
// UpdateStatus applies all collected status changes in a single batch update.
124+
// Requires the MCPRegistryStatus being the updated version from the cluster
125+
func (s *StatusCollector) UpdateStatus(ctx context.Context, mcpRegistryStatus *mcpv1alpha1.MCPRegistryStatus) bool {
130126

131127
ctxLogger := log.FromContext(ctx)
132128

133-
// Refetch the latest version of the resource to avoid conflicts
134-
latestRegistry := &mcpv1alpha1.MCPRegistry{}
135-
if err := k8sClient.Get(ctx, client.ObjectKeyFromObject(s.mcpRegistry), latestRegistry); err != nil {
136-
ctxLogger.Error(err, "Failed to fetch latest MCPRegistry version for status update")
137-
return fmt.Errorf("failed to fetch latest MCPRegistry version: %w", err)
138-
}
139-
140-
// Apply phase change
141-
if s.phase != nil {
142-
latestRegistry.Status.Phase = *s.phase
143-
}
144-
145-
// Apply message change
146-
if s.message != nil {
147-
latestRegistry.Status.Message = *s.message
129+
if s.hasChanges {
130+
// Apply phase change
131+
if s.phase != nil {
132+
mcpRegistryStatus.Phase = *s.phase
133+
}
134+
135+
// Apply message change
136+
if s.message != nil {
137+
mcpRegistryStatus.Message = *s.message
138+
}
139+
140+
// Apply sync status change
141+
if s.syncStatus != nil {
142+
mcpRegistryStatus.SyncStatus = s.syncStatus
143+
}
144+
145+
// Apply API status change
146+
if s.apiStatus != nil {
147+
mcpRegistryStatus.APIStatus = s.apiStatus
148+
}
149+
150+
// Apply condition changes
151+
for _, condition := range s.conditions {
152+
meta.SetStatusCondition(&mcpRegistryStatus.Conditions, condition)
153+
}
154+
155+
ctxLogger.V(1).Info("Batched status update applied",
156+
"phase", s.phase,
157+
"message", s.message,
158+
"conditionsCount", len(s.conditions))
159+
return true
148160
}
149-
150-
// Apply sync status change
151-
if s.syncStatus != nil {
152-
latestRegistry.Status.SyncStatus = s.syncStatus
153-
}
154-
155-
// Apply API status change
156-
if s.apiStatus != nil {
157-
latestRegistry.Status.APIStatus = s.apiStatus
158-
}
159-
160-
// Apply condition changes
161-
for _, condition := range s.conditions {
162-
meta.SetStatusCondition(&latestRegistry.Status.Conditions, condition)
163-
}
164-
165-
// Single status update using the latest version
166-
if err := k8sClient.Status().Update(ctx, latestRegistry); err != nil {
167-
ctxLogger.Error(err, "Failed to apply batched status update")
168-
return fmt.Errorf("failed to apply batched status update: %w", err)
169-
}
170-
171-
ctxLogger.V(1).Info("Applied batched status update",
172-
"phase", s.phase,
173-
"message", s.message,
174-
"conditionsCount", len(s.conditions))
175-
176-
return nil
161+
ctxLogger.V(1).Info("No batched status update needed")
162+
return false
177163
}
178164

179165
// StatusManager interface methods
@@ -196,6 +182,11 @@ func (s *StatusCollector) SetOverallStatus(phase mcpv1alpha1.MCPRegistryPhase, m
196182

197183
// SyncStatusCollector implementation
198184

185+
// Status returns the sync status
186+
func (sc *syncStatusCollector) Status() *mcpv1alpha1.SyncStatus {
187+
return sc.parent.syncStatus
188+
}
189+
199190
// SetSyncCondition sets a sync-related condition
200191
func (sc *syncStatusCollector) SetSyncCondition(condition metav1.Condition) {
201192
sc.parent.conditions[condition.Type] = condition
@@ -210,6 +201,11 @@ func (sc *syncStatusCollector) SetSyncStatus(phase mcpv1alpha1.SyncPhase, messag
210201

211202
// APIStatusCollector implementation
212203

204+
// Status returns the API status
205+
func (ac *apiStatusCollector) Status() *mcpv1alpha1.APIStatus {
206+
return ac.parent.apiStatus
207+
}
208+
213209
// SetAPIStatus delegates to the parent's SetAPIStatus method
214210
func (ac *apiStatusCollector) SetAPIStatus(phase mcpv1alpha1.APIPhase, message string, endpoint string) {
215211
ac.parent.SetAPIStatus(phase, message, endpoint)

‎cmd/thv-operator/pkg/mcpregistrystatus/collector_test.go‎

Lines changed: 6 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -322,9 +322,8 @@ func TestStatusCollector_Apply(t *testing.T) {
322322
t.Parallel()
323323
collector := NewStatusManager(registry).(*StatusCollector)
324324

325-
err := collector.Apply(ctx, k8sClient)
326-
327-
assert.NoError(t, err)
325+
hasUpdates := collector.UpdateStatus(ctx, &registry.Status)
326+
assert.False(t, hasUpdates)
328327
})
329328

330329
t.Run("verifies hasChanges behavior", func(t *testing.T) {
@@ -394,7 +393,7 @@ func TestStatusCollector_MultipleConditions(t *testing.T) {
394393
assert.Contains(t, collector.conditions, mcpv1alpha1.ConditionAPIReady)
395394
}
396395

397-
func TestStatusCollector_ApplyErrors(t *testing.T) {
396+
func TestStatusCollector_NoUpdates(t *testing.T) {
398397
t.Parallel()
399398

400399
ctx := context.Background()
@@ -406,9 +405,6 @@ func TestStatusCollector_ApplyErrors(t *testing.T) {
406405
t.Run("error fetching latest registry", func(t *testing.T) {
407406
t.Parallel()
408407

409-
// Create client that will fail on Get
410-
k8sClient := fake.NewClientBuilder().WithScheme(scheme).Build()
411-
412408
// Create collector with registry that doesn't exist in client
413409
registry := &mcpv1alpha1.MCPRegistry{
414410
ObjectMeta: metav1.ObjectMeta{
@@ -417,12 +413,10 @@ func TestStatusCollector_ApplyErrors(t *testing.T) {
417413
},
418414
}
419415

420-
collector := newStatusCollector(registry)
421-
collector.SetPhase(mcpv1alpha1.MCPRegistryPhaseReady) // Make some changes
416+
collector := newStatusCollector(registry) // No changes
417+
hasUpdates := collector.UpdateStatus(ctx, &registry.Status)
418+
assert.False(t, hasUpdates)
422419

423-
err := collector.Apply(ctx, k8sClient)
424-
assert.Error(t, err)
425-
assert.Contains(t, err.Error(), "failed to fetch latest MCPRegistry version")
426420
})
427421

428422
}

‎cmd/thv-operator/pkg/sync/constants.go‎ renamed to ‎cmd/thv-operator/pkg/mcpregistrystatus/constants.go‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
package sync
1+
package mcpregistrystatus
22

33
const (
44
// SyncTriggerAnnotation is the annotation key used to trigger registry synchronization

0 commit comments

Comments
 (0)