fix: disable istio inject on controller deployment for webhook (#652)

While working on the [deployment wiki](https://github.com/andyatmiami/kubeflow-notebooks/wiki/Deploying-Kubeflow-Notebooks-2.0-%5BDEVELOPMENT-ONLY%5D) - I ended up stumbling upon an issue whereby if an istio sidecar is injected on the deployment - webhooks traffic starts failing with the following:

```
2025-10-14T14:41:24Z	ERROR	Reconciler error	{"controller": "workspacekind", "controllerGroup": "kubeflow.org", "controllerKind": "WorkspaceKind", "WorkspaceKind": {"name":"jupyterlab"}, "namespace": "", "name": "jupyterlab", "reconcileID": "23977643-4d6a-4c03-93c0-a64fc0002ab2", "error": "Internal error occurred: failed calling webhook \"vworkspacekind.kb.io\": failed to call webhook: Post \"https://webhook-service.kubeflow-workspaces.svc:443/validate-kubeflow-org-v1beta1-workspacekind?timeout=10s\": EOF"}
```

The reason it has not been detected until now is due to the following:

1. istio installed
2. controller gets installed with "vanilla" namespace
    - no istio-inject label
3. backend/frontend update namespace with istio-inject label
4. controller redeployed
    - deployment now gets injected
5. webhooks calls fail

i.e. given the order of our component manifests getting deployed - its not until **after** `controller` is deployed once `backend` and/or `frontend` applied that this issue manifests.

For now, as a quick fix - this commit simply adds a `patch.yaml` file to decorate the `workspaces-controller` `Deployment` with the `sidecar.istio.io/inject: "false"`.
    - we will want to (eventually) restructure the `controller` `manifests` with `kustomize` to better align with `frontend` + `backend` (with `overlays` that specifically handle the `istio` case)
    - we will want to (probably) support the `istio` sidecar on `controller` - but configured in such a way to allow webhooks traffic to be allowed to pass

For consistency with `frontend` and `backend` components - this patch also defines the `istio-injection: enabled` label on the `Namespace` manifest of the `controller`.

Signed-off-by: Andy Stoneberg <astonebe@redhat.com>

Author: andyatmiami

workspaces/controller/config/default/components/common/kustomization.yaml

@@ -0,0 +1,9 @@
+apiVersion: kustomize.config.k8s.io/v1alpha1
+kind: Component
+
+labels:
+- includeSelectors: true
+  pairs:
+    app.kubernetes.io/managed-by: kustomize
+    app.kubernetes.io/name: workspaces-controller
+    app.kubernetes.io/part-of: kubeflow-workspaces

workspaces/controller/config/default/istio_inject_patch.yaml

@@ -0,0 +1,18 @@
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: kubeflow-workspaces
+  labels:
+    istio-injection: enabled
+    app.kubernetes.io/component: null
+    app.kubernetes.io/name: null
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: workspaces-controller
+spec:
+  template:
+    metadata:
+      annotations:
+        sidecar.istio.io/inject: "false"

workspaces/controller/config/default/kustomization.yaml

@@ -1,13 +1,6 @@
 # Adds namespace to all resources.
 namespace: kubeflow-workspaces
 
-labels:
-- includeSelectors: true
-  pairs:
-    app.kubernetes.io/managed-by: kustomize
-    app.kubernetes.io/name: workspaces-controller
-    app.kubernetes.io/part-of: kubeflow-workspaces
-
 resources:
 - ../crd
 - ../rbac
@@ -20,6 +13,9 @@ resources:
 # [PROMETHEUS] To enable prometheus monitor, uncomment all sections with 'PROMETHEUS'.
 #- ../prometheus
 
+components:
+- components/common
+
 patches:
 # [METRICS] The following patch will enable the metrics endpoint. Ensure that you also protect this endpoint.
 # More info: https://book.kubebuilder.io/reference/metrics
@@ -35,6 +31,10 @@ patches:
 # 'CERTMANAGER' needs to be enabled to use ca injection
 - path: webhookcainjection_patch.yaml
 
+# [ISTIO] Enable Istion injection on the kubeflow-workspaces namespace but disable it for the controller deployment
+# to prevent webhook issues
+- path: istio_inject_patch.yaml
+
 # [CERTMANAGER] To enable cert-manager, uncomment all sections with 'CERTMANAGER' prefix.
 # Uncomment the following replacements to add the cert-manager CA injection annotations
 replacements: