-
Notifications
You must be signed in to change notification settings - Fork 2.8k
Description
What happened:
I upgraded from 1.16.1 and now get this error:
time="2025-10-20T15:04:33Z" level=error msg="Failed to do run once: soft error\nOVHcloud API error (status code 404): Client::NotFound: \"Got an invalid (or empty) URL\" (X-OVH-Query-Id: EU.ext-2.68f64f81.842220.afb3a70c1c04ffdb199cc36e15ad7a9b) (consecutive soft errors: 2)"
I tried with v1.17.0, v1.18.0 and v1.19.0 and get the same error on each.
I removed all DNS records manually and deployed a new k8s cluster, without luck.
No DNS records were created by external-dns at all.
What you expected to happen:
I didn't expect to get this error, and I expected external-dns to create DNS records for my ingresses.
How to reproduce it (as minimally and precisely as possible):
- Create a DNS zone on OVH
- Create a new cluster
v1.33 - Deploy ingress-nginx
v1.13.3(default config) - Wait for loadBalancer to get ready
- Deploy external-dns
v1.17.0or newer with this config:
From the docs at https://github.com/kubernetes-sigs/external-dns/blob/v0.19.0/docs/tutorials/ovh.md
apiVersion: v1
kind: ServiceAccount
metadata:
name: external-dns
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: external-dns
rules:
- apiGroups: [""]
resources: ["services"]
verbs: ["get","watch","list"]
- apiGroups: [""]
resources: ["pods"]
verbs: ["get","watch","list"]
- apiGroups: ["discovery.k8s.io"]
resources: ["endpointslices"]
verbs: ["get","watch","list"]
- apiGroups: ["extensions","networking.k8s.io"]
resources: ["ingresses"]
verbs: ["get","watch","list"]
- apiGroups: [""]
resources: ["nodes"]
verbs: ["list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: external-dns-viewer
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: external-dns
subjects:
- kind: ServiceAccount
name: external-dns
namespace: default
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: external-dns
spec:
strategy:
type: Recreate
selector:
matchLabels:
app: external-dns
template:
metadata:
labels:
app: external-dns
spec:
serviceAccountName: external-dns
containers:
- name: external-dns
image: registry.k8s.io/external-dns/external-dns:v0.18.0
args:
- --provider=ovh
- --source=ingress
- --policy=upsert-only
- --registry=txt
- --txt-owner-id=my-cluster-1--
- --log-level=trace
- --interval=5m
- --txt-prefix=my-cluster-1--
env:
- name: OVH_ENDPOINT
valueFrom:
secretKeyRef:
name: provider-ovh
key: OVH_ENDPOINT
- name: OVH_APPLICATION_KEY
valueFrom:
secretKeyRef:
name: provider-ovh
key: OVH_APPLICATION_KEY
- name: OVH_APPLICATION_SECRET
valueFrom:
secretKeyRef:
name: provider-ovh
key: OVH_APPLICATION_SECRET
- name: OVH_CONSUMER_KEY
valueFrom:
secretKeyRef:
name: provider-ovh
key: OVH_CONSUMER_KEY
Pod log until the error (which is repeating)
time="2025-10-20T14:59:32Z" level=info msg="GitCommitShort=unknown, GoVersion=go1.24.4, Platform=linux/amd64, UserAgent=ExternalDNS/v20250626-v0.18.0"
time="2025-10-20T14:59:32Z" level=info msg="Instantiating new Kubernetes client"
time="2025-10-20T14:59:32Z" level=debug msg="apiServerURL: "
time="2025-10-20T14:59:32Z" level=debug msg="kubeConfig: "
time="2025-10-20T14:59:32Z" level=info msg="Using inCluster-config based on serviceaccount-token"
time="2025-10-20T14:59:32Z" level=debug msg="serving 'healthz' on ':7979/healthz'"
time="2025-10-20T14:59:32Z" level=debug msg="serving 'metrics' on ':7979/metrics'"
time="2025-10-20T14:59:32Z" level=debug msg="registered '19' metrics"
time="2025-10-20T14:59:32Z" level=info msg="Created Kubernetes client https://10.3.0.1:443"
time="2025-10-20T14:59:33Z" level=info msg="OVH: 3 zones found"
time="2025-10-20T14:59:33Z" level=debug msg="OVH: Getting records for domain_b.com from API"
time="2025-10-20T14:59:33Z" level=debug msg="OVH: Getting records for domain_a.com from API"
time="2025-10-20T14:59:33Z" level=debug msg="OVH: Getting records for domain_c.com from API"
time="2025-10-20T14:59:33Z" level=debug msg="OVH: Getting record 5386578359 for domain_a.com"
time="2025-10-20T14:59:33Z" level=debug msg="OVH: Getting record 5386578356 for domain_a.com"
time="2025-10-20T14:59:33Z" level=debug msg="OVH: Getting record 5386578355 for domain_a.com"
time="2025-10-20T14:59:33Z" level=debug msg="OVH: Getting record 5386578357 for domain_a.com"
time="2025-10-20T14:59:33Z" level=debug msg="OVH: Getting record 5386578358 for domain_a.com"
time="2025-10-20T14:59:33Z" level=debug msg="OVH: Getting record 5386578478 for domain_b.com"
time="2025-10-20T14:59:33Z" level=debug msg="OVH: Getting record 5386578477 for domain_b.com"
time="2025-10-20T14:59:33Z" level=debug msg="OVH: Getting record 5386578479 for domain_b.com"
time="2025-10-20T14:59:33Z" level=debug msg="OVH: Getting record 5386578480 for domain_b.com"
time="2025-10-20T14:59:33Z" level=debug msg="OVH: Getting record 5386578481 for domain_b.com"
time="2025-10-20T14:59:33Z" level=debug msg="OVH: Getting record 5386578526 for domain_b.com"
time="2025-10-20T14:59:33Z" level=debug msg="OVH: Getting record 5386578482 for domain_b.com"
time="2025-10-20T14:59:33Z" level=debug msg="OVH: Getting record 5386578525 for domain_b.com"
time="2025-10-20T14:59:33Z" level=debug msg="OVH: Getting record 5386578527 for domain_b.com"
time="2025-10-20T14:59:33Z" level=debug msg="OVH: Getting record 5386578528 for domain_b.com"
time="2025-10-20T14:59:33Z" level=debug msg="OVH: Getting record 5386578529 for domain_b.com"
time="2025-10-20T14:59:33Z" level=debug msg="OVH: Getting record 5386185734 for domain_c.com"
time="2025-10-20T14:59:33Z" level=debug msg="OVH: Getting record 5386186259 for domain_c.com"
time="2025-10-20T14:59:33Z" level=debug msg="OVH: Getting record 5386185735 for domain_c.com"
time="2025-10-20T14:59:33Z" level=debug msg="OVH: Getting record 5386185736 for domain_c.com"
time="2025-10-20T14:59:33Z" level=debug msg="OVH: Getting record 5386185742 for domain_c.com"
time="2025-10-20T14:59:33Z" level=debug msg="OVH: Getting record 5386185741 for domain_c.com"
time="2025-10-20T14:59:33Z" level=debug msg="OVH: Getting record 5386186256 for domain_c.com"
time="2025-10-20T14:59:33Z" level=debug msg="OVH: Getting record 5386186258 for domain_c.com"
time="2025-10-20T14:59:33Z" level=debug msg="OVH: Getting record 5386185740 for domain_c.com"
time="2025-10-20T14:59:33Z" level=debug msg="OVH: Record 5386578359 for domain_a.com is record#5386578359: NS | => dns16.ovh.net. (0)"
time="2025-10-20T14:59:33Z" level=debug msg="OVH: Record 5386578358 for domain_a.com is record#5386578358: NS | => ns16.ovh.net. (0)"
time="2025-10-20T14:59:33Z" level=debug msg="OVH: Record 5386578481 for domain_b.com is record#5386578481: NS | => dns16.ovh.net. (0)"
time="2025-10-20T14:59:33Z" level=debug msg="OVH: Record 5386578526 for domain_b.com is record#5386578526: A | master.prod => 51.91.161.159 (60)"
time="2025-10-20T14:59:33Z" level=debug msg="OVH: Record 5386578482 for domain_b.com is record#5386578482: NS | => ns16.ovh.net. (0)"
time="2025-10-20T14:59:33Z" level=debug msg="OVH: Record 5386578525 for domain_b.com is record#5386578525: TXT | myorg-3--prod--a-www => \"heritage=external-dns,external-dns/owner=myorg-3--prod--,external-dns/resource=ingress/default/app2--myorg-3--prod--master-live\" (0)"
time="2025-10-20T14:59:33Z" level=debug msg="OVH: Record 5386578527 for domain_b.com is record#5386578527: A | => 51.91.161.149 (60)"
time="2025-10-20T14:59:33Z" level=debug msg="OVH: Record 5386578528 for domain_b.com is record#5386578528: A | www => 51.91.161.149 (60)"
time="2025-10-20T14:59:33Z" level=debug msg="OVH: Record 5386578529 for domain_b.com is record#5386578529: TXT | myorg-3--prod--a-master.prod => \"heritage=external-dns,external-dns/owner=myorg-3--prod--,external-dns/resource=ingress/default/app2--myorg-3--prod--master\" (0)"
time="2025-10-20T14:59:34Z" level=debug msg="OVH: Record 5386185734 for domain_c.com is record#5386185734: NS | => dns16.ovh.net. (0)"
time="2025-10-20T14:59:34Z" level=debug msg="OVH: Record 5386185735 for domain_c.com is record#5386185735: NS | => ns16.ovh.net. (0)"
time="2025-10-20T14:59:34Z" level=debug msg="OVH: Record 5386185736 for domain_c.com is record#5386185736: A | => 213.186.33.5 (0)"
time="2025-10-20T14:59:34Z" level=debug msg="OVH: Record 5386185742 for domain_c.com is record#5386185742: TXT | www => \"3|welcome\" (0)"
time="2025-10-20T14:59:34Z" level=debug msg="OVH: Record 5386185741 for domain_c.com is record#5386185741: TXT | => \"1|www.domain_c.com\" (0)"
time="2025-10-20T14:59:34Z" level=info msg="OVH: 11 endpoints have been found"
time="2025-10-20T14:59:34Z" level=debug msg="Endpoints generated from ingress: default/app1--myorg-3--prod--master-live: [domain_a.com 30 IN A 51.91.161.149 [] domain_a.com 30 IN A 51.91.161.149 [] www.domain_a.com 30 IN A 51.91.161.149 []]"
time="2025-10-20T14:59:34Z" level=debug msg="Endpoints generated from ingress: default/app2--myorg-3--prod--master: [master.prod.domain_b.com 30 IN A 51.91.161.149 [] master.prod.domain_b.com 30 IN A 51.91.161.149 []]"
time="2025-10-20T14:59:34Z" level=debug msg="Endpoints generated from ingress: default/app2--myorg-3--prod--master-live: [domain_b.com 30 IN A 51.91.161.149 [] domain_b.com 30 IN A 51.91.161.149 [] www.domain_b.com 30 IN A 51.91.161.149 []]"
time="2025-10-20T14:59:34Z" level=debug msg="Endpoints generated from ingress: default/app1--myorg-3--prod--master: [master.prod.domain_a.com 30 IN A 51.91.161.149 [] master.prod.domain_a.com 30 IN A 51.91.161.149 []]"
time="2025-10-20T14:59:34Z" level=debug msg="Removing duplicate endpoint domain_a.com 30 IN A 51.91.161.149 []"
time="2025-10-20T14:59:34Z" level=debug msg="Removing duplicate endpoint master.prod.domain_b.com 30 IN A 51.91.161.149 []"
time="2025-10-20T14:59:34Z" level=debug msg="Removing duplicate endpoint domain_b.com 30 IN A 51.91.161.149 []"
time="2025-10-20T14:59:34Z" level=debug msg="Removing duplicate endpoint master.prod.domain_a.com 30 IN A 51.91.161.149 []"
time="2025-10-20T14:59:34Z" level=debug msg="Skipping endpoint domain_b.com 60 IN A 51.91.161.149 [] because owner id does not match, found: \"\", required: \"myorg-3--prod--\""
time="2025-10-20T14:59:34Z" level=debug msg="Skipping endpoint domain_b.com 30 IN A 51.91.161.149 [] because owner id does not match, found: \"\", required: \"myorg-3--prod--\""
time="2025-10-20T14:59:34Z" level=debug msg="OVH: changes CREATE dns:\"domain_a.com\" / targets:51.91.161.149 / type:A"
time="2025-10-20T14:59:34Z" level=debug msg="OVH: changes CREATE dns:\"www.domain_a.com\" / targets:51.91.161.149 / type:A"
time="2025-10-20T14:59:34Z" level=debug msg="OVH: changes CREATE dns:\"master.prod.domain_a.com\" / targets:51.91.161.149 / type:A"
time="2025-10-20T14:59:34Z" level=debug msg="OVH: changes CREATE dns:\"myorg-3--prod--a-domain_a.com\" / targets:\"heritage=external-dns,external-dns/owner=myorg-3--prod--,external-dns/resource=ingress/default/app1--myorg-3--prod--master-live\" / type:TXT"
time="2025-10-20T14:59:34Z" level=debug msg="OVH: changes CREATE dns:\"myorg-3--prod--a-www.domain_a.com\" / targets:\"heritage=external-dns,external-dns/owner=myorg-3--prod--,external-dns/resource=ingress/default/app1--myorg-3--prod--master-live\" / type:TXT"
time="2025-10-20T14:59:34Z" level=debug msg="OVH: changes CREATE dns:\"myorg-3--prod--a-master.prod.domain_a.com\" / targets:\"heritage=external-dns,external-dns/owner=myorg-3--prod--,external-dns/resource=ingress/default/app1--myorg-3--prod--master\" / type:TXT"
time="2025-10-20T14:59:34Z" level=debug msg="OVH: changes UPDATEOLD dns:\"master.prod.domain_b.com\" / targets:51.91.161.149 / type:A"
time="2025-10-20T14:59:34Z" level=debug msg="OVH: changes UPDATEOLD dns:\"www.domain_b.com\" / targets:51.91.161.149 / type:A"
time="2025-10-20T14:59:34Z" level=debug msg="OVH: changes UPDATEOLD dns:\"myorg-3--prod--a-master.prod.domain_b.com\" / targets:\"heritage=external-dns,external-dns/owner=myorg-3--prod--,external-dns/resource=ingress/default/app2--myorg-3--prod--master\" / type:TXT"
time="2025-10-20T14:59:34Z" level=debug msg="OVH: changes UPDATEOLD dns:\"myorg-3--prod--a-www.domain_b.com\" / targets:\"heritage=external-dns,external-dns/owner=myorg-3--prod--,external-dns/resource=ingress/default/app2--myorg-3--prod--master-live\" / type:TXT"
time="2025-10-20T14:59:34Z" level=debug msg="OVH: changes UPDATENEW dns:\"master.prod.domain_b.com\" / targets:51.91.161.149 / type:A"
time="2025-10-20T14:59:34Z" level=debug msg="OVH: changes UPDATENEW dns:\"www.domain_b.com\" / targets:51.91.161.149 / type:A"
time="2025-10-20T14:59:34Z" level=debug msg="OVH: changes UPDATENEW dns:\"myorg-3--prod--a-master.prod.domain_b.com\" / targets:\"heritage=external-dns,external-dns/owner=myorg-3--prod--,external-dns/resource=ingress/default/app2--myorg-3--prod--master\" / type:TXT"
time="2025-10-20T14:59:34Z" level=debug msg="OVH: changes UPDATENEW dns:\"myorg-3--prod--a-www.domain_b.com\" / targets:\"heritage=external-dns,external-dns/owner=myorg-3--prod--,external-dns/resource=ingress/default/app2--myorg-3--prod--master-live\" / type:TXT"
time="2025-10-20T14:59:34Z" level=info msg="OVH: \"\": 6 changes will be done"
time="2025-10-20T14:59:34Z" level=info msg="OVH: \"domain_b.com\": 0 changes will be done"
time="2025-10-20T14:59:34Z" level=debug msg="OVH: Refresh domain_b.com zone"
time="2025-10-20T14:59:34Z" level=debug msg="OVH: Add an entry to zone action(create) : myorg-3--prod--a-master.prod.domain_a.com 0 IN TXT \"heritage=external-dns,external-dns/owner=myorg-3--prod--,external-dns/resource=ingress/default/app1--myorg-3--prod--master\""
time="2025-10-20T14:59:34Z" level=debug msg="OVH: Add an entry to zone action(create) : domain_a.com 30 IN A 51.91.161.149"
time="2025-10-20T14:59:34Z" level=debug msg="OVH: Add an entry to zone action(create) : www.domain_a.com 30 IN A 51.91.161.149"
time="2025-10-20T14:59:34Z" level=debug msg="OVH: Add an entry to zone action(create) : master.prod.domain_a.com 30 IN A 51.91.161.149"
time="2025-10-20T14:59:34Z" level=debug msg="OVH: Add an entry to zone action(create) : myorg-3--prod--a-domain_a.com 0 IN TXT \"heritage=external-dns,external-dns/owner=myorg-3--prod--,external-dns/resource=ingress/default/app1--myorg-3--prod--master-live\""
time="2025-10-20T14:59:34Z" level=debug msg="OVH: Add an entry to zone action(create) : myorg-3--prod--a-www.domain_a.com 0 IN TXT \"heritage=external-dns,external-dns/owner=myorg-3--prod--,external-dns/resource=ingress/default/app1--myorg-3--prod--master-live\""
time="2025-10-20T14:59:34Z" level=error msg="Failed to do run once: soft error\nOVHcloud API error (status code 404): Client::NotFound: \"Got an invalid (or empty) URL\" (X-OVH-Query-Id: EU.ext-3.68f64e56.840806.eae7a155ecaed666ecb3897024503631) (consecutive soft errors: 1)"
Anything else we need to know?:
When I deploy external-dns with version v1.16.1 I don't get this error and records are created.
I think this output might reveal the error: Add an entry to zone action(create). There are two spaces between to and zone. Looks like an empty string is used instead of the zone-id.
Please let me know if you need more information.
I don't know how to further debug this, so reverting back to v1.16.1 for now.
Environment:
- External-DNS version
>= 1.17.0 - DNS provider: OVHCloud
- Others: