Disallow unknown keywords in meta-schema

[gregsdennis]

As a follow-up to #1610, there has been some discussion in Slack around a proposal I made for a way to allow for extensions to the meta-schema while also disallowing unknown keywords with unevaluatedProperties: false.

After iterating on the idea, we landed on this structure for the meta-schema:

{
  "$schema": "https://json-schema.org/1/2025",
  "$id": "https://json-schema.org/1/2025",

  "type": ["object", "boolean"],
  "properties": {
    "$schema": { "type": "string", "format": "iri" },
    "additionalProperties": { "$ref": "#" } // showing for self-reference
    // ...
  },
  "unevaluatedProperties": false,

  "$dynamicRef": "#extension",

  "$defs": {
    "extension": {
      "$dynamicAnchor": "extension"      
    }
  }
}

Then an extension meta-schema would need to look like this:

{
  "$schema": "https://json-schema.org/1/2025",
  "$id": "https://example.com/custom",
  
  "$ref": "https://json-schema.org/1/2025",

  "$defs": {
    "extension": {
      "$dynamicAnchor": "extension",
      "properties": {
        "foo": true
      }
    }
  }
}

Tracing this through:

1. Start validation with the custom meta-schema, https://example.com/custom. This registers the $dynamicAnchor.
2. The custom meta-schema references the base meta-schema (ours).
3. The base meta-schema has a $dynamicRef: #extension that resolves back to the definition in the custom meta-schema, where any additional properties are defined.
4. Because these extra properties are now defined via the $dynamicRef applicator, unevaluatedProperties can see them and will know to ignore them.
5. Lastly, applicators can once again be defined using just $ref: # (like draft 7 and earlier) because the reference back to the custom meta-schema is handled by the base meta-schema root.

It does feel a bit awkward to put your extension information in a $defs entry, but then you're also making a custom meta-schema, which isn't something I expect many people will do.

[Relequestual]

This got me thinking: Would you be able to do this combining multiple extensions (like multiple vocabularies)?

I think the answer is yes. You would have to "allOf" them in the subschema of the primary extension meta-schema which has the dynamicAnchor "extension".

[jdesrosiers]

This got me thinking: Would you be able to do this combining multiple extensions (like multiple vocabularies)?

I think the answer is yes. You would have to "allOf" them in the subschema of the primary extension meta-schema which has the dynamicAnchor "extension".

I see a small problem with this. If you're referencing multiple extensions, the applicators in that extension need to reference the main meta-schema instead of a self reference like the example given.

{
  "$id": "https://example.com/custom",
  
  "$ref": "https://json-schema.org/1/2025",

  "$defs": {
    "extension": {
      "$dynamicAnchor": "extension",
      "allOf": [
        { "$ref": "https://example.com/foo-extension" },
        { "$ref": "https://example.com/bar-extension" }
      ]
    }
  }
}

{
  "$id": "https://example.com/foo-extension",
  "properties": {
    "foo": { "$ref": "https://json-schema.org/1/2025" }
  }
}

{
  "$id": "https://example.com/bar-extension",
  "properties": {
    "bar": { "$ref": "https://json-schema.org/1/2025" }
  }
}

The problem with that is that it couples an extension to a specific release. Anyone providing an extension schema would have to provide a copy for each individual release. That shouldn't be necessary.

A solution is to reintroduce the "meta" dynamic anchor. Then extension schemas can use that dynamic reference instead which would decoupling it from any specific release schema.

{
  "$schema": "https://json-schema.org/1/2025",
  "$id": "https://json-schema.org/1/2025",
  "$dynamicAnchor": "meta-schema",
  // ...
}

{
  "$id": "https://example.com/custom",
  
  "$ref": "https://json-schema.org/1/2025",

  "$defs": {
    "extension": {
      "$dynamicAnchor": "extension",
      "allOf": [
        { "$ref": "https://example.com/foo-extension" }
      ]
    }
  }
}

{
  "$id": "https://example.com/foo-extension",
  "properties": {
    "foo": { "$dynamicRef": "#meta-schema" }
  }
}

Now, https://example.com/custom can change to referencing https://json-schema.org/1/2026 without the extension schema being affected.

[gregsdennis]

That's an interesting use of $dynamicAnchor: using it to look ahead instead of expecting it to be redefined. I wonder if any implementations would have problems with this. You'd be required to process the "$ref": "https://json-schema.org/1/2025" first so that the meta-schema dynamic anchor is defined; otherwise you'd get a reference resolution error.

---

[Thinking out loud ensues. Feel free to skip ahead.]

Also,

{
  "$id": "https://example.com/foo-extension",
  "properties": {
    "foo": { "$dynamicRef": "#meta-schema" }
  }
}

isn't an extension as we've defined that it needs to be. If you follow the example, then you actually have

{
  "$id": "https://example.com/custom",
  
  "$ref": "https://json-schema.org/1/2025",

  "$defs": {
    "extension": {
      "$dynamicAnchor": "extension",
      "allOf": [
        { "$ref": "https://example.com/foo-extension" }
      ]
    }
  }
}

{
  "$id": "https://example.com/foo-extension",
  
  "$ref": "https://json-schema.org/1/2025",

  "$defs": {
    "extension": {
      "$dynamicAnchor": "extension",
      "properties": {
        "foo": { "$dynamicRef": "#meta-schema" }
      }
    }
  }
}

This is an extension building on an extension. You can see that each sub-extension still directly references the spec release. If they're referencing different releases, then #meta-schema will resolve to whichever was encountered first in the evaluation, and that's not deterministic because keyword evaluation order isn't defined.

You can also have an extension combining multiple extensions:

{
  "$id": "https://example.com/custom",
  
  "$ref": "https://json-schema.org/1/2025",

  "$defs": {
    "extension": {
      "$dynamicAnchor": "extension",
      "allOf": [
        { "$ref": "https://example.com/foo-extension" },
        { "$ref": "https://example.com/bar-extension" }
      ]
    }
  }
}

{
  "$id": "https://example.com/foo-extension",
  
  "$ref": "https://json-schema.org/1/2025",

  "$defs": {
    "extension": {
      "$dynamicAnchor": "extension",
      "properties": {
        "foo": { "$dynamicRef": "#meta-schema" }
      }
    }
  }
}

{
  "$id": "https://example.com/bar-extension",
  
  "$ref": "https://json-schema.org/1/2025",

  "$defs": {
    "extension": {
      "$dynamicAnchor": "extension",
      "properties": {
        "bar": { "$dynamicRef": "#meta-schema" }
      }
    }
  }
}

Each extension stands on its own. An additional problem now is that $dynamicAnchor: extension is overridden at the custom level, and so the actual extensions are never found.

---

Initially, I had the extension defined in an allOf instead of a $defs. @Relequestual pointed out that it causes a double-evaluation of the properties, so he simplified it by moving it into a $def to avoid the double-eval. However, that may be our solution here.

{
  "$id": "https://example.com/custom",
  
  "$ref": "https://json-schema.org/1/2025",

  "allOf": [
    {
      "$dynamicAnchor": "extension",
      "allOf": [
        { "$ref": "https://example.com/foo-extension" },
        { "$ref": "https://example.com/bar-extension" }
      ]
    }
  ]
}

{
  "$id": "https://example.com/foo-extension",
  
  "$ref": "https://json-schema.org/1/2025",

  "allOf": [
    {
      "$dynamicAnchor": "extension",
      "properties": {
        "foo": { "$ref": "#" }
      }
    }
  ]
}

{
  "$id": "https://example.com/bar-extension",
  
  "$ref": "https://json-schema.org/1/2025",

  "allOf": [
    {
      "$dynamicAnchor": "extension",
      "properties": {
        "bar": { "ref": "#" }
      }
    }
  ]
}

You do get multiple evaluations of the properties, but it means that the properties defined in the foo- and bar-extensions are evaluated at all.

---

I think @jdesrosiers example actually does hold the answer - an extension needs two parts: the extension meta-schema and an extension definition schema.

{
  "$schema": "https://json-schema.org/1/2025",
  "$id": "https://example.com/foo-extension",
  
  "$ref": "https://json-schema.org/1/2025",

  "$defs": {
    "extension": {
      "$dynamicAnchor": "extension",
      "$ref": "foo-definition"
    }
  }
}

{
  "$schema": "https://json-schema.org/1/2025",
  "$id": "https://example.com/foo-definition",
  
  "properties": {
    "foo": { "$dynamicRef": "#meta-schema" }
  }
}

Then the extension definitions are highly composable, and the composition root is the extension meta-schema.

Composing multiple definitions works like @jdesrosiers suggests as well:

{
  "$schema": "https://json-schema.org/1/2025",
  "$id": "https://example.com/foo-extension",
  
  "$ref": "https://json-schema.org/1/2025",

  "$defs": {
    "extension": {
      "$dynamicAnchor": "extension",
      "allOf": [
        { "$ref": "foo-definition" },
        { "$ref": "bar-definition" }
      ]
    }
  }
}

Assuming we can work out the reference resolution issue on $dynamicRef: #meta-schema, my only remaining concern is that it puts a bit of a burden on would-be extension authors to write two files.

[jdesrosiers]

I think @jdesrosiers example actually does hold the answer - an extension needs two parts: the extension meta-schema and an extension definition schema.

That was what I had in mind. Sorry that wasn't clear, but I'm glad you figured out what I meant in the end.

my only remaining concern is that it puts a bit of a burden on would-be extension authors to write two files.

I see what you're saying. The vocabulary system works the same way and I know that was something people found confusing and difficult about it. It would be fantastic if we could come up with something that made it less cumbersome, but I don't expect to find a way for it to work without a separate extension schema and meta-schema schema unless we make some significant changes the current paradigm.

Assuming we can work out the reference resolution issue on $dynamicRef: #meta-schema

I don't see the problem you're alluding to. The only thing about this pattern that isn't a normal 2020-12 dynamic reference is that it doesn't have a bookending anchor, which we recently said will no longer be required. I've tested it on a branch of my implementation where I've implemented that change and everything works as expected.

[gregsdennis]

Assuming we can work out the reference resolution issue on $dynamicRef: #meta-schema

I don't see the problem you're alluding to. - @jdesrosiers

So let's just look at the foo extension for simplicity:

{
  "$id": "https://example.com/foo-extension",
  
  "$ref": "https://json-schema.org/1/2025",

  "$defs": {
    "extension": {
      "$dynamicAnchor": "extension",
      "properties": {
        "foo": { "$dynamicRef": "#meta-schema" }
      }
    }
  }
}

If a validator decided to run $defs prior to $ref, then it would encounter the $dynamicRef: #meta-schema before that anchor is defined. Of course, it is defined, but the validator hasn't encountered it yet. Because of this, it would raise a reference resolution error.

The only way to ensure that the anchor is encountered before the reference is to require that $ref is processed first, but that doesn't jive with a system that promotes the idea that keywords are run independently.

[jdesrosiers]

If a validator decided to run $defs prior to $ref, then it would encounter the $dynamicRef: #meta-schema before that anchor is defined.

That doesn't make sense to me. Evaluating $defs doesn't do anything. $defs doesn't have behavior, it's just a placeholder for schemas to be referenced. The schemas in $defs are only evaluated when they're referenced. /$defs/extension is only referenced from https://json-schema.org/1/2025, so there's no way it can run before the $ref to that schema is run.

[gregsdennis]

The schemas in $defs are only evaluated when they're referenced.

There's nothing in the spec to guarantee that. An implementation could attempt to resolve references in $defs as a pre-scan. Granted this is a $dynamicRef... I don't know. I haven't had the time to run this on my implementation yet, but I'd like to get other data points.

[jdesrosiers]

We do specifically say that you can't resolve dynamic references ahead of evaluation because it depends on the dynamic scope.

https://json-schema.org/draft/2020-12/json-schema-core#section-7.5.1

For some by-reference applicators, such as "$ref" (Section 8.2.3.1), the referenced schema can be determined by static analysis of the schema document's lexical scope. Others, such as "$dynamicRef" (with "$dynamicAnchor"), may make use of dynamic scoping, and therefore only be resolvable in the process of evaluating the schema with an instance.

[gregsdennis]

But even with an instance, you can go down the $defs path in a pre-scan first.

I realize I'm being pedantic, but I think that kind of scrutiny is necessary.

(That last sentence also needs some grammar ❤)