Suggestion: optional keyword to complement required

[Altreus]

I don't write objects with optional properties most of the time. I tend to prefer to use polymorphism to allow one of several data structures. When I do have an optional field, it's often something like "comments" or "extra info", and even then I don't have any philosophical objection to simply making it required but allowing it to contain the empty string.

An optional keyword to complement required would greatly benefit me (and hopefully others), without treading on the toes of people who are writing schemata with many optional values.

My proposition is thus:

• Either required or optional may be provided
• It is an error to provide both
• The default remains as it is (everything optional), which is the same as providing required with an empty array
• If optional is provided, anything not mentioned is now required
• Passing an empty array to optional means everything is required

Hope that's both clear and a useful idea.

[gregsdennis]

This seems like a duplicate/variant of #1096.

[jdesrosiers]

I can see how this could work. It would work similarly to additionalProperties in that it would collect the property names declared by properties and use that to make its assertion. The only change I would make to the proposal is to not forbid required when using optional. That would make a good linting rule, but it's unnecessary from JSON Schema architecture point of view.

Something like this has been requested for a long time, but this is the first proposal of this type that I think fits well into the JSON Schema architecture and I think we should consider it. Requiring properties by default (#1096) and requiredProperties (#846) are problematic because they assert more than one thing: (1) a property schema and (2) that the property is required. This proposal is better because it deals only with which properties are required.

Although I hate to add more keywords that break keyword independence, I think this is a good solution. In particular, this wouldn't have the same kind of problems with combining schemas that additionalProperties has.

[karenetheridge]

The only change I would make to the proposal is to not forbid required when using optional. That would make a good linting rule, but it's unnecessary from JSON Schema architecture point of view.

Agreed. But this also means we need to work out what happens when they are used together: for example if one schema pulls in another with $ref, and the reffed schema contains one of these keywords and the first schema contains the other. Would 'required' always overrule 'options'? Or the other way around? or should the innermost keyword always overrule the outermost (akin to a subclass overriding a superclass)?

edit: a subschema keyword overriding a superschema keyword isn't even doable, since both are evaluated, each in their own contexts. keywords generally need to stand on their own when not siblings, unless we explicitly define an annotation behaviour that can carry state.

[jdesrosiers]

we need to work out what happens when they are used together

When they are used together, they each assert independently just like any other keyword. That does mean it's possible for them to contradict each other, but not in any way that isn't possible already in JSON Schema. They don't need to interact or override each other. optional would be no more affected by a reference than additionalProperties is. That's what I like about this proposal.

[Altreus]

Grateful to hear my idea has legs! Happy you agree that it fits in without obviously breaking anything.

The only edge case I can think of is accidentally adding a property to a schema, forgetting that optional: [] was provided, thus breaking backward-compatibility. This is probably another linter case, however, rather than a strong reason to disallow that. Personally I would prefer to release a new version if I weren't going to put that new property in the optional array.

[jdesrosiers]

I implemented optional in my draft-future dialect. If you want to try it out, you can go to https://json-schema.hyperjump.io and use $schema": "https://json-schema.org/draft/future/schema".

Note: Things in my draft-future dialect are not necessarily going to appear in the next draft and this dialect is not available in npm. It should only be used for the purposes of evaluating potential changes.

[awwright]

Requiring properties by default (#1096) and requiredProperties (#846) are problematic because they assert more than one thing: (1) a property schema and (2) that the property is required

I don't believe doing multiple things is a problem as such. If there is a keyword does two things (an "authoring convenience keyword"), then as long as we also have keywords that do each of those things separately, there is no issue.

The problem would be if keywords do multiple things and there's no way around it. For example, suppose "minLength" (which only applies when the input is a string), also required that the value be a string, or if it were a number, applied to numbers as well. We would have to craft a more complicated if/then statement (if type="string" then minLength: 1), which would be more difficult to reason about.

[jdesrosiers]

I don't believe doing multiple things is a problem as such. If there is a keyword does two things (an "authoring convenience keyword"), then as long as we also have keywords that do each of those things separately, there is no issue.

Fair enough. I think your example is a bit of a stretch, but I agree with your point. However, I would still prefer a solution with a keyword that only does one thing. Requiring properties by default is off the table. A requiredProperties keyword is reasonable, but not ideal. An optional keyword is not perfect either, but I think it has the fewest issues.

[awwright]

However, I would still prefer a solution with a keyword that only does one thing. Requiring properties by default is off the table.

I would say that an "optional" keyword does at least two things.

It's presence means changes the behavior of "properties" so that it requires the listed properties; except the ones listed by "optional". This seems more complicated to me from both an authoring perspective as well as an implementation perspective.

If I understand correctly, these schemas have the same behavior:

{
   properties: {
      "name": { type: "string" },
      "phone": { type: "string" }
   },
   optional: [ "phone" ]
}

Versus:

{
   requiredProperties: {
      "name": { type: "string" }
   },
   properties: {
      "phone": { type: "string" }
   }
}

The only advantage I see to an "optional" keyword is it allows you to group/order properties the way you want, instead of forcing you to group them by if it's required/optional. But this has no effect on the validation, it's purely an authoring choice.

[gregsdennis]

It's presence means changes the behavior of "properties"

Not necessarily. optional can read the contents of properties to get a list of keywords, then enforce that the ones not declared within itself are present. properties can remain as is, oblivious to the presence of required or optional.

[gregsdennis]

I do like the idea of enforcing that both cannot be present. This can be enforced in the meta-schema.

[karenetheridge]

properties can remain as is, oblivious to the presence of required or optional.

Yes, this is the correct way forward.

This can be enforced in the meta-schema.

I agree with this too (but since the specification text itself will mention this restriction, it is not out of the question for an implementation to enforce this with code as well).

[jdesrosiers]

This seems more complicated to me from both an authoring perspective as well as an implementation perspective.

The whole reason I implemented this keyword was to explore things like this. I can assure you that optional was trivial to implement and requires nothing remotely unusual or unique to JSON Schema. As others have pointed out, it doesn't effect other keywords and makes it's own assertions which makes it a good fit.

As for making authoring complicated, I don't see how that would be, but having a web based implementation available makes it easy for us all to try it out and evaluate the developer experience implications.

I do like the idea of enforcing that both cannot be present.

It could be done, but I consider this kind of thing out of scope. I see it as a linting responsibility. I'm sure there's no good reason to use both at once, but there's no reason they couldn't both be used.

[Altreus]

$schema": "https://json-schema.org/draft/future/schema"

Brilliant. If I ever get back onto that project in a reasonable time frame I'll update the code to pull this schema and try it out. Hopefully, others will have an impetus to try it as well.

I'm sure there's no good reason to use both at once, but there's no reason they couldn't both be used.

Happy to stay on the sidelines on this point, but I would observe that if one allows a situation, one should handle it; therefore it should be defined what happens if you do use both at once, or defined that you cannot. Or it should at least be defined that it is undefined, which hedges against someone in future coming up with a genius idea that we can put in here.