Describe 'delay' as event-generating operator

[henrikt-ma]

In #3640 it was noted that both delay and spatialDistribution should be event-generating, so that they can preserve discontinuities. This is particularly useful for dealing with discrete-valued signals, and opens up for also supporting non-Real signals.

We have now a test implementation of event-generating delay, and I'd say it works great. At the moment, describing delay accordingly is the only thing this PR does. For the much less frequently used spatialDistribution we don't have a test implementation at the moment, but would be ready to also go ahead and change the specification based on test implementations in other tools.

Some questions to think about when making this change:

• Do we need to give more detailed semantics than before?
• Do we also need to give expression variability rules for constant and evaluable expressions? For example, this could make delay(1.0, delayTime, delayMax) a constant expression.
• Are we ready to maintain the pseudo-code for spatialDistribution if we were to add event generating semantics?

[gkurzbach]

@henrikt-ma You looking for an implementation in other tools: in SimulationX both delay and spatialDistribution reproduce events, as there were present in the input, unless they are called within noEvent(). delay() is also allowed for non-Real input (but strings are not supported).

[gkurzbach]

It is fine for me.

[HansOlsson]

For me there are two different use cases:

• delay for Boolean, Integer or enumeration. To me they should generate events, and preserve the type of the operand (the latter requires the first outside of noEvent); as in Don't require discrete-time Real variables to be assigned in when-clauses #3247 (comment)
• delay for Reals with discontinuities; here I see a number of challenges (I will list them later), and to me the simplest solution is to allow (but not require) it to generate events to handle discontinuities.

I don't see a need to generalize spatialDistribution to non-Reals yet; but I see that the second case also applies for spatialDistribution.

The issues with events for Reals as I see it are:

• Currently those events are not observable within the Modelica language. Thus saying that tools can generate events seems like a good compromise of compatibility and efficiency.
• The discontinuities can be of varying size, at some size they are small enough that they can be ignored.
• Whether a "large" discontinuity matters also depend on how it impacts the rest of the system; sometimes even a large discontinuity might be insignificant.
• Additionally, it is possible to have a signal that is a sum of discontinuity and a continuous signal. Saying that it is non-discrete time and thus shouldn't generate events, seem counter-intuitive - but it could also be a minor discontinuity from discretization that should be ignored. This is similar to the idea of "input smoothing".

If we later refine the delay-operator's description we might revisit this.

[henrikt-ma]

I don't see a need to generalize spatialDistribution to non-Reals yet; but I see that the second case also applies for spatialDistribution.

While I don't have a test implementation for this, I don't see that it would be more difficult than supporting it for delay, and then I think it should be supported for the sake of completeness.

[henrikt-ma]

The issues with events for Reals as I see it are:

• Currently those events are not observable within the Modelica language. Thus saying that tools can generate events seems like a good compromise of compatibility and efficiency.
• The discontinuities can be of varying size, at some size they are small enough that they can be ignored.

If the Real expression being delayed is discrete-time, I would say it is a very bad idea to not preserve the discrete-time variability. For tiny discontinuities this leaves two options, namely to preserve also the tiny discontinuity, or to completely ignore the change in the value. I can see that the latter can be desirable in some situations, but I believe that this can be left as a tool optimization that we don't have to describe in more detail in the specification.

• Whether a "large" discontinuity matters also depend on how it impacts the rest of the system; sometimes even a large discontinuity might be insignificant.

Yes, so it is important that a tool doesn't forget to support noEvent also for delay and spatialDistribution.

• Additionally, it is possible to have a signal that is a sum of discontinuity and a continuous signal. Saying that it is non-discrete time and thus shouldn't generate events, seem counter-intuitive - but it could also be a minor discontinuity from discretization that should be ignored. This is similar to the idea of "input smoothing".

It was not my intention that the text should be interpreted this way. The way I see it, the key change here is that delay (and spatialDistribution) will preserve discrete-time expression variability, and in order to be able to do that we need to make these operators event-generating.

[henrikt-ma]

@henrikt-ma You looking for an implementation in other tools: in SimulationX both delay and spatialDistribution reproduce events, as there were present in the input, unless they are called within noEvent(). delay() is also allowed for non-Real input (but strings are not supported).

Since both you and @HansOlsson bring up the non-Real cases, I will add this to the PR.

[HansOlsson]

I don't see a need to generalize spatialDistribution to non-Reals yet; but I see that the second case also applies for spatialDistribution.

While I don't have a test implementation for this, I don't see that it would be more difficult than supporting it for delay, and then I think it should be supported for the sake of completeness.

My experience with handling spatialDistribution is that it is already quite complicated to get a good result; and since I see no real use case of non-Real spatialDistribution I think that will be an excessive burden for something for no real gain.

[henrikt-ma]

I just realized that delay is becoming the first event-generating function which is automatically vectorized. Perhaps we should consider whether also some of the other event-generating functions should do the same?

[HansOlsson]

The issues with events for Reals as I see it are:

• Currently those events are not observable within the Modelica language. Thus saying that tools can generate events seems like a good compromise of compatibility and efficiency.
• The discontinuities can be of varying size, at some size they are small enough that they can be ignored.

If the Real expression being delayed is discrete-time, I would say it is a very bad idea to not preserve the discrete-time variability. For tiny discontinuities this leaves two options, namely to preserve also the tiny discontinuity, or to completely ignore the change in the value. I can see that the latter can be desirable in some situations, but I believe that this can be left as a tool optimization that we don't have to describe in more detail in the specification.

For me there are some issues:

• Instead of specifying one thing, and relying on tools to optimize to get the desired result, I think it is better to be clear about the allowed behavior.
• Discrete-time variability of Reals as currently defined in Modelica is quite restrictive; so in many cases the Real expression that would seem "discrete-time" isn't. It may still makes sense to propagate the discontinuity, regardless of whether the expression is formally discrete-time or not.
• Looking at MSL I don't see any uses of delay for Reals that are sort of discrete-time (except for something that is really an enumeration).

Since it is a trivial to write cases where generating events for discrete variables will cause an infinite explosion in smaller and smaller discontinuities, I think we should be careful and not add something "just in case".

• Additionally, it is possible to have a signal that is a sum of discontinuity and a continuous signal. Saying that it is non-discrete time and thus shouldn't generate events, seem counter-intuitive - but it could also be a minor discontinuity from discretization that should be ignored. This is similar to the idea of "input smoothing".

It was not my intention that the text should be interpreted this way. The way I see it, the key change here is that delay (and spatialDistribution) will preserve discrete-time expression variability, and in order to be able to do that we need to make these operators event-generating.

I can understand that delay of a non-Real needs to be a discrete-time expression (even if the delayTime is continuous-time) to not generate non-discrete-time Integer (etc), but I don't see a similar need for Reals.

[HansOlsson]

As indicated should not require an event for Reals, and support non-Real.
Could change to "next significant discontinuous change" and saying that changes for non-Reals are always significant.

[HansOlsson]

I just realized that delay is becoming the first event-generating function which is automatically vectorized. Perhaps we should consider whether also some of the other event-generating functions should do the same?

I would say all of them should be automatically vectorized, and see also: #3728

[henrikt-ma]

How about expressing the value simply as expr(max(time.start, time - delayTime)) -- it would be an expression small enough to fit nicely as inline math? Do we need to point out that expr(time.start) shall be interpreted as just expr in the initialization problem, or is this a reason to avoid the rewrite with max?

[henrikt-ma]

• Looking at MSL I don't see any uses of delay for Reals that are sort of discrete-time (except for something that is really an enumeration).

This is not a good reason to not let delay preserve variability when it is an event generating operator, especially considering that it isn't difficult to implement.

I can understand that delay of a non-Real needs to be a discrete-time expression (even if the delayTime is continuous-time) to not generate non-discrete-time Integer (etc), but I don't see a similar need for Reals.

But this is not the way it will work for non-Real signals either. A non-discrete-time Integer expression will not become discrete time just because you send it through delay. Discrete-time variability shall neither be introduced nor destroyed by delay, only preserved (as long as you don't wrap the whole thing in noEvent(…)).

[HansOlsson]

How about expressing the value simply as expr(max(time.start, time - delayTime)) -- it would be an expression small enough to fit nicely as inline math? Do we need to point out that expr(time.start) shall be interpreted as just expr in the initialization problem, or is this a reason to avoid the rewrite with max?

Interesting question - it's more complicated, at least Dymola seem to x.start for delay(x, 1) - and not the value of x at the start-time. I think that should be discussed separately.

[henrikt-ma]

While I don't have a test implementation for this, I don't see that it would be more difficult than supporting it for delay, and then I think it should be supported for the sake of completeness.

My experience with handling spatialDistribution is that it is already quite complicated to get a good result; and since I see no real use case of non-Real spatialDistribution I think that will be an excessive burden for something for no real gain.

I am just extrapolating from what I learnt when working on delay. Indeed, the continuous-time Real case is a challenge to implement with good results, but if we allow spatialDistribution to preserve discontinuity I think all the heavy lifting will already have been done by the time this has been implemented for Real. I believe the rest needed to support non-Real types would be a simple matter of carrying over a few simple ideas from delay?

[henrikt-ma]

Interesting question - it's more complicated, at least Dymola seem to x.start for delay(x, 1) - and not the value of x at the start-time. I think that should be discussed separately.

OK. System Modeler is also making the (a little bit stretched) interpretation that x(time.start) actually means x.start in the case of delay semantics.

Edit: Reported this as #3733.

[HansOlsson]

But this is not the way it will work for non-Real signals either. A non-discrete-time Integer expression will not become discrete time just because you send it through delay. Discrete-time variability shall neither be introduced nor destroyed by delay, only preserved (as long as you don't wrap the whole thing in noEvent(…)).

Discrete-time Integer and discrete-time Real variables are quite different, and thus many Real expression that only change at events are not formally discrete-time.
Consider:

  discrete Integer i1=integer(time), final i2=-i1; // Both i1 and i2 are discrete-time
  discrete Real r1, final r2=-r1; // Error, r2 cannot be discrete as it isn't assigned in a when-clause

Additionally, I thought we wanted delay(i1, 2+sin(time), 3) to be discrete-time - even though the second argument is continuous time.

Added So a Real expression are often sort of discrete-time without being formally discrete-time; that's not the case for Integer expressions.

[henrikt-ma]

Additionally, I thought we wanted delay(i1, 2+sin(time), 3) to be discrete-time - even though the second argument is continuous time.

Yes, absolutely! The variability of i1 shall be preserved. The proposed rule should capture this:

Unless inside \lstinline!noEvent!: \lstinline!delay($x$, $\ldots$)!, if $x$ is a discrete-time expression.

Added So a Real expression are often sort of discrete-time without being formally discrete-time; that's not the case for Integer expressions.

I know this is the case, but to me there is only one natural way of defining the variability semantics of delay, and that is that the variability has nothing to do with the type of the delayed expression. Natural and easy to implement, not necessarily a super-useful feature considering the overall poor support for discrete-time Real we have due to #3247.

[HansOlsson]

Added So a Real expression are often sort of discrete-time without being formally discrete-time; that's not the case for Integer expressions.

I know this is the case, but to me there is only one natural way of defining the variability semantics of delay, and that is that the variability has nothing to do with the type of the delayed expression. Natural and easy to implement, not necessarily a super-useful feature considering the overall poor support for discrete-time Real we have due to #3247.

To me the important part for Real expressions is that discontinuities are propagated (with an event that cannot be observed internally), not that the call is discrete.
For non-Reals it has to have discrete variability - but I view those as different issues.

[henrikt-ma]

To me the important part for Real expressions is that discontinuities are propagated (with an event that cannot be observed internally), not that the call is discrete.

Oh, so you mean that we should state more clearly that discontinuities should be preserved in the non-discrete-time case? Clarifying that wouldn't sound like a bad idea to me.

[casella]

@henrikt-ma You looking for an implementation in other tools: in SimulationX both delay and spatialDistribution reproduce events, as there were present in the input, unless they are called within noEvent(). delay() is also allowed for non-Real input (but strings are not supported).

Also OMC reproduces events that were present at the input of delay() and spatialDistribution() when they reach the output. In principle, adding noEvent() should suppress them, but that's not implemented correctly yet. @AnHeuermann, this will need to be fixed.

To me the important part for Real expressions is that discontinuities are propagated (with an event that cannot be observed internally), not that the call is discrete.

Oh, so you mean that we should state more clearly that discontinuities should be preserved in the non-discrete-time case? Clarifying that wouldn't sound like a bad idea to me.

To me this was the primary motivation to add events to delay() and spatialDistribution(). If I apply a Real (non-discrete) input with a step change to a pure time delay, I expect to see the same discontinuous step change at the outlet, not some smoothed-out, noEvent() version of it.

[casella]

LGTM

[HansOlsson]

The remaining issue (apart from merge conflict) is the variability of delay for a parameter expression, in particular delay(x, time+1):

• 🚀 Give delay(x, delayTime) variability according to what can be proven based on the variabilities of the sub-expressions x and delayTime, namely that the variability of x alone is preserved. For example, delaying a parameter expression x results in a parameter expression.
• 👀 Make delay(x, delayTime) have at least discrete-time variability. In particular, delaying a parameter expression x results in a discrete-time expression (which happens to have a constant value determined at initialization).

Status (for parameter Real p;Integer i;):

Expression	M 3.6 variability	🚀 Variability	👀Variability	🎉Variability
delay(p, 1)	Parameter	Parameter	Discrete-time	Discrete time
delay(p, sin(time)+2, 3)	Continuous	Parameter	Discete time	Continuous
delay(i, 1)	Discrete-time	Discrete-time	Discrete time	Discrete time
delay(i, sin(time)+2, 3)	Continuous	Discrete-time	Discrete time	Continuous

Checking in Dymola 2026x I notice that we don't make it parameter-variability and no-one has noticed, so I don't think delaying parameters is used in any widely distributed libraries.

[henrikt-ma]

The column for 👀 does not capture the intention of at least discrete-time. When I wrote that, I would still have expected delay(x, time + 1) to be discrete-time when x is a parameter. Thus, the Continuous would belong to a forth column which is a new option:

• 🎉 Treat delay variability as an ordinary function call. For example, delay(x, time + 1) would be non-discrete-time even if x is a constant expression.

[HansOlsson]

Language group: Seems preference for delay only variability-depending on first argument.

[HansOlsson]

Here are some simple tests for this. (Updated.)

model TestDelay
  final constant Real C=1 "Re-used, so don't change value";
  constant Real C1=delay(C, 1);
  constant Real C2=delay(C, 2+sin(time), 3);
  final parameter Real p=2;
  parameter Real p1=delay(p, 1);
  parameter Real p2=delay(p, 2+sin(time), 3);
  Real x=delay(time, 1);
  Integer i=integer(time);
  Integer i1=delay(i, 0.5);
  Integer i2=delay(i, 2+sin(time), 3);
  Boolean b=rem(i1, 2)>0;
  Boolean b1=delay(b, 0.5);
  Boolean b2=delay(b, 2+sin(time), 3);
  Real z=noEvent(delay(i2, 0.5));
  Real arr[10]=1:10;
  Real testInt=arr[1+delay(i2, 0.5)]; // Requires that delayed integer is integer
equation
  assert(abs(p1-p)<1e-12, "Parameter delayed should be the same");
  assert(abs(p2-p)<1e-12, "Parameter delayed should be the same");
  assert(abs(C1-C)<1e-12, "Constant delayed should be the same");
  assert(abs(C2-C)<1e-12, "Constant delayed should be the same");
  assert(time<1 or abs(x-time+1)<0.1, "Delayed value");
  assert(abs(z-integer(z))<1e-12, "Should be actual integer");
  assert(abs(i2-integer(i2*C))<1e-12, "Should be actual integer");
  annotation (experiment(StopTime=5));
end TestDelay;

[henrikt-ma]

Here are some simple tests for this.

There are several violations due to not providing delayMax:

  constant Real C2=delay(C, 2+sin(time));
  parameter Real p2=delay(p, 2+sin(time));
  Integer i2=delay(i, 2+sin(time));
  Boolean b2=delay(b, 2+sin(time));

The should all be changed to something like this:

  constant Real C2=delay(C, 2+sin(time), 3.1);

Apart from that, the example simulates fine in System Modeler.

[HansOlsson]

Here are some simple tests for this.

There are several violations due to not providing delayMax:

Now updated.

[HansOlsson]

Seems ok now.

[HansOlsson]

The conflicts must, however, be resolved.

[henrikt-ma]

The conflicts must, however, be resolved.

Done.