v0.38.1

DynamicPPL v0.38.1

Diff since v0.38.0

Added DynamicPPL.from_linked_vec_transform and DynamicPPL.from_vec_transform methods for ProductNamedTupleDistribution.
This patch allows sampling from ProductNamedTupleDistribution in DynamicPPL models.

Merged pull requests:

• ProductNamedTupleDistribution compatibility (#1079) (@penelopeysm)

Closed issues:

• Implement to_vec_transform for ProductNamedTupleDistribution (#1077)

v0.38.0

DynamicPPL v0.38.0

Diff since v0.37.5

Breaking changes

Introduction of InitContext

DynamicPPL 0.38 introduces a new evaluation context, InitContext.
It is used to generate fresh values for random variables in a model.

Evaluation contexts are stored inside a DynamicPPL.Model object, and control what happens with tilde-statements when a model is run.
The two major leaf (basic) contexts are DefaultContext and, now, InitContext.
DefaultContext is the default context, and it simply uses the values that are already stored in the VarInfo object passed to the model evaluation function.
On the other hand, InitContext ignores values in the VarInfo object and inserts new values obtained from a specified source.
(It follows also that the VarInfo being used may be empty, which means that InitContext is now also the way to obtain a fresh VarInfo for a model.)

DynamicPPL 0.38 provides three flavours of initialisation strategies, which are specified as the second argument to InitContext:

• InitContext(rng, InitFromPrior()): New values are sampled from the prior distribution (on the right-hand side of the tilde).
• InitContext(rng, InitFromUniform(a, b)): New values are sampled uniformly from the interval [a, b], and then invlinked to the support of the distribution on the right-hand side of the tilde.
• InitContext(rng, InitFromParams(p, fallback)): New values are obtained by indexing into the p object, which can be a NamedTuple or Dict{<:VarName}. If a variable is not found in p, then the fallback strategy is used, which is simply another of these strategies. In particular, InitFromParams enables the case where different variables are to be initialised from different sources.

(It is possible to define your own initialisation strategy; users who wish to do so are referred to the DynamicPPL API documentation and source code.)

The main impact on the upcoming Turing.jl release is that, instead of providing initial values for sampling, the user will be expected to provide an initialisation strategy instead.
This is a more flexible approach, and not only solves a number of pre-existing issues with initialisation of Turing models, but also improves the clarity of user code.
In particular:

• When providing a set of fixed parameters (i.e. InitFromParams(p)), p must now either be a NamedTuple or a Dict. Previously Vectors were allowed, which is error-prone because the ordering of variables in a VarInfo is not obvious.
• The parameters in p must now always be provided in unlinked space (i.e., in the space of the distribution on the right-hand side of the tilde). Previously, whether a parameter was expected to be in linked or unlinked space depended on whether the VarInfo was linked or not, which was confusing.

Removal of SamplingContext

For developers working on DynamicPPL, InitContext now completely replaces what used to be SamplingContext, SampleFromPrior, and SampleFromUniform.
Evaluating a model with SamplingContext(SampleFromPrior()) (e.g. with DynamicPPL.evaluate_and_sample!!(model, VarInfo(), SampleFromPrior()) has a direct one-to-one replacement in DynamicPPL.init!!(model, VarInfo(), InitFromPrior()).
Please see the docstring of init!! for more details.
Likewise SampleFromUniform() can be replaced with InitFromUniform().
InitFromParams() provides new functionality which was previously implemented in the roundabout way of manipulating the VarInfo (e.g. using unflatten, or even more hackily by directly modifying values in the VarInfo), and then evaluating using DefaultContext.

The main change that this is likely to create is for those who are implementing samplers or inference algorithms.
The exact way in which this happens will be detailed in the Turing.jl changelog when a new release is made.
Broadly speaking, though, SamplingContext(MySampler()) will be removed so if your sampler needs custom behaviour with the tilde-pipeline you will likely have to define your own context.

Removal of DynamicPPL.Sampler

DynamicPPL.Sampler and all associated interface functions have also been removed entirely.
If you were using these, the corresponding replacements are:

• DynamicPPL.Sampler(S): just don't wrap S; but make sure S subtypes AbstractMCMC.AbstractSampler
• DynamicPPL.initialstep: directly implement AbstractMCMC.step and AbstractMCMC.step_warmup as per the AbstractMCMC interface
• DynamicPPL.loadstate: Turing.loadstate (will be introduced in the next version)
• DynamicPPL.default_chain_type: removed, just use the chain_type keyword argument directly
• DynamicPPL.initialsampler: Turing.Inference.init_strategy (will be introduced in the next version; note that this function must return an AbstractInitStrategy, see above for explanation)
• DynamicPPL.default_varinfo: Turing.Inference.default_varinfo (will be introduced in the next version)
• DynamicPPL.TestUtils.test_sampler and related methods: removed, please implement your own testing utilities as needed

Simplification of the tilde-pipeline

There are now only two functions in the tilde-pipeline that need to be overloaded to change the behaviour of tilde-statements, namely, tilde_assume!! and tilde_observe!!.
Other functions such as tilde_assume and assume (and their observe counterparts) have been removed.

Note that this was effectively already the case in DynamicPPL 0.37 (where they were just wrappers around each other).
The separation of these functions was primarily implemented to avoid performing extra work where unneeded (e.g. to not calculate the log-likelihood when PriorContext was being used). This functionality has since been replaced with accumulators (see the 0.37 changelog for more details).

Removal of the "del" flag

Previously VarInfo (or more correctly, the Metadata object within a VarInfo), had a flag called "del" for all variables. If it was set to true the variable was to be overwritten with a new value at the next evaluation. The new InitContext and related changes above make this flag unnecessary, and it has been removed.

The only flag other than "del" that Metadata ever used was "trans". Thus the generic functions set_flag!, unset_flag! and is_flagged! have also been removed in favour of more specific ones. We've also used this opportunity to name the "trans" flag and the corresponding istrans function to be more explicit. The new, exported interface consists of the is_transformed and set_transformed!! functions.

Removal of resume_from

The resume_from=chn keyword argument to sample has been removed; please use the initial_state argument instead.
loadstate will be exported from Turing in the next release of Turing.

Change of output type for pointwise_logdensities

The functions pointwise_prior_logdensities, pointwise_logdensities, and pointwise_loglikelihoods when called on MCMCChains.Chains objects, now return new MCMCChains.Chains objects by default, instead of dictionaries of matrices.

If you want the old behaviour, you can pass OrderedDict as the third argument, i.e., pointwise_logdensities(model, chain, OrderedDict).

Other changes

predict(model, chain; include_all)

The include_all keyword argument for predict now works even when no RNG is specified (previously it would only work when an RNG was explicitly passed).

DynamicPPL.setleafcontext(model, context)

This convenience method has been added to quickly modify the leaf context of a model.

Reimplementation of functions using InitContext

A number of functions have been reimplemented and unified with the help of InitContext.
In particular, this release brings substantial performance improvements for returned and predict.
Their APIs are the same.

Upstreaming of VarName functionality

The implementation of the varname_leaves and varname_and_value_leaves functions have been moved to AbstractPPL.jl.
Their behaviour is otherwise identical, and they are still accessible from the DynamicPPL module (though still not exported).

Merged pull requests:

• v0.38 (#1018) (@penelopeysm)
• Remove Sampler and its interface (#1037) (@penelopeysm)
• Separate context code into smaller files; remove some dead code (#1050) (@penelopeysm)
• add Enzyme benchmarks and incorporate some CompatHelper bits (#1056) (@penelopeysm)
• Fixes for Turing 0.41 (#1057) (@penelopeysm)
• Delete the "del" flag (#1058) (@mhauru)
• no coveralls (#1059) (@penelopeysm)
• Replace Metadata.flags with Metadata.trans (#1060) (@mhauru)
• Remove resume_from and default_chain_type (#1061) (@penelopeysm)
• Allow more flexible initial_params (#1064) (@penelopeysm)
• Bump JET and SpecialFunctions test compat (#1065) (@penelopeysm)
• Fix CI for 1.12 (#1067) (@penelopeysm)
• fix include_all kwarg for predict, improve perf (#1068) (@penelopeysm)
• don't run Turing integration test on 1.12.0 (#1069) (@penelopeysm)
• Change pointwise_logdensities default key type to VarName (#1071) (@mhauru)
• Remove VarInfo internal docs from navigation (#1073) (@mhauru)
• Fix DynamicPPL / MCMCChains methods (#1076) (@penelopeysm)

Closed issues:

• Add scale parameter to SampleFromUniform (#375)
• Removed ignorable argument from set_flag! (#683)
• Return an AbstractChains object from appliying pointwise_logdensities to Abstractchains? (#688)
• Usage of VarName as keys (#712)
• Support using OrderedDict for initialization, also improve error message (#774)
• Inline (and more carefully use) islinked inside initialize_parameters!! (#797)
• SampleFromPrior, etc. cleanup (#859)
• tovec causing an error when sampling a Product named tuple distribution...

v0.37.5

DynamicPPL v0.37.5

Diff since v0.37.4

A minor optimisation for Enzyme AD on DynamicPPL models.

Merged pull requests:

• Don't use closure for enzyme (#1048) (@wsmoses)

v0.37.4

DynamicPPL v0.37.4

Diff since v0.37.3

An extension for MarginalLogDensities.jl has been added.

Loading DynamicPPL and MarginalLogDensities now provides the DynamicPPL.marginalize function to marginalise out variables from a model.
This is useful for averaging out random effects or nuisance parameters while improving inference on fixed effects/parameters of interest.
The marginalize function returns a MarginalLogDensities.MarginalLogDensity, a function-like callable struct that returns the approximate log-density of a subset of the parameters after integrating out the rest of them.
By default, this uses the Laplace approximation and sparse AD, making the marginalisation computationally very efficient.
Note that the Laplace approximation relies on the model being differentiable with respect to the marginalised variables, and that their posteriors are unimodal and approximately Gaussian.

Please see the MarginalLogDensities documentation and the new Marginalisation section of the DynamicPPL documentation for further information.

Merged pull requests:

• pretty tables 3 (#1026) (@penelopeysm)
• MarginalLogDensities extension (#1036) (@penelopeysm)
• pin JET to <= 0.10.6 (#1046) (@penelopeysm)

Closed issues:

• Enzyme doesn't like accumulators (#947)
• JET 1.12 tests are failing (#1040)

v0.37.3

DynamicPPL v0.37.3

Diff since v0.37.2

Prevents inlining of DynamicPPL.istrans with Enzyme, which allows Enzyme to differentiate models where VarNames have the same symbol but different types.

Merged pull requests:

• InitContext, part 4 - Use init!! to replace evaluate_and_sample!!, predict, returned, and initialize_values (#984) (@penelopeysm)
• Refactor inactive_noinl to inactive for istrans (#1043) (@wsmoses)

v0.37.2

DynamicPPL v0.37.2

Diff since v0.37.1

Make the resume_from keyword work for multiple-chain (parallel) sampling as well. Prior to this version, it was silently ignored. Note that to get the correct behaviour you also need to have a recent version of MCMCChains (v7.2.1).

Merged pull requests:

• InitContext, part 3 - Introduce InitContext (#981) (@penelopeysm)
• Remove unneeded argument from default_accumulators() (#1025) (@penelopeysm)
• use varname_leaves from AbstractPPL instead (#1030) (@penelopeysm)
• tidy occurrences of varname_leaves as well (#1031) (@penelopeysm)
• Bump actions/checkout from 4 to 5 (#1032) (@dependabot[bot])
• Fix resume_from for parallel sampling (#1035) (@penelopeysm)

Closed issues:

• VarInfo with custom accumulators (#744)
• varname_and_value_leaves etc should be in AbstractPPL (#1028)
• resume_from silently no-ops with multiple-chain sampling (#1033)

v0.37.1

DynamicPPL v0.37.1

Diff since v0.37.0

Update DynamicPPLMooncakeExt to work with Mooncake 0.4.147.

Merged pull requests:

• Fix MooncakeExt for 0.4.147 (#1021) (@penelopeysm)

v0.37.0

DynamicPPL v0.37.0

Diff since v0.36.15

DynamicPPL 0.37 comes with a substantial reworking of its internals.
Fundamentally, there is no change to the actual modelling syntax: if you are a Turing.jl user, for example, this release will not affect you too much (apart from the changes to @addlogprob!).
Any such changes will be covered separately in the Turing.jl changelog when a release is made.
However, if you are a package developer or someone who uses DynamicPPL's functionality directly, you will notice a number of changes.

To avoid overwhelming the reader, we begin by listing the most important, user-facing changes, before explaining the changes to the internals in more detail.

Note that virtually all changes listed here are breaking.

Public-facing changes

Submodel macro

The @submodel macro is fully removed; please use to_submodel instead.

DynamicPPL.TestUtils.AD.run_ad

The three keyword arguments, test, reference_backend, and expected_value_and_grad have been merged into a single test keyword argument.
Please see the API documentation for more details.
(The old test=true and test=false values are still valid, and you only need to adjust the invocation if you were explicitly passing the reference_backend or expected_value_and_grad arguments.)

There is now also an rng keyword argument to help seed parameter generation.

Finally, instead of specifying value_atol and grad_atol, you can now specify atol and rtol which are used for both value and gradient.
Their semantics are the same as in Julia's isapprox; two values are equal if they satisfy either atol or rtol.

DynamicPPL.TestUtils.check_model

You now need to explicitly pass a VarInfo argument to check_model and check_model_and_trace.
Previously, these functions would generate a new VarInfo for you (using an optionally provided rng).

Evaluating model log-probabilities in more detail

Previously, during evaluation of a model, DynamicPPL only had the capability to store a single log probability (logp) field.
DefaultContext, PriorContext, and LikelihoodContext were used to control what this field represented: they would accumulate the log joint, log prior, or log likelihood, respectively.

In this version, we have overhauled this quite substantially.
The technical details of exactly how this is done is covered in the 'Accumulators' section below, but the upshot is that the log prior, log likelihood, and log Jacobian terms (for any linked variables) are separately tracked.

Specifically, you will want to use the following functions to access these log probabilities:

• getlogprior(varinfo) to get the log prior. Note: This version introduces new, more consistent behaviour for this function, in that it always returns the log-prior of the values in the original, untransformed space, even if the varinfo has been linked.
• getloglikelihood(varinfo) to get the log likelihood.
• getlogjoint(varinfo) to get the log joint probability. Note: Similar to getlogprior, this function now always returns the log joint of the values in the original, untransformed space, even if the varinfo has been linked.

If you are using linked VarInfos (e.g. if you are writing a sampler), you may find that you need to obtain the log probability of the variables in the transformed space.
To this end, you can use:

• getlogjac(varinfo) to get the log Jacobian of the link transforms for any linked variables.
• getlogprior_internal(varinfo) to get the log prior of the variables in the transformed space.
• getlogjoint_internal(varinfo) to get the log joint probability of the variables in the transformed space.

Since transformations only apply to random variables, the likelihood is unaffected by linking.

Removal of PriorContext and LikelihoodContext

Following on from the above, a number of DynamicPPL's contexts have been removed, most notably PriorContext and LikelihoodContext.
Although these are not the only exported contexts, we consider unlikely that anyone was using other contexts manually: if you have a question about contexts other than these, please continue reading the 'Internals' section below.

If you were evaluating a model with PriorContext, you can now just evaluate it with DefaultContext, and instead of calling getlogp(varinfo), you can call getlogprior(varinfo) (and similarly for the likelihood).

If you were constructing a LogDensityFunction with PriorContext, you can now stick to DefaultContext.
LogDensityFunction now has an extra field, called getlogdensity, which represents a function that takes a VarInfo and returns the log density you want.
Thus, if you pass getlogprior_internal as the value of this parameter, you will get the same behaviour as with PriorContext.
(You should consider whether your use case needs the log prior in the transformed space, or the original space, and use (respectively) getlogprior_internal or getlogprior as needed.)

The other case where one might use PriorContext was to use @addlogprob! to add to the log prior.
Previously, this was accomplished by manually checking __context__ isa DynamicPPL.PriorContext.
Now, you can write @addlogprob (; logprior=x, loglikelihood=y) to add x to the log-prior and y to the log-likelihood.

Removal of order and num_produce

The VarInfo type used to carry with it:

• num_produce, an integer which recorded the number of observe tilde-statements that had been evaluated so far; and
• order, an integer per VarName which recorded the value of num_produce at the time that the variable was seen.

These fields were used in particle samplers in Turing.jl.
In DynamicPPL 0.37, these fields and the associated functions have been removed:

• get_num_produce
• set_num_produce!!
• reset_num_produce!!
• increment_num_produce!!
• set_retained_vns_del!
• setorder!!

Because this is one of the more arcane features of DynamicPPL, some extra explanation is warranted.

num_produce and order, along with the del flag in VarInfo, were used to control whether new values for variables were sampled during model execution.
For example, the particle Gibbs method has a reference particle, for which variables are never resampled.
However, if the reference particle is forked (i.e., if the reference particle is selected by a resampling step multiple times and thereby copied), then the variables that have not yet been evaluated must be sampled anew to ensure that the new particle is independent of the reference particle.

Previousy, this was accomplished by setting the del flag in the VarInfo object for all variables with order greater or equal to than num_produce.
Note that setting the del flag does not itself trigger a new value to be sampled; rather, it indicates that a new value should be sampled if the variable is encountered again.
This Turing.jl PR changes the implementation to set the del flag for all variables in the VarInfo.
Since the del flag only makes a difference when encountering a variable, this approach is entirely equivalent as long as the same variable is not seen multiple times in the model.
The interested reader is referred to that PR for more details.

Internals

Accumulators

This release overhauls how VarInfo objects track variables such as the log joint probability. The new approach is to use what we call accumulators: Objects that the VarInfo carries on it that may change their state at each tilde_assume!! and tilde_observe!! call based on the value of the variable in question. They replace both variables that were previously hard-coded in the VarInfo object (logp and num_produce) and some contexts. This brings with it a number of breaking changes:

• PriorContext and LikelihoodContext no longer exist. By default, a VarInfo tracks both the log prior and the log likelihood separately, and they can be accessed with getlogprior and getloglikelihood. If you want to execute a model while only accumulating one of the two (to save clock cycles), you can do so by creating a VarInfo that only has one accumulator in it, e.g. varinfo = setaccs!!(varinfo, (LogPriorAccumulator(),)).
• MiniBatchContext does not exist anymore. It can be replaced by creating and using a custom accumulator that replaces the default LikelihoodContext. We may introduce such an accumulator in DynamicPPL in the future, but for now you'll need to do it yourself.
• tilde_observe and observe have been removed. tilde_observe!! still exists, and any contexts should modify its behaviour. We may further rework the call stack under tilde_observe!! in the near future.
• tilde_assume no longer returns the log density of the current assumption as its second return value. We may further rework the tilde_assume!! call stack as well.
• For literal observation statements like 0.0 ~ Normal(blahblah) we used to call tilde_observe!! without the vn argument. This method no longer exists. Rather we call tilde_observe!! with vn set to nothing.
• @addlogprob! now always adds to the log likelihood. Previously it added to the log probability that the execution context specified, e.g. the log prior when using PriorContext.
• getlogp now returns a NamedTuple with keys logprior and loglikelihood. If you want the log joint probability, which is what getlogp used to return, use getlogjoint.
• Correspondingly setlogp!! and acclogp!! should now be called with a NamedTuple with keys logprior and loglikelihood. The acclogp!! method with a single scalar value has been deprecated and falls b...

v0.36.15

DynamicPPL v0.36.15

Diff since v0.36.14

Bumped minimum Julia version to 1.10.8 to avoid potential crashes with Core.Compiler.widenconst (which Mooncake uses).

Merged pull requests:

• Improve API for AD testing (#964) (@penelopeysm)

Closed issues:

• Data subsampling without reinstantiating the model (#208)
• Possibly confusing .~ meaning (#435)
• InferenceObjects integration (#464)
• Convenience macros to use within @model (#714)
• evaluate!! shenanigans (#720)
• to_submodel type instabilities (#794)
• Composing models with latent and to_distribution (#802)

v0.36.14

DynamicPPL v0.36.14

Diff since v0.36.13

Added compatibility with AbstractPPL@0.12.

Closed issues:

• change defaults: @addlogprob! not included in PriorContext() (#580)
• (Maybe) Find a way to avoid initialising with undef values (#784)
• values_as(vi, NamedTuple) doesn't preserve structure (#814)
• Is there a rationale for why a VarInfo should have some but not all variables linked? (#968)