[no ci] user guide: add stub on additional topics

Author: vpratz

docsrc/source/user_guide/approximators.ipynb

@@ -0,0 +1,35 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "2eb7a0f2-6017-4a45-90e4-d149d10193b7",
+   "metadata": {},
+   "source": [
+    "# Approximators\n",
+    "\n",
+    "Neural approximators provide an approximation of a distribution or a value. To achieve this, they combine the things we have discussed in the previous chapters: simulated data, adapters, summary networks and inference networks. Approximators are at the heart of BayesFlow, as they organize the different components and provide the `fit()` function used for training."
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.11"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docsrc/source/user_guide/diagnostics.ipynb

@@ -0,0 +1,62 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "71f2af04-8c30-44d7-a0c9-377ffd0d3c75",
+   "metadata": {},
+   "source": [
+    "# Diagnostics and Visualizations\n",
+    "\n",
+    "There are many factors that influence whether training succeeds and how well we can approximate a target. In this light, checking the results and diagnosing potential problems is an important part of the workflow.\n",
+    "\n",
+    "## Loss\n",
+    "\n",
+    "While the loss cannot show that the training has succeeded, it can indicate that something has gone wrong. Warning signs are an unstable loss with large upward jumps, and a lack of convergence (the loss still changes significantly at the end of training). We recommend to supply a validation dataset to the training, to diagnose potential overfitting. You can plot the loss using the {py:func}`bayesflow.diagnostics.loss` function.\n",
+    "\n",
+    "## Posterior\n",
+    "\n",
+    "For inference on simulated data, we can plot the posterior alongside the ground truth values. This can serve as a diagnostic for whether the approximator has learned to approximate the true posteriors well enough. The {py:func}`~bayesflow.diagnostics.pairs_posterior` function displays a set of one- and two-dimensional marginal posterior distributions.\n",
+    "\n",
+    "## Recovery\n",
+    "\n",
+    "For inference on simulated data, we can visualize how well the ground truth values are recovered over a larger number of datasets. {py:func}`~bayesflow.diagnostics.recovery` is a convencience function for this kind of plot.\n",
+    "\n",
+    "## Simulation-based Calibration (SBC)\n",
+    "\n",
+    "Simulation-based calibration provides an indication of the posterior approximations' accuracy, without requiring access to the ground-truth posterior. In short, if the true values are simulated from the prior used during inference, we would expect the rank of the true parameter value to be uniformly distributed from 1 to `num_samples`. There are multiple graphical methods that use this property for diagnostics. For example, we can use histograms together with an uncertainty band within which we would expect the histogram bars to be if the rank statistics were indeed uniform. This plot is provided by the {py:func}`~bayesflow.diagnostics.calibration_histogram` function.\n",
+    "\n",
+    "SBC histograms have some drawbacks on how the confidence bands are computed, so we recommend using another kind of plot that is based on the empirical cumulative distribution function (ECDF). For the ECDF, we can compute better confidence bands than for histograms, so the SBC ECDF plot is usually preferable. [This SBC interpretation guide by Martin Modrák](https://hyunjimoon.github.io/SBC/articles/rank_visualizations.html) gives further background information and also practical examples of how to interpret the SBC plots. To display SBC ECDF plots, use the {py:func}`~bayesflow.diagnostics.calibration_ecdf` function.\n",
+    "\n",
+    "## Posterior Contraction & z-Score\n",
+    "\n",
+    "After having convinced us that the posterior approximation are overall reasonable, we can check how much and what kind of information in the data we encode in the posterior. Specifically, we might want to look at two interesting scores:\n",
+    "\n",
+    "- The posterior contraction, which measures how much smaller the posterior variance is relative to the prior variance (higher values indicate more contraction relative to the prior).\n",
+    "- The posterior z-score which indicates the standardized difference between the posterior mean and the true parameter value. Since the posterior z-score requires the true parameter values, it can only be computed in simulated data settings.\n",
+    "\n",
+    "The {py:func}`~bayesflow.diagnostics.z_score_contraction` function provides a combined plot of both."
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.11"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docsrc/source/user_guide/index.md

@@ -14,4 +14,8 @@ generative_models.ipynb
 data_processing.ipynb
 summary_networks.ipynb
 inference_networks.ipynb
+approximators.ipynb
+workflows.ipynb
+diagnostics.ipynb
+saving_loading.ipynb
 ```

docsrc/source/user_guide/saving_loading.ipynb

@@ -0,0 +1,39 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "743f4071-009d-48e6-bc27-9785f37071c4",
+   "metadata": {},
+   "source": [
+    "# Saving & Loading Models\n",
+    "\n",
+    "Saving and loading of models takes place via our backend, [Keras 3](https://keras.io/). Objects that can be saved have a `save` method, which allows saving to a `.keras` file.\n",
+    "\n",
+    "The [`keras.saving.load_model`](https://keras.io/api/models/model_saving_apis/model_saving_and_loading/#load_model-function) function can be used to load the stored models. There is a lot more to say about this topic. For now, refer to the respective [Keras guide](https://keras.io/guides/serialization_and_saving/).\n",
+    "\n",
+    "**Important:** We try to avoid it, but changes to model architectures can lead to a situation where the model is not successfully loaded after updating BayesFlow. This can also happen silently (i.e., without an error/warning being shown), so we recommend to conduct a quick sanity check on the model outputs after loading. In practice, pinning the BayesFlow version for each project can also help to avoid problems in this regard."
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.11"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

docsrc/source/user_guide/workflows.ipynb

@@ -0,0 +1,54 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "2eb7a0f2-6017-4a45-90e4-d149d10193b7",
+   "metadata": {},
+   "source": [
+    "# Workflows\n",
+    "\n",
+    "Workflows are an abstraction on top of the approximator, that expose methods for training and inference in a more abstract, and therefore simplified, fashion.\n",
+    "\n",
+    "## BasicWorkflow\n",
+    "\n",
+    "For now, {py:class}`~bayesflow.workflows.BasicWorkflow` is the only available workflow. In its most basic form, you provide it with four things: simulator, adapter, inference network and summary network (if you use one).\n",
+    "\n",
+    "```python\n",
+    "workflow = bf.BasicWorkflow(\n",
+    "    simulator=simulator,\n",
+    "    adapter=adapter,\n",
+    "    inference_network=inference_network,\n",
+    "    summary_network=summary_network,\n",
+    ")\n",
+    "```\n",
+    "\n",
+    "You can then use the {py:meth}`~bayesflow.workflows.BasicWorkflow.fit_online` method to run the training. After training, the you can sample from the workflow using {py:meth}`~bayesflow.workflows.BasicWorkflow.sample`.\n",
+    "\n",
+    "Note that after the training, the work of the workflow is done, and you can just use the trained approximator (accessible via `workflow.approximator`) for downstream tasks. If you want to save you trained output, you would use the `approximator` as well, i.e., store it using `workflow.approximator.save(filepath=filepath)`. You can then use {py:meth}`~bayesflow.approximators.ContinuousApproximator.sample` and related methods directly from the approximator.\n",
+    "\n",
+    "{py:class}`~bayesflow.workflows.BasicWorkflow` is highly configurable, refer to the API reference for details. For practical usage, take a look at the {doc}`../examples`."
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.11"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}