bayesmith.evaluation.loo
Source-derived signatures and contracts. See the reference index for coverage and conventions.
Module contract
PSIS-LOO for a result that carries a pointwise log density (§0.9). Leave-one-out cross-validation by Pareto-smoothed importance sampling is a mature, general statistic, so §1.5 puts it upstream: ``arviz.loo`` computes it and this module owns only the three things arviz cannot know -- which result is being judged, whether LOO applies to it at all, and how the answer is filed. Nothing here recomputes an elpd, a Pareto k or a reliability rule. **The chain axis, and why it is supplied here rather than in the export.** ``az.loo`` reads arviz's ``chain``/``draw`` sample dims, and probe_28 §2 measured what happens without them: ``AttributeError: 'DataArray' object has no attribute 'chain'``. R2's export is nevertheless right to leave an iid result on one flat ``draw`` axis -- :class:`~bayesmith.artifacts.results.DrawsPosterior`'s ``chain_shape is None`` means "this run had no chain structure to diagnose", and manufacturing one there would hand a chain diagnostic something to split that never ran. So the ``(1, n)`` shape is a CONSUMER's adaptation to one upstream's axis convention, it lives in the consumer, and R2's export semantics are untouched. A result that DID run chains is read at the shape it recorded. **The verdict is arviz's own.** A high Pareto k says the importance-sampling ESTIMATE is unreliable; it does not say the model is wrong. So a warning maps to ABSTAIN, never FAIL (§0.2), and the rule that decides it is arviz's ``good_k`` -- this module adds no second cutoff of its own. That is §0.8's "one decision, one home" applied to a threshold: two Pareto-k rules would agree right up until somebody tuned one of them. **An importance-weighted sample is declined, not averaged over.** R2's export projects draws, replicated draws, pointwise log density and observed data; it carries no weights, and ``arviz.loo`` has none to honour. Handed a :class:`~bayesmith.artifacts.results.WeightedDrawsPosterior` it would therefore cross-validate the PROPOSAL rather than the posterior, and it would return the same numbers whether the weights were near-uniform or collapsed onto one draw -- measured on ``radiometer``: an identical elpd, se, p_loo and max Pareto k for a sample whose weight ESS is 1997.3 of 2000 and for the same sample with a weight ESS of 1.000. §0.2's contract is that a check which cannot apply says so, so this is UNVERIFIABLE / ABSTAIN with a finding naming the loss, and the branch sits BEFORE the arviz import so the answer is a property of the artifact rather than of the environment. **WAIC is not here, and its absence is recorded rather than patched.** ``hasattr(arviz, "waic")`` is False in 1.3.0 (probe_28 §2). §1.5 lists WAIC among the statistics this package reuses instead of reimplementing, so R3's "LOO/WAIC" ships as LOO-PSIS and ``tests/evaluation/test_loo.py`` holds the absence, so that a later arviz growing one is news rather than a silent divergence. **ArviZ stays optional.** It is imported inside :func:`loo_report`, never at module scope, so a clone without the dev extra gets an UNVERIFIABLE report (§7.3) rather than an ImportError at import time. **The version a report records is the ESTIMATOR's, and ``arviz`` is not it.** ``az.loo`` is a re-export: ``az.loo.__module__`` is ``arviz_stats.loo.loo``, so the elpd, ``se``, ``p_loo`` and Pareto k above are ``arviz-stats``'s numbers and ``az.__version__`` is a fact about the umbrella. Measured 2026-09-06: arviz-stats 1.3.1 -> 1.3.2 changed ``se`` by ``sqrt(n / (n - 1))`` -- a ddof 0->1 change that broke two pinned constants in ``tests/evaluation/test_loo.py`` -- while ``az.__version__`` read ``1.3.0`` on both sides. A report stored before that release and one stored after were therefore indistinguishable by their own provenance, on exactly the number that moved. So ``loo_estimator_version`` names the distribution that provides the callable that ran, and ``arviz_version`` stays beside it recording what it actually is. **That provenance is READ off the callable, not spelled.** A constant ``"arviz-stats"`` here would be a claim about today's arviz, and this finding exists because the last such claim went stale; a later arviz that computes ``loo`` in-house again would walk straight past it. :func:`_estimator_origin` asks ``az.loo`` where it lives instead, so the finding is true by construction rather than by maintenance.
REPORT_KIND
Defined in bayesmith.evaluation.loo, line 100.
REPORT_KIND = 'loo_psis'loo_report
Defined in bayesmith.evaluation.loo, line 264.
loo_report(result: Any, *, graph: Graph, chain_shape: tuple[int, int] | None=None, source_posterior: Any=None) -> EvaluationReportJudge one result's PSIS-LOO, via ``arviz.loo`` over R2's export.
Args:
result: a :class:`~bayesmith.artifacts.results.PosteriorResult` or
:class:`~bayesmith.artifacts.results.PredictiveResult`.
graph: the model the result came from, handed on to the export for the
observed data. It is READ, never modified -- this layer evaluates
results and does not touch the model (§2.4).
chain_shape: ``(chains, draws)`` to override what the result recorded.
A predictive result carries no chain shape of its own, so a caller
holding the source posterior's may supply it here; left out, an
unchained result is exported as one chain.
source_posterior: the :class:`~bayesmith.artifacts.results.PosteriorResult`
a predictive result came from, read ONLY for its representation --
it is what says whether the sample is importance-weighted, and a
``source_posterior_ref`` is a pointer, not a representation. It
deliberately does not supply ``chain_shape``: that override already
has a home above, and one argument quietly feeding two decisions is
how the second one stops being visible.
Returns:
An :class:`~bayesmith.artifacts.reports.EvaluationReport` of kind
``loo_psis``, filed against ``result``:
* APPLICABLE / PASS -- arviz reported no reliability warning.
* APPLICABLE / ABSTAIN -- arviz warned. The ESTIMATE is unreliable,
which is not a claim about the model (§0.2).
* INAPPLICABLE / ABSTAIN -- the result carries no pointwise log
density, so there is no LOO to compute.
* UNVERIFIABLE / ABSTAIN -- the sample is importance-weighted, and
R2's export carries no weights for arviz to honour.
* UNVERIFIABLE / ABSTAIN -- arviz is not installed (§7.3).