bayesmithDocumentation0.10.0 source

bayesmith.evaluation.loo

Source-derived signatures and contracts. See the reference index for coverage and conventions.

Module contract
PSIS-LOO for a result that carries a pointwise log density (§0.9).

Leave-one-out cross-validation by Pareto-smoothed importance sampling is a
mature, general statistic, so §1.5 puts it upstream: ``arviz.loo`` computes it
and this module owns only the three things arviz cannot know -- which result is
being judged, whether LOO applies to it at all, and how the answer is filed.
Nothing here recomputes an elpd, a Pareto k or a reliability rule.

**The chain axis, and why it is supplied here rather than in the export.**
``az.loo`` reads arviz's ``chain``/``draw`` sample dims, and probe_28 §2
measured what happens without them: ``AttributeError: 'DataArray' object has
no attribute 'chain'``. R2's export is nevertheless right to leave an iid
result on one flat ``draw`` axis --
:class:`~bayesmith.artifacts.results.DrawsPosterior`'s ``chain_shape is None``
means "this run had no chain structure to diagnose", and manufacturing one
there would hand a chain diagnostic something to split that never ran. So the
``(1, n)`` shape is a CONSUMER's adaptation to one upstream's axis convention,
it lives in the consumer, and R2's export semantics are untouched. A result
that DID run chains is read at the shape it recorded.

**The verdict is arviz's own.** A high Pareto k says the importance-sampling
ESTIMATE is unreliable; it does not say the model is wrong. So a warning maps
to ABSTAIN, never FAIL (§0.2), and the rule that decides it is arviz's
``good_k`` -- this module adds no second cutoff of its own. That is §0.8's
"one decision, one home" applied to a threshold: two Pareto-k rules would
agree right up until somebody tuned one of them.

**An importance-weighted sample is declined, not averaged over.** R2's export
projects draws, replicated draws, pointwise log density and observed data; it
carries no weights, and ``arviz.loo`` has none to honour. Handed a
:class:`~bayesmith.artifacts.results.WeightedDrawsPosterior` it would
therefore cross-validate the PROPOSAL rather than the posterior, and it would
return the same numbers whether the weights were near-uniform or collapsed
onto one draw -- measured on ``radiometer``: an identical elpd, se, p_loo and
max Pareto k for a sample whose weight ESS is 1997.3 of 2000 and for the same
sample with a weight ESS of 1.000. §0.2's contract is that a check which
cannot apply says so, so this is UNVERIFIABLE / ABSTAIN with a finding naming
the loss, and the branch sits BEFORE the arviz import so the answer is a
property of the artifact rather than of the environment.

**WAIC is not here, and its absence is recorded rather than patched.**
``hasattr(arviz, "waic")`` is False in 1.3.0 (probe_28 §2). §1.5 lists WAIC
among the statistics this package reuses instead of reimplementing, so R3's
"LOO/WAIC" ships as LOO-PSIS and ``tests/evaluation/test_loo.py`` holds the
absence, so that a later arviz growing one is news rather than a silent
divergence.

**ArviZ stays optional.** It is imported inside :func:`loo_report`, never at
module scope, so a clone without the dev extra gets an UNVERIFIABLE report
(§7.3) rather than an ImportError at import time.

**The version a report records is the ESTIMATOR's, and ``arviz`` is not it.**
``az.loo`` is a re-export: ``az.loo.__module__`` is ``arviz_stats.loo.loo``,
so the elpd, ``se``, ``p_loo`` and Pareto k above are ``arviz-stats``'s
numbers and ``az.__version__`` is a fact about the umbrella. Measured
2026-09-06: arviz-stats 1.3.1 -> 1.3.2 changed ``se`` by ``sqrt(n / (n - 1))``
-- a ddof 0->1 change that broke two pinned constants in
``tests/evaluation/test_loo.py`` -- while ``az.__version__`` read ``1.3.0`` on
both sides. A report stored before that release and one stored after were
therefore indistinguishable by their own provenance, on exactly the number
that moved. So ``loo_estimator_version`` names the distribution that provides
the callable that ran, and ``arviz_version`` stays beside it recording what it
actually is.

**That provenance is READ off the callable, not spelled.** A constant
``"arviz-stats"`` here would be a claim about today's arviz, and this finding
exists because the last such claim went stale; a later arviz that computes
``loo`` in-house again would walk straight past it. :func:`_estimator_origin`
asks ``az.loo`` where it lives instead, so the finding is true by
construction rather than by maintenance.

REPORT_KIND

Defined in bayesmith.evaluation.loo, line 100.

REPORT_KIND = 'loo_psis'

loo_report

Defined in bayesmith.evaluation.loo, line 264.

loo_report(result: Any, *, graph: Graph, chain_shape: tuple[int, int] | None=None, source_posterior: Any=None) -> EvaluationReport
Judge one result's PSIS-LOO, via ``arviz.loo`` over R2's export.

Args:
    result: a :class:`~bayesmith.artifacts.results.PosteriorResult` or
        :class:`~bayesmith.artifacts.results.PredictiveResult`.
    graph: the model the result came from, handed on to the export for the
        observed data. It is READ, never modified -- this layer evaluates
        results and does not touch the model (§2.4).
    chain_shape: ``(chains, draws)`` to override what the result recorded.
        A predictive result carries no chain shape of its own, so a caller
        holding the source posterior's may supply it here; left out, an
        unchained result is exported as one chain.
    source_posterior: the :class:`~bayesmith.artifacts.results.PosteriorResult`
        a predictive result came from, read ONLY for its representation --
        it is what says whether the sample is importance-weighted, and a
        ``source_posterior_ref`` is a pointer, not a representation. It
        deliberately does not supply ``chain_shape``: that override already
        has a home above, and one argument quietly feeding two decisions is
        how the second one stops being visible.

Returns:
    An :class:`~bayesmith.artifacts.reports.EvaluationReport` of kind
    ``loo_psis``, filed against ``result``:

    * APPLICABLE / PASS -- arviz reported no reliability warning.
    * APPLICABLE / ABSTAIN -- arviz warned. The ESTIMATE is unreliable,
      which is not a claim about the model (§0.2).
    * INAPPLICABLE / ABSTAIN -- the result carries no pointwise log
      density, so there is no LOO to compute.
    * UNVERIFIABLE / ABSTAIN -- the sample is importance-weighted, and
      R2's export carries no weights for arviz to honour.
    * UNVERIFIABLE / ABSTAIN -- arviz is not installed (§7.3).

Type a concept or API name.