bayesmith.evaluation.diagnostics

Source-derived signatures and contracts. See the reference index for coverage and conventions.

Module contract

Two design-time diagnostics, filed on §0 ruling 7's two axes.

:func:`~bayesmith.diagnose.identifiability.identifiability` and
:func:`~bayesmith.diagnose.sensitivity.prior_sensitivity` already reach a
verdict, against thresholds that already have an owner:
:data:`~bayesmith.diagnose.identifiability.DEFAULT_RANK_RTOL` and
:data:`~bayesmith.diagnose.sensitivity.CRITERION_SHIFT`, each justified where
it is declared.  What is missing is a way for a gate to READ those verdicts --
an :class:`~bayesmith.artifacts.reports.EvaluationReport` whose ``subject_ref``
names the result being judged and whose findings carry the numbers the verdict
was taken on.

So this module PROJECTS.  It reads ``nullity``, ``participation``, ``worst``,
``verified``, ``refit_converged`` and ``criterion_std`` off the reports and
files them.  It compares nothing against a number of its own -- the plan's §0.2
puts it plainly: a second copy of a threshold is how two answers to one
question come to be written down, and they agree until the day one of them is
retuned.  ``tests/evaluation/test_diagnostics.py`` asserts that this file
contains no float literal at all, which is the same rule stated as something
that can fail.

**Three verdict shapes, and the order between them is load-bearing.**

* APPLICABLE × PASS / FAIL -- the diagnostic ran and decided.
* APPLICABLE × ABSTAIN -- ``prior_sensitivity`` ran, and its two routes did
  not verify each other (the refit did not converge, or some element of
  ``verified`` is False).  A shift no second route confirmed is not evidence
  either way, so this is checked BEFORE the shift is compared: the measured
  fixture in the tests has a worst shift of +2.2 sigma with a False in
  ``verified``, and a projection that looked at the shift first would file a
  confident FAIL taken from a number nothing cross-checked.
* UNVERIFIABLE × ABSTAIN -- the diagnostic refused.  §0.10's headline case is
  a float32 ambient precision, which both functions refuse by name and which
  probe_28 §9 hit on its own first run; the refusal text goes into the finding
  verbatim, because half of it is the remedy (the GRAPH has to be built inside
  ``jax.enable_x64(True)``, not only the call).  A caller assembling a gate
  gets a report rather than an exception, which is §7.3's "degrade
  gracefully" and §2.4's "a run failure must not be dressed as a FAIL".

**Layering.** ``evaluation`` reads ``diagnose``, ``artifacts`` and ``graph``;
none of them reads back (``tests/test_layering.py``).  Nothing here imports
arviz, at module scope or otherwise.

IDENTIFIABILITY PRIOR_SENSITIVITY identifiability_report prior_sensitivity_report DEFAULT_RANK_RTOL CRITERION_SHIFT

IDENTIFIABILITY

Defined in bayesmith.evaluation.diagnostics, line 91.

IDENTIFIABILITY = 'identifiability'

PRIOR_SENSITIVITY

Defined in bayesmith.evaluation.diagnostics, line 92.

PRIOR_SENSITIVITY = 'prior_sensitivity'

identifiability_report

Defined in bayesmith.evaluation.diagnostics, line 154.

identifiability_report(graph: Graph, subject: PosteriorResult, *, names: Sequence[str] | str | None=None, rtol: float=DEFAULT_RANK_RTOL) -> EvaluationReport

File :func:`~bayesmith.diagnose.identifiability.identifiability`'s rank
verdict as a report about ``subject``.

PASS at nullity 0, FAIL above it; §0.2's row, and the whole comparison.
The rank cut is ``rtol``'s, decided in ``diagnose`` and passed through so
that a caller who has read
:attr:`~bayesmith.diagnose.identifiability.IdentifiabilityReport.weakest_identified`
can tighten it without this layer acquiring an opinion about the value.

Args:
    graph: the model ``subject``'s posterior was taken over. Build it
        inside ``with jax.enable_x64(True):`` -- see the module docstring.
    subject: the result this report judges. Only its identity and its
        fingerprints are read; no draw is touched.
    names: which latents to analyse, as
        :func:`~bayesmith.diagnose.identifiability.identifiability` takes
        them. ``None`` is the joint question over all of them, which is
        the one a per-block guard cannot ask.
    rtol: the relative rank tolerance.

Returns:
    APPLICABLE × PASS when nothing is degenerate, APPLICABLE × FAIL with a
    ``null_participation`` finding naming the latents the first blind
    direction mixes, or UNVERIFIABLE × ABSTAIN carrying the diagnostic's
    refusal.

prior_sensitivity_report

Defined in bayesmith.evaluation.diagnostics, line 264.

prior_sensitivity_report(graph: Graph, subject: PosteriorResult, *, names: Sequence[str] | str | None=None, at: dict[str, jax.Array] | None=None) -> EvaluationReport

File :func:`~bayesmith.diagnose.sensitivity.prior_sensitivity`'s worst
mode displacement as a report about ``subject``.

§0.2's row: PASS while the worst ``|shift_sigma|`` is below
:data:`~bayesmith.diagnose.sensitivity.CRITERION_SHIFT`, FAIL at or above
it, ABSTAIN when the closed form and the refit did not verify each other,
UNVERIFIABLE when the diagnostic refused. The verification is read FIRST;
the module docstring says why.

Args:
    graph: the model, built inside ``with jax.enable_x64(True):``.
    subject: the result this report judges.
    names: which latents' priors to analyse; ``None`` means all of them.
    at: where to expand, as ``prior_sensitivity`` takes it.

Returns:
    An :class:`~bayesmith.artifacts.reports.EvaluationReport` whose
    ``worst_shift`` finding carries ``(latent, index, shift)`` against
    ``CRITERION_SHIFT``, and whose ``prior_width_margin`` finding carries
    the declared width against the width at which that latent's shift
    would reach it.

DEFAULT_RANK_RTOL

Defined in bayesmith.diagnose.identifiability, line 154.

DEFAULT_RANK_RTOL: float = 1e-08

CRITERION_SHIFT

Defined in bayesmith.diagnose.sensitivity, line 128.

CRITERION_SHIFT: float = 0.1