Diagnostics and model checks

A successful run, a reliable estimator and an adequate model are different claims.

Ask the right diagnostic question

Question	Evidence
Did the computation complete?	Run termination, errors, elapsed work and budget.
Does the sample support estimation?	Chain behavior, ESS, R-hat, Monte Carlo error, divergences, weight concentration where relevant.
Can the data determine the parameters?	Identifiability, curvature and prior-sensitivity diagnostics.
Does the model reproduce specified features?	Prior/posterior predictive discrepancy reports and held-out prediction.
Does an inference route calibrate uncertainty?	Simulation-based calibration across independently simulated problems.

Read both report axes

An EvaluationReport separates applicability (APPLICABLE, INAPPLICABLE, UNVERIFIABLE) from conclusion (PASS, FAIL, ABSTAIN). Only applicable checks can pass or fail. Missing inputs, a missing optional backend or unsupported noise must not become a statistical PASS.

Applicability	Allowed conclusion
APPLICABLE	PASS, FAIL or ABSTAIN
INAPPLICABLE	ABSTAIN
UNVERIFIABLE	ABSTAIN

A quality gate aggregates reports under a declared policy. The evaluation layer reads completed results and existing diagnostic verdicts; it does not alter the posterior, choose the next sampler or invent a second numerical threshold.

Select and interpret checks

check_posterior is the orchestration entry point. Individual checks include posterior/prior prediction, held-out scores, optional ArviZ PSIS-LOO, SBC, identifiability and prior sensitivity. Observation grouping, draw floors and replicate budgets are part of applicability.

The declared false-positive rate is ALPHA = 0.05. Do not tune discrepancies, seeds or tolerances after seeing whether the model passes. Record which statistics were actually checked.

A PASS has a scope

A predictive PASS bounds the discrepancies evaluated, not all possible model errors. SBC can be satisfied by prior draws that ignore data; it is not evidence of useful posterior learning. Good importance-weight ESS cannot discover an unseen mode.

Use complementary checks and independent oracles. Experimental primitives need separate overlap and truncation-validity assessments.

Certify a point estimate

Since 0.10.0, minimize and fit accept certify=<limit>. This asks for a bound on the Newton decrement at the returned point, using the objective's gradient and Hessian. For a quadratic negative log posterior, that decrement is the distance to its minimum in curvature-standard-deviation units. For a nonlinear objective it describes the local quadratic model; it does not establish a global MAP or posterior sampling convergence.

import jax
import jax.numpy as jnp
from bayesmith.optimize import minimize

with jax.enable_x64(True):
    result = minimize(
        lambda p: 0.5 * jnp.sum(((p["x"] - 2.0) / 0.5) ** 2),
        {"x": jnp.asarray([0.0])},
        steps=200,
        certify=0.1,
    )
    print(result.converged, result.refusal)
    print(result.certificate.distance)

result.converged requires an accepted curvature floor, a successful certificate solve and an upper bound within the requested limit. A small estimated decrement alone is insufficient. Small problems use a dense Hessian calculation; above 1024 real parameters, certification requires a caller-supplied, justified floor=. The Lanczos probe cannot approve a certificate. The Newton polish runs by default when certification is requested.

Without certify=, converged is false because no certificate was requested; read refusal for the reason. With fit(names=...), the certificate concerns only the moved block conditional on held parameters. Factor sweeps do not attach a certificate automatically: a caller can measure the full joint objective afterwards through bayesmith.optimize.certify. The descent engine remains experimental; the certificate API is maintained within these numerical and local-model limits.