DOCUMENTATION · METHOD

Method, limitations, and claim boundaries

What the three channels mean

Discrimination (PR-AUC in this build) measures how well the model ranks fraud above non-fraud. Calibration (ECE) measures whether predicted probabilities match observed outcome frequencies. Explanation stability (attribution-profile cosine similarity) measures how stable the model's attribution profile is between the reference and comparison windows. It does not measure explanation correctness, causality, fairness, or deployment suitability.

The seeded executable model

Frozen logistic fraud model · synthetic demonstration fixture v1 is a deterministic, hand-authored logistic-regression demonstrator, not a fitted production classifier. Its 6 numeric inputs, intercept and coefficients are declared in one typed manifest. Scores are fraud probabilities from a sigmoid. Explanations are exact signed coefficient × feature contributions in logit space. Manifest/weights reference: sha256:475c696644118e7f58f3a4b40f210561a2d65196d641e4a4ac138de02df9462e. Deterministic, hand-authored demonstration model. It was not fitted to production data and is not evidence of real-world fraud performance.

Two execution tiers

Evidence replay accepts supplied scores, labels and optional attribution columns. It can recompute evidence metrics and run clearly labelled evidence-layer sensitivities, but it can't infer how a raw-input change would alter scores. Executable model stress test accepts raw numeric CSVs plus the constrained chaos-lab-logistic-manifest-1 JSON contract. It validates exact features and missing-value policy, reruns the logistic model, and derives exact linear contributions. It does not accept Python, pickle, or joblib execution.

What each operator changes

Signal substitution takes a declared raw source feature, picks rows deterministically, and interpolates them toward a declared substitute signal. It keeps that substitute feature and reruns the model. Feature outage, staleness and noise work the same way on raw inputs, when an executable contract exists. Historical attribution_reweight@1.0.0 records stay an evidence-only attribution transformation, scores don't change, and they're never silently reinterpreted. Label delay masks available labels. It doesn't fabricate outcomes.

What a scenario result is, and is not

Every scenario in this build is a declared, parameterised transformation of fixture data (e.g. signal substitution, feature staleness). Results are conditional on the declared transformation and its stated assumptions. They are not forecasts of future fraud, and they are not evidence about any real institution.

No composite score

This product deliberately does not produce a trust, readiness, resilience, safety, health, or approval score. Discrimination, calibration, and explanation stability are shown as separate evidence objects. The blind-spot search ranks scenarios against a single, visible, user-editable objective, it does not declare a scenario "safe" or "unsafe".

Current build status

This is a research prototype. The landing page and seeded demo use synthetic fixture data. When you upload evidence under Projects, the app computes discrimination (PR-AUC), calibration (ECE) and an explanation-stability proxy from your CSV rows. It stores the results with input and output hashes. The prototype also includes six versioned scenario operators, a bounded parameter search, deterministic evidence cases, separate human review records and schema-validated JSON exports. Multi-user authentication has not been built. The seeded demo's blind-spot search computes its trial ledger from the parameters and seed you set rather than reading a prewritten table.

RUNTIME NOTICE

Hosted showcase data retention

This public deployment uses temporary server storage so visitors can try the upload and scenario workflow without a managed customer database. Projects may disappear when the service restarts and must not contain sensitive or identifying data. Durable tenant storage and authentication are not enabled.

The explanation-stability proxy

The explanation-stability value shown on uploaded projects is this app's own cosine-similarity comparison between two attribution-share profiles. It is a reasonable, tested proxy for attribution-profile drift, but it is not the dissertation's validated ESI formulation and must not be compared numerically against BAF ESI references.

How bounded search qualifies a candidate

A trial qualifies only when its absolute PR-AUC change stays inside the declared discrimination tolerance and its chosen objective reaches the separate minimum-effect threshold. Among qualifying trials, the selected result has the smallest mean normalised distance from the lower bounds of the ranged operator parameters. The seeded default maximises Δ ECE. Search is seeded and budget-bounded, not exhaustive. If no trial qualifies, the product reports that and keeps the complete ledger.

Absence of an alert is not evidence of safety

A scenario that does not surface a divergence in this demo only means that scenario, with its declared parameters and seed, did not produce one in this fixture run. It is not evidence that a model is safe under other conditions.