Audit Walkthrough (for non-engineers)

Purpose: let auditors, lawyers, or compliance reviewers answer “Is this bundle trustworthy?” without reading code.

What you received

• A bundle directory or zip with manifest.json and hashed files.
• Optionally SHA256SUMS.txt and a short claim from the submitter.

Steps (5–10 minutes)

1. Verify hashes
   • If SHA256SUMS.txt exists: sha256sum -c SHA256SUMS.txt (expect all OK).
   • If not: run python -m json.tool manifest.json > /tmp/m.json (optional pretty-print), then spot-check a file: sha256sum <file> should match the sha256 for that path inside manifest.json.
2. Run the verifier (no internet needed)

   helix verify --kind auto --manifest /path/to/manifest.json
   

   • PASS means: file hashes match the manifest, all required schemas validated, and no checks required by the selected policy/profile were skipped.
   • FAIL prints a reason; you can stop here and ask the submitter to fix.
3. Check provenance
   • Open manifest.json and note: policy (or policy_profile), schema_versions, backends, helix_version, git_sha.
   • Confirm the policy/profile is acceptable for your org (e.g., audit-strict vs dev-fast).
   • Interpretation
     • helix_version: must be ≥ the minimum version listed in the selected policy documentation.
     • git_sha: required for non-dev profiles; absence requires escalation.
     • backends: any claim relying on a backend not listed here is invalid.
4. Review the claim
   • Match the claim to specific outputs (report PDF/HTML, JSON results). Ensure filenames referenced in the claim exist in the manifest.
   • Claims must reference outputs by exact filename or glob present in the manifest. Narrative-only claims are not acceptable evidence.
5. Optional replay (deterministic)
   • If a .spec.json is included: helix run <spec.json> --out /tmp/replay --backends cpu-reference.
   • Compare /tmp/replay/.../run.json hashes to the originals; expect identical for D0, tolerant match for D1.
   • D0: hashes must match exactly.
   • D1: differences must be within the tolerance defined in the contract (numeric noise only; no structural changes).

Fast trust smoke (60 seconds)

This does not replace the steps above; it is a quick sanity check only.

HELIX_POLICY_PATH=policies/audit-strict.json tools/demo_replay_sampler.sh
helix trust check --backend cpu-reference

Expected: DEMO REPLAY: PASS (...) with identical normalized hashes. If it fails, capture the two hashes and rerun with HELIX_BIN=<path to helix> to rule out PATH issues.

How to read the verifier output

• ok: true → bundle intact and schemas valid.
• warnings → non-fatal issues (deprecated schema, missing optional files). Note them in your report.
• taint → trust class (e.g., debug, deprecated, partial). Red taint means do not accept without additional justification.

When to escalate

• Hash mismatches, missing files, unknown schemas, or taint debug/partial with no explanation.
• Claims that rely on backends not listed in the manifest.
• Bundles older than the stated longevity window without a migration verifier.

This walkthrough is deliberately non-technical: you only need a shell and the helix command installed.

What this walkthrough does NOT establish

• Scientific correctness of the results.
• Suitability for a specific regulatory or clinical standard.
• Absence of bias, omissions, or alternative analyses.