Conformance Packs and Determinism Classes

A lightweight, repeatable suite that every Helix release must pass before tagging. It anchors determinism, schema compatibility, and policy enforcement.

Determinism classes

D0 (bitwise-deterministic): byte-for-byte identical outputs across runs on the same platform; hash-stable bundles.
D1 (deterministic within tolerance): numerically stable within fixed tolerances; hashes may differ but verifier must pass with exact schema + tolerance metadata.

Backends publish their target class:

Backend	Class	Expectation
cpu-reference	D0	Default; required for all releases.
native-cpu	D0	Must match cpu-reference on canonical cases.
gpu	D1	GPU math allowed to drift within documented tolerances; verifier compares against stored expectation.

Reference packs (ship with repo)

repro/helix_repro_bundle_v1: canonical CRISPR/Prime repro cases (spec + manifests + expected CPU/GPU runs). Powers D0/D1 checks.
tests/golden/cpu_minimal/config.json: fastest end-to-end CLI run for smoke + manifest hashing.
tests/golden/cpu_export: micro export parity fixture.

Runner command

Use tools/conformance.sh (added in this repo) to execute the curated subset:

./tools/conformance.sh

It prints PASS/FAIL per pack and exits non-zero on any drift. Under the hood it reuses tools/test.sh with the deterministic tests listed inside the script.

Release gating rules

Every release tag must run the conformance runner on at least cpu-reference and native-cpu; GPU optional but recommended when available.
New schema/spec versions must add a fixture or expectation to the pack before merging.
Any change to packs or tolerances requires updating this doc and the runner.

Adding a new conformance case

Add the case under repro/helix_repro_bundle_v1/inputs (spec + manifest + expected outputs) or a new pack directory under repro/.
Add a focused test in tests/ that consumes the new case without network access.
Append the test file to tools/conformance.sh and document the determinism class it covers.