Determinism regression agent
Same hash,
or no ship.
mregtest records one seeded run, replays it 10 times under a virtual clock, and requires every hash to match bit-for-bit. Then it injects a real model-change and requires the mismatch to be caught — not silently absorbed as noise.
mregtest · replay-based determinism gate · [higgsfield: noir gold ten overlapping oscilloscope traces collapsing into one line] · determinism gates
The failure mode
The bug you can't reproduce
“The bug only shows up sometimes. We can't reproduce it, so we can't say we fixed it — and neither can anyone reviewing the PR.”Staff engineer · distributed systems team
"Regression" is meaningless without a way to force determinism first. Flaky-test retries in CI hide non-determinism instead of catching it — a test that fails once and passes on rerun gets waved through, and the actual source of the flake (often a wall-clock read leaking into logic that should be seed-driven) ships anyway. Without a genuine replay guarantee, you can't tell a real model-change regression from ordinary noise.
How it works
Three steps. No faith required.
01
Record
One seeded run captures a canonical sha256 trace hash under a virtual clock — never wall-clock time.
02
Replay
The exact same seed re-derives and re-executes the system 10 times; every hash has to match, bit-for-bit.
03
Detect & seal
A real injected model-change has to produce a different hash — silence is a failure, not a pass — before the packet seals.
Proof band
One packet. Every claim accounted for.
packet rig/mregtest · replay-based regression testing claim "n=10 replays bit-identical, injected model-change is DETECTED" run seed 1 · 40 events · virtual clock recorded hash ‹001a11eb…› determinism PASS — 10/10 replays match recorded hash exactly regression PASS — mutant hash ‹8b76cb87…› ≠ baseline, flagged themis SEALED — terminal verifier signed
Force replay onto the live wall clock instead of the virtual clock and 10 replays produce 10 distinct hashes — the determinism gate goes RED on the spot, no ambiguity. How ProofPackets work →
Before you install
The obvious objections
Q We already have flaky-test retries in CI.
Retrying a flaky test hides non-determinism, it doesn't catch it. mregtest requires bit-for-bit identical output across 10 replays under a virtual clock — and separately proves that a real model-change gets DETECTED, not silently absorbed as noise.
Q Couldn't the replay just be a cache hit dressed up as "determinism"?
No — each replay re-derives the same seeded event stream and genuinely re-executes the system under test. The planted failure (swap the virtual clock for wall-clock time) makes that concrete: 10 replays produce 10 distinct hashes the instant real time leaks in.
Q What if the mutant model change is too big to be a fair test?
It's one realistic regression — the ack handler drops a single clamp — not a synthetic no-op. It still gets caught: mutant hash mismatches baseline hash on every seed tested.
Pricing
Start free. Pay when it pays.
Generalist ships today. Domain Expert adds vertical memory + priors. Custom is built to your stack.
Free
Priced after a short conversation
50 replay verifications a month. Public evidence, full gate output.
Team
Priced after a short conversation
Unlimited replays, custom mutant models, CI integration.
Pro
Priced after a short conversation
Dedicated verifier, CI-gated determinism regression, refute-turnaround SLA.
