Harness evolution agent
It tunes itself.
Or it dies.
harness runs an observability-driven hill-climb over your coding-agent's retry limit, temperature, and step budget — discarding any mutation that regresses failure rate or cost — then proves the final config actually wins on held-out tasks the verifier re-simulates from scratch.
harness · observability-driven config evolution · [higgsfield: noir gold hill-climb contour lines converging on a single lit peak] · agent engineering
The failure mode
The config nobody revisited
“We hand-tuned retry_limit and temperature once, in Q1. Nobody's touched it since — we don't even know if it still helps.”Eng lead · internal coding-agent platform
Agent configs rot the same way any manual tuning does: someone gets a good result once, ships the numbers, and nobody re-validates them against how the task distribution actually shifted. A retry budget that made sense for last quarter's task mix quietly becomes wasted spend — or worse, a silent failure-rate regression — and there's no held-out comparison proving the "known good" config is still good.
How it works
Three steps. No faith required.
01
Mutate
A population-based hill-climb perturbs retry_limit, temperature, and max_steps from the current best config.
02
Verify
A candidate that regresses failure rate or cost beyond threshold is discarded; the verifier re-simulates the paired comparison from scratch, never trusting the builder's self-reported win rate.
03
Seal
Win rate ≥60% and cost delta ≤10% on held-out tasks, or the packet is REFUTED and nothing enters the chain.
Proof band
One packet. Every claim accounted for.
packet rig/harness · observability-driven evolution claim "evolved config beats initial on ≥60% of held-out tasks, cost delta ≤10%" initial retry=0 · temp=1.400 · max_steps=4 final retry=0 · temp=0.329 · max_steps=8 win_rate PASS — 77.5% ≥ 60% floor, 40 held-out tasks, seed 42 cost_delta PASS — -17.3% (cheaper) ≤ 10% ceiling themis SEALED — terminal verifier signed
Disable the evolution loop and final_config == initial_config exactly — win rate collapses to 0.0%, gate goes RED, and the packet is REFUTED, not sealed. Across six tested seeds the real evolution loop clears 75-92.5% win rate, never a fluke single number. How ProofPackets work →
Before you install
The obvious objections
Q Config tuning sounds like something I could grid-search myself.
You could — and pay for every step attempt while you do it. harness runs a population-based hill-climb that only accepts a mutation when it beats the current best AND doesn't regress cost or per-step failure rate beyond threshold, then proves the final config wins on held-out tasks the verifier re-simulates independently.
Q What if the "evolved" config is just overfit to easy tasks?
The paired comparison runs the initial and final configs on the same held-out eval set with matched per-task seeds — no shared training signal. Across six tested seeds, win rate holds in the 75-92.5% range, never a single lucky number.
Q How do I know it isn't just always mutating and calling it "evolution"?
Disable the evolution loop and final_config == initial_config exactly — 0% win rate, gate goes RED. That's the planted failure that proves the mutation loop itself, not the reporting, is what's under test.
Pricing
Start free. Pay when it pays.
Generalist ships today. Domain Expert adds vertical memory + priors. Custom is built to your stack.
Free
Priced after a short conversation
10 evolution runs a month. Public evidence, full gate output.
Team
Priced after a short conversation
Unlimited evolution runs, custom knob sets, private eval sets.
Pro
Priced after a short conversation
Dedicated verifier, CI-gated config regression, refute-turnaround SLA.
