amazon-science/agent-ecc

amazon-science/agent-ecc

Language: Python

License: Apache-2.0

Stars: 0

Forks: 0

Open issues: 0

Created: 2026-10-21T08:16:13Z

Pushed: 2026-10-21T22:41:11Z

Default branch: main

Fork: no

Archived: no

README:

Agent ECC

Version-consistent experience consumption for agent training. Validate recorded experience against a chosen verifier version, re-evaluate it when needed, and track what each training step actually consumes.

Agent ECC is the open-source implementation of *Epoch-Consistent Commit*. In asynchronous agent training, trajectories are generated, graded and buffered long before a learner step consumes them, while the things that decide their labels keep changing: test labels are fixed, test suites are hardened, evaluators are patched, worlds are refreshed. A label recorded under one grader and trained after another has been published is a label nobody chose. Agent ECC makes the choice explicit: every training batch is sealed under one grader version, read once; every label the batch trains on is that version's verdict on what the trajectory actually observed (reused when nothing it depends on changed, re-parsed or re-executed when something did); a group whose label cannot be decided is neutralized rather than guessed; and what each learner step actually consumed is established from the learner's own log.

Layers

• ECC is the contract and the mechanism: content-addressed grader and world versions, a publication log that is

the pointer, the seal that reads it once per batch, and the invariants I1 to I5 ([docs/contract.md](docs/contract.md)).

• The SDK is how producers, graders and trainers use it: record types, scoring under a version, evaluation and

training manifests, the seal session (with re-executions prepared in the background from admission, and target pins for a control that never adopts), and audits of a finished run: the epochs its seals read and whether identical executions got different verdicts ([docs/interfaces.md](docs/interfaces.md)).

• A reference re-verification service does re-grading and re-execution behind a small HTTP API

(python -m agent_ecc.recompute.service).

• Adapters connect existing systems: SWE-bench (with the official harness, including SWE-ABS hardened suites),

AppWorld (answer re-grading and end-state re-grading with the AppWorld evaluator), and a trainer integration documented as an example ([docs/integrations/trainer.md](docs/integrations/trainer.md)).

It is a library and an SDK, not a platform: no scheduler, no database, no service you must run. The core package uses the Python standard library only. Scheduling stays with the trainer (staged releases, barriers that pause starts, bounds on how long a group may wait): agent-ecc gives it the registry, the seal and the records ([docs/integrations/trainer.md](docs/integrations/trainer.md), section 8).

Four record types

| Record | What it is | |---|---| | ExecutionRecord | What one trajectory did: its evidence by digest (a patch, a raw test log, a submitted answer, an end state) and the test runs made when it was graded. Immutable and content-addressed. | | ScoreRecord | One execution's verdict under one grader version: per-test outcomes, the method (recorded, reparse, reexec, undeterminable) and its cost. | | SealDecision | What one seal intends to train: the epoch it read once, a decision per member, neutralized groups, groups filtered after the decision, backfill and fillers. | | ConsumptionRecord | What one learner step actually consumed, confirmed by the learner's per-record log and the step's completion, with every mismatch against its seal decision (a record unsealed or unconsumed, consumed twice, in another group, with another reward or advantage, on other tokens or on none, or handed over under another epoch). A seal decision alone is not consumption. |

Install

Once the package is on PyPI (until then, install a built wheel or a source checkout, below):

pip install agent-ecc # the core: standard library only (Python >= 3.10)
pip install "agent-ecc[swebench]" # the official SWE-bench harness: log parsing, Docker re-execution
pip install "agent-ecc[appworld]" # the AppWorld evaluator: end-state re-grading
pip install "agent-ecc[service]" # the reference re-verification service (standard library server)

From a source checkout: pip install -e ".[test]" and python -m pytest. python -m build makes the sdist (with the documentation, the examples and the tests) and the wheel.

A tour

A sketch (dataset rows, logs, patches and groups come from your system; the runnable version is below):

from agent_ecc import (MemoryRegistry, MemoryStore, Scorer, ScoreInput, SealSession, SealJournal,
evaluation_manifest, training_manifest)
from agent_ecc.adapters.swebench import SWEBenchAdapter
from agent_ecc.recompute.service import HTTPBackend
from agent_ecc.scoring import grade_recorded

store, registry = MemoryStore(), MemoryRegistry()
swe = SWEBenchAdapter() # grades logs with the official harness

# grader versions are content-addressed maps (task, world) -> test spec
g1 = swe.version_from_rows(rows, store, name="verified")
g2 = swe.conjunction_version(rows, hardened_rows, store, name="verified+hardened") # original AND hardened suite
registry.publish_grader(g1)

# collect: a producer records what a trajectory did, and the verdict it got under the epoch it pinned at start
# (under a conjunction such as g2, one run per suite: docs/integrations/trainer.md, section 3)
check = g1.entries[task]["*"]
run = swe.run(check, store=store, log=test_log, applied=True)
record = swe.record(store=store, task_id=task, trajectory_id=traj, image=image, patch=patch, runs=[run])
recorded = grade_recorded(swe, record, check, store, grader_version=g1.version_id)

# score under any version: reuse, re-parse or re-execute, whichever the identities require
scorer = Scorer([swe], store, backend=HTTPBackend("http://reverify:8770"))
scored = scorer.score([ScoreInput(record, recorded)], g2)

# evaluate: one manifest per version over the same record set, undeterminable records counted apart
manifest = evaluation_manifest(records, [s.score for s in scored], g2)

# train: a seal reads the epoch once, decides every member under it, filters after deciding, backfills
registry.publish_grader(g2)
session = SealSession(registry, scorer, batch_id="step-42")
assembly = session.assemble(groups, backfill=more_groups) # more_groups(need,...