Wiki · Research
Follow-on research synthesis — owners, evidence, and non-equivalent boundaries
Shelf
Research

Follow-on research synthesis — owners, evidence, and non-equivalent boundaries

This document closes WI-028. It compares the eight completed extractions in their three tracks, then reconciles the overloaded vocabulary that appears across products, evaluators, optimizers, and protocols. It is a synthesis of pinned evidence, not a ranking, benchmark, or implementation plan. The canonical pattern catalogue is changed only when a reconciliation row supports an existing definition; no new catalogue entry is admitted by analogy alone.

Method and provenance

Documented — The synthesis applies WI-019’s shared spine, product template, evaluation template, and protocol template. Track comparisons come before cross-track claims. Every source claim below is routed through one of these extraction records, each of which freezes its own authority and read boundary:

WISubject/trackPinned authorityExtraction
WI-020Letta Code — durable/local productec4e23a85d4aa2a449ed5c7fb0801a0be1bd68d0products/letta-code.md
WI-021Inspect AI — evaluation/optimizationd482209d573cdde116ccf28abfb01712e91e80cevaluations/inspect-ai.md
WI-022Aider — durable/local product5dc9490bb35f9729ef2c95d00a19ccd30c26339cproducts/aider.md
WI-023goose — durable/local product3810898a7447ec3299be72e223d3570a7aabf0abproducts/goose.md
WI-024Google ADK — durable/local productPython 1d2d1eda3c9b795cd90ad643390f4da5a8cd27bf; Java 2b87d65d9704a61ff4668b8c9482a79fef9fe0d4products/google-adk.md
WI-025MCP — interoperability protocol4df2d6b6e3588efb46e7542d98498e5c630a0a86protocols/mcp.md
WI-026A2A — interoperability protocol1eb4aa03b07589d3a00ce7deab0dde679120ed30protocols/a2a.md
WI-027DSPy — evaluation/optimization80b118e52cb1f143a0d80d84685572000c59639eevaluations/dspy.md

Boundary — “Confirmed” below means that the candidate matches an existing catalogue meaning under a comparable evidence boundary. “Track-specific” means useful vocabulary whose owner or guarantee cannot be promoted to the shared catalogue from this corpus. “Contradicted” rejects the candidate as a safe positive rule. No score, adoption claim, or cross-source performance claim is introduced.

Within-track comparison: durable/local products

SubjectState and identityExternal interaction and authorityDurable evidence/recoveryStrongest limit
Letta CodeImplemented — agent-scoped memory roots, conversation metadata, Git-backed memory, and repository state are separate layers; parent/child sub-agent identity is explicitImplemented — child launch carries a continuation envelope and cross-agent permissions are guarded; unrestricted launch policy remains host-ownedImplemented — memory commits return a revision; prompt compilation projects metadata but is not a checkpoint or complete auditGit-backed memory is not truth, rollback, or human accountability; sync conflicts and external effects remain ambiguous (WI-020)
AiderImplemented — repository context, file edits, Git state, and chat history remain distinct; exact paths/search-replace anchors scope mutationImplemented — model-facing edits are confirmed/validated, commits are visible, and hooks/tests report results; shell/repository authority remains localImplemented — a commit hash is a durable candidate and /undo exposes reversal; lint/test output is returned to chatRepeated shell/test/commit effects are not exactly-once; user review and idempotence remain workflow obligations (WI-022)
gooseImplemented — session id reloads state-machine steps; working directory and extension context are session-scopedImplemented — MCP delegates capability, ACP delegates loop ownership, and action-required messages persist approval/denial; defaults do not create a general sandboxImplemented — recipe/session metadata and ACP context markers support restart and rollback around cancellationOptional macOS sandboxing is not a cross-platform default; neighbour is a useful analogy, not proof of shared state (WI-023)
Google ADKImplemented — session append/reload, branch/isolation scope, task-call ids, and confirmation events separate runtime continuity from delegated stateImplemented — confirmation is keyed to a function-call id; RemoteA2aAgent maps a remote protocol into local events; authentication is not impliedImplemented — events, EvalSet/metric results, migration metadata, and compatibility windows are distinct artifactsCross-language parity confirms contract intent, not production equivalence; schema migration and session storage remain application/runtime boundaries (WI-024)

Within-track conclusion — Opinion — The product subjects converge on explicit ownership and durable boundaries, but not on one “session” or “memory” primitive. Letta’s agent-owned memory, Aider’s repository candidate, goose’s session isolation, and ADK’s event/confirmation model are commensurable as boundary declarations; their state stores and recovery guarantees are not interchangeable.

Within-track comparison: evaluation and optimization

SubjectEvaluation contractOptimization/runtime relationEvidence and reproducibilityStrongest limit
Inspect AIImplemented/Documented — task, solver, scorer, samples, typed scores, logs, and aggregates are separate ownersImplemented — solver limits and sandbox outcomes bound a run; evaluation consumes a program rather than rewriting itImplemented — logs/transcripts and typed results support inspection; mocks substitute model calls in testsCrash/retry paths can lose in-flight data or repeat external effects; events are not automatically an audit log (WI-021)
DSPyImplemented — Evaluate applies a caller metric to a devset, retains row results, and aggregates scoresImplemented — ReAct emits a trajectory; BootstrapFewShot accepts metric-passing traces as demos and marks a compiled studentImplemented — JSON/PKL/program artifacts record state and dependency versions; deterministic DummyLM tests bound local behaviorOne trainset supplies teacher traces and leftover demos; no required held-out set, uncertainty estimate, provider seed, or cost ledger (WI-027)

Within-track conclusion — Opinion — Inspect AI is primarily an evidence-producing evaluator; DSPy is an optimizer whose metric changes a program artifact. Their typed scores, logs, and limits are comparable as evaluation-contract fields, but a score from one cannot prove the other’s optimization claim. Both reject the shortcut “a passing score is a trustworthy general improvement.”

Within-track comparison: interoperability protocols

SubjectAuthority and discoveryLifecycle and resultIdentity, trust, and deliveryStrongest limit
MCPNormative — versioned schema/specification defines initialize/version/capability negotiation and client/server rolesNormative — tools, resources, prompts, sampling, elicitation, progress/cancellation, and structured errors cross a session/transportNormative — hosts own user authority, roots, tokens, and model decisions; negotiated features are not proof of implementation safetyTransport and capability contracts do not define application identity, consent UI, or exactly-once side effects (WI-025)
A2ANormative — Agent Card selects interface, version, capabilities, security declaration, and optional signatureNormative — Message may create a Task; task states, streamed/push updates, artifacts, cancellation, and continuation are explicitNormative — server authenticates/authorizes and scopes visibility; context/task ids are opaque; duplicate push and disconnect are possibleAUTH_REQUIRED is not a grant; task history/streams and webhook delivery are not complete audit or exactly-once guarantees (WI-026)

Within-track conclusion — Opinion — MCP and A2A are commensurable at the level of negotiated capability, explicit lifecycle, error, and trust boundaries. They are not substitutes: MCP centers host-mediated capabilities and context; A2A centers independent-agent task ownership and artifacts. Normative text establishes requirements, not conformance of an arbitrary implementation.

Cross-track boundary table

TermProduct meaningEvaluation meaningProtocol meaningSynthesis result
StateSession stores, Git commits, memory files, event rowsRun/sample status, checkpoints, typed scores, compiled demosTask state, protocol session, capability/version stateNot commensurable — owner, retention, and replay semantics differ
SessionLetta/goose/ADK continuity identity, often with filesystem or event scopeInspect run context or DSPy in-memory call/compile contextMCP connection/session or A2A context/task groupingAnalogy only — shared name, different lifecycle and authority
MemoryAgent-owned files/metadata versus repository/documentary statePrompt examples, traces, logs, and evaluator evidenceProtocol messages/resources/artifacts carried across a boundaryNot commensurable — persistence and truth claims are different
HandoffChild/remote loop ownership plus continuation envelope or ACP contextSolver execution or compiled-program transitionA2A task/message delegation; MCP capability callCommensurable boundary only — ownership and state crossing must be stated
ToolProduct extension/provider capability with local permissionsSolver tool call or mocked model/tool executionMCP tool; A2A opaque remote capabilityAnalogy only — transport, authority, and result semantics differ
ApprovalProduct UI/action-required/confirmation event bound to a callUsually a scorer/limit outcome, not human authorityMCP human-in-the-loop guidance; A2A AUTH_REQUIRED stateNot commensurable — a pending state is not authenticated authority
Trace/logProduct messages, events, Git history, memory commitTranscript, sample evidence, score rows, optimizer traceTask history, status/artifact events, protocol errorsNot commensurable — an event stream is not automatically audit/accountability
Result/artifactCommit, recipe output, event, memory revisionScore, transcript, compiled state, CSV/JSON tableMCP result/content; A2A Artifact and status eventCommensurable output boundary only — durability and replay must be named
IdentityAgent/session/repository/user/child idsRun, sample, task, scorer, model/provider idsClient/server, Agent Card, context/task/message idsNot commensurable — identifiers do not transfer trust or ownership by themselves
OwnershipWho may mutate memory, repository, session, or continuationWho owns metric, dataset, model call, artifact, and test setClient/server/host/agent/transport responsibilityCommensurable as a required field — the concrete owner remains track-specific

Contradiction — Opinion from equivalent boundaries — None is claimed between equivalent normative/implemented statements. The apparent contradictions (“session is durable” versus “session is in-memory”, or “approval is authentication” versus “approval is a pending state”) disappear once the owner and evidence type are restored; treating them as one claim would be a category error.

Candidate adjudication

Each row below is one candidate from exactly one extraction. The evidence column links to the extraction’s pinned catalogue-consequence section; the provenance register above supplies the full SHA and authority type.

WI-020 — Letta Code

CandidateVerdictEvidence and reason
session-handoffconfirmedImplemented — WI-020 shows narrative metadata/prompt projection; it confirms the catalogue warning that this is not a machine checkpoint.
prompt-writes-artifactconfirmedImplemented — Letta’s Git memory commit returns author/revision; this matches the existing durable-artifact definition, without claiming truth or rollback.
ownership-changing-handoffconfirmedImplemented — child tags, continuation envelope, and statelessness make continuation ownership explicit; host launch policy stays outside the pattern.
replay-safe-side-effectconfirmedImplemented — sync conflict/retry evidence supports the existing warning that persistence is not exactly-once external effect.
candidate: memory-layer-separationtrack-specificImplemented — useful Letta vocabulary for conversation metadata, prompt state, memory files, and repository state; this corpus does not establish one universal memory API.
candidate: agent-memory-ownershiptrack-specificImplemented — agent-scoped roots and cross-agent denial are strong product evidence, but not a cross-track guarantee for protocols or evaluators.

WI-021 — Inspect AI

CandidateVerdictEvidence and reason
bounded-agent-loopconfirmedImplemented — WI-021 records per-sample message/token/turn/time/working/cost limits; this confirms the existing bounded-loop definition.
event-stream-not-audit-logconfirmedImplemented/Documented — Inspect events and crash-loss behavior match the catalogue warning that events are inputs to audit, not accountability by themselves.
isolation-boundary-declarationconfirmedImplemented — sandbox provider/cleanup and current-user subprocess authority reinforce the existing isolation-boundary definition.
deterministic-model-substitutionconfirmedImplemented — mock model outputs exercise agent/evaluation paths; this confirms substitution as bounded test evidence, not provider equivalence.
typed-output-gateconfirmedImplemented — typed Score/EvalScore/EvalMetric/EvalResults confirm structure-versus-truth separation.
replay-safe-side-effectconfirmedOpinion grounded in implementation — retry/requeue/checkpoint paths can repeat tools; this confirms the catalogue warning, not a guarantee of idempotence.

WI-022 — Aider

CandidateVerdictEvidence and reason
narrow-anchortrack-specificImplemented — Aider’s path and exact search/replace controls are a code-edit product mechanism; they support the broader narrowest-anchor-loop without warranting a second generic id.
structural-gateconfirmedImplemented — dirty-file/ignored-file/commit verification gates repository mutation and fits the existing structural-gate meaning.
land-candidatetrack-specificImplemented — an automatic Git commit is a useful repository candidate, but “land” depends on this product’s review/undo workflow and is not a universal completion state.
agent-facing-interfaceconfirmedImplemented — commit results, lint/test output, and edit errors are model-visible bounded artifacts, matching the catalogue interface definition.
prompt-writes-artifactconfirmedImplemented — model response → exact edit → commit hash is a direct confirmation of the durable-prompt-artifact pattern.
replay-safe-side-effectconfirmedOpinion grounded in implementation — retries can repeat shell/tests/commits; this confirms the existing warning and keeps review/idempotence explicit.

WI-023 — goose

CandidateVerdictEvidence and reason
session-isolationtrack-specificImplemented — session id reload and working-directory context are strong goose behavior, but the shared catalogue already expresses session ownership without adopting this storage design.
confirmation-gateconfirmedImplemented — inspection, goose.executable, action-required messages, and persisted allow/deny confirm a gate while leaving identity/authentication outside it.
protocol-escape-hatchconfirmedImplemented — MCP/ACP preserve a Goose loop while delegating capability or loop ownership, matching protocol-with-escape-hatch.
agent-facing-interfaceconfirmedImplemented — provider errors, tool results, approvals, notifications, and recipe output are represented as bounded messages/events.
handoffconfirmedImplemented — ACP context marker commits after completion and rolls back on cancellation/refusal; this is a concrete ownership-changing handoff.
neighbourtrack-specificOpinion — adjacent ACP/MCP identities are useful vocabulary, but addressability is not proof of shared state or a universal neighbour contract.

WI-024 — Google ADK

CandidateVerdictEvidence and reason
session-isolationtrack-specificImplemented — branch/isolation scopes and task-call ids constrain delegated visibility, but do not define the same session primitive as goose or Letta.
confirmation-gateconfirmedImplemented — persisted function-call-id confirmation and resumed invocation confirm request-bound approval; they do not establish authentication or safe execution.
schema-evolutiontrack-specificImplemented — v0 pickle/v1 JSON migration and compatibility windows are an ADK persistence contract, not a universal agent pattern from this corpus.
evaluation-boundarytrack-specificImplemented — EvalSet/inference/metric results are distinct from runtime events; this is useful within the evaluation track, not a cross-track score guarantee.
protocol-escape-hatchconfirmedImplemented — RemoteA2aAgent maps external protocol behavior into local events, confirming the escape-hatch boundary while WI-026 owns protocol meaning.

WI-025 — MCP

CandidateVerdictEvidence and reason
protocol-escape-hatchconfirmedNormative — WI-025 defines JSON-RPC/stdio/HTTP capability transport across process/network boundaries.
capability-negotiationtrack-specificNormative — initialize/version/capability negotiation is central MCP vocabulary; A2A has analogous cards, but this corpus does not justify a transport-neutral catalogue id.
confirmation-gateconfirmedNormative — MCP recommends human deny paths while leaving UI and identity to the application; this confirms the catalogue’s non-authentication warning.
agent-facing-interfaceconfirmedNormative — tools/list/call, resources, prompts, sampling, elicitation, and errors are structured interfaces with named sides.
external-authorityconfirmedNormative — roots, authorization, sampling, and elicitation assign filesystem, token, model, and user decisions to explicit owners, matching the existing external-authority-precedence definition.

WI-026 — A2A

CandidateVerdictEvidence and reason
handoffconfirmedNormative — WI-026 transfers task/context identity across an opaque-agent boundary without transferring internal state.
neighbourtrack-specificNormative — Agent Cards, Messages, Tasks, Artifacts, and declared capabilities make independent agents addressable, but the current catalogue has no generic neighbour id; addressability is retained as protocol vocabulary.
explicit-outputconfirmedNormative — Artifacts are named task outputs that may stream/append; this confirms explicit output designation while delivery durability remains separate.
confirmation-gateconfirmedNormative — AUTH_REQUIRED delegates an authorization request but explicitly does not grant credentials or consent.
audit-trailtrack-specificNormative — task ids/timestamps/history aid traceability, but incomplete history and duplicate/lost delivery prevent a general audit-trail guarantee; the catalogue’s event-stream warning is the safer mapping.
protocol-escape-hatchconfirmedNormative — JSON-RPC/gRPC/HTTP bindings carry one abstract task model across process/network boundaries, confirming the escape-hatch boundary.

WI-027 — DSPy

CandidateVerdictEvidence and reason
evidencetrack-specificImplemented — WI-027 retains example/prediction/metric rows and a ReAct trajectory; the catalogue has no generic evidence id, and retention/provenance remain the deciding boundary.
semantic-gatetrack-specificImplemented — a caller metric selects demonstrations, but metric validity is application-owned and not a universal semantic truth gate.
bounded-loopconfirmedImplemented — ReAct max_iters, Bootstrap rounds/demo limits, and executor max errors confirm the existing bounded-agent-loop meaning.
explicit-outputconfirmedImplemented — compiled state/demos and optional CSV/JSON results are explicit artifacts, with data/credentials/external effects excluded.
structural-gateconfirmedImplemented — matching predictor structure and _compiled state confirm structural compatibility as a separate gate from correctness.
test-substitutioncontradictedOpinion rejected by the pinned evidence — a metric can select a trace without an independent held-out test; treating optimization feedback as test substitution would reverse the extraction’s leakage warning.

Catalogue reconciliation and downstream boundary

Opinion — No edit to pattern-catalog.md is warranted by these rows. Confirmed candidates map to existing definitions (bounded-agent-loop, event-stream-not-audit-log, agent-facing-interface, external-authority-precedence, ownership-changing-handoff, prompt-writes-artifact, protocol-with-escape-hatch, structural-gates, typed-output-gate, explicit-output-designation, and the resumable-approval-state/approval-bound-to-request pair). The track-specific candidates are deliberately not promoted: doing so would turn one product’s storage or one protocol’s lifecycle into a guarantee for another track. The single contradicted candidate, test-substitution, is retained as a warning in the DSPy extraction and is not added to the catalogue.

Boundary — No module or CLI change follows from this synthesis. Existing WI-029 remains the only implementation-facing follow-up and is not silently expanded; any future adoption of follow-on evidence must be a separately planned work item with its own catalogue/ADR decision.

Findings and final conclusions

  1. Confirmed across tracks — Opinion. Explicit ownership, bounded loops, structured outputs, and evidence retention recur, but only when the owner and guarantee are carried with the noun.
  2. Track-specific — Implemented/Normative. Durable memory/session stores, evaluator scores and compiled demos, MCP capabilities, and A2A tasks/artifacts cannot be merged into one state model.
  3. Contradicted shortcut — Opinion. A metric or event stream is not a held-out test or audit trail merely because it is durable or typed.
  4. Strongest counter-evidence — Implemented/Normative. Letta memory commits, Inspect crash recovery, DSPy trainset reuse, MCP host authority, and A2A duplicate delivery each leave a different unrecovered boundary. No subject supplies a universal exactly-once, authenticated, human-accountable agent loop.

The synthesis therefore confirms the catalogue’s boundary-first vocabulary, keeps new nouns track-specific unless a current definition already covers them, and closes without changing modules/, CLI code, or the canonical catalogue.