Manthan × swarm_evolution — review + progressive integration

Manthan × swarm_evolution — review + progressive integration

Goal (operator): *review each feature of Manthan, run the cross-model swarm to

improve it, apply changes progressively, post every iteration. 2 rounds.*

The "swarm" is the conduir cross-model review panel (infra/scripts/
multi-agent-review.sh in conduir-evolution-engine): the same packet is fanned
out to independent model families (gemini / deepseek / copilot) each with a
distinct lens; findings are then ground-truthed against the live repo before
anything is acted on. Panel output is ~50%+ false-positive by reputation, so
every finding below was re-verified against Manthan's actual code; verdicts that
the panel got wrong are recorded as such.

The thesis under test: *Manthan is a generative substrate (the "churn") with no
selection; swarm_evolution is a selection/fitness/quality-diversity engine
pointed only at code. Wire the second into the first and Manthan can actually
surface the "amrita" instead of accumulating artefacts forever.*

Round 1 — make the substrate alive + open the selection feedback loop

Swarm findings (ground-truthed)

The biggest defect the panel under-weighted (caught on ground-truth)

Embeddings are never authored. loop._post_artefact posted no
content_embedding; nothing else computed one. So:

• the substrate's HNSW pgvector index indexed nothing;
• /ideas/search is a documented stub (test_ideas_search_skeleton asserts

results == [], the route comment says "vector search lands when the runtime
starts authoring embeddings");

• novelty-based fitness and MAP-Elites behaviour descriptors (R2) had **no

signal to stand on**.

This is the keystone: nothing downstream in the evolution thesis works until the
vector substrate carries data. So R1 makes it the centerpiece.

Changes shipped (R1)

• runtime/embeddings.py (new) — embed(text, gateway) authors an embedding

via the gateway /embeddings route. Best-effort, honest-failure: returns
None on a down/absent adapter, malformed response, or wrong dimension (a
wrong-dimension vector would silently poison every kNN query — worse than
none). Tolerates OpenAI / native / batched response shapes.

• loop.py — computes the embedding before posting and stores it on the

artefact (embedded: bool recorded in metadata). The pgvector substrate and
/ideas/search now carry data.

• runtime/selection.py (new) —
• SeenCache: bounded-LRU replacement for the unbounded set (F1).
• rank_candidates: orders unseen candidates by read-side signal with

ε-exploration (default-off; pure recency when no signal map is supplied),
so generation can preferentially build on what the graph finds promising
without the high-signal artefact becoming a mode-collapse attractor (F2).

• config.py — embed_enabled (default on, safe), embed_provider/model,

seen_cache_max, selection_signal_bias (default off), selection_epsilon.

• Tests — test_embeddings.py, test_selection.py (18 cases): dimension

guard, every failure path returns None, LRU eviction/refresh, explore-vs-
exploit ordering, recency-passthrough when signal absent.

All changes are additive and default to the existing behaviour (recency churn)
except embedding-authoring, so the recency baseline is preserved for the A/B
test below.

What the swarm changed about the plan (deepseek Part B — the load-bearing critique)

"swarm_evolution solves the machinery of selection (cycle state machine,

ledger, gates, MAP-Elites grid) but not the meaning of selection — what

makes an idea fit? Code fitness is objective and cheap; idea fitness is a

proxy and gameable. Any single scalar fitness will be gamed within days."

This is correct and reshapes Round 2:
1. No single scalar fitness. R2 fitness is a vector of proxies, explicitly
labelled as proxies.
2. MAP-Elites only after embeddings exist (R1), else its behaviour
descriptors are noise and the grid is "a colourful heatmap of garbage."
3. The thesis must be falsifiable, not theatre. Before deep integration, run
the A/B kill-test below.

Falsifiable kill-test (carried into R2, from deepseek)

Control = current recency churn (selection off). Treatment = signal/fitness
selection on. After N artefacts, archive the bottom X% from the served feed,
feature the top per cell. Human blind-scores 100 random artefacts/arm on "would
you read this voluntarily?" *Kill criterion: treatment not better than control
at p<0.05.* If it fails, the thesis is dead — spend effort on persona prompts
and diversity scheduling instead of an evolutionary death march.

Open follow-ups (filed, not yet done)

• F5 periodic agent re-fetch / NATS presence subscription.
• F6 validate gateway content + finish_reason (no empty artefacts).
• Durable cross-restart seen-cursor in the substrate (F1 part 2).
• The NATS bus is designed in ARCHITECTURE.md but the runtime polls HTTP — the

manthan.bus.* subjects/presence/events are unused.

Round 2 — the quality-diversity selection brain

Round 2 builds on top of Round 1: novelty-based behaviour descriptors only
exist because R1 made embeddings live. The swarm was re-run (gemini + deepseek +
copilot, now under the kulbirminhas-aiinitiative account so it produced
output this round) to (a) adversarially review the R1 code I shipped and (b)
stress-test the R2 design before building it.

Changes shipped (R2)

• api/fitness.py (new) — artefact fitness as a vector of proxies, never

a single public scalar: engagement (lineage children weighted by relation —
converge positive, contradicts mild, being replaces-d a penalty),
depth (converge-chain maturation), recency (14-day half-life). Carries an
explicit honesty block naming each component a proxy. The scalar is a
documented within-cell tie-break only.

• api/map_elites.py (new) — MAP-Elites quality-diversity archive. Cells =

(kind, novelty_band); novelty is the descriptor, not the fitness, so
"gibberish maximises distance" buys a far cell but never wins it. Bounded
long-tail protection (a young incumbent is shielded only within
grace_days AND a sub-evict_margin lead) protects the slow-maturing
"amrita" without freezing the archive. elites_from_lineage computes the
whole thing from the lineage graph; should_archive is the A/B-test primitive.

• /elites Read API route — serves the QD set (what Conduir should consume)

instead of naive recency.

• Tests — test_fitness.py, test_map_elites.py, test_elites_route.py

(+ R2 folds below). 57 tests green across runtime + api.

Swarm findings folded (ground-truthed; convergent = ≥2 families agreed)

Residual risks recorded honestly (not solved by reuse)

• Novelty is still a proxy. Descriptor-not-fitness defuses the win, but a

quality/dedup gate before novelty counts is the real fix (deferred).

• Fitness has no LLM-judge in this layer — deliberately, to avoid the

LLM-circularity failure mode. That caps how "smart" fitness can be; the
falsifiable A/B kill-test (control = recency, treatment = /elites, human
blind-score, kill if not better at p<0.05) is the gate before trusting it.

Net

Two progressive rounds: R1 made the substrate carry data; R2 turned that data
into a diverse-and-good selection surface that defuses the panel's three
top-rated attacks (gibberish-novelty, self-citation, archive-squatting) by
construction, with the rest recorded as honest follow-ups rather than hidden.
The swarm didn't just find bugs — deepseek's "machinery vs meaning" critique and
the 3/3 gaming findings shaped the design, which is the whole point of
running it.

Round 3 — sweep the un-reviewed components + close the end-to-end gaps

R1/R2 covered the runtime churn + the api selection brain. R3 ran the full panel
(gemini + deepseek + copilot) over the substrate data layer and the
runtime lifecycle — the components not yet reviewed — and closed the gaps
that stopped the R1/R2 work from functioning end-to-end.

Swarm findings (ground-truthed; all the top ones were 3/3 convergent)

Deliberately NOT done (panel agreed — gold-plating / unproven)

• children_count distinct-child accuracy (display-only stat).
• PATCH /artefacts/{id} backfill endpoint (a one-off script is simpler).
• DB-level per-tick lineage-dedup (bounded SeenCache re-reaction is a rare,

intentional edge case).

• content_text max-length / finish_reason rejection — noted as a small

follow-up, not load-bearing.

All three families explicitly confirmed *none of these fixes regress the
honest-failure / no-fabrication posture — the empty-content guard strengthens*
it.

Tests

65 green across runtime + api (pure logic: reconcile, content-guard, signal-map
parse, novelty-cells-light-up-with-embeddings). The two substrate routes are
exercised by import + pgvector-comparator resolution; full e2e needs Postgres
(same posture as the pre-existing ideas_search skeleton test).

Status after R3

The selection brain is now wired end-to-end in principle: embeddings are
authored (R1), exposed + searchable (R3), turned into a fitness vector + QD
archive (R2), and the generative loop can consume read-side signal (R3,
default-off). What remains is runtime validation (the falsifiable A/B
kill-test on a live stack) and the smaller deferred items above — not new
architecture.

Round 4 — read-side scoring hardening + convergence

R4 ran the full panel over the last un-reviewed product-layer components:
signal.py (signal-from-noise, now also feeding parent-selection), patina.py
(reputation), and maestro-bridge/translate.py.

This was the convergence round. gemini reported no MEDIUM+ findings
("architectural design … appears sound"); deepseek surfaced nothing actionable;
copilot found a small set of real edge-cases. Folded (ground-truthed):

19 new tests (signal, patina, maestro-bridge). 79 green across runtime + api

• maestro-bridge.

Convergence reached

Across four rounds the panel went from many CRITICAL/HIGH findings (R1/R2/R3) to
near-zero in R4 (one family found nothing at all). The remaining backlog is
either explicitly panel-agreed gold-plating (PATCH backfill, per-tick DB
dedup, children_count distinctness, adaptive novelty bands, Pareto-per-cell) or
requires a live Postgres + gateway stack (the falsifiable A/B kill-test,
substrate-route e2e) — i.e. runtime validation, not further code architecture.

Stopping here is the honest call (for the churn/selection components):
another swarm round on them would manufacture make-work. The integration is
architecturally complete and tested; the next real step is operational (stand up
the stack, run the A/B kill-test), not another review pass.

Round 5 — the infra / entrypoint components (db, llm, budget, the bridge)

R4 saturated the product-logic components, but the *infrastructure /
entrypoint layer had never been swept. R5 ran the panel over substrate/db.py,
runtime/llm.py, runtime/budget.py, and maestro-bridge/main.py (NATS
subscriber + promotion poller). This round found real* issues, so saturation was
not yet global.

Folded (ground-truthed; gemini + deepseek convergent):

19 cases added; 85 tests green across all four services.

Saturation reached (global)

Five rounds, descending yield: R1–R3 many CRITICAL/HIGH → R4 near-zero on product
logic → R5 a final real batch on infra, now also drained. What remains is
exclusively documented roadmap (persistent budget / promoted-ids, extract the
shared patina package, NATS durable subscriptions) or *panel-agreed
gold-plating*. No component has an un-actioned real defect. The honest call is
that the codebase is saturated against this review process — the next
increments are operational (run the stack + the A/B kill-test) and product-roadmap
decisions, not more review passes.

(Update: a later directive reopened the loop in exploratory mode — see R6+.)

Round 6 — config / bootstrap / entrypoints, Six-Hats (critical + exploratory)

The defect-axis saturated at R5, but the config/bootstrap/entrypoint layer
had never been reviewed, and a new directive asked for a critical AND
exploratory pass (de Bono's Six Hats). The panel (gemini + deepseek + copilot)
applied white/black/yellow/green/red/blue hats over shared/settings.py,
shared/logging.py, substrate/scripts/seed.py, and the FastAPI entrypoints.

Folded (ground-truthed; bounded, testable unit-improvements):

Rejected with reason: env_prefix="MANTHAN_" and extra="forbid" (both break
the existing .env contract), trace_id middleware (gemini cross-referenced the
engine repo's GEMINI.md, not manthan's), llm_model pinning (documented design
choice). 23 cases added; 94 tests green (runtime 46 / api 40 / substrate 1 /
maestro-bridge 7).

Takeaway: the exploratory axis is not yet saturated even where the defect
axis was — Six-Hats surfaced real, bounded unit-improvements (secret-handling,
dead-config wiring, readiness, admin-safety) that pure defect-hunting missed.
Remaining un-swept: the Next.js viewer, schema/migration, and infra
(Dockerfiles/compose) — next rounds.

Round 7 — schema + the Next.js viewer (Six-Hats)

The last two un-reviewed components: the DB schema (models + migration) and
the viewer frontend. The panel independently re-derived the schema
constraints (copilot/gemini/deepseek all flagged self-loop + non-blank — which I
had implemented from my own ground-truth, a nice cross-check) and surfaced real
frontend defects.

Schema (CI-verified where possible):

Viewer (TypeScript — no frontend test harness yet, so review-verified):

Rejected/deferred: inline-styles→CSS-modules (gemini: "not critical" for a demo),
explicit PrimaryKeyConstraint (SQLAlchemy handles it — stylistic), touching the
already-applied migration 0001 (DROP TYPE … CASCADE, tags default — uncertain

• risky on a deployed migration). copilot's yellow-hat confirmed the viewer has

no XSS (dangerouslySetInnerHTML absent).

Global saturation — restated

Every component now reviewed: backend logic (R1–R5), config/bootstrap/entrypoints
(R6), schema + frontend (R7). Across seven rounds the yield fell to bounded
edge-cases. What remains is infra polish (compose resource limits) and
documented roadmap (persistence, shared-patina extraction, a frontend test
harness) — not unit-improvements to existing logic. The codebase is saturated
on both the defect and exploratory axes against this review process. Note: the
viewer + maestro-bridge are not in the merge-blocking CI matrix (python =
substrate/runtime/api only), so viewer changes are review-verified, not CI-gated.

Round 8 — infra / containers / compose (FINAL, Six-Hats)

The last component category: service Dockerfiles, docker-compose.yml, Makefile.
Panel (deepseek + copilot; gemini empty this run). Folded the bounded,
safe-to-apply-without-a-build items:

Deferred (needs a build/lockfile, can't verify here): multi-stage slim images +
purge build tools (R8 image-bloat), install from a generated lock/constraints
file (dep-drift), digest-pin base images. Left as-is with reason: compose DB
password default (the R6 manthan_env app-guard already refuses it in prod;
requiring it would break dev make up).

FINAL STATUS — saturation across every component

Eight rounds, every component category reviewed with both a critical and (R6+)
an exploratory Six-Hats lens, every finding ground-truthed (false positives
rejected with reasons each round):

• R1–R5 backend logic — churn loop, selection brain (fitness + MAP-Elites),

substrate routes, signal/patina, maestro bridge, llm/budget/db.

• R6 config / bootstrap / entrypoints / shared lib.
• R7 schema + the Next.js viewer.
• R8 infra — containers / compose / Makefile.

The yield per new area followed the same curve: a real batch on first review,
then drained to bounded edge-cases. Everything still open is now one of:
documented roadmap (persistent budget/promoted-ids, extract the shared patina
package, NATS durable subscriptions, a frontend test harness, a dep-lock file) or
ops choices that can't be unit-verified in this partial-push repo (multi-stage
images, digest pinning). No component carries an un-actioned, in-scope,
unit-improvable defect.

The codebase is saturated against this review process. The next increments
are operational (stand up the stack; run the falsifiable A/B kill-test that
decides whether selection-over-fitness beats recency) and product-roadmap
decisions — not further review passes.