What building a second project taught me about the first

The first bug ludexel ever found in fabri had nothing to do with games. It was a Python import. ToolRegistry had a method literally named list(), and thanks to PEP 649’s deferred annotation evaluation, that method name shadowed the list[dict] type hint on the same class — a crash that only showed up on an import from outside my own test suite. The commit that fixed it, b639850, says it plainly: “Reported by the ludexel service which couldn’t import fabri.” Fourteen hours after fabri’s rename from “agent-memory framework” to something with a name, a real product broke it in a way none of my own passing tests had thought to check.

That’s the pattern that repeats for the next three weeks — the actual subject of this post. Building ludexel, an AI-native game studio that turns a prompt into a real Game Boy Advance ROM, taught me fabri faster than staring at fabri ever did. A real product kept giving the framework jobs it could fail at, and each failure was the lesson.

Eviction shipped before anyone filed a complaint

fabri’s memory loop — retrieve, run, mine, dedup, promote, retrieve again — sounds like a diagram you’d draw before writing code, and for a while it was one. Eviction is the one piece here I can’t honestly credit to ludexel-gba’s real usage: memory.max_entries defaults to None (unbounded), and ludexel-gba’s production yaml never sets it either — the store has never actually hit a cap in a live build. I shipped the eviction machinery anyway, on the bet that unbounded growth was a ticking bug rather than a complaint anyone had filed yet.

What shipped is ca14c33, “summarize-before-evict strategy for memory eviction” — MemGPT-style recursive summarization. Every guideline starts out tactical (session-scoped, disposable — the store’s default kind) and only gets promoted to strategic (protected from eviction) once a near-duplicate recurs across three separate sessions, so the store doesn’t amplify a fluke into permanent memory. Eviction chunks the rest in groups of five, compressing each group into one new tactical guideline; strategic entries are summarized individually instead so they don’t flatten into the wrong bucket; a plain hard delete is the fallback if summarizing fails. Five in, one out — a net reduction of four per pass, small enough that the source comment flags the store can sit “1-2 entries over limit until the next ingest triggers a follow-up eviction pass.”

Where the COGS number was lying

fabri shipped per-run COGS tracking in v0.5.0 — cost_usd, subagent_cost_usd, total_cost_usd on every usage event. It felt complete until v0.7.8, “close COGS leaks in planner, decompose, compress, crashed sub-agents,” found four LLM call sites burning provider tokens without rolling into the reported total — recorded COGS was underreporting real spend by roughly 10x on planner-engaged runs. A crashed sub-agent still cost money; the number just didn’t say so.

Ludexel-gba’s own .agent/fabri_agent.yaml carries the scar tissue directly:

max_steps: 48
max_cost_usd: 8.0        # hard halt -> budget_exceeded

subagent:
  max_steps: 14
  max_cost_usd: 3.0       # domain child needs far less than the orchestrator

That split exists because, without it, every spawn_subagent child inherited the orchestrator’s max_steps: 48 — a looping domain child could burn the entire run’s budget on its own (fixed in v0.7.5, commit 0734f15, which added the agent.subagent budget block). A normal cold build now runs about $2–4.

Measurement follows dependence

For most of fabri’s life, retrieval quality was a vibe. fabri searches memory two ways at once — dense (embedding similarity) and sparse (an FTS5 keyword index) — and fuses the two rankings; “hybrid” means both are supposed to be running together. It got measured in v0.10.0, “Measured retrieval” — the scoping commit says why: “Observability + an offline eval gate land before any retrieval-quality upgrade — quality was unmeasured.” That gate (40 guidelines, 24 queries, real IR metrics, CI floors set at measured-minus-0.05) immediately found a real bug: _fts5_query was space-joining tokens, and SQLite FTS5 reads a bare space as an implicit AND, so any multi-word query matched nothing and “hybrid” retrieval silently degraded to dense-only. Fixing the join pushed hybrid recall@5 — the fraction of relevant results that actually land in the top 5 — from 0.79 to 0.94.

Retrieval fuses the dense and sparse rankings with Reciprocal Rank Fusion (RRF): each result’s score is 1/(k + its rank) summed across both rankings, so two-way agreement can outrank a result only one search liked. The textbook k=60 is borrowed from web-scale search with pools in the thousands; at fabri’s scale — two short pools, a dozen candidates each — that flattens the formula enough that mere agreement beats the single best match either pool found. Retuning k down to 20 pushed hybrid recall@3 from 0.60 to 0.90.

Both fixes landed the same week fabri went from 0.8.2 to 0.10.0 across the Kiln Road build — a real 5-scene arc, authored end to end by the agent stack — the first stretch where what got retrieved actually mattered to whether a scene got authored right.

The retrospective became the prompt

The single cleanest artifact of the whole loop is a commit called 1ce60a1: “docs(retro)+prompts: fold Kiln Road v2 retrospective into fabri domains.” The Kiln Road v2 session ran 2026-07-07 to 2026-07-08, roughly 50 commits, structured as “every mistake, missed verification, and bug from this build, with its root cause and the reusable lesson.” Days later, that retrospective is literally inside the operating prompts: map.md got a camera-frame quality contract after three review passes all signed off on maps the user rejected on sight (“The whole maps are looking bad. Like really bad.”), because reviewers judged the 32×32 map file instead of the 15×10-tile window the camera actually shows.

The same session produced 13cae9d, the sharpest failure in the whole corpus: the orchestrator prompt had learned to say “New ROM packed — go play it” the moment validate_content passed — except validate_content only runs the Python generator chain (chargen, storygen, fontgen, scenegen), never the actual devkitARM compile. An undeclared ENEMY_SLIME symbol sailed straight through and shipped as “MISSION COMPLETE.” The fix wasn’t a smarter model — it was removing a claim the prompt had no right to make. ffed0b3 — the no-progress abort, the project_symbols pre-authoring lookup, the 720-second wall-clock backstop — reads like a checklist compiled directly from that bug list, because it was.

ludexel-app shipped its own Improver on July 3rd: a harness that mines failed runs, ranks failure classes, and writes a structured brief per class, sharing the docstring constraint — “NOT an autonomous editor… Nothing here edits code.” Two days later, fabri shipped its own Improver (fabri ingest, v0.9.2), opening the memory loop to any external log. Same author, same failure mode, twice — the two codebases never shared a line of code.

What’s still open

I’d like to end on the retrospective-becomes-prompt story — the tidiest proof the loop works. But onenter_battle_smoke is still quarantined in ludexel-gba’s known issues, at commit cd34691 (“quarantine onenter_battle_smoke — story-content drift”) — a battle whose flag layout drifted out from under its own smoke test, sitting there deliberately unfixed while the other six smokes stay green. The loop that turns ludexel’s failures into fabri’s features runs both directions, but it doesn’t run on its own — right now it’s just waiting on me to go fix a battle nobody’s gotten around to yet.