Phase 2 T44 deferred interjection regenerate — when the original turn
group included a follow-on interjection beat we left it untouched. Now
regenerate redoes BOTH halves:
- Detect a sibling interjection by looking up assistant_turn events
pinned to the same user_turn_id with `interjection_of` set.
- After streaming the new primary, run `detect_interjection` against
the new primary text.
- If True: stream a new interjection from the silent witness, append
with `interjection_of=<new primary speaker_id>`, supersede the
original interjection, and re-run memory + state-update for the new
beat.
- If False: supersede the original interjection without a replacement
(back-pointer goes to the new primary so the row stays consistently
hidden).
Also broadcast a `turn_html_replace` event for the new interjection so
the front-end can swap the prior interjection node in place (mirrors
T73.1's primary swap).
Tests:
- `test_regenerate_with_interjection_redoes_both_turns`: classifier
returns True; assert two new assistant_turns land for the same
user_turn, second carries `interjection_of`, originals superseded.
- `test_regenerate_drops_interjection_when_classifier_returns_false`:
classifier returns False; assert one new assistant_turn (primary
only) and the original interjection is superseded with no
replacement.
`interjection_of` carries the primary's *speaker_id* (matching the
existing convention in chat/web/turns.py) rather than the event_id.
After the new assistant_turn lands, publish a `turn_html_replace` SSE
event carrying the rendered HTML, the new turn_id, and the original
assistant_turn id as `supersedes_id` so connected tabs can swap the
prior DOM node in-place. Phase 1 T29 deferred this — page had to refresh
to see the regenerated turn.
Uses a new event name (not the existing `turn_html`) because the HTMX
`sse-swap="turn_html"` consumer expects raw HTML and an *append*
semantic; regenerate is a *replace*. The new event ships as JSON
(supersedes_id forces sse.py's JSON branch) so the front-end JS can
read the swap target from the payload.
Test: `test_regenerate_broadcasts_turn_html_over_sse` patches the
`publish` reference inside the regenerate module and asserts the
event shape.
Memories grow per-flag witness checkboxes (you / host / guest) that
auto-submit on change via HTMX. The new POST route emits a manual_edit
event with target_kind=memory_witness and a {flag, value} payload;
prior_value mirrors the same shape so an inverse edit restores the
flag. The drawer's recent-memories query now selects the three
witness columns alongside the existing fields so the template can
render checkbox state without a second query per row.
When a host->candidate edge already exists from a prior chat, the
Add-guest form renders the prose textarea disabled with an "already
know each other" note. Submission without the explicit "re-seed
anyway" toggle skips seed_inter_bot_edges so existing edge content
(affinity, trust, knowledge, summaries) survives — guest_added and
group_node_initialized still fire. A small inline script enables /
disables the textarea per-option based on a pre-computed
existing_guest_edges dict surfaced by the GET handler.
Adds the four POST routes whose state-layer support was already
dispatched by the manual_edit projector (edge_trust, edge_summary,
memory_pov_summary) plus a new edge_knowledge_fact dispatch branch for
add/remove fact list manipulation. Drawer template gains editable
textareas, sliders, and add/remove fact controls. Remove semantics on
knowledge_fact match by string (not index) so concurrent edge_update
events appending facts between drawer renders don't desync the form.
T18 review (Phase 1) noted the NICE-tier trim drops previous-scene
FIRST while §6.3 spec lists previous-scene LAST in the NICE tier
group. Decision: keep the existing greedy order (previous-scene
first), and document why.
Rationale (now in code at the trim ladder):
1. Cheapest-impact-first — a per-POV previous-scene summary loses
less narrative continuity than the older dialogue turns or
memory hits it competes with.
2. Greedy lookahead is more expensive than the marginal narrative
loss. Dropping previous-scene typically clears the soft-budget
slack in one step.
Test added: test_nice_trim_order_documented pins the observed order
(previous-scene -> memories -> dialogue) so a future refactor can't
silently invert it. Sized so that all-NICE config overflows soft but
dropping just previous-scene fits — proves memories and older
dialogue turns survive while previous-scene is the FIRST drop.
Phase 2 T43 added a SECOND ACTIVITIES: block to render guest activity
separately from you+speaker. Two consecutive ACTIVITIES: headers can
read as a duplicate-section bug to the LLM and bloat the prompt.
Consolidate to a single ACTIVITIES: block whose body is composed from
up to three bullets (you, speaker, guest). The block itself is
MUST-tier (always renders); bullet-level trim drops bullets in the
order guest -> group node -> you -> other edges, with the speaker
bullet as the MUST-tier floor (the speaker's own current activity is
the load-bearing slice).
Implementation chose Option B from the polish plan: pre-truncate the
bullets list at trim time before _build_activity_block runs, rather
than introduce a granular tier mode in the trim machinery. Rationale
documented in code; the existing block-level trim ladder gains a
single new toggle (include_you_activity) and the SHOULD-tier
guest_activity_block is gone.
Tests:
- test_single_activities_block_with_three_bullets_when_3_entities:
exactly one ACTIVITIES: header with all three entity bullets.
- test_tight_budget_drops_guest_activity_bullet_first: speaker bullet
survives, guest bullet absent under tight budget.
- Existing test_assemble_with_tight_budget_drops_guest_activity_first
still passes (asserts on bullet absence, not block-header absence).
Phase 2 T46 pinned the witness mask contract on search_memories with a
witness_role parameter (host/guest/you). The prompt-assembly call site
in assemble_narrative_prompt was hardcoded to "host", which silently
returned the wrong rows when the speaker was the guest bot.
Derive the witness role from chat membership via a new private helper
_witness_role_for(speaker_bot_id, host_bot_id), and apply it at the
search_memories call. Behaviour is identical when the speaker is the
host (or when no guest is present); the fix is load-bearing only when
the guest bot is the speaker — exactly the scenario Phase 2 T43 added
support for.
Tests: pin both directions (host-as-speaker and guest-as-speaker) by
patching the imported search_memories reference and asserting the
witness_role argument the call site emits.
19 tasks across 8 waves covering events with lifecycles, time skips
(elision + jump), active threads, significance/retrieval refinements,
and meanwhile scenes (host+guest with no 'you'). Mirrors the Phase 2
plan structure: pre-flight, parallel-execution strategy with worktree
isolation, file-disjointness analysis per wave, and per-task TDD spec
with commit messages.
Phase 3 schema: adds 0009_events.sql, 0010_threads.sql,
0011_meanwhile_scenes.sql (final version 11). Builds on Phase 2's
3-entity scene support and event-sourced architecture.
Rewrites post_turn for the multi-entity world:
- Addressee detection via case-insensitive whole-word match against the
guest name; defaults to host on no-match or both-match.
- Multi-entity prompt assembly: forwards guest_id so the prompt sees
the third party's activity / edges / group-node.
- Multi-witness memory write: record_turn_memory_for_present writes one
memory per present bot witness when a guest is in the room.
- Multi-pair state-update: compute_state_updates_for_present emits one
edge_update per directed pair (6 with a guest, 2 without).
- Interjection branch (T39): when a guest is present and the primary
beat completes, the silent witness may follow on. detect_interjection
decides; on True we stream a second narrative as the witness, append a
second assistant_turn linked to the same user_turn_id, and re-run the
multi-pair state update + memory write for the follow-on beat. Cancel
collapses both halves; a cancelled interjection skips its downstream
passes so we don't classifier-spam against a half-formed beat.
- Scene-close runs after both beats so apply_scene_close_summary sees
the full closing scene; T45's guest-aware summarizer handles per-POV
rewrites for each present witness.
regenerate.py mirrors the prompt / memory / state-update changes for
1:1 and multi-entity scenes. Per the Phase 2 spec, interjection
regeneration is deferred to Phase 2.5 — regenerate only re-streams the
addressee turn for v2.
Tests: adds 5 cases to tests/test_turn_flow.py covering the no-guest
regression, multi-bot without interjection, multi-bot with interjection,
scene-close per-POV rewrites, and addressee routing on a named-bot
prose. Each test pins its own canned MockLLMClient queue with the call
shape documented in the docstring.
Bot replies were running long (4 paragraphs of action+dialogue beats
per turn) because we never set max_tokens on the narrative call. Three
tunable knobs now in Settings (set in data/config.toml to override):
- narrative_max_tokens: int = 400
Hard cap on each generated response. ~400 tokens ≈ 1–2 short
paragraphs. Drop to 200 for terse banter, bump to 800+ for longer
scenes.
- narrative_temperature: float = 0.85
Sampling temperature. 0.7 = grounded/consistent (slightly stiff),
0.85 = creative-but-in-character (default), 1.0 = wide variety,
>1.0 = often off-the-rails.
- prompt closing instruction now nudges: "Keep your response to a
single beat — one or two short paragraphs at most. Don't monologue;
leave room for the other person to react."
Both turns.py (post_turn) and regenerate.py forward the params to
client.stream(). FeatherlessClient already passes **params through to
the OpenAI-compat endpoint.
Note: temperature doesn't control length — that was a common
misconception. max_tokens is the actual length cap. Lower temperature
makes word choice more predictable (slightly stiffer voice), not
shorter. Both knobs are useful for different goals.
The form-submit handler in chat.html was setting
``textarea.disabled = true`` synchronously before the browser actually
serialized the form. Disabled form fields are excluded from
submission, so the request body contained ``prose=""`` even when the
user had typed text — which the server (correctly) rejected with the
new empty-prose 400. Net effect: typing "hello" + Send gave a "prose
cannot be empty" error.
Switched to ``readOnly``: same UX (user can't edit while streaming)
but the field IS submitted. The unlock path now also clears the
textarea and refocuses for the next turn.
Empty submission was producing a blank user_turn event in the log and
firing the LLM stream anyway — the bot would invent a response from the
kickoff context alone, producing a monologue with no user input. Two-
layer fix:
- Browser: add `required` to the prose textarea in chat.html so the
form refuses to submit empty.
- Server: 400 in post_turn when prose.strip() is empty. Defense in
depth — if a client bypasses the textarea attribute (custom UI,
curl, etc.), the server still rejects.
Verified live: POST with empty body returns 400; POST with whitespace-
only returns 400; chat shell renders the textarea with required.
Full suite: 168 passed.
Two related issues blocking real-world use of the kickoff parse:
1. Classifier calls take ~12s end-to-end on Featherless for the
complex KickoffParse schema (Hermes-3-8B generating ~1.3KB of
structured JSON). The 10s timeout was firing on most attempts,
causing all 3 retries to time out and the empty-fallback to render
with blank form values. Bumping the default
classifier_timeout_s 10 → 30s gives generous headroom; measured
p99 is ~13s, so 30s is comfortable.
2. Featherless caps concurrent connections per account (2 on free /
lower paid tiers). Each turn flow can fire 4–5 calls (parse,
scene-close detect, narrative stream, two state-update passes)
plus the background significance worker. Without a gate, we'd
exceed the cap and fail.
Added a class-level ``asyncio.Semaphore`` to FeatherlessClient,
shared across all instances, configured once in lifespan from
``Settings.featherless_max_concurrent`` (default 2). Both
``generate`` and ``stream`` acquire the semaphore for the duration
of the call; the stream holds it until the async generator
completes, so token streaming is correctly accounted for.
Verified live: 4/4 sequential kickoff parses for the same bot all
succeed with real parsed values (previously ~50% blank-fallback).
Full suite: 168 passed.
Joseph Doherty