108 Commits

Author SHA1 Message Date
Joseph Doherty 29e6f346ef merge: T79 _witness_role_for defensive None handling 2026-04-26 21:42:24 -04:00
Joseph Doherty cb570a5adc merge: T78 search_memories docstring SQL-bias note 2026-04-26 21:42:24 -04:00
Joseph Doherty ce4f56adfa merge: T77 AddresseeDecision.confidence as Literal 2026-04-26 21:42:24 -04:00
Joseph Doherty bdc93b4b67 merge: T76 narrate_skip timeout_s plumbed through 2026-04-26 21:42:24 -04:00
Joseph Doherty 9c9d71eb31 fix: _witness_role_for defensive None handling (T79) 2026-04-26 21:41:15 -04:00
Joseph Doherty 4199038b8b fix: AddresseeDecision.confidence as Literal[high|medium|low] (T77) 2026-04-26 21:40:47 -04:00
Joseph Doherty e3dfe18811 docs: search_memories docstring mentions SQL-side significance bias (T78) 2026-04-26 21:40:40 -04:00
Joseph Doherty d759b90aa1 fix: plumb narrate_skip timeout_s through to client.generate (T76) 2026-04-26 21:40:29 -04:00
Joseph Doherty fb7e97260b docs: add Phase 3.5 cleanup plan (Phase 2.6/3 + 3.5/4 backlog burn-down)
12 tasks across 7 waves consolidating the 17-item backlog tracked in
CLAUDE.md (7 from Phase 2.6/3 + 10 from Phase 3.5/4). Items are
grouped by file ownership so each wave stays file-disjoint:

- Wave 1 (parallel 4-way): trivial single-line/single-file fixes
  (timeout_s plumbing, Literal type, docstring, defensive None)
- Wave 2 (single): scene_summarize.py polish bundle (5 T58 items)
- Wave 3 (single): typed ChatNotFoundError for skip routes
- Wave 4 (single): turns.py wiring (consume_pending_meanwhile_digests
  + natural-language skip runs scene close detection)
- Wave 5 (single): regenerate.py polish (cancel hook + DRY +
  sibling query + lifecycle rollback documentation + ordering)
- Wave 6 (parallel 3-way): unified record_turn_memory API + JSON
  audit + frontend turn_html_replace SSE handler
- Wave 7 (single): docs sweep

No schema migrations. Bundled tasks split into per-item sub-commits
for clean review bisection. Uses task ids T76-T87 to avoid collision
with prior phases (Phase 3 used T49-T67, Phase 2.5 used T68-T75).
2026-04-26 21:33:16 -04:00
dohertj2 753cec327f Merge pull request 'Phase 3: events, time skips, threads, meanwhile scenes' (#4) from phase-3 into main 2026-04-26 21:26:49 -04:00
Joseph Doherty 70a5ad3ecc docs: add T66-discovered consume_pending_meanwhile_digests backlog item 2026-04-26 21:19:11 -04:00
Joseph Doherty 6709cf46a7 merge: T67 phase 3 documentation update 2026-04-26 21:18:38 -04:00
Joseph Doherty c3947bbb68 merge: T66 phase 3 cross-feature integration coverage 2026-04-26 21:18:38 -04:00
Joseph Doherty f865ac2ee2 test: phase 3 cross-feature integration coverage (T66) 2026-04-26 21:16:30 -04:00
Joseph Doherty af6c54dd05 docs: phase 3 status, behavioral defaults, deferred items (T67) 2026-04-26 21:10:49 -04:00
Joseph Doherty dc35833534 test: feed meanwhile digest canned response after Wave 6b cross-feature merge 2026-04-26 21:07:44 -04:00
Joseph Doherty 0cd41636b3 merge: T65 meanwhile summary digest surfaces to next you-scene 2026-04-26 21:06:10 -04:00
Joseph Doherty 2c7aa68af9 merge: T64 meanwhile turn flow (host+guest, no you) 2026-04-26 21:06:10 -04:00
Joseph Doherty cf43ba0993 feat: meanwhile turn flow (host+guest, no you) (T64) 2026-04-26 21:05:40 -04:00
Joseph Doherty a781732ee6 feat: meanwhile summary digest surfaces to next you-scene (T65) 2026-04-26 20:59:35 -04:00
Joseph Doherty c9d58b8229 merge: T63 meanwhile scene schema + state 2026-04-26 20:52:51 -04:00
Joseph Doherty c463dc70b2 feat: meanwhile scene schema + state (T63) 2026-04-26 20:52:45 -04:00
Joseph Doherty 819803da84 merge: T62 natural-language skip command flow + shared skip controllers 2026-04-26 20:47:07 -04:00
Joseph Doherty a7eedb8037 feat: natural-language skip detection + skip command flow (T62)
Extend ParsedTurn with intent/landing_state_hint so the classifier can
flag skip-elision and skip-jump prose. The post_turn handler short-
circuits the regular narrative path when intent != "narrative":
elision runs through the shared controller in chat/web/skip.py;
jump returns 422 directing the user to the drawer's structured form
(simpler Phase 3 path — natural-language fiction-time delta parsing
is too fragile for v1 without a structured surface).

Extract the elision/jump logic that previously lived in drawer.py
into chat/web/skip.py so both the drawer T59 routes and the new
natural-language path share one canonical implementation. The drawer
routes become thin HTTP wrappers that translate ValueError to 400
and refresh the drawer partial; the existing drawer skip tests pass
unchanged.

The new natural-language elision derives ``new_time`` by bumping the
chat clock by 1 hour (Phase 3 stub) — the drawer's structured form
remains the path for picking a specific landing time.
2026-04-26 20:45:05 -04:00
Joseph Doherty e236bcadcd merge: T61 per-turn event-lifecycle detection + completion promotion 2026-04-26 20:37:21 -04:00
Joseph Doherty 3678bcaca6 merge: T60 prompt assembly active events + open threads 2026-04-26 20:37:21 -04:00
Joseph Doherty b582567521 feat: per-turn event-lifecycle detection + completion promotion (T61) 2026-04-26 20:35:34 -04:00
Joseph Doherty 21c4ffa63c feat: prompt assembly renders active events + open threads (T60) 2026-04-26 20:34:26 -04:00
Joseph Doherty 83f94a4325 merge: T59 drawer events / threads / skip controls 2026-04-26 20:29:40 -04:00
Joseph Doherty 2d14197553 feat: drawer events / threads / skip controls (T59) 2026-04-26 20:27:47 -04:00
Joseph Doherty 8efbcdf6c3 merge: T58 scene compression + thread emission on close 2026-04-26 20:21:01 -04:00
Joseph Doherty 8aeadfd0e4 merge: T57 significance-aware retrieval ranking 2026-04-26 20:21:01 -04:00
Joseph Doherty 88350d7d2e merge: T56 event-completion promotion service 2026-04-26 20:21:00 -04:00
Joseph Doherty 343f305587 feat: significance-driven quote retention + thread emission on close (T58) 2026-04-26 20:18:34 -04:00
Joseph Doherty 021587b3df feat: event-completion promotion service (T56) 2026-04-26 20:15:51 -04:00
Joseph Doherty 5e6b29e0c5 feat: significance-aware retrieval ranking (T57) 2026-04-26 20:15:19 -04:00
Joseph Doherty a34931375c merge: T55 thread-detection service 2026-04-26 20:12:12 -04:00
Joseph Doherty 959fe11410 merge: T54 synthesized-memories service 2026-04-26 20:12:12 -04:00
Joseph Doherty 2959e1ac2a merge: T53 skip narration service 2026-04-26 20:12:12 -04:00
Joseph Doherty afe940259a merge: T52 event-lifecycle detection service 2026-04-26 20:12:12 -04:00
Joseph Doherty c2144cd9df feat: skip narration service (T53) 2026-04-26 20:10:42 -04:00
Joseph Doherty 7857da4112 feat: thread-detection service (T55) 2026-04-26 20:10:36 -04:00
Joseph Doherty adbbd32873 feat: synthesized-memories service for jump skips (T54) 2026-04-26 20:10:05 -04:00
Joseph Doherty 98250644ad feat: event-lifecycle detection service (T52) 2026-04-26 20:09:13 -04:00
Joseph Doherty da1f67fb6a test: bump schema_version assertion to 10 (0009 events + 0010 threads) 2026-04-26 20:07:08 -04:00
Joseph Doherty 03ba34272b merge: T51 threads table + projector handlers 2026-04-26 20:06:45 -04:00
Joseph Doherty e26885b011 merge: T50 time_skip event handlers 2026-04-26 20:06:45 -04:00
Joseph Doherty 5b7a195cf5 merge: T49 events table + lifecycle handlers 2026-04-26 20:06:45 -04:00
Joseph Doherty 25bcbac055 feat: threads table + projector handlers (T51) 2026-04-26 20:05:09 -04:00
Joseph Doherty ab2b494c21 feat: time_skip event handlers (T50) 2026-04-26 20:04:46 -04:00
Joseph Doherty b6888ff36a feat: events table + lifecycle handlers (T49) 2026-04-26 20:04:36 -04:00
dohertj2 e4fd888b53 Merge pull request 'Phase 2.5 cleanup: 15-item backlog burndown' (#3) from phase-2.5 into main 2026-04-26 20:00:38 -04:00
dohertj2 079774dce5 Merge pull request 'Phase 2: multi-entity scene support (you + host + guest)' (#2) from phase-2 into main 2026-04-26 20:00:16 -04:00
dohertj2 3be7920f41 Merge pull request 'Phase 1: v1 single-bot roleplay engine' (#1) from phase-1 into main 2026-04-26 19:59:29 -04:00
Joseph Doherty e61bd9cb08 merge: T75 phase 2.5 docs sweep + phase 2.6 backlog 2026-04-26 17:47:01 -04:00
Joseph Doherty c6e0130e59 docs: phase 2.5 status, prune shipped backlog items, capture phase 2.6 follow-ups (T75) 2026-04-26 17:46:50 -04:00
Joseph Doherty 67d6f3fe68 merge: T74 turn-flow polish + addressee service 2026-04-26 17:43:04 -04:00
Joseph Doherty dbc9690358 merge: T73 regenerate.py polish (turn_html SSE + interjection regenerate + stale-guest cleanup) 2026-04-26 17:43:04 -04:00
Joseph Doherty 6d98728a2e chore: remove defensive stale-guest degrade in turns.py (T74.4)
T44 carried a defensive degrade-to-1:1 block in post_turn for the
case where chat.guest_bot_id pointed at a deleted bot. T47 then
fixed the root cause by adding a bot_reset cascade that clears
guest_bot_id from any chat that referenced the deleted bot, so the
post_turn defensive block was rendered dead.

Remove the orphan-clear branch and replace it with a comment
documenting that get_bot now returns a real row when guest_bot_id
is non-None. The cascade behavior is pinned by
test_reset_clears_guest_reference_in_other_chats in tests/test_reset.py.
2026-04-26 17:40:46 -04:00
Joseph Doherty bfb2ffb6f6 chore: pin scene-close-on-cancel behavior + comment rationale (T74.3)
Phase 2 T44 review noted that scene close still runs when a primary
turn is cancelled mid-stream and asked the implementer to review.

Review finding: the existing behavior is correct, not a bug. The
close-detection branch in post_turn consumes ONLY the user's prose
(fully appended to the event_log BEFORE streaming starts) and the
current container name. It does NOT consume the bot's output. A user
who types "we're done here, fade out" and then hits Stop mid-stream
still meant to close — the cancelled bot beat doesn't invalidate
that intent.

- Document the rationale with an inline comment near the
  close-detection branch in chat/web/turns.py.
- Add regression test
  test_cancelled_turn_still_closes_scene_when_user_prose_signals_close
  that drives a stream raising CancelledError on first iteration and
  asserts the scene_closed event still lands.
2026-04-26 17:40:12 -04:00
Joseph Doherty bd13b64959 chore: remove defensive stale-guest degrade in regenerate.py (T73.3)
Phase 2 T44 added a defensive degrade-to-1:1 here when
`chat.guest_bot_id` pointed at a deleted bot. T47 fixed the root cause:
`bot_reset` cascade-clears the column when the referenced bot is purged
(verified by tests/test_reset.py), so the guard was dead code.

No corresponding stale-guest test existed in tests/test_regenerate.py
to remove. The bot_reset cascade test in tests/test_reset.py already
covers the root-cause behavior.
2026-04-26 17:40:07 -04:00
Joseph Doherty f2a57005e5 feat: regenerate covers interjection turns (T73.2)
Phase 2 T44 deferred interjection regenerate — when the original turn
group included a follow-on interjection beat we left it untouched. Now
regenerate redoes BOTH halves:

- Detect a sibling interjection by looking up assistant_turn events
  pinned to the same user_turn_id with `interjection_of` set.
- After streaming the new primary, run `detect_interjection` against
  the new primary text.
- If True: stream a new interjection from the silent witness, append
  with `interjection_of=<new primary speaker_id>`, supersede the
  original interjection, and re-run memory + state-update for the new
  beat.
- If False: supersede the original interjection without a replacement
  (back-pointer goes to the new primary so the row stays consistently
  hidden).

Also broadcast a `turn_html_replace` event for the new interjection so
the front-end can swap the prior interjection node in place (mirrors
T73.1's primary swap).

Tests:
- `test_regenerate_with_interjection_redoes_both_turns`: classifier
  returns True; assert two new assistant_turns land for the same
  user_turn, second carries `interjection_of`, originals superseded.
- `test_regenerate_drops_interjection_when_classifier_returns_false`:
  classifier returns False; assert one new assistant_turn (primary
  only) and the original interjection is superseded with no
  replacement.

`interjection_of` carries the primary's *speaker_id* (matching the
existing convention in chat/web/turns.py) rather than the event_id.
2026-04-26 17:39:31 -04:00
Joseph Doherty 88fae33152 fix: enqueue significance for interjection memories (T74.2)
T44's interjection branch wrote interjection memories via
record_turn_memory_for_present but never enqueued a SignificanceJob,
so the interjection beat could land in memory but never be scored —
which meant it could never auto-pin even when it carried a pivotal
moment.

- Capture the host-POV memory id from the interjection's memory write
  result and enqueue a SignificanceJob mirroring the primary turn's
  pattern. One enqueue per beat (host id; guest POV piggybacks on the
  same score since the prose is identical for v2 — per-POV rewrite
  happens at scene close in T45).
- New test test_interjection_enqueues_significance_job pins the
  contract by intercepting worker.enqueue and asserting two distinct
  jobs land per 3-entity turn that fires an interjection.
2026-04-26 17:38:30 -04:00
Joseph Doherty c874883a84 feat: classifier-based addressee detection (T74.1)
Replace the substring _detect_addressee_id helper with a classifier
call for the multi-entity case. The substring helper is kept as a
fast-path for the no-guest case (no LLM round-trip needed when only
one bot is present, preserves throughput).

- New service chat/services/addressee.py wrapping the existing
  classifier wrapper. AddresseeDecision carries addressee_id +
  confidence + reason; classifier failure falls back to the host with
  reason="fallback" (graceful-degradation, matches the relationship_seed
  / interjection pattern).
- chat/web/turns.py post_turn now calls detect_addressee in the
  multi-entity branch; 1:1 keeps the substring path.
- tests/test_addressee.py: 3 new tests (guest pick, host pick,
  classifier-failure fallback).
- tests/test_turn_flow.py: existing multi-entity tests now feed a
  canned addressee response in the queue. The addressee-routing test
  is updated to assert classifier-driven routing rather than substring.
2026-04-26 17:37:26 -04:00
Joseph Doherty 6f22e86f54 feat: regenerate broadcasts turn_html over SSE (T73.1)
After the new assistant_turn lands, publish a `turn_html_replace` SSE
event carrying the rendered HTML, the new turn_id, and the original
assistant_turn id as `supersedes_id` so connected tabs can swap the
prior DOM node in-place. Phase 1 T29 deferred this — page had to refresh
to see the regenerated turn.

Uses a new event name (not the existing `turn_html`) because the HTMX
`sse-swap="turn_html"` consumer expects raw HTML and an *append*
semantic; regenerate is a *replace*. The new event ships as JSON
(supersedes_id forces sse.py's JSON branch) so the front-end JS can
read the swap target from the payload.

Test: `test_regenerate_broadcasts_turn_html_over_sse` patches the
`publish` reference inside the regenerate module and asserts the
event shape.
2026-04-26 17:36:16 -04:00
Joseph Doherty e632a6247d merge: T72 drawer polish (deferred edits + first-meeting gate + witness flag editing) 2026-04-26 17:32:02 -04:00
Joseph Doherty 607d0971c4 feat: drawer witness flag inline-edit (T72.3)
Memories grow per-flag witness checkboxes (you / host / guest) that
auto-submit on change via HTMX. The new POST route emits a manual_edit
event with target_kind=memory_witness and a {flag, value} payload;
prior_value mirrors the same shape so an inverse edit restores the
flag. The drawer's recent-memories query now selects the three
witness columns alongside the existing fields so the template can
render checkbox state without a second query per row.
2026-04-26 17:28:25 -04:00
Joseph Doherty c265e4ce0f feat: first-meeting gate on drawer Add-guest form (T72.2)
When a host->candidate edge already exists from a prior chat, the
Add-guest form renders the prose textarea disabled with an "already
know each other" note. Submission without the explicit "re-seed
anyway" toggle skips seed_inter_bot_edges so existing edge content
(affinity, trust, knowledge, summaries) survives — guest_added and
group_node_initialized still fire. A small inline script enables /
disables the textarea per-option based on a pre-computed
existing_guest_edges dict surfaced by the GET handler.
2026-04-26 17:26:31 -04:00
Joseph Doherty 21404a373b feat: drawer edits for edge_trust / edge_summary / memory_pov_summary / knowledge_facts (T72.1)
Adds the four POST routes whose state-layer support was already
dispatched by the manual_edit projector (edge_trust, edge_summary,
memory_pov_summary) plus a new edge_knowledge_fact dispatch branch for
add/remove fact list manipulation. Drawer template gains editable
textareas, sliders, and add/remove fact controls. Remove semantics on
knowledge_fact match by string (not index) so concurrent edge_update
events appending facts between drawer renders don't desync the form.
2026-04-26 17:24:24 -04:00
Joseph Doherty 789b9bd042 merge: T71 prompt.py polish (witness role + ACTIVITIES + NICE trim docs) 2026-04-26 17:18:02 -04:00
Joseph Doherty 73bb8c1f17 chore: document NICE trim order rationale (T71.3)
T18 review (Phase 1) noted the NICE-tier trim drops previous-scene
FIRST while §6.3 spec lists previous-scene LAST in the NICE tier
group. Decision: keep the existing greedy order (previous-scene
first), and document why.

Rationale (now in code at the trim ladder):
  1. Cheapest-impact-first — a per-POV previous-scene summary loses
     less narrative continuity than the older dialogue turns or
     memory hits it competes with.
  2. Greedy lookahead is more expensive than the marginal narrative
     loss. Dropping previous-scene typically clears the soft-budget
     slack in one step.

Test added: test_nice_trim_order_documented pins the observed order
(previous-scene -> memories -> dialogue) so a future refactor can't
silently invert it. Sized so that all-NICE config overflows soft but
dropping just previous-scene fits — proves memories and older
dialogue turns survive while previous-scene is the FIRST drop.
2026-04-26 17:16:02 -04:00
Joseph Doherty afd1a50958 refactor: single ACTIVITIES: block with bullet-level trim (T71.2)
Phase 2 T43 added a SECOND ACTIVITIES: block to render guest activity
separately from you+speaker. Two consecutive ACTIVITIES: headers can
read as a duplicate-section bug to the LLM and bloat the prompt.

Consolidate to a single ACTIVITIES: block whose body is composed from
up to three bullets (you, speaker, guest). The block itself is
MUST-tier (always renders); bullet-level trim drops bullets in the
order guest -> group node -> you -> other edges, with the speaker
bullet as the MUST-tier floor (the speaker's own current activity is
the load-bearing slice).

Implementation chose Option B from the polish plan: pre-truncate the
bullets list at trim time before _build_activity_block runs, rather
than introduce a granular tier mode in the trim machinery. Rationale
documented in code; the existing block-level trim ladder gains a
single new toggle (include_you_activity) and the SHOULD-tier
guest_activity_block is gone.

Tests:
- test_single_activities_block_with_three_bullets_when_3_entities:
  exactly one ACTIVITIES: header with all three entity bullets.
- test_tight_budget_drops_guest_activity_bullet_first: speaker bullet
  survives, guest bullet absent under tight budget.
- Existing test_assemble_with_tight_budget_drops_guest_activity_first
  still passes (asserts on bullet absence, not block-header absence).
2026-04-26 17:13:24 -04:00
Joseph Doherty 428438b223 fix: witness role parametric in prompt assembly (T71.1)
Phase 2 T46 pinned the witness mask contract on search_memories with a
witness_role parameter (host/guest/you). The prompt-assembly call site
in assemble_narrative_prompt was hardcoded to "host", which silently
returned the wrong rows when the speaker was the guest bot.

Derive the witness role from chat membership via a new private helper
_witness_role_for(speaker_bot_id, host_bot_id), and apply it at the
search_memories call. Behaviour is identical when the speaker is the
host (or when no guest is present); the fix is load-bearing only when
the guest bot is the speaker — exactly the scenario Phase 2 T43 added
support for.

Tests: pin both directions (host-as-speaker and guest-as-speaker) by
patching the imported search_memories reference and asserting the
witness_role argument the call site emits.
2026-04-26 17:11:20 -04:00
Joseph Doherty b13f3b4e47 merge: T70 LLM-merged group meta-summary 2026-04-26 17:09:16 -04:00
Joseph Doherty f701f9d7dd merge: T69 bot_reset purges orphaned 'you' activity rows 2026-04-26 17:09:16 -04:00
Joseph Doherty 1b9144442a merge: T68 open_db with check_same_thread parameter 2026-04-26 17:09:16 -04:00
Joseph Doherty 13c23fd898 feat: LLM-merged group meta-summary (T70) 2026-04-26 17:07:12 -04:00
Joseph Doherty c1e419e012 fix: bot_reset purges orphaned 'you' activity rows (T69) 2026-04-26 17:06:21 -04:00
Joseph Doherty 994728b5ed refactor: open_db with check_same_thread parameter (T68) 2026-04-26 17:05:29 -04:00
Joseph Doherty e05f28e9d5 docs: add Phase 2.5 cleanup plan (Phase 1.5 + 2.5/3 backlog)
8 tasks across 5 waves consolidating the 15-item backlog tracked in
CLAUDE.md (5 from Phase 1.5 cleanup + 10 from Phase 2.5/3). Items are
grouped by file ownership so each wave stays file-disjoint:

- Wave 1 (parallel): open_db refactor, bot_reset orphan cleanup,
  LLM-merged group meta-summary
- Wave 2 (single): prompt.py polish — witness role parametric, single
  ACTIVITIES block, NICE trim documented
- Wave 3 (single): drawer polish — deferred v1 edits, first-meeting
  gate, witness flag editing
- Wave 4 (parallel): regenerate.py polish (SSE + interjection
  regenerate + stale-guest cleanup); turn-flow polish + new addressee
  service (classifier addressee + significance for interjection +
  scene-close-on-cancel pinned + stale-guest cleanup)
- Wave 5 (single): docs sweep

No schema migrations. Bundled tasks split into per-item sub-commits
for clean review bisection. Uses task ids T68-T75 to avoid collision
with Phase 3 plan (T49-T67) regardless of merge order.
2026-04-26 17:02:46 -04:00
Joseph Doherty 379054755a docs: add Phase 3 implementation plan with parallel-safe waves
19 tasks across 8 waves covering events with lifecycles, time skips
(elision + jump), active threads, significance/retrieval refinements,
and meanwhile scenes (host+guest with no 'you'). Mirrors the Phase 2
plan structure: pre-flight, parallel-execution strategy with worktree
isolation, file-disjointness analysis per wave, and per-task TDD spec
with commit messages.

Phase 3 schema: adds 0009_events.sql, 0010_threads.sql,
0011_meanwhile_scenes.sql (final version 11). Builds on Phase 2's
3-entity scene support and event-sourced architecture.
2026-04-26 16:55:50 -04:00
Joseph Doherty bb87fcbd4a merge: T48 Phase 2 documentation update 2026-04-26 16:28:46 -04:00
Joseph Doherty f6b75b25eb merge: T47 bot_reset cascades to guest references 2026-04-26 16:28:46 -04:00
Joseph Doherty 9f35669936 merge: T46 witness filter coverage for multi-entity scenarios 2026-04-26 16:28:46 -04:00
Joseph Doherty 321810fa54 docs: phase 2 status, behavioral defaults, deferred items 2026-04-26 16:28:14 -04:00
Joseph Doherty fb17ba0657 fix: bot_reset cascades to guest references in other chats 2026-04-26 16:25:37 -04:00
Joseph Doherty d40313063c test: witness filter coverage for multi-entity scenarios 2026-04-26 16:25:03 -04:00
Joseph Doherty 60ac33a787 merge: T44 multi-entity turn flow with interjection support 2026-04-26 16:22:11 -04:00
Joseph Doherty c86b0df411 feat: T44 multi-entity turn flow with interjection support
Rewrites post_turn for the multi-entity world:

- Addressee detection via case-insensitive whole-word match against the
  guest name; defaults to host on no-match or both-match.
- Multi-entity prompt assembly: forwards guest_id so the prompt sees
  the third party's activity / edges / group-node.
- Multi-witness memory write: record_turn_memory_for_present writes one
  memory per present bot witness when a guest is in the room.
- Multi-pair state-update: compute_state_updates_for_present emits one
  edge_update per directed pair (6 with a guest, 2 without).
- Interjection branch (T39): when a guest is present and the primary
  beat completes, the silent witness may follow on. detect_interjection
  decides; on True we stream a second narrative as the witness, append a
  second assistant_turn linked to the same user_turn_id, and re-run the
  multi-pair state update + memory write for the follow-on beat. Cancel
  collapses both halves; a cancelled interjection skips its downstream
  passes so we don't classifier-spam against a half-formed beat.
- Scene-close runs after both beats so apply_scene_close_summary sees
  the full closing scene; T45's guest-aware summarizer handles per-POV
  rewrites for each present witness.

regenerate.py mirrors the prompt / memory / state-update changes for
1:1 and multi-entity scenes. Per the Phase 2 spec, interjection
regeneration is deferred to Phase 2.5 — regenerate only re-streams the
addressee turn for v2.

Tests: adds 5 cases to tests/test_turn_flow.py covering the no-guest
regression, multi-bot without interjection, multi-bot with interjection,
scene-close per-POV rewrites, and addressee routing on a named-bot
prose. Each test pins its own canned MockLLMClient queue with the call
shape documented in the docstring.
2026-04-26 16:18:38 -04:00
Joseph Doherty 44c8735b27 merge: T45 per-POV summaries on close for each present witness 2026-04-26 16:08:54 -04:00
Joseph Doherty 9b601650fb merge: T43 multi-entity prompt assembly 2026-04-26 16:08:54 -04:00
Joseph Doherty fcb111310a feat: multi-entity prompt assembly with guest activity, edges, group node 2026-04-26 16:07:15 -04:00
Joseph Doherty 4e240347b4 feat: per-POV summaries on close for each present witness 2026-04-26 16:06:05 -04:00
Joseph Doherty a90647dddb merge: T42 drawer guest add/remove + render 2026-04-26 16:01:17 -04:00
Joseph Doherty bb83d97088 feat: drawer guest add/remove + render 2026-04-26 15:59:48 -04:00
Joseph Doherty f24ffb8e4f merge: T41 multi-witness memory write helper 2026-04-26 15:54:25 -04:00
Joseph Doherty 9d80b9ae2b merge: T40 multi-entity state-update coordinator 2026-04-26 15:54:25 -04:00
Joseph Doherty 77b42f1ea5 merge: T39 interjection classifier service 2026-04-26 15:54:25 -04:00
Joseph Doherty e7793f2441 feat: multi-witness memory write helper 2026-04-26 15:52:48 -04:00
Joseph Doherty 4ec56dd475 feat: multi-entity state-update coordinator 2026-04-26 15:51:58 -04:00
Joseph Doherty 6a92253ae7 feat: interjection classifier service 2026-04-26 15:51:29 -04:00
Joseph Doherty 22db9f3554 test: bump schema_version assertion to 8 after 0008_group_node migration 2026-04-26 15:49:25 -04:00
Joseph Doherty 6b726b2a4a merge: T38 relationship-seed service 2026-04-26 15:49:03 -04:00
Joseph Doherty e58cdbd527 merge: T37 guest_added/guest_removed event handlers 2026-04-26 15:49:03 -04:00
Joseph Doherty b69a74e69b merge: T36 group_node schema + projector handlers 2026-04-26 15:49:03 -04:00
Joseph Doherty c6b3531c64 feat: relationship-seed service for first-co-appearance prompt 2026-04-26 15:47:12 -04:00
Joseph Doherty a0d7debce5 feat: group_node schema + projector handlers 2026-04-26 15:46:16 -04:00
Joseph Doherty a1b4e251c5 feat: guest_added / guest_removed event handlers 2026-04-26 15:46:09 -04:00
71 changed files with 17447 additions and 352 deletions
+126 -6
View File
@@ -50,6 +50,10 @@ The 3-entity cap is load-bearing: it makes the relationship graph fully enumerab
- **Snapshots**: periodic every 100 events / 30 min; pre-rewind always. 5 periodic retained; pre-rewind retained 14 days. - **Snapshots**: periodic every 100 events / 30 min; pre-rewind always. 5 periodic retained; pre-rewind retained 14 days.
- **Streaming**: Stop button on streaming row; mid-stream disconnect commits partial with `truncated: true`; Send disabled mid-stream; multi-tab streaming via per-chat SSE channel. - **Streaming**: Stop button on streaming row; mid-stream disconnect commits partial with `truncated: true`; Send disabled mid-stream; multi-tab streaming via per-chat SSE channel.
- **Display**: lightweight markdown; `*action*` italic; OOC `((parens))` shown dimmed/italic, never sent to bot. - **Display**: lightweight markdown; `*action*` italic; OOC `((parens))` shown dimmed/italic, never sent to bot.
- **Multi-entity defaults (Phase 2)**: when `chat.guest_bot_id is None`, behavior matches Phase 1 single-bot 1:1. With a guest, all 3 entities are present in the prompt, witness writes, and state-update fan-out (6 directed pairs).
- **Addressee detection**: simple substring match (whole-word, case-insensitive) over the user turn's body. If both bot names match or neither does, the host gets the floor.
- **Interjection**: classifier-driven, conservative bias (default false on classifier failure / refusal / parse error). When the classifier returns true, the addressee speaks first, then the non-addressee may interject in a follow-up turn.
- **Per-POV summaries (multi-entity)**: each present witness with a memory store gets their own per-POV summary on scene close. The summary differs per bot based on persona + their edge to "you". The group node summary is updated alongside.
## Core concepts (vocabulary) ## Core concepts (vocabulary)
@@ -170,10 +174,126 @@ Deferred to Phase 2: second bot, group node, scene configurations, witness filte
### Phase 1.5 cleanup backlog ### Phase 1.5 cleanup backlog
Small follow-ups identified during Phase 1 reviews. Pick up at any time; none are blocking. All items shipped — see Phase 2.5 status below.
- **`open_db` refactor.** `chat/web/bots.py:get_conn()` duplicates the context-manager body to add `check_same_thread=False`. Extend `open_db(path, *, check_same_thread=True)` and have `get_conn` call it directly — eliminates the duplicated PRAGMA setup and ensures any future PRAGMA tweak only happens in one place. ## Phase 2 status
- **Regenerate broadcasts `turn_html` over SSE.** Currently a refresh is needed (see T29 limitation above). Mirror the broadcast logic from `chat/web/turns.py:post_turn` after the new `assistant_turn` lands.
- **`bot_reset` purges orphaned "you" activity rows** (see limitation above). Either delete `activity` rows by chat-membership or accept the noise indefinitely; the projection-layer fix is one extra `DELETE FROM activity WHERE entity_id='you' AND container_id IN (SELECT id FROM containers WHERE chat_id IN (...))` clause inside `_apply_bot_reset`. Phase 2 shipped end-to-end across **13 tasks** (T36T48 wave). The multi-entity surface is functional: chats can host a guest bot, the prompt assembly is guest-aware, post-turn fans out across all directed pairs, and scene close writes a per-POV summary per present witness plus a group_node summary.
- **Drawer edits for the deferred v1 fields**: edge_trust slider, edge_summary textarea, memory pov_summary textarea, knowledge_facts add/remove. The `manual_edit` projector already supports `edge_trust` / `edge_summary` / `memory_pov_summary` target_kinds — only the routes are missing. Knowledge_facts needs a new dispatch branch.
- **NICE trim order in prompt assembly** drops previous-scene first instead of last (T18 review). Greedy-cuts heuristic vs spec listing order; revisit if v1 play surfaces a real regression. - **Multi-entity scene support**: chats can now have a guest bot (you + host + guest). The 3-entity cap holds. New event kinds: `guest_added`, `guest_removed`, `group_node_initialized`, `group_node_updated`. New table: `group_node` (members, summary, dynamic, threads).
- **Drawer guest UX**: add/remove guest from the drawer side panel. The "have they met?" prose seed is parsed by the `relationship_seed` classifier into inter-bot directed edges (host↔guest).
- **Multi-entity turn flow**: `post_turn` assembles narrative with the guest-aware prompt; writes memories for **all** present bot witnesses; runs state updates for **all** directed pairs (6 with 3 entities); detects interjections via classifier (default false; the addressee gets the floor first).
- **Per-POV scene close summaries**: each present witness with a memory store gets their own per-POV summary on close; `group_node` summary updated alongside.
- **Bot reset cascade**: resetting a bot now also clears `chats.guest_bot_id` references in other chats (root-cause fix for stale-guest references after T47).
### Phase 2.5 / 3 backlog
All items shipped — see Phase 2.5 status below.
## Phase 2.5 status
Phase 2.5 cleanup shipped end-to-end across 8 tasks (T68T75). Two CLAUDE.md backlogs (Phase 1.5 cleanup, Phase 2.5/3) are now empty; deferred follow-ups discovered during execution are tracked in a new "Phase 2.6 / 3 backlog" section below.
- **`open_db` with check_same_thread parameter (T68)**: refactored `chat/db/connection.py` so `chat/web/bots.py:get_conn` no longer duplicates the PRAGMA setup. Default behavior preserved.
- **`bot_reset` cross-chat cleanup (T69)**: now purges orphaned "you" activity rows. Note: this also fixed a latent FK constraint crash that was lurking in the projector — `activity.container_id` is FK-referenced and the prior code would have crashed on any reset of a bot whose chat had a non-NULL `container_id` "you" activity row. The bug was masked because no prior test seeded such a row.
- **LLM-merged group meta-summary (T70)**: replaces Phase 2 T45's naive concat with a classifier merge call. Falls back to the naive concat on classifier failure.
- **`prompt.py` polish (T71)**: witness role parametric (`host` vs `guest` derived from chat membership); single `ACTIVITIES:` block with bullet-level trim; NICE trim order kept with documented rationale (greedy cheapest-impact-first beats spec-listing order in practice).
- **Drawer polish (T72)**: deferred v1 edits (edge_trust slider, edge_summary textarea, memory pov_summary textarea, knowledge_facts add/remove) + first-meeting gate (Add-guest form disables prose textarea when host→guest edge already exists; "re-seed anyway" toggle re-enables) + witness flag inline-edit (per-memory checkboxes for [you, host, guest] flags). Two new `manual_edit` projector branches: `edge_knowledge_fact` and `memory_witness`.
- **Regenerate polish (T73)**: regenerate now broadcasts `turn_html_replace` over SSE (NEW event distinct from `turn_html` to avoid breaking the existing append-semantic consumer); regenerate covers interjection turns (re-detects + re-streams or supersedes); defensive stale-guest degrade removed.
- **Turn-flow polish + addressee service (T74)**: classifier-based addressee detection (substring helper kept as no-guest fast path); SignificanceJob enqueued for interjection memories; scene-close-on-cancel pinned with comment + regression test (close detection is genuinely user-prose-only); defensive stale-guest degrade removed.
### Phase 2.6 / 3 backlog
New follow-ups discovered during Phase 2.5 execution. None are blocking; pick up at any time.
- **Frontend handler for `turn_html_replace` SSE event (from T73.1 review)**: regenerate's backend broadcast lands, but no live tab swaps the regenerated turn until a JS handler is wired. The existing `turn_html` event uses HTMX `sse-swap` to append; `turn_html_replace` ships JSON with `supersedes_id` for replacement semantics. Phase 2.6 should wire the JS to swap the prior turn's DOM node in place.
- **Cancel/stop hook for in-flight regenerate streams (from T73 review)**: `post_turn` registers stream tasks in `_in_flight_tasks` so the user can stop them. Regenerate doesn't. A user clicking "Stop" mid-regenerate has no cancel hook today.
- **DRY: regenerate vs post_turn (from T73 review)**: recent-dialogue assembly and prior-edges block are duplicated between `chat/services/regenerate.py` and `chat/web/turns.py`. Extract to shared helpers analogous to `_gather_state_update_inputs`.
- **Sibling-discovery query optimization (from T73 review)**: `regenerate.py`'s sibling-assistant-turn lookup scans all non-superseded `assistant_turn` rows globally. Adding a `chat_id` predicate via JSON extraction (or a denormalized column) bounds the cost to per-chat scale.
- **`_witness_role_for` defensive coding (from T71 review)**: helper returns `"guest"` when `host_bot_id is None`, which is wrong for Phase-1 chats. Defensive: `return "host" if host_bot_id is None or speaker_bot_id == host_bot_id else "guest"`. Not exercised by current tests; harden as a precaution.
- **Confidence type tightening (from T74 review)**: `chat/services/addressee.py::AddresseeDecision.confidence` could be typed as `Literal["high","medium","low"]` for stricter validation. Currently `str` with a comment.
- **Scene-close-on-cancel UX revisit**: T74.3 pinned the existing behavior (close fires even on cancel). If real play-testing surfaces a regression, revisit.
## Phase 3 status
Phase 3 shipped end-to-end across 19 tasks (T49T67). Events with full lifecycle, time skips, active threads, significance refinements, and meanwhile scenes are functional. Schema baseline is now version 11 (migrations 0009 events, 0010 threads, 0011 meanwhile_scenes). Test count grew from ~247 (Phase 2) to ~315 (+68 new tests across the wave).
- **Wave 1 — schema + lifecycle handlers (parallel)**:
- **T49** `events` table + lifecycle handlers (`event_planned`, `event_started`, `event_completed`, `event_cancelled`, `event_expired`).
- **T50** `time_skip` event handlers (elision and jump variants).
- **T51** `threads` table + handlers (`thread_opened`, `thread_updated`, `thread_closed`).
- **Wave 2 — detection / narration services (parallel)**:
- **T52** event-lifecycle detection service (planned→active→completed transitions inferred from narration).
- **T53** skip narration service (elision + jump prose).
- **T54** synthesized-memories service for jump skips (LLM-summarized intervening time).
- **T55** thread-detection service (open/update/close inferred from recent dialogue).
- **Wave 3 — promotion + ranking (parallel)**:
- **T56** event-completion promotion service (objects → inventory, knowledge → edge knowledge, relationship deltas → edge summary; everything else stays in the closed event).
- **T57** significance-aware retrieval ranking — SQL-side `SIGNIFICANCE_RANK_BIAS` plus the existing Python composite re-rank.
- **T58** scene compression keeps key quotes when significance ≥ 2; thread emission piggybacks on scene close.
- **Wave 4 — drawer UX (single)**:
- **T59** drawer additions: events panel, threads panel, skip controls.
- **Wave 5a — prompt + turn flow integration (parallel)**:
- **T60** prompt assembly includes active events + open threads in the speaker's prompt.
- **T61** turn flow invokes event-detection + completion promotion alongside existing post-turn fan-out.
- **Wave 5b — natural-language skip surface (single)**:
- **T62** classifier-driven skip command at the user-input layer; shared skip controllers extracted into `chat/web/skip.py`.
- **Wave 6a — meanwhile schema (single)**:
- **T63** meanwhile-scene schema + state (scene config 4: host+guest, no "you").
- **Wave 6b — meanwhile turn flow (parallel)**:
- **T64** meanwhile turn flow (host+guest, no "you" in the prompt or witness writes).
- **T65** meanwhile summary digest surfaces to the next "you"-present scene.
- **Wave 7 — integration + docs (parallel)**:
- **T66** cross-feature integration tests covering events × skips × threads × meanwhile.
- **T67** documentation (this section).
### Phase 3.5 / 4 backlog
New follow-ups discovered during Phase 3 reviews and execution. None are blocking; pick up at any time.
#### From T53 review
- **`narrate_skip` `timeout_s` not piped through to `client.generate`**: parameter accepted but ignored. Fix: pass `timeout_s=timeout_s` to `client.generate(**...)`, or drop the parameter entirely if Featherless's client doesn't honor it.
#### From T57 review
- **`search_memories` docstring should mention SQL-side significance bias**: the function docstring still describes only the Python composite re-rank; add a one-line note about `SIGNIFICANCE_RANK_BIAS`.
#### From T58 review
- **Scene close re-close suffix bloat risk**: `_build_key_quotes_suffix` reads from `memories.pov_summary`. If a scene close runs twice, the second pass would read the rewritten text plus the previous "Key quotes:" suffix and append a second one. Either guard for double-suffix or source quotes from `event_log` `assistant_turn`/`user_turn` text instead.
- **Thread detection transcript scoping**: `_read_recent_dialogue` returns chat-wide history with no `scene_id` filter (Phase 1 turns lack one). Feeding chat-wide history to `detect_threads` will misattribute threads to the closing scene when the scene boundary falls inside the last 50 turns. Scope by `scene_id` once turns carry it, or by `started_at` against scene-open timestamp.
- **Swallowed exceptions in `detect_threads` try/except**: bare `Exception` swallows programmer errors silently. Log at debug level so silent regressions are recoverable.
- **Scene close `closed_at` clock divergence**: T58 uses `datetime.now(timezone.utc).isoformat()` instead of chat-clock time. Diverges from chat-clock semantics elsewhere; revisit if event reconstructions need chat-clock ordering.
- **Test coverage gaps in T58**: no test for 200-char quote truncation; no test for `thread_updated`/`thread_closed` candidate paths; no test for the `try/except` fallback.
#### From T61 review
- **Regenerate doesn't roll back lifecycle transitions from superseded turn**: `event_started`/`event_completed` rows from a superseded turn remain. Phase 3.5 should add a lifecycle-undo step. Caveat: regenerate-after-completion may double-emit promotion artifacts if the new text re-completes the same event.
- **Asymmetry in event-detection ordering**: post_turn runs lifecycle BETWEEN interjection and scene-close; regenerate runs lifecycle at the END. Benign because regenerate has no scene-close path, but worth tidying.
#### From T62 review
- **Error-message prefix sniff for 404 vs 400 routing**: drawer skip routes use `str(exc).startswith("chat not found")` to distinguish 404 from 400. Fragile if error wording changes. Use a typed exception subclass.
- **Skip command bypasses scene close detection**: a user typing "fade out, skip an hour" would skip without closing the scene. Acceptable for Phase 3 but worth noting.
#### From T63 review
- **`participants_json` JSON injection** (FIXED in T63 but worth noting in backlog as a "double-check other JSON-string-build sites" task): T63 originally used f-string interpolation; fixed to use `json.dumps`. Audit other state modules for similar patterns.
#### From T64 review
- **`record_meanwhile_memory` and `record_turn_memory_for_present` share private `_write_one_memory` helper**: minor DRY note; both helpers are similar enough that a unified API with a `you_present: bool` kwarg might be cleaner long-term.
- **Stop button cancellation for meanwhile turns**: T64 fix-up registered tasks in `_in_flight_tasks`; verify the `/turns/cancel` endpoint actually cancels meanwhile streams (the test pins registration but not the cancel-from-route path).
#### From cross-feature interactions discovered in Wave 6b merge
- **Cross-feature canned-queue brittleness**: meanwhile-scene close test required a canned response for T65's digest call after T64+T65 merge. Future close-path additions will keep extending the queue; consider a structured fixture builder rather than positional canned arrays.
#### From T66 integration tests
- **`consume_pending_meanwhile_digests` is defined but NOT wired into `post_turn`**: the helper lives in `chat/services/prompt.py` (T65) but `chat/web/turns.py` never calls it. Meanwhile digests stay pending forever in production. Phase 3.5 should call the helper after the first you-turn following a meanwhile close — probably right after the assistant_turn lands but before the next prompt assembly. Pinned by `tests/test_phase3_integration.py::test_meanwhile_close_digest_surfaces_then_consumed` which currently calls the helper directly.
#### Discovered during Phase 3 execution
- **`_witness_role_for` defensive `host_bot_id is None`** (carry-over from Phase 2.5 T71 backlog) — still pending.
+2 -2
View File
@@ -5,9 +5,9 @@ from pathlib import Path
@contextmanager @contextmanager
def open_db(path: Path): def open_db(path: Path, *, check_same_thread: bool = True):
path.parent.mkdir(parents=True, exist_ok=True) path.parent.mkdir(parents=True, exist_ok=True)
conn = sqlite3.connect(path) conn = sqlite3.connect(path, check_same_thread=check_same_thread)
conn.execute("PRAGMA journal_mode=WAL") conn.execute("PRAGMA journal_mode=WAL")
conn.execute("PRAGMA foreign_keys=ON") conn.execute("PRAGMA foreign_keys=ON")
try: try:
+8
View File
@@ -0,0 +1,8 @@
CREATE TABLE group_node (
chat_id TEXT PRIMARY KEY,
members_json TEXT NOT NULL,
summary TEXT NOT NULL DEFAULT '',
dynamic TEXT NOT NULL DEFAULT '',
threads_json TEXT NOT NULL DEFAULT '[]',
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
+14
View File
@@ -0,0 +1,14 @@
CREATE TABLE events (
id INTEGER PRIMARY KEY,
event_id TEXT NOT NULL UNIQUE,
chat_id TEXT NOT NULL,
kind TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'planned',
props_json TEXT NOT NULL DEFAULT '{}',
planned_for TEXT,
started_at TEXT,
completed_at TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX events_chat_idx ON events(chat_id, status);
+14
View File
@@ -0,0 +1,14 @@
CREATE TABLE threads (
id INTEGER PRIMARY KEY,
thread_id TEXT NOT NULL UNIQUE,
chat_id TEXT NOT NULL,
title TEXT NOT NULL,
summary TEXT NOT NULL DEFAULT '',
status TEXT NOT NULL DEFAULT 'open',
opened_at TEXT NOT NULL DEFAULT (datetime('now')),
closed_at TEXT,
last_referenced_scene_id INTEGER,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX threads_chat_status_idx ON threads(chat_id, status);
@@ -0,0 +1,27 @@
-- T63: Meanwhile scene support — extend scenes with a present-set discriminator
-- and a parent link (you-scene -> meanwhile child), plus a pending-digest queue.
--
-- Existing scenes table (0007) has columns:
-- id, chat_id, container_id, started_at, ended_at, significance,
-- participants_json
-- It has no `status` / `closed_at` columns. We treat `ended_at IS NULL` as
-- "active" and `ended_at IS NOT NULL` as "closed" — consistent with the
-- existing scene_opened/scene_closed handlers.
ALTER TABLE scenes ADD COLUMN present_set_kind TEXT NOT NULL DEFAULT 'you_host';
ALTER TABLE scenes ADD COLUMN parent_scene_id INTEGER;
CREATE INDEX scenes_present_set_idx
ON scenes(chat_id, present_set_kind, ended_at);
CREATE TABLE meanwhile_digest_pending (
id INTEGER PRIMARY KEY,
scene_id INTEGER NOT NULL,
chat_id TEXT NOT NULL,
summary TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
consumed_at TEXT
);
CREATE INDEX meanwhile_digest_chat_idx
ON meanwhile_digest_pending(chat_id) WHERE consumed_at IS NULL;
+110
View File
@@ -0,0 +1,110 @@
"""Addressee classifier service (T74.1).
Phase 2 (T44) detected the addressee — host vs. guest — with a simple
case-insensitive whole-word substring match against the bots' names.
That worked for the obvious case ("BotB, what do you think?") but lost
the long tail: pronouns, paraphrases, indirect address, narrative
focus on a particular party. T74.1 swaps the substring helper for a
classifier call that reads the prose holistically.
The substring helper in :mod:`chat.web.turns` is kept as a fast-path
for the no-guest case (only one bot present means there is nothing to
classify) and as a non-breaking fallback for the regenerate path. The
multi-entity branch in :func:`chat.web.turns.post_turn` calls
:func:`detect_addressee` from this module.
Failure mode: classifier flake or low-confidence response degrades to
the host (the default speaker per Phase 2's host-keeps-the-floor
bias). The decision carries ``confidence`` and ``reason`` so callers
that want to log degraded decisions can distinguish a real "host" call
from a fallback.
"""
from __future__ import annotations
from typing import Literal
from pydantic import BaseModel
from chat.llm.classify import classify
from chat.llm.client import LLMClient
class AddresseeDecision(BaseModel):
"""Which present bot the user is addressing.
``addressee_id`` is the chosen bot's id. ``confidence`` is one of
``"high"`` / ``"medium"`` / ``"low"`` — callers may treat ``"low"``
as a soft fallback to the host. ``reason`` is a short free-form
string. The classifier-failure fallback uses ``reason="fallback"``
so it's distinguishable from a real low-confidence call.
"""
addressee_id: str
confidence: Literal["high", "medium", "low"] = "medium"
reason: str = ""
_SYSTEM = (
"Given a user's turn prose and the names of present bots, decide "
"which bot the user is addressing. If the user is speaking to no "
"specific bot (descriptive narration, action without dialogue), "
"default to the host. Output strict JSON matching the schema. "
"The addressee_id MUST be one of the ids supplied in the user "
"message — do not invent ids."
)
async def detect_addressee(
client: LLMClient,
*,
classifier_model: str,
user_prose: str,
host_id: str,
host_name: str,
guest_id: str | None,
guest_name: str | None,
timeout_s: float = 30.0,
) -> AddresseeDecision:
"""Classify which present bot the user is addressing.
Defaults to host on classifier failure or when the classifier picks
an id that isn't one of the supplied ids. The caller is expected to
only invoke this in the multi-entity case (a guest is present);
when no guest is present the substring fast-path in
:mod:`chat.web.turns` is used instead and this function is not
called.
"""
fallback = AddresseeDecision(
addressee_id=host_id, confidence="low", reason="fallback"
)
user = (
f"Host: {host_name} (id={host_id})\n"
+ (
f"Guest: {guest_name} (id={guest_id})\n"
if guest_id is not None
else ""
)
+ f"\nUser prose:\n{user_prose}"
)
decision = await classify(
client,
model=classifier_model,
system=_SYSTEM,
user=user,
schema=AddresseeDecision,
default=fallback,
timeout_s=timeout_s,
)
# Defensive: if the classifier returned an id outside the supplied
# set, treat it as a fallback to the host. This catches pathological
# outputs that pass schema validation but pick a phantom id.
valid_ids = {host_id}
if guest_id is not None:
valid_ids.add(guest_id)
if decision.addressee_id not in valid_ids:
return fallback
return decision
__all__ = ["AddresseeDecision", "detect_addressee"]
+72
View File
@@ -0,0 +1,72 @@
"""Event-lifecycle detection (T52).
After each turn, classify whether any active events transitioned
(started, completed, cancelled). Conservative bias — most turns
return empty. T61 turn flow appends one event_started/completed/
cancelled per transition via append_and_apply.
"""
from __future__ import annotations
from pydantic import BaseModel, Field
from chat.llm.classify import classify
from chat.llm.client import LLMClient
class EventTransition(BaseModel):
event_id: str
new_status: str # "active" | "completed" | "cancelled"
reason: str = ""
class EventLifecycleDecision(BaseModel):
transitions: list[EventTransition] = Field(default_factory=list)
_SYSTEM = (
"You decide whether any active events transitioned this turn. "
"STRONGLY default to empty transitions — most turns do NOT resolve "
"or start a known event. Output only transitions that the narrative "
"text clearly resolves or starts. Each transition MUST reference an "
"event_id from the active_events list. new_status is one of "
"'active' (planned -> active), 'completed', or 'cancelled'. "
"Output strict JSON matching the schema."
)
async def detect_event_transitions(
client: LLMClient,
*,
classifier_model: str,
narrative_text: str,
active_events: list[dict], # [{event_id, kind, status, props}, ...]
timeout_s: float = 30.0,
) -> EventLifecycleDecision:
"""Classify event transitions for the latest turn. Empty active_events
short-circuits without an LLM call."""
if not active_events:
return EventLifecycleDecision()
user_lines = ["Active events:"]
for ev in active_events:
user_lines.append(
f"- event_id={ev['event_id']} kind={ev['kind']} "
f"status={ev['status']} props={ev.get('props', {})}"
)
user_lines.append("")
user_lines.append("Latest narrative:")
user_lines.append(narrative_text.strip())
user = "\n".join(user_lines)
return await classify(
client,
model=classifier_model,
system=_SYSTEM,
user=user,
schema=EventLifecycleDecision,
default=EventLifecycleDecision(),
timeout_s=timeout_s,
)
__all__ = ["EventTransition", "EventLifecycleDecision", "detect_event_transitions"]
+149
View File
@@ -0,0 +1,149 @@
"""Event-completion promotion (T56).
When an event reaches ``status='completed'``, read its ``props_json``
and emit promotion events into the appropriate state stores.
Synchronous, no LLM. Skips when the event status is not ``completed``
(cancelled / expired terminate the event without promoting).
Props recognized:
- ``acquired_objects: list[str]`` — emits a ``manual_edit`` with
``target_kind="memory_pov_summary"`` per object on the host's memory
row, recording the acquisition. Phase 3 is a stub: it requires both
``host_bot_id`` and ``host_memory_id`` (an existing memories.id) to
be present in props; missing either skips that object cleanly.
Phase 4 will introduce a real inventory schema.
- ``knowledge_facts: list[{owner_id, target_id, fact}]`` — emits an
``edge_update`` event on the directed ``owner_id -> target_id`` edge
with the fact appended to ``knowledge_facts``. The ``edge_update``
projector accepts ``knowledge_facts`` as a list and extends the
edge's stored ``knowledge_json``.
- ``relationship_change: {summary, source_id, target_id}`` — emits a
``manual_edit`` with ``target_kind="edge_summary"`` overwriting the
edge's ``summary`` field on the directed pair.
Anything else stays in the closed event record (the projector kept
the row; no further promotion).
"""
from __future__ import annotations
from sqlite3 import Connection
from chat.eventlog.log import append_and_apply
from chat.state.events import get_event
def promote_completed_event(
conn: Connection,
*,
event_id: str,
chat_id: str,
chat_clock_at: str | None,
) -> dict:
"""Read the completed event's props and emit promotion events.
Returns a dict of counts keyed by promoted artifact:
``{"acquired_objects", "knowledge_facts", "relationship_change"}``.
Skips silently if the event row is missing or its status is not
``completed`` — cancelled / expired events terminate without any
promotion.
"""
counts = {
"acquired_objects": 0,
"knowledge_facts": 0,
"relationship_change": 0,
}
event = get_event(conn, event_id)
if event is None or event["status"] != "completed":
return counts
props = event.get("props") or {}
# acquired_objects: each becomes a memory_pov_summary edit (Phase 3
# stub). The manual_edit projector requires a valid memory rowid as
# ``target_id`` (it does ``int(target_id)``), so skip cleanly when
# neither a host_bot_id nor a host_memory_id is supplied.
host_bot_id = props.get("host_bot_id")
host_memory_id = props.get("host_memory_id")
for obj in props.get("acquired_objects", []) or []:
if host_bot_id is None or host_memory_id is None:
continue
append_and_apply(
conn,
kind="manual_edit",
payload={
"target_kind": "memory_pov_summary",
"target_id": host_memory_id,
"owner_id": host_bot_id,
"chat_id": chat_id,
"prior_value": "",
"new_value": f"Acquired: {obj}",
"source": "event_promotion",
"event_id": event_id,
"chat_clock_at": chat_clock_at,
},
)
counts["acquired_objects"] += 1
# knowledge_facts: each becomes an edge_update appending the fact.
for fact_entry in props.get("knowledge_facts", []) or []:
owner_id = fact_entry.get("owner_id")
target_id = fact_entry.get("target_id")
fact = fact_entry.get("fact", "")
if not owner_id or not target_id or not fact:
continue
append_and_apply(
conn,
kind="edge_update",
payload={
"source_id": owner_id,
"target_id": target_id,
"chat_id": chat_id,
"affinity_delta": 0,
"trust_delta": 0,
"knowledge_facts": [fact],
"last_interaction_at": chat_clock_at,
"last_interaction_chat_id": chat_id,
"source": "event_promotion",
"event_id": event_id,
},
)
counts["knowledge_facts"] += 1
# relationship_change: edge_summary manual_edit on the directed pair.
# The manual_edit projector for ``edge_summary`` keys on a
# ``target_id`` dict ``{source_id, target_id}`` (see
# chat/state/manual_edit.py); we shape the payload to match.
rc = props.get("relationship_change") or {}
if rc:
source_id = rc.get("source_id")
rc_target_id = rc.get("target_id")
summary = rc.get("summary", "")
if source_id and rc_target_id and summary:
append_and_apply(
conn,
kind="manual_edit",
payload={
"target_kind": "edge_summary",
"target_id": {
"source_id": source_id,
"target_id": rc_target_id,
},
"chat_id": chat_id,
"prior_value": "",
"new_value": summary,
"source": "event_promotion",
"event_id": event_id,
"chat_clock_at": chat_clock_at,
},
)
counts["relationship_change"] += 1
return counts
__all__ = ["promote_completed_event"]
+100
View File
@@ -0,0 +1,100 @@
"""Interjection classifier service (T39).
Per Requirements §6.2, when a guest is present and the addressee bot has
just spoken, the *non-addressee* bot may follow on with a brief
interjection beat. This service decides whether that interjection
fires. Conservative bias: most turns return ``should_interject=False``
— the addressee has the floor and an interjection is the exception.
Trigger ``True`` only when the silent witness's character, given their
persona and edges, would plausibly speak up: jealousy, surprise, strong
agreement worth voicing, correcting a factual falsehood, urgency.
T44 (turn flow) calls this and, on ``True``, generates the brief
follow-on response as the silent witness. Classifier failure falls back
to ``should_interject=False`` with ``reason="fallback"`` so the chat
keeps moving (§3.3 graceful-degradation rule); callers that care can
distinguish a real "no" from a degraded "no" by the reason string.
"""
from __future__ import annotations
from pydantic import BaseModel
from chat.llm.classify import classify
from chat.llm.client import LLMClient
class InterjectionDecision(BaseModel):
"""Whether the silent witness interjects, plus a short reason.
Defaults are a deliberate no-op: ``should_interject=False`` with an
empty reason. The classifier-failure fallback uses
``reason="fallback"`` so it's distinguishable from a real "no".
"""
should_interject: bool = False
reason: str = ""
_SYSTEM = (
"You decide whether a silent witness character interjects after the "
"addressee character finishes speaking. STRONGLY default to false — "
"the addressee has the floor and most turns should NOT have an "
"interjection. Only return true when the silent witness's character, "
"given their persona and edges, would plausibly speak up: jealousy, "
"surprise, strong agreement worth voicing, correcting a factual "
"falsehood, urgency. Output strict JSON matching the schema."
)
async def detect_interjection(
client: LLMClient,
*,
classifier_model: str,
addressee_name: str,
addressee_just_said: str,
silent_witness_name: str,
silent_witness_persona: str,
silent_witness_edge_to_addressee: dict, # {affinity, trust, summary}
silent_witness_edge_to_you: dict,
you_just_said: str,
timeout_s: float = 30.0,
) -> InterjectionDecision:
"""Decide whether the silent witness bot interjects after the addressee
finishes speaking.
The two ``silent_witness_edge_*`` dicts carry the silent witness's
directed edges toward the addressee and toward the user ("you"),
each shaped ``{affinity: int, trust: int, summary: str}``. Missing
keys fall back to a 50/50 baseline with an empty summary so this
function tolerates partially-populated edge state without raising.
"""
user = (
f"You said: {you_just_said}\n\n"
f"{addressee_name} just said: {addressee_just_said}\n\n"
f"Silent witness: {silent_witness_name}\n"
f"Persona: {silent_witness_persona}\n"
f"Edge {silent_witness_name} -> {addressee_name}: "
f"affinity={silent_witness_edge_to_addressee.get('affinity', 50)}, "
f"trust={silent_witness_edge_to_addressee.get('trust', 50)}, "
f"summary={silent_witness_edge_to_addressee.get('summary', '')}\n"
f"Edge {silent_witness_name} -> you: "
f"affinity={silent_witness_edge_to_you.get('affinity', 50)}, "
f"trust={silent_witness_edge_to_you.get('trust', 50)}, "
f"summary={silent_witness_edge_to_you.get('summary', '')}\n\n"
f"Should {silent_witness_name} interject?"
)
return await classify(
client,
model=classifier_model,
system=_SYSTEM,
user=user,
schema=InterjectionDecision,
default=InterjectionDecision(
should_interject=False, reason="fallback"
),
timeout_s=timeout_s,
)
__all__ = ["InterjectionDecision", "detect_interjection"]
+157
View File
@@ -76,3 +76,160 @@ def record_turn_memory(
).fetchone() ).fetchone()
memory_id = row[0] if row else None memory_id = row[0] if row else None
return event_id, memory_id return event_id, memory_id
def _write_one_memory(
conn: Connection,
*,
owner_id: str,
chat_id: str,
narrative_text: str,
witness_you: int,
witness_host: int,
witness_guest: int,
scene_id: int | None,
chat_clock_at: str | None,
source: str,
significance: int,
) -> tuple[int, int | None]:
"""Append a single ``memory_written`` event for ``owner_id`` and return
``(event_id, memory_id)`` for the projected row."""
payload: dict = {
"owner_id": owner_id,
"chat_id": chat_id,
"pov_summary": narrative_text,
"witness_you": witness_you,
"witness_host": witness_host,
"witness_guest": witness_guest,
"source": source,
"reliability": 1.0,
"significance": significance,
"pinned": 0,
"auto_pinned": 0,
}
if scene_id is not None:
payload["scene_id"] = scene_id
if chat_clock_at is not None:
payload["chat_clock_at"] = chat_clock_at
event_id = append_and_apply(conn, kind="memory_written", payload=payload)
row = conn.execute(
"SELECT id FROM memories "
"WHERE owner_id = ? AND chat_id = ? "
"ORDER BY id DESC LIMIT 1",
(owner_id, chat_id),
).fetchone()
memory_id = row[0] if row else None
return event_id, memory_id
def record_turn_memory_for_present(
conn: Connection,
*,
chat_id: str,
host_bot_id: str,
guest_bot_id: str | None,
narrative_text: str,
scene_id: int | None = None,
chat_clock_at: str | None = None,
source: str = "direct",
significance: int = 1,
) -> dict[str, tuple[int, int | None]]:
"""Write a ``memory_written`` event for each present bot witness.
Host is always written. Guest is written iff ``guest_bot_id is not
None``. Witness flags are ``[you=1, host=1, guest=1]`` when a guest
is present, ``[you=1, host=1, guest=0]`` otherwise.
Returns a mapping ``{bot_id: (event_id, memory_id)}`` so callers can
look up the freshly-projected memory id per owner without re-querying
the database.
"""
witness_guest = 1 if guest_bot_id is not None else 0
result: dict[str, tuple[int, int | None]] = {}
result[host_bot_id] = _write_one_memory(
conn,
owner_id=host_bot_id,
chat_id=chat_id,
narrative_text=narrative_text,
witness_you=1,
witness_host=1,
witness_guest=witness_guest,
scene_id=scene_id,
chat_clock_at=chat_clock_at,
source=source,
significance=significance,
)
if guest_bot_id is not None:
result[guest_bot_id] = _write_one_memory(
conn,
owner_id=guest_bot_id,
chat_id=chat_id,
narrative_text=narrative_text,
witness_you=1,
witness_host=1,
witness_guest=1,
scene_id=scene_id,
chat_clock_at=chat_clock_at,
source=source,
significance=significance,
)
return result
def record_meanwhile_memory(
conn: Connection,
*,
chat_id: str,
host_bot_id: str,
guest_bot_id: str,
narrative_text: str,
scene_id: int | None = None,
chat_clock_at: str | None = None,
source: str = "direct",
significance: int = 1,
) -> dict[str, tuple[int, int | None]]:
"""Write per-POV ``memory_written`` events for a meanwhile turn (T64).
A meanwhile scene runs entirely between host + guest, with "you"
absent. Both bots are present witnesses, so each one gets a row with
witness flags ``[you=0, host=1, guest=1]`` — different from the
normal-turn ``record_turn_memory_for_present`` shape, which assumes
the user is always a witness (``witness_you=1``).
The ``guest_bot_id`` is required (a meanwhile scene by definition
has both bots) — callers passing ``None`` is a programming error.
Returns ``{bot_id: (event_id, memory_id)}`` mirroring
:func:`record_turn_memory_for_present` so downstream queues
(significance scoring) can pull memory ids without re-querying.
"""
result: dict[str, tuple[int, int | None]] = {}
result[host_bot_id] = _write_one_memory(
conn,
owner_id=host_bot_id,
chat_id=chat_id,
narrative_text=narrative_text,
witness_you=0,
witness_host=1,
witness_guest=1,
scene_id=scene_id,
chat_clock_at=chat_clock_at,
source=source,
significance=significance,
)
result[guest_bot_id] = _write_one_memory(
conn,
owner_id=guest_bot_id,
chat_id=chat_id,
narrative_text=narrative_text,
witness_you=0,
witness_host=1,
witness_guest=1,
scene_id=scene_id,
chat_clock_at=chat_clock_at,
source=source,
significance=significance,
)
return result
+62
View File
@@ -0,0 +1,62 @@
"""Multi-entity state-update coordinator (T40).
Wraps single-pair compute_state_update to run state updates for ALL
directed pairs of present entities. With 3 present entities (you, host,
guest) that's 6 directed pairs. With 2 present (you, host) it's 2 pairs.
Calls run sequentially to respect Featherless's 2-connection cap (the
client-level semaphore would serialize them anyway, but doing it here
keeps the failure surface clean — a hung pair doesn't queue behind
itself).
"""
from __future__ import annotations
from chat.llm.client import LLMClient
from chat.services.state_update import StateUpdate, compute_state_update
async def compute_state_updates_for_present(
client: LLMClient,
*,
classifier_model: str,
present_ids: list[str],
present_names: dict[str, str],
personas: dict[str, str],
prior_edges: dict[tuple[str, str], dict],
recent_dialogue: list[dict],
timeout_s: float = 30.0,
) -> list[tuple[str, str, StateUpdate]]:
"""Run compute_state_update for every directed pair (src != tgt) over
``present_ids``. Returns list of ``(source_id, target_id, update)``
tuples in the natural iteration order over ``present_ids x present_ids``.
A single failing pair falls back to the schema-default StateUpdate
(zero deltas, empty facts) inside ``compute_state_update``; the batch
keeps going.
"""
out: list[tuple[str, str, StateUpdate]] = []
for src in present_ids:
for tgt in present_ids:
if src == tgt:
continue
edge = prior_edges.get((src, tgt), {})
update = await compute_state_update(
client,
model=classifier_model,
source_id=src,
target_id=tgt,
source_name=present_names.get(src, src),
source_persona=personas.get(src, "") or "",
target_name=present_names.get(tgt, tgt),
prior_affinity=int(edge.get("affinity", 50)),
prior_trust=int(edge.get("trust", 50)),
prior_summary=edge.get("summary", "") or "",
recent_dialogue=recent_dialogue,
timeout_s=timeout_s,
)
out.append((src, tgt, update))
return out
__all__ = ["compute_state_updates_for_present"]
+488 -46
View File
@@ -37,7 +37,11 @@ import tiktoken
from chat.llm.client import Message from chat.llm.client import Message
from chat.state.edges import get_edge, list_edges_for from chat.state.edges import get_edge, list_edges_for
from chat.state.entities import get_bot, get_you from chat.state.entities import get_bot, get_you
from chat.state.events import list_active_events
from chat.state.group_node import get_group_node
from chat.state.meanwhile import list_pending_meanwhile_digests
from chat.state.memory import search_memories from chat.state.memory import search_memories
from chat.state.threads import list_open_threads
from chat.state.world import ( from chat.state.world import (
active_scene, active_scene,
get_activity, get_activity,
@@ -206,6 +210,121 @@ def _build_previous_scene_block(pov_summary: str | None) -> str | None:
return "PREVIOUS SCENE SUMMARY:\n" + pov_summary return "PREVIOUS SCENE SUMMARY:\n" + pov_summary
def _build_group_node_block(group_node: dict | None) -> str | None:
"""Render the group-node summary + dynamic as a SHOULD-tier block.
Used only in 3-entity scenes (you + host + guest). Returns None when
the row is missing or both summary and dynamic are empty.
"""
if not group_node:
return None
summary = (group_node.get("summary") or "").strip()
dynamic = (group_node.get("dynamic") or "").strip()
if not summary and not dynamic:
return None
lines = ["Group dynamic:"]
if summary:
lines.append(f"- Summary: {summary}")
if dynamic:
lines.append(f"- Dynamic: {dynamic}")
return "\n".join(lines)
def _props_excerpt(props: dict | None, limit: int = 80) -> str:
"""Return a one-line excerpt of an event's ``props`` dict.
Renders ``key=value`` pairs separated by ", " (deterministic by dict
insertion order) and truncates to ~``limit`` characters with a
trailing ellipsis. Returns empty string for falsy/empty props so the
caller can omit the line entirely.
"""
if not props:
return ""
pieces: list[str] = []
for k, v in props.items():
pieces.append(f"{k}={v}")
rendered = ", ".join(pieces)
if len(rendered) > limit:
# Reserve 1 char for the ellipsis so the total never exceeds limit.
rendered = rendered[: max(0, limit - 1)] + ""
return rendered
def _build_active_events_block(events: list[dict]) -> str | None:
"""Render the ``Active events:`` block for Phase 3 Task 60.
One bullet per event. The sub-label depends on status:
- ``planned`` → ``(planned for {planned_for})``
- ``active`` → ``(active, started_at={started_at})``
A second indented line carries a one-line excerpt of the event's
``props`` (truncated ~80 chars) when non-empty. Returns ``None`` when
there are no active events so the caller can omit the entire block.
"""
if not events:
return None
lines = ["Active events:"]
for ev in events:
kind = ev.get("kind") or "?"
status = ev.get("status") or ""
if status == "active":
started = ev.get("started_at") or ""
lines.append(f"- {kind} (active, started_at={started})")
else:
planned = ev.get("planned_for") or ""
lines.append(f"- {kind} (planned for {planned})")
excerpt = _props_excerpt(ev.get("props"))
if excerpt:
lines.append(f" {excerpt}")
return "\n".join(lines)
def _build_meanwhile_digests_block(digests: list[dict]) -> str | None:
"""Render the ``Meanwhile while you were away:`` block for T65.
One bullet per pending digest, formatted as ``- {summary}`` with the
summary truncated to ~200 characters per spec. Returns ``None`` when
there are no pending digests so the caller can omit the entire block.
The block is rendered ONLY when the prompt is for a regular you-scene
(``present_set_kind != "host_guest"``); the caller filters before
constructing the digests list.
"""
if not digests:
return None
lines = ["Meanwhile while you were away:"]
for d in digests:
summary = d.get("summary") or ""
if len(summary) > 200:
summary = summary[:199] + ""
if summary:
lines.append(f"- {summary}")
if len(lines) == 1:
return None
return "\n".join(lines)
def _build_open_threads_block(threads: list[dict]) -> str | None:
"""Render the ``Open threads:`` block for Phase 3 Task 60.
One bullet per thread, formatted as ``- {title}: {summary}`` with the
summary truncated to ~120 characters. Returns ``None`` when there are
no open threads so the caller can omit the entire block.
"""
if not threads:
return None
lines = ["Open threads:"]
for t in threads:
title = t.get("title") or "?"
summary = t.get("summary") or ""
if len(summary) > 120:
summary = summary[:119] + ""
if summary:
lines.append(f"- {title}: {summary}")
else:
lines.append(f"- {title}")
return "\n".join(lines)
def _closing_instruction(speaker_name: str, addressee_name: str) -> str: def _closing_instruction(speaker_name: str, addressee_name: str) -> str:
return ( return (
f"Continue the scene as {speaker_name}, in their voice, responding " f"Continue the scene as {speaker_name}, in their voice, responding "
@@ -252,6 +371,25 @@ def _resolve_previous_scene_summary(
return mem[0] return mem[0]
def _witness_role_for(speaker_bot_id: str, host_bot_id: str | None) -> str:
"""Return the witness POV role for the speaker's memory query.
The host bot of a chat queries memories with ``witness_role="host"``;
the guest bot queries with ``witness_role="guest"``. Phase 2 T46
pinned the contract on ``search_memories``; this helper applies it
at the call site so a guest-as-speaker doesn't silently retrieve
memories under the wrong POV mask.
When ``host_bot_id`` is ``None`` (degenerate case from a half-seeded
chat or Phase-1 path), the speaker is treated as the host so the
query falls back to the host POV mask rather than silently masking
the speaker's own memories as a guest.
"""
if host_bot_id is None or speaker_bot_id == host_bot_id:
return "host"
return "guest"
def _resolve_addressee( def _resolve_addressee(
conn: Connection, addressee: str, you: dict | None conn: Connection, addressee: str, you: dict | None
) -> tuple[str, str]: ) -> tuple[str, str]:
@@ -287,6 +425,8 @@ def assemble_narrative_prompt(
budget_soft: int = 6000, budget_soft: int = 6000,
budget_hard: int = 8000, budget_hard: int = 8000,
encoding_name: str = "cl100k_base", encoding_name: str = "cl100k_base",
guest_id: str | None = None,
present_set_kind: str = "you_host",
) -> list[Message]: ) -> list[Message]:
"""Assemble the narrative prompt for ``speaker_bot_id`` to respond. """Assemble the narrative prompt for ``speaker_bot_id`` to respond.
@@ -313,9 +453,26 @@ def assemble_narrative_prompt(
if chat is None: if chat is None:
raise ValueError(f"chat_id {chat_id!r} not found") raise ValueError(f"chat_id {chat_id!r} not found")
# Auto-detect guest from chat state when caller didn't pass one.
# Phase 1 chats have ``guest_bot_id is None``; the auto-detect is a
# no-op there and the function behaves exactly as before.
if guest_id is None:
guest_id = chat.get("guest_bot_id")
# A speaker addressing themself as guest doesn't add a third party.
if guest_id is not None and guest_id == speaker_bot_id:
guest_id = None
you = get_you(conn) you = get_you(conn)
addressee_id, addressee_name = _resolve_addressee(conn, addressee, you) addressee_id, addressee_name = _resolve_addressee(conn, addressee, you)
# T64: meanwhile-mode marker. When present_set_kind == "host_guest"
# the user ("you") is NOT a witness in the scene — bots speak only to
# each other. The local flag below is consumed by the activity-block
# builder (skip the "you" bullet entirely) and the other-edges filter
# (drop any speaker -> "you" rendering). Default "you_host" preserves
# the Phase 1/2/3 behavior for normal turns.
_exclude_you = present_set_kind == "host_guest"
# ---- Build all components as text strings ------------------------------ # ---- Build all components as text strings ------------------------------
speaker_identity = _build_speaker_identity(bot) speaker_identity = _build_speaker_identity(bot)
@@ -325,21 +482,111 @@ def assemble_narrative_prompt(
addressee_name, addressee_name,
) )
# Activity for present entities. Phase 1: you + speaker bot. (When a # Activity for present entities — single ACTIVITIES: block with up
# guest is added in Phase 1+, callers that know about it can pass # to three bullets (you, speaker, guest). The block itself is
# extra activities via a future hook; for now we keep it strict.) # MUST-tier and survives all trims, but bullet-level trim drops
activities: list[dict] = [] # bullets in the order guest -> you, keeping the speaker bullet
# (the speaker's own current activity is the load-bearing slice).
#
# T71.2 chose Option B from the polish plan: pre-truncate the
# bullets list at trim time before _build_activity_block runs,
# rather than introducing a granular tier mode in the trim
# machinery. The single-block render avoids the dual-ACTIVITIES:
# header that Phase 2 T43 introduced (read by some LLMs as a
# duplicate-section bug).
you_activity: dict | None = None
if not _exclude_you:
you_act = get_activity(conn, "you") you_act = get_activity(conn, "you")
if you_act is not None: if you_act is not None:
you_act = dict(you_act) you_activity = dict(you_act)
you_act["_display_name"] = (you or {}).get("name") or "you" you_activity["_display_name"] = (you or {}).get("name") or "you"
activities.append(you_act)
speaker_activity: dict | None = None
bot_act = get_activity(conn, speaker_bot_id) bot_act = get_activity(conn, speaker_bot_id)
if bot_act is not None: if bot_act is not None:
bot_act = dict(bot_act) speaker_activity = dict(bot_act)
bot_act["_display_name"] = bot["name"] speaker_activity["_display_name"] = bot["name"]
activities.append(bot_act)
activity_block = _build_activity_block(activities) guest_activity: dict | None = None
if guest_id is not None:
guest_act = get_activity(conn, guest_id)
if guest_act is not None:
guest_activity = dict(guest_act)
guest_bot = get_bot(conn, guest_id)
guest_activity["_display_name"] = (
guest_bot["name"] if guest_bot else guest_id
)
def _activity_block_for(
*, include_you: bool, include_guest: bool
) -> str | None:
"""Render the single ACTIVITIES: block with the requested bullets.
Speaker bullet is always included (it's the MUST-tier baseline);
``you`` and ``guest`` bullets are toggled by the caller during
trim. Returns None when no bullets remain.
"""
bullets: list[dict] = []
if include_you and you_activity is not None:
bullets.append(you_activity)
if speaker_activity is not None:
bullets.append(speaker_activity)
if include_guest and guest_activity is not None:
bullets.append(guest_activity)
return _build_activity_block(bullets)
# SHOULD-tier group-node block (Phase 2 / Task 43): rendered only
# when the group_node row is present AND it covers all three of
# you + host + guest (per the Task 43 spec).
group_node_block: str | None = None
if guest_id is not None:
gn = get_group_node(conn, chat_id)
if gn is not None:
members = set(gn.get("members") or [])
host_id = chat.get("host_bot_id")
required = {"you"}
if host_id is not None:
required.add(host_id)
required.add(guest_id)
if required.issubset(members):
group_node_block = _build_group_node_block(gn)
# SHOULD-tier active events + open threads (Phase 3 / Task 60).
# Auto-detect both from the chat_id per the Phase 2 T43 precedent —
# no new function parameter. Both blocks are omit-when-empty so a
# Phase 1 chat with no events/threads renders identically to before.
active_events_block = _build_active_events_block(
list_active_events(conn, chat_id)
)
open_threads_block = _build_open_threads_block(
list_open_threads(conn, chat_id)
)
# SHOULD-tier meanwhile digest (Phase 3 / Task 65). Only surfaces
# when the prompt is for a regular you-scene (NOT for a meanwhile
# child scene — the absent player is the audience, not the bots
# currently mid-meanwhile). We distinguish via the chat's active
# scene's ``present_set_kind``; a missing scene row defaults to a
# you-scene render so the block can still surface during the
# post-meanwhile-close transition before the next scene opens.
#
# Consumption is INTENTIONALLY left to the post_turn flow (a
# ``consume_pending_meanwhile_digests`` helper, see below) rather
# than emitted inline here. Surfacing has no side-effects; the
# caller appends ``meanwhile_digest_consumed`` after the response
# streams. This keeps prompt assembly pure and deterministic — the
# Phase 1 invariant T29's regenerate flow relies on.
meanwhile_digests_block: str | None = None
active_scene_kind: str | None = None
if chat.get("active_scene_id"):
active_sc = get_scene(conn, chat["active_scene_id"])
if active_sc:
active_scene_kind = active_sc.get("present_set_kind")
if active_scene_kind != "host_guest":
pending_digests = list_pending_meanwhile_digests(conn, chat_id)
meanwhile_digests_block = _build_meanwhile_digests_block(
pending_digests
)
container = None container = None
if chat.get("active_scene_id"): if chat.get("active_scene_id"):
@@ -352,9 +599,15 @@ def assemble_narrative_prompt(
container = get_container(conn, scene["container_id"]) container = get_container(conn, scene["container_id"])
scene_block = _build_scene_block(chat, container, scene) scene_block = _build_scene_block(chat, container, scene)
# Other edges: speaker → non-addressee. # Other edges: speaker → non-addressee. In meanwhile mode (host_guest)
# the speaker -> "you" edge is filtered out as well — "you" isn't
# part of the present set, so surfacing the speaker's relationship
# to the user inside a private bot-to-bot beat would leak context
# the bots aren't supposed to be drawing on right now.
all_outgoing = list_edges_for(conn, speaker_bot_id) all_outgoing = list_edges_for(conn, speaker_bot_id)
other_edges_raw = [e for e in all_outgoing if e.get("target_id") != addressee_id] other_edges_raw = [e for e in all_outgoing if e.get("target_id") != addressee_id]
if _exclude_you:
other_edges_raw = [e for e in other_edges_raw if e.get("target_id") != "you"]
for e in other_edges_raw: for e in other_edges_raw:
tid = e.get("target_id") tid = e.get("target_id")
if tid == "you": if tid == "you":
@@ -373,7 +626,12 @@ def assemble_narrative_prompt(
memory_summaries = [] memory_summaries = []
if query: if query:
try: try:
hits = search_memories(conn, speaker_bot_id, "host", query, k=4) witness_role = _witness_role_for(
speaker_bot_id, chat.get("host_bot_id")
)
hits = search_memories(
conn, speaker_bot_id, witness_role, query, k=4
)
memory_summaries = [h["pov_summary"] for h in hits] memory_summaries = [h["pov_summary"] for h in hits]
except Exception: except Exception:
memory_summaries = [] memory_summaries = []
@@ -392,11 +650,18 @@ def assemble_narrative_prompt(
last4 = dialogue_full[-4:] if dialogue_full else [] last4 = dialogue_full[-4:] if dialogue_full else []
must_dialogue_block = _build_dialogue_block(last4, earlier_summary=None) must_dialogue_block = _build_dialogue_block(last4, earlier_summary=None)
# MUST-tier ACTIVITIES floor: the speaker bullet alone (you and
# guest bullets are dropped first under bullet-level trim before
# the block bottoms out at speaker-only).
must_activity_block = _activity_block_for(
include_you=False, include_guest=False
)
must_blocks: list[str | None] = [ must_blocks: list[str | None] = [
speaker_identity, speaker_identity,
edge_to_addressee, edge_to_addressee,
scene_block, scene_block,
activity_block, must_activity_block,
must_dialogue_block, must_dialogue_block,
closing, closing,
] ]
@@ -421,6 +686,12 @@ def assemble_narrative_prompt(
include_previous_scene: bool, include_previous_scene: bool,
include_memories_top_k: int, include_memories_top_k: int,
dialogue_keep: int, dialogue_keep: int,
include_you_activity: bool = True,
include_guest_activity: bool = True,
include_group_node: bool = True,
include_active_events: bool = True,
include_open_threads: bool = True,
include_meanwhile_digests: bool = True,
) -> tuple[str, int, list[dict]]: ) -> tuple[str, int, list[dict]]:
# dialogue: keep the last `dialogue_keep` turns verbatim; older # dialogue: keep the last `dialogue_keep` turns verbatim; older
# turns become an "earlier:" placeholder line. # turns become an "earlier:" placeholder line.
@@ -441,12 +712,27 @@ def assemble_narrative_prompt(
if include_previous_scene else None if include_previous_scene else None
) )
# Single ACTIVITIES: block, bullet-level trim (T71.2). Guest
# bullet drops first, then the you bullet; speaker bullet is the
# MUST-tier floor and always present when an activity row exists.
activity_block = _activity_block_for(
include_you=include_you_activity,
include_guest=include_guest_activity,
)
body = _join_blocks([ body = _join_blocks([
speaker_identity, speaker_identity,
edge_to_addressee, edge_to_addressee,
other_edges_block if include_other_edges else None, other_edges_block if include_other_edges else None,
scene_block, scene_block,
activity_block, activity_block,
group_node_block if include_group_node else None,
active_events_block if include_active_events else None,
open_threads_block if include_open_threads else None,
(
meanwhile_digests_block
if include_meanwhile_digests else None
),
prev_block, prev_block,
memories_block, memories_block,
dialogue_block, dialogue_block,
@@ -463,12 +749,37 @@ def assemble_narrative_prompt(
nice_memories_k = min(4, len(memory_summaries)) nice_memories_k = min(4, len(memory_summaries))
include_prev = previous_scene_summary is not None include_prev = previous_scene_summary is not None
include_other = other_edges_block is not None include_other = other_edges_block is not None
include_you_activity = you_activity is not None
include_guest_activity = guest_activity is not None
include_group_node = group_node_block is not None
include_active_events = active_events_block is not None
include_open_threads = open_threads_block is not None
include_meanwhile_digests = meanwhile_digests_block is not None
def _build(*, prev: bool, mem_k: int, dlg: int, other: bool,
you_act: bool, guest_act: bool, group: bool,
events: bool, threads: bool,
digests: bool) -> tuple[str, int]:
body, total, _ = assemble( body, total, _ = assemble(
include_other_edges=include_other, include_other_edges=other,
include_previous_scene=include_prev, include_previous_scene=prev,
include_memories_top_k=nice_memories_k, include_memories_top_k=mem_k,
dialogue_keep=nice_dialogue_keep, dialogue_keep=dlg,
include_you_activity=you_act,
include_guest_activity=guest_act,
include_group_node=group,
include_active_events=events,
include_open_threads=threads,
include_meanwhile_digests=digests,
)
return body, total
body, total = _build(
prev=include_prev, mem_k=nice_memories_k, dlg=nice_dialogue_keep,
other=include_other, you_act=include_you_activity,
guest_act=include_guest_activity, group=include_group_node,
events=include_active_events, threads=include_open_threads,
digests=include_meanwhile_digests,
) )
# If under soft, we're done. # If under soft, we're done.
@@ -477,35 +788,58 @@ def assemble_narrative_prompt(
# Drop NICE in order: previous scene → memories beyond top-2 → # Drop NICE in order: previous scene → memories beyond top-2 →
# older dialogue turns (collapse to 4). # older dialogue turns (collapse to 4).
#
# T71.3 — order rationale: the §6.3 spec lists NICE-tier members
# with previous-scene LAST, which read as a literal trim order
# during T18 review. We deliberately keep the greedy order shown
# here (previous-scene FIRST) for two reasons:
#
# 1. Cheapest-impact-first: a per-POV previous-scene summary is
# a single short paragraph that loses very little narrative
# continuity when dropped, while the older dialogue turns it
# is competing with carry the speaker's last few beats — those
# ground the next response far more concretely.
# 2. Greedy lookahead is more expensive than the marginal
# narrative loss. Dropping previous-scene typically clears
# the soft-budget slack in one step; trying memories or
# dialogue first would routinely require multiple recompute
# passes through the assembler.
#
# The pin test test_nice_trim_order_documented locks this order so
# a future refactor can't quietly invert it without surfacing the
# decision.
if include_prev: if include_prev:
body, total, _ = assemble(
include_other_edges=include_other,
include_previous_scene=False,
include_memories_top_k=nice_memories_k,
dialogue_keep=nice_dialogue_keep,
)
include_prev = False include_prev = False
body, total = _build(
prev=include_prev, mem_k=nice_memories_k, dlg=nice_dialogue_keep,
other=include_other, you_act=include_you_activity,
guest_act=include_guest_activity, group=include_group_node,
events=include_active_events, threads=include_open_threads,
digests=include_meanwhile_digests,
)
if total <= budget_soft: if total <= budget_soft:
return _emit(body, user_turn_prose) return _emit(body, user_turn_prose)
if nice_memories_k > 2: if nice_memories_k > 2:
nice_memories_k = 2 nice_memories_k = 2
body, total, _ = assemble( body, total = _build(
include_other_edges=include_other, prev=include_prev, mem_k=nice_memories_k, dlg=nice_dialogue_keep,
include_previous_scene=False, other=include_other, you_act=include_you_activity,
include_memories_top_k=nice_memories_k, guest_act=include_guest_activity, group=include_group_node,
dialogue_keep=nice_dialogue_keep, events=include_active_events, threads=include_open_threads,
digests=include_meanwhile_digests,
) )
if total <= budget_soft: if total <= budget_soft:
return _emit(body, user_turn_prose) return _emit(body, user_turn_prose)
if nice_dialogue_keep > baseline_keep: if nice_dialogue_keep > baseline_keep:
nice_dialogue_keep = baseline_keep nice_dialogue_keep = baseline_keep
body, total, _ = assemble( body, total = _build(
include_other_edges=include_other, prev=include_prev, mem_k=nice_memories_k, dlg=nice_dialogue_keep,
include_previous_scene=False, other=include_other, you_act=include_you_activity,
include_memories_top_k=nice_memories_k, guest_act=include_guest_activity, group=include_group_node,
dialogue_keep=nice_dialogue_keep, events=include_active_events, threads=include_open_threads,
digests=include_meanwhile_digests,
) )
if total <= budget_soft: if total <= budget_soft:
return _emit(body, user_turn_prose) return _emit(body, user_turn_prose)
@@ -513,21 +847,95 @@ def assemble_narrative_prompt(
# Drop more NICE until we're under hard: memories all the way to 0. # Drop more NICE until we're under hard: memories all the way to 0.
while nice_memories_k > 0 and total > budget_hard: while nice_memories_k > 0 and total > budget_hard:
nice_memories_k = max(0, nice_memories_k - 1) nice_memories_k = max(0, nice_memories_k - 1)
body, total, _ = assemble( body, total = _build(
include_other_edges=include_other, prev=include_prev, mem_k=nice_memories_k, dlg=nice_dialogue_keep,
include_previous_scene=False, other=include_other, you_act=include_you_activity,
include_memories_top_k=nice_memories_k, guest_act=include_guest_activity, group=include_group_node,
dialogue_keep=nice_dialogue_keep, events=include_active_events, threads=include_open_threads,
digests=include_meanwhile_digests,
)
# Drop SHOULD-tier extras in order:
# 1. meanwhile digests block (T65: SHOULD-tier; refers to a past
# meanwhile scene — least critical to the speaker's immediate
# voice, so dropped first among SHOULD)
# 2. open threads block (T60: SHOULD-tier; least critical to the
# speaker's immediate voice — drop next among SHOULD)
# 3. active events block (T60: same tier, drops next)
# 4. guest activity bullet (T71.2: bullet-level trim within the
# single ACTIVITIES: block — guest goes first per Task 43 spec)
# 5. group node block
# 6. you activity bullet (still SHOULD-tier; speaker bullet is the
# MUST-tier floor and never dropped)
# 7. other edges
if include_meanwhile_digests and total > budget_hard:
include_meanwhile_digests = False
body, total = _build(
prev=include_prev, mem_k=nice_memories_k, dlg=nice_dialogue_keep,
other=include_other, you_act=include_you_activity,
guest_act=include_guest_activity, group=include_group_node,
events=include_active_events, threads=include_open_threads,
digests=include_meanwhile_digests,
)
if include_open_threads and total > budget_hard:
include_open_threads = False
body, total = _build(
prev=include_prev, mem_k=nice_memories_k, dlg=nice_dialogue_keep,
other=include_other, you_act=include_you_activity,
guest_act=include_guest_activity, group=include_group_node,
events=include_active_events, threads=include_open_threads,
digests=include_meanwhile_digests,
)
if include_active_events and total > budget_hard:
include_active_events = False
body, total = _build(
prev=include_prev, mem_k=nice_memories_k, dlg=nice_dialogue_keep,
other=include_other, you_act=include_you_activity,
guest_act=include_guest_activity, group=include_group_node,
events=include_active_events, threads=include_open_threads,
digests=include_meanwhile_digests,
)
if include_guest_activity and total > budget_hard:
include_guest_activity = False
body, total = _build(
prev=include_prev, mem_k=nice_memories_k, dlg=nice_dialogue_keep,
other=include_other, you_act=include_you_activity,
guest_act=include_guest_activity, group=include_group_node,
events=include_active_events, threads=include_open_threads,
digests=include_meanwhile_digests,
)
if include_group_node and total > budget_hard:
include_group_node = False
body, total = _build(
prev=include_prev, mem_k=nice_memories_k, dlg=nice_dialogue_keep,
other=include_other, you_act=include_you_activity,
guest_act=include_guest_activity, group=include_group_node,
events=include_active_events, threads=include_open_threads,
digests=include_meanwhile_digests,
)
if include_you_activity and total > budget_hard:
include_you_activity = False
body, total = _build(
prev=include_prev, mem_k=nice_memories_k, dlg=nice_dialogue_keep,
other=include_other, you_act=include_you_activity,
guest_act=include_guest_activity, group=include_group_node,
events=include_active_events, threads=include_open_threads,
digests=include_meanwhile_digests,
) )
# Drop SHOULD: other edges.
if include_other and total > budget_hard: if include_other and total > budget_hard:
include_other = False include_other = False
body, total, _ = assemble( body, total = _build(
include_other_edges=False, prev=include_prev, mem_k=nice_memories_k, dlg=nice_dialogue_keep,
include_previous_scene=False, other=include_other, you_act=include_you_activity,
include_memories_top_k=nice_memories_k, guest_act=include_guest_activity, group=include_group_node,
dialogue_keep=nice_dialogue_keep, events=include_active_events, threads=include_open_threads,
digests=include_meanwhile_digests,
) )
if total > budget_hard: if total > budget_hard:
@@ -553,4 +961,38 @@ def _emit(system_body: str, user_turn_prose: str | None) -> list[Message]:
return msgs return msgs
__all__ = ["assemble_narrative_prompt"] def consume_pending_meanwhile_digests(conn: Connection, chat_id: str) -> int:
"""Mark every pending meanwhile digest for ``chat_id`` as consumed.
Called by the post_turn flow AFTER the assistant response streams,
once for the first you-turn that surfaced any pending digests. We
keep this side-effect out of :func:`assemble_narrative_prompt` so
prompt assembly stays pure (T29's regenerate flow rebuilds prompts
repeatedly without state mutation).
Returns the number of digests consumed (0 when none were pending).
"""
from datetime import datetime, timezone
from chat.eventlog.log import append_and_apply
pending = list_pending_meanwhile_digests(conn, chat_id)
if not pending:
return 0
now = datetime.now(timezone.utc).isoformat()
for d in pending:
append_and_apply(
conn,
kind="meanwhile_digest_consumed",
payload={
"digest_id": d["id"],
"consumed_at": now,
},
)
return len(pending)
__all__ = [
"assemble_narrative_prompt",
"consume_pending_meanwhile_digests",
]
+447 -45
View File
@@ -26,6 +26,44 @@ Phase 1 simplifications (per the plan's "bound it" guidance):
so affinity/trust/knowledge reflect the new output. so affinity/trust/knowledge reflect the new output.
- The route does not broadcast a fresh ``turn_html`` SSE event; T34 - The route does not broadcast a fresh ``turn_html`` SSE event; T34
polishes UI swaps. The user refreshes the page to see the new turn. polishes UI swaps. The user refreshes the page to see the new turn.
*(T73.1 closed this gap — see Phase 2.5 changes below.)*
Phase 2 changes (T44):
- Multi-entity prompt assembly: ``guest_id`` is forwarded to the
prompt assembler so the regenerated narrative sees the same
guest-aware context the original turn did.
- Multi-witness memory write: ``record_turn_memory_for_present`` fans
out one ``memory_written`` event per witness when a guest is present.
- Multi-pair state-update: ``compute_state_updates_for_present`` emits
one ``edge_update`` per directed pair across present entities. With
three present that's six edges instead of two.
- Interjection regeneration is **deferred to Phase 2.5**. Regenerate
only re-streams the addressee turn for v2; ``detect_interjection``
is not invoked here. If the prior turn fired an interjection it
remains attached to the original assistant_turn (which is superseded
alongside the regenerated turn) — Phase 2.5 will revisit.
Phase 2.5 changes:
- T73.1: After the new ``assistant_turn`` lands we publish a
``turn_html_replace`` SSE event carrying the rendered HTML for the
regenerated turn plus the original assistant_turn's event_id as
``supersedes_id`` so connected tabs can swap the prior DOM node
in-place. We use a NEW event name (rather than re-using ``turn_html``)
because the existing HTMX ``sse-swap="turn_html"`` consumer expects a
raw-HTML body and an *append* semantic; ``turn_html_replace`` is a
JSON payload (sse.py auto-serialises when extra keys accompany
``data``) so the front-end JS can read ``supersedes_id`` and replace
the right node.
- T73.2: Interjection regeneration. When the original assistant_turn
group included an interjection beat we redo BOTH the primary and the
interjection — re-running ``detect_interjection`` against the new
primary text. If the classifier returns False this time we supersede
the original interjection without appending a replacement.
- T73.3: The defensive degrade-to-1:1 for stale ``guest_bot_id``
references was removed — Phase 2 T47 fixed the root cause (resets
clear the reference) so the guard is dead code.
""" """
from __future__ import annotations from __future__ import annotations
@@ -35,13 +73,18 @@ from sqlite3 import Connection
from chat.config import Settings from chat.config import Settings
from chat.eventlog.log import append_and_apply, append_event from chat.eventlog.log import append_and_apply, append_event
from chat.services.memory_write import record_turn_memory from chat.services.event_lifecycle import detect_event_transitions
from chat.services.event_promotion import promote_completed_event
from chat.services.interjection import detect_interjection
from chat.services.memory_write import record_turn_memory_for_present
from chat.services.multi_state_update import compute_state_updates_for_present
from chat.services.prompt import assemble_narrative_prompt from chat.services.prompt import assemble_narrative_prompt
from chat.services.state_update import compute_state_update
from chat.state.edges import get_edge from chat.state.edges import get_edge
from chat.state.entities import get_bot, get_you from chat.state.entities import get_bot, get_you
from chat.state.events import list_active_events
from chat.state.world import active_scene, get_chat from chat.state.world import active_scene, get_chat
from chat.web.pubsub import publish from chat.web.pubsub import publish
from chat.web.render import render_turn_html
async def regenerate_assistant_turn( async def regenerate_assistant_turn(
@@ -72,6 +115,16 @@ async def regenerate_assistant_turn(
"persona": "", "persona": "",
} }
# Phase 2: surface the guest (if any) so the prompt assembler and
# downstream multi-entity passes see the same shape post_turn does.
# Phase 2 T47 made bot_reset cascade-clear ``chat.guest_bot_id`` when
# the referenced bot is purged (verified by tests/test_reset.py), so
# we trust the column here: it's either a valid bot id or NULL.
guest_bot_id = chat.get("guest_bot_id")
guest_bot: dict | None = (
get_bot(conn, guest_bot_id) if guest_bot_id is not None else None
)
# 1. Locate the original assistant_turn event. # 1. Locate the original assistant_turn event.
row = conn.execute( row = conn.execute(
"SELECT payload_json FROM event_log " "SELECT payload_json FROM event_log "
@@ -83,6 +136,44 @@ async def regenerate_assistant_turn(
original_assistant_payload = json.loads(row[0]) original_assistant_payload = json.loads(row[0])
original_user_turn_id = original_assistant_payload.get("user_turn_id") original_user_turn_id = original_assistant_payload.get("user_turn_id")
# 1a. Look up any sibling interjection beat in the same turn group
# (T73.2). The original group is (primary + optional interjection),
# both pinned to the same ``user_turn_id``. The interjection has a
# populated ``interjection_of`` field in its payload — its speaker is
# the silent witness (the bot that wasn't the primary addressee).
# Filter on ``superseded_by IS NULL`` so prior regenerates of this
# group don't reappear as siblings.
original_interjection_event_id: int | None = None
original_interjection_payload: dict | None = None
if original_user_turn_id is not None:
sibling_cur = conn.execute(
"SELECT id, payload_json FROM event_log "
"WHERE kind = 'assistant_turn' "
" AND id != ? "
" AND superseded_by IS NULL",
(original_assistant_event_id,),
)
for sib_id, sib_payload_json in sibling_cur.fetchall():
sib_payload = json.loads(sib_payload_json)
if sib_payload.get("user_turn_id") != original_user_turn_id:
continue
if not sib_payload.get("interjection_of"):
continue
original_interjection_event_id = sib_id
original_interjection_payload = sib_payload
break
# Phase 2 v2 regenerates only the addressee turn — preserve whichever
# bot the original turn was attributed to, falling back to the host
# for legacy rows that pre-date multi-entity support.
speaker_bot_id = original_assistant_payload.get("speaker_id") or host_bot_id
if speaker_bot_id == host_bot_id:
speaker_bot = host_bot
elif guest_bot is not None and speaker_bot_id == guest_bot.get("id"):
speaker_bot = guest_bot
else:
speaker_bot = get_bot(conn, speaker_bot_id) or host_bot
speaker_bot_id = speaker_bot.get("id", host_bot_id)
# 2. Determine the prose for the new prompt and (when edited) capture # 2. Determine the prose for the new prompt and (when edited) capture
# the user_turn_edit event up front so the new event ids exist before # the user_turn_edit event up front so the new event ids exist before
# we link them from the assistant_turn payload. # we link them from the assistant_turn payload.
@@ -137,20 +228,26 @@ async def regenerate_assistant_turn(
if kind in ("user_turn", "user_turn_edit"): if kind in ("user_turn", "user_turn_edit"):
recent.append({"speaker": you_name, "text": p.get("prose", "")}) recent.append({"speaker": you_name, "text": p.get("prose", "")})
else: else:
recent.append( spk = p.get("speaker_id", "bot")
{"speaker": host_bot.get("name", "bot"), "text": p.get("text", "")} spk_name = host_bot.get("name", "bot")
) if spk == host_bot_id:
spk_name = host_bot.get("name", "bot")
elif guest_bot is not None and spk == guest_bot.get("id"):
spk_name = guest_bot.get("name", "bot")
recent.append({"speaker": spk_name, "text": p.get("text", "")})
# 4. Assemble the narrative prompt. ``recent`` already excludes the # 4. Assemble the narrative prompt. ``recent`` already excludes the
# current user prose, which we pass through ``user_turn_prose``. # current user prose, which we pass through ``user_turn_prose``.
# Phase 2: forward ``guest_id`` so the prompt sees the third party.
messages = assemble_narrative_prompt( messages = assemble_narrative_prompt(
conn, conn,
chat_id=chat_id, chat_id=chat_id,
speaker_bot_id=host_bot_id, speaker_bot_id=speaker_bot_id,
user_turn_prose=prose_for_prompt or None, user_turn_prose=prose_for_prompt or None,
recent_dialogue=recent, recent_dialogue=recent,
budget_soft=settings.narrative_budget_soft, budget_soft=settings.narrative_budget_soft,
budget_hard=settings.narrative_budget_hard, budget_hard=settings.narrative_budget_hard,
guest_id=guest_bot_id,
) )
# 5. Stream the new narrative. # 5. Stream the new narrative.
@@ -164,7 +261,7 @@ async def regenerate_assistant_turn(
accumulated.append(chunk) accumulated.append(chunk)
await publish( await publish(
chat_id, chat_id,
{"event": "token", "text": chunk, "speaker_id": host_bot_id}, {"event": "token", "text": chunk, "speaker_id": speaker_bot_id},
) )
new_text = "".join(accumulated) new_text = "".join(accumulated)
@@ -177,7 +274,7 @@ async def regenerate_assistant_turn(
kind="assistant_turn", kind="assistant_turn",
payload={ payload={
"chat_id": chat_id, "chat_id": chat_id,
"speaker_id": host_bot_id, "speaker_id": speaker_bot_id,
"text": new_text, "text": new_text,
"truncated": False, "truncated": False,
"user_turn_id": ( "user_turn_id": (
@@ -195,89 +292,394 @@ async def regenerate_assistant_turn(
(new_assistant_event_id, original_assistant_event_id), (new_assistant_event_id, original_assistant_event_id),
) )
# 7a. Broadcast a turn_html_replace SSE event so connected tabs can
# swap the prior assistant_turn DOM node in-place (T73.1, Phase 1.5
# backlog #2). Uses a separate event name from post_turn's
# ``turn_html`` (which is append-only) because regenerate is a
# *replace* operation — see module docstring for the rationale.
speaker_name_for_render = (
speaker_bot.get("name", "bot") if speaker_bot is not None else "bot"
)
new_turn_html = render_turn_html(
speaker_name_for_render, new_text, role="bot"
)
await publish(
chat_id,
{
"event": "turn_html_replace",
"data": new_turn_html,
"turn_id": new_assistant_event_id,
"supersedes_id": original_assistant_event_id,
},
)
# 8. Re-run downstream classifier passes (memory write + state update # 8. Re-run downstream classifier passes (memory write + state update
# for both directed edges). Significance is intentionally skipped on # for every directed pair across present entities). Significance is
# regenerate (the prior score remains attached to the prior memory). # intentionally skipped on regenerate (the prior score remains
# attached to the prior memory). Phase 2.5 will add interjection
# regeneration; v2 leaves any prior interjection beat in place.
scene = active_scene(conn, chat_id) scene = active_scene(conn, chat_id)
record_turn_memory( record_turn_memory_for_present(
conn, conn,
chat_id=chat_id, chat_id=chat_id,
host_bot_id=host_bot_id, host_bot_id=host_bot_id,
guest_bot_id=guest_bot_id,
narrative_text=new_text, narrative_text=new_text,
scene_id=scene["id"] if scene else None, scene_id=scene["id"] if scene else None,
chat_clock_at=chat.get("time"), chat_clock_at=chat.get("time"),
) )
last_at = chat.get("time") last_at = chat.get("time")
speaker_name = (
speaker_bot.get("name", "bot") if speaker_bot is not None else "bot"
)
recent_for_update = recent + [ recent_for_update = recent + [
{"speaker": host_bot.get("name", "bot"), "text": new_text} {"speaker": speaker_name, "text": new_text}
] ]
edge_b2y = get_edge(conn, host_bot_id, "you") or { # Build present-entity inputs for the multi-pair state-update pass.
# Host first preserves the Phase 1 directed-pair order (host->you,
# then you->host) so existing canned-response fixtures still line up.
present_ids: list[str] = [host_bot_id, "you"]
present_names: dict[str, str] = {
host_bot_id: host_bot.get("name", "bot"),
"you": you_name,
}
personas: dict[str, str] = {
host_bot_id: host_bot.get("persona") or "",
"you": you_entity.get("persona") or "",
}
if guest_bot is not None and guest_bot_id is not None:
present_ids.append(guest_bot_id)
present_names[guest_bot_id] = guest_bot.get("name", "bot")
personas[guest_bot_id] = guest_bot.get("persona") or ""
prior_edges: dict[tuple[str, str], dict] = {}
for src in present_ids:
for tgt in present_ids:
if src == tgt:
continue
edge = get_edge(conn, src, tgt) or {
"affinity": 50, "affinity": 50,
"trust": 50, "trust": 50,
"summary": "", "summary": "",
} }
update_b2y = await compute_state_update( prior_edges[(src, tgt)] = edge
state_updates = await compute_state_updates_for_present(
client, client,
model=settings.classifier_model, classifier_model=settings.classifier_model,
source_id=host_bot_id, present_ids=present_ids,
target_id="you", present_names=present_names,
source_name=host_bot.get("name", "bot"), personas=personas,
source_persona=host_bot.get("persona", "") or "", prior_edges=prior_edges,
target_name=you_name,
prior_affinity=edge_b2y["affinity"],
prior_trust=edge_b2y["trust"],
prior_summary=edge_b2y.get("summary", "") or "",
recent_dialogue=recent_for_update, recent_dialogue=recent_for_update,
timeout_s=settings.classifier_timeout_s,
) )
for src_id, tgt_id, update in state_updates:
append_and_apply( append_and_apply(
conn, conn,
kind="edge_update", kind="edge_update",
payload={ payload={
"source_id": host_bot_id, "source_id": src_id,
"target_id": "you", "target_id": tgt_id,
"chat_id": chat_id, "chat_id": chat_id,
"affinity_delta": update_b2y.affinity_delta, "affinity_delta": update.affinity_delta,
"trust_delta": update_b2y.trust_delta, "trust_delta": update.trust_delta,
"knowledge_facts": update_b2y.knowledge_facts, "knowledge_facts": update.knowledge_facts,
"last_interaction_at": last_at, "last_interaction_at": last_at,
"last_interaction_chat_id": chat_id, "last_interaction_chat_id": chat_id,
}, },
) )
edge_y2b = get_edge(conn, "you", host_bot_id) or { # 9. Interjection regenerate branch (T73.2). When the original
# assistant_turn group included a follow-on interjection beat we need
# to revisit that beat against the regenerated primary. Three outcomes:
#
# - No original interjection: nothing to do; we already short-circuit
# above by leaving ``original_interjection_event_id`` as None.
# - Original interjection + classifier returns True: stream a fresh
# interjection from the silent witness, append it (with
# ``interjection_of`` linking to the new primary speaker), and
# supersede the original interjection's row. Also re-run memory
# + state-update so the second beat moves edges + writes memories.
# - Original interjection + classifier returns False: supersede the
# original interjection without appending a replacement. The
# regenerated group becomes "primary only" because the new primary
# no longer warrants a follow-on. No memory / state work needed
# for the absent beat.
#
# ``superseded_by`` on the original interjection's row points at the
# *new primary* in the no-replacement case (rather than NULL or a
# nonexistent id) so the row is consistently hidden by the standard
# ``superseded_by IS NULL`` timeline filter and the back-pointer
# leads somewhere meaningful for an "originally said …" affordance.
if original_interjection_event_id is not None and guest_bot is not None:
# Identify the silent witness from the original interjection's
# speaker_id (which is the bot that interjected last time). When
# we regenerate we keep the *same pair of present entities*, so
# the silent witness is whichever bot isn't the new primary
# speaker — derive it from present rather than reusing the prior
# speaker_id verbatim, in case the regenerated primary swapped
# who held the floor.
if speaker_bot_id == host_bot_id:
silent_witness = guest_bot
else:
silent_witness = host_bot
silent_witness_id = silent_witness.get("id")
edge_w_to_addr = get_edge(conn, silent_witness_id, speaker_bot_id) or {
"affinity": 50, "affinity": 50,
"trust": 50, "trust": 50,
"summary": "", "summary": "",
} }
update_y2b = await compute_state_update( edge_w_to_you = get_edge(conn, silent_witness_id, "you") or {
"affinity": 50,
"trust": 50,
"summary": "",
}
decision = await detect_interjection(
client, client,
model=settings.classifier_model, classifier_model=settings.classifier_model,
source_id="you", addressee_name=speaker_bot.get("name", "bot"),
target_id=host_bot_id, addressee_just_said=new_text,
source_name=you_name, silent_witness_name=silent_witness.get("name", "bot"),
source_persona=you_entity.get("persona", "") or "", silent_witness_persona=silent_witness.get("persona") or "",
target_name=host_bot.get("name", "bot"), silent_witness_edge_to_addressee=edge_w_to_addr,
prior_affinity=edge_y2b["affinity"], silent_witness_edge_to_you=edge_w_to_you,
prior_trust=edge_y2b["trust"], you_just_said=prose_for_prompt or "",
prior_summary=edge_y2b.get("summary", "") or "", timeout_s=settings.classifier_timeout_s,
recent_dialogue=recent_for_update,
) )
if decision.should_interject:
# Re-read recent so the just-appended primary is in the prompt.
interject_cur = conn.execute(
"SELECT id, kind, payload_json FROM event_log "
"WHERE kind IN ('user_turn', 'user_turn_edit', 'assistant_turn') "
" AND superseded_by IS NULL AND hidden = 0 "
"ORDER BY id DESC LIMIT 20",
)
interject_rows = list(reversed(interject_cur.fetchall()))
interject_recent: list[dict] = []
for _eid, kind, payload_json in interject_rows:
p = json.loads(payload_json)
if p.get("chat_id") != chat_id:
continue
if kind in ("user_turn", "user_turn_edit"):
interject_recent.append(
{"speaker": you_name, "text": p.get("prose", "")}
)
else:
spk = p.get("speaker_id", "bot")
if spk == host_bot_id:
spk_name = host_bot.get("name", "bot")
elif spk == guest_bot.get("id"):
spk_name = guest_bot.get("name", "bot")
else:
spk_name = "bot"
interject_recent.append(
{"speaker": spk_name, "text": p.get("text", "")}
)
if interject_recent and interject_recent[-1].get("speaker") == you_name:
interject_recent = interject_recent[:-1]
interject_messages = assemble_narrative_prompt(
conn,
chat_id=chat_id,
speaker_bot_id=silent_witness_id,
addressee=speaker_bot_id,
user_turn_prose=prose_for_prompt or None,
recent_dialogue=interject_recent,
budget_soft=settings.narrative_budget_soft,
budget_hard=settings.narrative_budget_hard,
guest_id=guest_bot_id,
)
interject_accumulated: list[str] = []
async for chunk in client.stream(
interject_messages,
model=settings.narrative_model,
max_tokens=settings.narrative_max_tokens,
temperature=settings.narrative_temperature,
):
interject_accumulated.append(chunk)
await publish(
chat_id,
{
"event": "token",
"text": chunk,
"speaker_id": silent_witness_id,
},
)
interject_text = "".join(interject_accumulated)
new_interjection_event_id = append_event(
conn,
kind="assistant_turn",
payload={
"chat_id": chat_id,
"speaker_id": silent_witness_id,
"text": interject_text,
"truncated": False,
"user_turn_id": (
new_user_event_id
if new_user_event_id is not None
else original_user_turn_id
),
"regenerated_from": original_interjection_event_id,
"interjection_of": speaker_bot_id,
},
)
# Supersede the original interjection by the new one.
conn.execute(
"UPDATE event_log SET superseded_by = ? WHERE id = ?",
(new_interjection_event_id, original_interjection_event_id),
)
# Broadcast a replace event so connected tabs swap the prior
# interjection node in-place (mirrors T73.1's primary swap).
interject_html = render_turn_html(
silent_witness.get("name", "bot"), interject_text, role="bot"
)
await publish(
chat_id,
{
"event": "turn_html_replace",
"data": interject_html,
"turn_id": new_interjection_event_id,
"supersedes_id": original_interjection_event_id,
},
)
# Memory write for the new interjection beat (one event per
# present witness).
record_turn_memory_for_present(
conn,
chat_id=chat_id,
host_bot_id=host_bot_id,
guest_bot_id=guest_bot_id,
narrative_text=interject_text,
scene_id=scene["id"] if scene else None,
chat_clock_at=chat.get("time"),
)
# Re-run the multi-pair state-update with the post-interjection
# dialogue tail so deltas land on the post-primary baseline.
recent_post_interject = recent_for_update + [
{
"speaker": silent_witness.get("name", "bot"),
"text": interject_text,
}
]
prior_edges_post: dict[tuple[str, str], dict] = {}
for src in present_ids:
for tgt in present_ids:
if src == tgt:
continue
edge = get_edge(conn, src, tgt) or {
"affinity": 50,
"trust": 50,
"summary": "",
}
prior_edges_post[(src, tgt)] = edge
state_updates_post = await compute_state_updates_for_present(
client,
classifier_model=settings.classifier_model,
present_ids=present_ids,
present_names=present_names,
personas=personas,
prior_edges=prior_edges_post,
recent_dialogue=recent_post_interject,
timeout_s=settings.classifier_timeout_s,
)
for src_id, tgt_id, update in state_updates_post:
append_and_apply( append_and_apply(
conn, conn,
kind="edge_update", kind="edge_update",
payload={ payload={
"source_id": "you", "source_id": src_id,
"target_id": host_bot_id, "target_id": tgt_id,
"chat_id": chat_id, "chat_id": chat_id,
"affinity_delta": update_y2b.affinity_delta, "affinity_delta": update.affinity_delta,
"trust_delta": update_y2b.trust_delta, "trust_delta": update.trust_delta,
"knowledge_facts": update_y2b.knowledge_facts, "knowledge_facts": update.knowledge_facts,
"last_interaction_at": last_at, "last_interaction_at": last_at,
"last_interaction_chat_id": chat_id, "last_interaction_chat_id": chat_id,
}, },
) )
else:
# Classifier said "no follow-on this time" — supersede the
# original interjection without a replacement. Point the
# back-pointer at the new primary so the row is consistently
# hidden by the standard timeline filter.
conn.execute(
"UPDATE event_log SET superseded_by = ? WHERE id = ?",
(new_assistant_event_id, original_interjection_event_id),
)
# 10. Event-lifecycle detection (Phase 3, T61). Mirrors the post_turn
# block: classify whether any active events transitioned in the
# regenerated narrative and append the corresponding event_started /
# event_completed / event_cancelled. ``promote_completed_event``
# runs inline after a completion so promotion artifacts land in the
# same regenerate path.
#
# Phase 3.5 follow-up: when a regenerate replaces a turn that had
# already produced event transitions, those original transitions are
# NOT undone here. The superseded ``assistant_turn`` group keeps its
# prior ``event_started`` / ``event_completed`` events in the log
# (they remain projected onto the events table). Phase 3.5 will add
# an "undo lifecycle" step to roll back the prior transitions before
# re-classifying the regenerated text. For v3 we accept that a
# regenerate-after-completion will double-emit promotion artifacts
# if the new text re-completes the same event — narratively rare,
# and a true fix needs the lifecycle-undo pass.
new_active_events = list_active_events(conn, chat_id)
if new_active_events:
lifecycle_decision = await detect_event_transitions(
client,
classifier_model=settings.classifier_model,
narrative_text=new_text,
active_events=new_active_events,
timeout_s=settings.classifier_timeout_s,
)
for transition in lifecycle_decision.transitions:
if transition.new_status == "active":
append_and_apply(
conn,
kind="event_started",
payload={
"event_id": transition.event_id,
"started_at": chat.get("time"),
},
)
elif transition.new_status == "completed":
append_and_apply(
conn,
kind="event_completed",
payload={
"event_id": transition.event_id,
"completed_at": chat.get("time"),
},
)
promote_completed_event(
conn,
event_id=transition.event_id,
chat_id=chat_id,
chat_clock_at=chat.get("time"),
)
elif transition.new_status == "cancelled":
append_and_apply(
conn,
kind="event_cancelled",
payload={
"event_id": transition.event_id,
"completed_at": chat.get("time"),
},
)
return new_text return new_text
+107
View File
@@ -0,0 +1,107 @@
"""Parse user-supplied "have they met?" prose into per-direction seed
content for two bots' edges (T38).
Per Requirements §5.2, when two bots first co-appear in a chat, the user
is offered a small drawer asking "Have they met before? If yes, write a
short prose seed describing how." That prose lands here and is parsed
into a :class:`RelationshipSeed` whose two halves populate the
``botA -> botB`` and ``botB -> botA`` edges respectively (summary,
initial knowledge facts, and small affinity/trust deltas around the
default 50/50 baseline).
The two directions can differ — A may know more about B than B knows
about A, or A may trust B less than the reverse — so the schema carries
both halves independently.
Empty/whitespace-only prose short-circuits to a default
``RelationshipSeed`` (all zeroes, empty strings); the caller treats
that as "they haven't met" and writes no edge content. The wrapper uses
:func:`chat.llm.classify.classify` with ``default=RelationshipSeed()``
so a flapping classifier degrades to the same no-op rather than
blocking the chat-creation flow (§3.3 graceful-degradation rule).
T42 (the inter-bot relationship drawer) calls this from the route layer.
"""
from __future__ import annotations
from pydantic import BaseModel, Field
from chat.llm.classify import classify
from chat.llm.client import LLMClient
class RelationshipSeed(BaseModel):
"""Structured per-direction seed for two bots' edges.
Defaults are a deliberate no-op: empty summaries, empty knowledge
lists, zero deltas. Both the empty-prose short-circuit and the
classifier-failure fallback return this default so the caller can
treat them identically.
"""
a_to_b_summary: str = ""
a_to_b_knowledge_facts: list[str] = Field(default_factory=list)
a_to_b_affinity_delta: int = 0 # signed, -10..+10 typical
a_to_b_trust_delta: int = 0
b_to_a_summary: str = ""
b_to_a_knowledge_facts: list[str] = Field(default_factory=list)
b_to_a_affinity_delta: int = 0
b_to_a_trust_delta: int = 0
_SYSTEM = (
"You parse a short prose seed describing how two characters know each "
"other into structured per-direction edge content. For each direction "
"(A -> B, B -> A) extract: summary (one sentence from that POV), "
"knowledge_facts (list of factual claims that direction can carry "
"into future scenes), affinity_delta (-10..+10 — small adjustments to "
"the default 50/50 baseline), trust_delta (-10..+10). Default deltas "
"to 0 when prose is neutral. The two directions can differ — A may "
"trust B more than B trusts A. Output strict JSON matching the schema."
)
async def seed_inter_bot_edges(
client: LLMClient,
*,
classifier_model: str,
bot_a_id: str,
bot_a_name: str,
bot_b_id: str,
bot_b_name: str,
relationship_prose: str,
timeout_s: float = 30.0,
) -> RelationshipSeed:
"""Parse user-supplied prose into structured edge content for both
directed pairs.
Empty/whitespace prose short-circuits to an empty
:class:`RelationshipSeed` (the caller treats this as "they haven't
met" and writes no edge content). Classifier failure also returns
the default — see module docstring for the rationale.
The ``bot_a_id`` / ``bot_b_id`` arguments are accepted for symmetry
with the caller (T42's drawer route uses them when emitting
``edge_update`` events); they're embedded in the prompt alongside
the names so the classifier can disambiguate when names collide.
"""
if not relationship_prose or not relationship_prose.strip():
return RelationshipSeed()
user = (
f"Bot A: {bot_a_name} (id={bot_a_id})\n"
f"Bot B: {bot_b_name} (id={bot_b_id})\n\n"
f"Prose seed:\n{relationship_prose.strip()}"
)
return await classify(
client,
model=classifier_model,
system=_SYSTEM,
user=user,
schema=RelationshipSeed,
default=RelationshipSeed(),
timeout_s=timeout_s,
)
__all__ = ["RelationshipSeed", "seed_inter_bot_edges"]
+345 -37
View File
@@ -29,6 +29,8 @@ keeps moving.
from __future__ import annotations from __future__ import annotations
import json import json
import uuid
from datetime import datetime, timezone
from sqlite3 import Connection from sqlite3 import Connection
from pydantic import BaseModel, Field from pydantic import BaseModel, Field
@@ -156,70 +158,62 @@ def _read_recent_dialogue(
return out return out
async def apply_scene_close_summary( async def _summarize_and_apply_for_witness(
conn: Connection, conn: Connection,
client: LLMClient, client: LLMClient,
*, *,
classifier_model: str, classifier_model: str,
chat_id: str, chat_id: str,
scene_id: int, scene_id: int,
host_bot_id: str, bot_id: str,
timeout_s: float = 10.0, you_name: str,
dialogue: list[dict],
timeout_s: float,
key_quotes_suffix: str = "",
) -> ScenePOVSummary: ) -> ScenePOVSummary:
"""Drive the per-POV summary pipeline after ``scene_closed``. """Run :func:`summarize_scene` for one bot witness and apply the
three projected updates (memory pov_summary rewrite, edge summary
overwrite, edge knowledge_facts append).
Steps (Phase 1, single-bot): Tolerant of missing pieces in the same way Phase 1 was: no memory
1. Gather the closing scene's dialogue from the event_log. row -> skip the rewrite; no edge row -> skip the edge_summary write
2. Run :func:`summarize_scene` for the host bot. (the empty-default classifier output simply yields no rewrites).
3. Rewrite each scene-bound memory's ``pov_summary`` via
``manual_edit`` (target_kind ``memory_pov_summary``), capturing
the prior value for §6.4 reversibility.
4. Update the bot->you edge summary via ``manual_edit`` with the
new ``edge_summary`` target_kind. v1 combines prior + new by
concatenation — the classifier's ``relationship_summary`` is
already phrased as a continuation.
5. Append any new knowledge_facts to the same edge via
``edge_update``.
Tolerant of missing pieces: no memories -> skip step 3 silently; ``key_quotes_suffix`` is appended verbatim to the per-POV summary
no edge row -> skip step 4; empty knowledge_facts -> skip step 5. text before the rewrite lands (T58.1) — empty string is the no-op
The classifier's empty default flows through harmlessly. default for low-significance scenes.
""" """
# Local imports to keep the module-level surface tight and avoid
# any chance of a circular dep through chat.state.*.
from chat.state.edges import get_edge from chat.state.edges import get_edge
from chat.state.entities import get_bot, get_you from chat.state.entities import get_bot
host_bot = get_bot(conn, host_bot_id) or {"name": host_bot_id, "persona": ""} bot = get_bot(conn, bot_id) or {"name": bot_id, "persona": ""}
you_entity = get_you(conn) or {"name": "you", "persona": ""}
dialogue = _read_recent_dialogue(conn, chat_id) edge_b2y = get_edge(conn, bot_id, "you")
edge_b2y = get_edge(conn, host_bot_id, "you")
prior_summary = (edge_b2y or {}).get("summary", "") or "" prior_summary = (edge_b2y or {}).get("summary", "") or ""
pov = await summarize_scene( pov = await summarize_scene(
client, client,
model=classifier_model, model=classifier_model,
bot_name=host_bot.get("name", host_bot_id), bot_name=bot.get("name", bot_id),
bot_persona=host_bot.get("persona", "") or "", bot_persona=bot.get("persona", "") or "",
you_name=you_entity.get("name", "you") or "you", you_name=you_name,
prior_edge_summary=prior_summary, prior_edge_summary=prior_summary,
dialogue=dialogue, dialogue=dialogue,
timeout_s=timeout_s, timeout_s=timeout_s,
) )
# Update memories belonging to the closed scene for the host bot. # Update memories belonging to the closed scene for this witness.
cur = conn.execute( cur = conn.execute(
"SELECT id, pov_summary FROM memories " "SELECT id, pov_summary FROM memories "
"WHERE scene_id = ? AND owner_id = ?", "WHERE scene_id = ? AND owner_id = ?",
(scene_id, host_bot_id), (scene_id, bot_id),
) )
for memory_id, prior_pov in cur.fetchall(): for memory_id, prior_pov in cur.fetchall():
if not pov.summary: if not pov.summary:
# Empty default -> skip the memory rewrite; the seeded # Empty default -> skip the memory rewrite; the seeded
# per-turn pov_summary stays in place. # per-turn pov_summary stays in place.
continue continue
new_value = pov.summary + key_quotes_suffix
append_and_apply( append_and_apply(
conn, conn,
kind="manual_edit", kind="manual_edit",
@@ -227,11 +221,11 @@ async def apply_scene_close_summary(
"target_kind": "memory_pov_summary", "target_kind": "memory_pov_summary",
"target_id": int(memory_id), "target_id": int(memory_id),
"prior_value": prior_pov, "prior_value": prior_pov,
"new_value": pov.summary, "new_value": new_value,
}, },
) )
# Update the bot->you edge summary if we have an edge row and a # Update this bot->you edge summary if we have an edge row and a
# non-empty relationship_summary to merge. # non-empty relationship_summary to merge.
if edge_b2y is not None and pov.relationship_summary: if edge_b2y is not None and pov.relationship_summary:
new_summary = ( new_summary = (
@@ -245,7 +239,7 @@ async def apply_scene_close_summary(
payload={ payload={
"target_kind": "edge_summary", "target_kind": "edge_summary",
"target_id": { "target_id": {
"source_id": host_bot_id, "source_id": bot_id,
"target_id": "you", "target_id": "you",
}, },
"prior_value": prior_summary, "prior_value": prior_summary,
@@ -253,13 +247,13 @@ async def apply_scene_close_summary(
}, },
) )
# Append knowledge_facts to the bot->you edge if present. # Append knowledge_facts to this bot->you edge if present.
if pov.knowledge_facts: if pov.knowledge_facts:
append_and_apply( append_and_apply(
conn, conn,
kind="edge_update", kind="edge_update",
payload={ payload={
"source_id": host_bot_id, "source_id": bot_id,
"target_id": "you", "target_id": "you",
"chat_id": chat_id, "chat_id": chat_id,
"knowledge_facts": list(pov.knowledge_facts), "knowledge_facts": list(pov.knowledge_facts),
@@ -267,3 +261,317 @@ async def apply_scene_close_summary(
) )
return pov return pov
def _build_key_quotes_suffix(conn: Connection, scene_id: int) -> str:
"""If the scene's max-turn-significance is >= 2, build the
"Key quotes:" suffix from the top-3 highest-significance memory rows
(per requirements §11.1). Otherwise return the empty string so the
per-POV summaries collapse fully (low-significance scenes lose all
raw text in favor of the classifier rewrite).
Quote source is each memory's current ``pov_summary`` — the raw
per-turn narrative seeded by T21, since this helper is called BEFORE
the per-POV rewrite. Texts are truncated to 200 chars to bound
memory row growth across many witnesses.
"""
row = conn.execute(
"SELECT MAX(significance) FROM memories WHERE scene_id = ?",
(scene_id,),
).fetchone()
max_sig = (row[0] if row else None) or 0
if max_sig < 2:
return ""
cur = conn.execute(
"SELECT pov_summary FROM memories WHERE scene_id = ? "
"ORDER BY significance DESC, id ASC LIMIT 3",
(scene_id,),
)
quotes = [
(r[0] or "")[:200]
for r in cur.fetchall()
]
if not quotes:
return ""
lines = "\n".join(f'- "{q}"' for q in quotes)
return f"\n\nKey quotes:\n{lines}"
async def apply_scene_close_summary(
conn: Connection,
client: LLMClient,
*,
classifier_model: str,
chat_id: str,
scene_id: int,
host_bot_id: str,
timeout_s: float = 10.0,
) -> ScenePOVSummary:
"""Drive the per-POV summary pipeline after ``scene_closed``.
Phase 1 (single-bot) behavior — the host bot is summarized once and
the result drives memory + edge rewrites — is preserved exactly when
the chat has no guest. T45 extends this to fan out across each
present bot witness when a guest is also in the room:
1. Gather the closing scene's dialogue from the event_log.
2. For each present witness (host + guest if any), run
:func:`summarize_scene` once with that witness's persona and
their own prior ``bot -> you`` edge summary.
3. For each witness independently:
a. Rewrite each scene-bound memory's ``pov_summary`` via
``manual_edit`` (target_kind ``memory_pov_summary``).
b. Update that witness's ``bot -> you`` edge summary via
``manual_edit`` (target_kind ``edge_summary``). v2 combines
prior + classifier ``relationship_summary`` by simple
concatenation.
c. Append any ``knowledge_facts`` to the same edge via
``edge_update``.
4. If a ``group_node`` row exists for this chat, append a
``group_node_updated`` event whose ``summary`` is the naive
per-POV concat ``f"{name}: {summary}\\n\\n..."``. A true
LLM-merged group view is deferred to Phase 2.5; ``dynamic``
is left empty here for v2 (Phase 3 polishes it).
The host's :class:`ScenePOVSummary` is returned to preserve the
Phase 1 callers' contract.
"""
# Local imports to keep the module-level surface tight and avoid
# any chance of a circular dep through chat.state.*.
from chat.services.thread_detection import detect_threads
from chat.state.entities import get_bot, get_you
from chat.state.group_node import get_group_node
from chat.state.threads import list_open_threads
from chat.state.world import get_chat, get_scene
you_entity = get_you(conn) or {"name": "you", "persona": ""}
you_name = you_entity.get("name", "you") or "you"
chat = get_chat(conn, chat_id) or {}
guest_bot_id = chat.get("guest_bot_id")
# T65: detect meanwhile child scenes via the migration-0011
# ``present_set_kind`` column. The mechanism is a single field read
# — meanwhile scenes carry ``"host_guest"``, regular you-scenes
# carry the default ``"you_host"``. We read this once up front so
# both the dialogue source and the post-summary digest emission
# branches can reference it.
closing_scene = get_scene(conn, scene_id) or {}
is_meanwhile = closing_scene.get("present_set_kind") == "host_guest"
dialogue = _read_recent_dialogue(conn, chat_id)
# T58.1: build the "Key quotes:" suffix BEFORE the per-POV rewrites
# land — quote source is the raw seeded pov_summary text on each
# memory row, which the rewrite about to fire would clobber.
key_quotes_suffix = _build_key_quotes_suffix(conn, scene_id)
host_pov = await _summarize_and_apply_for_witness(
conn,
client,
classifier_model=classifier_model,
chat_id=chat_id,
scene_id=scene_id,
bot_id=host_bot_id,
you_name=you_name,
dialogue=dialogue,
timeout_s=timeout_s,
key_quotes_suffix=key_quotes_suffix,
)
guest_pov: ScenePOVSummary | None = None
if guest_bot_id is not None:
guest_pov = await _summarize_and_apply_for_witness(
conn,
client,
classifier_model=classifier_model,
chat_id=chat_id,
scene_id=scene_id,
bot_id=guest_bot_id,
you_name=you_name,
dialogue=dialogue,
timeout_s=timeout_s,
key_quotes_suffix=key_quotes_suffix,
)
# Group node update: T70 runs a third classifier call to merge the
# two per-POV summaries into a coherent group-level view + a brief
# group-dynamic note. Falls back to the Phase 2 naive concat on
# classifier failure (see :func:`merge_group_summary`). Only fires
# when both POVs ran (i.e. the guest is present) and a group_node
# row exists for this chat.
if guest_pov is not None and get_group_node(conn, chat_id) is not None:
host_bot = get_bot(conn, host_bot_id) or {"name": host_bot_id}
guest_bot = get_bot(conn, guest_bot_id) or {"name": guest_bot_id}
host_name = host_bot.get("name", host_bot_id) or host_bot_id
guest_name = guest_bot.get("name", guest_bot_id) or guest_bot_id
merged = await merge_group_summary(
client,
classifier_model=classifier_model,
host_name=host_name,
host_pov_summary=host_pov.summary,
guest_name=guest_name,
guest_pov_summary=guest_pov.summary,
timeout_s=timeout_s,
)
append_and_apply(
conn,
kind="group_node_updated",
payload={
"chat_id": chat_id,
"summary": merged.summary,
"dynamic": merged.dynamic,
},
)
# T65: when the closing scene was a meanwhile child (host_guest
# present set), generate an omniscient briefing for the absent
# "you" and queue it as a pending digest. We reuse summarize_scene
# with a narrator persona so the digest text is shaped by the same
# classifier — only the ``summary`` field is consumed downstream.
# Emitted AFTER per-POV summaries land so witness memories carry
# their own POV text first; this mirrors how group_node_updated is
# ordered relative to the per-POV writes above.
if is_meanwhile:
digest_pov = await summarize_scene(
client,
model=classifier_model,
bot_name="Narrator",
bot_persona=_MEANWHILE_DIGEST_PERSONA,
you_name=you_name,
prior_edge_summary="",
dialogue=dialogue,
timeout_s=timeout_s,
)
if digest_pov.summary:
append_and_apply(
conn,
kind="meanwhile_digest_created",
payload={
"chat_id": chat_id,
"scene_id": scene_id,
"summary": digest_pov.summary,
},
)
# T58.2: thread detection on close. Reuses the dialogue we already
# gathered for per-POV summarization — same {speaker, text} shape
# detect_threads expects. Failure-tolerant: classify() returns the
# empty default on retry-exhaustion, and the broad except below
# protects the close pipeline from any other classifier/mock flap.
try:
thread_result = await detect_threads(
client,
classifier_model=classifier_model,
scene_transcript=dialogue,
open_threads=list_open_threads(conn, chat_id),
timeout_s=timeout_s,
)
except Exception:
from chat.services.thread_detection import ThreadDetectionResult
thread_result = ThreadDetectionResult()
for cand in thread_result.candidates:
if cand.action == "open":
new_thread_id = f"thr_{uuid.uuid4().hex[:12]}"
append_and_apply(
conn,
kind="thread_opened",
payload={
"thread_id": new_thread_id,
"chat_id": chat_id,
"title": cand.title,
"summary": cand.summary,
},
)
elif cand.action == "update" and cand.existing_thread_id:
append_and_apply(
conn,
kind="thread_updated",
payload={
"thread_id": cand.existing_thread_id,
"summary": cand.summary,
"last_referenced_scene_id": scene_id,
},
)
elif cand.action == "close" and cand.existing_thread_id:
append_and_apply(
conn,
kind="thread_closed",
payload={
"thread_id": cand.existing_thread_id,
"closed_at": datetime.now(timezone.utc).isoformat(),
},
)
return host_pov
class GroupMetaSummary(BaseModel):
"""Classifier output: a merged group-level view of a closed scene.
Defaults are an empty no-op so callers can use the schema's default
as a sentinel; in practice :func:`merge_group_summary` builds an
explicit naive-concat fallback rather than returning these defaults
directly so existing Phase 2 behavior is preserved on classifier
failure.
"""
summary: str = ""
dynamic: str = ""
_GROUP_MERGE_SYSTEM = (
"Given two per-POV scene summaries from a 3-entity scene (you + "
"host + guest), produce a coherent group-level summary capturing "
"the shared events as both witnesses experienced them, plus a "
"brief 'dynamic' note describing the trio's group dynamic during "
"the scene. Output strict JSON matching schema."
)
# T65: meanwhile-scene digest. The "you" player was absent during this
# scene; the digest is a short neutral briefing they'll read on the next
# you-scene resume. Reuses the ScenePOVSummary schema so the same
# `summarize_scene` helper can be called with a different persona — only
# the ``summary`` field is used downstream.
_MEANWHILE_DIGEST_PERSONA = (
"an omniscient narrator briefing the absent player in 2-3 neutral "
"sentences on what happened while they were away — no editorializing, "
"no second-person address"
)
async def merge_group_summary(
client: LLMClient,
*,
classifier_model: str,
host_name: str,
host_pov_summary: str,
guest_name: str,
guest_pov_summary: str,
timeout_s: float = 30.0,
) -> GroupMetaSummary:
"""Merge two per-POV scene summaries into a coherent group-level
summary + group-dynamic note. Falls back to the naive concat (the
existing behavior) on classifier failure."""
user = (
f"{host_name} (host) POV summary:\n{host_pov_summary}\n\n"
f"{guest_name} (guest) POV summary:\n{guest_pov_summary}"
)
fallback = GroupMetaSummary(
summary=(
f"{host_name}: {host_pov_summary}\n\n"
f"{guest_name}: {guest_pov_summary}"
),
dynamic="",
)
return await classify(
client,
model=classifier_model,
system=_GROUP_MERGE_SYSTEM,
user=user,
schema=GroupMetaSummary,
default=fallback,
timeout_s=timeout_s,
)
+132
View File
@@ -0,0 +1,132 @@
"""Skip narration service (T53).
Generates brief transition prose for elision and jump skips.
Skips come in two flavors that read very differently:
* **Elision** — collapses an in-progress activity into its expected
end-state in 1-2 sentences, narrated from the speaker bot's POV.
Example: "skip ahead to when we arrive" while the characters are
driving — output describes pulling into the lot.
* **Jump** — bridges a longer fiction-time delta ("next morning", "a
week later") in 2-3 sentences, setting the scene at the new time.
Output is free-form prose, not structured JSON, so this service calls
``client.generate`` directly rather than going through the classifier
path used by, e.g., :mod:`chat.services.scene_summarize`. A
deterministic template fallback fires on any LLM failure so the skip
flow keeps moving even when the model is down — important because
skips are a UI-blocking operation; we'd rather show a parenthetical
sentence than hang the chat indefinitely.
"""
from __future__ import annotations
from chat.llm.client import LLMClient, Message
_ELISION_SYSTEM = (
"You write a brief 1-2 sentence transition that elides the time "
"between an in-progress activity and its expected end-state, "
"narrated from the speaker's POV. Keep it grounded and concrete. "
"Do not invent new events or characters."
)
_JUMP_SYSTEM = (
"You write a brief 2-3 sentence transition narrating a jump in "
"fiction time (e.g., 'next morning', 'a week later'), narrated "
"from the speaker's POV. Set the scene at the new time. Keep it "
"grounded — no invented major events. If a landing-state hint is "
"provided, weave it in naturally."
)
async def narrate_skip(
client: LLMClient,
*,
narrative_model: str,
skip_kind: str,
speaker_bot: dict,
you_name: str,
current_time: str,
new_time: str,
current_activity: str,
landing_state_hint: str = "",
timeout_s: float = 60.0,
) -> str:
"""Generate brief transition prose for a time skip.
``skip_kind`` is ``"elision"`` or ``"jump"``; any other value short-
circuits to the deterministic fallback (defensive — callers
shouldn't be inventing new kinds without updating this service).
Returns plain text. Never raises: any LLM error, an empty/blank
result, or an unknown ``skip_kind`` falls back to a parenthetical
template like ``"(next morning: having coffee in the kitchen.)"``
so the skip UI always has *something* to render.
"""
fallback = _build_fallback(
skip_kind=skip_kind,
new_time=new_time,
current_activity=current_activity,
landing_state_hint=landing_state_hint,
)
if skip_kind not in ("elision", "jump"):
return fallback
system = _ELISION_SYSTEM if skip_kind == "elision" else _JUMP_SYSTEM
user = (
f"Speaker: {speaker_bot.get('name', 'speaker')}\n"
f"Persona: {speaker_bot.get('persona', '')}\n"
f"Other party: {you_name}\n"
f"Current time: {current_time}\n"
f"New time: {new_time}\n"
f"Current activity: {current_activity}\n"
)
if landing_state_hint:
user += f"Landing state hint: {landing_state_hint}\n"
try:
result = await client.generate(
[
Message(role="system", content=system),
Message(role="user", content=user),
],
model=narrative_model,
max_tokens=200,
temperature=0.7,
timeout_s=timeout_s,
)
text = (result or "").strip()
if not text:
return fallback
return text
except Exception:
# Any failure — network blip, timeout, mock raising in tests —
# collapses to the deterministic template so the skip pipeline
# is never blocked on the LLM being available.
return fallback
def _build_fallback(
*,
skip_kind: str,
new_time: str,
current_activity: str,
landing_state_hint: str,
) -> str:
"""Deterministic parenthetical narration used when the LLM fails.
Both flavors render the same shape today: ``(<new_time>:
<detail>.)``. They're separated as branches to make it easy to
diverge later (e.g. an elision-specific template) without churning
the call site or the public signature.
"""
detail = landing_state_hint or current_activity or "moments later"
if skip_kind == "elision":
return f"({new_time}: {detail}.)"
return f"({new_time}: {detail}.)"
__all__ = ["narrate_skip"]
+74
View File
@@ -0,0 +1,74 @@
"""Synthesized-memories service (T54).
When the user jump-skips with 'anything notable happen?' prose, parse
that prose into 1-N synthesized memories per present bot. Each memory
carries source="synthesized" and reliability=0.7 (lower than direct).
Caller (T62 skip flow) writes the memories via record_turn_memory_for_present.
"""
from __future__ import annotations
from pydantic import BaseModel, Field
from chat.llm.classify import classify
from chat.llm.client import LLMClient
class SynthesizedMemory(BaseModel):
text: str
significance: int = 1 # 0..3, default 1
affinity_delta: int = 0
trust_delta: int = 0
class SynthesizedDigest(BaseModel):
memories: list[SynthesizedMemory] = Field(default_factory=list)
_SYSTEM = (
"You parse a short user-supplied prose describing 'anything notable' "
"that happened during a time skip into 1-N synthesized memories from "
"a single bot's POV. Each memory has: text (one factual sentence "
"from that bot's perspective), significance (0-3, default 1; only "
"use 2 or 3 for genuinely scene-level or relationship-altering "
"events), affinity_delta and trust_delta (-10..+10, default 0; "
"use small adjustments only when prose explicitly describes a shift). "
"Empty/whitespace prose returns an empty memories list. Output "
"strict JSON matching the schema."
)
async def synthesize_memories(
client: LLMClient,
*,
classifier_model: str,
prose: str,
bot_name: str,
bot_persona: str,
you_name: str,
timeout_s: float = 30.0,
) -> SynthesizedDigest:
"""Parse 'anything notable' prose into structured memories from a
single bot's POV. Empty/whitespace prose short-circuits to an
empty digest (no LLM call)."""
if not prose or not prose.strip():
return SynthesizedDigest()
user = (
f"Bot: {bot_name}\n"
f"Persona: {bot_persona}\n"
f"Other party: {you_name}\n\n"
f"Prose:\n{prose.strip()}"
)
return await classify(
client,
model=classifier_model,
system=_SYSTEM,
user=user,
schema=SynthesizedDigest,
default=SynthesizedDigest(),
timeout_s=timeout_s,
)
__all__ = ["SynthesizedMemory", "SynthesizedDigest", "synthesize_memories"]
+89
View File
@@ -0,0 +1,89 @@
"""Thread-detection service (T55).
On scene close, classify the transcript into thread open/update/close
candidates. Returns ThreadCandidate list; caller (T58 scene compression)
emits one thread_opened/thread_updated/thread_closed event per candidate.
"""
from __future__ import annotations
from pydantic import BaseModel, Field
from chat.llm.classify import classify
from chat.llm.client import LLMClient
class ThreadCandidate(BaseModel):
action: str # "open" | "update" | "close"
title: str = "" # required for "open"; ignored otherwise
summary: str = ""
existing_thread_id: str | None = None # required for "update" / "close"
class ThreadDetectionResult(BaseModel):
candidates: list[ThreadCandidate] = Field(default_factory=list)
_SYSTEM = (
"You analyze a closed scene's transcript to identify narrative "
"threads (unresolved arcs, dangling questions, promises made, "
"open obligations). Choose actions:\n"
"- 'open': a NEW thread the scene introduced. Provide title (short "
"noun phrase) + summary (one sentence).\n"
"- 'update': an EXISTING open thread that the scene developed. "
"Provide existing_thread_id + new summary.\n"
"- 'close': an EXISTING open thread that the scene resolved. "
"Provide existing_thread_id; summary may capture the resolution.\n"
"Conservative bias: most scenes do NOT open new threads. Only "
"produce candidates when the transcript clearly justifies them. "
"Output strict JSON matching the schema."
)
async def detect_threads(
client: LLMClient,
*,
classifier_model: str,
scene_transcript: list[dict], # [{speaker, text}, ...]
open_threads: list[dict], # [{thread_id, title, summary}, ...]
timeout_s: float = 30.0,
) -> ThreadDetectionResult:
"""Classify scene close into thread open/update/close candidates."""
if not scene_transcript:
return ThreadDetectionResult()
transcript_lines = [
f"{turn.get('speaker', 'unknown')}: {turn.get('text', '')}"
for turn in scene_transcript
]
threads_lines = []
if open_threads:
threads_lines.append("Currently open threads:")
for t in open_threads:
threads_lines.append(
f"- thread_id={t['thread_id']} "
f"title={t.get('title', '')} "
f"summary={t.get('summary', '')}"
)
else:
threads_lines.append("No currently open threads.")
user = (
"Scene transcript:\n"
+ "\n".join(transcript_lines)
+ "\n\n"
+ "\n".join(threads_lines)
)
return await classify(
client,
model=classifier_model,
system=_SYSTEM,
user=user,
schema=ThreadDetectionResult,
default=ThreadDetectionResult(),
timeout_s=timeout_s,
)
__all__ = ["ThreadCandidate", "ThreadDetectionResult", "detect_threads"]
+39 -8
View File
@@ -16,6 +16,16 @@ nested quotes, mixed punctuation), so v1 delegates the segmentation to
the classifier. The configurable ``Settings.ooc_marker`` is *not* read the classifier. The configurable ``Settings.ooc_marker`` is *not* read
here: the classifier figures OOC out from ``((`` ``))`` regardless of here: the classifier figures OOC out from ``((`` ``))`` regardless of
config-time choice; marker-based stripping is a downstream concern. config-time choice; marker-based stripping is a downstream concern.
T62 extends the parser with an ``intent`` field so the turn flow can
short-circuit time-skip phrases before the regular narrative path.
``intent`` defaults to ``"narrative"``; the classifier may set it to
``"skip_elision"`` when prose like "skip to when we arrive" or
``"skip_jump"`` when prose like "next morning" / "a week later" is
detected. ``landing_state_hint`` carries the residual descriptor for
elision skips (the "to when we ..." phrase). Existing callers that
don't read ``intent`` continue to work because the default keeps the
narrative path intact.
""" """
from __future__ import annotations from __future__ import annotations
@@ -39,9 +49,19 @@ class TurnSegment(BaseModel):
class ParsedTurn(BaseModel): class ParsedTurn(BaseModel):
"""A turn split into ordered, typed segments.""" """A turn split into ordered, typed segments.
``intent`` distinguishes a regular narrative beat (the default) from
a natural-language time-skip command (T62). ``landing_state_hint``
captures the descriptor following "skip to when we ..." for elision
skips so the downstream skip controller can pass it to the
narration helper. Both fields are optional and default-empty so
older fixtures and tests that don't supply them keep working.
"""
segments: list[TurnSegment] segments: list[TurnSegment]
intent: str = "narrative" # "narrative" | "skip_elision" | "skip_jump"
landing_state_hint: str = ""
_SYSTEM_PROMPT = ( _SYSTEM_PROMPT = (
@@ -52,13 +72,24 @@ _SYSTEM_PROMPT = (
"- ((text in double parens)) is an OOC (out-of-character) segment — " "- ((text in double parens)) is an OOC (out-of-character) segment — "
"the author talking to the system, not the in-fiction bot.\n\n" "the author talking to the system, not the in-fiction bot.\n\n"
"Output a JSON object with shape " "Output a JSON object with shape "
'{"segments": [{"kind": "...", "text": "..."}, ...]} ' '{"segments": [{"kind": "...", "text": "..."}, ...], '
"where each ``kind`` is exactly one of: dialogue, action, ooc. " '"intent": "...", "landing_state_hint": "..."} '
"Preserve the original substring text as ``text``: do not rewrite, " "where each segment ``kind`` is exactly one of: dialogue, action, "
"translate, or normalize punctuation — strip only the marker " "ooc. Preserve the original substring text as ``text``: do not "
"characters (asterisks, surrounding quotes, double parens) so " "rewrite, translate, or normalize punctuation — strip only the "
"``text`` is the inner content. Emit segments in the order they " "marker characters (asterisks, surrounding quotes, double parens) "
"appear in the input." "so ``text`` is the inner content. Emit segments in the order they "
"appear in the input.\n\n"
"``intent`` is exactly one of: narrative, skip_elision, skip_jump. "
"Default to ``narrative``. Use ``skip_elision`` when the prose is a "
"directive to fast-forward an in-progress activity to a near-term "
"landing state — e.g. 'skip to when we arrive', 'fast-forward to "
"after dinner'. Use ``skip_jump`` when the prose denotes a longer "
"fiction-time bridge — e.g. 'next morning', 'a week later', 'the "
"following day'.\n"
"``landing_state_hint`` is a short descriptor of the landing state "
"for ``skip_elision`` (e.g. 'we arrive at the park'). Empty string "
"for ``skip_jump`` and ``narrative``."
) )
+18 -2
View File
@@ -48,6 +48,17 @@ def _apply_bot_reset(conn: Connection, e: Event) -> None:
"SELECT id FROM chats WHERE host_bot_id = ?", (bot_id,) "SELECT id FROM chats WHERE host_bot_id = ?", (bot_id,)
).fetchall() ).fetchall()
] ]
# T69: purge orphaned "you" activity rows pointing at containers in this
# bot's chats BEFORE the containers/chats themselves are deleted, otherwise
# the subqueries find nothing and the FK constraint on activity.container_id
# blocks the container delete.
conn.execute(
"DELETE FROM activity WHERE entity_id = 'you' "
"AND container_id IN (SELECT id FROM containers WHERE chat_id IN ("
" SELECT id FROM chats WHERE host_bot_id = ?"
"))",
(bot_id,),
)
for chat_id in chat_ids: for chat_id in chat_ids:
conn.execute("DELETE FROM scenes WHERE chat_id = ?", (chat_id,)) conn.execute("DELETE FROM scenes WHERE chat_id = ?", (chat_id,))
conn.execute("DELETE FROM containers WHERE chat_id = ?", (chat_id,)) conn.execute("DELETE FROM containers WHERE chat_id = ?", (chat_id,))
@@ -66,9 +77,14 @@ def _apply_bot_reset(conn: Connection, e: Event) -> None:
"DELETE FROM edges WHERE source_id = ? OR target_id = ?", "DELETE FROM edges WHERE source_id = ? OR target_id = ?",
(bot_id, bot_id), (bot_id, bot_id),
) )
# Phase 2 cascade: clear guest references in other bots' chats so the host
# doesn't see a stale guest_bot_id pointing at this (now-purged) bot.
conn.execute(
"UPDATE chats SET guest_bot_id = NULL WHERE guest_bot_id = ?",
(bot_id,),
)
# NOTE: bots row itself is preserved (identity, kickoff_prose intact). # NOTE: bots row itself is preserved (identity, kickoff_prose intact).
# NOTE: "you" activity (entity_id="you") may linger from a deleted chat;
# acceptable for v1 — Phase 1.5 cleanup if needed.
def get_bot(conn: Connection, bot_id: str) -> dict | None: def get_bot(conn: Connection, bot_id: str) -> dict | None:
+127
View File
@@ -0,0 +1,127 @@
from __future__ import annotations
import json
from sqlite3 import Connection
from chat.eventlog.projector import on
from chat.eventlog.log import Event
_TERMINAL_STATUSES = {"completed", "cancelled", "expired"}
@on("event_planned")
def _apply_event_planned(conn: Connection, e: Event) -> None:
p = e.payload
conn.execute(
"INSERT OR IGNORE INTO events "
"(event_id, chat_id, kind, status, props_json, planned_for) "
"VALUES (?, ?, ?, 'planned', ?, ?)",
(
p["event_id"],
p["chat_id"],
p["kind"],
json.dumps(p.get("props", {})),
p.get("planned_for"),
),
)
@on("event_started")
def _apply_event_started(conn: Connection, e: Event) -> None:
p = e.payload
# Idempotent: only transition from non-terminal status.
conn.execute(
"UPDATE events SET status = 'active', started_at = ?, updated_at = datetime('now') "
"WHERE event_id = ? AND status NOT IN ('completed','cancelled','expired')",
(p.get("started_at"), p["event_id"]),
)
@on("event_completed")
def _apply_event_completed(conn: Connection, e: Event) -> None:
p = e.payload
conn.execute(
"UPDATE events SET status = 'completed', completed_at = ?, updated_at = datetime('now') "
"WHERE event_id = ? AND status NOT IN ('completed','cancelled','expired')",
(p.get("completed_at"), p["event_id"]),
)
@on("event_cancelled")
def _apply_event_cancelled(conn: Connection, e: Event) -> None:
p = e.payload
conn.execute(
"UPDATE events SET status = 'cancelled', completed_at = ?, updated_at = datetime('now') "
"WHERE event_id = ? AND status NOT IN ('completed','cancelled','expired')",
(p.get("completed_at"), p["event_id"]),
)
@on("event_expired")
def _apply_event_expired(conn: Connection, e: Event) -> None:
p = e.payload
conn.execute(
"UPDATE events SET status = 'expired', completed_at = ?, updated_at = datetime('now') "
"WHERE event_id = ? AND status NOT IN ('completed','cancelled','expired')",
(p.get("completed_at"), p["event_id"]),
)
def get_event(conn: Connection, event_id: str) -> dict | None:
row = conn.execute(
"SELECT event_id, chat_id, kind, status, props_json, planned_for, "
"started_at, completed_at, created_at, updated_at "
"FROM events WHERE event_id = ?",
(event_id,),
).fetchone()
if not row:
return None
return {
"event_id": row[0],
"chat_id": row[1],
"kind": row[2],
"status": row[3],
"props": json.loads(row[4]),
"planned_for": row[5],
"started_at": row[6],
"completed_at": row[7],
"created_at": row[8],
"updated_at": row[9],
}
def list_active_events(conn: Connection, chat_id: str) -> list[dict]:
rows = conn.execute(
"SELECT event_id, chat_id, kind, status, props_json, planned_for, "
"started_at, completed_at, created_at, updated_at "
"FROM events WHERE chat_id = ? AND status IN ('planned','active') "
"ORDER BY id ASC",
(chat_id,),
).fetchall()
return [
{
"event_id": r[0], "chat_id": r[1], "kind": r[2], "status": r[3],
"props": json.loads(r[4]),
"planned_for": r[5], "started_at": r[6], "completed_at": r[7],
"created_at": r[8], "updated_at": r[9],
}
for r in rows
]
def list_events_in_status(conn: Connection, chat_id: str, status: str) -> list[dict]:
rows = conn.execute(
"SELECT event_id, chat_id, kind, status, props_json, planned_for, "
"started_at, completed_at, created_at, updated_at "
"FROM events WHERE chat_id = ? AND status = ? ORDER BY id ASC",
(chat_id, status),
).fetchall()
return [
{
"event_id": r[0], "chat_id": r[1], "kind": r[2], "status": r[3],
"props": json.loads(r[4]),
"planned_for": r[5], "started_at": r[6], "completed_at": r[7],
"created_at": r[8], "updated_at": r[9],
}
for r in rows
]
+50
View File
@@ -0,0 +1,50 @@
from __future__ import annotations
import json
from sqlite3 import Connection
from chat.eventlog.projector import on
from chat.eventlog.log import Event
@on("group_node_initialized")
def _apply_group_node_initialized(conn: Connection, e: Event) -> None:
p = e.payload
conn.execute(
"INSERT OR REPLACE INTO group_node "
"(chat_id, members_json, summary, dynamic, threads_json) "
"VALUES (?, ?, ?, ?, ?)",
(
p["chat_id"],
json.dumps(p["members"]),
p.get("summary", ""),
p.get("dynamic", ""),
json.dumps(p.get("threads", [])),
),
)
@on("group_node_updated")
def _apply_group_node_updated(conn: Connection, e: Event) -> None:
p = e.payload
conn.execute(
"UPDATE group_node SET summary = ?, dynamic = ?, updated_at = datetime('now') "
"WHERE chat_id = ?",
(p.get("summary", ""), p.get("dynamic", ""), p["chat_id"]),
)
def get_group_node(conn: Connection, chat_id: str) -> dict | None:
row = conn.execute(
"SELECT chat_id, members_json, summary, dynamic, threads_json, updated_at "
"FROM group_node WHERE chat_id = ?",
(chat_id,),
).fetchone()
if not row:
return None
return {
"chat_id": row[0],
"members": json.loads(row[1]),
"summary": row[2],
"dynamic": row[3],
"threads": json.loads(row[4]),
"updated_at": row[5],
}
+55 -4
View File
@@ -6,7 +6,7 @@ be reversed by emitting an inverse ``manual_edit`` later. This module
applies the new value to the appropriate target table; the snapshot of applies the new value to the appropriate target table; the snapshot of
``prior_value`` is taken by the route handler before this fires. ``prior_value`` is taken by the route handler before this fires.
Phase 1 covers four target kinds: Phase 1 covers five target kinds:
- ``edge_affinity`` and ``edge_trust`` — slider edits on a specific edge, - ``edge_affinity`` and ``edge_trust`` — slider edits on a specific edge,
clamped to 0..100. clamped to 0..100.
- ``memory_significance`` — dropdown edit, clamped to 0..3. - ``memory_significance`` — dropdown edit, clamped to 0..3.
@@ -17,8 +17,18 @@ Phase 1 covers four target kinds:
field. Driven by T27 from the classifier's ``relationship_summary`` field. Driven by T27 from the classifier's ``relationship_summary``
output combined with the prior summary. output combined with the prior summary.
Other §6.4 editable fields (activity verb / attention / posture, T72.1 (Phase 2.5) adds one list-shaped edit:
knowledge_facts list manipulation) are deferred to Phase 1.5. - ``edge_knowledge_fact`` — add/remove a single fact on an edge's
``knowledge_json`` list. Payload carries an ``action`` of ``"add"`` or
``"remove"`` and a ``fact`` string; remove matches the first occurrence
by string equality so the route handler doesn't have to track fact
indices across re-renders.
T72.3 adds a per-flag witness toggle:
- ``memory_witness`` — flip one of ``witness_you`` / ``witness_host`` /
``witness_guest`` on a memory row. Payload's ``new_value`` is a dict
``{"flag": "you"|"host"|"guest", "value": 0|1}`` and ``prior_value``
mirrors the same shape so an inverse edit can restore the flag.
Pin toggles intentionally use the existing ``memory_pin_changed`` event Pin toggles intentionally use the existing ``memory_pin_changed`` event
(registered in :mod:`chat.state.memory`) rather than ``manual_edit`` so (registered in :mod:`chat.state.memory`) rather than ``manual_edit`` so
@@ -27,11 +37,14 @@ the projection writes both ``pinned`` and ``auto_pinned`` atomically.
from __future__ import annotations from __future__ import annotations
import json
from sqlite3 import Connection from sqlite3 import Connection
from chat.eventlog.log import Event from chat.eventlog.log import Event
from chat.eventlog.projector import on from chat.eventlog.projector import on
_VALID_WITNESS_FLAGS = {"you", "host", "guest"}
def _clamp(value: int, lo: int, hi: int) -> int: def _clamp(value: int, lo: int, hi: int) -> int:
return max(lo, min(hi, value)) return max(lo, min(hi, value))
@@ -87,5 +100,43 @@ def _apply_manual_edit(conn: Connection, e: Event) -> None:
target_id["target_id"], target_id["target_id"],
), ),
) )
elif kind == "edge_knowledge_fact":
# T72.1: add or remove a single fact on an edge's knowledge list.
# ``target_id`` is the {"source_id", "target_id"} edge pair;
# ``new_value`` carries ``{"action": "add"|"remove", "fact": str}``.
# Remove matches by string equality (first occurrence) so callers
# don't have to thread a fact_index through re-rendered drawers.
action = new_value["action"]
fact = str(new_value["fact"])
row = conn.execute(
"SELECT knowledge_json FROM edges "
"WHERE source_id = ? AND target_id = ?",
(target_id["source_id"], target_id["target_id"]),
).fetchone()
if row is not None:
knowledge = json.loads(row[0])
if action == "add":
knowledge.append(fact)
elif action == "remove" and fact in knowledge:
knowledge.remove(fact)
conn.execute(
"UPDATE edges SET knowledge_json = ? "
"WHERE source_id = ? AND target_id = ?",
(
json.dumps(knowledge),
target_id["source_id"],
target_id["target_id"],
),
)
elif kind == "memory_witness":
# T72.3: toggle one of the three witness flags on a memory row.
# ``new_value`` is the dict ``{"flag", "value"}``; ``prior_value``
# mirrors the same shape so an inverse edit restores the flag.
flag = new_value["flag"]
if flag in _VALID_WITNESS_FLAGS:
conn.execute(
f"UPDATE memories SET witness_{flag} = ? WHERE id = ?",
(1 if int(new_value["value"]) else 0, int(target_id)),
)
# Unknown target_kind: silently no-op for v1. Future kinds (activity # Unknown target_kind: silently no-op for v1. Future kinds (activity
# fields, knowledge_facts list manipulation) extend the dispatch above. # fields, etc.) extend the dispatch above.
+164
View File
@@ -0,0 +1,164 @@
"""Meanwhile-scene projection (T63).
A "meanwhile" scene is a 2-bot scene where ``present_set = {host_bot_id,
guest_bot_id}`` and "you" is absent. It runs alongside an active you-scene
(its parent) so bots can have private interactions whose outcome later
surfaces back to the you-scene as a pending digest.
The underlying ``scenes`` table (migration 0007) has no explicit ``status``
column; "active" is encoded as ``ended_at IS NULL`` and "closed" as
``ended_at IS NOT NULL``. This module preserves that convention and adds
two new columns introduced by migration 0011:
- ``present_set_kind`` — ``'you_host'`` (default) for normal scenes,
``'host_guest'`` for meanwhile child scenes.
- ``parent_scene_id`` — the you-scene a meanwhile child hangs off of.
Pending meanwhile digests live in their own table
(``meanwhile_digest_pending``) and are consumed when their summary is
surfaced in the next you-scene's prompt.
"""
from __future__ import annotations
import json
from sqlite3 import Connection
from chat.eventlog.projector import on
from chat.eventlog.log import Event
@on("meanwhile_scene_started")
def _apply_meanwhile_scene_started(conn: Connection, e: Event) -> None:
"""Insert a new scenes row for the meanwhile child.
The caller supplies an explicit ``scene_id`` so subsequent events
(close, digest) can reference it without round-tripping through
``lastrowid``.
"""
p = e.payload
conn.execute(
"INSERT INTO scenes ("
"id, chat_id, started_at, ended_at, significance, "
"participants_json, present_set_kind, parent_scene_id"
") VALUES (?, ?, ?, NULL, 0, ?, 'host_guest', ?)",
(
p["scene_id"],
p["chat_id"],
p.get("started_at"),
# participants_json mirrors the present_set: host + guest bot.
# json.dumps ensures bot ids with quotes/backslashes can't corrupt the JSON literal.
json.dumps([p["host_bot_id"], p["guest_bot_id"]]),
p["parent_scene_id"],
),
)
@on("meanwhile_scene_closed")
def _apply_meanwhile_scene_closed(conn: Connection, e: Event) -> None:
"""Mark the meanwhile scene closed by stamping ``ended_at``."""
p = e.payload
conn.execute(
"UPDATE scenes SET ended_at = ? "
"WHERE id = ? AND present_set_kind = 'host_guest' "
"AND ended_at IS NULL",
(p.get("closed_at"), p["scene_id"]),
)
@on("meanwhile_digest_created")
def _apply_meanwhile_digest_created(conn: Connection, e: Event) -> None:
"""Queue a digest for surfacing to the next you-scene's prompt."""
p = e.payload
conn.execute(
"INSERT INTO meanwhile_digest_pending (scene_id, chat_id, summary) "
"VALUES (?, ?, ?)",
(p["scene_id"], p["chat_id"], p["summary"]),
)
@on("meanwhile_digest_consumed")
def _apply_meanwhile_digest_consumed(conn: Connection, e: Event) -> None:
"""Mark a pending digest as consumed (idempotent on re-projection)."""
p = e.payload
conn.execute(
"UPDATE meanwhile_digest_pending SET consumed_at = ? "
"WHERE id = ? AND consumed_at IS NULL",
(p.get("consumed_at"), p["digest_id"]),
)
def _scene_row_to_dict(row: tuple) -> dict:
"""Shape a meanwhile-scene row.
``status`` is derived from ``ended_at`` for callers that prefer the
higher-level vocabulary; ``closed_at`` aliases ``ended_at`` for the
same reason. The underlying column remains ``ended_at``.
"""
ended_at = row[5]
return {
"id": row[0],
"chat_id": row[1],
"started_at": row[2],
"present_set_kind": row[3],
"parent_scene_id": row[4],
"closed_at": ended_at,
"status": "closed" if ended_at is not None else "active",
}
def list_meanwhile_scenes(
conn: Connection, chat_id: str, status: str = "active"
) -> list[dict]:
"""Return meanwhile scenes for ``chat_id`` filtered by derived status."""
if status == "active":
ended_clause = "s.ended_at IS NULL"
elif status == "closed":
ended_clause = "s.ended_at IS NOT NULL"
else:
raise ValueError(f"unknown status: {status!r}")
rows = conn.execute(
"SELECT s.id, s.chat_id, s.started_at, s.present_set_kind, "
"s.parent_scene_id, s.ended_at "
"FROM scenes s "
"WHERE s.chat_id = ? AND s.present_set_kind = 'host_guest' "
f"AND {ended_clause} "
"ORDER BY s.id ASC",
(chat_id,),
).fetchall()
return [_scene_row_to_dict(r) for r in rows]
def get_parent_scene(conn: Connection, scene_id: int) -> dict | None:
"""Given a meanwhile scene id, return its parent (you-scene) row."""
row = conn.execute(
"SELECT s.id, s.chat_id, s.started_at, s.present_set_kind, "
"s.parent_scene_id, s.ended_at "
"FROM scenes s JOIN scenes m ON m.parent_scene_id = s.id "
"WHERE m.id = ?",
(scene_id,),
).fetchone()
if row is None:
return None
return _scene_row_to_dict(row)
def list_pending_meanwhile_digests(
conn: Connection, chat_id: str
) -> list[dict]:
"""Return digests for ``chat_id`` that haven't been consumed yet."""
rows = conn.execute(
"SELECT id, scene_id, chat_id, summary, created_at "
"FROM meanwhile_digest_pending "
"WHERE chat_id = ? AND consumed_at IS NULL "
"ORDER BY id ASC",
(chat_id,),
).fetchall()
return [
{
"id": r[0],
"scene_id": r[1],
"chat_id": r[2],
"summary": r[3],
"created_at": r[4],
}
for r in rows
]
+25 -2
View File
@@ -94,6 +94,14 @@ def get_pinned(conn: Connection, owner_id: str) -> list[dict]:
_SIGNIFICANCE_WEIGHT = 0.3 _SIGNIFICANCE_WEIGHT = 0.3
_RECENCY_WEIGHT = 0.5 _RECENCY_WEIGHT = 0.5
# T57 (Phase 3, §11.1): significance multiplier applied to the SQL ORDER BY in
# ``search_memories`` so that the FTS over-fetch already prefers
# higher-significance rows for tied / near-tied BM25 ranks. Module-level so it
# can be tuned without a code change. BM25 ``rank`` is lower-is-better, so the
# bias is *subtracted* from rank in the ASC ordering — equivalent to multiplying
# a higher-is-better score by a positive constant per the spec wording.
SIGNIFICANCE_RANK_BIAS = 0.5
def search_memories( def search_memories(
conn: Connection, conn: Connection,
@@ -117,6 +125,16 @@ def search_memories(
so that stronger candidates yield smaller composite scores; the result is so that stronger candidates yield smaller composite scores; the result is
sorted ascending and truncated to ``k``. The unmodified ``fts_rank`` and a sorted ascending and truncated to ``k``. The unmodified ``fts_rank`` and a
debug-friendly ``composite_score`` are kept on each returned dict. debug-friendly ``composite_score`` are kept on each returned dict.
The result ordering applies TWO independent significance boosts:
* **SQL-side** — ``ORDER BY (rank - significance * SIGNIFICANCE_RANK_BIAS)``
pushes higher-significance memories ahead in the FTS5 candidate set so
the over-fetch already prefers them for tied / near-tied BM25 ranks
(T57, §11.1).
* **Python-side** — a composite re-rank with ``_SIGNIFICANCE_WEIGHT``
reinforces the ordering after candidate retrieval, alongside the
recency boost above.
""" """
if witness_role not in _VALID_WITNESS_ROLES: if witness_role not in _VALID_WITNESS_ROLES:
raise ValueError( raise ValueError(
@@ -137,10 +155,15 @@ def search_memories(
"JOIN memories m ON m.id = memories_fts.rowid " "JOIN memories m ON m.id = memories_fts.rowid "
f"WHERE m.owner_id = ? AND m.{witness_col} = 1 " f"WHERE m.owner_id = ? AND m.{witness_col} = 1 "
"AND memories_fts MATCH ? " "AND memories_fts MATCH ? "
"ORDER BY memories_fts.rank " # T57: significance multiplier biases the FTS over-fetch order. BM25
# ``rank`` is lower-is-better, so subtracting ``significance * BIAS``
# surfaces higher-significance rows above lower-significance rows with
# equal/near-equal match strength. Equivalent to ``score × constant``
# per §11.1 once the rank is inverted to a higher-is-better score.
"ORDER BY (memories_fts.rank - m.significance * ?) ASC "
"LIMIT ?" "LIMIT ?"
) )
cur = conn.execute(sql, (owner_id, query, over_fetch)) cur = conn.execute(sql, (owner_id, query, SIGNIFICANCE_RANK_BIAS, over_fetch))
rows = cur.fetchall() rows = cur.fetchall()
if not rows: if not rows:
return [] return []
+123
View File
@@ -0,0 +1,123 @@
from __future__ import annotations
from sqlite3 import Connection
from chat.eventlog.projector import on
from chat.eventlog.log import Event
@on("thread_opened")
def _apply_thread_opened(conn: Connection, e: Event) -> None:
p = e.payload
conn.execute(
"INSERT OR IGNORE INTO threads "
"(thread_id, chat_id, title, summary, status) "
"VALUES (?, ?, ?, ?, 'open')",
(
p["thread_id"],
p["chat_id"],
p["title"],
p.get("summary", ""),
),
)
@on("thread_updated")
def _apply_thread_updated(conn: Connection, e: Event) -> None:
p = e.payload
# Idempotent: closed threads ignore subsequent updates.
conn.execute(
"UPDATE threads SET summary = ?, last_referenced_scene_id = ?, "
"updated_at = datetime('now') "
"WHERE thread_id = ? AND status = 'open'",
(
p.get("summary", ""),
p.get("last_referenced_scene_id"),
p["thread_id"],
),
)
@on("thread_closed")
def _apply_thread_closed(conn: Connection, e: Event) -> None:
p = e.payload
conn.execute(
"UPDATE threads SET status = 'closed', closed_at = ?, "
"updated_at = datetime('now') "
"WHERE thread_id = ? AND status = 'open'",
(p.get("closed_at"), p["thread_id"]),
)
def get_thread(conn: Connection, thread_id: str) -> dict | None:
row = conn.execute(
"SELECT thread_id, chat_id, title, summary, status, "
"opened_at, closed_at, last_referenced_scene_id, "
"created_at, updated_at "
"FROM threads WHERE thread_id = ?",
(thread_id,),
).fetchone()
if not row:
return None
return {
"thread_id": row[0],
"chat_id": row[1],
"title": row[2],
"summary": row[3],
"status": row[4],
"opened_at": row[5],
"closed_at": row[6],
"last_referenced_scene_id": row[7],
"created_at": row[8],
"updated_at": row[9],
}
def list_open_threads(conn: Connection, chat_id: str) -> list[dict]:
rows = conn.execute(
"SELECT thread_id, chat_id, title, summary, status, "
"opened_at, closed_at, last_referenced_scene_id, "
"created_at, updated_at "
"FROM threads WHERE chat_id = ? AND status = 'open' "
"ORDER BY id ASC",
(chat_id,),
).fetchall()
return [
{
"thread_id": r[0], "chat_id": r[1], "title": r[2],
"summary": r[3], "status": r[4],
"opened_at": r[5], "closed_at": r[6],
"last_referenced_scene_id": r[7],
"created_at": r[8], "updated_at": r[9],
}
for r in rows
]
def list_threads(conn: Connection, chat_id: str, status: str | None = None) -> list[dict]:
if status is None:
rows = conn.execute(
"SELECT thread_id, chat_id, title, summary, status, "
"opened_at, closed_at, last_referenced_scene_id, "
"created_at, updated_at "
"FROM threads WHERE chat_id = ? ORDER BY id ASC",
(chat_id,),
).fetchall()
else:
rows = conn.execute(
"SELECT thread_id, chat_id, title, summary, status, "
"opened_at, closed_at, last_referenced_scene_id, "
"created_at, updated_at "
"FROM threads WHERE chat_id = ? AND status = ? "
"ORDER BY id ASC",
(chat_id, status),
).fetchall()
return [
{
"thread_id": r[0], "chat_id": r[1], "title": r[2],
"summary": r[3], "status": r[4],
"opened_at": r[5], "closed_at": r[6],
"last_referenced_scene_id": r[7],
"created_at": r[8], "updated_at": r[9],
}
for r in rows
]
+46
View File
@@ -29,6 +29,52 @@ def _apply_chat_created(conn: Connection, e: Event) -> None:
) )
@on("time_skip_elision")
def _apply_time_skip_elision(conn: Connection, e: Event) -> None:
p = e.payload
conn.execute(
"UPDATE chat_state SET time = ? WHERE chat_id = ?",
(p["new_time"], p["chat_id"]),
)
@on("time_skip_jump")
def _apply_time_skip_jump(conn: Connection, e: Event) -> None:
p = e.payload
chat_id = p["chat_id"]
conn.execute(
"UPDATE chat_state SET time = ? WHERE chat_id = ?",
(p["new_time"], chat_id),
)
if p.get("reset_activity", False):
# Activity rows are keyed by entity_id with a container_id FK.
# Each chat owns its containers, so deleting activity rows whose
# container_id belongs to this chat clears every present entity.
conn.execute(
"DELETE FROM activity "
"WHERE container_id IN (SELECT id FROM containers WHERE chat_id = ?)",
(chat_id,),
)
@on("guest_added")
def _apply_guest_added(conn: Connection, e: Event) -> None:
p = e.payload
conn.execute(
"UPDATE chats SET guest_bot_id = ? WHERE id = ?",
(p["guest_bot_id"], p["chat_id"]),
)
@on("guest_removed")
def _apply_guest_removed(conn: Connection, e: Event) -> None:
p = e.payload
conn.execute(
"UPDATE chats SET guest_bot_id = NULL WHERE id = ?",
(p["chat_id"],),
)
@on("container_created") @on("container_created")
def _apply_container_created(conn: Connection, e: Event) -> None: def _apply_container_created(conn: Connection, e: Event) -> None:
p = e.payload p = e.payload
+365 -5
View File
@@ -41,8 +41,265 @@
{% endif %} {% endif %}
</div> </div>
{% endfor %} {% endfor %}
<details class="skip-controls">
<summary>Elision skip</summary>
<form class="inline-edit"
hx-post="/chats/{{ chat.id }}/drawer/skip/elision"
hx-target="#drawer" hx-swap="innerHTML">
<label>
Landing state hint:
<input type="text" name="landing_state_hint"
placeholder="e.g. arriving at the office">
</label>
<label>
New time (ISO 8601):
<input type="text" name="new_time" required
placeholder="2026-04-26T20:30:00+00:00">
</label>
<button type="submit">Skip ahead</button>
</form>
</details>
<details class="skip-controls">
<summary>Jump skip</summary>
<form class="inline-edit"
hx-post="/chats/{{ chat.id }}/drawer/skip/jump"
hx-target="#drawer" hx-swap="innerHTML">
<label>
New time (ISO 8601):
<input type="text" name="new_time" required
placeholder="2026-04-27T08:00:00+00:00">
</label>
<label>
Anything notable happen? (optional)
<textarea name="notable_prose" rows="3"
placeholder="leave blank to jump without synthesizing memories"></textarea>
</label>
<label>
<input type="checkbox" name="reset_activity" value="1">
Reset activity at landing
</label>
<button type="submit">Jump ahead</button>
</form>
</details>
</section> </section>
<section class="drawer-section">
<h3>Events</h3>
{% if active_events %}
<ul class="event-list">
{% for ev in active_events %}
<li class="event-row">
<strong>{{ ev.kind }}</strong>
<span class="muted"> ({{ ev.status }})</span>
{% if ev.planned_for %}
<p class="muted">planned for: {{ ev.planned_for }}</p>
{% endif %}
{% if ev.props %}
<p class="muted">{{ ev.props|tojson }}</p>
{% endif %}
<form class="inline-edit"
hx-post="/chats/{{ chat.id }}/drawer/event/cancel/{{ ev.event_id }}"
hx-target="#drawer" hx-swap="innerHTML">
<button type="submit">Cancel</button>
</form>
</li>
{% endfor %}
</ul>
{% else %}
<p class="muted">No active events.</p>
{% endif %}
<details>
<summary>Plan event</summary>
<form class="inline-edit"
hx-post="/chats/{{ chat.id }}/drawer/event/plan"
hx-target="#drawer" hx-swap="innerHTML">
<label>
Kind:
<input type="text" name="kind" required
placeholder="e.g. dinner_reservation">
</label>
<label>
Planned for (ISO 8601):
<input type="text" name="planned_for" required
placeholder="2026-04-26T19:00:00+00:00">
</label>
<label>
Props (JSON):
<textarea name="props_json" rows="3"
placeholder='{"location": "Bistro X"}'>{}</textarea>
</label>
<button type="submit">Plan event</button>
</form>
</details>
</section>
<section class="drawer-section">
<h3>Threads</h3>
{% if open_threads %}
<ul class="thread-list">
{% for th in open_threads %}
<li class="thread-row">
<strong>{{ th.title }}</strong>
{% if th.summary %}
<p>{{ th.summary }}</p>
{% endif %}
<form class="inline-edit"
hx-post="/chats/{{ chat.id }}/drawer/thread/close/{{ th.thread_id }}"
hx-target="#drawer" hx-swap="innerHTML">
<button type="submit">Close</button>
</form>
</li>
{% endfor %}
</ul>
{% else %}
<p class="muted">No open threads.</p>
{% endif %}
</section>
{% if guest_bot %}
<section class="drawer-section">
<h3>Guest</h3>
<p><strong>{{ guest_bot.name }}</strong></p>
{% if guest_activity %}
<p>{{ guest_activity.posture or "—" }} / {{ (guest_activity.action or {}).verb or "—" }}</p>
{% if guest_activity.attention %}<p class="muted">attention: {{ guest_activity.attention }}</p>{% endif %}
{% if guest_activity.holding %}<p class="muted">holding: {{ guest_activity.holding|join(", ") }}</p>{% endif %}
{% else %}
<p class="muted">No activity recorded.</p>
{% endif %}
{% if edge_h2g %}
<div class="edge-row">
<strong>{{ host_bot.name }} &rarr; {{ guest_bot.name }}</strong>
<p>Affinity: {{ edge_h2g.affinity }}/100 &middot; Trust: {{ edge_h2g.trust }}/100</p>
{% if edge_h2g.knowledge %}
<details><summary>Knowledge ({{ edge_h2g.knowledge|length }})</summary>
<ul>{% for fact in edge_h2g.knowledge %}<li>{{ fact }}</li>{% endfor %}</ul>
</details>
{% endif %}
</div>
{% endif %}
{% if edge_g2h %}
<div class="edge-row">
<strong>{{ guest_bot.name }} &rarr; {{ host_bot.name }}</strong>
<p>Affinity: {{ edge_g2h.affinity }}/100 &middot; Trust: {{ edge_g2h.trust }}/100</p>
{% if edge_g2h.knowledge %}
<details><summary>Knowledge ({{ edge_g2h.knowledge|length }})</summary>
<ul>{% for fact in edge_g2h.knowledge %}<li>{{ fact }}</li>{% endfor %}</ul>
</details>
{% endif %}
</div>
{% endif %}
{% if edge_y2g %}
<div class="edge-row">
<strong>you &rarr; {{ guest_bot.name }}</strong>
<p>Affinity: {{ edge_y2g.affinity }}/100 &middot; Trust: {{ edge_y2g.trust }}/100</p>
</div>
{% endif %}
{% if edge_g2y %}
<div class="edge-row">
<strong>{{ guest_bot.name }} &rarr; you</strong>
<p>Affinity: {{ edge_g2y.affinity }}/100 &middot; Trust: {{ edge_g2y.trust }}/100</p>
</div>
{% endif %}
<form class="inline-edit"
hx-post="/chats/{{ chat.id }}/drawer/guest/remove"
hx-target="#drawer" hx-swap="innerHTML">
<button type="submit">Remove guest</button>
</form>
</section>
{% else %}
<section class="drawer-section">
<h3>Add guest</h3>
{% if available_guests %}
{% set first_guest_id = available_guests[0].id %}
{% set first_existing = existing_guest_edges.get(first_guest_id, False) %}
<form class="inline-edit add-guest-form"
hx-post="/chats/{{ chat.id }}/drawer/guest/add"
hx-target="#drawer" hx-swap="innerHTML">
<label>
Bot:
<select name="guest_bot_id" required class="add-guest-select">
{% for b in available_guests %}
<option value="{{ b.id }}"
data-existing-edge="{{ 'true' if existing_guest_edges.get(b.id) else 'false' }}">
{{ b.name }}{% if existing_guest_edges.get(b.id) %} (already met){% endif %}
</option>
{% endfor %}
</select>
</label>
<p class="muted add-guest-existing-note"
{% if not first_existing %}hidden{% endif %}>
they already know each other (edge exists from a prior chat)
</p>
<label class="add-guest-reseed-label"
{% if not first_existing %}hidden{% endif %}>
<input type="checkbox" name="reseed" value="1" class="add-guest-reseed">
re-seed anyway
</label>
<label>
Have they met before? Describe how (leave blank if not):
<textarea name="relationship_prose" rows="3"
class="add-guest-prose"
{% if first_existing %}disabled{% endif %}
placeholder="e.g. Old college friends who studied physics together."></textarea>
</label>
<button type="submit">Add guest</button>
</form>
<script>
(function () {
var form = document.currentScript.previousElementSibling;
while (form && !form.classList.contains('add-guest-form')) {
form = form.previousElementSibling;
}
if (!form) return;
var sel = form.querySelector('.add-guest-select');
var prose = form.querySelector('.add-guest-prose');
var reseed = form.querySelector('.add-guest-reseed');
var note = form.querySelector('.add-guest-existing-note');
var reseedLabel = form.querySelector('.add-guest-reseed-label');
function refresh() {
var opt = sel.options[sel.selectedIndex];
var existing = opt && opt.getAttribute('data-existing-edge') === 'true';
if (existing) {
note.removeAttribute('hidden');
reseedLabel.removeAttribute('hidden');
prose.disabled = !reseed.checked;
} else {
note.setAttribute('hidden', '');
reseedLabel.setAttribute('hidden', '');
reseed.checked = false;
prose.disabled = false;
}
}
sel.addEventListener('change', refresh);
reseed.addEventListener('change', refresh);
refresh();
})();
</script>
{% else %}
<p class="muted">No other bots authored yet.</p>
{% endif %}
</section>
{% endif %}
{% if group_node %}
<section class="drawer-section">
<h3>Group</h3>
{% if group_node.summary %}
<p>{{ group_node.summary }}</p>
{% else %}
<p class="muted">No group summary yet.</p>
{% endif %}
{% if group_node.dynamic %}
<p class="muted">Dynamic: {{ group_node.dynamic }}</p>
{% endif %}
</section>
{% endif %}
<section class="drawer-section"> <section class="drawer-section">
<h3>Edges</h3> <h3>Edges</h3>
{% if edge_b2y %} {% if edge_b2y %}
@@ -61,19 +318,95 @@
</label> </label>
<button type="submit">Save</button> <button type="submit">Save</button>
</form> </form>
{% if edge_b2y.summary %}<p class="muted">{{ edge_b2y.summary }}</p>{% endif %} <form class="inline-edit"
hx-post="/chats/{{ chat.id }}/drawer/edge/trust"
hx-target="#drawer" hx-swap="innerHTML">
<input type="hidden" name="source_id" value="{{ host_bot.id }}">
<input type="hidden" name="target_id" value="you">
<label>
Trust:
<input type="range" name="new_value" min="0" max="100"
value="{{ edge_b2y.trust }}"
oninput="this.nextElementSibling.value = this.value">
<output>{{ edge_b2y.trust }}</output>
</label>
<button type="submit">Save</button>
</form>
<form class="inline-edit"
hx-post="/chats/{{ chat.id }}/drawer/edge/summary"
hx-target="#drawer" hx-swap="innerHTML">
<input type="hidden" name="source_id" value="{{ host_bot.id }}">
<input type="hidden" name="target_id" value="you">
<label>
Summary:
<textarea name="new_summary" rows="3" maxlength="2000">{{ edge_b2y.summary or "" }}</textarea>
</label>
<button type="submit">Save summary</button>
</form>
<details>
<summary>Knowledge ({{ (edge_b2y.knowledge or [])|length }})</summary>
{% if edge_b2y.knowledge %} {% if edge_b2y.knowledge %}
<details><summary>Knowledge ({{ edge_b2y.knowledge|length }})</summary> <ul>
<ul>{% for fact in edge_b2y.knowledge %}<li>{{ fact }}</li>{% endfor %}</ul> {% for fact in edge_b2y.knowledge %}
</details> <li>
{{ fact }}
<form class="inline-edit"
hx-post="/chats/{{ chat.id }}/drawer/edge/knowledge-facts"
hx-target="#drawer" hx-swap="innerHTML">
<input type="hidden" name="source_id" value="{{ host_bot.id }}">
<input type="hidden" name="target_id" value="you">
<input type="hidden" name="action" value="remove">
<input type="hidden" name="fact" value="{{ fact }}">
<button type="submit">Remove</button>
</form>
</li>
{% endfor %}
</ul>
{% endif %} {% endif %}
<form class="inline-edit"
hx-post="/chats/{{ chat.id }}/drawer/edge/knowledge-facts"
hx-target="#drawer" hx-swap="innerHTML">
<input type="hidden" name="source_id" value="{{ host_bot.id }}">
<input type="hidden" name="target_id" value="you">
<input type="hidden" name="action" value="add">
<label>
Add fact:
<input type="text" name="fact" maxlength="500" required>
</label>
<button type="submit">Add</button>
</form>
</details>
</div> </div>
{% endif %} {% endif %}
{% if edge_y2b %} {% if edge_y2b %}
<div class="edge-row"> <div class="edge-row">
<strong>you &rarr; {{ host_bot.name }}</strong> <strong>you &rarr; {{ host_bot.name }}</strong>
<p>Affinity: {{ edge_y2b.affinity }}/100 &middot; Trust: {{ edge_y2b.trust }}/100</p> <p>Affinity: {{ edge_y2b.affinity }}/100 &middot; Trust: {{ edge_y2b.trust }}/100</p>
{% if edge_y2b.summary %}<p class="muted">{{ edge_y2b.summary }}</p>{% endif %} <form class="inline-edit"
hx-post="/chats/{{ chat.id }}/drawer/edge/trust"
hx-target="#drawer" hx-swap="innerHTML">
<input type="hidden" name="source_id" value="you">
<input type="hidden" name="target_id" value="{{ host_bot.id }}">
<label>
Trust:
<input type="range" name="new_value" min="0" max="100"
value="{{ edge_y2b.trust }}"
oninput="this.nextElementSibling.value = this.value">
<output>{{ edge_y2b.trust }}</output>
</label>
<button type="submit">Save</button>
</form>
<form class="inline-edit"
hx-post="/chats/{{ chat.id }}/drawer/edge/summary"
hx-target="#drawer" hx-swap="innerHTML">
<input type="hidden" name="source_id" value="you">
<input type="hidden" name="target_id" value="{{ host_bot.id }}">
<label>
Summary:
<textarea name="new_summary" rows="3" maxlength="2000">{{ edge_y2b.summary or "" }}</textarea>
</label>
<button type="submit">Save summary</button>
</form>
</div> </div>
{% endif %} {% endif %}
{% if not edge_b2y and not edge_y2b %} {% if not edge_b2y and not edge_y2b %}
@@ -129,6 +462,33 @@
<input type="hidden" name="pinned" value="{{ 0 if m.pinned else 1 }}"> <input type="hidden" name="pinned" value="{{ 0 if m.pinned else 1 }}">
<button type="submit">{{ 'Unpin' if m.pinned else 'Pin' }}</button> <button type="submit">{{ 'Unpin' if m.pinned else 'Pin' }}</button>
</form> </form>
<div class="witness-row">
{% for flag in ['you', 'host', 'guest'] %}
{% set witnessed = m['witness_' ~ flag] %}
<form class="inline-edit"
hx-post="/chats/{{ chat.id }}/drawer/memory/witness"
hx-target="#drawer" hx-swap="innerHTML">
<input type="hidden" name="memory_id" value="{{ m.id }}">
<input type="hidden" name="flag" value="{{ flag }}">
<input type="hidden" name="new_value" value="{{ 0 if witnessed else 1 }}">
<label>
<input type="checkbox" {% if witnessed %}checked{% endif %}
onchange="this.form.requestSubmit()">
{{ flag }}
</label>
</form>
{% endfor %}
</div>
<details>
<summary>Edit POV summary</summary>
<form class="inline-edit"
hx-post="/chats/{{ chat.id }}/drawer/memory/pov-summary"
hx-target="#drawer" hx-swap="innerHTML">
<input type="hidden" name="memory_id" value="{{ m.id }}">
<textarea name="new_summary" rows="3" maxlength="2000">{{ m.pov_summary }}</textarea>
<button type="submit">Save</button>
</form>
</details>
</li> </li>
{% endfor %} {% endfor %}
</ul> </ul>
+2 -9
View File
@@ -1,10 +1,10 @@
from __future__ import annotations from __future__ import annotations
import sqlite3
from pathlib import Path from pathlib import Path
from fastapi import APIRouter, Depends, Form, HTTPException, Request from fastapi import APIRouter, Depends, Form, HTTPException, Request
from fastapi.responses import RedirectResponse, HTMLResponse from fastapi.responses import RedirectResponse, HTMLResponse
from fastapi.templating import Jinja2Templates from fastapi.templating import Jinja2Templates
from chat.db.connection import open_db
from chat.eventlog.log import append_event from chat.eventlog.log import append_event
from chat.eventlog.projector import project from chat.eventlog.projector import project
from chat.state.entities import list_bots from chat.state.entities import list_bots
@@ -19,15 +19,8 @@ REQUIRED_FIELDS = ("id", "name", "persona", "initial_relationship_to_you", "kick
def get_conn(request: Request): def get_conn(request: Request):
settings = request.app.state.settings settings = request.app.state.settings
db_path: Path = settings.db_path db_path: Path = settings.db_path
db_path.parent.mkdir(parents=True, exist_ok=True) with open_db(db_path, check_same_thread=False) as conn:
conn = sqlite3.connect(db_path, check_same_thread=False)
conn.execute("PRAGMA journal_mode=WAL")
conn.execute("PRAGMA foreign_keys=ON")
try:
yield conn yield conn
conn.commit()
finally:
conn.close()
def _split_voice_samples(text: str) -> list[str]: def _split_voice_samples(text: str) -> list[str]:
+781 -10
View File
@@ -1,4 +1,4 @@
"""Chat drawer — read view (T24) and inline edits (T25). """Chat drawer — read view (T24) and inline edits (T25, T72).
The GET endpoint renders an HTML partial showing the current scene + The GET endpoint renders an HTML partial showing the current scene +
container, per-entity activity, host <-> you edges, pinned memories with container, per-entity activity, host <-> you edges, pinned memories with
@@ -13,18 +13,22 @@ returning the refreshed drawer partial so HTMX can swap it in:
* pin toggle on a memory (emits ``memory_pin_changed`` with * pin toggle on a memory (emits ``memory_pin_changed`` with
``auto_pinned=0`` so a manual pin is not subject to auto-eviction). ``auto_pinned=0`` so a manual pin is not subject to auto-eviction).
T72 (Phase 2.5) extends the inline-edit set to cover the remaining
§6.4 editable fields whose state-layer support already lands in the
``manual_edit`` projector: edge trust slider, edge summary textarea,
memory POV summary textarea, and per-edge knowledge-fact add/remove. It
also exposes a witness-flag toggle (``you/host/guest``) per memory row
and a "first-meeting gate" on the Add-guest form so an existing edge
isn't quietly overwritten by a re-seed.
Each ``manual_edit`` payload snapshots the prior value alongside the new Each ``manual_edit`` payload snapshots the prior value alongside the new
one so a later inverse edit can restore state (§6.4 final paragraph). one so a later inverse edit can restore state (§6.4 final paragraph).
Other §6.4 editable fields (activity verb/attention/posture, edge_trust,
edge summary, knowledge_facts list, memory pov_summary) are deferred to
a Phase 1.5 follow-up — the dispatch in :mod:`chat.state.manual_edit`
already accepts more ``target_kind`` values, so adding their routes is a
mechanical extension.
""" """
from __future__ import annotations from __future__ import annotations
import json
import uuid
from pathlib import Path from pathlib import Path
from fastapi import APIRouter, Depends, Form, HTTPException, Request from fastapi import APIRouter, Depends, Form, HTTPException, Request
@@ -32,13 +36,22 @@ from fastapi.responses import HTMLResponse
from fastapi.templating import Jinja2Templates from fastapi.templating import Jinja2Templates
from chat.eventlog.log import append_and_apply from chat.eventlog.log import append_and_apply
from chat.services.relationship_seed import seed_inter_bot_edges
from chat.services.scene_summarize import apply_scene_close_summary from chat.services.scene_summarize import apply_scene_close_summary
from chat.state.edges import get_edge from chat.state.edges import get_edge
from chat.state.entities import get_bot, get_you from chat.state.entities import get_bot, get_you, list_bots
from chat.state.events import list_active_events
from chat.state.group_node import get_group_node
from chat.state.memory import get_pinned from chat.state.memory import get_pinned
from chat.state.threads import list_open_threads
from chat.state.world import active_scene, get_activity, get_chat, get_container from chat.state.world import active_scene, get_activity, get_chat, get_container
from chat.web.bots import get_conn from chat.web.bots import get_conn
from chat.web.kickoff import get_llm_client from chat.web.kickoff import get_llm_client
from chat.web.skip import (
_now_iso,
process_elision_skip,
process_jump_skip,
)
TEMPLATES = Jinja2Templates( TEMPLATES = Jinja2Templates(
directory=str(Path(__file__).resolve().parent.parent / "templates") directory=str(Path(__file__).resolve().parent.parent / "templates")
@@ -53,6 +66,13 @@ PIN_CAP = 8
# Recent-memories list is bounded to keep the drawer cheap to render. # Recent-memories list is bounded to keep the drawer cheap to render.
RECENT_LIMIT = 10 RECENT_LIMIT = 10
# T72.1 caps on free-form textarea edits. Edge summaries and per-POV
# memory summaries are drawer-driven prose — bound them so a stray paste
# can't blow up the projected row size or the SSE drawer refresh payload.
EDGE_SUMMARY_MAX = 2000
MEMORY_POV_SUMMARY_MAX = 2000
KNOWLEDGE_FACT_MAX = 500
@router.get("/chats/{chat_id}/drawer", response_class=HTMLResponse) @router.get("/chats/{chat_id}/drawer", response_class=HTMLResponse)
async def drawer(chat_id: str, request: Request, conn=Depends(get_conn)): async def drawer(chat_id: str, request: Request, conn=Depends(get_conn)):
@@ -78,12 +98,49 @@ async def drawer(chat_id: str, request: Request, conn=Depends(get_conn)):
edge_b2y = get_edge(conn, chat["host_bot_id"], "you") edge_b2y = get_edge(conn, chat["host_bot_id"], "you")
edge_y2b = get_edge(conn, "you", chat["host_bot_id"]) edge_y2b = get_edge(conn, "you", chat["host_bot_id"])
# T42: guest + group context. Empty defaults keep the template happy
# when no guest is present (the relevant sections render conditionally).
guest_bot = None
guest_activity = None
edge_h2g = None
edge_g2h = None
edge_y2g = None
edge_g2y = None
available_guests: list[dict] = []
group_node = None
if chat.get("guest_bot_id"):
guest_bot_id = chat["guest_bot_id"]
guest_bot = get_bot(conn, guest_bot_id)
guest_activity = get_activity(conn, guest_bot_id)
edge_h2g = get_edge(conn, chat["host_bot_id"], guest_bot_id)
edge_g2h = get_edge(conn, guest_bot_id, chat["host_bot_id"])
edge_y2g = get_edge(conn, "you", guest_bot_id)
edge_g2y = get_edge(conn, guest_bot_id, "you")
else:
# Candidates for the "Add guest" dropdown — every authored bot
# except the host (and "you", which is implicit, never a bot row).
available_guests = [
b for b in list_bots(conn) if b["id"] != chat["host_bot_id"]
]
# T72.2 first-meeting gate: pre-compute whether a host->candidate edge
# already exists. Template renders the prose textarea disabled and the
# POST handler skips ``seed_inter_bot_edges`` (preserving the existing
# edge content) unless the user explicitly toggles "re-seed anyway".
existing_guest_edges = {
b["id"]: get_edge(conn, chat["host_bot_id"], b["id"]) is not None
for b in available_guests
}
group_node = get_group_node(conn, chat_id)
# Recent memories from host's POV (witness_host = 1), most recent first. # Recent memories from host's POV (witness_host = 1), most recent first.
# Raw query keeps this read self-contained — no projector helper exposes # Raw query keeps this read self-contained — no projector helper exposes
# "latest N for an owner" yet and the drawer is the only consumer. # "latest N for an owner" yet and the drawer is the only consumer. The
# three witness flags ride along so T72.3's per-row checkboxes can
# render the current state without a second query per memory.
recent_rows = conn.execute( recent_rows = conn.execute(
""" """
SELECT id, pov_summary, significance, pinned, created_at SELECT id, pov_summary, significance, pinned, created_at,
witness_you, witness_host, witness_guest
FROM memories FROM memories
WHERE owner_id = ? AND witness_host = 1 WHERE owner_id = ? AND witness_host = 1
ORDER BY id DESC ORDER BY id DESC
@@ -98,12 +155,19 @@ async def drawer(chat_id: str, request: Request, conn=Depends(get_conn)):
"significance": r[2], "significance": r[2],
"pinned": r[3], "pinned": r[3],
"created_at": r[4], "created_at": r[4],
"witness_you": r[5],
"witness_host": r[6],
"witness_guest": r[7],
} }
for r in recent_rows for r in recent_rows
] ]
pinned = get_pinned(conn, chat["host_bot_id"]) pinned = get_pinned(conn, chat["host_bot_id"])
# T59: active events + open threads for the new drawer sections.
active_events = list_active_events(conn, chat_id)
open_threads = list_open_threads(conn, chat_id)
return TEMPLATES.TemplateResponse( return TEMPLATES.TemplateResponse(
request, request,
"_drawer.html", "_drawer.html",
@@ -117,9 +181,20 @@ async def drawer(chat_id: str, request: Request, conn=Depends(get_conn)):
"bot_activity": bot_activity, "bot_activity": bot_activity,
"edge_b2y": edge_b2y, "edge_b2y": edge_b2y,
"edge_y2b": edge_y2b, "edge_y2b": edge_y2b,
"guest_bot": guest_bot,
"guest_activity": guest_activity,
"edge_h2g": edge_h2g,
"edge_g2h": edge_g2h,
"edge_y2g": edge_y2g,
"edge_g2y": edge_g2y,
"available_guests": available_guests,
"existing_guest_edges": existing_guest_edges,
"group_node": group_node,
"recent_memories": recent_memories, "recent_memories": recent_memories,
"pinned": pinned, "pinned": pinned,
"pin_cap": PIN_CAP, "pin_cap": PIN_CAP,
"active_events": active_events,
"open_threads": open_threads,
}, },
) )
@@ -304,3 +379,699 @@ async def toggle_memory_pin(
}, },
) )
return await drawer(chat_id, request, conn) return await drawer(chat_id, request, conn)
# --- T72.1 deferred v1 drawer edits --------------------------------------
#
# These four endpoints round out the §6.4 editable surface — the
# ``manual_edit`` projector already dispatches ``edge_trust``,
# ``edge_summary``, and ``memory_pov_summary`` (T25); ``edge_knowledge_fact``
# is a new dispatch branch added alongside this commit. Each route follows
# the T25 pattern: snapshot the prior value, append + apply ``manual_edit``,
# then re-render the drawer partial.
@router.post(
"/chats/{chat_id}/drawer/edge/trust",
response_class=HTMLResponse,
)
async def edit_edge_trust(
chat_id: str,
request: Request,
source_id: str = Form(...),
target_id: str = Form(...),
new_value: int = Form(...),
conn=Depends(get_conn),
):
chat = get_chat(conn, chat_id)
if chat is None:
raise HTTPException(status_code=404, detail=f"chat not found: {chat_id}")
if not 0 <= int(new_value) <= 100:
raise HTTPException(
status_code=400,
detail=f"trust must be in [0, 100], got {new_value}",
)
edge = get_edge(conn, source_id, target_id)
if edge is None:
raise HTTPException(
status_code=404,
detail=f"edge not found: {source_id}->{target_id}",
)
prior = int(edge["trust"])
append_and_apply(
conn,
kind="manual_edit",
payload={
"target_kind": "edge_trust",
"target_id": {"source_id": source_id, "target_id": target_id},
"prior_value": prior,
"new_value": int(new_value),
},
)
return await drawer(chat_id, request, conn)
@router.post(
"/chats/{chat_id}/drawer/edge/summary",
response_class=HTMLResponse,
)
async def edit_edge_summary(
chat_id: str,
request: Request,
source_id: str = Form(...),
target_id: str = Form(...),
new_summary: str = Form(...),
conn=Depends(get_conn),
):
chat = get_chat(conn, chat_id)
if chat is None:
raise HTTPException(status_code=404, detail=f"chat not found: {chat_id}")
if len(new_summary) > EDGE_SUMMARY_MAX:
raise HTTPException(
status_code=400,
detail=(
f"edge summary exceeds {EDGE_SUMMARY_MAX} chars "
f"(got {len(new_summary)})"
),
)
edge = get_edge(conn, source_id, target_id)
if edge is None:
raise HTTPException(
status_code=404,
detail=f"edge not found: {source_id}->{target_id}",
)
prior = edge.get("summary") or ""
append_and_apply(
conn,
kind="manual_edit",
payload={
"target_kind": "edge_summary",
"target_id": {"source_id": source_id, "target_id": target_id},
"prior_value": prior,
"new_value": new_summary,
},
)
return await drawer(chat_id, request, conn)
@router.post(
"/chats/{chat_id}/drawer/memory/pov-summary",
response_class=HTMLResponse,
)
async def edit_memory_pov_summary(
chat_id: str,
request: Request,
memory_id: int = Form(...),
new_summary: str = Form(...),
conn=Depends(get_conn),
):
chat = get_chat(conn, chat_id)
if chat is None:
raise HTTPException(status_code=404, detail=f"chat not found: {chat_id}")
if len(new_summary) > MEMORY_POV_SUMMARY_MAX:
raise HTTPException(
status_code=400,
detail=(
f"memory pov_summary exceeds {MEMORY_POV_SUMMARY_MAX} chars "
f"(got {len(new_summary)})"
),
)
# 404 when the memory either doesn't exist or belongs to a different
# chat — the drawer never surfaces cross-chat memories so editing one
# would be a path-traversal-style mistake.
row = conn.execute(
"SELECT pov_summary FROM memories WHERE id = ? AND chat_id = ?",
(int(memory_id), chat_id),
).fetchone()
if row is None:
raise HTTPException(
status_code=404,
detail=f"memory not found in chat: {memory_id}",
)
prior = row[0] or ""
append_and_apply(
conn,
kind="manual_edit",
payload={
"target_kind": "memory_pov_summary",
"target_id": int(memory_id),
"prior_value": prior,
"new_value": new_summary,
},
)
return await drawer(chat_id, request, conn)
@router.post(
"/chats/{chat_id}/drawer/edge/knowledge-facts",
response_class=HTMLResponse,
)
async def edit_edge_knowledge_facts(
chat_id: str,
request: Request,
source_id: str = Form(...),
target_id: str = Form(...),
action: str = Form(...),
fact: str = Form(...),
conn=Depends(get_conn),
):
"""Add or remove a single knowledge_fact on an edge.
Remove semantics are by string match (first occurrence) — the drawer
re-renders after every edit so threading a stable index through is
fragile when concurrent ``edge_update`` events can append more facts
between renders. The projector is a no-op when the fact isn't found,
keeping the route idempotent for stale form submissions.
"""
chat = get_chat(conn, chat_id)
if chat is None:
raise HTTPException(status_code=404, detail=f"chat not found: {chat_id}")
if action not in ("add", "remove"):
raise HTTPException(
status_code=400,
detail=f"action must be 'add' or 'remove', got {action!r}",
)
if len(fact) > KNOWLEDGE_FACT_MAX:
raise HTTPException(
status_code=400,
detail=(
f"fact exceeds {KNOWLEDGE_FACT_MAX} chars (got {len(fact)})"
),
)
if not fact.strip():
raise HTTPException(status_code=400, detail="fact must not be empty")
edge = get_edge(conn, source_id, target_id)
if edge is None:
raise HTTPException(
status_code=404,
detail=f"edge not found: {source_id}->{target_id}",
)
prior = list(edge.get("knowledge") or [])
append_and_apply(
conn,
kind="manual_edit",
payload={
"target_kind": "edge_knowledge_fact",
"target_id": {"source_id": source_id, "target_id": target_id},
"prior_value": prior,
"new_value": {"action": action, "fact": fact},
},
)
return await drawer(chat_id, request, conn)
# --- T72.3 witness flag inline-edit --------------------------------------
#
# Witness flags decide which entities can recall a memory (§7 retrieval).
# Editing them is rare but high-impact — flipping ``witness_guest`` from 0
# to 1 makes the memory available to the guest's prompt context. The route
# follows the T25 / T72.1 pattern: snapshot prior, append + apply
# ``manual_edit`` with a ``{flag, value}`` payload, refresh the partial.
_VALID_WITNESS_FLAGS = ("you", "host", "guest")
@router.post(
"/chats/{chat_id}/drawer/memory/witness",
response_class=HTMLResponse,
)
async def edit_memory_witness(
chat_id: str,
request: Request,
memory_id: int = Form(...),
flag: str = Form(...),
new_value: int = Form(...),
conn=Depends(get_conn),
):
chat = get_chat(conn, chat_id)
if chat is None:
raise HTTPException(status_code=404, detail=f"chat not found: {chat_id}")
if flag not in _VALID_WITNESS_FLAGS:
raise HTTPException(
status_code=400,
detail=(
f"flag must be one of {list(_VALID_WITNESS_FLAGS)}, "
f"got {flag!r}"
),
)
row = conn.execute(
f"SELECT witness_{flag} FROM memories "
"WHERE id = ? AND chat_id = ?",
(int(memory_id), chat_id),
).fetchone()
if row is None:
raise HTTPException(
status_code=404,
detail=f"memory not found in chat: {memory_id}",
)
prior_int = int(row[0])
new_int = 1 if int(new_value) else 0
append_and_apply(
conn,
kind="manual_edit",
payload={
"target_kind": "memory_witness",
"target_id": int(memory_id),
"prior_value": {"flag": flag, "value": prior_int},
"new_value": {"flag": flag, "value": new_int},
},
)
return await drawer(chat_id, request, conn)
# --- T42 guest add/remove -------------------------------------------------
#
# Adding a guest fans out into up to four events: a ``guest_added`` to flip
# ``chats.guest_bot_id``, two ``edge_update`` events seeded from the
# user-supplied prose (skipped when the prose is empty / the seed comes back
# default), and a ``group_node_initialized`` if no row exists yet — three
# entities now share the chat so the §8.4 group node becomes meaningful.
#
# Removing a guest first emits ``scene_closed`` for the active scene (so any
# host -> you scene closes cleanly with the guest still in scope) before
# clearing the guest_bot_id; per spec the next user message implicitly opens
# a fresh you+host scene via Phase 1's mid-chat reset behavior.
def _seed_is_default(seed) -> bool:
"""Treat a seed as a no-op when both summaries are empty AND both
delta pairs are zero AND both fact lists are empty.
"""
return (
not seed.a_to_b_summary
and not seed.b_to_a_summary
and seed.a_to_b_affinity_delta == 0
and seed.a_to_b_trust_delta == 0
and seed.b_to_a_affinity_delta == 0
and seed.b_to_a_trust_delta == 0
and not seed.a_to_b_knowledge_facts
and not seed.b_to_a_knowledge_facts
)
@router.post(
"/chats/{chat_id}/drawer/guest/add",
response_class=HTMLResponse,
)
async def add_guest(
chat_id: str,
request: Request,
guest_bot_id: str = Form(...),
relationship_prose: str = Form(""),
reseed: str = Form(""),
conn=Depends(get_conn),
client=Depends(get_llm_client),
):
chat = get_chat(conn, chat_id)
if chat is None:
raise HTTPException(status_code=404, detail=f"chat not found: {chat_id}")
if chat.get("guest_bot_id") is not None:
raise HTTPException(
status_code=400,
detail="a guest is already present in this chat",
)
if guest_bot_id == chat["host_bot_id"]:
raise HTTPException(
status_code=400, detail="guest must differ from host"
)
guest_bot = get_bot(conn, guest_bot_id)
if guest_bot is None:
raise HTTPException(
status_code=404, detail=f"guest bot not found: {guest_bot_id}"
)
host_bot = get_bot(conn, chat["host_bot_id"])
if host_bot is None:
raise HTTPException(
status_code=404,
detail=f"host bot not found: {chat['host_bot_id']}",
)
# T72.2 first-meeting gate: when an edge already exists from a prior
# chat, the textarea is rendered disabled. Submission without the
# explicit "re-seed anyway" toggle skips ``seed_inter_bot_edges``
# entirely so the existing edge content (affinity, trust, knowledge,
# summaries) survives. ``guest_added`` and ``group_node_initialized``
# still fire so the chat picks up the new participant.
existing_edge = (
get_edge(conn, chat["host_bot_id"], guest_bot_id) is not None
)
reseed_requested = reseed.lower() in ("1", "true", "on", "yes")
skip_seed = existing_edge and not reseed_requested
settings = request.app.state.settings
if skip_seed:
seed = None
else:
seed = await seed_inter_bot_edges(
client,
classifier_model=settings.classifier_model,
bot_a_id=chat["host_bot_id"],
bot_a_name=host_bot["name"],
bot_b_id=guest_bot_id,
bot_b_name=guest_bot["name"],
relationship_prose=relationship_prose,
timeout_s=settings.classifier_timeout_s,
)
append_and_apply(
conn,
kind="guest_added",
payload={"chat_id": chat_id, "guest_bot_id": guest_bot_id},
)
# Emit edge_update only when the seed carries content. Empty prose
# short-circuits inside ``seed_inter_bot_edges`` to a default seed,
# so this skips the two extra log entries on the no-prose path.
# NOTE: ``_apply_edge_update`` does not accept a ``summary`` field —
# per-direction summary is set via the per-pov scene-close path
# (T27), not direct edge_update. We therefore drop seed.*_summary
# here; the deltas + knowledge_facts are what materializes.
if seed is not None and not _seed_is_default(seed):
append_and_apply(
conn,
kind="edge_update",
payload={
"source_id": chat["host_bot_id"],
"target_id": guest_bot_id,
"chat_id": chat_id,
"affinity_delta": seed.a_to_b_affinity_delta,
"trust_delta": seed.a_to_b_trust_delta,
"knowledge_facts": seed.a_to_b_knowledge_facts,
"last_interaction_at": chat.get("time"),
"last_interaction_chat_id": chat_id,
},
)
append_and_apply(
conn,
kind="edge_update",
payload={
"source_id": guest_bot_id,
"target_id": chat["host_bot_id"],
"chat_id": chat_id,
"affinity_delta": seed.b_to_a_affinity_delta,
"trust_delta": seed.b_to_a_trust_delta,
"knowledge_facts": seed.b_to_a_knowledge_facts,
"last_interaction_at": chat.get("time"),
"last_interaction_chat_id": chat_id,
},
)
# Three entities now share the chat (you, host, guest) — initialize
# the group node row if Wave 1's reader doesn't see one yet.
if get_group_node(conn, chat_id) is None:
append_and_apply(
conn,
kind="group_node_initialized",
payload={
"chat_id": chat_id,
"members": ["you", chat["host_bot_id"], guest_bot_id],
"summary": "",
"dynamic": "",
"threads": [],
},
)
return await drawer(chat_id, request, conn)
@router.post(
"/chats/{chat_id}/drawer/guest/remove",
response_class=HTMLResponse,
)
async def remove_guest(
chat_id: str,
request: Request,
conn=Depends(get_conn),
):
chat = get_chat(conn, chat_id)
if chat is None:
raise HTTPException(status_code=404, detail=f"chat not found: {chat_id}")
if chat.get("guest_bot_id") is None:
raise HTTPException(
status_code=400, detail="no guest present in this chat"
)
# Close the active scene (if any) before flipping guest_bot_id so
# the scene record carries the guest as a participant.
scene = active_scene(conn, chat_id)
if scene is not None:
append_and_apply(
conn,
kind="scene_closed",
payload={
"scene_id": scene["id"],
"ended_at": chat.get("time"),
"significance": 0,
},
)
append_and_apply(
conn,
kind="guest_removed",
payload={"chat_id": chat_id},
)
return await drawer(chat_id, request, conn)
# --- T59 events / threads / skip controls --------------------------------
#
# Five drawer-driven endpoints that emit Phase 3 event-log entries:
#
# * ``event_planned`` / ``event_cancelled`` for the events panel — props
# arrive as a JSON-encoded form field so the user can author arbitrary
# structured side-info without a custom HTMX widget per kind.
# * ``time_skip_elision`` / ``time_skip_jump`` for the skip panel —
# each emits the projector event AND an ``assistant_turn`` carrying the
# narration prose from :mod:`chat.services.skip_narration`. Jump skips
# ALSO write per-bot synthesized memories from any user-supplied
# ``notable_prose`` via :func:`synthesize_memories` +
# :func:`record_turn_memory_for_present`.
# * ``thread_closed`` for the threads panel.
#
# Skip narration is appended via plain ``append_event`` (assistant_turn
# has no projector handler — it's a transcript-only kind, see
# :func:`chat.web.turns._read_recent_dialogue`). The user will see the
# new turn on the next chat-detail page load; we do NOT broadcast via
# ``publish`` here because the SSE channel is scoped to the chat-detail
# page and the drawer partial is the response body — adding cross-cutting
# SSE here would require dragging the publish import + chat-channel state
# into the drawer module without a meaningful UX gain (the drawer only
# rerenders itself on these submissions).
@router.post(
"/chats/{chat_id}/drawer/event/plan",
response_class=HTMLResponse,
)
async def plan_event(
chat_id: str,
request: Request,
kind: str = Form(...),
planned_for: str = Form(...),
props_json: str = Form("{}"),
conn=Depends(get_conn),
):
"""Append an ``event_planned`` row from the drawer's "Plan event" form.
``props_json`` is parsed into a dict before being attached to the
payload so the projector can treat it as structured data. Bad JSON
yields ``400`` — the form template renders an inline error in that
case so the user can fix-and-resubmit without losing their input.
"""
chat = get_chat(conn, chat_id)
if chat is None:
raise HTTPException(status_code=404, detail=f"chat not found: {chat_id}")
try:
props = json.loads(props_json) if props_json.strip() else {}
except json.JSONDecodeError as exc:
raise HTTPException(
status_code=400, detail=f"props_json must be valid JSON: {exc}"
)
if not isinstance(props, dict):
raise HTTPException(
status_code=400, detail="props_json must encode a JSON object"
)
event_id = f"evt_{uuid.uuid4().hex[:12]}"
append_and_apply(
conn,
kind="event_planned",
payload={
"event_id": event_id,
"chat_id": chat_id,
"kind": kind,
"props": props,
"planned_for": planned_for,
},
)
return await drawer(chat_id, request, conn)
@router.post(
"/chats/{chat_id}/drawer/event/cancel/{event_id}",
response_class=HTMLResponse,
)
async def cancel_event(
chat_id: str,
event_id: str,
request: Request,
conn=Depends(get_conn),
):
"""Append an ``event_cancelled`` row for ``event_id``.
``completed_at`` is sourced from the chat clock (so cancellations
timeline-align with the rest of the fiction) with a UTC-now fallback
when the clock isn't set. The projector is idempotent on terminal
statuses so a stale double-submit is harmless.
"""
chat = get_chat(conn, chat_id)
if chat is None:
raise HTTPException(status_code=404, detail=f"chat not found: {chat_id}")
completed_at = chat.get("time") or _now_iso()
append_and_apply(
conn,
kind="event_cancelled",
payload={
"event_id": event_id,
"completed_at": completed_at,
},
)
return await drawer(chat_id, request, conn)
@router.post(
"/chats/{chat_id}/drawer/skip/elision",
response_class=HTMLResponse,
)
async def skip_elision(
chat_id: str,
request: Request,
landing_state_hint: str = Form(""),
new_time: str = Form(...),
conn=Depends(get_conn),
client=Depends(get_llm_client),
):
"""Elision skip: collapse in-progress activity into its end-state.
Thin HTTP wrapper around :func:`chat.web.skip.process_elision_skip`
(T62 extracted the controller). Validation failures surface as
``400`` and the route still returns the refreshed drawer partial on
success so HTMX swaps in the new chat clock.
"""
settings = request.app.state.settings
try:
await process_elision_skip(
conn,
client,
settings,
chat_id=chat_id,
new_time=new_time,
landing_state_hint=landing_state_hint,
)
except ValueError as exc:
# ``process_elision_skip`` raises on missing-chat or malformed /
# backwards new_time. The drawer used to 404 / 400 these
# separately — preserve the 404-vs-400 split by sniffing the
# error message so existing tests keep passing without changes.
if str(exc).startswith("chat not found"):
raise HTTPException(status_code=404, detail=str(exc))
raise HTTPException(status_code=400, detail=str(exc))
return await drawer(chat_id, request, conn)
@router.post(
"/chats/{chat_id}/drawer/skip/jump",
response_class=HTMLResponse,
)
async def skip_jump(
chat_id: str,
request: Request,
new_time: str = Form(...),
notable_prose: str = Form(""),
reset_activity: str = Form(""),
conn=Depends(get_conn),
client=Depends(get_llm_client),
):
"""Jump skip: bridge a longer fiction-time delta.
Thin HTTP wrapper around :func:`chat.web.skip.process_jump_skip`
(T62 extracted the controller). ``reset_activity`` is parsed
permissively here ("1" / "true" / "on" / "yes" — same shape as the
add-guest reseed flag) since HTML checkboxes typically post the
literal "1" or omit the field entirely.
"""
reset_flag = reset_activity.lower() in ("1", "true", "on", "yes")
settings = request.app.state.settings
try:
await process_jump_skip(
conn,
client,
settings,
chat_id=chat_id,
new_time=new_time,
notable_prose=notable_prose,
reset_activity=reset_flag,
)
except ValueError as exc:
if str(exc).startswith("chat not found"):
raise HTTPException(status_code=404, detail=str(exc))
raise HTTPException(status_code=400, detail=str(exc))
return await drawer(chat_id, request, conn)
@router.post(
"/chats/{chat_id}/drawer/thread/close/{thread_id}",
response_class=HTMLResponse,
)
async def close_thread(
chat_id: str,
thread_id: str,
request: Request,
conn=Depends(get_conn),
):
"""Append a ``thread_closed`` row for ``thread_id``.
Mirrors :func:`cancel_event` — chat-clock-or-now timestamp, projector
handles idempotency. The drawer's open-threads list is sourced from
``list_open_threads`` which filters by ``status='open'`` so a stale
double-submit is a no-op visually.
"""
chat = get_chat(conn, chat_id)
if chat is None:
raise HTTPException(status_code=404, detail=f"chat not found: {chat_id}")
closed_at = chat.get("time") or _now_iso()
append_and_apply(
conn,
kind="thread_closed",
payload={
"thread_id": thread_id,
"closed_at": closed_at,
},
)
return await drawer(chat_id, request, conn)
+398
View File
@@ -0,0 +1,398 @@
"""Meanwhile-mode turn controller (T64).
A meanwhile scene is a private 2-bot scene running alongside an active
you-scene (its parent). The user manually advances it by POSTing to the
existing ``/chats/<id>/turns`` endpoint; the route detects an active
meanwhile scene at the start of ``post_turn`` and dispatches here.
Unlike the normal turn flow, "you" is NOT a witness to the scene. The
controller mirrors ``post_turn`` shape but with:
- Speaker alternation derived from the latest meanwhile ``assistant_turn``
scoped to this scene_id (host first, then alternating).
- Prompt assembly with ``present_set_kind="host_guest"`` so the prompt
builder drops the "you" activity bullet and any speaker -> "you" edge.
- Memory writes via ``record_meanwhile_memory`` — both bots get rows
with witness flags ``[you=0, host=1, guest=1]``.
- State updates over exactly 2 directed pairs (host <-> guest); no
you-related pairs fire.
- The ``assistant_turn`` payload carries ``meanwhile_scene_id`` and
``present_set_kind="host_guest"`` so downstream filters (alternation
lookup, drawer rendering, scene-close detection) can scope to the
meanwhile slice without conflating it with the parent you-scene's
history.
Scene-close detection for meanwhile scenes is not auto-fired here —
T65 covers the close + digest pipeline. The controller's job ends
after the post-turn classifier passes land.
"""
from __future__ import annotations
import asyncio
import json
from chat.config import Settings
from chat.eventlog.log import append_and_apply, append_event
from chat.llm.client import LLMClient
from chat.services.memory_write import record_meanwhile_memory
from chat.services.multi_state_update import compute_state_updates_for_present
from chat.services.prompt import assemble_narrative_prompt
from chat.services.turn_parse import parse_turn
from chat.state.edges import get_edge
from chat.state.entities import get_bot
from chat.state.meanwhile import list_meanwhile_scenes
from chat.state.world import get_chat
from chat.web.pubsub import publish
from chat.web.render import render_turn_html as _render_turn_html
def _strip_ooc_for_prompt(parsed) -> str:
"""Mirror of the helper in turns.py — concatenate non-OOC segments."""
keep = [s.text for s in parsed.segments if s.kind != "ooc"]
return " ".join(keep).strip()
def _read_recent_meanwhile_dialogue(
conn, chat_id: str, scene_id: int, limit: int = 50
) -> list[dict]:
"""Return the meanwhile scene's prior turns shaped as
``{"speaker": <id>, "text": <prose>}``.
Pulls ``user_turn`` rows for the chat (the user-side prose driving
this meanwhile scene rides through the same chat) plus only those
``assistant_turn`` rows whose ``meanwhile_scene_id`` matches the
given scene id. Other meanwhile scenes on the same chat — and the
parent you-scene's assistant_turns — are excluded so the prompt
context stays scoped to the private beat.
Filters chat_id (and meanwhile_scene_id for assistant_turn) via
``json_extract`` in SQL so SQLite stops at the first ``limit`` rows
that already match — avoids an unbounded scan as ``event_log``
grows. The user-side rows match on chat_id only since they aren't
tagged with a scene id (they ride the chat-wide log).
"""
cur = conn.execute(
"SELECT id, kind, payload_json FROM event_log "
"WHERE kind IN ('user_turn', 'user_turn_edit', 'assistant_turn') "
" AND superseded_by IS NULL AND hidden = 0 "
" AND json_extract(payload_json, '$.chat_id') = ? "
" AND ("
" kind IN ('user_turn', 'user_turn_edit') "
" OR json_extract(payload_json, '$.meanwhile_scene_id') = ?"
" ) "
"ORDER BY id DESC LIMIT ?",
(chat_id, scene_id, limit),
)
rows = cur.fetchall()
rows.reverse()
out: list[dict] = []
for _row_id, kind, payload_json in rows:
p = json.loads(payload_json)
if kind in ("user_turn", "user_turn_edit"):
out.append({"speaker": "you", "text": p.get("prose", "")})
else:
out.append(
{
"speaker": p.get("speaker_id", "bot"),
"text": p.get("text", ""),
}
)
return out
def _last_meanwhile_speaker(conn, chat_id: str, scene_id: int) -> str | None:
"""Return the speaker_id of the latest assistant_turn linked to
``scene_id`` for ``chat_id``, or ``None`` if no prior turn exists.
Used to alternate the speaker across consecutive meanwhile turns —
the OTHER bot speaks next. Pushes both filters into SQL via
``json_extract`` and bounds with ``LIMIT 1`` so SQLite stops at the
first match instead of scanning the whole assistant_turn slice.
"""
row = conn.execute(
"SELECT json_extract(payload_json, '$.speaker_id') AS speaker "
"FROM event_log "
"WHERE kind = 'assistant_turn' "
" AND superseded_by IS NULL AND hidden = 0 "
" AND json_extract(payload_json, '$.chat_id') = ? "
" AND json_extract(payload_json, '$.meanwhile_scene_id') = ? "
"ORDER BY id DESC "
"LIMIT 1",
(chat_id, scene_id),
).fetchone()
return row[0] if row else None
async def process_meanwhile_turn(
conn,
client: LLMClient,
settings: Settings,
*,
chat_id: str,
prose: str,
) -> dict:
"""Run one meanwhile turn end-to-end.
Returns a small dict shape ``{"assistant_text": ..., "speaker_id":
..., "scene_id": ..., "user_turn_id": ..., "assistant_event_id":
...}`` so callers can introspect the produced beat (HTTP route maps
to a ``204``; future SSE rebroadcast may use the dict directly).
Raises ``ValueError`` when there is no active meanwhile scene on
``chat_id`` — the caller (turns.py) only dispatches here after a
positive ``list_meanwhile_scenes`` lookup, but the defensive raise
keeps the controller usable in isolation.
"""
chat = get_chat(conn, chat_id)
if chat is None:
raise ValueError(f"chat not found: {chat_id}")
scenes = list_meanwhile_scenes(conn, chat_id, status="active")
if not scenes:
raise ValueError(f"no active meanwhile scene on chat: {chat_id}")
scene = scenes[0]
scene_id = scene["id"]
host_bot_id = chat["host_bot_id"]
guest_bot_id = chat.get("guest_bot_id")
if guest_bot_id is None:
# A meanwhile scene without a guest is a schema violation —
# the projector requires both ids on meanwhile_scene_started.
raise ValueError(
f"meanwhile scene {scene_id} on chat {chat_id} lacks a guest"
)
host_bot = get_bot(conn, host_bot_id)
guest_bot = get_bot(conn, guest_bot_id)
if host_bot is None or guest_bot is None:
raise ValueError(
f"meanwhile bots missing: host={host_bot_id} guest={guest_bot_id}"
)
# 1. Parse the user prose with the classifier — same shape as the
# normal turn flow so OOC-stripping, segment-typing, etc. all work.
parsed = await parse_turn(
client, model=settings.classifier_model, prose=prose
)
prompt_prose = _strip_ooc_for_prompt(parsed)
# 2. Append user_turn — kept on the chat-wide log so the user can
# see their own prose in the timeline. Tagged with the meanwhile
# scene_id so future renderers can group it with the right scene.
user_turn_event_id = append_event(
conn,
kind="user_turn",
payload={
"chat_id": chat_id,
"prose": prose,
"segments": [s.model_dump() for s in parsed.segments],
"meanwhile_scene_id": scene_id,
},
)
# 3. Alternate the speaker. First turn -> host speaks; each
# subsequent turn -> the OTHER bot from the previous beat. Lookup
# is scoped by ``meanwhile_scene_id`` so unrelated assistant_turns
# on the same chat don't perturb the alternation.
last_speaker = _last_meanwhile_speaker(conn, chat_id, scene_id)
if last_speaker is None or last_speaker == guest_bot_id:
speaker_bot = host_bot
addressee_bot = guest_bot
else:
speaker_bot = guest_bot
addressee_bot = host_bot
# 4. Placeholder marker so SSE observers see "in flight". No
# projector handler is registered for this kind — it's transcript-
# only, same as the normal turn flow.
append_event(
conn,
kind="assistant_turn_started",
payload={
"chat_id": chat_id,
"speaker_id": speaker_bot["id"],
"user_turn_id": user_turn_event_id,
"meanwhile_scene_id": scene_id,
},
)
# 5. Build the narrative prompt. ``present_set_kind="host_guest"``
# tells the assembler to drop the "you" activity bullet and any
# speaker -> "you" edge — both irrelevant inside a private beat.
# Addressee is the OTHER bot, not "you".
recent_dialogue = _read_recent_meanwhile_dialogue(conn, chat_id, scene_id)
if recent_dialogue and recent_dialogue[-1].get("speaker") == "you":
recent_dialogue = recent_dialogue[:-1]
messages = assemble_narrative_prompt(
conn,
chat_id=chat_id,
speaker_bot_id=speaker_bot["id"],
addressee=addressee_bot["id"],
user_turn_prose=prompt_prose if prompt_prose else None,
recent_dialogue=recent_dialogue,
budget_soft=settings.narrative_budget_soft,
budget_hard=settings.narrative_budget_hard,
guest_id=guest_bot_id,
present_set_kind="host_guest",
)
# 6. Stream + accumulate. Same SSE pattern as the normal flow —
# tokens publish under the speaker's id so the UI can label the
# right bubble. Register the streaming task in the chat-keyed
# in-flight registry so POST /chats/<id>/turns/cancel can call
# ``.cancel()`` on it; without this, the Stop button is a no-op for
# meanwhile beats. We import the underscore name from turns.py
# deliberately — it's the same single-process registry the cancel
# route reads, and exposing it via a public alias would require
# touching every existing call site for no behavioural gain.
from chat.web.turns import _in_flight_tasks # noqa: PLC0415
accumulated: list[str] = []
truncated = False
cancelled = False
async def _stream() -> None:
async for chunk in client.stream(
messages,
model=settings.narrative_model,
max_tokens=settings.narrative_max_tokens,
temperature=settings.narrative_temperature,
):
accumulated.append(chunk)
await publish(
chat_id,
{
"event": "token",
"text": chunk,
"speaker_id": speaker_bot["id"],
},
)
stream_task = asyncio.create_task(_stream())
_in_flight_tasks[chat_id] = stream_task
try:
await stream_task
except asyncio.CancelledError:
truncated = True
cancelled = True
except Exception:
truncated = True
finally:
# Always unregister so a subsequent turn can register a fresh
# task. Mirrors the cleanup in turns.py::post_turn.
_in_flight_tasks.pop(chat_id, None)
text = "".join(accumulated)
# 7. Append assistant_turn — tagged with meanwhile_scene_id so the
# next turn's alternation lookup can find it, and present_set_kind
# so downstream renderers / digesters can filter scope.
assistant_event_id = append_event(
conn,
kind="assistant_turn",
payload={
"chat_id": chat_id,
"speaker_id": speaker_bot["id"],
"text": text,
"truncated": truncated,
"user_turn_id": user_turn_event_id,
"meanwhile_scene_id": scene_id,
"present_set_kind": "host_guest",
},
)
# 8. Per-turn memory writes — both bots get a row with witness flags
# [you=0, host=1, guest=1]. Skipped on cancellation so we don't
# record memory for a partial beat the user never read.
if not cancelled and text.strip():
record_meanwhile_memory(
conn,
chat_id=chat_id,
host_bot_id=host_bot_id,
guest_bot_id=guest_bot_id,
narrative_text=text,
scene_id=scene_id,
chat_clock_at=chat.get("time"),
)
# 9. Post-turn state-update — exactly 2 directed pairs over the
# bot pair. No you-related pairs fire (you isn't present).
present_ids = [host_bot_id, guest_bot_id]
present_names = {
host_bot_id: host_bot["name"],
guest_bot_id: guest_bot["name"],
}
personas = {
host_bot_id: host_bot.get("persona") or "",
guest_bot_id: guest_bot.get("persona") or "",
}
prior_edges: dict[tuple[str, str], dict] = {}
for src in present_ids:
for tgt in present_ids:
if src == tgt:
continue
edge = get_edge(conn, src, tgt) or {
"affinity": 50,
"trust": 50,
"summary": "",
}
prior_edges[(src, tgt)] = edge
state_updates = await compute_state_updates_for_present(
client,
classifier_model=settings.classifier_model,
present_ids=present_ids,
present_names=present_names,
personas=personas,
prior_edges=prior_edges,
recent_dialogue=recent_dialogue,
timeout_s=settings.classifier_timeout_s,
)
last_at = chat.get("time")
for src_id, tgt_id, update in state_updates:
append_and_apply(
conn,
kind="edge_update",
payload={
"source_id": src_id,
"target_id": tgt_id,
"chat_id": chat_id,
"affinity_delta": update.affinity_delta,
"trust_delta": update.trust_delta,
"knowledge_facts": update.knowledge_facts,
"last_interaction_at": last_at,
"last_interaction_chat_id": chat_id,
},
)
# 10. SSE broadcast for the timeline UI — completion event + an HTML
# fragment for the HTMX SSE swap. Same pattern as the normal turn
# flow so the rendered transcript shows the meanwhile beat inline.
await publish(
chat_id,
{
"event": "assistant_turn_complete",
"speaker_id": speaker_bot["id"],
"text": text,
"truncated": truncated,
},
)
turn_html = _render_turn_html(speaker_bot["name"], text, role="bot")
await publish(chat_id, {"event": "turn_html", "data": turn_html})
if cancelled:
# Re-raise after the partial-turn has been recorded so callers
# see the cancel propagate (mirrors normal turn flow).
raise asyncio.CancelledError
return {
"assistant_text": text,
"speaker_id": speaker_bot["id"],
"scene_id": scene_id,
"user_turn_id": user_turn_event_id,
"assistant_event_id": assistant_event_id,
}
__all__ = ["process_meanwhile_turn"]
+287
View File
@@ -0,0 +1,287 @@
"""Shared skip-flow controllers (T62).
Both the drawer skip routes (T59) and the natural-language skip parse
(T62) call into these controllers. Keep the controllers free of HTTP
concerns they take ``conn`` + ``client`` + ``settings`` and structured
args, append events, and return a small result dict the caller can map
to whatever response shape it owes (drawer partial, 204, 422, etc.).
``ValueError`` is the controller-level signal for caller-mappable
validation failure (bad ISO timestamp, backwards skip). The drawer
routes translate it to ``HTTP 400``; the natural-language path either
swallows it (the parser handed us a degenerate hint) or surfaces it the
same way. Anything else (LLM failure, unexpected exception) propagates
uncaught :func:`narrate_skip` already has its own deterministic
fallback for the routine LLM-down case, so a real exception here means
something we want to see.
The two controllers mirror the drawer T59 logic closely so the v1
guarantees (``time_skip_*`` lands first memory writes ride the
post-skip clock narration ``assistant_turn`` is appended last) hold
identically across the two entry points.
"""
from __future__ import annotations
from datetime import datetime, timezone
from sqlite3 import Connection
from chat.config import Settings
from chat.eventlog.log import append_and_apply, append_event
from chat.llm.client import LLMClient
from chat.services.memory_write import record_turn_memory_for_present
from chat.services.skip_narration import narrate_skip
from chat.services.synthesized_memories import synthesize_memories
from chat.state.entities import get_bot, get_you
from chat.state.world import get_activity, get_chat
def _parse_iso_time(value: str) -> datetime | None:
"""Permissive ISO 8601 parser shared with the drawer routes (T59).
``datetime.fromisoformat`` doesn't accept a trailing ``Z`` until
Python 3.11; we normalize it to ``+00:00`` so older interpreters
parse the same set of strings the drawer accepts.
"""
if not value:
return None
try:
v = value.strip()
if v.endswith("Z"):
v = v[:-1] + "+00:00"
return datetime.fromisoformat(v)
except (TypeError, ValueError):
return None
def _validate_new_time(chat: dict, new_time: str) -> None:
"""Raise ``ValueError`` if ``new_time`` is unparseable or backwards.
The drawer route maps the raised error to ``HTTP 400``; the
natural-language path may also surface it as a ``400``. Centralizing
the rule here means both entry points enforce the same invariant
(no causality-corrupting backwards jumps).
"""
new_dt = _parse_iso_time(new_time)
if new_dt is None:
raise ValueError(f"new_time must be ISO 8601, got {new_time!r}")
cur_dt = _parse_iso_time(chat.get("time") or "")
if cur_dt is not None and new_dt < cur_dt:
raise ValueError(
"new_time must not be earlier than the current chat clock"
)
async def process_elision_skip(
conn: Connection,
client: LLMClient,
settings: Settings,
*,
chat_id: str,
new_time: str,
landing_state_hint: str = "",
) -> dict:
"""Run an elision skip end-to-end.
Validates ``new_time`` against the current chat clock, appends a
``time_skip_elision`` event (chat clock advances), generates a
transition narration via :func:`narrate_skip`, and appends an
``assistant_turn`` carrying the narration. ``narrate_skip`` has its
own deterministic fallback so this never blocks on the model.
Returns ``{"assistant_text": ..., "speaker_id": ..., "skip_event_id":
..., "assistant_event_id": ...}`` so callers can introspect the
generated turn (e.g. for SSE rebroadcast or test assertions).
Raises ``ValueError`` on validation failure or when the chat row
can't be located (the drawer maps it to ``HTTP 400`` / ``404``
respectively; the natural-language path follows the same shape).
"""
chat = get_chat(conn, chat_id)
if chat is None:
raise ValueError(f"chat not found: {chat_id}")
_validate_new_time(chat, new_time)
host_bot = get_bot(conn, chat["host_bot_id"]) or {
"id": chat["host_bot_id"],
"name": "host",
"persona": "",
}
you_entity = get_you(conn) or {"name": "you"}
# The drawer route reaches into the host bot's current activity to
# surface the verb to the narration helper — we do the same so both
# entry points produce the same prose for the same chat state.
bot_activity = get_activity(conn, chat["host_bot_id"]) or {}
current_activity = (bot_activity.get("action") or {}).get("verb") or ""
narration = await narrate_skip(
client,
narrative_model=settings.narrative_model,
skip_kind="elision",
speaker_bot=host_bot,
you_name=you_entity.get("name") or "you",
current_time=chat.get("time") or "",
new_time=new_time,
current_activity=current_activity,
landing_state_hint=landing_state_hint,
timeout_s=settings.classifier_timeout_s,
)
skip_event_id = append_and_apply(
conn,
kind="time_skip_elision",
payload={"chat_id": chat_id, "new_time": new_time},
)
speaker_id = host_bot.get("id") or chat["host_bot_id"]
assistant_event_id = append_event(
conn,
kind="assistant_turn",
payload={
"chat_id": chat_id,
"speaker_id": speaker_id,
"text": narration,
"truncated": False,
},
)
return {
"assistant_text": narration,
"speaker_id": speaker_id,
"skip_event_id": skip_event_id,
"assistant_event_id": assistant_event_id,
}
async def process_jump_skip(
conn: Connection,
client: LLMClient,
settings: Settings,
*,
chat_id: str,
new_time: str,
notable_prose: str = "",
reset_activity: bool = False,
) -> dict:
"""Run a jump skip end-to-end.
Same validations as :func:`process_elision_skip`. Emits
``time_skip_jump`` *before* synthesizing memories so per-bot writes
record the post-jump chat clock (mirroring how a regular turn's
memory carries the chat clock). When ``notable_prose`` is non-empty,
runs :func:`synthesize_memories` once per present bot witness, then
fans the resulting memories out via
:func:`record_turn_memory_for_present` with ``source="synthesized"``.
Finally appends the narration ``assistant_turn``.
Returns ``{"assistant_text": ..., "speaker_id": ..., "skip_event_id":
..., "assistant_event_id": ...}``.
Raises ``ValueError`` on validation failure (caller maps to ``400``).
"""
chat = get_chat(conn, chat_id)
if chat is None:
raise ValueError(f"chat not found: {chat_id}")
_validate_new_time(chat, new_time)
host_bot = get_bot(conn, chat["host_bot_id"]) or {
"id": chat["host_bot_id"],
"name": "host",
"persona": "",
}
you_entity = get_you(conn) or {"name": "you"}
you_name = you_entity.get("name") or "you"
guest_bot_id = chat.get("guest_bot_id")
guest_bot = get_bot(conn, guest_bot_id) if guest_bot_id else None
# Emit time_skip_jump up front so subsequent memory writes ride the
# post-jump chat clock (matches the drawer T59 behavior pinned by
# test_post_skip_jump_with_notable_prose_writes_synthesized_memories).
skip_event_id = append_and_apply(
conn,
kind="time_skip_jump",
payload={
"chat_id": chat_id,
"new_time": new_time,
"reset_activity": reset_activity,
},
)
# Synthesize per-bot memories when prose is non-empty. The helper
# short-circuits on whitespace prose, but gating the loop here keeps
# the canned-LLM-queue accounting predictable for tests.
if notable_prose.strip():
present_bots: list[dict] = [host_bot]
if guest_bot is not None:
present_bots.append(guest_bot)
for bot in present_bots:
digest = await synthesize_memories(
client,
classifier_model=settings.classifier_model,
prose=notable_prose,
bot_name=bot.get("name") or "",
bot_persona=bot.get("persona") or "",
you_name=you_name,
timeout_s=settings.classifier_timeout_s,
)
for mem in digest.memories:
# ``record_turn_memory_for_present`` writes one row per
# present bot per call — we already iterate by bot here,
# so guest_bot_id=None avoids double-writing the guest's
# row when bot==guest.
record_turn_memory_for_present(
conn,
chat_id=chat_id,
host_bot_id=bot["id"],
guest_bot_id=None,
narrative_text=mem.text,
chat_clock_at=new_time,
source="synthesized",
significance=mem.significance,
)
narration = await narrate_skip(
client,
narrative_model=settings.narrative_model,
skip_kind="jump",
speaker_bot=host_bot,
you_name=you_name,
current_time=chat.get("time") or "",
new_time=new_time,
current_activity="",
landing_state_hint=notable_prose,
timeout_s=settings.classifier_timeout_s,
)
speaker_id = host_bot.get("id") or chat["host_bot_id"]
assistant_event_id = append_event(
conn,
kind="assistant_turn",
payload={
"chat_id": chat_id,
"speaker_id": speaker_id,
"text": narration,
"truncated": False,
},
)
return {
"assistant_text": narration,
"speaker_id": speaker_id,
"skip_event_id": skip_event_id,
"assistant_event_id": assistant_event_id,
}
def _now_iso() -> str:
"""UTC ISO timestamp used by callers as a chat-clock fallback."""
return datetime.now(timezone.utc).isoformat()
__all__ = [
"process_elision_skip",
"process_jump_skip",
"_now_iso",
"_parse_iso_time",
]
+626 -130
View File
@@ -1,32 +1,47 @@
"""POST ``/chats/<id>/turns`` — narrative turn flow with SSE streaming. """POST ``/chats/<id>/turns`` — narrative turn flow with SSE streaming.
The turn flow strings together the pieces built in T17 (turn parser), T18 The turn flow strings together the pieces built in T17 (turn parser), T18
(prompt assembler), and T16 (SSE channel): (prompt assembler), and T16 (SSE channel). Phase 2 (T44) extends it to
multi-entity scenes with optional guest support and a follow-on
interjection beat.
1. Parse the user's prose with the classifier into typed segments. 1. Parse the user's prose with the classifier into typed segments.
2. Append a ``user_turn`` event capturing both the original prose and the 2. Append a ``user_turn`` event capturing both the original prose and the
parsed segments. parsed segments.
3. Append a placeholder ``assistant_turn_started`` marker so observers know 3. Append a placeholder ``assistant_turn_started`` marker so observers know
a response is in flight. a response is in flight.
4. Build the narrative prompt, dropping OOC segments before they reach the 4. Detect the addressee (host vs. guest) from the prose using a simple
bot (per Requirements §6.1 the OOC convention is for the author to talk word-boundary substring match see :func:`_detect_addressee_id`.
to the system, not to the in-fiction bot). 5. Build the narrative prompt for the addressee, dropping OOC segments
5. Stream tokens from the LLM, broadcasting each chunk over the chat's SSE before they reach the bot (per Requirements §6.1 the OOC convention is
for the author to talk to the system, not to the in-fiction bot).
6. Stream tokens from the LLM, broadcasting each chunk over the chat's SSE
channel as a ``token`` event so any subscribed browser tab sees them channel as a ``token`` event so any subscribed browser tab sees them
arrive in real time. arrive in real time.
6. On stream complete, append an ``assistant_turn`` event with the full 7. On stream complete, append an ``assistant_turn`` event with the full
text and ``truncated=False``. Then run a post-turn state-update pass text and ``truncated=False``. Then run a post-turn state-update pass
(Requirements §3.4): one classifier call per directed edge between (Requirements §3.4): one classifier call per directed edge between
present entities, each producing an ``edge_update`` event with present entities, each producing an ``edge_update`` event with
affinity/trust/knowledge deltas. Finally publish a ``turn_html`` affinity/trust/knowledge deltas.
event with a ready-to-swap HTML fragment so HTMX's SSE extension can 8. When a guest is present, run the interjection classifier (§6.2). If it
append it to the timeline without a page reload. fires we stream a second narrative as the silent witness, append a
7. Return ``204 No Content`` the SSE channel is the real conveyor of second ``assistant_turn`` event linked to the same ``user_turn_id``,
and re-run memory + state-update for the interjector. The same
in-flight task covers both halves so cancel collapses both.
9. Scene-close detection runs after the (primary + optional interjection)
beats land so the close summary sees the full closing scene. T45's
guest-aware ``apply_scene_close_summary`` writes per-POV summaries for
each present witness.
10. Publish a ``turn_html`` event for each turn so HTMX's SSE extension
can append it to the timeline without a page reload.
11. Return ``204 No Content`` the SSE channel is the real conveyor of
state, not the POST response body. state, not the POST response body.
Errors during streaming flip the assistant_turn's ``truncated`` flag to Errors during streaming flip the assistant_turn's ``truncated`` flag to
``True`` and we still commit what we received. ``asyncio.CancelledError`` ``True`` and we still commit what we received. ``asyncio.CancelledError``
is treated identically and re-raised after recording the partial turn. is treated identically and re-raised after recording the partial turn.
A cancellation mid-interjection skips the interjector's state/memory
follow-up so we don't run classifiers against a half-formed beat.
""" """
from __future__ import annotations from __future__ import annotations
@@ -34,26 +49,37 @@ from __future__ import annotations
import asyncio import asyncio
import html import html
import json import json
import re
from datetime import timedelta
from fastapi import APIRouter, Depends, Form, HTTPException, Request from fastapi import APIRouter, Depends, Form, HTTPException, Request
from fastapi.responses import HTMLResponse, RedirectResponse, Response from fastapi.responses import HTMLResponse, JSONResponse, RedirectResponse, Response
from chat.eventlog.log import append_and_apply, append_event from chat.eventlog.log import append_and_apply, append_event
from chat.services.addressee import detect_addressee
from chat.services.background import SignificanceJob from chat.services.background import SignificanceJob
from chat.services.memory_write import record_turn_memory from chat.services.event_lifecycle import detect_event_transitions
from chat.services.event_promotion import promote_completed_event
from chat.services.interjection import detect_interjection
from chat.services.memory_write import record_turn_memory_for_present
from chat.services.multi_state_update import compute_state_updates_for_present
from chat.services.prompt import assemble_narrative_prompt from chat.services.prompt import assemble_narrative_prompt
from chat.services.rewind import compute_rewind_preview, execute_rewind from chat.services.rewind import compute_rewind_preview, execute_rewind
from chat.services.scene_close import detect_scene_close from chat.services.scene_close import detect_scene_close
from chat.services.scene_summarize import apply_scene_close_summary from chat.services.scene_summarize import apply_scene_close_summary
from chat.services.state_update import compute_state_update
from chat.services.turn_parse import ParsedTurn, parse_turn from chat.services.turn_parse import ParsedTurn, parse_turn
from chat.state.edges import get_edge from chat.state.edges import get_edge
from chat.state.entities import get_bot, get_you from chat.state.entities import get_bot, get_you
from chat.state.events import list_active_events
from chat.state.meanwhile import list_meanwhile_scenes
from chat.state.world import active_scene, get_chat, get_container from chat.state.world import active_scene, get_chat, get_container
from chat.web.bots import get_conn from chat.web.bots import get_conn
from chat.web.kickoff import get_llm_client from chat.web.kickoff import get_llm_client
from chat.web.meanwhile import process_meanwhile_turn
from chat.web.pubsub import publish from chat.web.pubsub import publish
from chat.web.render import render_turn_html as _render_turn_html from chat.web.render import render_turn_html as _render_turn_html
from chat.web.skip import _parse_iso_time, process_elision_skip
router = APIRouter() router = APIRouter()
@@ -114,6 +140,84 @@ def _read_recent_dialogue(conn, chat_id: str, limit: int = 200) -> list[dict]:
return out return out
def _detect_addressee_id(
prose: str, host_bot: dict, guest_bot: dict | None
) -> str:
"""Return the bot id of the addressee for ``prose``.
Phase 2 v1 uses a simple case-insensitive whole-word match. The host
is the default addressee flips to guest only when the guest's name
appears in the prose AND the host's does not. If both names match
or neither matches, the host keeps the floor. This bias keeps the
primary speaker stable across ambiguous prose; the interjection
branch (later in the turn flow) is how the silent witness gets a word
in edgewise when warranted.
"""
if guest_bot is None:
return host_bot["id"]
host_name = host_bot.get("name") or ""
guest_name = guest_bot.get("name") or ""
host_match = bool(
host_name
and re.search(rf"\b{re.escape(host_name)}\b", prose, re.IGNORECASE)
)
guest_match = bool(
guest_name
and re.search(rf"\b{re.escape(guest_name)}\b", prose, re.IGNORECASE)
)
if guest_match and not host_match:
return guest_bot["id"]
return host_bot["id"]
def _gather_state_update_inputs(
conn,
*,
host_bot: dict,
guest_bot: dict | None,
you_entity: dict,
) -> tuple[list[str], dict[str, str], dict[str, str], dict[tuple[str, str], dict]]:
"""Collect ``(present_ids, present_names, personas, prior_edges)`` for
a multi-entity state-update pass.
Phase 2 v1 always pairs ``you`` with the host and (when present) the
guest. ``prior_edges`` falls back to the schema default 50/50 baseline
when no row exists yet that mirrors the Phase 1 single-pair flow.
Order matters: the host comes first so the directed-pair iteration
in :func:`compute_state_updates_for_present` matches the Phase 1
sequence (host->you, then you->host). Existing tests pin the canned-
response queue to that order keeping it stable means we don't
have to reshuffle test fixtures across the Phase 2 cutover.
"""
present_ids: list[str] = [host_bot["id"], "you"]
present_names: dict[str, str] = {
host_bot["id"]: host_bot["name"],
"you": you_entity.get("name") or "you",
}
personas: dict[str, str] = {
host_bot["id"]: host_bot.get("persona") or "",
"you": you_entity.get("persona") or "",
}
if guest_bot is not None:
present_ids.append(guest_bot["id"])
present_names[guest_bot["id"]] = guest_bot["name"]
personas[guest_bot["id"]] = guest_bot.get("persona") or ""
prior_edges: dict[tuple[str, str], dict] = {}
for src in present_ids:
for tgt in present_ids:
if src == tgt:
continue
edge = get_edge(conn, src, tgt) or {
"affinity": 50,
"trust": 50,
"summary": "",
}
prior_edges[(src, tgt)] = edge
return present_ids, present_names, personas, prior_edges
@router.post("/chats/{chat_id}/turns") @router.post("/chats/{chat_id}/turns")
async def post_turn( async def post_turn(
chat_id: str, chat_id: str,
@@ -137,14 +241,106 @@ async def post_turn(
detail=f"host bot not found: {chat['host_bot_id']}", detail=f"host bot not found: {chat['host_bot_id']}",
) )
guest_bot = None
guest_bot_id = chat.get("guest_bot_id")
if guest_bot_id is not None:
# T47's bot_reset cascade clears guest_bot_id from any chat that
# referenced the deleted bot, so by the time we read it here it's
# either None or a live bot id. The previous defensive
# degrade-to-1:1 block (T44) was rendered dead by T47 and removed
# in T74.4 — get_bot now returns a real row.
guest_bot = get_bot(conn, guest_bot_id)
settings = request.app.state.settings settings = request.app.state.settings
# 0. Meanwhile-mode short-circuit (T64). When an active meanwhile
# scene is running on this chat, the turn flow is entirely between
# the two bots — "you" is absent. The meanwhile controller mirrors
# the post_turn shape but with no-you semantics: present_set_kind
# ``host_guest`` in the prompt assembler, ``record_meanwhile_memory``
# for witness flags, only 2 directed pairs in the state update, and
# the assistant_turn payload tagged with ``meanwhile_scene_id`` so
# alternation lookups can scope to this scene specifically. The
# T62 skip-intent dispatch and the regular narrative path below
# are skipped — a meanwhile beat is its own self-contained flow.
if list_meanwhile_scenes(conn, chat_id, status="active"):
try:
await process_meanwhile_turn(
conn,
client,
settings,
chat_id=chat_id,
prose=prose,
)
except ValueError as exc:
raise HTTPException(status_code=400, detail=str(exc))
return Response(status_code=204)
# 1. Parse turn (classifier). # 1. Parse turn (classifier).
parsed = await parse_turn( parsed = await parse_turn(
client, model=settings.classifier_model, prose=prose client, model=settings.classifier_model, prose=prose
) )
prompt_prose = _strip_ooc_for_prompt(parsed) prompt_prose = _strip_ooc_for_prompt(parsed)
# 1a. Skip-command short-circuit (T62). The parser may classify the
# prose as a time-skip directive — in which case the regular
# narrative path (addressee detection, narrative stream, post-turn
# state-update + scene-close passes) is skipped entirely. Elision
# runs through the shared controller in :mod:`chat.web.skip`; jump
# is drawer-only for Phase 3 (the natural-language path returns
# 422 directing the user to the drawer's jump form, where they can
# supply structured ``notable_prose`` and a target time). Anything
# not matching these intents falls through to the narrative branch.
intent = getattr(parsed, "intent", "narrative") or "narrative"
if intent == "skip_jump":
# Drawer-only jump for Phase 3: parsing a free-form fiction-time
# delta out of natural language ("next morning" -> ?) is fragile
# enough that we'd rather route the user to the drawer form,
# where they pick a concrete ISO time and an optional notable-
# prose field. 422 = "request shape is understood, but the
# required structured input lives on a different surface".
return JSONResponse(
{
"error": (
"Jump skip requires the drawer's jump form for "
"notable_prose."
)
},
status_code=422,
)
if intent == "skip_elision":
# Derive ``new_time`` from the chat clock. Phase 3 stub: bump by
# 1 hour. The drawer's elision form is the structured path when
# the author wants a specific landing time; here the goal is
# "elide the dull bit" and any sensible forward step is fine —
# ``narrate_skip`` weaves the landing-state hint into the
# transition prose so the prose carries the semantic time, not
# the timestamp itself.
cur_dt = _parse_iso_time(chat.get("time") or "")
new_time = (
(cur_dt + timedelta(hours=1)).isoformat()
if cur_dt is not None
else (chat.get("time") or "")
)
try:
await process_elision_skip(
conn,
client,
settings,
chat_id=chat_id,
new_time=new_time,
landing_state_hint=getattr(parsed, "landing_state_hint", "")
or "",
)
except ValueError as exc:
# The controller raises on missing chat / bad new_time.
# Missing chat is already handled above (we'd have 404'd);
# a bad new_time here is a stub-derivation bug rather than
# user input — surface as 400 with the controller message.
raise HTTPException(status_code=400, detail=str(exc))
return Response(status_code=204)
# 2. Append user_turn event. # 2. Append user_turn event.
user_turn_event_id = append_event( user_turn_event_id = append_event(
conn, conn,
@@ -156,7 +352,33 @@ async def post_turn(
}, },
) )
# 3. Append assistant_turn_started placeholder. ``user_turn``, # 3. Determine the addressee. Done before assistant_turn_started so the
# placeholder reflects the bot the user is actually talking to (host
# in 1:1, host-or-guest in multi-entity). T74.1 routes the multi-entity
# case through the addressee classifier; the no-guest case still uses
# the substring fast-path because there is nothing to classify when
# only one bot is present (and a classifier round-trip there would
# just be throughput overhead).
if guest_bot is None:
addressee_id = _detect_addressee_id(prose, host_bot, guest_bot)
else:
decision = await detect_addressee(
client,
classifier_model=settings.classifier_model,
user_prose=prose,
host_id=host_bot["id"],
host_name=host_bot["name"],
guest_id=guest_bot["id"],
guest_name=guest_bot["name"],
timeout_s=settings.classifier_timeout_s,
)
addressee_id = decision.addressee_id
addressee_bot = (
guest_bot if (guest_bot is not None and addressee_id == guest_bot["id"])
else host_bot
)
# 4. Append assistant_turn_started placeholder. ``user_turn``,
# ``assistant_turn_started``, and ``assistant_turn`` have no registered # ``assistant_turn_started``, and ``assistant_turn`` have no registered
# projector handlers — they live in the event_log purely for transcript # projector handlers — they live in the event_log purely for transcript
# rendering — so we don't call ``project`` here. (Re-projecting now would # rendering — so we don't call ``project`` here. (Re-projecting now would
@@ -166,12 +388,15 @@ async def post_turn(
kind="assistant_turn_started", kind="assistant_turn_started",
payload={ payload={
"chat_id": chat_id, "chat_id": chat_id,
"speaker_id": host_bot["id"], "speaker_id": addressee_bot["id"],
"user_turn_id": user_turn_event_id, "user_turn_id": user_turn_event_id,
}, },
) )
# 4. Build the narrative prompt. # 5. Build the narrative prompt for the addressee. ``guest_id`` is
# passed explicitly so the prompt assembler renders the guest's
# activity / group-node block when applicable. The assembler is
# tolerant of ``guest_id is None`` so this is a no-op for 1:1 chats.
recent = _read_recent_dialogue(conn, chat_id, limit=20) recent = _read_recent_dialogue(conn, chat_id, limit=20)
# Drop the just-appended user turn from ``recent`` — it's passed as # Drop the just-appended user turn from ``recent`` — it's passed as
# ``user_turn_prose`` to the assembler and would otherwise duplicate. # ``user_turn_prose`` to the assembler and would otherwise duplicate.
@@ -180,195 +405,439 @@ async def post_turn(
messages = assemble_narrative_prompt( messages = assemble_narrative_prompt(
conn, conn,
chat_id=chat_id, chat_id=chat_id,
speaker_bot_id=host_bot["id"], speaker_bot_id=addressee_bot["id"],
user_turn_prose=prompt_prose if prompt_prose else None, user_turn_prose=prompt_prose if prompt_prose else None,
recent_dialogue=recent, recent_dialogue=recent,
budget_soft=settings.narrative_budget_soft, budget_soft=settings.narrative_budget_soft,
budget_hard=settings.narrative_budget_hard, budget_hard=settings.narrative_budget_hard,
guest_id=guest_bot_id,
) )
# 5. Stream and accumulate tokens. The stream runs as a Task so the # 6. Stream and accumulate tokens. The stream runs as a Task so the
# /turns/cancel route can invoke ``Task.cancel()`` to abort it # /turns/cancel route can invoke ``Task.cancel()`` to abort it
# mid-stream. ``accumulated`` is a closure over the inner coroutine, # mid-stream. ``accumulated`` is a closure over the inner coroutine,
# so when the await on ``stream_task`` raises CancelledError below # so when the await on ``stream_task`` raises CancelledError below
# we still see whatever tokens were appended before cancellation. # we still see whatever tokens were appended before cancellation.
accumulated: list[str] = [] primary_accumulated: list[str] = []
truncated = False primary_truncated = False
cancelled = False cancelled = False
async def _stream() -> None: async def _stream_primary() -> None:
async for chunk in client.stream( async for chunk in client.stream(
messages, messages,
model=settings.narrative_model, model=settings.narrative_model,
max_tokens=settings.narrative_max_tokens, max_tokens=settings.narrative_max_tokens,
temperature=settings.narrative_temperature, temperature=settings.narrative_temperature,
): ):
accumulated.append(chunk) primary_accumulated.append(chunk)
await publish( await publish(
chat_id, chat_id,
{ {
"event": "token", "event": "token",
"text": chunk, "text": chunk,
"speaker_id": host_bot["id"], "speaker_id": addressee_bot["id"],
}, },
) )
stream_task = asyncio.create_task(_stream()) stream_task = asyncio.create_task(_stream_primary())
_in_flight_tasks[chat_id] = stream_task _in_flight_tasks[chat_id] = stream_task
try: try:
await stream_task await stream_task
except asyncio.CancelledError: except asyncio.CancelledError:
# Preserve the partial output before letting the cancellation # Preserve the partial output before letting the cancellation
# propagate so the transcript reflects what the user actually saw. # propagate so the transcript reflects what the user actually saw.
truncated = True primary_truncated = True
cancelled = True cancelled = True
except Exception: except Exception:
# Surface as a truncated turn rather than losing the partial output. # Surface as a truncated turn rather than losing the partial output.
truncated = True primary_truncated = True
finally: finally:
# Always unregister so a subsequent turn can register a fresh task. # Always unregister so a subsequent turn can register a fresh task.
_in_flight_tasks.pop(chat_id, None) _in_flight_tasks.pop(chat_id, None)
full_text = "".join(accumulated) primary_text = "".join(primary_accumulated)
# 6. Append the assistant_turn with the final text. (See note above on # 7. Append the assistant_turn with the final text. (See note above on
# why we skip ``project`` for these transcript-only event kinds.) # why we skip ``project`` for these transcript-only event kinds.)
append_event( append_event(
conn, conn,
kind="assistant_turn", kind="assistant_turn",
payload={ payload={
"chat_id": chat_id, "chat_id": chat_id,
"speaker_id": host_bot["id"], "speaker_id": addressee_bot["id"],
"text": full_text, "text": primary_text,
"truncated": truncated, "truncated": primary_truncated,
"user_turn_id": user_turn_event_id, "user_turn_id": user_turn_event_id,
}, },
) )
# 6a. Per-turn memory write (Plan §11.1, T21). Phase 1 single-bot: # 7a. Per-turn memory write (Plan §11.1, T21 / T41). With a guest
# only the host bot has a memory store, witness flags are # present this fans out to one ``memory_written`` event per witness
# ``[you=1, host=1, guest=0]``, and ``pov_summary`` is the raw # (host + guest); without a guest it preserves the Phase 1 single
# narrative text (T27 will rewrite at scene close). Significance # write keyed on the host. Witness flags are set inside the helper.
# defaults to 1; T22's async classifier pass will overwrite it.
scene = active_scene(conn, chat_id) scene = active_scene(conn, chat_id)
_event_id, memory_id = record_turn_memory( memory_results = record_turn_memory_for_present(
conn, conn,
chat_id=chat_id, chat_id=chat_id,
host_bot_id=host_bot["id"], host_bot_id=host_bot["id"],
narrative_text=full_text, guest_bot_id=guest_bot_id,
narrative_text=primary_text,
scene_id=scene["id"] if scene else None, scene_id=scene["id"] if scene else None,
chat_clock_at=chat.get("time"), chat_clock_at=chat.get("time"),
) )
# 6b. Post-turn state-update pass (Requirements §3.4). For Phase 1 # 7b. Post-turn state-update pass (Requirements §3.4 / T40). All
# the only present entities are ``you`` and ``host_bot`` so we run # directed pairs over the present entities — 2 pairs for 1:1, 6 for
# two classifier calls — one per directed edge — and append the # 3-entity scenes. Run sequentially via the inner helper which honors
# resulting ``edge_update`` events. The recent-dialogue slice is # the Featherless 2-conn cap.
# re-read here so the pass sees the just-appended assistant turn.
# We use ``append_and_apply`` (vs append + project) because the
# edge_update handler is *not* replay-safe: re-projecting prior
# events would re-apply their deltas on top of the live row.
recent_for_update = _read_recent_dialogue(conn, chat_id, limit=10)
you_entity = get_you(conn) or {"name": "you", "persona": ""} you_entity = get_you(conn) or {"name": "you", "persona": ""}
last_at = chat.get("time") last_at = chat.get("time")
recent_for_update = _read_recent_dialogue(conn, chat_id, limit=10)
edge_b2y = get_edge(conn, host_bot["id"], "you") or { present_ids, present_names, personas, prior_edges = (
"affinity": 50, _gather_state_update_inputs(
"trust": 50, conn,
"summary": "", host_bot=host_bot,
} guest_bot=guest_bot,
update_b2y = await compute_state_update( you_entity=you_entity,
client,
model=settings.classifier_model,
source_id=host_bot["id"],
target_id="you",
source_name=host_bot["name"],
source_persona=host_bot.get("persona", ""),
target_name=you_entity.get("name", "you"),
prior_affinity=edge_b2y["affinity"],
prior_trust=edge_b2y["trust"],
prior_summary=edge_b2y.get("summary", "") or "",
recent_dialogue=recent_for_update,
) )
)
state_updates = await compute_state_updates_for_present(
client,
classifier_model=settings.classifier_model,
present_ids=present_ids,
present_names=present_names,
personas=personas,
prior_edges=prior_edges,
recent_dialogue=recent_for_update,
timeout_s=settings.classifier_timeout_s,
)
for src_id, tgt_id, update in state_updates:
append_and_apply( append_and_apply(
conn, conn,
kind="edge_update", kind="edge_update",
payload={ payload={
"source_id": host_bot["id"], "source_id": src_id,
"target_id": "you", "target_id": tgt_id,
"chat_id": chat_id, "chat_id": chat_id,
"affinity_delta": update_b2y.affinity_delta, "affinity_delta": update.affinity_delta,
"trust_delta": update_b2y.trust_delta, "trust_delta": update.trust_delta,
"knowledge_facts": update_b2y.knowledge_facts, "knowledge_facts": update.knowledge_facts,
"last_interaction_at": last_at, "last_interaction_at": last_at,
"last_interaction_chat_id": chat_id, "last_interaction_chat_id": chat_id,
}, },
) )
edge_y2b = get_edge(conn, "you", host_bot["id"]) or { # 7c. Enqueue the async significance pass (Plan §11.1, T22). The
"affinity": 50,
"trust": 50,
"summary": "",
}
update_y2b = await compute_state_update(
client,
model=settings.classifier_model,
source_id="you",
target_id=host_bot["id"],
source_name=you_entity.get("name", "you"),
source_persona=you_entity.get("persona", "") or "",
target_name=host_bot["name"],
prior_affinity=edge_y2b["affinity"],
prior_trust=edge_y2b["trust"],
prior_summary=edge_y2b.get("summary", "") or "",
recent_dialogue=recent_for_update,
)
append_and_apply(
conn,
kind="edge_update",
payload={
"source_id": "you",
"target_id": host_bot["id"],
"chat_id": chat_id,
"affinity_delta": update_y2b.affinity_delta,
"trust_delta": update_y2b.trust_delta,
"knowledge_facts": update_y2b.knowledge_facts,
"last_interaction_at": last_at,
"last_interaction_chat_id": chat_id,
},
)
# 6c. Enqueue the async significance pass (Plan §11.1, T22). The
# worker scores the just-written memory 0-3, updates significance, # worker scores the just-written memory 0-3, updates significance,
# and auto-pins on score 3 with the §8.5 soft-cap eviction rule. # and auto-pins on score 3 with the §8.5 soft-cap eviction rule.
# Enqueued before the broadcast so it's outstanding by the time the # Phase 2 picks the host's memory id as the canonical input — guest
# client sees ``turn_html`` — but the worker is async, so the user # POV memories piggyback on the same significance score (the prose
# never blocks on it. # they record is identical for v2; per-POV rewrite happens at scene
# close in T45 and downstream-of-significance).
worker = getattr(request.app.state, "background_worker", None) worker = getattr(request.app.state, "background_worker", None)
if worker is not None and memory_id is not None: host_event_memory = memory_results.get(host_bot["id"])
host_memory_id = host_event_memory[1] if host_event_memory else None
if worker is not None and host_memory_id is not None:
worker.enqueue( worker.enqueue(
SignificanceJob( SignificanceJob(
memory_id=memory_id, memory_id=host_memory_id,
narrative_text=full_text, narrative_text=primary_text,
prior_dialogue=recent_for_update, prior_dialogue=recent_for_update,
host_bot_id=host_bot["id"], host_bot_id=host_bot["id"],
) )
) )
# 6d. Scene-close detection (Plan §7.2, T26). Runs AFTER assistant_turn # 8. Interjection branch (T39 / T44). Only fires when the chat has a
# so the bot's response is the closing scene's final beat — closing # guest AND the addressee was the bot we *can* interject for (i.e.
# before narrative would force the bot to speak "in no scene", which # not the lone bot in a 1:1 chat). The silent witness is whichever
# is awkward. Hard signals only in Phase 1: container change parsed # bot didn't get the addressee slot. We only run this when the
# from prose, or explicit "fade out" / "we're done here" patterns. # primary stream actually completed — a cancelled or errored primary
# On classifier failure the service returns ``should_close=False`` # short-circuits the follow-on so we don't classifier-spam against a
# so the turn flow keeps moving; the manual close button in the # half-formed beat.
# drawer is the always-available fallback. interjection_text: str | None = None
interjection_speaker_id: str | None = None
interjection_truncated = False
if (
guest_bot is not None
and not cancelled
and not primary_truncated
and primary_text.strip()
):
# Identify the silent witness — the bot that is NOT the addressee.
if addressee_id == host_bot["id"]:
silent_witness = guest_bot
else:
silent_witness = host_bot
edge_w_to_addr = get_edge(
conn, silent_witness["id"], addressee_bot["id"]
) or {"affinity": 50, "trust": 50, "summary": ""}
edge_w_to_you = get_edge(conn, silent_witness["id"], "you") or {
"affinity": 50,
"trust": 50,
"summary": "",
}
decision = await detect_interjection(
client,
classifier_model=settings.classifier_model,
addressee_name=addressee_bot["name"],
addressee_just_said=primary_text,
silent_witness_name=silent_witness["name"],
silent_witness_persona=silent_witness.get("persona") or "",
silent_witness_edge_to_addressee=edge_w_to_addr,
silent_witness_edge_to_you=edge_w_to_you,
you_just_said=prose,
timeout_s=settings.classifier_timeout_s,
)
if decision.should_interject:
interjection_speaker_id = silent_witness["id"]
# Re-read recent_dialogue so the just-appended assistant_turn
# (the addressee's beat) is in the prompt context.
interject_recent = _read_recent_dialogue(conn, chat_id, limit=20)
if interject_recent and interject_recent[-1].get("speaker") == "you":
interject_recent = interject_recent[:-1]
interject_messages = assemble_narrative_prompt(
conn,
chat_id=chat_id,
speaker_bot_id=silent_witness["id"],
addressee=addressee_bot["id"],
user_turn_prose=prompt_prose if prompt_prose else None,
recent_dialogue=interject_recent,
budget_soft=settings.narrative_budget_soft,
budget_hard=settings.narrative_budget_hard,
guest_id=guest_bot_id,
)
interject_accumulated: list[str] = []
async def _stream_interjection() -> None:
async for chunk in client.stream(
interject_messages,
model=settings.narrative_model,
max_tokens=settings.narrative_max_tokens,
temperature=settings.narrative_temperature,
):
interject_accumulated.append(chunk)
await publish(
chat_id,
{
"event": "token",
"text": chunk,
"speaker_id": silent_witness["id"],
},
)
interject_task = asyncio.create_task(_stream_interjection())
_in_flight_tasks[chat_id] = interject_task
try:
await interject_task
except asyncio.CancelledError:
interjection_truncated = True
cancelled = True
except Exception:
interjection_truncated = True
finally:
_in_flight_tasks.pop(chat_id, None)
interjection_text = "".join(interject_accumulated)
append_event(
conn,
kind="assistant_turn",
payload={
"chat_id": chat_id,
"speaker_id": silent_witness["id"],
"text": interjection_text,
"truncated": interjection_truncated,
"user_turn_id": user_turn_event_id,
"interjection_of": addressee_bot["id"],
},
)
# Skip the downstream classifier passes if the interjection
# was cancelled mid-stream — we don't want to score a partial
# beat the user never got to read in full.
if not interjection_truncated:
# Re-run the multi-pair state update — the interjector
# adding their voice plausibly shifts edges for everyone
# in the room. Idempotent enough for v2 (deltas accumulate;
# no stale state). Re-read recent so the just-appended
# interjection turn is in scope.
recent_post_interject = _read_recent_dialogue(
conn, chat_id, limit=10
)
# Re-fetch prior edges so deltas land on the post-primary
# state rather than the pre-turn baseline.
_, _, _, prior_edges_post = _gather_state_update_inputs(
conn,
host_bot=host_bot,
guest_bot=guest_bot,
you_entity=you_entity,
)
state_updates_post = await compute_state_updates_for_present(
client,
classifier_model=settings.classifier_model,
present_ids=present_ids,
present_names=present_names,
personas=personas,
prior_edges=prior_edges_post,
recent_dialogue=recent_post_interject,
timeout_s=settings.classifier_timeout_s,
)
for src_id, tgt_id, update in state_updates_post:
append_and_apply(
conn,
kind="edge_update",
payload={
"source_id": src_id,
"target_id": tgt_id,
"chat_id": chat_id,
"affinity_delta": update.affinity_delta,
"trust_delta": update.trust_delta,
"knowledge_facts": update.knowledge_facts,
"last_interaction_at": last_at,
"last_interaction_chat_id": chat_id,
},
)
# Memory write for the interjection beat — a second pair
# of memory_written events (host + guest POVs).
interject_memory_results = record_turn_memory_for_present(
conn,
chat_id=chat_id,
host_bot_id=host_bot["id"],
guest_bot_id=guest_bot_id,
narrative_text=interjection_text,
scene_id=scene["id"] if scene else None,
chat_clock_at=chat.get("time"),
)
# T74.2: enqueue a significance pass for the interjection
# memory. Mirrors the primary-turn enqueue pattern above —
# we score on the host's memory id since the prose is
# identical across both POVs (per-POV rewrite happens at
# scene close in T45). Without this enqueue the
# interjection beat lands in memory but never gets scored,
# so it can never auto-pin even when it carries a pivotal
# moment.
interject_host_event = interject_memory_results.get(
host_bot["id"]
)
interject_host_memory_id = (
interject_host_event[1] if interject_host_event else None
)
if (
worker is not None
and interject_host_memory_id is not None
):
worker.enqueue(
SignificanceJob(
memory_id=interject_host_memory_id,
narrative_text=interjection_text,
prior_dialogue=recent_post_interject,
host_bot_id=host_bot["id"],
)
)
# 8a. Event-lifecycle detection (Phase 3, T61). Runs after the post-turn
# classifier passes (memory write + state update + optional
# interjection) and BEFORE scene-close detection. The classifier reads
# ``primary_text`` against the chat's currently-active events and
# returns a (usually empty) list of transitions. Each transition lands
# an ``event_started`` / ``event_completed`` / ``event_cancelled``
# event via ``append_and_apply`` so the events projection updates
# synchronously. A completion is followed inline by
# ``promote_completed_event`` so any structured artifacts the event
# carries (knowledge_facts, relationship_change, acquired_objects)
# land in state in the same turn — see chat/services/event_promotion.
#
# ``detect_event_transitions`` short-circuits when ``active_events``
# is empty (per T52), so chats without active events don't pay a
# classifier round-trip and existing fixtures need no extra canned
# slots.
active_events = list_active_events(conn, chat_id)
if active_events:
lifecycle_decision = await detect_event_transitions(
client,
classifier_model=settings.classifier_model,
narrative_text=primary_text,
active_events=active_events,
timeout_s=settings.classifier_timeout_s,
)
for transition in lifecycle_decision.transitions:
if transition.new_status == "active":
append_and_apply(
conn,
kind="event_started",
payload={
"event_id": transition.event_id,
"started_at": chat.get("time"),
},
)
elif transition.new_status == "completed":
append_and_apply(
conn,
kind="event_completed",
payload={
"event_id": transition.event_id,
"completed_at": chat.get("time"),
},
)
# Run promotion inline so the artifact-emitting events
# (edge_update / manual_edit) land synchronously after
# the completion. ``promote_completed_event`` is
# synchronous (no await) and skips silently when the
# event row's status isn't 'completed' — a safety net
# for races, not expected to trigger in practice.
promote_completed_event(
conn,
event_id=transition.event_id,
chat_id=chat_id,
chat_clock_at=chat.get("time"),
)
elif transition.new_status == "cancelled":
append_and_apply(
conn,
kind="event_cancelled",
payload={
"event_id": transition.event_id,
"completed_at": chat.get("time"),
},
)
# Any other ``new_status`` value falls through silently —
# the lifecycle service constrains the schema to the three
# valid transitions, and a defensive no-op here keeps the
# turn flow tolerant of unexpected outputs.
# 9. Scene-close detection (Plan §7.2, T26). Runs AFTER assistant_turn
# and the optional interjection so the bots' responses are part of
# the closing scene's final beat — closing before narrative would
# force the bot to speak "in no scene", which is awkward. Hard
# signals only in Phase 1: container change parsed from prose, or
# explicit "fade out" / "we're done here" patterns. On classifier
# failure the service returns ``should_close=False`` so the turn
# flow keeps moving; the manual close button in the drawer is the
# always-available fallback.
# #
# Skip empty prose — no signal to classify and no point spending a # Skip empty prose — no signal to classify and no point spending a
# round-trip. Skip when there's no active scene (e.g. after a prior # round-trip. Skip when there's no active scene (e.g. after a prior
# close in the same chat) — we have nothing to close. T13 (kickoff) # close in the same chat) — we have nothing to close. T13 (kickoff)
# is the only scene-opener path in v1; Phase 2-3 will handle # is the only scene-opener path in v1; Phase 2-3 will handle
# automatic re-opening with the next container. # automatic re-opening with the next container.
#
# T74.3: this branch deliberately runs even when ``cancelled`` is
# True. Close detection consumes only the user's prose (which is
# fully appended to the event_log BEFORE streaming starts) and the
# current container name; it does NOT consume the bot's output.
# A user who types "we're done here, fade out" and then hits Stop
# mid-stream still meant to close the scene — the cancelled bot
# beat doesn't invalidate that intent. Pinned by
# test_cancelled_turn_still_closes_scene_when_user_prose_signals_close.
if scene is not None and prose.strip(): if scene is not None and prose.strip():
container = None container = None
if scene.get("container_id") is not None: if scene.get("container_id") is not None:
@@ -393,11 +862,12 @@ async def post_turn(
"significance": 0, "significance": 0,
}, },
) )
# T27: per-POV summary + edge summary update + knowledge # T27 / T45: per-POV summary + edge summary update + knowledge
# promotion. Runs synchronously after the close so the # promotion for each present witness (host always; guest when
# next turn (or a subsequent GET /chats/<id>) sees the # present). Runs synchronously after the close so the next
# rewritten memories and edge summary. Tolerates classifier # turn (or a subsequent GET /chats/<id>) sees the rewritten
# failure (returns the empty default and skips the writes). # memories and edge summaries. Tolerates classifier failure
# (returns the empty default and skips the writes).
await apply_scene_close_summary( await apply_scene_close_summary(
conn, conn,
client, client,
@@ -408,22 +878,48 @@ async def post_turn(
timeout_s=settings.classifier_timeout_s, timeout_s=settings.classifier_timeout_s,
) )
# 7. Broadcast a JSON completion event (for JS consumers) and an HTML # 10. Broadcast a JSON completion event (for JS consumers) and an HTML
# fragment event (for HTMX SSE swap-into-timeline). # fragment event (for HTMX SSE swap-into-timeline). One pair per
# written assistant_turn so the timeline ends up with both the
# primary and the interjection beat in the right order.
await publish( await publish(
chat_id, chat_id,
{ {
"event": "assistant_turn_complete", "event": "assistant_turn_complete",
"speaker_id": host_bot["id"], "speaker_id": addressee_bot["id"],
"text": full_text, "text": primary_text,
"truncated": truncated, "truncated": primary_truncated,
}, },
) )
assistant_html = _render_turn_html( primary_html = _render_turn_html(
host_bot["name"], full_text, role="bot" addressee_bot["name"], primary_text, role="bot"
) )
await publish( await publish(
chat_id, {"event": "turn_html", "data": assistant_html} chat_id, {"event": "turn_html", "data": primary_html}
)
if interjection_text is not None and interjection_speaker_id is not None:
# The interjector's display name is whichever bot wasn't the
# addressee — pull it from the in-scope variable directly.
interject_speaker_name = (
host_bot["name"]
if interjection_speaker_id == host_bot["id"]
else (guest_bot["name"] if guest_bot is not None else "bot")
)
await publish(
chat_id,
{
"event": "assistant_turn_complete",
"speaker_id": interjection_speaker_id,
"text": interjection_text,
"truncated": interjection_truncated,
},
)
interject_html = _render_turn_html(
interject_speaker_name, interjection_text, role="bot"
)
await publish(
chat_id, {"event": "turn_html", "data": interject_html}
) )
if cancelled: if cancelled:
@@ -499,6 +499,8 @@ Written per witness when a scene closes. Different details, different interpreta
### Phase 2 — multi-entity ### Phase 2 — multi-entity
**Status: shipped 2026-04-26** — multi-entity scene support, guest add/remove drawer UX, guest-aware prompt assembly, multi-entity turn flow with interjection classifier, per-POV scene close summaries for every present witness, group_node initialization/update, and bot reset cascade clearing stale `chats.guest_bot_id` references all landed across the wave5 task series (see `CLAUDE.md` § "Phase 2 status" for the deliverable summary and follow-ups).
- Guest bot in chat (3-entity scene config). - Guest bot in chat (3-entity scene config).
- Interjection classifier call. - Interjection classifier call.
- Witness filtering across multiple owners. - Witness filtering across multiple owners.
@@ -508,6 +510,8 @@ Written per witness when a scene closes. Different details, different interpreta
### Phase 3 — events, skips, threads ### Phase 3 — events, skips, threads
**Status: shipped 2026-04-26** (T49T67, 19 tasks across 8 waves; schema baseline now version 11; +68 tests). See "Phase 3 status" in CLAUDE.md for the per-task breakdown.
- Events with lifecycles and scoped props. - Events with lifecycles and scoped props.
- Time skips: elision and jump. - Time skips: elision and jump.
- Active threads. - Active threads.
@@ -0,0 +1,596 @@
# Roleplay Engine — Phase 2.5 Cleanup Plan
> **For Claude:** REQUIRED SUB-SKILL: Use `superpowers-extended-cc:executing-plans` to implement this plan task-by-task. Use the parallel-dispatch pattern documented under "Parallel-Execution Strategy" for waves that fan out to multiple subagents.
**Goal:** Burn down the combined Phase 1.5 + Phase 2.5/3 backlog tracked in [`CLAUDE.md`](../../CLAUDE.md) §"Phase 1.5 cleanup backlog" and §"Phase 2.5 / 3 backlog". 15 follow-up items consolidated into 8 tasks (file-disjoint across waves) so several can run in parallel.
**Architecture:** No new architecture. Every change here is either a refactor (T68 `open_db`), a polish on an existing service/route (most tasks), or a UI affordance for state that already exists (T72 drawer edits, witness-flag editing). No new tables, no new event kinds, no schema migrations.
**Tech Stack:** Same as Phase 2. No new dependencies.
**Source-of-truth references:**
- Backlog list: [`CLAUDE.md`](../../CLAUDE.md) §"Phase 1.5 cleanup backlog" (5 items) + §"Phase 2.5 / 3 backlog" (10 items) = 15 items total.
- Conventions: [`CLAUDE.md`](../../CLAUDE.md) §"Behavioral defaults" + §"Phase 2 status".
- Phase 2 plan (style, TDD pattern, parallel-dispatch mechanics): [2026-04-26-v2-phase2-implementation.md](2026-04-26-v2-phase2-implementation.md).
- Phase 3 plan (in flight on a separate branch): [2026-04-26-v3-phase3-implementation.md](2026-04-26-v3-phase3-implementation.md).
When a task says "see §X", that's the requirements doc unless stated otherwise.
---
## Pre-flight
**Branch:** create `phase-2.5` from the latest `main` after Phase 2 has merged. If Phase 2 is still in PR review, branch off `phase-2` directly:
```bash
# Option A: after main has phase-2 merged
git checkout main && git pull && git checkout -b phase-2.5
# Option B: continue from phase-2 directly
git checkout phase-2 && git pull && git checkout -b phase-2.5
```
**Schema baseline:** Phase 2 leaves the DB at version 8. Phase 2.5 adds **no migrations**. Schema-version assertion in `tests/test_world.py` stays at 8.
**Relationship to Phase 3:** Phase 3 (`phase-3` branch, plan committed but not yet executed) uses task ids T49T67. Phase 2.5 uses **T68T75** to avoid collision regardless of merge order.
**Pinned non-negotiables (carried forward from Phases 1 + 2):**
- State changes go through the event log. Use `append_and_apply(conn, kind, payload)` for the live path; `apply_event` only after a fresh `append_event` returning the new id.
- Witness filter every memory read at SQL level (hard `WHERE` constraint; never a soft signal).
- Edges are directed; `botA → botB` and `botB → botA` are independent records.
- Per-POV scene summaries — never write omniscient narration.
- TDD: every task starts with a failing test (or, for refactors that preserve behavior, a regression test that pins the existing contract before any change).
- One commit per task minimum. Tasks that bundle 3+ small backlog items SHOULD split commits within the task — one commit per backlog item — so review can bisect cleanly.
**Verification before claiming done:** Use `superpowers-extended-cc:verification-before-completion` — run the test command, paste actual output. Don't assume green.
---
## Backlog item → task mapping
15 items consolidated into 8 tasks by **file ownership** (so each wave's tasks stay file-disjoint). Bundled tasks may split commits internally.
| # | Backlog item | Source | Task |
|---|--------------|--------|------|
| 1 | `open_db` refactor with `check_same_thread` parameter | Phase 1.5 | **T68** |
| 2 | Regenerate broadcasts `turn_html` over SSE | Phase 1.5 | **T73** |
| 3 | `bot_reset` purges orphaned "you" activity rows | Phase 1.5 | **T69** |
| 4 | Drawer edits for deferred v1 fields (edge_trust, edge_summary, memory pov_summary, knowledge_facts) | Phase 1.5 | **T72** |
| 5 | NICE trim order in prompt assembly | Phase 1.5 | **T71** |
| 6 | Interjection regenerate | Phase 2.5 | **T73** |
| 7 | Classifier-based addressee detection | Phase 2.5 | **T74** |
| 8 | LLM-merged group meta-summary | Phase 2.5 | **T70** |
| 9 | First-meeting gate (drawer "have they met?" toggle) | Phase 2.5 | **T72** |
| 10 | Witness flag editing in drawer | Phase 2.5 | **T72** |
| 11 | Significance for interjection memories | Phase 2.5 | **T74** |
| 12 | Stale guest reference defensive degrade removal | Phase 2.5 | **T73 + T74** (split by file) |
| 13 | Scene close on cancel review | Phase 2.5 | **T74** |
| 14 | Dual `ACTIVITIES:` block consolidation | Phase 2.5 | **T71** |
| 15 | Witness role hardcode in prompt assembly | Phase 2.5 | **T71** |
| — | Docs sweep — remove shipped items from CLAUDE.md | (this plan) | **T75** |
---
## Parallel-Execution Strategy
Same pattern as Phases 2 and 3. Five waves: parallel within each wave (file-disjoint), serial across waves. Cross-wave merges keep `phase-2.5` green between dispatches.
### How to dispatch a wave in parallel
Use the **Agent tool with `isolation: "worktree"`** so each subagent gets its own git worktree. (If the controlling session's working directory is **not** the chat repo, create worktrees manually with `git worktree add .worktrees/<wave>-<task> -b <wave>/<task> phase-2.5` from inside the chat repo and pass the worktree path explicitly into each subagent prompt — that is the pattern Phase 2 used.)
In a single message, dispatch all tasks in the wave:
```
Agent({
description: "Wave 1 — T68 open_db refactor",
subagent_type: "general-purpose",
isolation: "worktree",
prompt: "<full task text from below>",
})
Agent({ ...T69... })
Agent({ ...T70... })
```
### After a wave completes
1. Each subagent returns its worktree path and commit SHA(s).
2. **Run a spec + code-quality reviewer subagent on each completed task.** Combined review is acceptable for purely mechanical refactors (T68, T69); separate spec + quality reviewers for tasks that bundle multiple backlog items (T71, T72, T74).
3. **Merge the wave into `phase-2.5`** in any order (file-disjointness guarantees no conflict). Use `--no-ff`:
```bash
git checkout phase-2.5
for branch in <wave-branches>; do
git merge --no-ff "$branch" -m "merge: <task description>"
done
```
4. **Run the full test suite** on the merged `phase-2.5`. If it's red, the wave's mutual-independence assumption was violated — bisect the offending pair, fix, re-merge.
5. **Push `phase-2.5`** to gitea so the work is durable before the next wave starts.
6. Optionally clean up worktrees: `git worktree remove .worktrees/<branch>` and `git branch -D <branch>`.
### Conflict prevention checklist (apply before dispatch)
For each parallel wave, verify the **Files** sections of all tasks have **no overlapping paths**. The waves below are designed to satisfy this; if you decide to add or merge tasks, re-check.
The hot files in this plan are: `chat/web/turns.py`, `chat/services/regenerate.py`, `chat/web/drawer.py`, `chat/templates/_drawer.html`, `chat/services/prompt.py`. Each is owned by exactly one task in this plan.
### Failure recovery
If one subagent fails: cancel it, merge the others' successful work, re-dispatch the failed task as a single follow-up. Don't block the wave.
If a failure exposes a bad assumption shared by multiple tasks (e.g., a refactor that requires a wider blast radius than the plan accounted for), pause the wave and revisit.
### Why each wave is parallel-safe
| Wave | Tasks | Hot files touched | Disjoint? |
|------|-------|-------------------|-----------|
| 1 | T68, T69, T70 | `chat/db/connection.py` + `chat/web/bots.py` (T68); `chat/state/entities.py` (T69); `chat/services/scene_summarize.py` (T70) | ✅ |
| 2 | T71 | `chat/services/prompt.py` | (single task) |
| 3 | T72 | `chat/web/drawer.py` + `chat/templates/_drawer.html` | (single task) |
| 4 | T73, T74 | `chat/services/regenerate.py` (T73); `chat/web/turns.py` + new `chat/services/addressee.py` (T74) | ✅ |
| 5 | T75 | `CLAUDE.md` | (single task) |
---
## Task overview
```
Wave 1 ─┬─ T68: open_db refactor with check_same_thread param
├─ T69: bot_reset purges orphaned "you" activity rows
└─ T70: LLM-merged group meta-summary
Wave 2 ─── T71: prompt.py polish (NICE trim order + dual ACTIVITIES + witness role parametric)
Wave 3 ─── T72: drawer.py polish (deferred v1 edits + first-meeting gate + witness flag editing)
Wave 4 ─┬─ T73: regenerate.py polish (turn_html SSE + interjection regenerate + stale-guest cleanup)
└─ T74: turn-flow polish + addressee service (classifier addressee detection +
significance for interjection + scene close on cancel + stale-guest cleanup)
Wave 5 ─── T75: docs sweep — remove shipped items from CLAUDE.md backlogs
```
Critical path: 5 sequential merge points. Total tasks: 8. Wall-clock parallelism advantage: Waves 1 and 4 dispatch concurrently; Waves 2, 3, 5 are single-task by file constraint.
---
## Wave 1 — Independent small fixes (parallel)
Three tasks, fully file-disjoint.
### Task 68: `open_db` refactor with `check_same_thread` parameter
**Files:**
- Modify: `chat/db/connection.py` (extend `open_db(path, *, check_same_thread=True)` so callers can opt out of SQLite's main-thread requirement)
- Modify: `chat/web/bots.py` (use the new parameter in `get_conn` rather than hand-rolling its own context-manager body)
- Modify: tests in `tests/test_connection.py` (or wherever `open_db` is tested; add 1 test for the new parameter)
**Spec:** Currently `chat/web/bots.py:get_conn()` duplicates the body of `open_db` so it can pass `check_same_thread=False`. Extend `open_db` to accept this as a kwarg (default True, preserving existing behavior). Then have `get_conn` call `open_db(...)` directly. The PRAGMA setup (WAL, foreign_keys, synchronous, etc.) stays in one place.
**Step 1: failing test** — add a regression test that pins the existing contract:
```python
def test_open_db_default_uses_check_same_thread_true(tmp_path):
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
# Default is check_same_thread=True; calling from another thread should fail.
...
def test_open_db_can_disable_check_same_thread(tmp_path):
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db, check_same_thread=False) as conn:
# Same conn callable from another thread now.
...
```
**Step 3: implementation** — add `check_same_thread: bool = True` to `open_db`. Pass through to `sqlite3.connect`. Then in `chat/web/bots.py`, replace the duplicated context-manager body with `open_db(path, check_same_thread=False)`.
**Step 5: commit** — `refactor: open_db with check_same_thread parameter (T68)`.
**Notes for implementer:**
- This is a refactor — the full test suite must be GREEN before AND after. Run before to baseline, run after to confirm no regressions. Pay special attention to `tests/test_bots.py` if it exercises the `get_conn` path.
- Do NOT change the default. Existing callers don't pass `check_same_thread` and must continue to get `True`.
---
### Task 69: `bot_reset` purges orphaned "you" activity rows
**Files:**
- Modify: `chat/state/entities.py` (extend `_apply_bot_reset` with one more `DELETE` clause for "you" activity rows tied to chats that this bot hosted)
- Modify: tests in `tests/test_reset.py` (add 2 tests)
**Spec:** Currently `_apply_bot_reset` purges the bot's chats, the bot's own activity rows, the bot's memories, and edges involving the bot. Phase 2 T47 added a `chats.guest_bot_id` cascade. Still missing: when bot A's chats are deleted, "you"-owned activity rows that were associated with those chats' containers are not cleaned up. They linger as orphaned activity entries pointing at deleted containers.
The fix per the existing CLAUDE.md note:
```sql
DELETE FROM activity
WHERE entity_id = 'you'
AND container_id IN (SELECT id FROM containers WHERE chat_id IN (
SELECT id FROM chats WHERE host_bot_id = ?
));
```
Order matters: this `DELETE` must run BEFORE the `DELETE FROM containers` and `DELETE FROM chats` clauses — otherwise the subqueries return no rows. Verify ordering in the existing handler before placing the new line.
**Tests:** 2 added.
1. `test_reset_purges_orphaned_you_activity_rows`: seed bot_a, chat_bot_a, a container in chat_bot_a, and a "you" activity row pointing at that container. Reset bot_a. Assert `SELECT COUNT(*) FROM activity WHERE entity_id = 'you'` is 0.
2. `test_reset_does_not_purge_you_activity_in_other_chats`: seed bot_a + bot_b, both with chats and "you" activity in each. Reset bot_a. Assert "you" activity in chat_bot_a is gone, but "you" activity in chat_bot_b is preserved.
**Commit:** `fix: bot_reset purges orphaned 'you' activity rows (T69)`.
---
### Task 70: LLM-merged group meta-summary
**Files:**
- Modify: `chat/services/scene_summarize.py` (replace the naive `f"{host_name}: {host_summary}\n\n{guest_name}: {guest_summary}"` with an LLM-merged group view via a new classifier wrapper)
- Modify: tests in `tests/test_per_pov_summary.py` (replace the regression test for naive concat with one that asserts the merged text uses the classifier output; keep the existing per-POV memory tests intact)
**Spec:** Phase 2 T45 wrote a stub for `group_node.summary` that just concatenated the two per-POV summaries. Replace it with a small classifier call that produces a coherent group-level summary from both POVs.
Add a new helper at the bottom of `scene_summarize.py`:
```python
class GroupMetaSummary(BaseModel):
summary: str = ""
dynamic: str = ""
async def merge_group_summary(
client: LLMClient,
*,
classifier_model: str,
host_name: str,
host_pov_summary: str,
guest_name: str,
guest_pov_summary: str,
timeout_s: float = 30.0,
) -> GroupMetaSummary:
"""Merge two per-POV scene summaries into a coherent group-level
summary + group-dynamic note. Falls back to the naive concat on
classifier failure."""
```
System prompt: "Given two per-POV scene summaries from a 3-entity scene (you + host + guest), produce a coherent group-level summary capturing the shared events as both witnesses experienced them, plus a brief 'dynamic' note describing the trio's group dynamic during the scene." Output strict JSON matching schema. Default = `GroupMetaSummary(summary=f"{host_name}: {host_pov_summary}\n\n{guest_name}: {guest_pov_summary}", dynamic="")` (the existing naive concat preserved as fallback so a classifier failure doesn't degrade behavior).
In `apply_scene_close_summary`, replace the naive concat call site (the existing `summary=` kwarg of the `group_node_updated` event) with `await merge_group_summary(...)` and use its `.summary` and `.dynamic` outputs.
**Tests:** 3 in `tests/test_per_pov_summary.py`.
1. `test_group_summary_merges_per_pov_via_classifier_when_guest_present`: mock the classifier with `GroupMetaSummary(summary="merged summary", dynamic="warm rapport")`. Close a scene with guest. Assert `get_group_node(...).summary == "merged summary"` and `.dynamic == "warm rapport"`.
2. `test_group_summary_falls_back_to_naive_concat_on_classifier_failure`: mock classifier with bad JSON across all 3 retries. Close scene. Assert `summary` matches the old naive concat format. `dynamic` is empty.
3. `test_group_summary_skipped_when_no_guest`: no-guest path unchanged — `group_node_updated` not emitted at all (existing behavior).
**Commit:** `feat: LLM-merged group meta-summary (T70)`.
---
## Wave 2 — `prompt.py` polish (single task)
T71 bundles three prompt-assembly cleanups. All touch `chat/services/prompt.py`. Single task because the file is hot; the implementer SHOULD split into 3 commits within the task for clean review bisection.
### Task 71: prompt.py polish (NICE trim order + dual ACTIVITIES + witness role parametric)
**Files:**
- Modify: `chat/services/prompt.py`
- Modify: `tests/test_prompt.py` (add tests; preserve existing 10 tests)
**Spec:** Three independent cleanups bundled because the file is hot.
#### 71.1 — Witness role parametric (Phase 2.5 backlog #15)
`chat/services/prompt.py:436` (or wherever the call site is — verify) calls `search_memories(conn, speaker_bot_id, "host", query, k=4)` with `witness_role="host"` hardcoded. This is wrong when the speaker is the guest (the guest queries with `witness_role="guest"` should hit a different SQL filter).
Fix: derive the role from chat membership.
```python
def _witness_role_for(speaker_bot_id: str, host_bot_id: str) -> str:
return "host" if speaker_bot_id == host_bot_id else "guest"
```
Apply at the call site. The test contract is already pinned in `tests/test_witness_filter_multi.py` from Phase 2 T46 — those tests will continue to pass; this change unblocks guest-as-speaker in production.
**Commit:** `fix: witness role parametric in prompt assembly (T71.1)`.
#### 71.2 — Dual `ACTIVITIES:` block consolidation (Phase 2.5 backlog #14)
T43 (Phase 2) added a second `ACTIVITIES:` block to render guest activity separately from you+speaker activity (so the trim ladder could drop guest activity first under tight budget). Two consecutive `ACTIVITIES:` headers can read as a duplicate-section bug to the LLM.
Refactor to a single `ACTIVITIES:` block with three bullets (you, speaker, guest), where each bullet is independently trimmable: under tight budget, drop the guest bullet first, then the you bullet, keeping the speaker bullet (the speaker's own current activity is MUST-tier).
Implementation: the existing trim machinery uses block-level granularity. Extend it to bullet-level granularity for this block (one new helper or one new tier name like `MUST-bullet` / `SHOULD-bullet` / `NICE-bullet` — pick whichever is least disruptive).
**Commit:** `refactor: single ACTIVITIES: block with bullet-level trim (T71.2)`.
#### 71.3 — NICE trim order revisit (Phase 1.5 backlog #5)
Per T18 review: the NICE trim drops previous-scene first instead of last (the spec listing order was previous-scene last). Greedy-cuts heuristic vs. spec.
Revisit: review the trim ordering carefully. If real play surfaces a regression (the previous-scene block is genuinely important to bot continuity), reverse the NICE order so previous-scene drops last. If not, document the intentional deviation in a code comment and call it done.
**This is a judgment call.** Default action: leave the order as-is and add a comment explaining why (the heuristic is "drop the cheapest-impact thing first; greedy lookahead is more expensive than the marginal narrative loss"). If review feedback during execution disagrees, reverse the order.
**Commit:** `chore: document NICE trim order rationale (T71.3)` OR `fix: NICE trim order drops previous-scene last (T71.3)`.
#### Tests for T71
Add to `tests/test_prompt.py`:
1. `test_speaker_is_guest_uses_guest_witness_role`: speaker=guest_id. Patch `search_memories` to record its `witness_role` argument. Assert called with `"guest"`, not `"host"`.
2. `test_single_activities_block_with_three_bullets_when_3_entities`: 3-entity prompt. Assert exactly one `ACTIVITIES:` header present. Assert bullets for you, speaker, guest.
3. `test_tight_budget_drops_guest_activity_bullet_first`: 3-entity prompt with budget tight enough to force trim. Assert speaker activity bullet survives, guest activity bullet is dropped.
4. (Optional, depends on 71.3 outcome) `test_nice_trim_order_drops_previous_scene_last`: only add if you choose to fix the order.
**Verification gates:**
- `pytest tests/test_prompt.py -v` — 10 existing + 3-4 new all pass.
- `pytest tests/test_witness_filter_multi.py -v` — Phase 2 T46 tests still pass (proves the witness-role fix didn't break anything).
- Full suite green.
---
## Wave 3 — `drawer.py` polish (single task)
T72 bundles three drawer affordances. All touch `chat/web/drawer.py` and `chat/templates/_drawer.html`. Single task by file constraint; implementer SHOULD split into 3 commits.
### Task 72: drawer polish (deferred v1 edits + first-meeting gate + witness flag editing)
**Files:**
- Modify: `chat/web/drawer.py` (add 4-5 new POST routes for the deferred v1 edits + 1 GET extension for first-meeting gate + 1 POST for witness flag editing)
- Modify: `chat/templates/_drawer.html` (forms for each new edit affordance)
- Create: `tests/test_drawer_edits_extended.py` (new tests for the new routes; existing `tests/test_drawer_edits.py` and `tests/test_drawer_guest.py` stay unchanged)
**Spec:** Three independent backlog items.
#### 72.1 — Deferred v1 drawer edits (Phase 1.5 backlog #4)
The `manual_edit` projector already supports `target_kind` values for `edge_trust`, `edge_summary`, `memory_pov_summary`. These work end-to-end at the state layer; only the drawer routes are missing.
Add 4 new POST routes:
1. `POST /chats/{chat_id}/drawer/edge/trust` — form `{source_id, target_id, new_value}` (0100 int). Appends `manual_edit` with `target_kind="edge_trust"`, `prior_value=current_trust`, `new_value=...`. Validate range; 400 on out-of-bounds.
2. `POST /chats/{chat_id}/drawer/edge/summary` — form `{source_id, target_id, new_summary}` (text). Appends `manual_edit` with `target_kind="edge_summary"`. No validation beyond non-empty + reasonable length cap (e.g., 2000 chars).
3. `POST /chats/{chat_id}/drawer/memory/pov-summary` — form `{memory_id, new_summary}`. Appends `manual_edit` with `target_kind="memory_pov_summary"`. 404 if memory not in this chat or not owned by a present bot.
4. `POST /chats/{chat_id}/drawer/edge/knowledge-facts` — form `{source_id, target_id, action: 'add'|'remove', fact: str}`. Knowledge_facts needs a NEW dispatch branch in the `manual_edit` projector — add it as part of this task: `target_kind="edge_knowledge_fact"` with payload action + fact.
The existing drawer template has read-only renders for these fields. Replace with editable forms (textarea + slider + button).
Tests in `tests/test_drawer_edits_extended.py`:
- One test per route (4 tests minimum) asserting: the manual_edit event lands; the projected state changes; the response contains the updated drawer partial.
**Commit:** `feat: drawer edits for edge_trust / edge_summary / memory_pov_summary / knowledge_facts (T72.1)`.
#### 72.2 — First-meeting gate (Phase 2.5 backlog #9)
The "Add guest" form's `relationship_prose` textarea fires every time. In Phase 2 T42's notes: "fire it every time a `(host, guest)` pair has no existing `host → guest` edge."
Implement the gate: when the user opens the Add-guest form, check whether `get_edge(conn, host_bot_id, guest_bot_id)` already exists. If yes:
- Render the textarea disabled with the message "they already know each other (edge exists from a prior chat)" + a small "re-seed anyway" toggle that re-enables the textarea.
- If the user submits without toggling, skip the relationship-seed call (existing edge content stays).
- If the user toggles re-seed and submits prose, the existing flow runs — `seed_inter_bot_edges` produces deltas, two `edge_update` events fire on top of the existing edge content.
Tests:
1. `test_add_guest_form_disables_prose_when_edge_exists`: pre-seed a host→guest edge from a prior chat; render the form; assert the textarea has `disabled` attribute AND the "they already know each other" message is in the body.
2. `test_add_guest_with_existing_edge_skips_seed_call`: pre-seed edge; submit form without toggling re-seed; assert classifier mock was NOT called (count check on canned-response queue).
**Commit:** `feat: first-meeting gate on drawer Add-guest form (T72.2)`.
#### 72.3 — Witness flag editing (Phase 2.5 backlog #10)
Memories show witness flags `[you, host, guest]` read-only in the drawer. Add an inline-edit affordance: each flag becomes a checkbox; toggling submits a `manual_edit` event with `target_kind="memory_witness"`, payload `{memory_id, flag: 'you'|'host'|'guest', new_value: bool}`.
The `manual_edit` projector needs a new dispatch branch for `memory_witness` — same as the knowledge_facts branch in 72.1; do them together if cleaner.
Tests: 2.
1. `test_witness_flag_toggle_updates_memory_row`: seed memory with witness `[1, 1, 0]`. POST toggle on `guest` flag → 1. Project. Assert `memories.witness_guest = 1`.
2. `test_witness_flag_toggle_emits_manual_edit_event`: same setup; assert the manual_edit event has the right `target_kind` and `prior_value`/`new_value`.
**Commit:** `feat: drawer witness flag inline-edit (T72.3)`.
---
## Wave 4 — Turn-flow polish (parallel)
Two tasks, file-disjoint. T73 owns `chat/services/regenerate.py`; T74 owns `chat/web/turns.py` + adds a new addressee-detection service.
Each task bundles multiple backlog items. Implementer should split commits within each task.
### Task 73: `regenerate.py` polish
**Files:**
- Modify: `chat/services/regenerate.py`
- Modify: `tests/test_regenerate.py` (add tests; existing tests preserved)
**Spec:** Three regenerate-related backlog items.
#### 73.1 — Regenerate broadcasts `turn_html` over SSE (Phase 1.5 backlog #2)
After the new `assistant_turn` lands, broadcast a `turn_html` event over the chat's pub/sub channel — mirror the broadcast logic in `chat/web/turns.py:post_turn`. The existing `post_turn` does this via `publish(chat_id, {"event": "turn_html", "html": ...})` (or similar — verify). Use the same render path so connected tabs swap the regenerated turn live, no refresh required.
Test: `test_regenerate_broadcasts_turn_html_over_sse` — mock `publish` and assert it was called with the new `assistant_turn`'s rendered HTML.
**Commit:** `feat: regenerate broadcasts turn_html over SSE (T73.1)`.
#### 73.2 — Interjection regenerate (Phase 2.5 backlog #6)
Phase 2 T44 deferred interjection regenerate: regenerate currently only acts on the addressee turn. Extend so that when a turn group has both a primary `assistant_turn` and an `assistant_turn` flagged as `interjection_of=...`, regenerate redoes BOTH — the primary first, then the interjection (using the same interjection-decision classifier path as `post_turn`). The interjection branch may decide `should_interject=False` on the regenerate, in which case the previous interjection_turn is superseded but no new interjection is appended.
Test: `test_regenerate_with_interjection_redoes_both_turns` — seed a 3-entity scene with a prior primary + interjection; regenerate; assert two new assistant_turns land (or one new + a supersede-without-replace if the regenerated decision was "no interjection").
**Commit:** `feat: regenerate covers interjection turns (T73.2)`.
#### 73.3 — Stale-guest defensive degrade cleanup in regenerate.py (Phase 2.5 backlog #12, partial)
Phase 2 T44 added a defensive degrade-to-1:1 in `regenerate.py` when `chat.guest_bot_id` points at a deleted bot. T47 fixed the root cause (resets clear the reference). The defensive degrade is now dead code.
Remove the degrade block; let the function trust that `chat.guest_bot_id` is either valid or NULL. The corresponding existing test for the defensive degrade can be removed (the bot_reset cascade test in `tests/test_reset.py` already covers the root-cause behavior).
**Commit:** `chore: remove defensive stale-guest degrade in regenerate.py (T73.3)`.
#### Verification gates
- `pytest tests/test_regenerate.py -v` — existing + new all pass.
- Full suite green.
---
### Task 74: turn-flow polish + new addressee-detection service
**Files:**
- Modify: `chat/web/turns.py`
- Create: `chat/services/addressee.py` (new classifier wrapper for addressee detection)
- Create: `tests/test_addressee.py`
- Modify: `tests/test_turn_flow.py` (add tests; existing 8 tests preserved)
**Spec:** Four turn-flow backlog items.
#### 74.1 — Classifier-based addressee detection (Phase 2.5 backlog #7)
Phase 2 T44's `_detect_addressee_id` uses a substring whole-word regex match. This is brittle: bot names that are common English words (e.g., a bot named "Sam"), names appearing inside a quoted aside ("Did you see what Sam wrote in his letter?" — addressed to host, not Sam), or fuzzy references all break it.
Replace with a small classifier call. New module `chat/services/addressee.py`:
```python
class AddresseeDecision(BaseModel):
addressee_id: str # bot id, "you", or "host" as fallback
confidence: str = "medium" # "high" | "medium" | "low"
reason: str = ""
async def detect_addressee(
client: LLMClient,
*,
classifier_model: str,
user_prose: str,
host_id: str,
host_name: str,
guest_id: str | None,
guest_name: str | None,
timeout_s: float = 30.0,
) -> AddresseeDecision:
"""Classify which present bot the user is addressing in this turn.
Defaults to host on failure or low confidence."""
```
System prompt: "Given a user's turn prose and the names of present bots, decide which bot the user is addressing. If the user is speaking to no specific bot (descriptive narration, action without dialogue), default to the host. Output strict JSON."
Default fallback (classifier failure) = `AddresseeDecision(addressee_id=host_id, confidence="low", reason="fallback")`.
In `chat/web/turns.py`, replace `_detect_addressee_id` with a call to `detect_addressee`. Keep the substring helper as a low-confidence pre-filter for the no-guest case (no LLM call needed when only one bot is present — preserves throughput).
Tests:
- `tests/test_addressee.py` (new file): 3 tests — classifier returns guest, classifier returns host, classifier failure falls back to host.
- `tests/test_turn_flow.py`: update `test_addressee_detection_routes_to_named_bot` from Phase 2 T44 to use the new classifier path. (Existing test should keep passing with the new mock orchestration; canned-response queue may need an extra slot for the addressee decision.)
**Commit:** `feat: classifier-based addressee detection (T74.1)`.
#### 74.2 — Significance for interjection memories (Phase 2.5 backlog #11)
Phase 2 T44 noted: the interjection branch's `memory_written` event doesn't enqueue a `SignificanceJob`. Wire it in: after the interjection memory write (the `record_turn_memory_for_present` call in the interjection branch), enqueue a `SignificanceJob` with the interjection's host memory id (mirror the primary turn's enqueue at the end of the primary branch).
If both host and guest memory ids exist for the interjection (as they will when both are present), enqueue once for the host id (the existing pattern for primary turns — the score applies to both POVs since the prose is identical at the time of write).
Test: `test_interjection_enqueues_significance_job` — mock the worker; trigger an interjection; assert `SignificanceJob` was enqueued with the interjection memory id.
**Commit:** `fix: enqueue significance for interjection memories (T74.2)`.
#### 74.3 — Scene close on cancel review (Phase 2.5 backlog #13)
Phase 2 T44 review noted: when a primary turn is cancelled mid-stream, scene close still runs. Behavior may be intentional (close detection looks at user prose, not bot output) or wrong (a cancelled turn is incomplete; closing the scene on it is premature).
**Decision for this task:** review the call path. If the close detection truly only consults user prose AND the user prose is fully present at the moment of cancel (it is — user prose is appended before the stream starts), the existing behavior is correct: a cancelled turn doesn't invalidate the user's intent to close the scene. Document this in a code comment near the close-detection branch.
If a play-test surfaces a regression (e.g., a user cancels because the bot misread their close intent), revisit. Default: document and close as a no-op.
Test: `test_cancelled_turn_still_closes_scene_when_user_prose_signals_close` — pin the existing behavior so a future refactor doesn't quietly change it.
**Commit:** `chore: pin scene-close-on-cancel behavior + comment rationale (T74.3)`.
#### 74.4 — Stale-guest defensive degrade cleanup in turns.py (Phase 2.5 backlog #12, partial)
Same as T73.3 but for `chat/web/turns.py`: T44's defensive degrade-to-1:1 in `post_turn` (lines 235-242 per the T44 implementer note) is dead code now that T47 fixed the root cause. Remove it.
**Commit:** `chore: remove defensive stale-guest degrade in turns.py (T74.4)`.
#### Verification gates
- `pytest tests/test_addressee.py -v` — 3/3 new tests pass.
- `pytest tests/test_turn_flow.py -v` — existing 8 + new 2-3 all pass.
- `pytest tests/test_reset.py -v` — Phase 2 T47 root-cause cascade still green.
- Full suite green.
---
## Wave 5 — Docs sweep (single task)
### Task 75: Remove shipped items from CLAUDE.md backlogs
**Files:**
- Modify: `CLAUDE.md`
**Spec:** Walk through the 15 backlog items in `CLAUDE.md` §"Phase 1.5 cleanup backlog" and §"Phase 2.5 / 3 backlog". For each item shipped during Phases 2.5 (T68T74), remove it from the backlog list. Add a new section "Phase 2.5 status" near the existing "Phase 2 status" section listing what shipped:
- `open_db` refactor (T68).
- `bot_reset` purges orphaned "you" activity rows (T69).
- LLM-merged group meta-summary (T70).
- Prompt assembly polish: witness role parametric, single ACTIVITIES block, NICE trim documented (T71).
- Drawer edits for deferred v1 fields, first-meeting gate, witness flag editing (T72).
- Regenerate over SSE + interjection regenerate + stale-guest cleanup (T73).
- Classifier-based addressee detection + significance for interjection + scene-close-on-cancel pinned + stale-guest cleanup (T74).
If any task during execution chose NOT to ship a sub-item (e.g., T71.3 left NICE trim unchanged with a documented rationale), keep that sub-item in a "Phase 3.5+ deferred" section with the rationale. The goal is for the backlog list to reflect actual repo state, not aspirational scope.
If any new follow-ups were discovered during T68T74 reviews, add them to the appropriate backlog section.
**Commit:** `docs: phase 2.5 status, prune shipped backlog items (T75)`.
---
## Wrap-up
After Wave 5 lands:
1. **Run full suite** on `phase-2.5`: should be ~225+ tests passing (212 from Phase 2 + ~15 new across the 8 tasks).
2. **Manual smoke** (recommended before opening the PR):
- Drawer: edit edge_trust on a chat; verify the new value sticks after refresh.
- Drawer: edit edge_summary on a chat; refresh; verify.
- Drawer: toggle a memory's witness flag; refresh; verify.
- Drawer: open Add-guest form for a (host, guest) pair that already shares an edge; verify the gate disables the prose textarea.
- Drawer: open Add-guest form for a fresh pair; verify the textarea is enabled.
- Reset a bot; verify "you" activity rows for that bot's chats are gone (run `sqlite3 data/db.sqlite "SELECT * FROM activity WHERE entity_id='you'"` before/after).
- Multi-tab: open two tabs on the same chat; click Regenerate on one; verify the other tab sees the new turn live (no refresh).
- Trigger an interjection turn; check the worker queue or `significance_jobs` table; verify a job was enqueued for the interjection memory.
- Use a bot with a name that's a common word ("Sam"); ask "did you see what Sam wrote?" — verify host gets the floor (classifier addressee detection, not substring).
3. **Push `phase-2.5`** to gitea.
4. **Open PR** `phase-2.5 → main`.
5. **No new Phase 3+ backlog items expected** — if review surfaces any, add to CLAUDE.md.
---
## Notes for the controller running this plan
- **Don't dispatch Wave 4 until Wave 3 is merged AND tested green on `phase-2.5`.** T74 references the new addressee service path that's stand-alone, but the existing tests in `tests/test_turn_flow.py` may have shifted from Wave 3 if the drawer-test fixture interactions touch shared state. Verify green before fanning out.
- **After each parallel wave**, run a code-review subagent (`subagent-driven-development` skill's two-stage review pattern) on each task. For purely mechanical tasks (T68, T69), combined spec+quality is acceptable. For bundled tasks (T71, T72, T74), use separate spec + quality reviewers — the surface area is larger.
- **If Phase 3 (`phase-3` branch) is in flight in parallel**, T75 (the docs sweep) should land on `phase-2.5` only — Phase 3's docs sweep (T67) is independent. Both will resolve when the two branches merge to `main` in some order; expect a small CLAUDE.md merge to reconcile any overlapping backlog edits.
- **If a task's "split commits" guidance proves impractical** (e.g., bundling means a test pins 3 fixes at once), one consolidated commit is acceptable. The split is an aid for review bisection, not a hard rule.
- **Token-spend rough estimate**: Phase 2.5 should be ~50% the size of Phase 2 (smaller scope, all reuse). Per-task token spend similar to Phase 2's smaller tasks (T36, T37, T47).
- **DO NOT break existing v1 / v2 surface contracts.** Every test file that was green at the start of Phase 2.5 must stay green at the end. The `tests/test_witness_filter_multi.py` contracts pinned in Phase 2 T46 are particularly load-bearing for T71.1 — verify them after the witness-role parametric fix lands.
@@ -0,0 +1,15 @@
{
"planPath": "docs/plans/2026-04-26-v2.5-phase2.5-cleanup.md",
"tasks": [
{"id": 68, "subject": "T68: open_db refactor with check_same_thread parameter", "status": "pending", "wave": 1, "parallelGroup": "wave-1"},
{"id": 69, "subject": "T69: bot_reset purges orphaned 'you' activity rows", "status": "pending", "wave": 1, "parallelGroup": "wave-1"},
{"id": 70, "subject": "T70: LLM-merged group meta-summary", "status": "pending", "wave": 1, "parallelGroup": "wave-1"},
{"id": 71, "subject": "T71: prompt.py polish (NICE trim + dual ACTIVITIES + witness role)", "status": "pending", "wave": 2, "parallelGroup": null},
{"id": 72, "subject": "T72: drawer polish (deferred v1 edits + first-meeting gate + witness flag editing)", "status": "pending", "wave": 3, "parallelGroup": null},
{"id": 73, "subject": "T73: regenerate.py polish (turn_html SSE + interjection regenerate + stale-guest cleanup)", "status": "pending", "wave": 4, "parallelGroup": "wave-4", "blockedBy": [72]},
{"id": 74, "subject": "T74: turn-flow polish + addressee service (classifier addressee + significance interjection + scene close on cancel + stale-guest cleanup)", "status": "pending", "wave": 4, "parallelGroup": "wave-4", "blockedBy": [72]},
{"id": 75, "subject": "T75: docs sweep — remove shipped items from CLAUDE.md", "status": "pending", "wave": 5, "parallelGroup": null, "blockedBy": [73, 74]}
],
"lastUpdated": "2026-04-26T00:00:00Z",
"notes": "8 tasks across 5 waves consolidating 15 backlog items (5 from Phase 1.5, 10 from Phase 2.5/3). Waves 1 and 4 are parallel-safe (file-disjoint within each). Waves 2, 3, 5 are single-task by hot-file constraint (prompt.py, drawer.py, CLAUDE.md). Bundled tasks (T71, T72, T74) split into sub-commits per backlog item for clean review bisection. No schema migrations — schema baseline stays at version 8. Phase 3 plan uses T49-T67; this plan uses T68-T75 to avoid id collision regardless of merge order."
}
@@ -0,0 +1,891 @@
# Roleplay Engine — Phase 3 Implementation Plan
> **For Claude:** REQUIRED SUB-SKILL: Use `superpowers-extended-cc:executing-plans` to implement this plan task-by-task. Use the parallel-dispatch pattern documented under "Parallel-Execution Strategy" for waves that fan out to multiple subagents.
**Goal:** Add events with lifecycles, time skips (elision + jump), active threads, significance/retrieval refinements, and "Meanwhile…" scenes (host+guest with no "you" present). All scoped to a single chat; the cross-chat surface remains unchanged.
**Architecture:** Builds on Phase 2's event-sourced architecture and 3-entity scene support. New event kinds (`event_planned`, `event_started`, `event_completed`, `event_cancelled`, `event_expired`, `time_skip_elision`, `time_skip_jump`, `thread_opened`, `thread_updated`, `thread_closed`, `meanwhile_scene_started`, `meanwhile_scene_closed`, `synthesized_memories`) carry the new state changes. Two new tables (`events`, `threads`) hold lifecycle state. Existing handlers (`memory_written`, `edge_update`) gain new payload sources without changes — promotion logic lives in services, not in projector handlers.
**Tech Stack:** Same as Phase 2 (Python 3.11+, FastAPI, HTMX, SQLite, Featherless). No new dependencies.
**Source-of-truth references:**
- Phase 3 scope: requirements doc §13 "Phase 3 — events, skips, threads"
- Behavioral details: §4 (per-chat clocks), §6.3 (prompt assembly), §6.4 (drawer), §8.1 (retrieved-memory inputs), §9 ("Time, Skips, Events — Phase 3 surface"), §11 (significance & compression)
- Conventions: [../../CLAUDE.md](../../CLAUDE.md) §"Behavioral defaults" + §"Phase 2 status"
- Phase 2 plan (style, TDD pattern, parallel-dispatch mechanics): [2026-04-26-v2-phase2-implementation.md](2026-04-26-v2-phase2-implementation.md)
When a task says "see §X", that's the requirements doc unless stated otherwise.
---
## Pre-flight
**Branch:** create `phase-3` from the latest `main` after Phase 2 has merged. If Phase 2 is still in PR review, branch off `phase-2` directly:
```bash
# Option A: after main has phase-2 merged
git checkout main && git pull && git checkout -b phase-3
# Option B: continue from phase-2 directly
git checkout phase-2 && git pull && git checkout -b phase-3
```
**Schema baseline:** Phase 2 leaves the DB at version 8. Phase 3 adds two migrations: `0009_events.sql` and `0010_threads.sql`. No other migrations expected.
**Phase 2.5 backlog:** the items in CLAUDE.md §"Phase 2.5 / 3 backlog" are NOT scoped here — they should be cleaned up in a separate branch off `main` (suggested name `phase-2.5`) before or in parallel with Phase 3. None of them blocks Phase 3.
**Pinned non-negotiables (carried forward):**
- State changes go through the event log. Use `append_and_apply(conn, kind, payload)` for the live path; `apply_event` only after a fresh `append_event` returning the new id.
- Witness filter every memory read at SQL level (hard `WHERE` constraint; never a soft signal).
- Edges are directed; `botA → botB` and `botB → botA` are independent records.
- Per-POV scene summaries — never write omniscient narration. (Meanwhile scenes write per-POV summaries for both present bots; you receive a digest later, not during the scene.)
- TDD: every task starts with a failing test.
- One commit per task minimum, more if it splits naturally.
**Verification before claiming done:** Use `superpowers-extended-cc:verification-before-completion` — run the test command, paste actual output. Don't assume green.
---
## Parallel-Execution Strategy
Same pattern as Phase 2. Eight waves: parallel within each wave (file-disjoint), serial across waves. The controller (you, the controlling Claude session) merges each subagent's commits and verifies the suite stays green before dispatching the next wave.
### How to dispatch a wave in parallel
Use the **Agent tool with `isolation: "worktree"`** so each subagent gets its own git worktree. The runtime cleans up the worktree automatically if no changes are made; otherwise it returns the path + branch for the controller to merge. (If the controlling session's working directory is **not** the chat repo, create worktrees manually with `git worktree add .worktrees/<wave>-<task> -b <wave>/<task> phase-3` from inside the chat repo and pass the worktree path explicitly into each subagent prompt — that is the pattern Phase 2 used.)
In a single message, dispatch all tasks in the wave:
```
Agent({
description: "Wave 1 — T49 events table + handlers",
subagent_type: "general-purpose",
isolation: "worktree",
prompt: "<full task text from below>",
})
Agent({
description: "Wave 1 — T50 time_skip handlers",
subagent_type: "general-purpose",
isolation: "worktree",
prompt: "<full task text from below>",
})
Agent({
description: "Wave 1 — T51 threads table + handlers",
subagent_type: "general-purpose",
isolation: "worktree",
prompt: "<full task text from below>",
})
```
All subagents start simultaneously, each working on a private worktree branched off `phase-3`. They cannot see each other's changes (no shared filesystem state) — that's the safety guarantee.
### After a wave completes
1. Each subagent returns its worktree path and commit SHA.
2. **Run a spec + code-quality reviewer subagent on each completed task** (combined review is acceptable for purely mechanical schema/handler tasks; large or integration tasks like T62, T63 deserve separate spec + quality reviewers).
3. **Merge the wave into `phase-3`** in any order (file-disjointness guarantees no conflict). Use `--no-ff` so each task's history stays grouped:
```bash
git checkout phase-3
for branch in <wave-branches>; do
git merge --no-ff "$branch" -m "merge: <task description>"
done
```
4. **Run the full test suite** on the merged `phase-3`. If it's red, the wave's mutual-independence assumption was violated — bisect to find the offending pair, fix in a follow-up commit, re-merge.
5. **Push `phase-3`** to gitea so the work is durable before the next wave starts.
6. Optionally clean up worktrees: `git worktree remove .worktrees/<branch>` and `git branch -D <branch>`.
### Conflict prevention checklist (apply before dispatch)
For each parallel wave, verify the **Files** sections of all tasks have **no overlapping paths**. The waves below are designed to satisfy this; if you decide to add or merge tasks, re-check.
If a hot file (`chat/web/turns.py`, `chat/services/prompt.py`, `chat/web/drawer.py`, `chat/templates/_drawer.html`, `chat/services/regenerate.py`) needs changes from multiple tasks, do **not** parallelize them — serialize within the wave or split into separate waves.
### Failure recovery
If one subagent fails (test failures, blocked, infinite loop):
- **Do not block the wave on a failure.** Cancel the failed subagent, merge the others' successful work, and re-dispatch the failed task as a single follow-up.
- If a failure exposes a bad assumption shared by multiple tasks (e.g. an event-payload schema mismatch), pause the wave and revisit the plan.
### Why each wave is parallel-safe
| Wave | Tasks | Hot files touched | Disjoint? |
|------|-------|-------------------|-----------|
| 1 | T49, T50, T51 | new SQL migrations + new state modules; T50 also extends `chat/state/world.py` (additive) | ✅ |
| 2 | T52, T53, T54, T55 | new service modules only | ✅ |
| 3 | T56, T57, T58 | new service module (T56) + `chat/state/memory.py` retrieval extension (T57) + `chat/services/scene_summarize.py` (T58) | ✅ |
| 4 | T59 | `chat/web/drawer.py`, `chat/templates/_drawer.html` | (single task) |
| 5a | T60, T61 | `chat/services/prompt.py` (T60), `chat/web/turns.py` (T61) | ✅ |
| 5b | T62 | `chat/web/turns.py`, plus a new skip route module | (single task; depends on 5a) |
| 6 | T63, T64, T65 | meanwhile is tightly coupled — see Wave 6 sub-structure below | ⚠️ partial |
| 7 | T66, T67 | new test file + docs only | ✅ |
**Wave 6 sub-structure:** T63 is schema/state (new files); T64 is service + extends `chat/web/turns.py`; T65 is service + extends `chat/services/prompt.py`. T64 and T65 are file-disjoint relative to each other but both depend on T63's schema landing first. Dispatch as: T63 alone → merge → T64+T65 in parallel → merge.
---
## Task overview
```
Wave 1 ─┬─ T49: events table + lifecycle handlers
├─ T50: time_skip event kinds + handlers (advance chat clock)
└─ T51: threads table + open/update/close handlers
Wave 2 ─┬─ T52: event-lifecycle detection service (narrative → state changes)
├─ T53: skip narration service (elision + jump prose)
├─ T54: synthesized-memories service (jump skip "anything notable?")
└─ T55: thread-detection service (on scene close, identify open threads)
Wave 3 ─┬─ T56: event-completion promotion (inventory / edges / memories)
├─ T57: significance retrieval ranking refinements
└─ T58: scene compression keeps key quotes when significance ≥ 2
Wave 4 ─── T59: drawer additions — events panel, threads panel, skip controls
Wave 5a ─┬─ T60: prompt assembly includes active events + active threads
└─ T61: turn flow invokes event-detection + thread-update per turn
Wave 5b ─── T62: skip command surface (parse + route + jump UI prompt)
Wave 6 ─┬─ T63: meanwhile scene config — schema + state + scene-config-4 marker
└─ (after T63 merges)
├─ T64: meanwhile turn flow (host+guest, no "you")
└─ T65: meanwhile summary digest (briefs you on next active scene)
Wave 7 ─┬─ T66: cross-feature integration tests (events × skips × threads × meanwhile)
└─ T67: Phase 3 documentation update
```
Critical path: 8 sequential merge points (Waves 1, 2, 3, 4, 5a, 5b, 6a, 6b, 7). Total tasks: 19. Wall-clock parallelism advantage depends on subagent dispatch overhead, but in principle each wave's tasks can run concurrently in ~the time of one task.
---
## Wave 1 — Schema & state foundation
These three tasks are **fully independent**: each adds a new SQL migration + new state module. T50 also adds two handlers to `chat/state/world.py` (additive, alongside Phase 2's `_apply_guest_added`).
### Task 49: Events table + lifecycle handlers
**Files:**
- Create: `chat/db/migrations/0009_events.sql`
- Create: `chat/state/events.py`
- Create: `tests/test_events_state.py`
**Spec:** Adds the `events` table and projector handlers for the lifecycle: `event_planned`, `event_started`, `event_completed`, `event_cancelled`, `event_expired`. Each event row carries `chat_id`, `kind` (free-form domain-event tag like `"date_at_park"`), `status` (`planned|active|completed|cancelled|expired`), `props_json` (arbitrary blob), `planned_for` (ISO-8601 chat-clock string, optional), `started_at` / `completed_at` (chat-clock strings).
**Step 1: failing test** — see pattern in `tests/test_group_node.py` (Phase 2 T36). Three tests minimum:
1. `test_event_planned_creates_row`: append `event_planned` with `kind`, `props_json`, `planned_for`; project; assert `get_event(conn, event_id)` returns the row with `status="planned"`.
2. `test_event_started_then_completed_updates_status`: append `event_planned``event_started``event_completed`; assert `status` transitions and `completed_at` populated.
3. `test_event_cancelled_terminal`: append `event_planned``event_cancelled`; assert `status="cancelled"`. A subsequent `event_started` is ignored (handler no-op when status is terminal).
**Step 3: implementation** — `0009_events.sql`:
```sql
CREATE TABLE events (
id INTEGER PRIMARY KEY,
chat_id TEXT NOT NULL,
kind TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'planned',
props_json TEXT NOT NULL DEFAULT '{}',
planned_for TEXT,
started_at TEXT,
completed_at TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX events_chat_idx ON events(chat_id, status);
```
`chat/state/events.py`:
- `@on("event_planned")` inserts a new row with status `planned`. Payload provides a stable `event_id` (caller-allocated UUID) so the projector is idempotent.
- `@on("event_started")` updates status to `active` and sets `started_at` from payload (or current chat clock).
- `@on("event_completed")`, `@on("event_cancelled")`, `@on("event_expired")` each move to the named terminal state and stamp `completed_at` (the column doubles as "ended at").
- `get_event(conn, event_id)`, `list_active_events(conn, chat_id)`, `list_events_in_status(conn, chat_id, status)` readers.
- All handlers no-op when the row is already in a terminal state (idempotent re-projection safety).
**Step 5: commit** — `feat: events table + lifecycle handlers (T49)`.
**Notes for the implementer:**
- Use UUID-style ids (e.g., `f"evt_{uuid.uuid4().hex[:12]}"`) created by the caller; pass as `event_id` in payload. Don't auto-generate inside the projector.
- Schema version after this migration alone: 9. The full Phase 3 baseline is 10 (T51 adds 0010_threads.sql).
- `tests/test_world.py::test_schema_version_after_migration_is_8` will need to bump after Wave 1 merges — handle in the wave-merge step (mirrors Phase 2 T36's pattern).
---
### Task 50: Time-skip event kinds + chat-clock handlers
**Files:**
- Modify: `chat/state/world.py` (add `_apply_time_skip_elision`, `_apply_time_skip_jump`; both update `chats.time` and may reset `activity` rows)
- Create: `tests/test_time_skip_handlers.py`
**Spec:** Two new event kinds.
- `time_skip_elision` payload: `{chat_id, new_time}`. Handler updates `chats.time = ?`. Activity rows are NOT reset (the activity that was elided to its end-state is the resolution itself; the caller passes a follow-up `activity_changed` event when needed).
- `time_skip_jump` payload: `{chat_id, new_time, reset_activity: bool}`. Handler updates `chats.time = ?`; if `reset_activity` is true, deletes per-chat `activity` rows for the participants in that chat (a fresh landing state will be set by a follow-up `activity_changed` event from the skip service).
These are pure state mutations. T54 and T62 fire them via `append_and_apply`.
**Tests:** 3 minimum.
1. `test_elision_advances_chat_clock_only`: seed chat at time T0; append `time_skip_elision` with `new_time=T1`; project; assert `get_chat(...)["time"] == T1` and activity unchanged.
2. `test_jump_with_reset_clears_activity`: seed chat with one activity row; append `time_skip_jump` with `reset_activity=True`; assert chat clock advanced AND activity table empty for that chat.
3. `test_jump_without_reset_preserves_activity`: same seed; `reset_activity=False`; assert activity row still present and clock advanced.
**Implementation:** new handlers next to `_apply_chat_created` in `chat/state/world.py`. Use the same parameterized SQL patterns. Do NOT add UI here — T62 wires the skip command flow.
**Commit:** `feat: time_skip event handlers (T50)`.
---
### Task 51: Threads table + open/update/close handlers
**Files:**
- Create: `chat/db/migrations/0010_threads.sql`
- Create: `chat/state/threads.py`
- Create: `tests/test_threads_state.py`
**Spec:** Adds the `threads` table and projector handlers for `thread_opened`, `thread_updated`, `thread_closed`. A thread is a per-chat narrative continuity tag — open during scenes, surfaced to prompt assembly so successor scenes can reference unresolved arcs.
`0010_threads.sql`:
```sql
CREATE TABLE threads (
id INTEGER PRIMARY KEY,
chat_id TEXT NOT NULL,
title TEXT NOT NULL,
summary TEXT NOT NULL DEFAULT '',
status TEXT NOT NULL DEFAULT 'open', -- open | closed
opened_at TEXT NOT NULL DEFAULT (datetime('now')),
closed_at TEXT,
last_referenced_scene_id INTEGER,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX threads_chat_status_idx ON threads(chat_id, status);
```
`chat/state/threads.py`:
- `@on("thread_opened")` payload: `{thread_id, chat_id, title, summary?}`. Inserts a new row with `status='open'`.
- `@on("thread_updated")` payload: `{thread_id, summary, last_referenced_scene_id?}`. Updates summary + optional last-referenced-scene pointer.
- `@on("thread_closed")` payload: `{thread_id, closed_at?}`. Sets status='closed', stamps `closed_at`.
- Readers: `get_thread(conn, thread_id)`, `list_open_threads(conn, chat_id)`, `list_threads(conn, chat_id, status=None)`.
**Tests:** 3 minimum.
1. `test_thread_opened_creates_row`.
2. `test_thread_updated_changes_summary_and_last_referenced`.
3. `test_thread_closed_terminal`: subsequent `thread_updated` is ignored (matches the design's "closed threads are kept for replay but don't surface in prompt").
**Note:** the Phase 2 `group_node.threads_json` column was a Phase-3 placeholder and is NOT used as authoritative storage now — `threads` table is the source of truth. The drawer can choose to render either, but Phase 3 onward should treat the table as canonical and treat `group_node.threads_json` as a deprecated cache that we leave alone (or clear in the next migration).
**Commit:** `feat: threads table + projector handlers (T51)`.
---
## Wave 2 — Classifier services (parallel)
Four tasks, all new service modules — fully file-disjoint.
### Task 52: Event-lifecycle detection service
**Files:**
- Create: `chat/services/event_lifecycle.py`
- Create: `tests/test_event_lifecycle.py`
**Spec:** A classifier-wrapped service that inspects a freshly-narrated turn and decides whether any active events transitioned this turn (started, completed, cancelled). Returns a structured `EventLifecycleDecision` with one or more `EventTransition(event_id, new_status, reason)` items, or empty when nothing changed.
Schema:
```python
class EventTransition(BaseModel):
event_id: str
new_status: str # "active" | "completed" | "cancelled"
reason: str = ""
class EventLifecycleDecision(BaseModel):
transitions: list[EventTransition] = Field(default_factory=list)
```
Public API:
```python
async def detect_event_transitions(
client: LLMClient,
*,
classifier_model: str,
narrative_text: str,
active_events: list[dict], # [{id, kind, status, props}, ...] from list_active_events
timeout_s: float = 30.0,
) -> EventLifecycleDecision:
"""Decide whether any active events transitioned this turn. Conservative
bias — most turns return empty transitions. Trigger only when the
narrative text clearly resolves or starts a known active event.
"""
```
Caller (T61 turn flow) appends one `event_started` / `event_completed` / `event_cancelled` event per transition via `append_and_apply`.
**Tests:** 3 minimum — happy path with one transition, empty active_events short-circuits without classifier call, classifier failure returns empty default.
**Commit:** `feat: event-lifecycle detection service (T52)`.
---
### Task 53: Skip narration service
**Files:**
- Create: `chat/services/skip_narration.py`
- Create: `tests/test_skip_narration.py`
**Spec:** Generates the brief transition narration that bridges a time skip. Two flavors mirroring §9:
- **Elision:** "skip to when we arrive". Input: current activity ("walking to park"), expected end-state ("at the park, sitting on a bench"). Output: 1-2 sentence transition prose narrated from the host bot's POV. New chat-clock value is provided by the caller.
- **Jump:** "next morning". Input: time delta + landing-state hint (optional). Output: 2-3 sentences setting the scene at the new time.
Public API:
```python
async def narrate_skip(
client: LLMClient,
*,
narrative_model: str,
skip_kind: str, # "elision" | "jump"
speaker_bot: dict, # {id, name, persona}
you_name: str,
current_time: str,
new_time: str,
current_activity: str,
landing_state_hint: str = "",
timeout_s: float = 60.0,
) -> str:
"""Generate brief transition prose. Returns plain text, not JSON."""
```
Uses `client.generate(...)` (not `classify`) since output is free-form prose. Falls back to a deterministic template string on failure (e.g., `f"({new_time}: {landing_state_hint or current_activity}.)"`). The fallback ensures the skip flow never blocks even when the LLM is down.
**Tests:** 3 minimum — happy elision, happy jump, generation failure returns fallback string with the new time visible.
**Commit:** `feat: skip narration service (T53)`.
---
### Task 54: Synthesized-memories service
**Files:**
- Create: `chat/services/synthesized_memories.py`
- Create: `tests/test_synthesized_memories.py`
**Spec:** When the user does a jump skip ("a week later") they're prompted "anything notable happen?" If they answer with prose, this service parses that prose into 1-N synthesized memories per present bot. Each memory carries `source="synthesized"`, `reliability=0.7`, witness mask `[1, 1, 0]` or `[1, 1, 1]` per present set, and a one-sentence text body.
Schema:
```python
class SynthesizedMemory(BaseModel):
text: str
significance: int = 1 # 0..3, default 1
affinity_delta: int = 0
trust_delta: int = 0
class SynthesizedDigest(BaseModel):
memories: list[SynthesizedMemory] = Field(default_factory=list)
```
Public API:
```python
async def synthesize_memories(
client: LLMClient,
*,
classifier_model: str,
prose: str,
bot_name: str, # which witness's POV
bot_persona: str,
you_name: str,
timeout_s: float = 30.0,
) -> SynthesizedDigest:
"""Parse 'anything notable happen?' prose into structured memories
from a single bot's POV. Empty/whitespace prose short-circuits."""
```
Caller (T62 skip flow) calls this once per present bot (host always; guest if present), then writes via `record_turn_memory_for_present` with `source="synthesized"` and the synthesized text in place of narrative_text.
**Tests:** 3 minimum — happy path returns parseable memories, empty prose short-circuits, classifier failure returns empty digest.
**Commit:** `feat: synthesized-memories service for jump skips (T54)`.
---
### Task 55: Thread-detection service
**Files:**
- Create: `chat/services/thread_detection.py`
- Create: `tests/test_thread_detection.py`
**Spec:** On scene close, classify the scene transcript to detect open threads (unresolved arcs, dangling questions, promises made). Returns a list of `ThreadCandidate(title, summary, action: "open"|"update"|"close", existing_thread_id?)`.
The service receives the current set of open threads so it can decide to **update** an existing thread rather than open a duplicate. It can also signal **close** when the transcript clearly resolves an open thread.
Schema:
```python
class ThreadCandidate(BaseModel):
action: str # "open" | "update" | "close"
title: str = "" # required for "open"; ignored otherwise
summary: str = ""
existing_thread_id: str | None = None # required for "update"/"close"
class ThreadDetectionResult(BaseModel):
candidates: list[ThreadCandidate] = Field(default_factory=list)
```
Public API:
```python
async def detect_threads(
client: LLMClient,
*,
classifier_model: str,
scene_transcript: list[dict], # [{speaker, text}, ...]
open_threads: list[dict], # [{id, title, summary}, ...]
timeout_s: float = 30.0,
) -> ThreadDetectionResult:
"""Classify scene close into thread open/update/close candidates."""
```
Caller (T58 scene compression — added in Wave 3) loops over candidates and emits one `thread_opened`, `thread_updated`, or `thread_closed` event per candidate.
**Tests:** 3 minimum — opens a new thread, updates an existing thread (test asserts `existing_thread_id` is honored), classifier failure returns empty.
**Commit:** `feat: thread-detection service (T55)`.
---
## Wave 3 — Promotion & retrieval refinements
Three tasks. T56 is a new service module (event-completion promotion). T57 modifies `chat/state/memory.py` to add a significance-aware retrieval rank. T58 modifies `chat/services/scene_summarize.py` to integrate compression hints + the thread-detection service from T55. File-disjoint.
### Task 56: Event-completion promotion
**Files:**
- Create: `chat/services/event_promotion.py`
- Create: `tests/test_event_promotion.py`
**Spec:** When an event reaches `completed` (the only terminal state that promotes; cancelled/expired do NOT promote per §9 last paragraph), the orchestrator promotes any structured artifacts the event carried into the appropriate target store:
- `event.props.acquired_objects: list[str]` → append `inventory_added` events (Phase 4 schema; Phase 3 stub: just append a `manual_edit` with `target_kind="memory_pov_summary"` describing the acquisition into the host's memory).
- `event.props.knowledge_facts: list[{owner_id, target_id, fact}]` → append `edge_update` events with the facts on the named directed edge.
- `event.props.relationship_change: {summary, source_id, target_id}` → append `manual_edit` with `target_kind="edge_summary"` for that pair.
- Everything else stays in the closed event record (the projector kept the row; no further promotion).
Public API:
```python
def promote_completed_event(
conn,
*,
event_id: str,
chat_id: str,
chat_clock_at: str | None,
) -> dict:
"""Read the completed event's props_json and emit promotion events.
Returns a summary dict {inventory: int, knowledge: int, relationship: int}
of how many promotion events fired. No classifier calls — purely
structural. Skips if event status isn't 'completed'."""
```
This is **synchronous** (no async, no LLM). It reads a row, parses JSON, emits events via `append_and_apply`.
**Tests:** 4 minimum — empty props no-op, knowledge_facts produces edge_update events, relationship_change produces manual_edit, cancelled-event-doesn't-promote.
**Commit:** `feat: event-completion promotion service (T56)`.
---
### Task 57: Significance-aware retrieval ranking
**Files:**
- Modify: `chat/state/memory.py` (extend `search_memories(conn, owner_id, witness_role, query, k)` to add a significance bias to the rank ordering)
- Modify: `tests/test_memory_search.py` (or wherever the existing search tests live; add 2 tests)
**Spec:** Currently `search_memories` orders by FTS rank only. §11.1 says "Retrieval ranking: significance multiplier applied as `score × constant` to FTS / vector rank." Phase 3 implements this for FTS only (vector retrieval is Phase 4).
Change the SQL `ORDER BY` from `ORDER BY rank` to `ORDER BY (rank + significance * 0.5) DESC` (or whatever scaling produces sane results — this is a tuning knob, document the choice in a comment). The constant may need adjustment after manual play; surface it as a module-level constant `SIGNIFICANCE_RANK_BIAS`.
**Tests:** 2 added.
1. `test_higher_significance_outranks_equal_rank`: seed two memories with identical FTS-matching text but different significance scores; assert the higher-significance row appears first in results.
2. `test_significance_bias_is_constant_module_level`: verify the constant is accessible as `chat.state.memory.SIGNIFICANCE_RANK_BIAS` (so it's tunable without a code change in calling sites).
**Commit:** `feat: significance-aware retrieval ranking (T57)`.
---
### Task 58: Scene compression keeps key quotes when significance ≥ 2
**Files:**
- Modify: `chat/services/scene_summarize.py` (extend `apply_scene_close_summary` to also call `detect_threads` from T55 and emit thread events; extend the per-POV summary to include up to 3 verbatim "key quotes" from the closing scene when scene-max-significance ≥ 2)
- Modify: `tests/test_per_pov_summary.py` (add 3 tests for the new behavior)
**Spec:** §11.1 specifies "Compression: scenes with max-turn-significance ≥ 2 retain key quotes; ≤ 1 collapse fully into the per-POV summary." Implement this:
- Compute scene max significance from `memories.significance` rows in this scene.
- When max < 2: existing behavior unchanged (per-POV summary, no extra quotes).
- When max ≥ 2: include up to 3 verbatim quote spans (each ≤ 200 chars) in the per-POV summary text. Format: append `\n\nKey quotes:\n- "..."\n- "..."` to the summary. The `summarize_scene` classifier already produces the prose; the quote-selection step is a deterministic post-process that picks the top-3 highest-significance turn texts from the scene transcript (truncated).
Additionally, after writing per-POV summaries (existing behavior), call `detect_threads` (from T55) once per close. For each candidate emit the matching `thread_opened` / `thread_updated` / `thread_closed` event via `append_and_apply`. Failures fall back to no thread changes (existing memory + edge updates still land).
**Tests:** 3 added.
1. `test_low_significance_scene_omits_quotes`: max significance = 1; assert summary text contains no "Key quotes:" header.
2. `test_high_significance_scene_includes_top_3_quotes`: seed 4 memories with significance 3, 2, 1, 2; assert summary contains the top-3 (by significance) verbatim turn texts.
3. `test_thread_detection_emits_events`: stub `detect_threads` to return one `ThreadCandidate(action="open", ...)`; assert a `thread_opened` event landed.
**Commit:** `feat: significance-driven quote retention + thread emission on close (T58)`.
---
## Wave 4 — Drawer additions (single task)
This wave is one task because all Phase 3 drawer additions touch `chat/web/drawer.py` and `chat/templates/_drawer.html` together — splitting would force serial execution with conflicts.
### Task 59: Drawer events / threads / skip controls
**Files:**
- Modify: `chat/web/drawer.py` (extend `GET /chats/{chat_id}/drawer`; add `POST /chats/{chat_id}/drawer/event/plan`, `/drawer/event/cancel/{event_id}`, `/drawer/skip/elision`, `/drawer/skip/jump`, `/drawer/thread/close/{thread_id}`)
- Modify: `chat/templates/_drawer.html` (3 new sections: Events, Threads, Skip controls)
- Create: `tests/test_drawer_events_threads_skip.py`
**Spec:**
**GET extension:**
- `list_active_events(conn, chat_id)` → render in a new "Events" section.
- `list_open_threads(conn, chat_id)` → render in a new "Threads" section.
- A "Skip" subsection with two buttons: "Elision skip" (opens an inline form taking a `landing_state_hint`) and "Jump skip" (opens an inline form taking `target_time` ISO + optional `notable_prose` for the synthesized-memories prompt).
**POST routes:**
1. `POST /drawer/event/plan` — form `{kind, planned_for, props_json}` → 400-validates JSON, appends `event_planned`, returns refreshed drawer.
2. `POST /drawer/event/cancel/{event_id}` — appends `event_cancelled`, returns refreshed drawer.
3. `POST /drawer/skip/elision` — form `{landing_state_hint, new_time}` → calls `narrate_skip` (T53), appends `time_skip_elision` + an `assistant_turn` carrying the narration, returns refreshed drawer + chat partial.
4. `POST /drawer/skip/jump` — form `{new_time, notable_prose, reset_activity}` → calls `narrate_skip` for transition prose, calls `synthesize_memories` (T54) for each present bot, appends `time_skip_jump` + memories + transition turn, returns refreshed drawer + chat partial.
5. `POST /drawer/thread/close/{thread_id}` — appends `thread_closed`, returns refreshed drawer.
**Template additions:**
- "Events" section listing each active event by kind + planned_for + props.
- "Threads" section listing each open thread title + summary + a Close button.
- "Skip" controls under existing Activity section.
- Forms use HTMX (`hx-post`, `hx-target="#drawer"`, `hx-swap="innerHTML"`) consistent with Phase 2 drawer patterns.
**Tests (`tests/test_drawer_events_threads_skip.py`):** 6 minimum.
1. GET drawer with no events/threads → no Events/Threads sections rendered.
2. POST event/plan with valid form → event_planned event appended; drawer body now contains the event title.
3. POST event/cancel → event_cancelled appended; drawer no longer lists the event under "Active".
4. POST skip/elision → time_skip_elision appended, chat clock advanced, narration assistant_turn present in chat history.
5. POST skip/jump with notable_prose → time_skip_jump + N synthesized memory_written events; assert reliability=0.7 on those rows.
6. POST thread/close → thread_closed appended; thread no longer in open list.
**Commit:** `feat: drawer events / threads / skip controls (T59)`.
**Notes for implementer:**
- The existing `available_guests` dropdown helper from T42 is the reference for form-population patterns.
- For the Jump skip's `notable_prose` field, treat empty as "no synthesized memories" (just advance the clock) — the spec allows this.
- Validate `target_time` ISO format; 400 on parse failure. Do not allow target_time earlier than current chat clock.
---
## Wave 5a — Prompt + turn-flow integration (parallel)
T60 modifies `chat/services/prompt.py`. T61 modifies `chat/web/turns.py`. File-disjoint.
### Task 60: Prompt assembly includes active events + active threads
**Files:**
- Modify: `chat/services/prompt.py` (extend `assemble_narrative_prompt`)
- Modify: `tests/test_prompt.py` (add 3 tests)
**Spec:** Two new SHOULD-tier blocks added between the existing scene-context block and retrieved-memories block:
1. **Active events** — title `Active events:`. Lists each active event in this chat: `- {kind} (planned for {planned_for})` plus a one-line props excerpt (truncate to ~80 chars). Trim-tier SHOULD; drops before retrieved memories under tight budget.
2. **Active threads** — title `Open threads:`. Lists each open thread: `- {title}: {summary}` (summary truncated to ~120 chars). SHOULD-tier.
Both blocks are omitted entirely when their lists are empty (no header rendered).
Per Phase 2 T43's auto-detection precedent, the function reads `list_active_events(conn, chat_id)` and `list_open_threads(conn, chat_id)` itself; no new parameters.
**Tests:** 3 added.
1. `test_assemble_with_no_events_or_threads_omits_blocks` — regression; no events/threads → assembled prompt has neither block.
2. `test_assemble_with_active_events_renders_block` — seed one event_planned + event_started; assert "Active events:" header and event kind appear in prompt.
3. `test_assemble_with_open_thread_renders_block` — seed one thread_opened; assert "Open threads:" header and thread title appear.
**Commit:** `feat: prompt assembly renders active events + open threads (T60)`.
---
### Task 61: Turn flow invokes event-detection + thread-update per turn
**Files:**
- Modify: `chat/web/turns.py` (after the primary narrative + memory + state-update block, call `detect_event_transitions` from T52; emit `event_started`/`event_completed`/`event_cancelled` events accordingly)
- Modify: `chat/services/regenerate.py` (mirror — regenerate also re-detects event transitions for the regenerated turn)
- Modify: `tests/test_turn_flow.py` (add 3 tests)
**Spec:** After the existing post-turn classifier passes (memory write, state update, interjection check) and BEFORE scene-close detection, call `detect_event_transitions` with `narrative_text=primary_text` and `active_events=list_active_events(conn, chat_id)`.
For each `EventTransition` returned:
- `new_status="active"` → append `event_started` payload `{event_id, started_at: chat.time}`.
- `new_status="completed"` → append `event_completed` payload `{event_id, completed_at: chat.time}` AND THEN call `promote_completed_event` (T56) inline so promotion events emit synchronously after completion.
- `new_status="cancelled"` → append `event_cancelled`. Promotion is skipped.
Empty transitions list = no-op (most turns; no extra events written).
`regenerate.py` mirrors the same logic for the regenerated turn (existing event transitions from the superseded turn are NOT undone — that's a Phase 3.5 follow-up; document the limitation).
**Tests:** 3 added to `tests/test_turn_flow.py`.
1. `test_turn_with_event_transition_appends_started_event`: mock `detect_event_transitions` to return one transition; assert `event_started` lands in event log; canned-response queue matches.
2. `test_turn_with_event_completion_runs_promotion`: same mock returning `new_status="completed"`; seed a planned event with knowledge_facts in props; assert `event_completed` + `edge_update` (from promotion) both land.
3. `test_turn_with_no_active_events_skips_classifier`: no active events; assert `detect_event_transitions` is never called (its canned response slot would still be in the queue at end of test).
**Commit:** `feat: per-turn event-lifecycle detection + completion promotion (T61)`.
---
## Wave 5b — Skip command flow (single task)
Single task because it modifies `chat/web/turns.py` (which Wave 5a also touched). Run after Wave 5a is merged so the file's recent additions are stable.
### Task 62: Skip command surface
**Files:**
- Modify: `chat/web/turns.py` (extend `parse_turn` to detect natural-language skip commands like "skip to the park", "next morning", "a week later" and route to a skip-handling branch BEFORE the normal narrative flow)
- Create: `chat/web/skip.py` (new module hosting `process_elision_skip(...)` and `process_jump_skip(...)` controllers; called by both turns.py and the drawer skip routes from T59)
- Modify: `tests/test_turn_flow.py` (add 3 tests)
**Spec:** Currently `parse_turn` extracts the user's prose into structured fields (addressee inferred, etc.). Phase 3 adds detection of skip commands as a separate intent.
The classifier-based parse already produces an `intent` field (or similar — verify in code). Extend the schema with `intent="skip_elision"` and `intent="skip_jump"`. When intent is one of these, the turn flow short-circuits the normal narrative path and routes to:
- `process_elision_skip(conn, client, settings, *, chat_id, landing_state_hint=parsed.landing_state)` — calls `narrate_skip(skip_kind="elision")`, appends `time_skip_elision`, `assistant_turn` carrying narration, returns 204.
- `process_jump_skip(conn, client, settings, *, chat_id, target_time=parsed.target_time, notable_prose=parsed.notable_prose)` — appends `time_skip_jump`, calls `synthesize_memories` per present bot, appends synthesized `memory_written` events, calls `narrate_skip(skip_kind="jump")`, appends `assistant_turn` carrying transition prose, returns 204.
The drawer routes from T59 share these functions (don't duplicate the logic across drawer.py and turns.py).
For Phase 3's first cut, JUMP skip's `notable_prose` is NOT collected from natural-language ("a week later, anything notable?" requires a UI prompt). Two options:
- **(simpler)** Drawer-only entry for jump skip; natural-language jump short-circuits to drawer prompt.
- **(better UX)** Natural-language jump returns a 422 with an HTMX-swap that injects the "anything notable?" textarea into the chat surface; user submits prose to a follow-up `/chats/{chat_id}/skip/jump/confirm` endpoint.
Pick the simpler path for Phase 3 (drawer-only jump). Document the second option as a Phase 3.5 polish.
**Tests:** 3 added.
1. `test_elision_skip_via_natural_language` — user prose "skip to when we arrive at the park"; assert `time_skip_elision` event landed and chat clock advanced; an `assistant_turn` carrying transition prose was appended.
2. `test_jump_skip_via_natural_language_redirects_to_drawer` — user prose "next morning"; assert response is 422 with an HTMX swap pointing at the drawer's jump form (or whatever the chosen Phase 3 fallback is).
3. `test_skip_command_does_not_run_narrative_classifier` — same user prose as test 1; assert `assemble_narrative_prompt` was NOT called for a regular bot turn (the skip path bypasses it).
**Commit:** `feat: natural-language skip detection + skip command flow (T62)`.
---
## Wave 6 — Meanwhile scenes
Phase 3's capstone feature. Most ambitious: scene config 4 (host + guest, no "you"). Per §13 the cap stays at 2 bots in any scene; meanwhile is two-bot bot↔bot. "You" receives a digest later, not during.
Decomposed into 3 tasks. T63 lands first (schema + state); then T64 + T65 in parallel.
### Task 63: Meanwhile scene config — schema + state
**Files:**
- Create: `chat/db/migrations/0011_meanwhile_scenes.sql`
- Create: `chat/state/meanwhile.py`
- Create: `tests/test_meanwhile_state.py`
**Spec:** A meanwhile scene is a special kind of scene where `present_set = {host_bot_id, guest_bot_id}` (no "you"). The existing `scenes` table can carry it via a new `present_set_kind` column distinguishing `you_host`, `you_host_guest`, `host_guest`. Alternatively, `meanwhile_scenes` is a sidecar table — pick the lower-disruption option.
**Recommended:** add a `present_set_kind` column to `scenes` (default `'you_host'` for back-compat) via migration `0011_meanwhile_scenes.sql`:
```sql
ALTER TABLE scenes ADD COLUMN present_set_kind TEXT NOT NULL DEFAULT 'you_host';
ALTER TABLE scenes ADD COLUMN parent_scene_id INTEGER; -- the active you-scene this meanwhile branched off from
CREATE INDEX scenes_present_set_idx ON scenes(chat_id, present_set_kind, status);
```
New event kinds with `chat/state/meanwhile.py` handlers:
- `@on("meanwhile_scene_started")` payload: `{chat_id, scene_id, host_bot_id, guest_bot_id, parent_scene_id, started_at}`. Inserts a new scene row with `present_set_kind="host_guest"`, links to parent.
- `@on("meanwhile_scene_closed")` payload: `{scene_id, closed_at}`. Updates status to `closed`; subsequent per-POV summary writes for both bots happen via existing scene-close path (host + guest are the "present witnesses"; "you" is excluded).
Readers: `list_meanwhile_scenes(conn, chat_id, status='active')`, `get_parent_scene(conn, scene_id)`.
**Tests:** 3 minimum.
1. `test_meanwhile_started_creates_scene_with_correct_present_set_kind`.
2. `test_meanwhile_closed_marks_scene_closed`.
3. `test_active_you_scene_can_coexist_with_active_meanwhile_scene` (one chat, two active scenes — meanwhile + the main you-scene that spawned it).
**Commit:** `feat: meanwhile scene schema + state (T63)`.
---
### Task 64: Meanwhile turn flow
**Files:**
- Modify: `chat/web/turns.py` (add meanwhile-mode detection at the start of `post_turn`; if active meanwhile scene exists for this chat, route to `process_meanwhile_turn`)
- Create: `chat/web/meanwhile.py` (new module hosting `process_meanwhile_turn(...)` controller; mirrors post_turn but with no "you" in present_set)
- Modify: `chat/services/prompt.py` (small addition: when `present_set_kind="host_guest"`, exclude "you" from edges + activity blocks; addressee is always the other bot)
- Create: `tests/test_meanwhile_turn_flow.py`
**Spec:** A meanwhile scene runs entirely between two bots. The user can advance it manually via a meanwhile-mode chat surface (T65 wires the UI), but turn-flow logic is:
1. Read active meanwhile scene; identify `speaker_bot_id` (alternates each turn — start with host, then guest, etc.) and `addressee_bot_id` (the other one).
2. Assemble narrative prompt with `speaker_bot_id`, `addressee=addressee_bot.name`, `present_set_kind="host_guest"` (so "you" is omitted from edges/activities).
3. Stream narrative; commit `assistant_turn` event with `present_set_kind="host_guest"` and `meanwhile_scene_id` populated.
4. Memory writes: BOTH host and guest get a memory_written with witness `[0, 1, 1]` (you=0; you wasn't present). Use `record_turn_memory_for_present` adapted to the no-you case (or extend it with a `you_present: bool = True` parameter).
5. State updates: 2 directed pairs (host↔guest only). Skip you-related pairs.
6. Scene close detection: same path as regular scenes; on close, per-POV summaries fire for both bots; group_node updates if applicable.
Addressee-alternation: simple — each turn alternates speaker. (Phase 3.5 may add classifier-driven turn-taking with refusals.)
**Tests:** 4 minimum.
1. `test_meanwhile_turn_writes_memories_with_witness_0_1_1`.
2. `test_meanwhile_turn_emits_2_edge_updates_only` (host→guest, guest→host).
3. `test_meanwhile_turn_alternates_speaker` (turn 1: host speaks; turn 2: guest speaks).
4. `test_meanwhile_scene_close_writes_per_pov_for_both_bots_only` (no "you" memory; existing T45 path is hit but with `you_present=False`).
**Commit:** `feat: meanwhile turn flow (host+guest, no you) (T64)`.
---
### Task 65: Meanwhile summary digest
**Files:**
- Modify: `chat/services/scene_summarize.py` (when a meanwhile scene closes, generate ALSO a "you-facing digest" — a brief narrated summary that will surface to "you" the next time the main you-scene resumes)
- Modify: `chat/services/prompt.py` (when assembling for a regular you-scene and any closed-but-not-yet-surfaced meanwhile digests exist, include them as a SHOULD-tier block titled "Meanwhile while you were away:")
- Create: `chat/state/meanwhile_digest.py` (a small state module: `meanwhile_digest_pending` table; handlers for `meanwhile_digest_created` / `meanwhile_digest_consumed`)
- Modify: `tests/test_per_pov_summary.py` and `tests/test_prompt.py` (add tests)
**Spec:** When a meanwhile scene closes (T64's path), also append `meanwhile_digest_created` with `{chat_id, scene_id, summary}`. The summary is generated via a fresh `summarize_scene` call with `bot_persona="omniscient narrator briefing the absent player"`; output is a 2-3 sentence neutral summary of what happened.
When the next you-scene starts (or the prompt is assembled for the next active you-scene's turn), `assemble_narrative_prompt` queries `list_pending_meanwhile_digests(conn, chat_id)` and:
- Includes them as a SHOULD-tier block: `"Meanwhile while you were away:\n- {summary}\n- {summary}"`.
- After they're surfaced once, the caller (T64 in the post-meanwhile turn or the first you-turn after meanwhile-close) appends `meanwhile_digest_consumed` per digest, marking them as surfaced.
Migration `0011_meanwhile_scenes.sql` (T63) can include the `meanwhile_digest_pending` table OR T65 adds a thin `0012_meanwhile_digest.sql`. Pick lower-disruption — likely add to T63's migration for simplicity. Document the choice.
(If you choose to add the table in T65 via a new migration, add `0012_meanwhile_digest.sql`. The schema-version assertion bump in `tests/test_world.py` happens once after Wave 6 merges.)
**Tests:** 3 added.
1. `test_meanwhile_close_creates_digest`: close a meanwhile scene; assert `meanwhile_digest_pending` row exists with non-empty summary.
2. `test_pending_digest_renders_in_you_scene_prompt`: seed a pending digest; assemble prompt for a you-host scene; assert the "Meanwhile while you were away:" header and summary appear.
3. `test_consumed_digest_does_not_render_again`: append `meanwhile_digest_consumed`; reassemble prompt; digest no longer appears.
**Commit:** `feat: meanwhile summary digest surfaces to next you-scene (T65)`.
---
## Wave 7 — Polish (parallel)
Two independent tasks. New test file (T66) + docs only (T67). Dispatch in parallel after Wave 6 merges.
### Task 66: Cross-feature integration tests
**Files:**
- Create: `tests/test_phase3_integration.py`
**Spec:** Phase 3 introduces a lot of cross-feature interaction surfaces. This task adds tests that exercise multi-feature flows end-to-end:
1. Plan an event → play turns → event_started detected → event_completed detected → promotion fires → memory + edge updates land.
2. Open a thread on close → next scene's prompt includes the open thread → close thread via drawer → next scene's prompt no longer includes it.
3. Jump skip → synthesized memories land per present bot → next turn's prompt retrieves them via search.
4. Meanwhile scene → close → digest pending → first you-turn prompt includes digest → after that turn, digest is consumed.
5. Meanwhile while a regular you-scene is active → both scenes have memories; querying memories for either bot at the post-meanwhile main scene correctly returns both sets witness-filtered.
5 tests minimum.
**Commit:** `test: phase 3 cross-feature integration coverage (T66)`.
---
### Task 67: Phase 3 documentation update
**Files:**
- Modify: `CLAUDE.md` (add "Phase 3 status" section; update "Behavioral defaults"; add "Phase 3.5 / 4 backlog" with carry-overs from review feedback during execution)
- Modify: `docs/plans/2026-04-26-v1-requirements-design.md` (annotate §13 "Phase 3 — events, skips, threads" as **Status: shipped <date>**)
**Spec:** Documentation-only. Run last so it captures any deviations and review-noted follow-ups discovered during execution. Reflect:
- Events with full lifecycle (planned → active → completed/cancelled/expired).
- Time skips: elision (immediate end-state) + jump (synthesized memories from "anything notable?").
- Threads opened/updated/closed; surfaced in prompt assembly + drawer.
- Significance retrieval bias + key-quote retention at significance ≥ 2.
- Meanwhile scenes: bot+bot without "you"; per-POV summaries for both bots; you-facing digest on next you-scene.
- Phase 3 known limitations / 3.5 backlog candidates:
- Natural-language jump skip falls back to drawer form (no inline "anything notable?" prompt).
- Regenerate doesn't undo prior event transitions from the superseded turn.
- Meanwhile turn-taking is alternation (no classifier-driven refusals or initiative).
- Vector retrieval is still Phase 4.
**Commit:** `docs: phase 3 status, behavioral defaults, deferred items (T67)`.
---
## Wrap-up
After Wave 7 lands:
1. **Run full suite** on `phase-3`: should be ~260+ tests passing (212 from Phase 2 + ~50 new).
2. **Manual smoke** (recommended before opening the PR):
- Plan an event from the drawer; play turns until it completes; verify promotion landed (drawer shows updated edges / memories).
- Use elision and jump skips both via natural language and the drawer.
- Close a scene that opened a thread; verify the thread renders in the next scene's prompt.
- Trigger a meanwhile scene from the drawer; play 2 turns; close it; resume the main you-scene; verify the digest renders once and not again.
3. **Push `phase-3`** to gitea.
4. **Open PR** `phase-3 → main`.
5. **Phase 3.5 backlog candidates** (track in CLAUDE.md): inline natural-language jump prompt UI, regenerate-aware event-transition undo, classifier-driven meanwhile turn-taking, drawer surface for closed-event browsing, event template library (kind presets with default props).
---
## Notes for the controller running this plan
- **Don't dispatch Wave 5b until Wave 5a is merged AND green on `phase-3`.** Wave 5b's `turns.py` modifications layer on top of T61's recent additions; missing that produces merge conflicts or import-time failures.
- **Don't dispatch T64+T65 until T63 merges.** Both depend on the new `present_set_kind` column and the meanwhile event kinds.
- **After each parallel wave**, run a code-review subagent (`subagent-driven-development` skill's two-stage review pattern) on each task before merging to `phase-3`. For purely mechanical tasks (schema migrations, projector handlers), a combined spec+quality review is acceptable. For T62, T64, T65 (large or integration tasks), use separate spec + quality reviewers.
- **If a parallel wave's merge produces a conflict**, the wave's file-disjointness assumption was violated. Bisect the affected pair, fix the offending task in a follow-up commit on `phase-3`, and proceed.
- **Schema-version test bumps** happen at Wave 1 merge (8 → 10) and Wave 6 merge (10 → 11 or 12 depending on T65's migration choice). Update `tests/test_world.py` once per affected merge — same pattern as Phase 2 T36.
- **Token-spend rough estimate**: Phase 3 should be larger than Phase 2 (~1.5×) — events / skips / meanwhile each carry their own state + service + UI surfaces. Per-task token spend similar to Phase 2's larger tasks (T42, T44).
- **DO NOT modify Phase 1 / 2 code paths** unless explicitly required (e.g., T58 modifies `scene_summarize.py` because the new behavior is genuinely additive). Existing 1- and 2-entity flows must continue to work end-to-end after each wave.
@@ -0,0 +1,26 @@
{
"planPath": "docs/plans/2026-04-26-v3-phase3-implementation.md",
"tasks": [
{"id": 49, "subject": "T49: events table + lifecycle handlers", "status": "pending", "wave": 1, "parallelGroup": "wave-1"},
{"id": 50, "subject": "T50: time_skip event kinds + chat-clock handlers", "status": "pending", "wave": 1, "parallelGroup": "wave-1"},
{"id": 51, "subject": "T51: threads table + open/update/close handlers", "status": "pending", "wave": 1, "parallelGroup": "wave-1"},
{"id": 52, "subject": "T52: event-lifecycle detection service", "status": "pending", "wave": 2, "parallelGroup": "wave-2", "blockedBy": [49]},
{"id": 53, "subject": "T53: skip narration service (elision + jump)", "status": "pending", "wave": 2, "parallelGroup": "wave-2", "blockedBy": [50]},
{"id": 54, "subject": "T54: synthesized-memories service for jump skips", "status": "pending", "wave": 2, "parallelGroup": "wave-2", "blockedBy": [50]},
{"id": 55, "subject": "T55: thread-detection service", "status": "pending", "wave": 2, "parallelGroup": "wave-2", "blockedBy": [51]},
{"id": 56, "subject": "T56: event-completion promotion service", "status": "pending", "wave": 3, "parallelGroup": "wave-3", "blockedBy": [49, 52]},
{"id": 57, "subject": "T57: significance-aware retrieval ranking", "status": "pending", "wave": 3, "parallelGroup": "wave-3"},
{"id": 58, "subject": "T58: scene compression keeps key quotes + emits thread events", "status": "pending", "wave": 3, "parallelGroup": "wave-3", "blockedBy": [55]},
{"id": 59, "subject": "T59: drawer events / threads / skip controls", "status": "pending", "wave": 4, "parallelGroup": null, "blockedBy": [49, 50, 51, 53, 54]},
{"id": 60, "subject": "T60: prompt assembly includes active events + open threads", "status": "pending", "wave": 5, "parallelGroup": "wave-5a", "blockedBy": [49, 51]},
{"id": 61, "subject": "T61: turn flow invokes event-detection + completion promotion", "status": "pending", "wave": 5, "parallelGroup": "wave-5a", "blockedBy": [52, 56]},
{"id": 62, "subject": "T62: skip command surface (parse + route + jump UI)", "status": "pending", "wave": 5, "parallelGroup": null, "blockedBy": [50, 53, 54, 60, 61]},
{"id": 63, "subject": "T63: meanwhile scene config — schema + state", "status": "pending", "wave": 6, "parallelGroup": null},
{"id": 64, "subject": "T64: meanwhile turn flow (host+guest, no you)", "status": "pending", "wave": 6, "parallelGroup": "wave-6b", "blockedBy": [63]},
{"id": 65, "subject": "T65: meanwhile summary digest surfaces to next you-scene", "status": "pending", "wave": 6, "parallelGroup": "wave-6b", "blockedBy": [63]},
{"id": 66, "subject": "T66: cross-feature integration tests", "status": "pending", "wave": 7, "parallelGroup": "wave-7", "blockedBy": [62, 64, 65]},
{"id": 67, "subject": "T67: Phase 3 documentation update", "status": "pending", "wave": 7, "parallelGroup": "wave-7", "blockedBy": [62, 64, 65]}
],
"lastUpdated": "2026-04-26T00:00:00Z",
"notes": "19 tasks across 8 waves (1, 2, 3, 4, 5a, 5b, 6a, 6b, 7). Waves 1, 2, 3, 5a, and 7 are fully parallel-safe (file-disjoint within each). Waves 4, 5b, and 6a are single-task. Wave 6b is parallel after 6a (T63) merges. Use Agent tool with isolation: 'worktree' to dispatch parallel tasks. Merge each wave's worktrees back into phase-3 before dispatching the next wave. See plan §Parallel-Execution Strategy for full guidance. Schema baseline: Phase 2 ends at version 8; Phase 3 adds 0009_events.sql, 0010_threads.sql, 0011_meanwhile_scenes.sql (final version 11)."
}
@@ -0,0 +1,709 @@
# Roleplay Engine — Phase 3.5 Cleanup Plan
> **For Claude:** REQUIRED SUB-SKILL: Use `superpowers-extended-cc:executing-plans` to implement this plan task-by-task. Use the parallel-dispatch pattern documented under "Parallel-Execution Strategy" for waves that fan out to multiple subagents.
**Goal:** Burn down the combined Phase 2.6/3 + Phase 3.5/4 backlog tracked in [`CLAUDE.md`](../../CLAUDE.md). 17 follow-up items consolidated into 12 tasks (file-disjoint where possible) across 7 waves so several can run in parallel.
**Architecture:** No new architecture, no schema migrations, no new event kinds. Every change here is either a polish on an existing service/route, a refactor for DRY/typing/observability, or a UI affordance for state that already exists (frontend `turn_html_replace` consumer).
**Tech Stack:** Same as Phase 3. No new dependencies.
**Source-of-truth references:**
- Backlog list: [`CLAUDE.md`](../../CLAUDE.md) §"Phase 2.6 / 3 backlog" (7 items, 6 actionable + 1 deferred) + §"Phase 3.5 / 4 backlog" (~13 items grouped by source review) = 17 items net.
- Conventions: [`CLAUDE.md`](../../CLAUDE.md) §"Behavioral defaults" + §"Phase 3 status".
- Phase 2.5 cleanup plan (style, file-bundling pattern): [2026-04-26-v2.5-phase2.5-cleanup.md](2026-04-26-v2.5-phase2.5-cleanup.md).
When a task says "see §X", that's the requirements doc unless stated otherwise.
---
## Pre-flight
**Branch:** create `phase-3.5` from the latest `main` after Phase 3 has merged (it has — main is at `753cec3`):
```bash
git checkout main && git pull && git checkout -b phase-3.5
```
**Schema baseline:** Phase 3 leaves the DB at version 11. Phase 3.5 adds **no migrations**. Schema-version assertion in `tests/test_world.py` stays at 11.
**Pinned non-negotiables (carried forward):**
- State changes go through the event log. Use `append_and_apply(conn, kind, payload)` for the live path; `apply_event` only after a fresh `append_event` returning the new id.
- Witness filter every memory read at SQL level (hard `WHERE` constraint; never a soft signal).
- Edges are directed; `botA → botB` and `botB → botA` are independent records.
- Per-POV scene summaries — never write omniscient narration.
- TDD: every task starts with a failing test (or a regression test pinning existing contract before refactor).
- One commit per task minimum. Tasks that bundle multiple backlog items SHOULD split commits within the task — one commit per item — so review can bisect cleanly.
**Verification before claiming done:** Use `superpowers-extended-cc:verification-before-completion` — run the test command, paste actual output. Don't assume green.
---
## Backlog item → task mapping
17 items consolidated into 12 tasks by **file ownership** (so each wave's tasks stay file-disjoint). Bundled tasks split commits internally.
| # | Backlog item | Source | Task |
|---|--------------|--------|------|
| 1 | `narrate_skip` `timeout_s` not piped through | T53 review | **T76** |
| 2 | `AddresseeDecision.confidence` should be `Literal[...]` | T74 review (Phase 2.5 carry-over) | **T77** |
| 3 | `search_memories` docstring missing SQL-bias note | T57 review | **T78** |
| 4 | `_witness_role_for` defensive coding for `host_bot_id is None` | T71 review (Phase 2.5 carry-over) | **T79** |
| 5 | Scene close re-close suffix bloat risk | T58 review | **T80** |
| 6 | Thread detection transcript scoping (chat-wide vs scene-scoped) | T58 review | **T80** |
| 7 | Swallowed `Exception` in `detect_threads` try/except — log at debug | T58 review | **T80** |
| 8 | Scene close `closed_at` clock divergence | T58 review | **T80** |
| 9 | T58 test coverage gaps (truncation, update/close, fallback) | T58 review | **T80** |
| 10 | Error-message prefix sniff for 404 vs 400 routing in skip routes | T62 review | **T81** |
| 11 | `consume_pending_meanwhile_digests` not wired into `post_turn` | T66 integration tests | **T82** |
| 12 | Skip command bypasses scene close detection | T62 review | **T82** |
| 13 | Cancel/stop hook for in-flight regenerate streams | T73 review (Phase 2.5 carry-over) | **T83** |
| 14 | DRY: regenerate vs post_turn (recent-dialogue + prior-edges duplication) | T73 review (Phase 2.5 carry-over) | **T83** |
| 15 | Sibling-discovery query optimization in regenerate.py | T73 review (Phase 2.5 carry-over) | **T83** |
| 16 | Regenerate doesn't roll back lifecycle transitions from superseded turn | T61 review | **T83** |
| 17 | Asymmetry in event-detection ordering (turns.py vs regenerate.py) | T61 review | **T83** |
| 18 | `record_meanwhile_memory` / `record_turn_memory_for_present` unified API | T64 review | **T84** |
| 19 | `participants_json` JSON-build audit (other state modules) | T63 review | **T85** |
| 20 | Stop-button cancellation route-level coverage for meanwhile turns | T64 review | **T85** |
| 21 | Frontend handler for `turn_html_replace` SSE event | T73.1 review (Phase 2.5 carry-over) | **T86** |
| — | Docs sweep — remove shipped items, capture residuals | (this plan) | **T87** |
**Deferred (not in this plan):**
- **Cross-feature canned-queue brittleness** (Wave 6b cross-feature): would require a structured-fixture refactor across 3+ test files. Tracking but not in scope here — open as its own work if it surfaces a real test-maintenance pain.
- **Scene-close-on-cancel UX revisit** (T74.3): pinned to existing behavior; no action needed unless real play surfaces a regression.
---
## Parallel-Execution Strategy
Same pattern as Phases 2.5 and 3. Seven waves: parallel within each wave (file-disjoint), serial across waves.
### How to dispatch a wave in parallel
Use the **Agent tool with `isolation: "worktree"`** so each subagent gets its own git worktree. (If the controlling session's working directory is **not** the chat repo, create worktrees manually with `git worktree add .worktrees/<wave>-<task> -b <wave>/<task> phase-3.5` from inside the chat repo and pass the worktree path explicitly into each subagent prompt.)
In a single message, dispatch all tasks in the wave:
```
Agent({
description: "Wave 1 — T76 narrate_skip timeout_s",
subagent_type: "general-purpose",
isolation: "worktree",
prompt: "<full task text from below>",
})
Agent({ ...T77... })
Agent({ ...T78... })
Agent({ ...T79... })
```
### After a wave completes
1. Each subagent returns its worktree path and commit SHA(s).
2. **Run a spec + code-quality reviewer subagent on each completed task.** Combined review acceptable for purely mechanical fixes (T76T79). Separate spec + quality reviewers for bundled tasks (T80, T82, T83).
3. **Merge the wave into `phase-3.5`** in any order (file-disjointness guarantees no conflict). Use `--no-ff`.
4. **Run the full test suite** on the merged `phase-3.5`. If red, the wave's mutual-independence assumption was violated — bisect, fix, re-merge.
5. **Push `phase-3.5`** to gitea.
6. Optionally clean up worktrees: `git worktree remove .worktrees/<branch>` and `git branch -D <branch>`.
### Conflict prevention checklist
For each parallel wave, verify the **Files** sections of all tasks have **no overlapping paths**. Hot files in this plan: `chat/web/turns.py` (T82 only), `chat/services/regenerate.py` (T83 only), `chat/web/skip.py` + `chat/web/drawer.py` (T81 only). Each is owned by exactly one task.
### Failure recovery
If one subagent fails: cancel it, merge the others' successful work, re-dispatch the failed task as a single follow-up. Don't block the wave.
### Why each wave is parallel-safe
| Wave | Tasks | Hot files touched | Disjoint? |
|------|-------|-------------------|-----------|
| 1 | T76, T77, T78, T79 | 4 different services/state files; no overlap | ✅ |
| 2 | T80 | `chat/services/scene_summarize.py` | (single task) |
| 3 | T81 | `chat/web/skip.py` + `chat/web/drawer.py` (typed exception) | (single task) |
| 4 | T82 | `chat/web/turns.py` | (single task) |
| 5 | T83 | `chat/services/regenerate.py` (+ optional new shared-helpers module) | (single task) |
| 6 | T84, T85, T86 | `chat/services/memory_write.py` (T84); audit + tests (T85); frontend HTML/JS (T86) | ✅ |
| 7 | T87 | `CLAUDE.md` | (single task) |
---
## Task overview
```
Wave 1 ─┬─ T76: narrate_skip timeout_s plumbed through (skip_narration.py)
├─ T77: AddresseeDecision.confidence as Literal (addressee.py)
├─ T78: search_memories docstring SQL-bias note (memory.py)
└─ T79: _witness_role_for defensive None handling (prompt.py)
Wave 2 ─── T80: scene_summarize.py polish bundle (5 T58 items)
Wave 3 ─── T81: typed exception for skip controllers (skip.py + drawer.py)
Wave 4 ─── T82: turns.py wiring (consume_pending_meanwhile_digests + skip-bypass scene close)
Wave 5 ─── T83: regenerate.py polish bundle (cancel hook + DRY + sibling query +
lifecycle rollback + ordering asymmetry)
Wave 6 ─┬─ T84: memory_write.py unified record-memory API
├─ T85: JSON-build audit + meanwhile stop-button route-level test
└─ T86: frontend turn_html_replace SSE handler
Wave 7 ─── T87: docs sweep — prune CLAUDE.md backlogs, capture residuals
```
Critical path: 7 sequential merge points. Total tasks: 12. Wall-clock parallelism advantage: Waves 1 and 6 dispatch concurrently (4-way and 3-way respectively). Waves 2, 3, 4, 5, 7 are single-task by hot-file constraint.
---
## Wave 1 — Independent small fixes (parallel, 4 tasks)
All 4 tasks are tiny (1-line + tests). Fully file-disjoint.
### Task 76: `narrate_skip` timeout_s plumbed through
**Files:**
- Modify: `chat/services/skip_narration.py`
- Modify: `tests/test_skip_narration.py` (add 1 test)
**Spec:** `narrate_skip(timeout_s=60.0)` accepts the parameter but doesn't pass it to `client.generate(...)`. Per T53 review — fix:
```python
# Inside narrate_skip:
result = await client.generate(
[
Message(role="system", content=system),
Message(role="user", content=user),
],
model=narrative_model,
max_tokens=200,
temperature=0.7,
timeout_s=timeout_s, # NEW: pipe through
)
```
If the underlying `client.generate` doesn't honor `timeout_s` (Featherless client signature accepts `**params` so it'll ride through harmlessly), this is a no-op at runtime but documents intent. Read `chat/llm/featherless.py::FeatherlessClient.generate` to confirm.
**Test added:** `test_narrate_skip_passes_timeout_through` — capture the kwargs passed to `client.generate` via a custom mock that records them; assert `timeout_s` is present with the expected value.
**Commit:** `fix: plumb narrate_skip timeout_s through to client.generate (T76)`.
---
### Task 77: `AddresseeDecision.confidence` as Literal type
**Files:**
- Modify: `chat/services/addressee.py`
- Modify: `tests/test_addressee.py` (existing tests should still pass; add 1 test for the typed validation)
**Spec:** Per T74 review (Phase 2.5 carry-over). `AddresseeDecision.confidence` is currently `str` with a comment noting valid values. Tighten to `Literal["high", "medium", "low"]`:
```python
from typing import Literal
class AddresseeDecision(BaseModel):
addressee_id: str
confidence: Literal["high", "medium", "low"] = "medium"
reason: str = ""
```
Pydantic will reject classifier output with values outside the literal set, falling back via the `default=` in `classify(...)`. The existing fallback path returns `confidence="low"` so no behavior change for legitimate calls.
**Test added:** `test_invalid_confidence_value_falls_back_to_default` — feed canned classifier output with `"confidence": "VERY_HIGH"`; assert the result is the default fallback (Pydantic validation failed → `classify` falls back).
**Commit:** `fix: AddresseeDecision.confidence as Literal[high|medium|low] (T77)`.
---
### Task 78: `search_memories` docstring mentions SQL-bias
**Files:**
- Modify: `chat/state/memory.py` — extend the `search_memories` docstring with a one-liner about `SIGNIFICANCE_RANK_BIAS`.
- No test changes needed (docstring-only).
**Spec:** Per T57 review. The current `search_memories` docstring describes only the Python-side composite re-rank with `_SIGNIFICANCE_WEIGHT`. T57 added a SQL-side `SIGNIFICANCE_RANK_BIAS` constant. Add to the docstring:
```
... (existing prose) ...
The result ordering applies TWO independent significance boosts:
- SQL-side: ``ORDER BY (rank - significance * SIGNIFICANCE_RANK_BIAS)``
pushes higher-significance memories ahead in the FTS5 candidate set.
- Python-side: a composite re-rank with `_SIGNIFICANCE_WEIGHT` reinforces
the ordering after candidate retrieval.
```
**Commit:** `docs: search_memories docstring mentions SQL-side significance bias (T78)`.
---
### Task 79: `_witness_role_for` defensive None handling
**Files:**
- Modify: `chat/services/prompt.py`
- Modify: `tests/test_prompt.py` (add 1 test)
**Spec:** Per T71 review (Phase 2.5 carry-over). Current `_witness_role_for(speaker_bot_id, host_bot_id)` returns `"host" if speaker_bot_id == host_bot_id else "guest"`. When `host_bot_id` is `None` (degenerate case — can happen if a chat row is half-seeded in a test), this returns `"guest"` even though the speaker is logically the host. Defensive fix:
```python
def _witness_role_for(speaker_bot_id: str, host_bot_id: str | None) -> str:
"""..."""
if host_bot_id is None or speaker_bot_id == host_bot_id:
return "host"
return "guest"
```
**Test added:** `test_witness_role_for_none_host_returns_host` — call helper with `host_bot_id=None`; assert returns `"host"`.
**Commit:** `fix: _witness_role_for defensive None handling (T79)`.
---
## Wave 2 — `scene_summarize.py` polish bundle (single task)
T80 bundles 5 backlog items (4 fixes + 1 test gap) all touching `chat/services/scene_summarize.py` and its test file. Single task by hot-file constraint; split into 5 commits internally for clean review bisection.
### Task 80: scene_summarize.py polish
**Files:**
- Modify: `chat/services/scene_summarize.py`
- Modify: `tests/test_per_pov_summary.py`
**Spec:** Five sub-fixes per the T58 review.
#### 80.1 — Re-close suffix bloat guard
**Problem:** `_build_key_quotes_suffix` reads from `memories.pov_summary`. If a scene close runs twice (replay, manual re-close, idempotent retry), the second pass reads the rewritten text plus the previous "Key quotes:" suffix and appends a second one — recursive bloat.
**Fix (recommended option):** source quotes from `event_log` `assistant_turn`/`user_turn` text instead of `memories.pov_summary`. The event-log text is immutable per turn, so re-close produces identical key-quote text.
Implementation: in `_build_key_quotes_suffix(conn, scene_id)`, replace the SELECT from `memories` with a JOIN that pulls turn text from `event_log` filtered by scene_id (via `payload_json` JSON extraction), keeping the significance-from-memories filter.
If event-log queries prove too expensive or fragile, fall back to: at the start of `_build_key_quotes_suffix`, strip any existing `"\n\nKey quotes:\n"` suffix from each `pov_summary` before computing the new one. Document the choice in a code comment.
**Test added:** `test_scene_close_re_run_does_not_double_suffix` — close a scene with significance ≥ 2; assert each pov_summary contains "Key quotes:" exactly ONCE. Then call `apply_scene_close_summary` again on the same scene; assert each pov_summary STILL contains "Key quotes:" exactly ONCE (no second append).
**Commit:** `fix: guard scene close key-quote suffix against re-close bloat (T80.1)`.
#### 80.2 — Thread detection transcript scoping
**Problem:** `_read_recent_dialogue` returns chat-wide history with no `scene_id` filter. Feeding chat-wide history to `detect_threads` will misattribute threads to the closing scene when a scene boundary falls inside the last 50 turns.
**Fix:** add a `scene_started_at` lookup at the top of `apply_scene_close_summary`. Filter the transcript to turns where the event_log row's `created_at` (or `chat_clock_at`) is `>= scene_started_at`. Read `chat/state/world.py::get_scene` for the started_at column.
If `get_scene` doesn't expose `started_at`, query directly: `SELECT started_at FROM scenes WHERE id = ?`.
**Test added:** `test_thread_detection_uses_scene_scoped_transcript` — seed a scene with 3 turns. Then close the scene. Then start a new scene and seed 3 more turns. Then close the second scene. Mock `detect_threads` to capture its `scene_transcript` arg. Assert the second close's transcript contains ONLY the 3 second-scene turns (not the first scene's turns).
**Commit:** `fix: scope thread detection transcript to closing scene (T80.2)`.
#### 80.3 — Log swallowed exceptions in `detect_threads` try/except
**Problem:** T58's `try/except Exception:` around `detect_threads` swallows programmer errors silently.
**Fix:** add a `logging.getLogger(__name__).debug("detect_threads failed: %s", exc, exc_info=True)` (or appropriate level) inside the except block. Use the standard library `logging` module (it's already configured by Phase 1's `chat.app`).
**Test added:** `test_detect_threads_failure_is_logged` — patch `detect_threads` to raise `RuntimeError("test")`. Use `caplog` (pytest fixture) to capture log output. Assert the log contains the error message.
**Commit:** `fix: log swallowed exceptions in detect_threads try/except (T80.3)`.
#### 80.4 — Scene close `closed_at` chat-clock semantics
**Problem:** T58 uses `datetime.now(timezone.utc).isoformat()` for `closed_at` on emitted events instead of chat-clock time. Diverges from chat-clock semantics elsewhere.
**Fix:** read `chat["time"]` at the top of `apply_scene_close_summary` (already loaded for per-POV writes via `chat_clock_at`). Pass `chat_clock_at` through to `thread_closed` events' `closed_at` field instead of `datetime.now(...)`.
**Test added:** `test_thread_closed_uses_chat_clock_time` — seed a chat with `chat["time"] = "2026-04-26T10:00:00+00:00"`. Mock `detect_threads` to return one `ThreadCandidate(action="close", existing_thread_id="thr_x")`. Close the scene. Inspect the emitted `thread_closed` event payload — assert `closed_at == "2026-04-26T10:00:00+00:00"`, NOT a `datetime.now(...)` value.
**Commit:** `fix: thread_closed uses chat-clock time, not wall clock (T80.4)`.
#### 80.5 — T58 test coverage gaps
**Spec:** Per T58 review, these specific scenarios lack coverage. Add tests:
1. `test_key_quote_truncation_at_200_chars` — seed a memory with significance 2 and a 500-char `pov_summary`. Close scene. Assert the key-quote bullet for that memory is exactly 200 chars (or 199 + ellipsis, depending on the truncation impl).
2. `test_thread_detection_update_candidate_emits_thread_updated` — mock `detect_threads` to return `ThreadCandidate(action="update", existing_thread_id="thr_x", summary="updated")`. Close. Assert a `thread_updated` event landed.
3. `test_thread_detection_close_candidate_emits_thread_closed` — same setup with `action="close"`. Assert a `thread_closed` event landed.
(The `try/except` fallback test from 80.3 covers the exception path; no separate test needed.)
**Commit:** `test: T58 coverage gaps (truncation, update/close paths) (T80.5)`.
#### Verification gates for T80
- All 5 sub-fix commits cleanly split per `git show`.
- All 13 existing per_pov_summary tests still pass.
- 5+ new tests pass.
- Full suite green.
- `git diff phase-3.5 --stat` shows ONLY `chat/services/scene_summarize.py` and `tests/test_per_pov_summary.py`.
---
## Wave 3 — Typed exception for skip controllers (single task)
### Task 81: `ChatNotFoundError` for skip route routing
**Files:**
- Modify: `chat/web/skip.py` — define a typed exception and raise it instead of generic `ValueError("chat not found")`.
- Modify: `chat/web/drawer.py` — replace string-prefix sniff with `except ChatNotFoundError: 404`.
- Modify: `tests/test_drawer_events_threads_skip.py` (add 1 test).
**Spec:** Per T62 review. Drawer skip routes use `str(exc).startswith("chat not found")` to distinguish 404 from 400. Fragile if error wording changes. Replace with a typed exception subclass.
```python
# chat/web/skip.py top:
class ChatNotFoundError(Exception):
"""Raised when a chat_id doesn't resolve. Maps to 404 in HTTP routes."""
# Inside process_elision_skip / process_jump_skip:
chat = get_chat(conn, chat_id)
if chat is None:
raise ChatNotFoundError(f"chat {chat_id!r} not found")
# Other validation failures still raise ValueError (mapped to 400).
```
```python
# chat/web/drawer.py skip routes:
try:
result = await process_elision_skip(...)
except ChatNotFoundError:
raise HTTPException(status_code=404, detail="chat not found")
except ValueError as exc:
raise HTTPException(status_code=400, detail=str(exc))
```
**Test added:** `test_skip_route_raises_404_via_typed_exception` — POST to `/chats/nonexistent/drawer/skip/elision`; assert response 404. (Existing tests should still pass — they check 404 for missing chats and 400 for validation errors.)
**Commit:** `fix: typed ChatNotFoundError replaces string-prefix sniff in skip routes (T81)`.
---
## Wave 4 — `turns.py` wiring (single task)
### Task 82: post_turn meanwhile-digest consumption + skip-command scene close
**Files:**
- Modify: `chat/web/turns.py`
- Modify: `tests/test_turn_flow.py` (add 2 tests)
- Modify: `tests/test_phase3_integration.py` (T66 test 4 currently calls `consume_pending_meanwhile_digests` directly — update it to assert the helper is now invoked AUTOMATICALLY by post_turn)
**Spec:** Two related fixes both touching `post_turn`.
#### 82.1 — Wire `consume_pending_meanwhile_digests` into post_turn
**Problem (T66):** `chat/services/prompt.py::consume_pending_meanwhile_digests` is defined but never called. Meanwhile digests stay pending forever in production.
**Fix:** call the helper at the END of `post_turn` (after the assistant_turn lands and all classifier passes finish, BEFORE the response returns). The helper appends `meanwhile_digest_consumed` events for each pending digest, marking them as surfaced. Idempotent — re-calling produces zero events.
Place the call only when the chat has just-surfaced digests. The simplest pattern: always call after the primary turn — the helper short-circuits when nothing's pending. (Cost: 1 trivial DB query per turn.)
```python
from chat.services.prompt import consume_pending_meanwhile_digests
# Near the end of post_turn, before publish/return:
consume_pending_meanwhile_digests(conn, chat_id)
```
**Test added:** `test_post_turn_consumes_pending_meanwhile_digests` — seed a pending digest. POST a turn. Assert: a `meanwhile_digest_consumed` event landed in event_log AND `list_pending_meanwhile_digests` is now empty.
**Update existing T66 test 4:** the integration test currently calls `consume_pending_meanwhile_digests` directly to drive the consumption side-effect. After T82, the consumption happens automatically inside post_turn. Remove the explicit call in the test (or assert the count even without calling explicitly).
**Commit:** `fix: post_turn consumes pending meanwhile digests (T82.1)`.
#### 82.2 — Skip command runs scene close detection
**Problem (T62):** A user typing "skip an hour" via natural-language skip causes the skip path to bypass scene close detection. The user's prose may have signaled close intent ("fade out, skip an hour") but the skip path returns 204 without checking.
**Fix:** in the natural-language skip dispatch (T62 `intent="skip_elision"` branch), call `detect_scene_close` BEFORE the skip controller runs. If close is detected, append `scene_closed` first, then run the skip flow.
Order matters: scene close → skip narration → time advance. The narration should reflect the new scene start at the new time.
```python
if parsed.intent == "skip_elision":
# Run scene close detection first - the user may have signaled close.
if scene := active_scene(conn, chat_id):
close_decision = await detect_scene_close(client, classifier_model=...,
prose=parsed.prose, ...)
if close_decision.should_close:
append_and_apply(conn, kind="scene_closed", payload={"scene_id": scene["id"], ...})
await apply_scene_close_summary(conn, client, ...)
# ... existing skip dispatch ...
```
**Test added:** `test_natural_language_skip_with_close_signal_closes_scene` — user prose "fade out, skip to morning". Mock `detect_scene_close` to return True. Mock `narrate_skip`. Assert: `scene_closed` event lands AND `time_skip_elision` event lands AND ordering is `scene_closed` BEFORE `time_skip_elision`.
**Commit:** `fix: natural-language skip runs scene close detection (T82.2)`.
#### Verification gates for T82
- 2 new tests pass.
- All existing turn_flow + phase3_integration tests still pass (T66 test 4 may need a small update; document in commit).
- Full suite green.
- `git diff phase-3.5 --stat`: only `chat/web/turns.py`, `tests/test_turn_flow.py`, `tests/test_phase3_integration.py`.
---
## Wave 5 — `regenerate.py` polish bundle (single task)
T83 bundles 5 regenerate-related backlog items. All touch `chat/services/regenerate.py`. Single task by hot-file constraint; 5 internal commits.
### Task 83: regenerate.py polish
**Files:**
- Modify: `chat/services/regenerate.py`
- Optional: create `chat/services/turn_common.py` (new shared-helpers module for 83.2 DRY extraction)
- Modify: `chat/web/turns.py` (only if 83.2 extracts shared helpers — minimal)
- Modify: `tests/test_regenerate.py` (add 5+ tests)
**Spec:** Five sub-fixes per the T73 + T61 reviews.
#### 83.1 — Cancel/stop hook for in-flight regenerate streams
**Problem:** `post_turn` registers stream tasks in `_in_flight_tasks` so `/turns/cancel` cancels them. Regenerate doesn't.
**Fix:** mirror the registration pattern from `post_turn` (Phase 2.5 T74.4 documented this). Wrap the regenerate stream in `asyncio.create_task(...)`, register in `_in_flight_tasks[chat_id]`, await, unregister in `finally`.
**Test added:** `test_regenerate_registers_task_in_in_flight_tasks` — mid-stream snapshot pattern (mirror Phase 3 T64.fix-up's `_SnapshotMock`). Assert the chat_id is registered during streaming.
**Commit:** `feat: regenerate registers stream task in _in_flight_tasks for cancellation (T83.1)`.
#### 83.2 — DRY: extract recent-dialogue + prior-edges helpers
**Problem:** Recent-dialogue assembly + prior-edges block are duplicated between `chat/services/regenerate.py` and `chat/web/turns.py`. Diff drift risk.
**Fix:** create `chat/services/turn_common.py` with:
```python
def read_recent_dialogue(conn, chat_id: str, limit: int = 50) -> list[dict]:
"""Pull the last N user_turn + assistant_turn rows for the chat as
structured [{speaker, text}, ...]. Filters superseded_by IS NULL
AND hidden = 0. Used by post_turn AND regenerate AND scene_summarize."""
def gather_prior_edges(conn, chat_id: str, present_ids: list[str]) -> dict[tuple, dict]:
"""Build {(src, tgt): {affinity, trust, summary}} dict for all directed
pairs where both endpoints are in present_ids."""
```
Update `chat/web/turns.py::post_turn` to use the new helpers. Update `chat/services/regenerate.py::regenerate_assistant_turn` to use them too. Existing tests should pass unchanged.
**Test added:** `tests/test_turn_common.py` (NEW file) with 2-3 tests: dialogue read filters superseded; prior_edges builds correct keys.
**Commit:** `refactor: extract turn_common helpers from regenerate + turns (T83.2)`.
#### 83.3 — Sibling-discovery query optimization
**Problem (T73):** `regenerate.py`'s sibling-assistant-turn lookup scans ALL non-superseded `assistant_turn` rows globally with no `chat_id` predicate.
**Fix:** add a `json_extract(payload_json, '$.chat_id') = ?` predicate in the SQL query, plus `LIMIT 50` for safety. Mirror Phase 3 T64's `_last_meanwhile_speaker` SQL pattern.
**Test added:** `test_regenerate_sibling_lookup_scoped_to_chat` — seed two chats both with regenerate-able turn groups. Regenerate in chat A. Assert the sibling lookup didn't return any chat-B rows (verify via SQL query interception or by ensuring cross-chat sibling pollution wouldn't break).
**Commit:** `perf: scope regenerate sibling-lookup to chat_id (T83.3)`.
#### 83.4 — Lifecycle-transition rollback on regenerate
**Problem (T61):** Regenerate doesn't roll back lifecycle transitions from the superseded turn. `event_started`/`event_completed` rows from a superseded turn remain. May double-emit promotion artifacts.
**Fix (Phase 3.5 first cut — minimal):** at the START of regenerate, scan the superseded turn's downstream events. For any `event_started`/`event_completed`/`event_cancelled` linked to the superseded turn (via `superseded_by` chain on the original assistant_turn AND a back-pointer in the lifecycle event payload), revert the projected state by appending compensating events:
- Original was `event_started` for evt_x → emit `event_started_undone` (NEW event kind?) OR mark the original superseded.
- Original was `event_completed` → similarly.
**This is too invasive for one Phase 3.5 commit.** Instead, the simpler Phase 3.5 fix: **document the limitation explicitly** in the regenerate flow with a TODO + add an `assert` or warning when regenerate detects prior lifecycle transitions on the superseded turn. Real rollback support is a Phase 4 item.
**Test added:** `test_regenerate_with_prior_lifecycle_logs_warning` — seed a turn that emits `event_completed`. Regenerate. Assert a warning log entry mentions the un-rolled-back transition.
**Commit:** `chore: document regenerate lifecycle-rollback limitation (T83.4)`.
(If the implementer judges that proper rollback IS feasible in the time available, they can implement it instead — but the safe Phase 3.5 default is documentation + warning. The choice is the implementer's; report which path was taken.)
#### 83.5 — Event-detection ordering symmetry
**Problem (T61):** `post_turn` runs lifecycle BETWEEN interjection and scene-close; `regenerate.py` runs lifecycle at the END. Benign because regenerate has no scene-close path, but worth tidying.
**Fix:** move the event-detection block in `regenerate.py` to mirror `post_turn`'s position (between interjection regenerate and the function's tail). Cosmetic — no behavior change.
**Test:** verify all existing regenerate tests still pass after the move.
**Commit:** `refactor: regenerate event-detection ordering mirrors post_turn (T83.5)`.
#### Verification gates for T83
- 5 internal commits visible.
- All 6 existing regenerate tests still pass.
- All 10+ existing turn_flow tests still pass.
- 5+ new tests pass.
- Full suite green.
- `git diff phase-3.5 --stat`: `chat/services/regenerate.py`, `chat/services/turn_common.py` (if extracted), `chat/web/turns.py` (only if helper-call sites updated), `tests/test_regenerate.py`, `tests/test_turn_common.py` (if new).
---
## Wave 6 — Final polish (parallel, 3 tasks)
### Task 84: Unified record-memory API
**Files:**
- Modify: `chat/services/memory_write.py`
- Modify: `tests/test_memory_write.py` (add tests)
- Modify: `chat/web/meanwhile.py` (call site update — minimal)
**Spec:** Per T64 review. `record_meanwhile_memory` and `record_turn_memory_for_present` share private `_write_one_memory` but expose two separate public APIs. Unify:
```python
def record_turn_memory(
conn,
*,
chat_id: str,
host_bot_id: str,
guest_bot_id: str | None,
narrative_text: str,
scene_id: int | None = None,
chat_clock_at: str | None = None,
source: str = "direct",
significance: int = 1,
you_present: bool = True, # NEW: False for meanwhile turns
) -> dict[str, tuple[int, int | None]]:
"""Single entry-point for memory writes. When you_present is True,
witnesses are [you=1, host=1, guest if present]. When False (meanwhile),
witnesses are [you=0, host=1, guest=1] and host_bot_id + guest_bot_id
are both required."""
```
`record_meanwhile_memory` becomes a thin wrapper (kept for backward compat) that calls `record_turn_memory(..., you_present=False)`. `record_turn_memory_for_present` is renamed/aliased to `record_turn_memory` for the same reason.
Update `chat/web/meanwhile.py` to call the unified function with `you_present=False`. Verify the existing meanwhile turn tests still pass.
**Tests added:** 2 new tests in `tests/test_memory_write.py`:
- `test_record_turn_memory_you_present_false_writes_meanwhile_witness_mask` — assert witness `[0, 1, 1]`.
- `test_record_turn_memory_you_present_true_default_writes_normal_witness_mask` — assert `[1, 1, 0|1]`.
**Commit:** `refactor: unified record_turn_memory API with you_present kwarg (T84)`.
---
### Task 85: JSON-build audit + meanwhile stop-button route-level test
**Files:**
- Audit: read all `chat/state/*.py` for f-string JSON construction. NO PRODUCTION CHANGES unless an issue is found.
- Modify (only if issue found): the affected state module.
- Modify: `tests/test_meanwhile_turn_flow.py` (add stop-button route-level test).
**Spec:** Two unrelated minor follow-ups bundled by being small.
#### 85.1 — JSON-build audit (T63)
T63 originally used f-string interpolation for `participants_json` (fixed during review). Audit other state modules for the same pattern. Grep:
```bash
grep -rn 'f["\']\[' chat/state/ chat/services/ chat/eventlog/
```
For each f-string that constructs a JSON string from user-controllable data, replace with `json.dumps(...)`. If NONE found (likely — most code already uses `json.dumps`), add a one-line note in the commit body confirming the audit.
If issues are found, fix them and add tests asserting JSON validity for inputs containing quote/backslash characters.
**Commit:** `chore: audit JSON-string construction sites; replace any f-string usages with json.dumps (T85.1)` (or "audit confirms no issues").
#### 85.2 — Meanwhile stop-button route-level test
T64's fix-up registered stream tasks in `_in_flight_tasks`. There's a unit test pinning registration but no test that drives `/turns/cancel` against a meanwhile turn.
**Test added:** `test_meanwhile_turn_can_be_cancelled_via_route` — start a meanwhile turn (POST /turns), capture the in-flight task, immediately POST /turns/cancel, assert the task transitions to cancelled state and the assistant_turn payload has `truncated=True`.
This test may be tricky to write reliably (timing). If the existing `_SnapshotMock` pattern doesn't support mid-stream cancel injection, document the limitation and add a smaller test that just verifies the cancel endpoint works on the chat (asserts no tasks remaining after cancel).
**Commit:** `test: meanwhile turn cancellation via /turns/cancel route (T85.2)`.
---
### Task 86: Frontend `turn_html_replace` SSE handler
**Files:**
- Modify: `chat/templates/chat.html` (or wherever the chat page's SSE consumer lives) — add a JS event handler for `turn_html_replace`.
- Optional: `chat/static/app.css` (if styling needs adjustment for replaced turns).
- Modify: `tests/test_streaming_ux.py` (add 1 test).
**Spec:** Per Phase 2.5 T73.1 (carry-over). Regenerate's backend broadcasts a `turn_html_replace` SSE event but no live tab swaps the prior turn. Wire the JS:
```html
<!-- In chat.html, near the existing SSE setup: -->
<script>
const sse = new EventSource('/chats/{{ chat.id }}/events');
// Existing: turn_html appends new turn HTML.
sse.addEventListener('turn_html', (ev) => {
const turnsContainer = document.getElementById('turns');
turnsContainer.insertAdjacentHTML('beforeend', ev.data);
});
// NEW: turn_html_replace swaps an existing turn's DOM in place.
sse.addEventListener('turn_html_replace', (ev) => {
const data = JSON.parse(ev.data);
// data: {html, turn_id, supersedes_id}
const oldNode = document.getElementById(`turn-${data.supersedes_id}`);
if (oldNode) {
const tmpl = document.createElement('template');
tmpl.innerHTML = data.html.trim();
oldNode.replaceWith(tmpl.content.firstChild);
} else {
// Fallback: append if the prior turn isn't in the DOM (edge case).
document.getElementById('turns').insertAdjacentHTML('beforeend', data.html);
}
});
</script>
```
(Adapt the actual template structure — read `chat/templates/chat.html` first to see the existing SSE setup. The `turn-{id}` DOM-id pattern needs to be consistent with how the backend renders turn HTML — verify by checking `chat/web/render.py`.)
**Test added:** Hard to write a true browser-driven test in pytest. The simplest pin: a unit test that asserts the rendered HTML for a regenerate response contains the `turn_html_replace` SSE event AND the event payload contains the expected `supersedes_id`. (Already exists in T73.1 tests — don't duplicate.) Optionally add a Selenium/playwright test if the project has frontend testing infrastructure.
If pytest-only is the available coverage, document the manual smoke step in CLAUDE.md ("multi-tab regenerate" smoke test) and rely on backend pin tests + manual verification.
**Commit:** `feat: frontend turn_html_replace SSE handler (T86)`.
---
## Wave 7 — Docs sweep (single task)
### Task 87: CLAUDE.md backlog burn-down
**Files:**
- Modify: `CLAUDE.md`
**Spec:** Walk through the Phase 2.6/3 backlog and Phase 3.5/4 backlog sections. For each item shipped in T76T86, remove from backlog list. Add a new section "Phase 3.5 status" near the existing "Phase 3 status" listing what shipped. Add a new "Phase 3.6 / 4 backlog" section if any new items were discovered during T76T86 reviews (likely some — the review cycle always surfaces fresh follow-ups).
If any task during T76T86 chose to defer a sub-item (e.g., T83.4 chose documentation over rollback impl), keep that sub-item in the new "Phase 3.6+ deferred" section with the rationale.
Mark §13 Phase 3 deliverables in the requirements doc as **shipped** (already done in T67) — verify and don't re-mark.
**Commit:** `docs: phase 3.5 status, prune shipped backlog items (T87)`.
---
## Wrap-up
After Wave 7 lands:
1. **Run full suite** on `phase-3.5`: should be ~330+ tests passing (315 from Phase 3 + ~15-25 new across the wave).
2. **Manual smoke** (recommended before opening the PR):
- Multi-tab regenerate: open two tabs on the same chat; click Regenerate on one; verify the other tab swaps the regenerated turn live (T86).
- Trigger a meanwhile scene; close it; resume the main chat; verify the digest renders ONCE on the next you-turn AND is consumed automatically (T82.1).
- Type "fade out, skip an hour" — verify both scene close + skip fire in correct order (T82.2).
- Plan an event, complete it via a turn, then regenerate that turn — observe the warning log mentioning un-rolled-back lifecycle transitions (T83.4).
3. **Push `phase-3.5`** to gitea.
4. **Open PR** `phase-3.5 → main`.
5. **Phase 3.6+ residuals** (track in CLAUDE.md): genuine lifecycle-rollback impl, structured fixture builder for canned-queue tests, scene-close-on-cancel revisit if play-testing surfaces a regression.
---
## Notes for the controller running this plan
- **Don't dispatch Wave 5 until Wave 4 is merged AND green.** T82 (turns.py) + T83 (regenerate.py + optionally turns.py for shared-helper extraction) both touch `turns.py`. Sequential ordering prevents merge conflict.
- **After each parallel wave**, run a code-review subagent. Combined spec+quality review is acceptable for trivial tasks (T76, T77, T78, T79). Separate spec + quality reviewers for bundled tasks (T80, T82, T83) — each bundles 4-5 sub-fixes.
- **Token-spend rough estimate**: Phase 3.5 should be ~50-60% the size of Phase 3 (smaller scope, all reuse, no new schema). Per-task spend similar to Phase 2.5's bundled tasks.
- **DO NOT break existing v1/v2/v3 surface contracts.** Every test file that was green at the start of Phase 3.5 must stay green at the end. The cross-feature integration tests (`tests/test_phase3_integration.py`) are particularly load-bearing — if any of T80/T82/T83 break them, that's a regression to investigate, NOT to suppress.
@@ -0,0 +1,19 @@
{
"planPath": "docs/plans/2026-04-26-v3.5-phase3.5-cleanup.md",
"tasks": [
{"id": 76, "subject": "T76: narrate_skip timeout_s plumbed through", "status": "pending", "wave": 1, "parallelGroup": "wave-1"},
{"id": 77, "subject": "T77: AddresseeDecision.confidence as Literal", "status": "pending", "wave": 1, "parallelGroup": "wave-1"},
{"id": 78, "subject": "T78: search_memories docstring SQL-bias note", "status": "pending", "wave": 1, "parallelGroup": "wave-1"},
{"id": 79, "subject": "T79: _witness_role_for defensive None handling", "status": "pending", "wave": 1, "parallelGroup": "wave-1"},
{"id": 80, "subject": "T80: scene_summarize.py polish bundle (re-close suffix + transcript scoping + log + clock + tests)", "status": "pending", "wave": 2, "parallelGroup": null},
{"id": 81, "subject": "T81: ChatNotFoundError replaces string-prefix sniff in skip routes", "status": "pending", "wave": 3, "parallelGroup": null},
{"id": 82, "subject": "T82: post_turn wiring (consume_pending_meanwhile_digests + skip command scene close)", "status": "pending", "wave": 4, "parallelGroup": null},
{"id": 83, "subject": "T83: regenerate.py polish (cancel hook + DRY + sibling query + lifecycle rollback note + ordering)", "status": "pending", "wave": 5, "parallelGroup": null, "blockedBy": [82]},
{"id": 84, "subject": "T84: unified record_turn_memory API with you_present kwarg", "status": "pending", "wave": 6, "parallelGroup": "wave-6", "blockedBy": [83]},
{"id": 85, "subject": "T85: JSON-build audit + meanwhile stop-button route-level test", "status": "pending", "wave": 6, "parallelGroup": "wave-6", "blockedBy": [83]},
{"id": 86, "subject": "T86: frontend turn_html_replace SSE handler", "status": "pending", "wave": 6, "parallelGroup": "wave-6", "blockedBy": [83]},
{"id": 87, "subject": "T87: docs sweep — prune CLAUDE.md backlogs, capture residuals", "status": "pending", "wave": 7, "parallelGroup": null, "blockedBy": [84, 85, 86]}
],
"lastUpdated": "2026-04-26T00:00:00Z",
"notes": "12 tasks across 7 waves consolidating 17 backlog items (7 from Phase 2.6/3, 10 from Phase 3.5/4). Wave 1 (4-way parallel) and Wave 6 (3-way parallel) are file-disjoint. Waves 2, 3, 4, 5, 7 are single-task by hot-file constraint. Bundled tasks (T80, T82, T83) split into per-item sub-commits for clean review bisection. No schema migrations — schema baseline stays at version 11. Uses task ids T76-T87 to avoid id collision with prior phases (Phase 1: T0-T35, Phase 2: T36-T48, Phase 3: T49-T67, Phase 2.5: T68-T75)."
}
+134
View File
@@ -0,0 +1,134 @@
"""Addressee classifier service tests (T74.1).
Covers :func:`chat.services.addressee.detect_addressee`:
- Classifier picks the guest -> ``addressee_id == guest_id``.
- Classifier picks the host -> ``addressee_id == host_id``.
- Classifier flakes (3 bad-JSON responses, exhausting the built-in
retry budget in :func:`chat.llm.classify.classify`) -> fallback to
the host with ``reason="fallback"``.
"""
from __future__ import annotations
import json
import pytest
from chat.llm.mock import MockLLMClient
from chat.services.addressee import AddresseeDecision, detect_addressee
@pytest.mark.asyncio
async def test_classifier_picks_guest():
"""Classifier returns the guest id verbatim — caller propagates it."""
canned = [
json.dumps(
{
"addressee_id": "bot_b",
"confidence": "high",
"reason": "user named BotB",
}
)
]
client = MockLLMClient(canned=canned)
result = await detect_addressee(
client,
classifier_model="test-model",
user_prose="BotB, what do you think?",
host_id="bot_a",
host_name="BotA",
guest_id="bot_b",
guest_name="BotB",
)
assert isinstance(result, AddresseeDecision)
assert result.addressee_id == "bot_b"
assert result.confidence == "high"
@pytest.mark.asyncio
async def test_classifier_picks_host():
"""Classifier returns the host id — caller propagates it."""
canned = [
json.dumps(
{
"addressee_id": "bot_a",
"confidence": "medium",
"reason": "narration aimed at host",
}
)
]
client = MockLLMClient(canned=canned)
result = await detect_addressee(
client,
classifier_model="test-model",
user_prose="I lean back and stretch.",
host_id="bot_a",
host_name="BotA",
guest_id="bot_b",
guest_name="BotB",
)
assert result.addressee_id == "bot_a"
assert result.confidence == "medium"
@pytest.mark.asyncio
async def test_classifier_failure_falls_back_to_host():
"""Three bad-JSON responses exhaust the retry budget and the
classifier-failure fallback returns ``host_id`` with
``reason="fallback"``."""
canned = ["not json", "still not json", "garbage"]
client = MockLLMClient(canned=canned)
result = await detect_addressee(
client,
classifier_model="test-model",
user_prose="anything",
host_id="bot_a",
host_name="BotA",
guest_id="bot_b",
guest_name="BotB",
)
assert result.addressee_id == "bot_a"
assert result.reason == "fallback"
assert result.confidence == "low"
@pytest.mark.asyncio
async def test_invalid_confidence_value_falls_back_to_default():
"""Pydantic rejects ``confidence`` values outside the literal set
(``high`` / ``medium`` / ``low``). After the retry budget is
exhausted, classify returns the configured fallback default
here that's ``confidence="low"`` with ``reason="fallback"``.
"""
canned = [
json.dumps(
{
"addressee_id": "bot_a",
"confidence": "VERY_HIGH",
"reason": "out-of-range value",
}
),
"still_bad",
"still_bad",
]
client = MockLLMClient(canned=canned)
result = await detect_addressee(
client,
classifier_model="test-model",
user_prose="anything",
host_id="bot_a",
host_name="BotA",
guest_id="bot_b",
guest_name="BotB",
)
assert result.addressee_id == "bot_a"
assert result.confidence == "low"
assert result.reason == "fallback"
+403
View File
@@ -0,0 +1,403 @@
"""T72: deferred v1 drawer edits + witness flag inline-edit.
T25 shipped affinity / significance / pin. T72.1 fills in the rest of the
§6.4 editable surface whose ``manual_edit`` projector dispatch was already
in place (or, for ``edge_knowledge_fact``, added alongside the route):
* ``POST /chats/{chat_id}/drawer/edge/trust`` slider 0..100.
* ``POST /chats/{chat_id}/drawer/edge/summary`` textarea, capped 2000.
* ``POST /chats/{chat_id}/drawer/memory/pov-summary`` textarea, capped.
* ``POST /chats/{chat_id}/drawer/edge/knowledge-facts`` add/remove fact.
T72.3 layers a witness-flag toggle on top:
* ``POST /chats/{chat_id}/drawer/memory/witness`` ``manual_edit`` with
``target_kind`` = ``memory_witness`` and a ``{flag, value}`` payload.
Each test asserts (a) the ``manual_edit`` event lands in the log,
(b) the projected table reflects the new value, and (c) the response is
the refreshed drawer partial.
"""
from __future__ import annotations
import json
from pathlib import Path
import pytest
from fastapi.testclient import TestClient
from chat.app import app
from chat.db.connection import open_db
from chat.eventlog.log import append_event
from chat.eventlog.projector import project
@pytest.fixture
def client(tmp_path, monkeypatch):
cfg = tmp_path / "config.toml"
cfg.write_text('featherless_api_key = "test"\n')
monkeypatch.setenv("CHAT_CONFIG_PATH", str(cfg))
db = tmp_path / "test.db"
monkeypatch.setenv("CHAT_DB_PATH", str(db))
with TestClient(app) as c:
if hasattr(app.state, "background_worker"):
app.state.background_worker.enabled = False
yield c
def _seed(db: Path) -> None:
"""Seed a chat with one host bot, one host->you edge with a fact and
summary already set, and one memory authored by ``bot_a`` witnessed by
all three roles. Tests reach into projected state to verify edits.
"""
with open_db(db) as conn:
append_event(
conn,
kind="bot_authored",
payload={
"id": "bot_a",
"name": "BotA",
"persona": "...",
"voice_samples": [],
"traits": [],
"backstory": "",
"initial_relationship_to_you": "",
"kickoff_prose": "",
},
)
append_event(
conn,
kind="chat_created",
payload={
"id": "chat_bot_a",
"host_bot_id": "bot_a",
"initial_time": "2026-04-26T20:00:00+00:00",
"narrative_anchor": "Day 1",
"weather": "",
},
)
# Materialise edge bot_a -> you with a knowledge_fact already on it
# so the remove path has something to consume.
append_event(
conn,
kind="edge_update",
payload={
"source_id": "bot_a",
"target_id": "you",
"chat_id": "chat_bot_a",
"affinity_delta": 0,
"knowledge_facts": ["studied physics together"],
},
)
append_event(
conn,
kind="memory_written",
payload={
"owner_id": "bot_a",
"chat_id": "chat_bot_a",
"pov_summary": "Original summary text.",
"witness_you": 1,
"witness_host": 1,
"witness_guest": 0,
"significance": 1,
},
)
project(conn)
# --- T72.1 tests ----------------------------------------------------------
def test_edit_edge_trust_emits_manual_edit_and_updates(client, tmp_path):
_seed(tmp_path / "test.db")
response = client.post(
"/chats/chat_bot_a/drawer/edge/trust",
data={"source_id": "bot_a", "target_id": "you", "new_value": "73"},
)
assert response.status_code == 200
# Refresh shows the new trust value somewhere in the partial.
assert "73" in response.text
with open_db(tmp_path / "test.db") as conn:
rows = conn.execute(
"SELECT payload_json FROM event_log WHERE kind = 'manual_edit'"
).fetchall()
assert len(rows) == 1
payload = json.loads(rows[0][0])
assert payload["target_kind"] == "edge_trust"
assert payload["prior_value"] == 50
assert payload["new_value"] == 73
assert payload["target_id"] == {
"source_id": "bot_a",
"target_id": "you",
}
from chat.state.edges import get_edge
edge = get_edge(conn, "bot_a", "you")
assert edge["trust"] == 73
def test_edit_edge_trust_400_on_out_of_range(client, tmp_path):
_seed(tmp_path / "test.db")
response = client.post(
"/chats/chat_bot_a/drawer/edge/trust",
data={"source_id": "bot_a", "target_id": "you", "new_value": "150"},
)
assert response.status_code == 400
def test_edit_edge_summary_emits_manual_edit_and_updates(client, tmp_path):
_seed(tmp_path / "test.db")
response = client.post(
"/chats/chat_bot_a/drawer/edge/summary",
data={
"source_id": "bot_a",
"target_id": "you",
"new_summary": "BotA respects you and shares lab notes.",
},
)
assert response.status_code == 200
with open_db(tmp_path / "test.db") as conn:
rows = conn.execute(
"SELECT payload_json FROM event_log WHERE kind = 'manual_edit'"
).fetchall()
assert len(rows) == 1
payload = json.loads(rows[0][0])
assert payload["target_kind"] == "edge_summary"
assert payload["new_value"].startswith("BotA respects")
assert payload["target_id"] == {
"source_id": "bot_a",
"target_id": "you",
}
summary = conn.execute(
"SELECT summary FROM edges "
"WHERE source_id = ? AND target_id = ?",
("bot_a", "you"),
).fetchone()[0]
assert "respects" in summary
# And the refreshed partial echoes the new summary back.
assert "respects" in response.text
def test_edit_edge_summary_400_on_overflow(client, tmp_path):
_seed(tmp_path / "test.db")
response = client.post(
"/chats/chat_bot_a/drawer/edge/summary",
data={
"source_id": "bot_a",
"target_id": "you",
"new_summary": "x" * 2001,
},
)
assert response.status_code == 400
def test_edit_memory_pov_summary_emits_manual_edit_and_updates(
client, tmp_path
):
_seed(tmp_path / "test.db")
with open_db(tmp_path / "test.db") as conn:
memory_id = conn.execute("SELECT id FROM memories LIMIT 1").fetchone()[0]
response = client.post(
"/chats/chat_bot_a/drawer/memory/pov-summary",
data={
"memory_id": str(memory_id),
"new_summary": "Cleaner per-POV restatement of the moment.",
},
)
assert response.status_code == 200
with open_db(tmp_path / "test.db") as conn:
rows = conn.execute(
"SELECT payload_json FROM event_log WHERE kind = 'manual_edit'"
).fetchall()
assert len(rows) == 1
payload = json.loads(rows[0][0])
assert payload["target_kind"] == "memory_pov_summary"
assert payload["prior_value"] == "Original summary text."
assert payload["new_value"].startswith("Cleaner per-POV")
assert payload["target_id"] == memory_id
pov = conn.execute(
"SELECT pov_summary FROM memories WHERE id = ?", (memory_id,)
).fetchone()[0]
assert pov.startswith("Cleaner per-POV")
assert "Cleaner per-POV" in response.text
def test_edit_memory_pov_summary_404_when_wrong_chat(client, tmp_path):
_seed(tmp_path / "test.db")
with open_db(tmp_path / "test.db") as conn:
memory_id = conn.execute("SELECT id FROM memories LIMIT 1").fetchone()[0]
# Re-home the memory to a different chat to confirm the route's
# cross-chat guard fires.
conn.execute(
"UPDATE memories SET chat_id = 'other_chat' WHERE id = ?",
(memory_id,),
)
conn.commit()
response = client.post(
"/chats/chat_bot_a/drawer/memory/pov-summary",
data={"memory_id": str(memory_id), "new_summary": "..."},
)
assert response.status_code == 404
def test_edit_edge_knowledge_facts_add_emits_event_and_appends(client, tmp_path):
_seed(tmp_path / "test.db")
response = client.post(
"/chats/chat_bot_a/drawer/edge/knowledge-facts",
data={
"source_id": "bot_a",
"target_id": "you",
"action": "add",
"fact": "lent you a textbook",
},
)
assert response.status_code == 200
with open_db(tmp_path / "test.db") as conn:
rows = conn.execute(
"SELECT payload_json FROM event_log WHERE kind = 'manual_edit'"
).fetchall()
assert len(rows) == 1
payload = json.loads(rows[0][0])
assert payload["target_kind"] == "edge_knowledge_fact"
assert payload["new_value"] == {
"action": "add",
"fact": "lent you a textbook",
}
# Prior value snapshots the entire knowledge list before the edit.
assert payload["prior_value"] == ["studied physics together"]
from chat.state.edges import get_edge
edge = get_edge(conn, "bot_a", "you")
assert "lent you a textbook" in edge["knowledge"]
assert "studied physics together" in edge["knowledge"]
assert "lent you a textbook" in response.text
def test_edit_edge_knowledge_facts_remove_drops_matching_fact(client, tmp_path):
_seed(tmp_path / "test.db")
response = client.post(
"/chats/chat_bot_a/drawer/edge/knowledge-facts",
data={
"source_id": "bot_a",
"target_id": "you",
"action": "remove",
"fact": "studied physics together",
},
)
assert response.status_code == 200
with open_db(tmp_path / "test.db") as conn:
from chat.state.edges import get_edge
edge = get_edge(conn, "bot_a", "you")
assert "studied physics together" not in edge["knowledge"]
rows = conn.execute(
"SELECT payload_json FROM event_log WHERE kind = 'manual_edit'"
).fetchall()
payload = json.loads(rows[0][0])
assert payload["target_kind"] == "edge_knowledge_fact"
assert payload["new_value"]["action"] == "remove"
def test_edit_edge_knowledge_facts_400_on_bad_action(client, tmp_path):
_seed(tmp_path / "test.db")
response = client.post(
"/chats/chat_bot_a/drawer/edge/knowledge-facts",
data={
"source_id": "bot_a",
"target_id": "you",
"action": "delete",
"fact": "x",
},
)
assert response.status_code == 400
# --- T72.3 tests (witness flag inline-edit) -------------------------------
def test_witness_flag_toggle_updates_memory_row(client, tmp_path):
"""Memory seeded with witness [you=1, host=1, guest=0]; toggling
``guest`` to 1 lands as ``witness_guest = 1`` after projection.
"""
_seed(tmp_path / "test.db")
with open_db(tmp_path / "test.db") as conn:
memory_id = conn.execute("SELECT id FROM memories LIMIT 1").fetchone()[0]
response = client.post(
"/chats/chat_bot_a/drawer/memory/witness",
data={
"memory_id": str(memory_id),
"flag": "guest",
"new_value": "1",
},
)
assert response.status_code == 200
with open_db(tmp_path / "test.db") as conn:
row = conn.execute(
"SELECT witness_you, witness_host, witness_guest "
"FROM memories WHERE id = ?",
(memory_id,),
).fetchone()
assert row == (1, 1, 1)
def test_witness_flag_toggle_emits_manual_edit_event(client, tmp_path):
_seed(tmp_path / "test.db")
with open_db(tmp_path / "test.db") as conn:
memory_id = conn.execute("SELECT id FROM memories LIMIT 1").fetchone()[0]
response = client.post(
"/chats/chat_bot_a/drawer/memory/witness",
data={
"memory_id": str(memory_id),
"flag": "guest",
"new_value": "1",
},
)
assert response.status_code == 200
with open_db(tmp_path / "test.db") as conn:
rows = conn.execute(
"SELECT payload_json FROM event_log WHERE kind = 'manual_edit'"
).fetchall()
assert len(rows) == 1
payload = json.loads(rows[0][0])
assert payload["target_kind"] == "memory_witness"
assert payload["target_id"] == memory_id
assert payload["prior_value"] == {"flag": "guest", "value": 0}
assert payload["new_value"] == {"flag": "guest", "value": 1}
def test_witness_flag_toggle_400_on_bad_flag(client, tmp_path):
_seed(tmp_path / "test.db")
with open_db(tmp_path / "test.db") as conn:
memory_id = conn.execute("SELECT id FROM memories LIMIT 1").fetchone()[0]
response = client.post(
"/chats/chat_bot_a/drawer/memory/witness",
data={
"memory_id": str(memory_id),
"flag": "narrator",
"new_value": "1",
},
)
assert response.status_code == 400
+452
View File
@@ -0,0 +1,452 @@
"""T59: drawer events / threads / skip controls.
Extends the chat drawer with three new sections (Events, Threads, Skip)
and five new POST endpoints:
* ``POST /chats/{chat_id}/drawer/event/plan`` emits ``event_planned``.
* ``POST /chats/{chat_id}/drawer/event/cancel/{event_id}`` emits
``event_cancelled``.
* ``POST /chats/{chat_id}/drawer/skip/elision`` validates new_time,
emits ``time_skip_elision`` plus an ``assistant_turn`` carrying the
narrated transition prose from :mod:`chat.services.skip_narration`.
* ``POST /chats/{chat_id}/drawer/skip/jump`` validates new_time, emits
``time_skip_jump`` plus per-bot synthesized ``memory_written`` events
derived from the user-supplied "anything notable" prose, and an
``assistant_turn`` carrying the narration.
* ``POST /chats/{chat_id}/drawer/thread/close/{thread_id}`` emits
``thread_closed``.
Each route returns the refreshed drawer partial (HTMX swap target) so
the tests assert both the persisted event_log effect AND the rendered
section content. Wire-up follows the T42 ``MockLLMClient`` pattern.
"""
from __future__ import annotations
import json
from pathlib import Path
import pytest
from fastapi.testclient import TestClient
from chat.app import app
from chat.db.connection import open_db
from chat.eventlog.log import append_and_apply, append_event
from chat.eventlog.projector import project
from chat.llm.mock import MockLLMClient
@pytest.fixture
def client(tmp_path, monkeypatch):
cfg = tmp_path / "config.toml"
cfg.write_text('featherless_api_key = "test"\n')
monkeypatch.setenv("CHAT_CONFIG_PATH", str(cfg))
db = tmp_path / "test.db"
monkeypatch.setenv("CHAT_DB_PATH", str(db))
with TestClient(app) as c:
if hasattr(app.state, "background_worker"):
app.state.background_worker.enabled = False
yield c
def _bot_payload(bot_id: str, name: str) -> dict:
return {
"id": bot_id,
"name": name,
"persona": "...",
"voice_samples": [],
"traits": [],
"backstory": "",
"initial_relationship_to_you": "",
"kickoff_prose": "",
}
def _seed_chat(db: Path, *, with_scene: bool = True) -> None:
"""Seed a chat hosted by ``bot_a`` (with ``bot_b`` authored as a
candidate guest) so the skip-jump path can write per-bot synthesized
memories when a guest is present.
"""
with open_db(db) as conn:
append_event(conn, kind="bot_authored", payload=_bot_payload("bot_a", "BotA"))
append_event(conn, kind="bot_authored", payload=_bot_payload("bot_b", "BotB"))
append_event(
conn,
kind="you_authored",
payload={"name": "Me", "pronouns": "they/them", "persona": ""},
)
append_event(
conn,
kind="chat_created",
payload={
"id": "chat_bot_a",
"host_bot_id": "bot_a",
"initial_time": "2026-04-26T20:00:00+00:00",
"narrative_anchor": "Day 1",
"weather": "",
},
)
if with_scene:
append_event(
conn,
kind="scene_opened",
payload={
"chat_id": "chat_bot_a",
"container_id": None,
"started_at": "2026-04-26T20:00:00+00:00",
"participants": ["you", "bot_a"],
},
)
project(conn)
def _override_llm(canned: list[str]):
"""Wire a ``MockLLMClient`` into the drawer's LLM dependency."""
from chat.web.kickoff import get_llm_client
app.dependency_overrides[get_llm_client] = lambda: MockLLMClient(
canned=list(canned)
)
# ---------------------------------------------------------------------------
# 1. Empty drawer state — Events + Threads sections render but show
# empty-state copy (no row markup) when no events / threads exist.
# ---------------------------------------------------------------------------
def test_get_drawer_with_no_events_or_threads_omits_sections(client, tmp_path):
_seed_chat(tmp_path / "test.db")
response = client.get("/chats/chat_bot_a/drawer")
assert response.status_code == 200
body = response.text
# Sections render with empty-state copy — deterministic markers.
assert "<h3>Events</h3>" in body
assert "<h3>Threads</h3>" in body
assert "No active events" in body
assert "No open threads" in body
# Skip controls always render under Activity (gated by chat clock).
assert "Elision skip" in body
assert "Jump skip" in body
# ---------------------------------------------------------------------------
# 2. POST event/plan — event_planned lands and the drawer lists it.
# ---------------------------------------------------------------------------
def test_post_event_plan_appends_event_planned_and_renders(client, tmp_path):
_seed_chat(tmp_path / "test.db")
response = client.post(
"/chats/chat_bot_a/drawer/event/plan",
data={
"kind": "dinner_reservation",
"planned_for": "2026-04-26T19:00:00+00:00",
"props_json": json.dumps({"restaurant": "Bistro X"}),
},
)
assert response.status_code == 200
with open_db(tmp_path / "test.db") as conn:
rows = conn.execute(
"SELECT payload_json FROM event_log WHERE kind = 'event_planned'"
).fetchall()
assert len(rows) == 1
payload = json.loads(rows[0][0])
assert payload["kind"] == "dinner_reservation"
assert payload["chat_id"] == "chat_bot_a"
assert payload["planned_for"] == "2026-04-26T19:00:00+00:00"
assert payload["props"] == {"restaurant": "Bistro X"}
assert payload["event_id"].startswith("evt_")
# Refreshed partial lists the new event by kind.
body = response.text
assert "dinner_reservation" in body
def test_post_event_plan_invalid_props_json_returns_400(client, tmp_path):
_seed_chat(tmp_path / "test.db")
response = client.post(
"/chats/chat_bot_a/drawer/event/plan",
data={
"kind": "dinner_reservation",
"planned_for": "2026-04-26T19:00:00+00:00",
"props_json": "not valid json {",
},
)
assert response.status_code == 400
# ---------------------------------------------------------------------------
# 3. POST event/cancel — event_cancelled lands and the active list drops it.
# ---------------------------------------------------------------------------
def test_post_event_cancel_appends_event_cancelled(client, tmp_path):
_seed_chat(tmp_path / "test.db")
# Plan first via the route so the test exercises both sides.
plan_resp = client.post(
"/chats/chat_bot_a/drawer/event/plan",
data={
"kind": "doctor_visit",
"planned_for": "2026-04-27T09:00:00+00:00",
"props_json": "{}",
},
)
assert plan_resp.status_code == 200
with open_db(tmp_path / "test.db") as conn:
row = conn.execute(
"SELECT payload_json FROM event_log WHERE kind = 'event_planned' "
"ORDER BY id DESC LIMIT 1"
).fetchone()
event_id = json.loads(row[0])["event_id"]
cancel_resp = client.post(
f"/chats/chat_bot_a/drawer/event/cancel/{event_id}"
)
assert cancel_resp.status_code == 200
with open_db(tmp_path / "test.db") as conn:
cancelled = conn.execute(
"SELECT payload_json FROM event_log WHERE kind = 'event_cancelled'"
).fetchall()
assert len(cancelled) == 1
cp = json.loads(cancelled[0][0])
assert cp["event_id"] == event_id
# Active-events query should no longer surface this event.
from chat.state.events import list_active_events
assert list_active_events(conn, "chat_bot_a") == []
# ---------------------------------------------------------------------------
# 4. POST skip/elision — emits time_skip_elision + assistant_turn narration.
# ---------------------------------------------------------------------------
def test_post_skip_elision_advances_clock_and_emits_narration(client, tmp_path):
_seed_chat(tmp_path / "test.db")
canned_narration = "We pull up to the curb just before sunset."
_override_llm([canned_narration])
try:
response = client.post(
"/chats/chat_bot_a/drawer/skip/elision",
data={
"landing_state_hint": "arriving at the venue",
"new_time": "2026-04-26T20:30:00+00:00",
},
)
assert response.status_code == 200
finally:
app.dependency_overrides.clear()
with open_db(tmp_path / "test.db") as conn:
from chat.state.world import get_chat
chat = get_chat(conn, "chat_bot_a")
assert chat["time"] == "2026-04-26T20:30:00+00:00"
skip_rows = conn.execute(
"SELECT payload_json FROM event_log WHERE kind = 'time_skip_elision'"
).fetchall()
assert len(skip_rows) == 1
sp = json.loads(skip_rows[0][0])
assert sp["chat_id"] == "chat_bot_a"
assert sp["new_time"] == "2026-04-26T20:30:00+00:00"
# An assistant_turn event landed with the narration text.
turn_rows = conn.execute(
"SELECT payload_json FROM event_log WHERE kind = 'assistant_turn'"
).fetchall()
assert len(turn_rows) == 1
tp = json.loads(turn_rows[0][0])
assert tp["chat_id"] == "chat_bot_a"
assert tp["text"].strip() # non-empty narration
assert tp["speaker_id"] == "bot_a"
def test_post_skip_elision_invalid_time_returns_400(client, tmp_path):
_seed_chat(tmp_path / "test.db")
_override_llm([])
try:
# Garbled ISO timestamp.
bad_resp = client.post(
"/chats/chat_bot_a/drawer/skip/elision",
data={"landing_state_hint": "x", "new_time": "not-a-time"},
)
assert bad_resp.status_code == 400
# Backwards-in-time skip: chat seeded at 20:00, asking 19:00.
backwards_resp = client.post(
"/chats/chat_bot_a/drawer/skip/elision",
data={
"landing_state_hint": "x",
"new_time": "2026-04-26T19:00:00+00:00",
},
)
assert backwards_resp.status_code == 400
finally:
app.dependency_overrides.clear()
# ---------------------------------------------------------------------------
# 5. POST skip/jump — synthesized memories per present bot + narration.
# ---------------------------------------------------------------------------
def test_post_skip_jump_with_notable_prose_writes_synthesized_memories(
client, tmp_path
):
_seed_chat(tmp_path / "test.db")
# Single host present (no guest) — exactly one synthesize call,
# one narration call. The synthesize digest carries two memories so
# we can assert N writes lands the right shape.
digest_json = json.dumps(
{
"memories": [
{
"text": "We bumped into an old friend at the cafe.",
"significance": 1,
"affinity_delta": 0,
"trust_delta": 0,
},
{
"text": "It started raining on the walk home.",
"significance": 1,
"affinity_delta": 0,
"trust_delta": 0,
},
]
}
)
narration = "The afternoon slipped by quickly."
_override_llm([digest_json, narration])
try:
response = client.post(
"/chats/chat_bot_a/drawer/skip/jump",
data={
"new_time": "2026-04-27T08:00:00+00:00",
"notable_prose": (
"We ran into an old friend, and it rained on the way back."
),
"reset_activity": "1",
},
)
assert response.status_code == 200
finally:
app.dependency_overrides.clear()
with open_db(tmp_path / "test.db") as conn:
from chat.state.world import get_chat
chat = get_chat(conn, "chat_bot_a")
assert chat["time"] == "2026-04-27T08:00:00+00:00"
jump_rows = conn.execute(
"SELECT payload_json FROM event_log WHERE kind = 'time_skip_jump'"
).fetchall()
assert len(jump_rows) == 1
jp = json.loads(jump_rows[0][0])
assert jp["chat_id"] == "chat_bot_a"
assert jp["new_time"] == "2026-04-27T08:00:00+00:00"
assert jp["reset_activity"] is True
# Two synthesized memories land for the lone host bot
# (record_turn_memory_for_present writes one row per present bot
# per call — host only here, so 2 memories x 1 bot = 2 events).
mem_rows = conn.execute(
"SELECT payload_json FROM event_log WHERE kind = 'memory_written'"
).fetchall()
synth_payloads = [
json.loads(r[0])
for r in mem_rows
if json.loads(r[0]).get("source") == "synthesized"
]
assert len(synth_payloads) == 2
for p in synth_payloads:
assert p["owner_id"] == "bot_a"
assert p["chat_id"] == "chat_bot_a"
# And the assistant_turn narration landed.
turn_rows = conn.execute(
"SELECT payload_json FROM event_log WHERE kind = 'assistant_turn'"
).fetchall()
assert len(turn_rows) == 1
tp = json.loads(turn_rows[0][0])
assert tp["text"].strip()
assert tp["speaker_id"] == "bot_a"
def test_post_skip_jump_with_empty_prose_skips_memory_writes(client, tmp_path):
_seed_chat(tmp_path / "test.db")
# Empty prose short-circuits in synthesize_memories before any LLM call,
# so the canned queue only needs the narration.
narration = "(next morning: still in the kitchen.)"
_override_llm([narration])
try:
response = client.post(
"/chats/chat_bot_a/drawer/skip/jump",
data={
"new_time": "2026-04-27T08:00:00+00:00",
"notable_prose": " ",
"reset_activity": "",
},
)
assert response.status_code == 200
finally:
app.dependency_overrides.clear()
with open_db(tmp_path / "test.db") as conn:
synth = conn.execute(
"SELECT COUNT(*) FROM event_log WHERE kind = 'memory_written' "
"AND payload_json LIKE '%synthesized%'"
).fetchone()[0]
assert synth == 0
# ---------------------------------------------------------------------------
# 6. POST thread/close — thread_closed lands and the open list drops it.
# ---------------------------------------------------------------------------
def test_post_thread_close_appends_thread_closed(client, tmp_path):
_seed_chat(tmp_path / "test.db")
# Open a thread directly via append_and_apply so the test focuses on
# the close route's effect.
with open_db(tmp_path / "test.db") as conn:
append_and_apply(
conn,
kind="thread_opened",
payload={
"thread_id": "thr_alpha",
"chat_id": "chat_bot_a",
"title": "the missing key",
"summary": "Couldn't find the key.",
},
)
response = client.post("/chats/chat_bot_a/drawer/thread/close/thr_alpha")
assert response.status_code == 200
with open_db(tmp_path / "test.db") as conn:
closed = conn.execute(
"SELECT payload_json FROM event_log WHERE kind = 'thread_closed'"
).fetchall()
assert len(closed) == 1
cp = json.loads(closed[0][0])
assert cp["thread_id"] == "thr_alpha"
from chat.state.threads import list_open_threads
assert list_open_threads(conn, "chat_bot_a") == []
+478
View File
@@ -0,0 +1,478 @@
"""T42: drawer guest add/remove + render.
The drawer grows a "Guest" section (when a guest bot is present in the
chat), a "Group" section sourced from the ``group_node`` row, an
"Add guest" form (visible while no guest is present), and a "Remove
guest" button (visible while one is). The two new POST endpoints emit
``guest_added`` / ``guest_removed`` events plus ancillary updates:
* ``POST /chats/{chat_id}/drawer/guest/add`` runs the relationship-seed
classifier (T38) over the user-supplied prose and emits an
``edge_update`` per direction when the seed comes back non-default.
It also seeds a ``group_node_initialized`` row when none exists yet.
* ``POST /chats/{chat_id}/drawer/guest/remove`` first emits
``scene_closed`` for the active scene so the host -> you scene closes
cleanly before the guest leaves.
"""
from __future__ import annotations
import json
from pathlib import Path
import pytest
from fastapi.testclient import TestClient
from chat.app import app
from chat.db.connection import open_db
from chat.eventlog.log import append_event
from chat.eventlog.projector import project
from chat.llm.mock import MockLLMClient
@pytest.fixture
def client(tmp_path, monkeypatch):
cfg = tmp_path / "config.toml"
cfg.write_text('featherless_api_key = "test"\n')
monkeypatch.setenv("CHAT_CONFIG_PATH", str(cfg))
db = tmp_path / "test.db"
monkeypatch.setenv("CHAT_DB_PATH", str(db))
with TestClient(app) as c:
if hasattr(app.state, "background_worker"):
app.state.background_worker.enabled = False
yield c
def _bot_payload(bot_id: str, name: str) -> dict:
return {
"id": bot_id,
"name": name,
"persona": "...",
"voice_samples": [],
"traits": [],
"backstory": "",
"initial_relationship_to_you": "",
"kickoff_prose": "",
}
def _seed_chat(db: Path, *, with_scene: bool = True) -> None:
"""Seed a chat hosted by ``bot_a`` (with ``bot_b`` authored as a
candidate guest) and, by default, an open scene so the
``guest_removed`` flow has something to close.
"""
with open_db(db) as conn:
append_event(conn, kind="bot_authored", payload=_bot_payload("bot_a", "BotA"))
append_event(conn, kind="bot_authored", payload=_bot_payload("bot_b", "BotB"))
append_event(
conn,
kind="you_authored",
payload={"name": "Me", "pronouns": "they/them", "persona": ""},
)
append_event(
conn,
kind="chat_created",
payload={
"id": "chat_bot_a",
"host_bot_id": "bot_a",
"initial_time": "2026-04-26T20:00:00+00:00",
"narrative_anchor": "Day 1",
"weather": "",
},
)
if with_scene:
append_event(
conn,
kind="scene_opened",
payload={
"chat_id": "chat_bot_a",
"container_id": None,
"started_at": "2026-04-26T20:00:00+00:00",
"participants": ["you", "bot_a"],
},
)
project(conn)
def _override_llm(canned: list[str]):
"""Wire a ``MockLLMClient`` into the drawer's LLM dependency."""
from chat.web.kickoff import get_llm_client
app.dependency_overrides[get_llm_client] = lambda: MockLLMClient(
canned=list(canned)
)
def test_drawer_no_guest_omits_guest_section(client, tmp_path):
_seed_chat(tmp_path / "test.db")
response = client.get("/chats/chat_bot_a/drawer")
assert response.status_code == 200
body = response.text
# No guest-section header; the "Add guest" form should be visible instead.
assert "<h3>Guest</h3>" not in body
assert "Add guest" in body
def test_drawer_add_guest_seeds_edges_and_group_node(client, tmp_path):
_seed_chat(tmp_path / "test.db")
canned = json.dumps(
{
"a_to_b_summary": "old college friend",
"a_to_b_knowledge_facts": ["studied physics together"],
"a_to_b_affinity_delta": 4,
"a_to_b_trust_delta": -1,
"b_to_a_summary": "former roommate",
"b_to_a_knowledge_facts": ["lived together junior year"],
"b_to_a_affinity_delta": 3,
"b_to_a_trust_delta": 0,
}
)
_override_llm([canned])
try:
response = client.post(
"/chats/chat_bot_a/drawer/guest/add",
data={
"guest_bot_id": "bot_b",
"relationship_prose": (
"Alice and Bob met in college and studied physics together."
),
},
)
assert response.status_code == 200
finally:
app.dependency_overrides.clear()
with open_db(tmp_path / "test.db") as conn:
from chat.state.edges import get_edge
from chat.state.group_node import get_group_node
from chat.state.world import get_chat
chat = get_chat(conn, "chat_bot_a")
assert chat["guest_bot_id"] == "bot_b"
edge_a_to_b = get_edge(conn, "bot_a", "bot_b")
edge_b_to_a = get_edge(conn, "bot_b", "bot_a")
# Seed deltas applied around the 50/50 default.
assert edge_a_to_b["affinity"] == 54
assert edge_a_to_b["trust"] == 49
assert "studied physics together" in edge_a_to_b["knowledge"]
assert edge_b_to_a["affinity"] == 53
assert edge_b_to_a["trust"] == 50
assert "lived together junior year" in edge_b_to_a["knowledge"]
group = get_group_node(conn, "chat_bot_a")
assert group is not None
assert set(group["members"]) == {"you", "bot_a", "bot_b"}
def test_drawer_add_guest_empty_prose_skips_edge_update(client, tmp_path):
_seed_chat(tmp_path / "test.db")
# No canned responses: the seed function short-circuits on empty prose
# so no LLM call should happen.
_override_llm([])
try:
response = client.post(
"/chats/chat_bot_a/drawer/guest/add",
data={"guest_bot_id": "bot_b", "relationship_prose": " "},
)
assert response.status_code == 200
finally:
app.dependency_overrides.clear()
with open_db(tmp_path / "test.db") as conn:
from chat.state.world import get_chat
chat = get_chat(conn, "chat_bot_a")
assert chat["guest_bot_id"] == "bot_b"
# guest_added fires but no edge_update events between bot_a and bot_b.
added = conn.execute(
"SELECT COUNT(*) FROM event_log WHERE kind = 'guest_added'"
).fetchone()[0]
assert added == 1
edge_updates = conn.execute(
"SELECT payload_json FROM event_log WHERE kind = 'edge_update'"
).fetchall()
for (payload_json,) in edge_updates:
payload = json.loads(payload_json)
pair = {payload.get("source_id"), payload.get("target_id")}
assert pair != {"bot_a", "bot_b"}, (
"no edge_update should be emitted between host and guest "
"when prose is empty"
)
def test_drawer_add_guest_when_already_present_returns_400(client, tmp_path):
_seed_chat(tmp_path / "test.db")
# Pre-attach a guest directly via append_and_apply so we don't replay
# the prior chat_created (which would violate UNIQUE on chats.id).
from chat.eventlog.log import append_and_apply
with open_db(tmp_path / "test.db") as conn:
append_and_apply(
conn,
kind="bot_authored",
payload=_bot_payload("bot_c", "BotC"),
)
append_and_apply(
conn,
kind="guest_added",
payload={"chat_id": "chat_bot_a", "guest_bot_id": "bot_b"},
)
_override_llm([])
try:
response = client.post(
"/chats/chat_bot_a/drawer/guest/add",
data={"guest_bot_id": "bot_c", "relationship_prose": ""},
)
assert response.status_code == 400
finally:
app.dependency_overrides.clear()
def test_drawer_remove_guest_clears_and_closes_scene(client, tmp_path):
_seed_chat(tmp_path / "test.db")
from chat.eventlog.log import append_and_apply
with open_db(tmp_path / "test.db") as conn:
append_and_apply(
conn,
kind="guest_added",
payload={"chat_id": "chat_bot_a", "guest_bot_id": "bot_b"},
)
response = client.post("/chats/chat_bot_a/drawer/guest/remove")
assert response.status_code == 200
with open_db(tmp_path / "test.db") as conn:
from chat.state.world import active_scene, get_chat
chat = get_chat(conn, "chat_bot_a")
assert chat["guest_bot_id"] is None
assert active_scene(conn, "chat_bot_a") is None
kinds = [
row[0]
for row in conn.execute(
"SELECT kind FROM event_log ORDER BY id"
).fetchall()
]
# scene_closed must precede guest_removed in the log.
assert "scene_closed" in kinds
assert "guest_removed" in kinds
assert kinds.index("scene_closed") < kinds.index("guest_removed")
# --- T72.2 first-meeting gate ----------------------------------------------
def _seed_host_to_guest_edge(db: Path) -> None:
"""Materialise a bot_a -> bot_b edge so the gate's check fires."""
from chat.eventlog.log import append_and_apply
with open_db(db) as conn:
append_and_apply(
conn,
kind="edge_update",
payload={
"source_id": "bot_a",
"target_id": "bot_b",
"chat_id": "chat_bot_a",
"affinity_delta": 0,
"knowledge_facts": ["already met before"],
},
)
def test_add_guest_form_disables_prose_when_edge_exists(client, tmp_path):
"""When host->candidate edge already exists, the GET partial renders
the textarea disabled and surfaces the "already know each other"
message so the user knows submitting will skip the seed.
"""
_seed_chat(tmp_path / "test.db")
_seed_host_to_guest_edge(tmp_path / "test.db")
response = client.get("/chats/chat_bot_a/drawer")
assert response.status_code == 200
body = response.text
# Note + disabled state both present. The textarea sits next to the
# ``add-guest-prose`` class so we can match it specifically.
assert "already know each other" in body
assert 'class="add-guest-prose"' in body
# The textarea for the first (auto-selected) candidate should be
# disabled in the initial markup since an edge exists.
assert "disabled" in body.split('class="add-guest-prose"', 1)[1].split(">", 1)[0]
# And the option carries the ``data-existing-edge="true"`` attribute
# the inline JS uses to flip state on subsequent select changes.
assert 'data-existing-edge="true"' in body
def test_add_guest_with_existing_edge_skips_seed_call(client, tmp_path):
"""Submitting the Add-guest form WITHOUT toggling re-seed must skip
``seed_inter_bot_edges`` entirely. We assert this via an empty mock
queue: if the seed function had been called it would have consumed
a canned response (or raised because none was available).
"""
_seed_chat(tmp_path / "test.db")
_seed_host_to_guest_edge(tmp_path / "test.db")
# Empty queue: any classifier call would raise inside MockLLMClient.
canned_queue: list[str] = []
_override_llm(canned_queue)
try:
response = client.post(
"/chats/chat_bot_a/drawer/guest/add",
data={
"guest_bot_id": "bot_b",
"relationship_prose": "ignored prose",
# NO reseed flag — gate should suppress the seed call.
},
)
assert response.status_code == 200
finally:
app.dependency_overrides.clear()
with open_db(tmp_path / "test.db") as conn:
from chat.state.edges import get_edge
from chat.state.world import get_chat
chat = get_chat(conn, "chat_bot_a")
assert chat["guest_bot_id"] == "bot_b"
# The pre-seeded knowledge fact survives — proof the seed didn't run
# and overwrite the existing edge.
edge = get_edge(conn, "bot_a", "bot_b")
assert "already met before" in edge["knowledge"]
# Exactly one guest_added; no new edge_update events between
# bot_a and bot_b (the pre-seed edge_update from the test setup
# is the only edge_update on this pair).
added = conn.execute(
"SELECT COUNT(*) FROM event_log WHERE kind = 'guest_added'"
).fetchone()[0]
assert added == 1
edge_updates = conn.execute(
"SELECT payload_json FROM event_log WHERE kind = 'edge_update'"
).fetchall()
# Only the pre-seed edge_update from _seed_host_to_guest_edge.
ab_updates = [
json.loads(p[0])
for p in edge_updates
if {
json.loads(p[0]).get("source_id"),
json.loads(p[0]).get("target_id"),
}
== {"bot_a", "bot_b"}
]
assert len(ab_updates) == 1
assert ab_updates[0]["knowledge_facts"] == ["already met before"]
def test_add_guest_with_existing_edge_and_reseed_runs_seed(client, tmp_path):
"""Toggling ``re-seed anyway`` flips the gate off — the existing
flow runs (seed produces deltas, two ``edge_update`` events fire).
"""
_seed_chat(tmp_path / "test.db")
_seed_host_to_guest_edge(tmp_path / "test.db")
canned = json.dumps(
{
"a_to_b_summary": "reconnected",
"a_to_b_knowledge_facts": ["new fact"],
"a_to_b_affinity_delta": 2,
"a_to_b_trust_delta": 1,
"b_to_a_summary": "reconnected",
"b_to_a_knowledge_facts": [],
"b_to_a_affinity_delta": 1,
"b_to_a_trust_delta": 0,
}
)
_override_llm([canned])
try:
response = client.post(
"/chats/chat_bot_a/drawer/guest/add",
data={
"guest_bot_id": "bot_b",
"relationship_prose": "fresh prose",
"reseed": "1",
},
)
assert response.status_code == 200
finally:
app.dependency_overrides.clear()
with open_db(tmp_path / "test.db") as conn:
edge_updates = conn.execute(
"SELECT payload_json FROM event_log WHERE kind = 'edge_update'"
).fetchall()
# Pre-seed (1) + two from the re-seed = 3 edge_updates total.
ab_updates = [
json.loads(p[0])
for p in edge_updates
if {
json.loads(p[0]).get("source_id"),
json.loads(p[0]).get("target_id"),
}
== {"bot_a", "bot_b"}
]
assert len(ab_updates) == 3
def test_drawer_with_guest_renders_guest_and_group_sections(client, tmp_path):
_seed_chat(tmp_path / "test.db")
from chat.eventlog.log import append_and_apply
with open_db(tmp_path / "test.db") as conn:
append_and_apply(
conn,
kind="guest_added",
payload={"chat_id": "chat_bot_a", "guest_bot_id": "bot_b"},
)
# Activity for the guest so the section has content to render.
append_and_apply(
conn,
kind="activity_change",
payload={
"entity_id": "bot_b",
"posture": "leaning",
"action": {"verb": "smirking"},
"attention": "BotA",
},
)
# Edges in all four directions involving the guest.
for src, tgt in (("bot_a", "bot_b"), ("bot_b", "bot_a"), ("you", "bot_b"), ("bot_b", "you")):
append_and_apply(
conn,
kind="edge_update",
payload={
"source_id": src,
"target_id": tgt,
"chat_id": "chat_bot_a",
"affinity_delta": 1,
},
)
append_and_apply(
conn,
kind="group_node_initialized",
payload={
"chat_id": "chat_bot_a",
"members": ["you", "bot_a", "bot_b"],
"summary": "Three friends catching up over drinks.",
"dynamic": "warm and conspiratorial",
},
)
response = client.get("/chats/chat_bot_a/drawer")
assert response.status_code == 200
body = response.text
assert "<h3>Guest</h3>" in body
assert "BotB" in body
assert "smirking" in body
assert "<h3>Group</h3>" in body
assert "Three friends catching up over drinks." in body
assert "warm and conspiratorial" in body
# "Remove guest" button is visible when a guest is present.
assert "Remove guest" in body
+103
View File
@@ -0,0 +1,103 @@
"""Tests for the event-lifecycle detection service (T52).
Per Phase 3, after each narrated turn we ask a classifier whether any
active events transitioned (started, completed, cancelled). The bias is
strongly toward an empty result most turns do NOT resolve or start a
known event, and the turn-flow caller (T61) only appends an
event_started/completed/cancelled record when this service yields one.
These tests cover:
* The classifier returning a single transition is honored end-to-end.
* An empty ``active_events`` list short-circuits before any LLM call,
so callers that hold no live events pay zero classifier cost.
* Three rounds of malformed JSON exhaust ``classify``'s retries and we
fall back to the empty default graceful degradation per §3.3.
"""
from __future__ import annotations
import json
import pytest
from chat.llm.mock import MockLLMClient
from chat.services.event_lifecycle import (
EventLifecycleDecision,
detect_event_transitions,
)
@pytest.mark.asyncio
async def test_detects_one_transition_happy_path():
canned = json.dumps(
{
"transitions": [
{
"event_id": "evt_1",
"new_status": "completed",
"reason": "they arrived at the park",
}
]
}
)
mock = MockLLMClient(canned=[canned])
result = await detect_event_transitions(
mock,
classifier_model="x",
narrative_text="They walked through the park gate, finally there.",
active_events=[
{
"event_id": "evt_1",
"kind": "date_at_park",
"status": "active",
"props": {},
}
],
)
assert isinstance(result, EventLifecycleDecision)
assert len(result.transitions) == 1
assert result.transitions[0].event_id == "evt_1"
assert result.transitions[0].new_status == "completed"
assert result.transitions[0].reason == "they arrived at the park"
@pytest.mark.asyncio
async def test_empty_active_events_short_circuits_without_classifier_call():
"""No active events -> no classifier call.
The mock has an empty canned list; any ``generate`` call would raise
``IndexError`` from ``list.pop(0)``. The test passing proves the
short-circuit holds.
"""
mock = MockLLMClient(canned=[])
result = await detect_event_transitions(
mock,
classifier_model="x",
narrative_text="Just a quiet moment between them.",
active_events=[],
)
assert isinstance(result, EventLifecycleDecision)
assert result.transitions == []
@pytest.mark.asyncio
async def test_classifier_failure_returns_empty_default():
"""``classify`` retries 3 times; after all fail it returns the empty
default so the turn flow keeps moving (§3.3 graceful degradation)."""
mock = MockLLMClient(canned=["bad", "bad", "bad"])
result = await detect_event_transitions(
mock,
classifier_model="x",
narrative_text="Some text the classifier will choke on.",
active_events=[
{
"event_id": "evt_1",
"kind": "date_at_park",
"status": "active",
"props": {},
}
],
)
assert isinstance(result, EventLifecycleDecision)
assert result.transitions == []
+256
View File
@@ -0,0 +1,256 @@
"""Tests for the event-completion promotion service (T56).
When an event reaches ``status='completed'``, the orchestrator promotes
structured artifacts the event carried (``acquired_objects``,
``knowledge_facts``, ``relationship_change``) into the appropriate
state stores via downstream events. Cancelled / expired events do NOT
promote the closed event row is left in place but no follow-on
events fire.
"""
from __future__ import annotations
import json
from chat.db.connection import open_db
from chat.db.migrate import apply_migrations
from chat.eventlog.log import append_event
from chat.eventlog.projector import project
from chat.services.event_promotion import promote_completed_event
from chat.state.edges import get_edge
import chat.state.edges # noqa: F401 - register edge_update handler
import chat.state.entities # noqa: F401 - register handlers
import chat.state.events # noqa: F401 - register events handlers
import chat.state.manual_edit # noqa: F401 - register manual_edit handler
import chat.state.world # noqa: F401 - register handlers
def _bot_payload(bot_id: str, name: str) -> dict:
return {
"id": bot_id,
"name": name,
"persona": "thoughtful, observant",
"voice_samples": [],
"traits": [],
"backstory": "",
"initial_relationship_to_you": "coworker",
"kickoff_prose": "",
}
def _chat_payload(chat_id: str = "chat_bot_a") -> dict:
return {
"id": chat_id,
"host_bot_id": "bot_a",
"guest_bot_id": "bot_b",
"initial_time": "2026-04-26T20:00:00+00:00",
"narrative_anchor": "Day 1 evening",
"weather": "clear",
}
def _seed_chat(conn) -> None:
append_event(conn, kind="bot_authored", payload=_bot_payload("bot_a", "BotA"))
append_event(conn, kind="bot_authored", payload=_bot_payload("bot_b", "BotB"))
append_event(conn, kind="chat_created", payload=_chat_payload())
def _seed_event(
conn,
*,
event_id: str,
props: dict,
terminal_kind: str = "event_completed",
) -> None:
"""Append event_planned, then a terminal transition (default completed)."""
append_event(
conn,
kind="event_planned",
payload={
"event_id": event_id,
"chat_id": "chat_bot_a",
"kind": "story_event",
"props": props,
"planned_for": "2026-04-30T18:00:00+00:00",
},
)
append_event(
conn,
kind=terminal_kind,
payload={
"event_id": event_id,
"completed_at": "2026-04-30T20:00:00+00:00",
},
)
project(conn)
def _max_event_id(conn) -> int:
return conn.execute("SELECT COALESCE(MAX(id), 0) FROM event_log").fetchone()[0]
def _events_after(conn, after_id: int, kind: str) -> list[dict]:
rows = conn.execute(
"SELECT id, kind, payload_json FROM event_log "
"WHERE id > ? AND kind = ? ORDER BY id ASC",
(after_id, kind),
).fetchall()
return [
{"id": r[0], "kind": r[1], "payload": json.loads(r[2])} for r in rows
]
def test_empty_props_no_op(tmp_path):
"""Completed event with empty props produces no promotion events."""
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
_seed_chat(conn)
_seed_event(conn, event_id="evt_empty", props={})
before = _max_event_id(conn)
counts = promote_completed_event(
conn,
event_id="evt_empty",
chat_id="chat_bot_a",
chat_clock_at="2026-04-30T20:00:00+00:00",
)
assert counts == {
"acquired_objects": 0,
"knowledge_facts": 0,
"relationship_change": 0,
}
# No new edge_update or manual_edit rows after the promote call.
assert _events_after(conn, before, "edge_update") == []
assert _events_after(conn, before, "manual_edit") == []
def test_knowledge_facts_emits_edge_update(tmp_path):
"""A knowledge_facts entry promotes to an edge_update on the directed edge."""
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
_seed_chat(conn)
_seed_event(
conn,
event_id="evt_kf",
props={
"knowledge_facts": [
{
"owner_id": "bot_a",
"target_id": "you",
"fact": "Maya prefers tea over coffee",
}
]
},
)
before = _max_event_id(conn)
counts = promote_completed_event(
conn,
event_id="evt_kf",
chat_id="chat_bot_a",
chat_clock_at="2026-04-30T20:00:00+00:00",
)
assert counts["knowledge_facts"] == 1
assert counts["acquired_objects"] == 0
assert counts["relationship_change"] == 0
# An edge_update event landed in the event_log AFTER the promote call.
new_edge_updates = _events_after(conn, before, "edge_update")
assert len(new_edge_updates) == 1
payload = new_edge_updates[0]["payload"]
assert payload["source_id"] == "bot_a"
assert payload["target_id"] == "you"
assert payload["knowledge_facts"] == ["Maya prefers tea over coffee"]
# And the projected edge has the fact applied.
edge = get_edge(conn, "bot_a", "you")
assert edge is not None
assert "Maya prefers tea over coffee" in edge["knowledge"]
def test_relationship_change_emits_manual_edit(tmp_path):
"""A relationship_change promotes to a manual_edit edge_summary."""
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
_seed_chat(conn)
_seed_event(
conn,
event_id="evt_rc",
props={
"relationship_change": {
"source_id": "bot_a",
"target_id": "you",
"summary": "they're now dating",
}
},
)
before = _max_event_id(conn)
counts = promote_completed_event(
conn,
event_id="evt_rc",
chat_id="chat_bot_a",
chat_clock_at="2026-04-30T20:00:00+00:00",
)
assert counts["relationship_change"] == 1
assert counts["knowledge_facts"] == 0
assert counts["acquired_objects"] == 0
new_manual_edits = _events_after(conn, before, "manual_edit")
# Filter to edge_summary only — Phase 3 stub may also emit
# memory_pov_summary entries for acquired_objects, but here there
# are none.
edge_summary_edits = [
m for m in new_manual_edits
if m["payload"].get("target_kind") == "edge_summary"
]
assert len(edge_summary_edits) == 1
payload = edge_summary_edits[0]["payload"]
assert payload["target_kind"] == "edge_summary"
assert payload["target_id"] == {"source_id": "bot_a", "target_id": "you"}
assert payload["new_value"] == "they're now dating"
def test_cancelled_event_does_not_promote(tmp_path):
"""Cancelled events have promotable props ignored — no follow-on events."""
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
_seed_chat(conn)
_seed_event(
conn,
event_id="evt_canx",
props={
"knowledge_facts": [
{"owner_id": "bot_a", "target_id": "you", "fact": "x"}
],
"relationship_change": {
"source_id": "bot_a",
"target_id": "you",
"summary": "ignored",
},
},
terminal_kind="event_cancelled",
)
before = _max_event_id(conn)
counts = promote_completed_event(
conn,
event_id="evt_canx",
chat_id="chat_bot_a",
chat_clock_at="2026-04-30T20:00:00+00:00",
)
assert counts == {
"acquired_objects": 0,
"knowledge_facts": 0,
"relationship_change": 0,
}
assert _events_after(conn, before, "edge_update") == []
assert _events_after(conn, before, "manual_edit") == []
+235
View File
@@ -0,0 +1,235 @@
from __future__ import annotations
from chat.db.connection import open_db
from chat.db.migrate import apply_migrations
from chat.eventlog.log import append_and_apply, append_event
from chat.eventlog.projector import project
import chat.state.entities # registers handlers
import chat.state.world # registers handlers
import chat.state.group_node # registers handlers
import chat.state.events # registers handlers
from chat.state.events import (
get_event,
list_active_events,
list_events_in_status,
)
def _bot_payload(bot_id: str, name: str) -> dict:
return {
"id": bot_id,
"name": name,
"persona": "thoughtful, observant",
"voice_samples": [],
"traits": [],
"backstory": "",
"initial_relationship_to_you": "coworker",
"kickoff_prose": "",
}
def _chat_payload(chat_id: str = "chat_bot_a") -> dict:
return {
"id": chat_id,
"host_bot_id": "bot_a",
"guest_bot_id": "bot_b",
"initial_time": "2026-04-26T20:00:00+00:00",
"narrative_anchor": "Day 1 evening",
"weather": "clear",
}
def _seed_chat(conn) -> None:
append_event(conn, kind="bot_authored", payload=_bot_payload("bot_a", "BotA"))
append_event(conn, kind="bot_authored", payload=_bot_payload("bot_b", "BotB"))
append_event(conn, kind="chat_created", payload=_chat_payload())
def test_event_planned_creates_row(tmp_path):
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
_seed_chat(conn)
append_event(
conn,
kind="event_planned",
payload={
"event_id": "evt_abc",
"chat_id": "chat_bot_a",
"kind": "date_at_park",
"props": {"location": "park"},
"planned_for": "2026-04-30T18:00:00+00:00",
},
)
project(conn)
ev = get_event(conn, "evt_abc")
assert ev is not None
assert ev["event_id"] == "evt_abc"
assert ev["chat_id"] == "chat_bot_a"
assert ev["kind"] == "date_at_park"
assert ev["status"] == "planned"
assert ev["props"]["location"] == "park"
assert ev["planned_for"] == "2026-04-30T18:00:00+00:00"
assert ev["started_at"] is None
assert ev["completed_at"] is None
def test_event_started_then_completed_updates_status(tmp_path):
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
_seed_chat(conn)
append_event(
conn,
kind="event_planned",
payload={
"event_id": "evt_abc",
"chat_id": "chat_bot_a",
"kind": "date_at_park",
"props": {},
"planned_for": "2026-04-30T18:00:00+00:00",
},
)
append_event(
conn,
kind="event_started",
payload={
"event_id": "evt_abc",
"started_at": "2026-04-30T18:01:00+00:00",
},
)
append_event(
conn,
kind="event_completed",
payload={
"event_id": "evt_abc",
"completed_at": "2026-04-30T20:00:00+00:00",
},
)
project(conn)
ev = get_event(conn, "evt_abc")
assert ev is not None
assert ev["status"] == "completed"
assert ev["started_at"] == "2026-04-30T18:01:00+00:00"
assert ev["completed_at"] == "2026-04-30T20:00:00+00:00"
def test_event_cancelled_terminal_subsequent_transitions_ignored(tmp_path):
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
_seed_chat(conn)
append_event(
conn,
kind="event_planned",
payload={
"event_id": "evt_abc",
"chat_id": "chat_bot_a",
"kind": "date_at_park",
"props": {},
"planned_for": "2026-04-30T18:00:00+00:00",
},
)
append_event(
conn,
kind="event_cancelled",
payload={
"event_id": "evt_abc",
"completed_at": "2026-04-30T17:00:00+00:00",
},
)
project(conn)
ev = get_event(conn, "evt_abc")
assert ev is not None
assert ev["status"] == "cancelled"
assert ev["completed_at"] == "2026-04-30T17:00:00+00:00"
# Subsequent event_started must be no-oped because status is terminal.
# Use append_and_apply so we apply ONLY this new event without
# replaying earlier non-idempotent handlers (e.g. chat_created).
append_and_apply(
conn,
kind="event_started",
payload={
"event_id": "evt_abc",
"started_at": "2026-04-30T18:01:00+00:00",
},
)
ev2 = get_event(conn, "evt_abc")
assert ev2 is not None
assert ev2["status"] == "cancelled"
assert ev2["started_at"] is None
# completed_at unchanged from the cancelled transition
assert ev2["completed_at"] == "2026-04-30T17:00:00+00:00"
def test_list_active_events_filters_to_planned_and_active(tmp_path):
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
_seed_chat(conn)
# Four events: one planned, one active, one completed, one cancelled.
for ev_id, kind in [
("evt_planned", "date_at_park"),
("evt_active", "movie_night"),
("evt_done", "dinner"),
("evt_canx", "trip"),
]:
append_event(
conn,
kind="event_planned",
payload={
"event_id": ev_id,
"chat_id": "chat_bot_a",
"kind": kind,
"props": {},
"planned_for": "2026-04-30T18:00:00+00:00",
},
)
append_event(
conn,
kind="event_started",
payload={
"event_id": "evt_active",
"started_at": "2026-04-30T18:01:00+00:00",
},
)
append_event(
conn,
kind="event_started",
payload={
"event_id": "evt_done",
"started_at": "2026-04-30T18:01:00+00:00",
},
)
append_event(
conn,
kind="event_completed",
payload={
"event_id": "evt_done",
"completed_at": "2026-04-30T20:00:00+00:00",
},
)
append_event(
conn,
kind="event_cancelled",
payload={
"event_id": "evt_canx",
"completed_at": "2026-04-30T17:00:00+00:00",
},
)
project(conn)
active = list_active_events(conn, "chat_bot_a")
active_ids = {e["event_id"] for e in active}
assert active_ids == {"evt_planned", "evt_active"}
completed = list_events_in_status(conn, "chat_bot_a", "completed")
assert [e["event_id"] for e in completed] == ["evt_done"]
cancelled = list_events_in_status(conn, "chat_bot_a", "cancelled")
assert [e["event_id"] for e in cancelled] == ["evt_canx"]
+101
View File
@@ -0,0 +1,101 @@
from __future__ import annotations
from chat.db.connection import open_db
from chat.db.migrate import apply_migrations
from chat.eventlog.log import append_event
from chat.eventlog.projector import project
import chat.state.entities # registers handlers
import chat.state.world # registers handlers
import chat.state.group_node # registers handlers
from chat.state.group_node import get_group_node
def _bot_payload(bot_id: str, name: str) -> dict:
return {
"id": bot_id,
"name": name,
"persona": "thoughtful, observant",
"voice_samples": [],
"traits": [],
"backstory": "",
"initial_relationship_to_you": "coworker",
"kickoff_prose": "",
}
def _chat_payload(chat_id: str = "chat_bot_a") -> dict:
return {
"id": chat_id,
"host_bot_id": "bot_a",
"guest_bot_id": "bot_b",
"initial_time": "2026-04-26T20:00:00+00:00",
"narrative_anchor": "Day 1 evening",
"weather": "clear",
}
def test_group_node_initialized_creates_row(tmp_path):
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
append_event(conn, kind="bot_authored", payload=_bot_payload("bot_a", "BotA"))
append_event(conn, kind="bot_authored", payload=_bot_payload("bot_b", "BotB"))
append_event(conn, kind="chat_created", payload=_chat_payload())
append_event(
conn,
kind="group_node_initialized",
payload={
"chat_id": "chat_bot_a",
"members": ["you", "bot_a", "bot_b"],
},
)
project(conn)
gn = get_group_node(conn, "chat_bot_a")
assert gn is not None
assert gn["chat_id"] == "chat_bot_a"
assert gn["members"] == ["you", "bot_a", "bot_b"]
assert gn["summary"] == ""
assert gn["dynamic"] == ""
assert gn["threads"] == []
def test_group_node_updated_changes_summary_and_dynamic(tmp_path):
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
append_event(conn, kind="bot_authored", payload=_bot_payload("bot_a", "BotA"))
append_event(conn, kind="bot_authored", payload=_bot_payload("bot_b", "BotB"))
append_event(conn, kind="chat_created", payload=_chat_payload())
append_event(
conn,
kind="group_node_initialized",
payload={
"chat_id": "chat_bot_a",
"members": ["you", "bot_a", "bot_b"],
},
)
append_event(
conn,
kind="group_node_updated",
payload={
"chat_id": "chat_bot_a",
"summary": "Three coworkers chatting about the project.",
"dynamic": "Tense but cordial.",
},
)
project(conn)
gn = get_group_node(conn, "chat_bot_a")
assert gn is not None
assert gn["summary"] == "Three coworkers chatting about the project."
assert gn["dynamic"] == "Tense but cordial."
# Members preserved across update
assert gn["members"] == ["you", "bot_a", "bot_b"]
def test_get_group_node_returns_none_for_missing_chat(tmp_path):
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
assert get_group_node(conn, "chat_missing") is None
+96
View File
@@ -0,0 +1,96 @@
from __future__ import annotations
from chat.db.connection import open_db
from chat.db.migrate import apply_migrations
from chat.eventlog.log import append_event
from chat.eventlog.projector import project
import chat.state.entities # registers bot_authored handler
import chat.state.world # registers chat_created / guest_added / guest_removed
from chat.state.world import get_chat
def _bot_payload(bot_id: str, name: str) -> dict:
return {
"id": bot_id,
"name": name,
"persona": "...",
"voice_samples": ["sample"],
"traits": ["shy"],
"backstory": "...",
"initial_relationship_to_you": "coworker",
"kickoff_prose": "you stay late",
}
def _chat_payload(**overrides) -> dict:
payload = {
"id": "chat_bot_a",
"host_bot_id": "bot_a",
"initial_time": "2026-04-26T20:00:00+00:00",
"narrative_anchor": "Day 1 evening",
"weather": "clear",
}
payload.update(overrides)
return payload
def test_guest_added_sets_guest_bot_id(tmp_path):
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
append_event(conn, kind="bot_authored", payload=_bot_payload("bot_a", "BotA"))
append_event(conn, kind="bot_authored", payload=_bot_payload("bot_b", "BotB"))
append_event(conn, kind="chat_created", payload=_chat_payload())
append_event(conn, kind="guest_added", payload={
"chat_id": "chat_bot_a",
"guest_bot_id": "bot_b",
})
project(conn)
chat = get_chat(conn, "chat_bot_a")
assert chat is not None
assert chat["guest_bot_id"] == "bot_b"
def test_guest_removed_clears_guest_bot_id(tmp_path):
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
append_event(conn, kind="bot_authored", payload=_bot_payload("bot_a", "BotA"))
append_event(conn, kind="bot_authored", payload=_bot_payload("bot_b", "BotB"))
append_event(conn, kind="chat_created", payload=_chat_payload())
append_event(conn, kind="guest_added", payload={
"chat_id": "chat_bot_a",
"guest_bot_id": "bot_b",
})
append_event(conn, kind="guest_removed", payload={
"chat_id": "chat_bot_a",
})
project(conn)
chat = get_chat(conn, "chat_bot_a")
assert chat is not None
assert chat["guest_bot_id"] is None
def test_guest_added_idempotent_overwrite(tmp_path):
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
append_event(conn, kind="bot_authored", payload=_bot_payload("bot_a", "BotA"))
append_event(conn, kind="bot_authored", payload=_bot_payload("bot_b", "BotB"))
append_event(conn, kind="bot_authored", payload=_bot_payload("bot_c", "BotC"))
append_event(conn, kind="chat_created", payload=_chat_payload())
append_event(conn, kind="guest_added", payload={
"chat_id": "chat_bot_a",
"guest_bot_id": "bot_b",
})
append_event(conn, kind="guest_added", payload={
"chat_id": "chat_bot_a",
"guest_bot_id": "bot_c",
})
project(conn)
chat = get_chat(conn, "chat_bot_a")
assert chat is not None
assert chat["guest_bot_id"] == "bot_c"
+89
View File
@@ -0,0 +1,89 @@
"""Tests for the interjection classifier service (T39).
Per Requirements §6.2, when a guest is present and the addressee bot has
just spoken, the *non-addressee* bot may interject with a brief follow-on
beat. The classifier wrapped here decides whether that interjection
should fire. The default bias is strongly toward False the addressee
has the floor so an interjection only fires when the silent witness's
character would plausibly speak up.
These tests cover:
* The classifier returning ``should_interject=True`` is honored.
* The classifier returning ``should_interject=False`` is honored.
* Repeated invalid JSON exhausts the classifier retries and falls back
to ``should_interject=False`` with ``reason="fallback"``.
"""
from __future__ import annotations
import json
import pytest
from chat.llm.mock import MockLLMClient
from chat.services.interjection import (
InterjectionDecision,
detect_interjection,
)
def _kwargs() -> dict:
"""Reasonable, non-empty kwargs for ``detect_interjection``."""
return dict(
classifier_model="x",
addressee_name="Alice",
addressee_just_said="I think we should leave now.",
silent_witness_name="Bob",
silent_witness_persona="Skeptical engineer, blunt, protective of the user.",
silent_witness_edge_to_addressee={
"affinity": 40,
"trust": 30,
"summary": "old rival; mild distrust",
},
silent_witness_edge_to_you={
"affinity": 70,
"trust": 80,
"summary": "long-time confidant",
},
you_just_said="Where do you both think we should go?",
)
@pytest.mark.asyncio
async def test_interjection_returns_true_when_classifier_decides_yes():
canned = json.dumps({"should_interject": True, "reason": "jealousy"})
mock = MockLLMClient(canned=[canned])
result = await detect_interjection(mock, **_kwargs())
assert isinstance(result, InterjectionDecision)
assert result.should_interject is True
assert result.reason == "jealousy"
@pytest.mark.asyncio
async def test_interjection_returns_false_when_classifier_decides_no():
canned = json.dumps(
{"should_interject": False, "reason": "addressee has the floor"}
)
mock = MockLLMClient(canned=[canned])
result = await detect_interjection(mock, **_kwargs())
assert isinstance(result, InterjectionDecision)
assert result.should_interject is False
assert result.reason == "addressee has the floor"
@pytest.mark.asyncio
async def test_interjection_falls_back_to_false_on_classifier_failure():
"""``classify`` retries 3 times; after all fail it returns the default.
The default carries ``should_interject=False`` and
``reason="fallback"`` so callers can tell a real "no" from a
classifier-degraded "no" if they ever care to.
"""
mock = MockLLMClient(
canned=["this is not json", "still not json", "still not json"]
)
result = await detect_interjection(mock, **_kwargs())
assert isinstance(result, InterjectionDecision)
assert result.should_interject is False
assert result.reason == "fallback"
+269
View File
@@ -0,0 +1,269 @@
from __future__ import annotations
from chat.db.connection import open_db
from chat.db.migrate import apply_migrations
from chat.eventlog.log import append_event
from chat.eventlog.projector import project
import chat.state.entities # registers handlers
import chat.state.world # registers handlers
import chat.state.meanwhile # registers handlers
from chat.state.meanwhile import (
get_parent_scene,
list_meanwhile_scenes,
list_pending_meanwhile_digests,
)
from chat.state.world import active_scene
def _bot_payload(bot_id: str, name: str) -> dict:
return {
"id": bot_id,
"name": name,
"persona": "thoughtful, observant",
"voice_samples": [],
"traits": [],
"backstory": "",
"initial_relationship_to_you": "coworker",
"kickoff_prose": "",
}
def _chat_payload(chat_id: str = "chat_bot_a") -> dict:
return {
"id": chat_id,
"host_bot_id": "bot_a",
"guest_bot_id": "bot_b",
"initial_time": "2026-04-26T20:00:00+00:00",
"narrative_anchor": "Day 1 evening",
"weather": "clear",
}
def test_meanwhile_started_creates_scene_with_correct_present_set_kind(tmp_path):
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
append_event(
conn, kind="bot_authored", payload=_bot_payload("bot_a", "BotA")
)
append_event(
conn, kind="bot_authored", payload=_bot_payload("bot_b", "BotB")
)
append_event(conn, kind="chat_created", payload=_chat_payload())
# Parent (you-scene) — uses existing scene_opened handler. Will get
# the default present_set_kind='you_host' from the new column.
append_event(
conn,
kind="scene_opened",
payload={
"chat_id": "chat_bot_a",
"started_at": "2026-04-26T20:00:00+00:00",
"participants": ["you", "bot_a", "bot_b"],
},
)
# Now the meanwhile child scene — bot_a + bot_b only.
append_event(
conn,
kind="meanwhile_scene_started",
payload={
"scene_id": 2,
"chat_id": "chat_bot_a",
"parent_scene_id": 1,
"host_bot_id": "bot_a",
"guest_bot_id": "bot_b",
"started_at": "2026-04-26T20:05:00+00:00",
},
)
project(conn)
meanwhile_scenes = list_meanwhile_scenes(
conn, "chat_bot_a", status="active"
)
assert len(meanwhile_scenes) == 1
m = meanwhile_scenes[0]
assert m["id"] == 2
assert m["chat_id"] == "chat_bot_a"
assert m["status"] == "active"
assert m["present_set_kind"] == "host_guest"
assert m["parent_scene_id"] == 1
assert m["started_at"] == "2026-04-26T20:05:00+00:00"
assert m["closed_at"] is None
# Parent linkage helper.
parent = get_parent_scene(conn, 2)
assert parent is not None
assert parent["id"] == 1
assert parent["present_set_kind"] == "you_host"
def test_meanwhile_closed_marks_scene_closed(tmp_path):
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
append_event(
conn, kind="bot_authored", payload=_bot_payload("bot_a", "BotA")
)
append_event(
conn, kind="bot_authored", payload=_bot_payload("bot_b", "BotB")
)
append_event(conn, kind="chat_created", payload=_chat_payload())
append_event(
conn,
kind="scene_opened",
payload={
"chat_id": "chat_bot_a",
"started_at": "2026-04-26T20:00:00+00:00",
"participants": ["you", "bot_a", "bot_b"],
},
)
append_event(
conn,
kind="meanwhile_scene_started",
payload={
"scene_id": 2,
"chat_id": "chat_bot_a",
"parent_scene_id": 1,
"host_bot_id": "bot_a",
"guest_bot_id": "bot_b",
"started_at": "2026-04-26T20:05:00+00:00",
},
)
append_event(
conn,
kind="meanwhile_scene_closed",
payload={
"scene_id": 2,
"closed_at": "2026-04-26T20:15:00+00:00",
},
)
project(conn)
assert list_meanwhile_scenes(conn, "chat_bot_a", status="active") == []
closed = list_meanwhile_scenes(conn, "chat_bot_a", status="closed")
assert len(closed) == 1
assert closed[0]["id"] == 2
assert closed[0]["status"] == "closed"
assert closed[0]["closed_at"] == "2026-04-26T20:15:00+00:00"
def test_active_you_scene_can_coexist_with_active_meanwhile_scene(tmp_path):
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
append_event(
conn, kind="bot_authored", payload=_bot_payload("bot_a", "BotA")
)
append_event(
conn, kind="bot_authored", payload=_bot_payload("bot_b", "BotB")
)
append_event(conn, kind="chat_created", payload=_chat_payload())
append_event(
conn,
kind="scene_opened",
payload={
"chat_id": "chat_bot_a",
"started_at": "2026-04-26T20:00:00+00:00",
"participants": ["you", "bot_a", "bot_b"],
},
)
append_event(
conn,
kind="meanwhile_scene_started",
payload={
"scene_id": 2,
"chat_id": "chat_bot_a",
"parent_scene_id": 1,
"host_bot_id": "bot_a",
"guest_bot_id": "bot_b",
"started_at": "2026-04-26T20:05:00+00:00",
},
)
project(conn)
# The you-scene is still the active scene from the chat's POV.
you_scene = active_scene(conn, "chat_bot_a")
assert you_scene is not None
assert you_scene["id"] == 1
# And the meanwhile child is independently active.
meanwhile_scenes = list_meanwhile_scenes(
conn, "chat_bot_a", status="active"
)
assert len(meanwhile_scenes) == 1
assert meanwhile_scenes[0]["id"] == 2
assert meanwhile_scenes[0]["present_set_kind"] == "host_guest"
# Cross-check via raw query: one row per present_set_kind, both unended.
rows = conn.execute(
"SELECT id, present_set_kind FROM scenes "
"WHERE chat_id = ? AND ended_at IS NULL ORDER BY id",
("chat_bot_a",),
).fetchall()
kinds = sorted(r[1] for r in rows)
assert kinds == ["host_guest", "you_host"]
def test_meanwhile_digest_created_and_consumed(tmp_path):
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
append_event(
conn, kind="bot_authored", payload=_bot_payload("bot_a", "BotA")
)
append_event(
conn, kind="bot_authored", payload=_bot_payload("bot_b", "BotB")
)
append_event(conn, kind="chat_created", payload=_chat_payload())
append_event(
conn,
kind="scene_opened",
payload={
"chat_id": "chat_bot_a",
"started_at": "2026-04-26T20:00:00+00:00",
"participants": ["you", "bot_a", "bot_b"],
},
)
append_event(
conn,
kind="meanwhile_scene_started",
payload={
"scene_id": 2,
"chat_id": "chat_bot_a",
"parent_scene_id": 1,
"host_bot_id": "bot_a",
"guest_bot_id": "bot_b",
"started_at": "2026-04-26T20:05:00+00:00",
},
)
append_event(
conn,
kind="meanwhile_digest_created",
payload={
"scene_id": 2,
"chat_id": "chat_bot_a",
"summary": "BotA confides in BotB about the missing file.",
},
)
project(conn)
pending = list_pending_meanwhile_digests(conn, "chat_bot_a")
assert len(pending) == 1
digest_id = pending[0]["id"]
assert pending[0]["scene_id"] == 2
assert pending[0]["summary"].startswith("BotA confides")
# Use append_and_apply for the second beat: re-running project()
# would re-fire non-idempotent handlers (chat_created, scene_opened)
# whose INSERTs conflict on UNIQUE constraints.
from chat.eventlog.log import append_and_apply
append_and_apply(
conn,
kind="meanwhile_digest_consumed",
payload={
"digest_id": digest_id,
"consumed_at": "2026-04-26T20:30:00+00:00",
},
)
assert list_pending_meanwhile_digests(conn, "chat_bot_a") == []
+572
View File
@@ -0,0 +1,572 @@
"""Meanwhile-mode turn flow (T64).
A meanwhile scene runs entirely between two bots host + guest with
"you" absent. The user manually advances the scene by POSTing prose to
the existing ``/chats/<id>/turns`` endpoint; the route detects the active
meanwhile scene at the start of ``post_turn`` and dispatches to the
``process_meanwhile_turn`` controller in ``chat/web/meanwhile.py``.
Coverage:
1. Memory writes for a meanwhile turn carry witness ``[you=0, host=1,
guest=1]`` for both the host's and the guest's per-POV memory rows.
2. State updates after a meanwhile turn run for exactly 2 directed pairs
(host -> guest, guest -> host) no you-related pairs fire.
3. Speakers alternate across consecutive meanwhile turns: the host
speaks first (no prior meanwhile assistant_turn), the guest speaks
second (the prior turn's speaker was the host, so this turn's
speaker is the OTHER bot).
4. Scene-close on a meanwhile scene writes per-POV summaries for host +
guest only no "you" POV row is written, mirroring the no-you
present_set of the meanwhile scene.
"""
from __future__ import annotations
import json
from pathlib import Path
import pytest
from fastapi.testclient import TestClient
from chat.app import app
from chat.db.connection import open_db
from chat.eventlog.log import append_event
from chat.eventlog.projector import project
from chat.llm.mock import MockLLMClient
import chat.state.meanwhile # noqa: F401 (registers handlers)
def _bot_payload(bot_id: str, name: str) -> dict:
return {
"id": bot_id,
"name": name,
"persona": f"persona for {name}",
"voice_samples": [],
"traits": [],
"backstory": "",
"initial_relationship_to_you": "",
"kickoff_prose": "...",
}
def _seed_meanwhile_chat(db_path: Path) -> None:
"""Seed two bots, you, a chat with both wired in, an open parent
you-scene, AND an active meanwhile child scene with bot_a + bot_b.
Edges are seeded for both directed pairs between bot_a and bot_b at
schema-default 50/50 so post-turn state-update writes land cleanly.
Activities for both bots are recorded so the prompt assembler has
something to render.
"""
with open_db(db_path) as conn:
append_event(conn, kind="bot_authored", payload=_bot_payload("bot_a", "BotA"))
append_event(conn, kind="bot_authored", payload=_bot_payload("bot_b", "BotB"))
append_event(
conn,
kind="you_authored",
payload={"name": "Me", "pronouns": "they/them", "persona": ""},
)
append_event(
conn,
kind="chat_created",
payload={
"id": "chat_bot_a",
"host_bot_id": "bot_a",
"guest_bot_id": "bot_b",
"initial_time": "2026-04-26T20:00:00+00:00",
"narrative_anchor": "Day 1",
"weather": "",
},
)
append_event(
conn,
kind="container_created",
payload={
"chat_id": "chat_bot_a",
"name": "office",
"type": "workplace",
"properties": {},
},
)
# Parent (you-scene) opens first.
append_event(
conn,
kind="scene_opened",
payload={
"chat_id": "chat_bot_a",
"container_id": 1,
"started_at": "2026-04-26T20:00:00+00:00",
"participants": ["you", "bot_a", "bot_b"],
},
)
# Meanwhile child scene — bot_a + bot_b only, parent linked.
append_event(
conn,
kind="meanwhile_scene_started",
payload={
"scene_id": 2,
"chat_id": "chat_bot_a",
"parent_scene_id": 1,
"host_bot_id": "bot_a",
"guest_bot_id": "bot_b",
"started_at": "2026-04-26T20:05:00+00:00",
},
)
# Seed both directed edges between the bots so state-update
# writes land on initialized rows.
for src, tgt in [("bot_a", "bot_b"), ("bot_b", "bot_a")]:
append_event(
conn,
kind="edge_update",
payload={
"source_id": src,
"target_id": tgt,
"chat_id": "chat_bot_a",
"knowledge_facts": [],
},
)
for entity_id, verb in [("bot_a", "listening"), ("bot_b", "talking")]:
append_event(
conn,
kind="activity_change",
payload={
"entity_id": entity_id,
"posture": "sitting",
"action": {
"verb": verb,
"interruptible": True,
"required_attention": "low",
"expected_duration": "ongoing",
},
"attention": "",
"holding": [],
"status": {},
},
)
project(conn)
def _override_llm(canned: list[str]) -> MockLLMClient:
from chat.web.kickoff import get_llm_client
mock = MockLLMClient(canned=list(canned))
app.dependency_overrides[get_llm_client] = lambda: mock
return mock
def _zero_state() -> str:
return json.dumps(
{"affinity_delta": 0, "trust_delta": 0, "knowledge_facts": []}
)
@pytest.fixture
def app_state_setup(tmp_path, monkeypatch):
cfg = tmp_path / "config.toml"
cfg.write_text('featherless_api_key = "test"\n')
monkeypatch.setenv("CHAT_CONFIG_PATH", str(cfg))
db = tmp_path / "test.db"
monkeypatch.setenv("CHAT_DB_PATH", str(db))
with TestClient(app) as c:
app.state.background_worker.enabled = False
yield c
app.dependency_overrides.clear()
def test_meanwhile_turn_writes_memories_with_witness_0_1_1(
app_state_setup, tmp_path
):
"""A meanwhile turn writes one ``memory_written`` event per bot — host
and guest with witness flags ``[you=0, host=1, guest=1]``. "You" is
not present in the scene, so the witness_you flag must be 0 for both
rows.
Canned queue (4 calls):
1. parse_turn (user prose classification)
2. narrative stream (host speaks first; no prior meanwhile turn)
3. state-update for bot_a -> bot_b
4. state-update for bot_b -> bot_a
"""
_seed_meanwhile_chat(tmp_path / "test.db")
canned_parse = json.dumps(
{"segments": [{"kind": "narration", "text": "they exchange a glance"}]}
)
canned = [
canned_parse,
"BotA leans in. *quietly* Tell me what you saw.",
_zero_state(),
_zero_state(),
]
mock = _override_llm(canned)
try:
response = app_state_setup.post(
"/chats/chat_bot_a/turns",
data={"prose": "they exchange a glance"},
)
assert response.status_code == 204
finally:
app.dependency_overrides.clear()
assert mock._canned == []
with open_db(tmp_path / "test.db") as conn:
rows = conn.execute(
"SELECT payload_json FROM event_log WHERE kind = 'memory_written' "
"ORDER BY id"
).fetchall()
payloads = [json.loads(r[0]) for r in rows]
assert len(payloads) == 2
owners = sorted(p["owner_id"] for p in payloads)
assert owners == ["bot_a", "bot_b"]
for p in payloads:
assert p["witness_you"] == 0, p
assert p["witness_host"] == 1, p
assert p["witness_guest"] == 1, p
def test_meanwhile_turn_emits_2_edge_updates_only(app_state_setup, tmp_path):
"""A meanwhile turn runs state-update for exactly 2 directed pairs:
host -> guest and guest -> host. No you-related pairs fire.
"""
_seed_meanwhile_chat(tmp_path / "test.db")
canned_parse = json.dumps(
{"segments": [{"kind": "narration", "text": "they whisper"}]}
)
canned = [
canned_parse,
"BotA whispers. *softly* I noticed something today.",
_zero_state(),
_zero_state(),
]
mock = _override_llm(canned)
try:
response = app_state_setup.post(
"/chats/chat_bot_a/turns", data={"prose": "they whisper"}
)
assert response.status_code == 204
finally:
app.dependency_overrides.clear()
assert mock._canned == []
with open_db(tmp_path / "test.db") as conn:
# Edge updates landed AFTER the assistant_turn (i.e. excluding
# the seed updates done before the request).
max_at = conn.execute(
"SELECT MAX(id) FROM event_log WHERE kind = 'assistant_turn'"
).fetchone()[0]
rows = conn.execute(
"SELECT payload_json FROM event_log "
"WHERE kind = 'edge_update' AND id > ? ORDER BY id",
(max_at,),
).fetchall()
payloads = [json.loads(r[0]) for r in rows]
# Exactly 2 post-turn edge_update events.
assert len(payloads) == 2
pairs = sorted((p["source_id"], p["target_id"]) for p in payloads)
assert pairs == [("bot_a", "bot_b"), ("bot_b", "bot_a")]
# And NO you-related pair leaked in.
for p in payloads:
assert p["source_id"] != "you", p
assert p["target_id"] != "you", p
def test_meanwhile_turn_alternates_speaker(app_state_setup, tmp_path):
"""Successive meanwhile turns alternate which bot speaks.
The first turn has no prior meanwhile ``assistant_turn`` linked to
this scene, so the host speaks. The second turn finds the latest
such ``assistant_turn``'s speaker (the host) and picks the OTHER
bot, so the guest speaks. Each ``assistant_turn`` payload carries
``meanwhile_scene_id`` so the alternation lookup is unambiguous.
"""
_seed_meanwhile_chat(tmp_path / "test.db")
canned_parse_1 = json.dumps(
{"segments": [{"kind": "narration", "text": "they pause"}]}
)
canned_1 = [
canned_parse_1,
"BotA speaks first. *quietly*",
_zero_state(),
_zero_state(),
]
mock = _override_llm(canned_1)
try:
response = app_state_setup.post(
"/chats/chat_bot_a/turns", data={"prose": "they pause"}
)
assert response.status_code == 204
finally:
app.dependency_overrides.clear()
assert mock._canned == []
canned_parse_2 = json.dumps(
{"segments": [{"kind": "narration", "text": "and again"}]}
)
canned_2 = [
canned_parse_2,
"BotB replies. *thoughtfully*",
_zero_state(),
_zero_state(),
]
mock = _override_llm(canned_2)
try:
response = app_state_setup.post(
"/chats/chat_bot_a/turns", data={"prose": "and again"}
)
assert response.status_code == 204
finally:
app.dependency_overrides.clear()
assert mock._canned == []
with open_db(tmp_path / "test.db") as conn:
rows = conn.execute(
"SELECT payload_json FROM event_log "
"WHERE kind = 'assistant_turn' ORDER BY id"
).fetchall()
payloads = [json.loads(r[0]) for r in rows]
assert len(payloads) == 2
# First turn — host speaks.
assert payloads[0]["speaker_id"] == "bot_a"
# Second turn — guest speaks (alternation).
assert payloads[1]["speaker_id"] == "bot_b"
# Both payloads tag this meanwhile scene id so the alternation
# lookup can scope to it specifically (not any other assistant_turn
# that might exist on the chat).
assert payloads[0]["meanwhile_scene_id"] == 2
assert payloads[1]["meanwhile_scene_id"] == 2
# Both also carry the present_set_kind discriminator for downstream
# filters (digest creation, drawer rendering).
assert payloads[0]["present_set_kind"] == "host_guest"
assert payloads[1]["present_set_kind"] == "host_guest"
def test_meanwhile_scene_close_writes_per_pov_for_both_bots_only(
app_state_setup, tmp_path
):
"""When a meanwhile scene closes, per-POV summary rewrites land for
the host and the guest. No write fires for "you" there is no
"you" memory store and no "you" POV in the meanwhile present set.
"""
from chat.services.scene_summarize import apply_scene_close_summary
from chat.eventlog.log import append_and_apply
_seed_meanwhile_chat(tmp_path / "test.db")
# Run a meanwhile turn first so each bot has a memory row scoped to
# the meanwhile scene_id (=2). The per-POV rewrite targets these
# rows by ``scene_id``.
canned_parse = json.dumps(
{"segments": [{"kind": "narration", "text": "they speak quietly"}]}
)
canned = [
canned_parse,
"BotA speaks. *quietly*",
_zero_state(),
_zero_state(),
]
mock = _override_llm(canned)
try:
response = app_state_setup.post(
"/chats/chat_bot_a/turns",
data={"prose": "they speak quietly"},
)
assert response.status_code == 204
finally:
app.dependency_overrides.clear()
assert mock._canned == []
# Close the meanwhile scene and run the close-summary pipeline.
# Two POV summaries (host + guest) — no "you" POV.
pov_payload_host = json.dumps(
{
"summary": "BotA reflects on the quiet moment with BotB.",
"knowledge_facts": [],
"relationship_summary": "",
}
)
pov_payload_guest = json.dumps(
{
"summary": "BotB notices BotA's reserved manner.",
"knowledge_facts": [],
"relationship_summary": "",
}
)
# T65 added a meanwhile digest summarize call after per-POV writes
# for meanwhile scenes. T58's thread detection is wrapped in try/except
# so its IndexError is swallowed gracefully.
digest_payload = json.dumps(
{
"summary": "While you were away, BotA and BotB talked quietly.",
"knowledge_facts": [],
"relationship_summary": "",
}
)
close_mock = MockLLMClient(
canned=[pov_payload_host, pov_payload_guest, digest_payload]
)
import asyncio as _asyncio
with open_db(tmp_path / "test.db") as conn:
# asyncio.run() can't nest under TestClient's loop, but the
# close pipeline is awaitable — drive it via a fresh loop here.
_loop = _asyncio.new_event_loop()
# Mark the meanwhile scene closed via the projector handler.
append_and_apply(
conn,
kind="meanwhile_scene_closed",
payload={
"scene_id": 2,
"closed_at": "2026-04-26T20:30:00+00:00",
},
)
# apply_scene_close_summary takes host_bot_id; here we tell it to
# operate on the meanwhile scene id (2). With no "you" memory
# row to rewrite (witness_you=0 means "you" doesn't have a
# memory for this scene), the call must produce per-POV writes
# ONLY for bot_a and bot_b.
try:
_loop.run_until_complete(
apply_scene_close_summary(
conn,
close_mock,
classifier_model="x",
chat_id="chat_bot_a",
scene_id=2,
host_bot_id="bot_a",
)
)
finally:
_loop.close()
# Per-POV memory rewrites: count manual_edits with target_kind
# ``memory_pov_summary`` whose target_id maps to a memory row
# scoped to scene 2.
edits = conn.execute(
"SELECT payload_json FROM event_log WHERE kind = 'manual_edit'"
).fetchall()
pov_edits = []
for (raw,) in edits:
payload = json.loads(raw)
if payload.get("target_kind") != "memory_pov_summary":
continue
mem_row = conn.execute(
"SELECT owner_id, scene_id FROM memories WHERE id = ?",
(payload["target_id"],),
).fetchone()
if mem_row is None or mem_row[1] != 2:
continue
pov_edits.append({"owner": mem_row[0], "new": payload["new_value"]})
# Verify the actual current pov_summary on each bot's memory row
# for scene 2 reflects the rewrite.
host_pov = conn.execute(
"SELECT pov_summary FROM memories WHERE owner_id = ? AND scene_id = ?",
("bot_a", 2),
).fetchone()
guest_pov = conn.execute(
"SELECT pov_summary FROM memories WHERE owner_id = ? AND scene_id = ?",
("bot_b", 2),
).fetchone()
# No "you" memory row should exist for the meanwhile scene —
# "you" was never a witness.
you_row = conn.execute(
"SELECT id FROM memories WHERE owner_id = 'you' AND scene_id = ?",
(2,),
).fetchone()
# Exactly two memory_pov_summary rewrites — one per bot witness.
assert len(pov_edits) == 2
owners = sorted(e["owner"] for e in pov_edits)
assert owners == ["bot_a", "bot_b"]
assert host_pov is not None and "BotA reflects" in host_pov[0]
assert guest_pov is not None and "BotB notices" in guest_pov[0]
# No "you" POV row — meanwhile scenes don't surface a you-memory.
assert you_row is None
def test_meanwhile_turn_registered_in_in_flight_tasks(
app_state_setup, tmp_path
):
"""A meanwhile turn registers its streaming task in the chat-keyed
``_in_flight_tasks`` registry the cancel route reads from, and clears
the entry after the stream completes.
Without registration, ``POST /chats/<id>/turns/cancel`` would be a
silent no-op for meanwhile beats the Stop button wouldn't actually
stop them. We pin the behaviour via a streaming mock that snapshots
``_in_flight_tasks`` at the moment of its first yield (mid-flight),
then assert the entry is removed after the response returns.
"""
from typing import AsyncIterator, Sequence
from chat.llm.client import Message
from chat.web.turns import _in_flight_tasks
_seed_meanwhile_chat(tmp_path / "test.db")
# Snapshot of (chat_id-present?, registered task object) captured
# at the first stream yield. The closure runs inside the streaming
# coroutine, so when it executes the task is alive and registered.
in_flight_snapshot: dict = {}
class _SnapshotMock(MockLLMClient):
async def stream(
self, messages: Sequence[Message], *, model: str, **params
) -> AsyncIterator[str]:
text = self._canned.pop(0)
for i, ch in enumerate(text):
if i == 0:
# Snapshot at first yield — the post_turn coroutine
# is awaiting our generator and the streaming Task
# is registered in _in_flight_tasks[chat_id].
in_flight_snapshot["present"] = (
"chat_bot_a" in _in_flight_tasks
)
in_flight_snapshot["task"] = _in_flight_tasks.get(
"chat_bot_a"
)
yield ch
canned_parse = json.dumps(
{"segments": [{"kind": "narration", "text": "they exchange a glance"}]}
)
mock = _SnapshotMock(
canned=[
canned_parse,
"BotA leans in. *quietly*",
_zero_state(),
_zero_state(),
]
)
from chat.web.kickoff import get_llm_client
app.dependency_overrides[get_llm_client] = lambda: mock
try:
# Pre-condition: registry is empty for this chat.
assert "chat_bot_a" not in _in_flight_tasks
response = app_state_setup.post(
"/chats/chat_bot_a/turns",
data={"prose": "they exchange a glance"},
)
assert response.status_code == 204
finally:
app.dependency_overrides.clear()
# Mid-flight: the streaming task was present in the registry, and
# the captured value was an asyncio.Task (not None / not some other
# placeholder).
import asyncio
assert in_flight_snapshot.get("present") is True, (
"_in_flight_tasks was empty at first yield — meanwhile stream "
"isn't registering its task"
)
assert isinstance(in_flight_snapshot.get("task"), asyncio.Task)
# Post-flight: the entry has been cleaned up so the next turn (or
# the cancel route) doesn't see a stale task.
assert "chat_bot_a" not in _in_flight_tasks
+34
View File
@@ -125,3 +125,37 @@ def test_search_invalid_witness_role_raises(tmp_path):
with open_db(db) as conn: with open_db(db) as conn:
with pytest.raises(ValueError): with pytest.raises(ValueError):
search_memories(conn, "bot_a", "invalid_role", "anything", k=4) search_memories(conn, "bot_a", "invalid_role", "anything", k=4)
def test_higher_significance_outranks_equal_rank(tmp_path):
"""T57: significance multiplier biases the SQL ORDER BY.
Two memories with IDENTICAL FTS-matching text yield (effectively) equal
BM25 ranks. The significance bias applied in the SQL ORDER BY must
surface the higher-significance row first.
"""
db = tmp_path / "t.db"
_seed(
db,
memory_specs=[
# Identical pov_summary text -> FTS BM25 rank is the same for both.
{"pov_summary": "she swore an oath", "significance": 0},
{"pov_summary": "she swore an oath", "significance": 3},
],
)
with open_db(db) as conn:
out = search_memories(conn, "bot_a", "host", "oath", k=5)
assert len(out) == 2
# Higher significance wins despite tied FTS rank.
assert out[0]["significance"] == 3
assert out[1]["significance"] == 0
def test_significance_bias_is_constant_module_level():
"""T57: pin ``SIGNIFICANCE_RANK_BIAS`` as a tunable module-level numeric."""
from chat.state.memory import SIGNIFICANCE_RANK_BIAS
assert isinstance(SIGNIFICANCE_RANK_BIAS, (int, float))
# Must be non-negative -- a negative bias would invert the desired
# "higher significance ranks higher" semantics.
assert SIGNIFICANCE_RANK_BIAS >= 0
+150 -1
View File
@@ -22,7 +22,7 @@ from chat.db.migrate import apply_migrations
from chat.eventlog.log import append_event from chat.eventlog.log import append_event
from chat.eventlog.projector import project from chat.eventlog.projector import project
from chat.llm.mock import MockLLMClient from chat.llm.mock import MockLLMClient
from chat.services.memory_write import record_turn_memory from chat.services.memory_write import record_turn_memory, record_turn_memory_for_present
import chat.state.entities # noqa: F401 - register handlers import chat.state.entities # noqa: F401 - register handlers
import chat.state.memory # noqa: F401 import chat.state.memory # noqa: F401
import chat.state.world # noqa: F401 import chat.state.world # noqa: F401
@@ -295,3 +295,152 @@ def test_post_turn_writes_memory_for_host_bot(client, tmp_path):
assert w_guest == 0 assert w_guest == 0
assert source == "direct" assert source == "direct"
assert sig == 1 assert sig == 1
# ---------------------------------------------------------------------------
# T41: record_turn_memory_for_present — multi-witness helper.
# ---------------------------------------------------------------------------
def _seed_two_bots(db_path: Path) -> None:
"""Author host + guest bots and create a two-bot chat."""
with open_db(db_path) as conn:
for bot_id, name in (("bot_a", "BotA"), ("bot_b", "BotB")):
append_event(
conn,
kind="bot_authored",
payload={
"id": bot_id,
"name": name,
"persona": "...",
"voice_samples": [],
"traits": [],
"backstory": "",
"initial_relationship_to_you": "",
"kickoff_prose": "",
},
)
append_event(
conn,
kind="chat_created",
payload={
"id": "chat_ab",
"host_bot_id": "bot_a",
"guest_bot_id": "bot_b",
"initial_time": "2026-04-26T20:00:00+00:00",
"narrative_anchor": "Day 1",
"weather": "",
},
)
project(conn)
def test_record_for_present_no_guest_writes_single_memory_with_witness_1_1_0(tmp_path):
db = tmp_path / "t.db"
apply_migrations(db)
_seed_minimal(db)
with open_db(db) as conn:
result = record_turn_memory_for_present(
conn,
chat_id="chat_bot_a",
host_bot_id="bot_a",
guest_bot_id=None,
narrative_text="BotA glances out the window.",
scene_id=None,
chat_clock_at="2026-04-26T20:00:00+00:00",
)
# Returned dict has only the host key.
assert set(result.keys()) == {"bot_a"}
eid_h, mid_h = result["bot_a"]
assert eid_h > 0
assert mid_h is not None and mid_h > 0
rows = conn.execute(
"SELECT owner_id, witness_you, witness_host, witness_guest "
"FROM memories"
).fetchall()
assert len(rows) == 1
owner_id, w_you, w_host, w_guest = rows[0]
assert owner_id == "bot_a"
assert w_you == 1
assert w_host == 1
assert w_guest == 0
# Exactly one memory_written event was appended.
cur = conn.execute(
"SELECT COUNT(*) FROM event_log WHERE kind = 'memory_written'"
)
assert cur.fetchone()[0] == 1
def test_record_for_present_with_guest_writes_two_memories_with_witness_1_1_1(tmp_path):
db = tmp_path / "t.db"
apply_migrations(db)
_seed_two_bots(db)
with open_db(db) as conn:
result = record_turn_memory_for_present(
conn,
chat_id="chat_ab",
host_bot_id="bot_a",
guest_bot_id="bot_b",
narrative_text="BotA and BotB share a glance.",
scene_id=None,
chat_clock_at="2026-04-26T20:00:00+00:00",
)
# Returned dict has both keys.
assert set(result.keys()) == {"bot_a", "bot_b"}
eid_h, mid_h = result["bot_a"]
eid_g, mid_g = result["bot_b"]
assert eid_h > 0 and eid_g > 0
assert mid_h is not None and mid_h > 0
assert mid_g is not None and mid_g > 0
# Distinct event ids and memory ids.
assert eid_h != eid_g
assert mid_h != mid_g
rows = conn.execute(
"SELECT owner_id, witness_you, witness_host, witness_guest "
"FROM memories ORDER BY owner_id"
).fetchall()
assert len(rows) == 2
owners = {r[0] for r in rows}
assert owners == {"bot_a", "bot_b"}
# All rows should have witness mask [1, 1, 1].
for _owner, w_you, w_host, w_guest in rows:
assert w_you == 1
assert w_host == 1
assert w_guest == 1
# Two memory_written events were appended.
cur = conn.execute(
"SELECT COUNT(*) FROM event_log WHERE kind = 'memory_written'"
)
assert cur.fetchone()[0] == 2
def test_record_for_present_dict_keys_match(tmp_path):
db = tmp_path / "t.db"
apply_migrations(db)
_seed_two_bots(db)
with open_db(db) as conn:
# No guest: keys == {host_bot_id}.
result_no_guest = record_turn_memory_for_present(
conn,
chat_id="chat_ab",
host_bot_id="bot_a",
guest_bot_id=None,
narrative_text="Just BotA's POV.",
)
assert set(result_no_guest.keys()) == {"bot_a"}
# With guest: keys == {host_bot_id, guest_bot_id}.
result_with_guest = record_turn_memory_for_present(
conn,
chat_id="chat_ab",
host_bot_id="bot_a",
guest_bot_id="bot_b",
narrative_text="Both bots witness this.",
)
assert set(result_with_guest.keys()) == {"bot_a", "bot_b"}
+147
View File
@@ -0,0 +1,147 @@
"""Multi-entity state-update coordinator (T40).
Wraps the single-pair :func:`compute_state_update` to run state updates
for ALL directed pairs of present entities. With 3 present entities
(you, host, guest) that's 6 directed pairs; with 2 (you, host) it's 2.
Calls run sequentially to respect Featherless's 2-connection cap.
"""
from __future__ import annotations
import json
import pytest
from chat.llm.mock import MockLLMClient
from chat.services.multi_state_update import compute_state_updates_for_present
from chat.services.state_update import StateUpdate
def _canned_update(affinity: int, trust: int, facts: list[str] | None = None) -> str:
return json.dumps(
{
"affinity_delta": affinity,
"trust_delta": trust,
"knowledge_facts": facts or [],
}
)
@pytest.mark.asyncio
async def test_two_entities_returns_two_updates():
"""you + bot_a -> 2 directed pairs (you->bot_a, bot_a->you)."""
canned = [
_canned_update(2, 1, ["likes coffee"]), # you -> bot_a
_canned_update(1, 0, ["greets warmly"]), # bot_a -> you
]
mock = MockLLMClient(canned=canned)
results = await compute_state_updates_for_present(
mock,
classifier_model="x",
present_ids=["you", "bot_a"],
present_names={"you": "Me", "bot_a": "BotA"},
personas={"you": "", "bot_a": "thoughtful"},
prior_edges={
("you", "bot_a"): {"affinity": 50, "trust": 50, "summary": ""},
("bot_a", "you"): {"affinity": 50, "trust": 50, "summary": ""},
},
recent_dialogue=[
{"speaker": "you", "text": "hi"},
{"speaker": "BotA", "text": "Hello!"},
],
)
assert len(results) == 2
assert results[0][0] == "you"
assert results[0][1] == "bot_a"
assert isinstance(results[0][2], StateUpdate)
assert results[0][2].affinity_delta == 2
assert results[0][2].trust_delta == 1
assert results[0][2].knowledge_facts == ["likes coffee"]
assert results[1][0] == "bot_a"
assert results[1][1] == "you"
assert isinstance(results[1][2], StateUpdate)
assert results[1][2].affinity_delta == 1
assert results[1][2].trust_delta == 0
assert results[1][2].knowledge_facts == ["greets warmly"]
@pytest.mark.asyncio
async def test_three_entities_returns_six_updates():
"""you + bot_a + bot_b -> 6 directed pairs (no self-pairs)."""
canned = [_canned_update(i, 0) for i in range(6)]
mock = MockLLMClient(canned=canned)
results = await compute_state_updates_for_present(
mock,
classifier_model="x",
present_ids=["you", "bot_a", "bot_b"],
present_names={"you": "Me", "bot_a": "BotA", "bot_b": "BotB"},
personas={"you": "", "bot_a": "thoughtful", "bot_b": "cheerful"},
prior_edges={}, # all default to 50/50/""
recent_dialogue=[{"speaker": "you", "text": "hello all"}],
)
assert len(results) == 6
pairs = [(src, tgt) for src, tgt, _ in results]
# No self-pairs.
assert all(src != tgt for src, tgt in pairs)
# All 6 directed combinations present.
expected = {
("you", "bot_a"),
("you", "bot_b"),
("bot_a", "you"),
("bot_a", "bot_b"),
("bot_b", "you"),
("bot_b", "bot_a"),
}
assert set(pairs) == expected
# Every entry is a StateUpdate.
assert all(isinstance(u, StateUpdate) for _, _, u in results)
@pytest.mark.asyncio
async def test_failure_in_one_pair_does_not_kill_batch():
"""First pair fails all 3 classify retries -> default; second parses OK."""
canned = [
# Pair 1 (you -> bot_a): 3 malformed responses -> default StateUpdate.
"bad",
"still bad",
"nope",
# Pair 2 (bot_a -> you): valid JSON.
_canned_update(3, 2, ["was warm"]),
]
mock = MockLLMClient(canned=canned)
results = await compute_state_updates_for_present(
mock,
classifier_model="x",
present_ids=["you", "bot_a"],
present_names={"you": "Me", "bot_a": "BotA"},
personas={"you": "", "bot_a": "thoughtful"},
prior_edges={
("you", "bot_a"): {"affinity": 60, "trust": 40, "summary": "some prior"},
("bot_a", "you"): {"affinity": 50, "trust": 50, "summary": ""},
},
recent_dialogue=[{"speaker": "you", "text": "hi"}],
)
assert len(results) == 2
# First pair: default (zero-delta) StateUpdate.
src1, tgt1, update1 = results[0]
assert (src1, tgt1) == ("you", "bot_a")
assert update1.affinity_delta == 0
assert update1.trust_delta == 0
assert update1.knowledge_facts == []
# Second pair: parsed valid JSON.
src2, tgt2, update2 = results[1]
assert (src2, tgt2) == ("bot_a", "you")
assert update2.affinity_delta == 3
assert update2.trust_delta == 2
assert update2.knowledge_facts == ["was warm"]
+57
View File
@@ -0,0 +1,57 @@
from __future__ import annotations
import sqlite3
import threading
from chat.db.connection import open_db
def test_open_db_default_uses_check_same_thread_true(tmp_path):
"""Default open_db must reject cross-thread use (safe default)."""
db = tmp_path / "t.db"
captured: list[BaseException | None] = []
with open_db(db) as conn:
conn.execute("CREATE TABLE t (x INTEGER)")
def worker():
try:
conn.execute("SELECT 1").fetchall()
captured.append(None)
except BaseException as e: # noqa: BLE001
captured.append(e)
t = threading.Thread(target=worker)
t.start()
t.join()
assert len(captured) == 1
err = captured[0]
assert isinstance(err, sqlite3.ProgrammingError), (
f"expected sqlite3.ProgrammingError on cross-thread use, got {err!r}"
)
def test_open_db_can_disable_check_same_thread(tmp_path):
"""open_db(check_same_thread=False) must allow cross-thread use."""
db = tmp_path / "t.db"
captured: list[BaseException | None] = []
rows: list[object] = []
with open_db(db, check_same_thread=False) as conn:
conn.execute("CREATE TABLE t (x INTEGER)")
conn.execute("INSERT INTO t (x) VALUES (42)")
def worker():
try:
result = conn.execute("SELECT x FROM t").fetchall()
rows.extend(result)
captured.append(None)
except BaseException as e: # noqa: BLE001
captured.append(e)
t = threading.Thread(target=worker)
t.start()
t.join()
assert captured == [None], f"worker thread raised: {captured}"
assert rows == [(42,)]
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+608 -2
View File
@@ -12,14 +12,16 @@ import pytest
from chat.db.connection import open_db from chat.db.connection import open_db
from chat.db.migrate import apply_migrations from chat.db.migrate import apply_migrations
from chat.eventlog.log import append_event from chat.eventlog.log import append_and_apply, append_event
from chat.eventlog.projector import project from chat.eventlog.projector import project
import chat.state.entities # noqa: F401 (registers handlers) import chat.state.entities # noqa: F401 (registers handlers)
import chat.state.edges # noqa: F401 import chat.state.edges # noqa: F401
import chat.state.memory # noqa: F401 import chat.state.memory # noqa: F401
import chat.state.world # noqa: F401 import chat.state.world # noqa: F401
import chat.state.events # noqa: F401
import chat.state.threads # noqa: F401
from chat.llm.client import Message from chat.llm.client import Message
from chat.services.prompt import assemble_narrative_prompt from chat.services.prompt import _witness_role_for, assemble_narrative_prompt
def _seed_basic(conn) -> None: def _seed_basic(conn) -> None:
@@ -253,3 +255,607 @@ def test_must_exceeds_budget_hard_raises_value_error(tmp_path):
budget_soft=5, budget_soft=5,
budget_hard=10, budget_hard=10,
) )
# ---------------------------------------------------------------------------
# Task 43: multi-entity prompt assembly (guest_id support)
# ---------------------------------------------------------------------------
def _seed_with_guest(conn) -> None:
"""Seed a 3-entity scene: you (Sam) + host (Aria, bot_a) + guest (Iris, bot_b).
Group node row is initialized with summary + dynamic, edges in all
relevant directions are seeded, and activities are recorded for all
three entities.
"""
append_event(conn, kind="bot_authored", payload={
"id": "bot_a",
"name": "Aria",
"persona": "reserved coworker who notices things",
"voice_samples": ["I — sorry, I didn't mean to.", "Right. Of course."],
"traits": ["introverted", "observant"],
"backstory": "An archivist who joined the firm last spring.",
"initial_relationship_to_you": "coworker; mild crush; never voiced",
"kickoff_prose": "you stay late at the office",
})
append_event(conn, kind="bot_authored", payload={
"id": "bot_b",
"name": "Iris",
"persona": "wry transplant from the Boston office",
"voice_samples": ["Oh, please.", "Don't make me say it twice."],
"traits": ["sardonic", "loyal"],
"backstory": "Met Aria at a conference two years back.",
"initial_relationship_to_you": "stranger; curious",
"kickoff_prose": "",
})
append_event(conn, kind="you_authored", payload={
"name": "Sam",
"pronouns": "they/them",
"persona": "tired analyst",
})
append_event(conn, kind="chat_created", payload={
"id": "chat_bot_a",
"host_bot_id": "bot_a",
"guest_bot_id": "bot_b",
"initial_time": "2026-04-26T20:00:00+00:00",
"narrative_anchor": "Day 1 evening",
"weather": "clear",
})
append_event(conn, kind="container_created", payload={
"chat_id": "chat_bot_a",
"name": "office bullpen",
"type": "workplace",
"properties": {"public": False, "moving": False, "audible_range": "room"},
})
# Edges: host -> you, guest -> you, host -> guest, guest -> host.
append_event(conn, kind="edge_update", payload={
"source_id": "bot_a",
"target_id": "you",
"affinity_delta": 12,
"trust_delta": 5,
"knowledge_facts": ["they work on the same floor"],
})
append_event(conn, kind="edge_update", payload={
"source_id": "bot_a",
"target_id": "bot_b",
"affinity_delta": 20,
"trust_delta": 15,
"knowledge_facts": ["studied physics together"],
})
append_event(conn, kind="edge_update", payload={
"source_id": "bot_b",
"target_id": "you",
"affinity_delta": 4,
"trust_delta": 0,
"knowledge_facts": ["Aria's coworker"],
})
append_event(conn, kind="edge_update", payload={
"source_id": "bot_b",
"target_id": "bot_a",
"affinity_delta": 18,
"trust_delta": 12,
"knowledge_facts": ["former roommate"],
})
# Activity for all three entities — note distinct verbs so we can
# check whose activity got dropped under tight budget.
append_event(conn, kind="activity_change", payload={
"entity_id": "you",
"container_id": 1,
"posture": "sitting at your desk",
"action": {"verb": "finishing emails"},
"attention": "the screen",
"holding": ["coffee mug"],
})
append_event(conn, kind="activity_change", payload={
"entity_id": "bot_a",
"container_id": 1,
"posture": "sitting at her desk",
"action": {"verb": "pretending to work"},
"attention": "you, in glances",
})
append_event(conn, kind="activity_change", payload={
"entity_id": "bot_b",
"container_id": 1,
"posture": "leaning against the doorframe",
"action": {"verb": "smirking-distinctively"},
"attention": "Aria",
})
append_event(conn, kind="scene_opened", payload={
"chat_id": "chat_bot_a",
"container_id": 1,
"started_at": "2026-04-26T20:00:00+00:00",
"participants": ["you", "bot_a", "bot_b"],
})
append_event(conn, kind="group_node_initialized", payload={
"chat_id": "chat_bot_a",
"members": ["you", "bot_a", "bot_b"],
"summary": "Three coworkers catching up after hours UNIQUE-GROUP-SUMMARY.",
"dynamic": "warm-but-prickly UNIQUE-GROUP-DYNAMIC",
})
project(conn)
def test_assemble_with_no_guest_matches_phase1(tmp_path):
"""Regression: 2-entity scenario without guest_id behaves exactly as Phase 1."""
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
_seed_basic(conn)
msgs = assemble_narrative_prompt(
conn,
chat_id="chat_bot_a",
speaker_bot_id="bot_a",
recent_dialogue=[],
retrieved_memory_summaries=[],
)
body = msgs[0].content
# Phase 1 must blocks present.
assert "Aria" in body
assert "PERSONA" in body
assert "Sam" in body
assert "ACTIVITIES" in body
assert "62/100" in body # speaker → addressee edge intact
# No guest content leaks in.
assert "Group dynamic" not in body
assert "Iris" not in body
def test_assemble_with_guest_includes_group_node_summary(tmp_path):
"""When guest is present (auto-detected via chat.guest_bot_id) and a
group_node row exists, its summary + dynamic are rendered."""
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
_seed_with_guest(conn)
msgs = assemble_narrative_prompt(
conn,
chat_id="chat_bot_a",
speaker_bot_id="bot_a",
recent_dialogue=[],
retrieved_memory_summaries=[],
)
body = msgs[0].content
assert "Group dynamic" in body
assert "UNIQUE-GROUP-SUMMARY" in body
assert "UNIQUE-GROUP-DYNAMIC" in body
# Guest activity also present (SHOULD-tier, fits at default budget).
assert "smirking-distinctively" in body
# Speaker's other edges include the host -> guest direction.
assert "Iris" in body
def test_assemble_when_speaker_is_guest_orients_edges_correctly(tmp_path):
"""When the guest is the speaker, identity is the guest, the
addressee edge is guest you, and other edges include guest host."""
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
_seed_with_guest(conn)
msgs = assemble_narrative_prompt(
conn,
chat_id="chat_bot_a",
speaker_bot_id="bot_b", # guest as speaker
recent_dialogue=[],
retrieved_memory_summaries=[],
)
body = msgs[0].content
# Speaker identity is the guest's persona.
assert "You are Iris." in body
assert "wry transplant from the Boston office" in body
# Edge to addressee is guest → you (Sam) with the seeded values
# (default 50 + 4 affinity = 54).
assert "YOUR EDGE TO Sam" in body
assert "54/100" in body
# Other edges include guest → host (Aria) with seeded value
# (default 50 + 18 = 68).
assert "OTHER EDGES" in body
assert "Aria" in body
assert "68/100" in body
def test_speaker_is_guest_uses_guest_witness_role(tmp_path, monkeypatch):
"""T71.1: when the guest is the speaker, ``search_memories`` is
called with ``witness_role="guest"``, not the previously-hardcoded
``"host"``. Pins the parametric witness role at the prompt call site
so guest-as-speaker honours the witness mask via Phase 2 T46.
"""
db = tmp_path / "t.db"
apply_migrations(db)
captured: dict = {}
def _fake_search(conn, owner_id, witness_role, query, k=4):
captured["owner_id"] = owner_id
captured["witness_role"] = witness_role
captured["query"] = query
return []
# Patch the imported reference inside the prompt module so the
# production call site uses the fake.
import chat.services.prompt as prompt_mod
monkeypatch.setattr(prompt_mod, "search_memories", _fake_search)
with open_db(db) as conn:
_seed_with_guest(conn)
# Guest as speaker — must request memories with witness_role="guest".
assemble_narrative_prompt(
conn,
chat_id="chat_bot_a",
speaker_bot_id="bot_b",
recent_dialogue=[],
# retrieved_memory_summaries=None forces the search path.
retrieved_memory_summaries=None,
)
assert captured["owner_id"] == "bot_b"
assert captured["witness_role"] == "guest"
def test_speaker_is_host_uses_host_witness_role(tmp_path, monkeypatch):
"""T71.1 (regression): host-as-speaker still queries with
``witness_role="host"``."""
db = tmp_path / "t.db"
apply_migrations(db)
captured: dict = {}
def _fake_search(conn, owner_id, witness_role, query, k=4):
captured["witness_role"] = witness_role
return []
import chat.services.prompt as prompt_mod
monkeypatch.setattr(prompt_mod, "search_memories", _fake_search)
with open_db(db) as conn:
_seed_with_guest(conn)
assemble_narrative_prompt(
conn,
chat_id="chat_bot_a",
speaker_bot_id="bot_a", # host as speaker
recent_dialogue=[],
retrieved_memory_summaries=None,
)
assert captured["witness_role"] == "host"
def test_single_activities_block_with_three_bullets_when_3_entities(tmp_path):
"""T71.2: with you + host + guest present, the assembled prompt
contains exactly ONE ``ACTIVITIES:`` header and bullets for all
three entities (no duplicate header from the prior dual-block
rendering).
"""
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
_seed_with_guest(conn)
msgs = assemble_narrative_prompt(
conn,
chat_id="chat_bot_a",
speaker_bot_id="bot_a",
recent_dialogue=[],
retrieved_memory_summaries=[],
)
body = msgs[0].content
# Exactly one ACTIVITIES: header.
assert body.count("ACTIVITIES:") == 1
# Bullets for all three entities (you=Sam, host=Aria, guest=Iris)
# — pin on the unique action verbs from the seed data.
assert "finishing emails" in body # you bullet
assert "pretending to work" in body # speaker (host) bullet
assert "smirking-distinctively" in body # guest bullet
def test_tight_budget_drops_guest_activity_bullet_first(tmp_path):
"""T71.2: under tight budget the speaker bullet survives but the
guest activity bullet is the first ACTIVITIES: bullet to drop. The
block as a whole stays present (it's MUST-tier); only its body
contracts.
"""
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
_seed_with_guest(conn)
dialogue = [
{"speaker": "you", "text": "line-16 hi there"},
{"speaker": "bot_a", "text": "line-17 hey"},
{"speaker": "you", "text": "line-18 quiet night"},
{"speaker": "bot_a", "text": "line-19 indeed"},
]
msgs = assemble_narrative_prompt(
conn,
chat_id="chat_bot_a",
speaker_bot_id="bot_a",
recent_dialogue=dialogue,
retrieved_memory_summaries=[],
budget_soft=250,
budget_hard=340,
)
body = msgs[0].content
# Speaker bullet survives (MUST-tier floor).
assert "pretending to work" in body
assert "ACTIVITIES:" in body
# Guest bullet is dropped first under budget pressure.
assert "smirking-distinctively" not in body
def test_nice_trim_order_documented(tmp_path):
"""T71.3: pin the NICE-tier trim order so a future refactor can't
quietly invert it.
Order under NICE pressure is:
1. previous-scene summary (dropped FIRST)
2. memories beyond top-2
3. older dialogue turns (collapsed to last-4)
We size the budget so that all-NICE-included is over soft, but
dropping ONLY previous-scene gets us back under soft. The observed
behaviour we pin: previous-scene gone, memories/dialogue intact.
"""
db = tmp_path / "t.db"
apply_migrations(db)
# Heavy previous-scene summary — large enough that dropping it
# alone clears the soft-budget overage. Defined out here so the
# marker is in scope for the assertions below.
prev_scene_blob = "PREVSCENE-MARKER " + ("filler " * 200)
with open_db(db) as conn:
# Append all events first, project once at the end (project is
# not idempotent — it replays every event in the log).
from chat.eventlog.log import append_event as _append
_append(conn, kind="bot_authored", payload={
"id": "bot_a",
"name": "Aria",
"persona": "reserved coworker who notices things",
"voice_samples": ["I — sorry, I didn't mean to."],
"traits": ["introverted"],
"backstory": "An archivist who joined the firm last spring.",
"initial_relationship_to_you": "coworker",
"kickoff_prose": "you stay late at the office",
})
_append(conn, kind="you_authored", payload={
"name": "Sam",
"pronouns": "they/them",
"persona": "tired analyst",
})
_append(conn, kind="chat_created", payload={
"id": "chat_bot_a",
"host_bot_id": "bot_a",
"guest_bot_id": None,
"initial_time": "2026-04-26T20:00:00+00:00",
"narrative_anchor": "Day 1 evening",
"weather": "clear",
})
_append(conn, kind="container_created", payload={
"chat_id": "chat_bot_a",
"name": "office bullpen",
"type": "workplace",
"properties": {"public": False, "moving": False, "audible_range": "room"},
})
_append(conn, kind="edge_update", payload={
"source_id": "bot_a",
"target_id": "you",
"affinity_delta": 12,
"trust_delta": 5,
"knowledge_facts": ["they work on the same floor"],
})
_append(conn, kind="activity_change", payload={
"entity_id": "you",
"container_id": 1,
"posture": "sitting at your desk",
"action": {"verb": "finishing emails"},
"attention": "the screen",
})
_append(conn, kind="activity_change", payload={
"entity_id": "bot_a",
"container_id": 1,
"posture": "sitting at her desk",
"action": {"verb": "pretending to work"},
"attention": "you, in glances",
})
_append(conn, kind="scene_opened", payload={
"chat_id": "chat_bot_a",
"container_id": 1,
"started_at": "2026-04-26T20:00:00+00:00",
"participants": ["you", "bot_a"],
})
# Close the seeded scene and write a per-POV summary memory so
# _resolve_previous_scene_summary returns a non-empty string.
_append(conn, kind="scene_closed", payload={
"scene_id": 1,
"ended_at": "2026-04-26T20:30:00+00:00",
"significance": 2,
})
_append(conn, kind="memory_written", payload={
"owner_id": "bot_a",
"chat_id": "chat_bot_a",
"scene_id": 1,
"pov_summary": prev_scene_blob,
"witness_you": 1,
"witness_host": 1,
"witness_guest": 0,
"source": "direct",
"reliability": 1.0,
"significance": 2,
})
project(conn)
# Six dialogue turns — last 4 plus 2 older. If older turns are
# dropped under NICE pressure, the unique markers for turns 0/1
# disappear; we'll assert they REMAIN to prove dialogue trim
# didn't fire.
dialogue = [
{"speaker": "you", "text": "DLG-OLD-00 hello"},
{"speaker": "bot_a", "text": "DLG-OLD-01 hi"},
{"speaker": "you", "text": "DLG-LAST-16 ok"},
{"speaker": "bot_a", "text": "DLG-LAST-17 sure"},
{"speaker": "you", "text": "DLG-LAST-18 night"},
{"speaker": "bot_a", "text": "DLG-LAST-19 indeed"},
]
# Four small memories — if "memories beyond top-2" trim fires,
# MEM-C/MEM-D disappear; we'll assert they REMAIN to prove
# memories trim didn't fire either.
memories = ["MEM-A short", "MEM-B short", "MEM-C short", "MEM-D short"]
# Soft tuned so the all-NICE config (with the heavy previous
# scene summary) overflows, but dropping just previous-scene
# fits comfortably. Hard set high so SHOULD-tier never trims.
msgs = assemble_narrative_prompt(
conn,
chat_id="chat_bot_a",
speaker_bot_id="bot_a",
recent_dialogue=dialogue,
retrieved_memory_summaries=memories,
budget_soft=400,
budget_hard=8000,
)
body = msgs[0].content
# Previous-scene summary was the FIRST NICE drop — its unique
# marker must be absent.
assert "PREVSCENE-MARKER" not in body
# Memories beyond top-2 stayed (proves memories trim did NOT fire).
assert "MEM-A" in body
assert "MEM-B" in body
assert "MEM-C" in body
assert "MEM-D" in body
# Older dialogue turns stayed (proves dialogue trim did NOT fire).
assert "DLG-OLD-00" in body
assert "DLG-OLD-01" in body
# Last-4 dialogue turns of course present.
assert "DLG-LAST-19" in body
def test_assemble_with_tight_budget_drops_guest_activity_first(tmp_path):
"""Under tight budget MUST blocks survive but SHOULD-tier guest
activity is dropped first."""
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
_seed_with_guest(conn)
# Short dialogue so MUST core (speaker identity + edge + last 4
# turns + closing) sits comfortably under the hard budget while
# SHOULD-tier additions (guest activity, group node, other edges)
# would push over.
dialogue = [
{"speaker": "you", "text": "line-16 hi there"},
{"speaker": "bot_a", "text": "line-17 hey"},
{"speaker": "you", "text": "line-18 quiet night"},
{"speaker": "bot_a", "text": "line-19 indeed"},
]
msgs = assemble_narrative_prompt(
conn,
chat_id="chat_bot_a",
speaker_bot_id="bot_a",
recent_dialogue=dialogue,
retrieved_memory_summaries=[],
# MUST core ~310 tokens; SHOULD additions (guest activity +
# group node + other edges) push it well over 380. budget_hard
# is set just above MUST core so SHOULD-tier blocks must be
# trimmed away.
budget_soft=250,
budget_hard=340,
)
body = msgs[0].content
# MUST: speaker identity, edge to addressee, last 4 dialogue turns.
assert "Aria" in body
assert "YOUR EDGE TO Sam" in body
for i in range(16, 20):
assert f"line-{i:02d}" in body
# Guest activity (SHOULD-tier) must be dropped under tight budget.
assert "smirking-distinctively" not in body
# Token budget honoured.
import tiktoken
enc = tiktoken.get_encoding("cl100k_base")
assert len(enc.encode(body)) <= 340
# ---------------------------------------------------------------------------
# Task 60: Active events + open threads in prompt assembly
# ---------------------------------------------------------------------------
def test_assemble_with_no_events_or_threads_omits_blocks(tmp_path):
"""Regression: with the basic 2-entity scenario (no events seeded, no
threads seeded), the assembled prompt must NOT contain the
``Active events:`` or ``Open threads:`` headers both blocks are
omit-when-empty."""
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
_seed_basic(conn)
msgs = assemble_narrative_prompt(
conn,
chat_id="chat_bot_a",
speaker_bot_id="bot_a",
recent_dialogue=[],
retrieved_memory_summaries=[],
)
body = msgs[0].content
assert "Active events:" not in body
assert "Open threads:" not in body
def test_assemble_with_active_events_renders_block(tmp_path):
"""Seed a planned event then transition it to active; the assembled
prompt should render the ``Active events:`` block listing the event
by kind."""
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
_seed_basic(conn)
# event_planned then event_started → status="active". Use
# append_and_apply because _seed_basic already projected; calling
# project() again would replay every prior event (and trip
# UNIQUE constraints on chat_created etc.).
append_and_apply(conn, kind="event_planned", payload={
"event_id": "evt_park",
"chat_id": "chat_bot_a",
"kind": "date_at_park",
"props": {"location": "Riverside Park", "vibe": "casual"},
"planned_for": "2026-04-30T18:00:00+00:00",
})
append_and_apply(conn, kind="event_started", payload={
"event_id": "evt_park",
"started_at": "2026-04-30T18:05:00+00:00",
})
msgs = assemble_narrative_prompt(
conn,
chat_id="chat_bot_a",
speaker_bot_id="bot_a",
recent_dialogue=[],
retrieved_memory_summaries=[],
)
body = msgs[0].content
assert "Active events:" in body
assert "date_at_park" in body
def test_assemble_with_open_thread_renders_block(tmp_path):
"""Seed a single open thread; the assembled prompt should render the
``Open threads:`` block listing the thread by title."""
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
_seed_basic(conn)
# _seed_basic already projected; use append_and_apply for the
# post-seed event so we don't re-trigger UNIQUE constraint
# collisions on the prior chat_created/etc. events.
append_and_apply(conn, kind="thread_opened", payload={
"thread_id": "thr_job",
"chat_id": "chat_bot_a",
"title": "Maya's job hunt",
"summary": "Maya is looking for a new job",
})
msgs = assemble_narrative_prompt(
conn,
chat_id="chat_bot_a",
speaker_bot_id="bot_a",
recent_dialogue=[],
retrieved_memory_summaries=[],
)
body = msgs[0].content
assert "Open threads:" in body
assert "Maya's job hunt" in body
def test_witness_role_for_none_host_returns_host():
assert _witness_role_for("bot_a", None) == "host"
# Sanity check: existing semantics preserved.
assert _witness_role_for("bot_a", "bot_a") == "host"
assert _witness_role_for("bot_a", "bot_b") == "guest"
+391
View File
@@ -271,3 +271,394 @@ def test_regenerate_404_when_assistant_turn_missing(client, tmp_path):
assert response.status_code == 404 assert response.status_code == 404
finally: finally:
app.dependency_overrides.clear() app.dependency_overrides.clear()
def _seed_with_interjection_group(db_path):
"""Seed a multi-entity scene with a (primary + interjection) group.
Returns ``(user_turn_id, primary_at_id, interjection_at_id)``.
The primary speaker is the host (bot_a); the silent witness who
interjected is the guest (bot_b). Mirrors the convention in
chat/web/turns.py both assistant_turns share the same
``user_turn_id`` and the interjection's payload carries
``interjection_of=<primary speaker_id>``.
"""
with open_db(db_path) as conn:
for bot_id, name, persona in (
("bot_a", "BotA", "thoughtful"),
("bot_b", "BotB", "loud"),
):
append_event(
conn,
kind="bot_authored",
payload={
"id": bot_id,
"name": name,
"persona": persona,
"voice_samples": [],
"traits": [],
"backstory": "",
"initial_relationship_to_you": "",
"kickoff_prose": "",
},
)
append_event(
conn,
kind="chat_created",
payload={
"id": "chat_multi",
"host_bot_id": "bot_a",
"guest_bot_id": "bot_b",
"initial_time": "2026-04-26T20:00:00+00:00",
"narrative_anchor": "Day 1",
"weather": "",
},
)
for src, tgt in (
("bot_a", "you"),
("you", "bot_a"),
("bot_b", "you"),
("you", "bot_b"),
("bot_a", "bot_b"),
("bot_b", "bot_a"),
):
append_event(
conn,
kind="edge_update",
payload={
"source_id": src,
"target_id": tgt,
"chat_id": "chat_multi",
},
)
for entity_id in ("you", "bot_a", "bot_b"):
append_event(
conn,
kind="activity_change",
payload={
"entity_id": entity_id,
"posture": "sitting",
"action": {"verb": "talking"},
"attention": "",
"holding": [],
"status": {},
},
)
ut_id = append_event(
conn,
kind="user_turn",
payload={
"chat_id": "chat_multi",
"prose": "hello",
"segments": [],
},
)
primary_id = append_event(
conn,
kind="assistant_turn",
payload={
"chat_id": "chat_multi",
"speaker_id": "bot_a",
"text": "Original primary.",
"truncated": False,
"user_turn_id": ut_id,
},
)
interjection_id = append_event(
conn,
kind="assistant_turn",
payload={
"chat_id": "chat_multi",
"speaker_id": "bot_b",
"text": "Original interjection!",
"truncated": False,
"user_turn_id": ut_id,
"interjection_of": "bot_a",
},
)
project(conn)
return ut_id, primary_id, interjection_id
def test_regenerate_broadcasts_turn_html_over_sse(
tmp_path, monkeypatch
):
"""T73.1: regenerate publishes a ``turn_html_replace`` SSE event so
connected tabs swap the prior turn's DOM node in place.
The event carries:
- ``data``: rendered HTML for the new turn
- ``turn_id``: event_id of the new assistant_turn
- ``supersedes_id``: event_id of the original assistant_turn
"""
import asyncio
from chat.config import Settings
from chat.db.migrate import apply_migrations
from chat.services import regenerate as regenerate_module
from chat.services.regenerate import regenerate_assistant_turn
db_path = tmp_path / "test.db"
cfg = tmp_path / "config.toml"
cfg.write_text('featherless_api_key = "test"\n')
monkeypatch.setenv("CHAT_CONFIG_PATH", str(cfg))
monkeypatch.setenv("CHAT_DB_PATH", str(db_path))
apply_migrations(db_path)
ut_id, at_id = _seed_with_one_turn(db_path)
published: list[tuple[str, dict]] = []
async def _capture(chat_id, event):
published.append((chat_id, event))
# Patch the imported reference inside the regenerate module so the
# service's call site goes through our spy.
monkeypatch.setattr(regenerate_module, "publish", _capture)
narrative_canned = "Refreshed reply."
state_canned = json.dumps(
{"affinity_delta": 0, "trust_delta": 0, "knowledge_facts": []}
)
canned = [narrative_canned, state_canned, state_canned]
mock_client = MockLLMClient(canned=list(canned))
settings = Settings(featherless_api_key="test")
with open_db(db_path) as conn:
new_text = asyncio.run(
regenerate_assistant_turn(
conn,
mock_client,
settings=settings,
chat_id="chat_bot_a",
original_assistant_event_id=at_id,
)
)
assert new_text == narrative_canned
# Find the new assistant_turn event_id for cross-checking.
cur = conn.execute(
"SELECT id FROM event_log "
"WHERE kind = 'assistant_turn' AND id != ? "
"AND superseded_by IS NULL",
(at_id,),
).fetchone()
new_at_id = cur[0]
# Filter out per-token publishes; we want the replace broadcast.
replace_calls = [
ev for (_cid, ev) in published if ev.get("event") == "turn_html_replace"
]
assert len(replace_calls) == 1
payload = replace_calls[0]
assert payload["supersedes_id"] == at_id
assert payload["turn_id"] == new_at_id
# The HTML carries the new narrative text and the speaker name.
assert "Refreshed reply." in payload["data"]
assert "BotA" in payload["data"]
# Sanity: every publish targeted this chat.
for cid, _ev in published:
assert cid == "chat_bot_a"
def test_regenerate_with_interjection_redoes_both_turns(tmp_path, monkeypatch):
"""T73.2: when the original turn group included an interjection, both
the primary and the interjection are regenerated.
Setup: 3-entity scene (host BotA + guest BotB + you) with a prior
(primary by BotA + interjection by BotB) group. Mock the
interjection classifier to return ``should_interject=True`` so the
follow-on regenerates too.
Assert: 2 new assistant_turns exist for the same user_turn_id, the
second carrying ``interjection_of`` pointing at the new primary's
speaker_id. Both originals are superseded.
"""
import asyncio
from chat.config import Settings
from chat.db.migrate import apply_migrations
from chat.services import regenerate as regenerate_module
from chat.services.interjection import InterjectionDecision
from chat.services.regenerate import regenerate_assistant_turn
db_path = tmp_path / "test.db"
cfg = tmp_path / "config.toml"
cfg.write_text('featherless_api_key = "test"\n')
monkeypatch.setenv("CHAT_CONFIG_PATH", str(cfg))
monkeypatch.setenv("CHAT_DB_PATH", str(db_path))
apply_migrations(db_path)
ut_id, primary_id, interjection_id = _seed_with_interjection_group(db_path)
# Stub detect_interjection so the classifier "fires" with new prose.
async def _stub_should_interject(*_args, **_kwargs):
return InterjectionDecision(should_interject=True, reason="fired")
monkeypatch.setattr(
regenerate_module, "detect_interjection", _stub_should_interject
)
# Canned queue:
# 1. New primary narrative stream.
# 2-7. Six state-update classifier calls (one per directed pair
# across host/you/guest = 6 pairs) for the primary pass.
# 8. New interjection narrative stream.
# 9-14. Six state-update classifier calls for the post-interjection
# pass.
state_canned = json.dumps(
{"affinity_delta": 0, "trust_delta": 0, "knowledge_facts": []}
)
canned: list[str] = []
canned.append("New primary text.")
canned.extend([state_canned] * 6)
canned.append("New interjection text!")
canned.extend([state_canned] * 6)
mock_client = MockLLMClient(canned=list(canned))
settings = Settings(featherless_api_key="test")
with open_db(db_path) as conn:
new_text = asyncio.run(
regenerate_assistant_turn(
conn,
mock_client,
settings=settings,
chat_id="chat_multi",
original_assistant_event_id=primary_id,
)
)
assert new_text == "New primary text."
# Both originals are superseded.
primary_super = conn.execute(
"SELECT superseded_by FROM event_log WHERE id = ?", (primary_id,)
).fetchone()[0]
interjection_super = conn.execute(
"SELECT superseded_by FROM event_log WHERE id = ?",
(interjection_id,),
).fetchone()[0]
assert primary_super is not None
assert interjection_super is not None
# Two NEW assistant_turn events exist (the regenerated primary
# and the regenerated interjection), both pinned to the same
# user_turn_id as the originals.
cur = conn.execute(
"SELECT id, payload_json FROM event_log "
"WHERE kind = 'assistant_turn' AND id NOT IN (?, ?) "
"ORDER BY id",
(primary_id, interjection_id),
).fetchall()
assert len(cur) == 2
new_primary_id, new_primary_payload_json = cur[0]
new_interjection_id, new_interjection_payload_json = cur[1]
new_primary_payload = json.loads(new_primary_payload_json)
new_interjection_payload = json.loads(new_interjection_payload_json)
assert new_primary_payload["text"] == "New primary text."
assert new_primary_payload["speaker_id"] == "bot_a"
assert new_primary_payload["user_turn_id"] == ut_id
assert new_primary_payload["regenerated_from"] == primary_id
assert "interjection_of" not in new_primary_payload
assert new_interjection_payload["text"] == "New interjection text!"
assert new_interjection_payload["speaker_id"] == "bot_b"
assert new_interjection_payload["user_turn_id"] == ut_id
assert new_interjection_payload["regenerated_from"] == interjection_id
# interjection_of links to the new primary's speaker (matches
# the existing convention in chat/web/turns.py).
assert new_interjection_payload["interjection_of"] == "bot_a"
# The originals' supersede pointers reach the new ones.
assert primary_super == new_primary_id
assert interjection_super == new_interjection_id
def test_regenerate_drops_interjection_when_classifier_returns_false(
tmp_path, monkeypatch
):
"""T73.2: when the original group included an interjection but the
classifier returns False this time, the new group is primary-only.
The original interjection is still superseded (we don't leave it
visible in the timeline alongside a regenerated primary it no longer
follows from), but no replacement assistant_turn is appended.
"""
import asyncio
from chat.config import Settings
from chat.db.migrate import apply_migrations
from chat.services import regenerate as regenerate_module
from chat.services.interjection import InterjectionDecision
from chat.services.regenerate import regenerate_assistant_turn
db_path = tmp_path / "test.db"
cfg = tmp_path / "config.toml"
cfg.write_text('featherless_api_key = "test"\n')
monkeypatch.setenv("CHAT_CONFIG_PATH", str(cfg))
monkeypatch.setenv("CHAT_DB_PATH", str(db_path))
apply_migrations(db_path)
ut_id, primary_id, interjection_id = _seed_with_interjection_group(db_path)
async def _stub_no_interject(*_args, **_kwargs):
return InterjectionDecision(
should_interject=False, reason="quiet"
)
monkeypatch.setattr(
regenerate_module, "detect_interjection", _stub_no_interject
)
# Canned queue: primary narrative + 6 state-update calls. No
# interjection stream because the classifier short-circuits.
state_canned = json.dumps(
{"affinity_delta": 0, "trust_delta": 0, "knowledge_facts": []}
)
canned: list[str] = ["New primary text."] + [state_canned] * 6
mock_client = MockLLMClient(canned=list(canned))
settings = Settings(featherless_api_key="test")
with open_db(db_path) as conn:
new_text = asyncio.run(
regenerate_assistant_turn(
conn,
mock_client,
settings=settings,
chat_id="chat_multi",
original_assistant_event_id=primary_id,
)
)
assert new_text == "New primary text."
# Original primary superseded by the new primary.
primary_super = conn.execute(
"SELECT superseded_by FROM event_log WHERE id = ?", (primary_id,)
).fetchone()[0]
# Original interjection ALSO superseded — we don't leave a
# dangling beat attached to a regenerated primary that no longer
# warrants a follow-on. Back-pointer goes to the new primary.
interjection_super = conn.execute(
"SELECT superseded_by FROM event_log WHERE id = ?",
(interjection_id,),
).fetchone()[0]
assert primary_super is not None
assert interjection_super is not None
assert interjection_super == primary_super # both point at new primary
# Exactly ONE new assistant_turn — the primary; no replacement
# interjection.
cur = conn.execute(
"SELECT payload_json FROM event_log "
"WHERE kind = 'assistant_turn' AND id NOT IN (?, ?) "
"AND superseded_by IS NULL",
(primary_id, interjection_id),
).fetchall()
assert len(cur) == 1
new_primary_payload = json.loads(cur[0][0])
assert new_primary_payload["text"] == "New primary text."
assert "interjection_of" not in new_primary_payload
+109
View File
@@ -0,0 +1,109 @@
"""Tests for the relationship-seed service (T38).
Per Requirements §5.2, when two bots first co-appear in a chat, the user
is prompted with "Have they met before? If yes, write a short prose
seed." The prose is parsed via classifier into structured directed-edge
content for the ``botA -> botB`` and ``botB -> botA`` edges.
These tests cover:
* The happy path: a canned classifier response parses cleanly into a
populated :class:`RelationshipSeed` with both directions filled.
* Empty prose short-circuits before any classifier call (mock has no
canned responses; an accidental call would raise ``IndexError``).
* Whitespace-only prose has the same short-circuit behavior.
"""
from __future__ import annotations
import json
import pytest
from chat.llm.mock import MockLLMClient
from chat.services.relationship_seed import (
RelationshipSeed,
seed_inter_bot_edges,
)
@pytest.mark.asyncio
async def test_seed_parses_canned_prose():
canned = json.dumps(
{
"a_to_b_summary": "old college friend who now distrusts him slightly",
"a_to_b_knowledge_facts": [
"studied physics together",
"lost touch after a falling out",
],
"a_to_b_affinity_delta": 2,
"a_to_b_trust_delta": -1,
"b_to_a_summary": "former roommate; warm memories, mild resentment",
"b_to_a_knowledge_facts": ["lived together junior year"],
"b_to_a_affinity_delta": 3,
"b_to_a_trust_delta": 0,
}
)
mock = MockLLMClient(canned=[canned])
result = await seed_inter_bot_edges(
mock,
classifier_model="x",
bot_a_id="bot_a",
bot_a_name="Alice",
bot_b_id="bot_b",
bot_b_name="Bob",
relationship_prose=(
"Alice and Bob met in college. They studied physics together and "
"lived as roommates junior year, but drifted apart after a fight."
),
)
assert isinstance(result, RelationshipSeed)
assert (
result.a_to_b_summary
== "old college friend who now distrusts him slightly"
)
assert result.a_to_b_knowledge_facts == [
"studied physics together",
"lost touch after a falling out",
]
assert result.a_to_b_affinity_delta == 2
assert result.a_to_b_trust_delta == -1
assert (
result.b_to_a_summary
== "former roommate; warm memories, mild resentment"
)
assert result.b_to_a_knowledge_facts == ["lived together junior year"]
assert result.b_to_a_affinity_delta == 3
assert result.b_to_a_trust_delta == 0
@pytest.mark.asyncio
async def test_seed_empty_prose_returns_empty():
"""Empty prose short-circuits — classifier must not be called."""
mock = MockLLMClient(canned=[])
result = await seed_inter_bot_edges(
mock,
classifier_model="x",
bot_a_id="bot_a",
bot_a_name="Alice",
bot_b_id="bot_b",
bot_b_name="Bob",
relationship_prose="",
)
assert result == RelationshipSeed()
@pytest.mark.asyncio
async def test_seed_whitespace_only_prose_returns_empty():
"""Whitespace-only prose is treated the same as empty."""
mock = MockLLMClient(canned=[])
result = await seed_inter_bot_edges(
mock,
classifier_model="x",
bot_a_id="bot_a",
bot_a_name="Alice",
bot_b_id="bot_b",
bot_b_name="Bob",
relationship_prose=" \n ",
)
assert result == RelationshipSeed()
+347
View File
@@ -183,3 +183,350 @@ def test_bot_list_renders_reset_form(client, tmp_path):
assert response.status_code == 200 assert response.status_code == 200
assert "Reset" in response.text assert "Reset" in response.text
assert "confirm_name" in response.text assert "confirm_name" in response.text
def _seed_two_bots_with_guest_link(
db: Path, *, extra_events: list[dict] | None = None
) -> None:
"""Seed bot_a + bot_b, each hosting their own chat, with bot_b a guest in chat_bot_a.
``extra_events`` is appended after the guest_added event and projected
together with the rest of the seed (so handlers run only once per event).
"""
with open_db(db) as conn:
# bot_a + its chat
append_event(
conn,
kind="bot_authored",
payload={
"id": "bot_a",
"name": "BotA",
"persona": "thoughtful",
"voice_samples": [],
"traits": [],
"backstory": "",
"initial_relationship_to_you": "coworker",
"kickoff_prose": "",
},
)
append_event(
conn,
kind="chat_created",
payload={
"id": "chat_bot_a",
"host_bot_id": "bot_a",
"initial_time": "2026-04-26T20:00:00+00:00",
"narrative_anchor": "Day 1",
"weather": "",
},
)
# bot_b + its own chat
append_event(
conn,
kind="bot_authored",
payload={
"id": "bot_b",
"name": "BotB",
"persona": "curious",
"voice_samples": [],
"traits": [],
"backstory": "",
"initial_relationship_to_you": "friend",
"kickoff_prose": "",
},
)
append_event(
conn,
kind="chat_created",
payload={
"id": "chat_bot_b",
"host_bot_id": "bot_b",
"initial_time": "2026-04-26T20:00:00+00:00",
"narrative_anchor": "Day 1",
"weather": "",
},
)
# bot_b joins chat_bot_a as a guest.
append_event(
conn,
kind="guest_added",
payload={
"chat_id": "chat_bot_a",
"guest_bot_id": "bot_b",
},
)
for ev in extra_events or []:
append_event(conn, kind=ev["kind"], payload=ev["payload"])
project(conn)
def test_reset_clears_guest_reference_in_other_chats(client, tmp_path):
db = tmp_path / "test.db"
_seed_two_bots_with_guest_link(db)
# Sanity-check the seed: bot_b is the guest in bot_a's chat.
from chat.state.world import get_chat
with open_db(db) as conn:
assert get_chat(conn, "chat_bot_a")["guest_bot_id"] == "bot_b"
assert get_chat(conn, "chat_bot_b") is not None
response = client.post(
"/bots/bot_b/reset",
data={"confirm_name": "BotB"},
follow_redirects=False,
)
assert response.status_code == 303
with open_db(db) as conn:
# The guest reference in bot_a's chat is cleared.
chat_a = get_chat(conn, "chat_bot_a")
assert chat_a is not None
assert chat_a["guest_bot_id"] is None
# bot_b's own chat is gone (Phase 1 host purge behavior).
assert get_chat(conn, "chat_bot_b") is None
# bot_a is untouched.
assert conn.execute(
"SELECT COUNT(*) FROM bots WHERE id = 'bot_a'"
).fetchone()[0] == 1
def test_reset_purges_orphaned_you_activity_rows(client, tmp_path):
"""T69: when a bot's chats are deleted, "you" activity rows tied to those
chats' containers should also be purged (otherwise they linger orphaned)."""
db = tmp_path / "test.db"
with open_db(db) as conn:
append_event(
conn,
kind="bot_authored",
payload={
"id": "bot_a",
"name": "BotA",
"persona": "thoughtful",
"voice_samples": [],
"traits": [],
"backstory": "",
"initial_relationship_to_you": "coworker",
"kickoff_prose": "",
},
)
append_event(
conn,
kind="chat_created",
payload={
"id": "chat_bot_a",
"host_bot_id": "bot_a",
"initial_time": "2026-04-26T20:00:00+00:00",
"narrative_anchor": "Day 1",
"weather": "",
},
)
append_event(
conn,
kind="container_created",
payload={
"chat_id": "chat_bot_a",
"name": "office",
"type": "workplace",
"properties": {},
},
)
append_event(
conn,
kind="activity_change",
payload={
"entity_id": "you",
"container_id": 1,
"posture": "standing",
"action": {"verb": "watching"},
},
)
project(conn)
# Sanity: the "you" activity row exists and points at the container.
assert conn.execute(
"SELECT COUNT(*) FROM activity WHERE entity_id = 'you'"
).fetchone()[0] == 1
response = client.post(
"/bots/bot_a/reset",
data={"confirm_name": "BotA"},
follow_redirects=False,
)
assert response.status_code == 303
with open_db(db) as conn:
# The orphaned "you" activity row tied to bot_a's purged container is gone.
assert conn.execute(
"SELECT COUNT(*) FROM activity WHERE entity_id = 'you'"
).fetchone()[0] == 0
def test_reset_does_not_purge_you_activity_in_other_chats(client, tmp_path):
"""T69: resetting bot_a must leave a "you" activity row pointing at
bot_b's container intact — only orphans from the reset bot's chats go."""
db = tmp_path / "test.db"
with open_db(db) as conn:
# bot_a + its chat + container.
append_event(
conn,
kind="bot_authored",
payload={
"id": "bot_a",
"name": "BotA",
"persona": "thoughtful",
"voice_samples": [],
"traits": [],
"backstory": "",
"initial_relationship_to_you": "coworker",
"kickoff_prose": "",
},
)
append_event(
conn,
kind="chat_created",
payload={
"id": "chat_bot_a",
"host_bot_id": "bot_a",
"initial_time": "2026-04-26T20:00:00+00:00",
"narrative_anchor": "Day 1",
"weather": "",
},
)
append_event(
conn,
kind="container_created",
payload={
"chat_id": "chat_bot_a",
"name": "office",
"type": "workplace",
"properties": {},
},
)
# bot_b + its chat + container.
append_event(
conn,
kind="bot_authored",
payload={
"id": "bot_b",
"name": "BotB",
"persona": "curious",
"voice_samples": [],
"traits": [],
"backstory": "",
"initial_relationship_to_you": "friend",
"kickoff_prose": "",
},
)
append_event(
conn,
kind="chat_created",
payload={
"id": "chat_bot_b",
"host_bot_id": "bot_b",
"initial_time": "2026-04-26T20:00:00+00:00",
"narrative_anchor": "Day 1",
"weather": "",
},
)
append_event(
conn,
kind="container_created",
payload={
"chat_id": "chat_bot_b",
"name": "kitchen",
"type": "home",
"properties": {},
},
)
# The activity table is keyed on entity_id (PRIMARY KEY), so only one
# "you" row exists at a time. Point it at bot_b's container so reset of
# bot_a should NOT touch it.
append_event(
conn,
kind="activity_change",
payload={
"entity_id": "you",
"container_id": 2, # kitchen, in chat_bot_b
"posture": "sitting",
"action": {"verb": "reading"},
},
)
project(conn)
# Sanity: the "you" activity row is in bot_b's container.
row = conn.execute(
"SELECT container_id FROM activity WHERE entity_id = 'you'"
).fetchone()
assert row is not None and row[0] == 2
response = client.post(
"/bots/bot_a/reset",
data={"confirm_name": "BotA"},
follow_redirects=False,
)
assert response.status_code == 303
with open_db(db) as conn:
# The "you" activity in bot_b's container is preserved.
row = conn.execute(
"SELECT container_id FROM activity WHERE entity_id = 'you'"
).fetchone()
assert row is not None
assert row[0] == 2
def test_reset_purges_guest_memories_from_other_chats(client, tmp_path):
db = tmp_path / "test.db"
_seed_two_bots_with_guest_link(
db,
extra_events=[
# bot_b is a guest in chat_bot_a and remembers things from there.
{
"kind": "memory_written",
"payload": {
"owner_id": "bot_b",
"chat_id": "chat_bot_a",
"pov_summary": "Met BotA; she was tense.",
"witness_you": 1,
"witness_host": 1,
"witness_guest": 1,
"significance": 3,
},
},
# And a memory from bot_b's own chat for good measure.
{
"kind": "memory_written",
"payload": {
"owner_id": "bot_b",
"chat_id": "chat_bot_b",
"pov_summary": "A quiet evening at home.",
"witness_you": 1,
"witness_host": 1,
"witness_guest": 0,
"significance": 1,
},
},
],
)
with open_db(db) as conn:
# Sanity: bot_b owns 2 memories pre-reset, one in each chat.
assert conn.execute(
"SELECT COUNT(*) FROM memories WHERE owner_id = 'bot_b'"
).fetchone()[0] == 2
response = client.post(
"/bots/bot_b/reset",
data={"confirm_name": "BotB"},
follow_redirects=False,
)
assert response.status_code == 303
with open_db(db) as conn:
# ALL of bot_b's memories are gone, including the cross-chat one in chat_bot_a.
assert conn.execute(
"SELECT COUNT(*) FROM memories WHERE owner_id = 'bot_b'"
).fetchone()[0] == 0
assert conn.execute(
"SELECT COUNT(*) FROM memories WHERE owner_id = 'bot_b' AND chat_id = 'chat_bot_a'"
).fetchone()[0] == 0
+160
View File
@@ -0,0 +1,160 @@
"""Skip narration service tests (T53).
The skip-narration service generates short transition prose between an
in-progress moment and a post-skip moment. Two flavors:
* ``elision`` collapses an in-progress activity to its expected
end-state in 1-2 sentences (e.g. "skip ahead to when we arrive").
* ``jump`` bridges a longer fiction-time delta in 2-3 sentences
(e.g. "next morning", "a week later").
Output is free-form prose, not structured JSON, so the service goes
through ``client.generate`` directly rather than the classifier path.
A deterministic template fallback fires on any LLM failure so the skip
flow never blocks even when the model is down.
"""
from __future__ import annotations
from typing import AsyncIterator, Sequence
import pytest
from chat.llm.client import Message
from chat.llm.mock import MockLLMClient
from chat.services.skip_narration import narrate_skip
_SPEAKER = {
"id": "bot1",
"name": "Aria",
"persona": "thoughtful, observant",
}
@pytest.mark.asyncio
async def test_narrate_elision_returns_classifier_output():
canned = (
"She closes her laptop and slings her bag over her shoulder. "
"The office shrinks behind her as she steps into the late "
"afternoon light."
)
mock = MockLLMClient(canned=[canned])
result = await narrate_skip(
mock,
narrative_model="x",
skip_kind="elision",
speaker_bot=_SPEAKER,
you_name="Me",
current_time="3:42 PM",
new_time="5:10 PM",
current_activity="finishing up at her desk",
landing_state_hint="walking out into the parking lot",
)
assert "office" in result or result == canned
@pytest.mark.asyncio
async def test_narrate_jump_returns_classifier_output():
canned = (
"Morning light spills through the kitchen window. The coffee "
"maker hums. She's already at the table, scrolling her phone."
)
mock = MockLLMClient(canned=[canned])
result = await narrate_skip(
mock,
narrative_model="x",
skip_kind="jump",
speaker_bot=_SPEAKER,
you_name="Me",
current_time="late evening",
new_time="next morning",
current_activity="winding down for the night",
landing_state_hint="having coffee in the kitchen",
)
assert result
lower = result.lower()
assert "morning" in lower or "coffee" in lower
class _RaisingMock:
"""Mock LLMClient whose ``generate`` always raises.
``MockLLMClient.generate`` raises ``IndexError`` once the canned
list is empty, but the test wants a clear, unambiguous failure
regardless of canned-list state, so we ship a tiny dedicated mock
instead.
"""
async def generate(
self, messages: Sequence[Message], *, model: str, **params
) -> str:
raise RuntimeError("LLM is down")
async def stream(
self, messages: Sequence[Message], *, model: str, **params
) -> AsyncIterator[str]:
raise RuntimeError("LLM is down")
yield # pragma: no cover - make this a generator
class _RecordingMock:
"""Mock LLMClient that records the kwargs passed to ``generate``.
Used to assert that callers plumb through optional parameters like
``timeout_s`` instead of swallowing them. Returns a fixed string so
the surrounding fallback path is not exercised.
"""
def __init__(self) -> None:
self.captured_kwargs: dict | None = None
async def generate(
self, messages: Sequence[Message], *, model: str, **params
) -> str:
self.captured_kwargs = dict(params)
return "ok"
async def stream(
self, messages: Sequence[Message], *, model: str, **params
) -> AsyncIterator[str]:
raise RuntimeError("not used")
yield # pragma: no cover - make this a generator
@pytest.mark.asyncio
async def test_narrate_skip_passes_timeout_through():
mock = _RecordingMock()
await narrate_skip(
mock,
narrative_model="x",
skip_kind="jump",
speaker_bot=_SPEAKER,
you_name="Me",
current_time="late evening",
new_time="next morning",
current_activity="winding down for the night",
landing_state_hint="having coffee in the kitchen",
timeout_s=12.5,
)
assert mock.captured_kwargs is not None
assert mock.captured_kwargs.get("timeout_s") == 12.5
@pytest.mark.asyncio
async def test_narrate_falls_back_on_generation_failure():
new_time = "next morning"
result = await narrate_skip(
_RaisingMock(),
narrative_model="x",
skip_kind="jump",
speaker_bot=_SPEAKER,
you_name="Me",
current_time="late evening",
new_time=new_time,
current_activity="winding down for the night",
landing_state_hint="having coffee in the kitchen",
)
# Fallback template includes the new_time so callers can see *what*
# we skipped to even when the LLM never answered.
assert new_time in result
+98
View File
@@ -0,0 +1,98 @@
"""Tests for the synthesized-memories service (T54).
When the user jump-skips ("a week later") they are prompted "anything
notable happen?" If they answer with prose, this service parses it into
1-N synthesized memories per present bot. Each memory carries
``source="synthesized"`` and ``reliability=0.7`` (the caller T62 skip
flow applies those tags when persisting; this service just produces
the structured digest).
These tests cover:
* The happy path: a canned classifier response parses cleanly into a
populated :class:`SynthesizedDigest` with one memory.
* Empty prose short-circuits before any classifier call the mock has
no canned responses, so an accidental call would raise
``IndexError``.
* Classifier failure (3 bad responses, exhausting :func:`classify`'s
retry budget) falls back to an empty default digest.
"""
from __future__ import annotations
import json
import pytest
from chat.llm.mock import MockLLMClient
from chat.services.synthesized_memories import (
SynthesizedDigest,
SynthesizedMemory,
synthesize_memories,
)
@pytest.mark.asyncio
async def test_synthesize_parses_canned_prose():
canned = json.dumps(
{
"memories": [
{
"text": "Maya started a new pottery class.",
"significance": 1,
"affinity_delta": 0,
"trust_delta": 0,
}
]
}
)
mock = MockLLMClient(canned=[canned])
result = await synthesize_memories(
mock,
classifier_model="x",
prose="we saw each other at her pottery class once",
bot_name="Maya",
bot_persona="warm potter, mid-30s",
you_name="Sam",
)
assert isinstance(result, SynthesizedDigest)
assert len(result.memories) == 1
mem = result.memories[0]
assert isinstance(mem, SynthesizedMemory)
assert mem.text == "Maya started a new pottery class."
assert mem.significance == 1
assert mem.affinity_delta == 0
assert mem.trust_delta == 0
@pytest.mark.asyncio
async def test_empty_prose_returns_empty_digest():
"""Empty prose short-circuits — the classifier must not be called."""
mock = MockLLMClient(canned=[])
result = await synthesize_memories(
mock,
classifier_model="x",
prose="",
bot_name="Maya",
bot_persona="warm potter, mid-30s",
you_name="Sam",
)
assert result == SynthesizedDigest()
assert result.memories == []
@pytest.mark.asyncio
async def test_classifier_failure_returns_empty_default():
"""Three bad responses exhaust the classifier's retry budget; the
service then returns the empty default digest."""
mock = MockLLMClient(canned=["bad", "bad", "bad"])
result = await synthesize_memories(
mock,
classifier_model="x",
prose="we saw each other at her pottery class once",
bot_name="Maya",
bot_persona="warm potter, mid-30s",
you_name="Sam",
)
assert result == SynthesizedDigest()
assert result.memories == []
+128
View File
@@ -0,0 +1,128 @@
"""Tests for the thread-detection service (T55).
On scene close, the transcript is classified to detect open threads
(unresolved arcs, dangling questions, promises made). The service can
also signal **update** to an existing thread when the scene developed
it, or **close** when the scene resolved it.
These tests cover:
* A new thread the scene introduced action="open" with a fresh title.
* An update to an existing thread action="update" with
``existing_thread_id`` referencing the prior thread.
* Classifier failure three bad responses degrade to an empty
candidates list (graceful degradation, §3.3).
* Empty transcript short-circuits before any classifier call.
"""
from __future__ import annotations
import json
import pytest
from chat.llm.mock import MockLLMClient
from chat.services.thread_detection import (
ThreadCandidate,
ThreadDetectionResult,
detect_threads,
)
@pytest.mark.asyncio
async def test_detects_new_thread_open():
canned = json.dumps(
{
"candidates": [
{
"action": "open",
"title": "Maya's job hunt",
"summary": "Maya is looking for a new job",
"existing_thread_id": None,
}
]
}
)
mock = MockLLMClient(canned=[canned])
result = await detect_threads(
mock,
classifier_model="x",
scene_transcript=[
{"speaker": "Maya", "text": "I need to find a new job soon."},
{"speaker": "Sam", "text": "What kind of role are you looking for?"},
],
open_threads=[],
)
assert isinstance(result, ThreadDetectionResult)
assert len(result.candidates) == 1
cand = result.candidates[0]
assert isinstance(cand, ThreadCandidate)
assert cand.action == "open"
assert cand.title == "Maya's job hunt"
assert cand.summary == "Maya is looking for a new job"
assert cand.existing_thread_id is None
@pytest.mark.asyncio
async def test_detects_update_to_existing_thread():
canned = json.dumps(
{
"candidates": [
{
"action": "update",
"title": "",
"summary": "Maya interviewed at Acme today",
"existing_thread_id": "thr_jobhunt",
}
]
}
)
mock = MockLLMClient(canned=[canned])
result = await detect_threads(
mock,
classifier_model="x",
scene_transcript=[
{"speaker": "Maya", "text": "I had the Acme interview today."},
{"speaker": "Sam", "text": "How did it go?"},
],
open_threads=[
{
"thread_id": "thr_jobhunt",
"title": "Maya's job hunt",
"summary": "Maya is looking for a new job",
}
],
)
assert len(result.candidates) == 1
cand = result.candidates[0]
assert cand.action == "update"
assert cand.existing_thread_id == "thr_jobhunt"
assert cand.summary == "Maya interviewed at Acme today"
@pytest.mark.asyncio
async def test_classifier_failure_returns_empty():
"""Three malformed classifier responses → empty candidates list."""
mock = MockLLMClient(canned=["not json", "still not json", "{bad"])
result = await detect_threads(
mock,
classifier_model="x",
scene_transcript=[
{"speaker": "Maya", "text": "Anything could happen here."},
],
open_threads=[],
)
assert result.candidates == []
@pytest.mark.asyncio
async def test_empty_transcript_short_circuits():
"""Empty transcript short-circuits — classifier must not be called."""
mock = MockLLMClient(canned=[])
result = await detect_threads(
mock,
classifier_model="x",
scene_transcript=[],
open_threads=[],
)
assert result.candidates == []
+181
View File
@@ -0,0 +1,181 @@
from __future__ import annotations
from chat.db.connection import open_db
from chat.db.migrate import apply_migrations
from chat.eventlog.log import append_and_apply, append_event
from chat.eventlog.projector import project
import chat.state.entities # registers handlers
import chat.state.world # registers handlers
import chat.state.threads # registers handlers
from chat.state.threads import get_thread, list_open_threads
def _bot_payload(bot_id: str, name: str) -> dict:
return {
"id": bot_id,
"name": name,
"persona": "thoughtful, observant",
"voice_samples": [],
"traits": [],
"backstory": "",
"initial_relationship_to_you": "coworker",
"kickoff_prose": "",
}
def _chat_payload(chat_id: str = "chat_bot_a") -> dict:
return {
"id": chat_id,
"host_bot_id": "bot_a",
"guest_bot_id": "bot_b",
"initial_time": "2026-04-26T20:00:00+00:00",
"narrative_anchor": "Day 1 evening",
"weather": "clear",
}
def test_thread_opened_creates_row(tmp_path):
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
append_event(conn, kind="bot_authored", payload=_bot_payload("bot_a", "BotA"))
append_event(conn, kind="chat_created", payload=_chat_payload())
append_event(
conn,
kind="thread_opened",
payload={
"thread_id": "thr_abc",
"chat_id": "chat_bot_a",
"title": "Maya's job hunt",
"summary": "Maya is looking for a new job",
},
)
project(conn)
t = get_thread(conn, "thr_abc")
assert t is not None
assert t["thread_id"] == "thr_abc"
assert t["chat_id"] == "chat_bot_a"
assert t["title"] == "Maya's job hunt"
assert t["summary"] == "Maya is looking for a new job"
assert t["status"] == "open"
assert t["closed_at"] is None
assert t["last_referenced_scene_id"] is None
def test_thread_updated_changes_summary_and_last_referenced(tmp_path):
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
append_event(conn, kind="bot_authored", payload=_bot_payload("bot_a", "BotA"))
append_event(conn, kind="chat_created", payload=_chat_payload())
append_event(
conn,
kind="thread_opened",
payload={
"thread_id": "thr_abc",
"chat_id": "chat_bot_a",
"title": "Maya's job hunt",
"summary": "Maya is looking for a new job",
},
)
append_event(
conn,
kind="thread_updated",
payload={
"thread_id": "thr_abc",
"summary": "Maya landed an interview at a startup",
"last_referenced_scene_id": 42,
},
)
project(conn)
t = get_thread(conn, "thr_abc")
assert t is not None
assert t["summary"] == "Maya landed an interview at a startup"
assert t["last_referenced_scene_id"] == 42
assert t["status"] == "open"
def test_thread_closed_terminal(tmp_path):
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
append_event(conn, kind="bot_authored", payload=_bot_payload("bot_a", "BotA"))
append_event(conn, kind="chat_created", payload=_chat_payload())
append_event(
conn,
kind="thread_opened",
payload={
"thread_id": "thr_abc",
"chat_id": "chat_bot_a",
"title": "Maya's job hunt",
"summary": "Maya is looking for a new job",
},
)
append_event(
conn,
kind="thread_closed",
payload={
"thread_id": "thr_abc",
"closed_at": "2026-04-26T21:00:00+00:00",
},
)
project(conn)
t = get_thread(conn, "thr_abc")
assert t is not None
assert t["status"] == "closed"
assert t["closed_at"] == "2026-04-26T21:00:00+00:00"
# Subsequent updates to a closed thread are no-ops.
append_and_apply(
conn,
kind="thread_updated",
payload={
"thread_id": "thr_abc",
"summary": "should not be applied",
},
)
t2 = get_thread(conn, "thr_abc")
assert t2 is not None
assert t2["summary"] == "Maya is looking for a new job"
assert t2["status"] == "closed"
def test_list_open_threads_filters_to_open_only(tmp_path):
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
append_event(conn, kind="bot_authored", payload=_bot_payload("bot_a", "BotA"))
append_event(conn, kind="chat_created", payload=_chat_payload())
for tid, title in [
("thr_1", "Arc 1"),
("thr_2", "Arc 2"),
("thr_3", "Arc 3"),
]:
append_event(
conn,
kind="thread_opened",
payload={
"thread_id": tid,
"chat_id": "chat_bot_a",
"title": title,
"summary": "",
},
)
append_event(
conn,
kind="thread_closed",
payload={
"thread_id": "thr_3",
"closed_at": "2026-04-26T21:00:00+00:00",
},
)
project(conn)
open_threads = list_open_threads(conn, "chat_bot_a")
assert len(open_threads) == 2
ids = {t["thread_id"] for t in open_threads}
assert ids == {"thr_1", "thr_2"}
+132
View File
@@ -0,0 +1,132 @@
from __future__ import annotations
from chat.db.connection import open_db
from chat.db.migrate import apply_migrations
from chat.eventlog.log import append_event
from chat.eventlog.projector import project
import chat.state.world # registers handlers
from chat.state.world import get_activity, get_chat
def _chat_payload(**overrides):
payload = {
"id": "chat_bot_a",
"host_bot_id": "bot_a",
"guest_bot_id": None,
"initial_time": "2026-04-26T20:00:00+00:00",
"narrative_anchor": "Day 1 evening",
"weather": "clear",
}
payload.update(overrides)
return payload
def _container_payload(**overrides):
payload = {
"chat_id": "chat_bot_a",
"name": "office",
"type": "workplace",
"properties": {
"public": True,
"moving": False,
"audible_range": "normal",
"slots": [],
},
"parent_id": None,
}
payload.update(overrides)
return payload
def _activity_payload(**overrides):
payload = {
"entity_id": "bot_a",
"container_id": 1,
"slot": "desk_chair",
"posture": "sitting",
"action": {"verb": "writing email"},
"attention": "the screen",
"holding": ["pen"],
"status": {"hungry": False},
}
payload.update(overrides)
return payload
def _seed_events(conn):
"""Append seed events but do NOT project — caller appends more then projects once."""
append_event(conn, kind="chat_created", payload=_chat_payload())
append_event(conn, kind="container_created", payload=_container_payload())
append_event(conn, kind="activity_change", payload=_activity_payload())
def test_elision_advances_chat_clock_only(tmp_path):
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
_seed_events(conn)
append_event(conn, kind="time_skip_elision", payload={
"chat_id": "chat_bot_a",
"new_time": "2026-04-26T20:30:00+00:00",
})
project(conn)
chat = get_chat(conn, "chat_bot_a")
assert chat["time"] == "2026-04-26T20:30:00+00:00"
# Activity row preserved with the same fields it was seeded with.
a = get_activity(conn, "bot_a")
assert a is not None
assert a["entity_id"] == "bot_a"
assert a["container_id"] == 1
assert a["slot"] == "desk_chair"
assert a["posture"] == "sitting"
assert a["action"] == {"verb": "writing email"}
assert a["attention"] == "the screen"
assert a["holding"] == ["pen"]
assert a["status"] == {"hungry": False}
def test_jump_with_reset_clears_activity(tmp_path):
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
_seed_events(conn)
append_event(conn, kind="time_skip_jump", payload={
"chat_id": "chat_bot_a",
"new_time": "2026-04-27T08:00:00+00:00",
"reset_activity": True,
})
project(conn)
chat = get_chat(conn, "chat_bot_a")
assert chat["time"] == "2026-04-27T08:00:00+00:00"
count = conn.execute(
"SELECT COUNT(*) FROM activity "
"WHERE container_id IN (SELECT id FROM containers WHERE chat_id = ?)",
("chat_bot_a",),
).fetchone()[0]
assert count == 0
assert get_activity(conn, "bot_a") is None
def test_jump_without_reset_preserves_activity(tmp_path):
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
_seed_events(conn)
append_event(conn, kind="time_skip_jump", payload={
"chat_id": "chat_bot_a",
"new_time": "2026-04-27T08:00:00+00:00",
"reset_activity": False,
})
project(conn)
chat = get_chat(conn, "chat_bot_a")
assert chat["time"] == "2026-04-27T08:00:00+00:00"
a = get_activity(conn, "bot_a")
assert a is not None
assert a["posture"] == "sitting"
assert a["action"]["verb"] == "writing email"
+1116 -1
View File
File diff suppressed because it is too large Load Diff
+269
View File
@@ -0,0 +1,269 @@
"""Task 46 — Witness filter coverage for multi-entity scenarios.
The witness filter is enforced at the SQL layer in
``chat.state.memory.search_memories``. Each memory row carries three witness
flags ``(witness_you, witness_host, witness_guest)``. A retrieval is scoped
to a *bot's own memory store* via ``owner_id`` and a *POV role*
(``"you"``/``"host"``/``"guest"``); the SQL filter is
``WHERE owner_id = ? AND witness_<role> = 1``.
This module exercises the cross-witness scenarios called out in §"Witnessed-By
Tracking" (rp-engine-design.md L108-L116) — multi-witness masks, secondhand
provenance, and the per-owner separation that prevents bleed between bots'
private memory stores.
These are tests-only. ``search_memories`` already accepts ``witness_role``,
so the cases land green without any production-code change. The host-only
hardcode in ``chat/services/prompt.py`` is a separate concern (the v1 prompt
builder always queries from the host POV); these tests pin the underlying
retrieval contract so a future viewer-aware caller has something to lean on.
"""
from __future__ import annotations
from pathlib import Path
from chat.db.connection import open_db
from chat.db.migrate import apply_migrations
from chat.eventlog.log import append_event
from chat.eventlog.projector import project
from chat.state.memory import search_memories
import chat.state.memory # noqa: F401 (registers memory_written handler)
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _seed_memories(db: Path, specs: list[dict]) -> None:
"""Apply migrations and project a list of ``memory_written`` events.
Each spec dict supplies the witness mask + provenance fields explicitly so
the test can name the exact mask under test (``[you, host, guest]``).
"""
apply_migrations(db)
with open_db(db) as conn:
for spec in specs:
payload = {
"owner_id": spec["owner_id"],
"chat_id": spec.get("chat_id", "chat_ab"),
"pov_summary": spec["pov_summary"],
"witness_you": spec["witness_you"],
"witness_host": spec["witness_host"],
"witness_guest": spec["witness_guest"],
"source": spec.get("source", "direct"),
"reliability": spec.get("reliability", 1.0),
"significance": spec.get("significance", 1),
"pinned": 0,
"auto_pinned": 0,
}
append_event(conn, kind="memory_written", payload=payload)
project(conn)
# ---------------------------------------------------------------------------
# Scenario 1 — mask [1, 1, 0]: visible to host, NOT to guest.
# ---------------------------------------------------------------------------
def test_witness_1_1_0_visible_to_host_not_guest(tmp_path):
"""A private host moment ([you=1, host=1, guest=0]) must surface for the
host's own POV query and stay hidden when the guest queries the same
memory store."""
db = tmp_path / "t.db"
_seed_memories(
db,
[
{
"owner_id": "bot_a",
"pov_summary": "BotA quietly noticed the broken vase",
"witness_you": 1,
"witness_host": 1,
"witness_guest": 0,
},
],
)
with open_db(db) as conn:
host_hits = search_memories(conn, "bot_a", "host", "vase", k=4)
assert len(host_hits) == 1
assert host_hits[0]["pov_summary"] == "BotA quietly noticed the broken vase"
# Same store, guest POV: filtered out (witness_guest = 0).
guest_hits = search_memories(conn, "bot_a", "guest", "vase", k=4)
assert guest_hits == []
# ---------------------------------------------------------------------------
# Scenario 2 — mask [0, 1, 1]: visible to BOTH host and guest queries.
# ---------------------------------------------------------------------------
def test_witness_0_1_1_visible_to_both_host_and_guest(tmp_path):
"""A bot-only side scene ([you=0, host=1, guest=1]) must surface from
*both* POV queries against bot stores that recorded it."""
db = tmp_path / "t.db"
_seed_memories(
db,
[
# bot_a recorded the moment from its own (host) POV.
{
"owner_id": "bot_a",
"pov_summary": "the bots whispered about the secret meeting",
"witness_you": 0,
"witness_host": 1,
"witness_guest": 1,
},
# bot_b recorded the same moment from its own (guest) POV.
{
"owner_id": "bot_b",
"pov_summary": "the bots whispered about the secret meeting",
"witness_you": 0,
"witness_host": 1,
"witness_guest": 1,
},
],
)
with open_db(db) as conn:
host_hits = search_memories(conn, "bot_a", "host", "secret", k=4)
assert len(host_hits) == 1
assert host_hits[0]["owner_id"] == "bot_a"
guest_hits = search_memories(conn, "bot_b", "guest", "secret", k=4)
assert len(guest_hits) == 1
assert guest_hits[0]["owner_id"] == "bot_b"
# Cross-check the "you" POV doesn't pick it up — witness_you = 0.
you_hits_a = search_memories(conn, "bot_a", "you", "secret", k=4)
you_hits_b = search_memories(conn, "bot_b", "you", "secret", k=4)
assert you_hits_a == []
assert you_hits_b == []
# ---------------------------------------------------------------------------
# Scenario 3 — mask [1, 0, 0]: degenerate "you-only" memory; filtered out for
# both bot queries because neither host nor guest witness flag is set.
# ---------------------------------------------------------------------------
def test_witness_1_0_0_filtered_out_for_bot_queries(tmp_path):
"""`you` doesn't have a memory store in v1, so a row with only
``witness_you = 1`` is degenerate. From either bot POV the filter must
drop it (it would only ever surface via a ``"you"`` role query, which
isn't a path the v1 prompt builder uses)."""
db = tmp_path / "t.db"
_seed_memories(
db,
[
{
"owner_id": "bot_a",
"pov_summary": "you alone caught the slip of the tongue",
"witness_you": 1,
"witness_host": 0,
"witness_guest": 0,
},
],
)
with open_db(db) as conn:
host_hits = search_memories(conn, "bot_a", "host", "tongue", k=4)
guest_hits = search_memories(conn, "bot_a", "guest", "tongue", k=4)
assert host_hits == []
assert guest_hits == []
# And a ``you`` POV query still finds it — the row exists, just isn't
# reachable from either of the v1 bot retrieval paths.
you_hits = search_memories(conn, "bot_a", "you", "tongue", k=4)
assert len(you_hits) == 1
# ---------------------------------------------------------------------------
# Scenario 4 — secondhand source carries reduced reliability and is still
# witness-filtered. Per design.md L114: "BotA tells BotB about it secondhand:
# creates a new memory in BotB's store flagged [0,0,1] with source: botA".
# We park the mask at [0, 0, 1] (you=0, host=0, guest=1) so that bot_b's
# guest-POV query reaches it, and assert reliability < 1.0 surfaces.
# ---------------------------------------------------------------------------
def test_secondhand_memory_visible_with_reduced_reliability(tmp_path):
"""A secondhand memory ([0, 0, 1] in bot_b's store, ``source = "told_by:bot_a"``)
must surface for bot_b's guest-POV query and carry ``reliability < 1.0``
so downstream callers can tag it as hearsay."""
db = tmp_path / "t.db"
_seed_memories(
db,
[
{
"owner_id": "bot_b",
"pov_summary": "BotA mentioned a fight at the dockyard",
"witness_you": 0,
"witness_host": 0,
"witness_guest": 1,
"source": "told_by:bot_a",
"reliability": 0.6,
},
],
)
with open_db(db) as conn:
hits = search_memories(conn, "bot_b", "guest", "dockyard", k=4)
assert len(hits) == 1
m = hits[0]
assert m["source"] == "told_by:bot_a"
assert m["reliability"] < 1.0
assert m["reliability"] == 0.6
# And it's *not* visible from bot_b's host-POV query — bot_b is the
# guest in this chat, not the host. The mask enforces that.
host_hits = search_memories(conn, "bot_b", "host", "dockyard", k=4)
assert host_hits == []
# ---------------------------------------------------------------------------
# Scenario 5 — owner separation. Two bots both have [1, 1, 1] memories about
# the same event, but the queries are scoped per owner store and must not
# bleed across owners.
# ---------------------------------------------------------------------------
def test_owner_separation_no_cross_owner_bleed(tmp_path):
"""Each bot only sees memories it OWNS, regardless of witness flags. A
fully-witnessed memory in ``bot_a``'s store must not leak into a query
against ``bot_b``'s store and vice versa."""
db = tmp_path / "t.db"
_seed_memories(
db,
[
{
"owner_id": "bot_a",
"pov_summary": "the lighthouse beam swept across all three of them",
"witness_you": 1,
"witness_host": 1,
"witness_guest": 1,
"significance": 2,
},
{
"owner_id": "bot_b",
"pov_summary": "the lighthouse beam swept across all three of them",
"witness_you": 1,
"witness_host": 1,
"witness_guest": 1,
"significance": 2,
},
],
)
with open_db(db) as conn:
# bot_a's host-POV query: only bot_a's row.
a_hits = search_memories(conn, "bot_a", "host", "lighthouse", k=4)
assert len(a_hits) == 1
assert a_hits[0]["owner_id"] == "bot_a"
# bot_b's guest-POV query: only bot_b's row.
b_hits = search_memories(conn, "bot_b", "guest", "lighthouse", k=4)
assert len(b_hits) == 1
assert b_hits[0]["owner_id"] == "bot_b"
# Even though bot_a's memory is fully witnessed, switching to bot_b's
# store with bot_a's POV role still confines us to bot_b's rows.
cross_hits = search_memories(conn, "bot_b", "host", "lighthouse", k=4)
assert len(cross_hits) == 1
assert cross_hits[0]["owner_id"] == "bot_b"
+2 -2
View File
@@ -324,11 +324,11 @@ def test_get_scene_returns_none_for_missing(tmp_path):
assert active_scene(conn, "chat_missing") is None assert active_scene(conn, "chat_missing") is None
def test_schema_version_after_migration_is_7(tmp_path): def test_schema_version_after_migration_is_11(tmp_path):
db = tmp_path / "t.db" db = tmp_path / "t.db"
apply_migrations(db) apply_migrations(db)
with open_db(db) as conn: with open_db(db) as conn:
row = conn.execute( row = conn.execute(
"SELECT value FROM meta WHERE key = 'schema_version'" "SELECT value FROM meta WHERE key = 'schema_version'"
).fetchone() ).fetchone()
assert int(row[0]) == 7 assert int(row[0]) == 11