99 Commits

Author SHA1 Message Date
Joseph Doherty e61bd9cb08 merge: T75 phase 2.5 docs sweep + phase 2.6 backlog 2026-04-26 17:47:01 -04:00
Joseph Doherty c6e0130e59 docs: phase 2.5 status, prune shipped backlog items, capture phase 2.6 follow-ups (T75) 2026-04-26 17:46:50 -04:00
Joseph Doherty 67d6f3fe68 merge: T74 turn-flow polish + addressee service 2026-04-26 17:43:04 -04:00
Joseph Doherty dbc9690358 merge: T73 regenerate.py polish (turn_html SSE + interjection regenerate + stale-guest cleanup) 2026-04-26 17:43:04 -04:00
Joseph Doherty 6d98728a2e chore: remove defensive stale-guest degrade in turns.py (T74.4)
T44 carried a defensive degrade-to-1:1 block in post_turn for the
case where chat.guest_bot_id pointed at a deleted bot. T47 then
fixed the root cause by adding a bot_reset cascade that clears
guest_bot_id from any chat that referenced the deleted bot, so the
post_turn defensive block was rendered dead.

Remove the orphan-clear branch and replace it with a comment
documenting that get_bot now returns a real row when guest_bot_id
is non-None. The cascade behavior is pinned by
test_reset_clears_guest_reference_in_other_chats in tests/test_reset.py.
2026-04-26 17:40:46 -04:00
Joseph Doherty bfb2ffb6f6 chore: pin scene-close-on-cancel behavior + comment rationale (T74.3)
Phase 2 T44 review noted that scene close still runs when a primary
turn is cancelled mid-stream and asked the implementer to review.

Review finding: the existing behavior is correct, not a bug. The
close-detection branch in post_turn consumes ONLY the user's prose
(fully appended to the event_log BEFORE streaming starts) and the
current container name. It does NOT consume the bot's output. A user
who types "we're done here, fade out" and then hits Stop mid-stream
still meant to close — the cancelled bot beat doesn't invalidate
that intent.

- Document the rationale with an inline comment near the
  close-detection branch in chat/web/turns.py.
- Add regression test
  test_cancelled_turn_still_closes_scene_when_user_prose_signals_close
  that drives a stream raising CancelledError on first iteration and
  asserts the scene_closed event still lands.
2026-04-26 17:40:12 -04:00
Joseph Doherty bd13b64959 chore: remove defensive stale-guest degrade in regenerate.py (T73.3)
Phase 2 T44 added a defensive degrade-to-1:1 here when
`chat.guest_bot_id` pointed at a deleted bot. T47 fixed the root cause:
`bot_reset` cascade-clears the column when the referenced bot is purged
(verified by tests/test_reset.py), so the guard was dead code.

No corresponding stale-guest test existed in tests/test_regenerate.py
to remove. The bot_reset cascade test in tests/test_reset.py already
covers the root-cause behavior.
2026-04-26 17:40:07 -04:00
Joseph Doherty f2a57005e5 feat: regenerate covers interjection turns (T73.2)
Phase 2 T44 deferred interjection regenerate — when the original turn
group included a follow-on interjection beat we left it untouched. Now
regenerate redoes BOTH halves:

- Detect a sibling interjection by looking up assistant_turn events
  pinned to the same user_turn_id with `interjection_of` set.
- After streaming the new primary, run `detect_interjection` against
  the new primary text.
- If True: stream a new interjection from the silent witness, append
  with `interjection_of=<new primary speaker_id>`, supersede the
  original interjection, and re-run memory + state-update for the new
  beat.
- If False: supersede the original interjection without a replacement
  (back-pointer goes to the new primary so the row stays consistently
  hidden).

Also broadcast a `turn_html_replace` event for the new interjection so
the front-end can swap the prior interjection node in place (mirrors
T73.1's primary swap).

Tests:
- `test_regenerate_with_interjection_redoes_both_turns`: classifier
  returns True; assert two new assistant_turns land for the same
  user_turn, second carries `interjection_of`, originals superseded.
- `test_regenerate_drops_interjection_when_classifier_returns_false`:
  classifier returns False; assert one new assistant_turn (primary
  only) and the original interjection is superseded with no
  replacement.

`interjection_of` carries the primary's *speaker_id* (matching the
existing convention in chat/web/turns.py) rather than the event_id.
2026-04-26 17:39:31 -04:00
Joseph Doherty 88fae33152 fix: enqueue significance for interjection memories (T74.2)
T44's interjection branch wrote interjection memories via
record_turn_memory_for_present but never enqueued a SignificanceJob,
so the interjection beat could land in memory but never be scored —
which meant it could never auto-pin even when it carried a pivotal
moment.

- Capture the host-POV memory id from the interjection's memory write
  result and enqueue a SignificanceJob mirroring the primary turn's
  pattern. One enqueue per beat (host id; guest POV piggybacks on the
  same score since the prose is identical for v2 — per-POV rewrite
  happens at scene close in T45).
- New test test_interjection_enqueues_significance_job pins the
  contract by intercepting worker.enqueue and asserting two distinct
  jobs land per 3-entity turn that fires an interjection.
2026-04-26 17:38:30 -04:00
Joseph Doherty c874883a84 feat: classifier-based addressee detection (T74.1)
Replace the substring _detect_addressee_id helper with a classifier
call for the multi-entity case. The substring helper is kept as a
fast-path for the no-guest case (no LLM round-trip needed when only
one bot is present, preserves throughput).

- New service chat/services/addressee.py wrapping the existing
  classifier wrapper. AddresseeDecision carries addressee_id +
  confidence + reason; classifier failure falls back to the host with
  reason="fallback" (graceful-degradation, matches the relationship_seed
  / interjection pattern).
- chat/web/turns.py post_turn now calls detect_addressee in the
  multi-entity branch; 1:1 keeps the substring path.
- tests/test_addressee.py: 3 new tests (guest pick, host pick,
  classifier-failure fallback).
- tests/test_turn_flow.py: existing multi-entity tests now feed a
  canned addressee response in the queue. The addressee-routing test
  is updated to assert classifier-driven routing rather than substring.
2026-04-26 17:37:26 -04:00
Joseph Doherty 6f22e86f54 feat: regenerate broadcasts turn_html over SSE (T73.1)
After the new assistant_turn lands, publish a `turn_html_replace` SSE
event carrying the rendered HTML, the new turn_id, and the original
assistant_turn id as `supersedes_id` so connected tabs can swap the
prior DOM node in-place. Phase 1 T29 deferred this — page had to refresh
to see the regenerated turn.

Uses a new event name (not the existing `turn_html`) because the HTMX
`sse-swap="turn_html"` consumer expects raw HTML and an *append*
semantic; regenerate is a *replace*. The new event ships as JSON
(supersedes_id forces sse.py's JSON branch) so the front-end JS can
read the swap target from the payload.

Test: `test_regenerate_broadcasts_turn_html_over_sse` patches the
`publish` reference inside the regenerate module and asserts the
event shape.
2026-04-26 17:36:16 -04:00
Joseph Doherty e632a6247d merge: T72 drawer polish (deferred edits + first-meeting gate + witness flag editing) 2026-04-26 17:32:02 -04:00
Joseph Doherty 607d0971c4 feat: drawer witness flag inline-edit (T72.3)
Memories grow per-flag witness checkboxes (you / host / guest) that
auto-submit on change via HTMX. The new POST route emits a manual_edit
event with target_kind=memory_witness and a {flag, value} payload;
prior_value mirrors the same shape so an inverse edit restores the
flag. The drawer's recent-memories query now selects the three
witness columns alongside the existing fields so the template can
render checkbox state without a second query per row.
2026-04-26 17:28:25 -04:00
Joseph Doherty c265e4ce0f feat: first-meeting gate on drawer Add-guest form (T72.2)
When a host->candidate edge already exists from a prior chat, the
Add-guest form renders the prose textarea disabled with an "already
know each other" note. Submission without the explicit "re-seed
anyway" toggle skips seed_inter_bot_edges so existing edge content
(affinity, trust, knowledge, summaries) survives — guest_added and
group_node_initialized still fire. A small inline script enables /
disables the textarea per-option based on a pre-computed
existing_guest_edges dict surfaced by the GET handler.
2026-04-26 17:26:31 -04:00
Joseph Doherty 21404a373b feat: drawer edits for edge_trust / edge_summary / memory_pov_summary / knowledge_facts (T72.1)
Adds the four POST routes whose state-layer support was already
dispatched by the manual_edit projector (edge_trust, edge_summary,
memory_pov_summary) plus a new edge_knowledge_fact dispatch branch for
add/remove fact list manipulation. Drawer template gains editable
textareas, sliders, and add/remove fact controls. Remove semantics on
knowledge_fact match by string (not index) so concurrent edge_update
events appending facts between drawer renders don't desync the form.
2026-04-26 17:24:24 -04:00
Joseph Doherty 789b9bd042 merge: T71 prompt.py polish (witness role + ACTIVITIES + NICE trim docs) 2026-04-26 17:18:02 -04:00
Joseph Doherty 73bb8c1f17 chore: document NICE trim order rationale (T71.3)
T18 review (Phase 1) noted the NICE-tier trim drops previous-scene
FIRST while §6.3 spec lists previous-scene LAST in the NICE tier
group. Decision: keep the existing greedy order (previous-scene
first), and document why.

Rationale (now in code at the trim ladder):
  1. Cheapest-impact-first — a per-POV previous-scene summary loses
     less narrative continuity than the older dialogue turns or
     memory hits it competes with.
  2. Greedy lookahead is more expensive than the marginal narrative
     loss. Dropping previous-scene typically clears the soft-budget
     slack in one step.

Test added: test_nice_trim_order_documented pins the observed order
(previous-scene -> memories -> dialogue) so a future refactor can't
silently invert it. Sized so that all-NICE config overflows soft but
dropping just previous-scene fits — proves memories and older
dialogue turns survive while previous-scene is the FIRST drop.
2026-04-26 17:16:02 -04:00
Joseph Doherty afd1a50958 refactor: single ACTIVITIES: block with bullet-level trim (T71.2)
Phase 2 T43 added a SECOND ACTIVITIES: block to render guest activity
separately from you+speaker. Two consecutive ACTIVITIES: headers can
read as a duplicate-section bug to the LLM and bloat the prompt.

Consolidate to a single ACTIVITIES: block whose body is composed from
up to three bullets (you, speaker, guest). The block itself is
MUST-tier (always renders); bullet-level trim drops bullets in the
order guest -> group node -> you -> other edges, with the speaker
bullet as the MUST-tier floor (the speaker's own current activity is
the load-bearing slice).

Implementation chose Option B from the polish plan: pre-truncate the
bullets list at trim time before _build_activity_block runs, rather
than introduce a granular tier mode in the trim machinery. Rationale
documented in code; the existing block-level trim ladder gains a
single new toggle (include_you_activity) and the SHOULD-tier
guest_activity_block is gone.

Tests:
- test_single_activities_block_with_three_bullets_when_3_entities:
  exactly one ACTIVITIES: header with all three entity bullets.
- test_tight_budget_drops_guest_activity_bullet_first: speaker bullet
  survives, guest bullet absent under tight budget.
- Existing test_assemble_with_tight_budget_drops_guest_activity_first
  still passes (asserts on bullet absence, not block-header absence).
2026-04-26 17:13:24 -04:00
Joseph Doherty 428438b223 fix: witness role parametric in prompt assembly (T71.1)
Phase 2 T46 pinned the witness mask contract on search_memories with a
witness_role parameter (host/guest/you). The prompt-assembly call site
in assemble_narrative_prompt was hardcoded to "host", which silently
returned the wrong rows when the speaker was the guest bot.

Derive the witness role from chat membership via a new private helper
_witness_role_for(speaker_bot_id, host_bot_id), and apply it at the
search_memories call. Behaviour is identical when the speaker is the
host (or when no guest is present); the fix is load-bearing only when
the guest bot is the speaker — exactly the scenario Phase 2 T43 added
support for.

Tests: pin both directions (host-as-speaker and guest-as-speaker) by
patching the imported search_memories reference and asserting the
witness_role argument the call site emits.
2026-04-26 17:11:20 -04:00
Joseph Doherty b13f3b4e47 merge: T70 LLM-merged group meta-summary 2026-04-26 17:09:16 -04:00
Joseph Doherty f701f9d7dd merge: T69 bot_reset purges orphaned 'you' activity rows 2026-04-26 17:09:16 -04:00
Joseph Doherty 1b9144442a merge: T68 open_db with check_same_thread parameter 2026-04-26 17:09:16 -04:00
Joseph Doherty 13c23fd898 feat: LLM-merged group meta-summary (T70) 2026-04-26 17:07:12 -04:00
Joseph Doherty c1e419e012 fix: bot_reset purges orphaned 'you' activity rows (T69) 2026-04-26 17:06:21 -04:00
Joseph Doherty 994728b5ed refactor: open_db with check_same_thread parameter (T68) 2026-04-26 17:05:29 -04:00
Joseph Doherty e05f28e9d5 docs: add Phase 2.5 cleanup plan (Phase 1.5 + 2.5/3 backlog)
8 tasks across 5 waves consolidating the 15-item backlog tracked in
CLAUDE.md (5 from Phase 1.5 cleanup + 10 from Phase 2.5/3). Items are
grouped by file ownership so each wave stays file-disjoint:

- Wave 1 (parallel): open_db refactor, bot_reset orphan cleanup,
  LLM-merged group meta-summary
- Wave 2 (single): prompt.py polish — witness role parametric, single
  ACTIVITIES block, NICE trim documented
- Wave 3 (single): drawer polish — deferred v1 edits, first-meeting
  gate, witness flag editing
- Wave 4 (parallel): regenerate.py polish (SSE + interjection
  regenerate + stale-guest cleanup); turn-flow polish + new addressee
  service (classifier addressee + significance for interjection +
  scene-close-on-cancel pinned + stale-guest cleanup)
- Wave 5 (single): docs sweep

No schema migrations. Bundled tasks split into per-item sub-commits
for clean review bisection. Uses task ids T68-T75 to avoid collision
with Phase 3 plan (T49-T67) regardless of merge order.
2026-04-26 17:02:46 -04:00
Joseph Doherty 379054755a docs: add Phase 3 implementation plan with parallel-safe waves
19 tasks across 8 waves covering events with lifecycles, time skips
(elision + jump), active threads, significance/retrieval refinements,
and meanwhile scenes (host+guest with no 'you'). Mirrors the Phase 2
plan structure: pre-flight, parallel-execution strategy with worktree
isolation, file-disjointness analysis per wave, and per-task TDD spec
with commit messages.

Phase 3 schema: adds 0009_events.sql, 0010_threads.sql,
0011_meanwhile_scenes.sql (final version 11). Builds on Phase 2's
3-entity scene support and event-sourced architecture.
2026-04-26 16:55:50 -04:00
Joseph Doherty bb87fcbd4a merge: T48 Phase 2 documentation update 2026-04-26 16:28:46 -04:00
Joseph Doherty f6b75b25eb merge: T47 bot_reset cascades to guest references 2026-04-26 16:28:46 -04:00
Joseph Doherty 9f35669936 merge: T46 witness filter coverage for multi-entity scenarios 2026-04-26 16:28:46 -04:00
Joseph Doherty 321810fa54 docs: phase 2 status, behavioral defaults, deferred items 2026-04-26 16:28:14 -04:00
Joseph Doherty fb17ba0657 fix: bot_reset cascades to guest references in other chats 2026-04-26 16:25:37 -04:00
Joseph Doherty d40313063c test: witness filter coverage for multi-entity scenarios 2026-04-26 16:25:03 -04:00
Joseph Doherty 60ac33a787 merge: T44 multi-entity turn flow with interjection support 2026-04-26 16:22:11 -04:00
Joseph Doherty c86b0df411 feat: T44 multi-entity turn flow with interjection support
Rewrites post_turn for the multi-entity world:

- Addressee detection via case-insensitive whole-word match against the
  guest name; defaults to host on no-match or both-match.
- Multi-entity prompt assembly: forwards guest_id so the prompt sees
  the third party's activity / edges / group-node.
- Multi-witness memory write: record_turn_memory_for_present writes one
  memory per present bot witness when a guest is in the room.
- Multi-pair state-update: compute_state_updates_for_present emits one
  edge_update per directed pair (6 with a guest, 2 without).
- Interjection branch (T39): when a guest is present and the primary
  beat completes, the silent witness may follow on. detect_interjection
  decides; on True we stream a second narrative as the witness, append a
  second assistant_turn linked to the same user_turn_id, and re-run the
  multi-pair state update + memory write for the follow-on beat. Cancel
  collapses both halves; a cancelled interjection skips its downstream
  passes so we don't classifier-spam against a half-formed beat.
- Scene-close runs after both beats so apply_scene_close_summary sees
  the full closing scene; T45's guest-aware summarizer handles per-POV
  rewrites for each present witness.

regenerate.py mirrors the prompt / memory / state-update changes for
1:1 and multi-entity scenes. Per the Phase 2 spec, interjection
regeneration is deferred to Phase 2.5 — regenerate only re-streams the
addressee turn for v2.

Tests: adds 5 cases to tests/test_turn_flow.py covering the no-guest
regression, multi-bot without interjection, multi-bot with interjection,
scene-close per-POV rewrites, and addressee routing on a named-bot
prose. Each test pins its own canned MockLLMClient queue with the call
shape documented in the docstring.
2026-04-26 16:18:38 -04:00
Joseph Doherty 44c8735b27 merge: T45 per-POV summaries on close for each present witness 2026-04-26 16:08:54 -04:00
Joseph Doherty 9b601650fb merge: T43 multi-entity prompt assembly 2026-04-26 16:08:54 -04:00
Joseph Doherty fcb111310a feat: multi-entity prompt assembly with guest activity, edges, group node 2026-04-26 16:07:15 -04:00
Joseph Doherty 4e240347b4 feat: per-POV summaries on close for each present witness 2026-04-26 16:06:05 -04:00
Joseph Doherty a90647dddb merge: T42 drawer guest add/remove + render 2026-04-26 16:01:17 -04:00
Joseph Doherty bb83d97088 feat: drawer guest add/remove + render 2026-04-26 15:59:48 -04:00
Joseph Doherty f24ffb8e4f merge: T41 multi-witness memory write helper 2026-04-26 15:54:25 -04:00
Joseph Doherty 9d80b9ae2b merge: T40 multi-entity state-update coordinator 2026-04-26 15:54:25 -04:00
Joseph Doherty 77b42f1ea5 merge: T39 interjection classifier service 2026-04-26 15:54:25 -04:00
Joseph Doherty e7793f2441 feat: multi-witness memory write helper 2026-04-26 15:52:48 -04:00
Joseph Doherty 4ec56dd475 feat: multi-entity state-update coordinator 2026-04-26 15:51:58 -04:00
Joseph Doherty 6a92253ae7 feat: interjection classifier service 2026-04-26 15:51:29 -04:00
Joseph Doherty 22db9f3554 test: bump schema_version assertion to 8 after 0008_group_node migration 2026-04-26 15:49:25 -04:00
Joseph Doherty 6b726b2a4a merge: T38 relationship-seed service 2026-04-26 15:49:03 -04:00
Joseph Doherty e58cdbd527 merge: T37 guest_added/guest_removed event handlers 2026-04-26 15:49:03 -04:00
Joseph Doherty b69a74e69b merge: T36 group_node schema + projector handlers 2026-04-26 15:49:03 -04:00
Joseph Doherty c6b3531c64 feat: relationship-seed service for first-co-appearance prompt 2026-04-26 15:47:12 -04:00
Joseph Doherty a0d7debce5 feat: group_node schema + projector handlers 2026-04-26 15:46:16 -04:00
Joseph Doherty a1b4e251c5 feat: guest_added / guest_removed event handlers 2026-04-26 15:46:09 -04:00
Joseph Doherty b8335895e1 docs: add Phase 2 implementation plan with parallel-safe waves
13 tasks across 6 waves (1, 2, 3, 4a, 4b, 5). Designed for parallel
subagent execution where file-disjointness allows.

Waves 1, 2, 4a, and 5 each contain 2-3 tasks that touch disjoint files
and can be dispatched concurrently via the Agent tool with
isolation: "worktree". Waves 3 (drawer guest support) and 4b (multi-
entity turn flow) are single-task because they touch hot files
(_drawer.html, turns.py) that cannot be safely co-modified.

Plan covers:
- T36: group_node schema + handlers (new migration 0008)
- T37: guest_added / guest_removed event handlers (modifies world.py)
- T38: relationship-seed service ("have they met?")
- T39: interjection classifier service
- T40: multi-entity state-update coordinator (6 directed pairs)
- T41: multi-witness memory write helper
- T42: drawer guest add/remove UI + render
- T43: multi-entity prompt assembly (extends T18)
- T44: multi-entity turn flow (rewrites post_turn)
- T45: multi-entity per-POV summaries on scene close
- T46: witness filter cross-coverage tests
- T47: bot_reset cascades to guest references
- T48: Phase 2 documentation update

Plan also documents:
- Worktree-per-subagent dispatch pattern using Agent isolation flag
- Merge ordering per wave (file-disjointness = conflict-free merges)
- Failure recovery (cancel failed parallel task, re-dispatch as solo)
- Conflict prevention checklist (verify Files sections disjoint per wave)

Tasks file (.tasks.json) carries dependency DAG with `blockedBy` and
`parallelGroup` so a future executing-plans run can dispatch correctly.

NOT EXECUTING. Plan only.
2026-04-26 15:37:07 -04:00
Joseph Doherty d161e7b8e9 feat: cap narrative response length + tune sampling
Bot replies were running long (4 paragraphs of action+dialogue beats
per turn) because we never set max_tokens on the narrative call. Three
tunable knobs now in Settings (set in data/config.toml to override):

- narrative_max_tokens: int = 400
  Hard cap on each generated response. ~400 tokens ≈ 1–2 short
  paragraphs. Drop to 200 for terse banter, bump to 800+ for longer
  scenes.

- narrative_temperature: float = 0.85
  Sampling temperature. 0.7 = grounded/consistent (slightly stiff),
  0.85 = creative-but-in-character (default), 1.0 = wide variety,
  >1.0 = often off-the-rails.

- prompt closing instruction now nudges: "Keep your response to a
  single beat — one or two short paragraphs at most. Don't monologue;
  leave room for the other person to react."

Both turns.py (post_turn) and regenerate.py forward the params to
client.stream(). FeatherlessClient already passes **params through to
the OpenAI-compat endpoint.

Note: temperature doesn't control length — that was a common
misconception. max_tokens is the actual length cap. Lower temperature
makes word choice more predictable (slightly stiffer voice), not
shorter. Both knobs are useful for different goals.
2026-04-26 15:28:08 -04:00
Joseph Doherty f0742dd4f9 fix: use readOnly (not disabled) to lock textarea during stream
The form-submit handler in chat.html was setting
``textarea.disabled = true`` synchronously before the browser actually
serialized the form. Disabled form fields are excluded from
submission, so the request body contained ``prose=""`` even when the
user had typed text — which the server (correctly) rejected with the
new empty-prose 400. Net effect: typing "hello" + Send gave a "prose
cannot be empty" error.

Switched to ``readOnly``: same UX (user can't edit while streaming)
but the field IS submitted. The unlock path now also clears the
textarea and refocuses for the next turn.
2026-04-26 15:23:06 -04:00
Joseph Doherty 52555e0455 fix: reject empty prose on turn submit
Empty submission was producing a blank user_turn event in the log and
firing the LLM stream anyway — the bot would invent a response from the
kickoff context alone, producing a monologue with no user input. Two-
layer fix:

- Browser: add `required` to the prose textarea in chat.html so the
  form refuses to submit empty.
- Server: 400 in post_turn when prose.strip() is empty. Defense in
  depth — if a client bypasses the textarea attribute (custom UI,
  curl, etc.), the server still rejects.

Verified live: POST with empty body returns 400; POST with whitespace-
only returns 400; chat shell renders the textarea with required.
Full suite: 168 passed.
2026-04-26 15:20:02 -04:00
Joseph Doherty 5c039c8e56 fix: classifier timeout + Featherless concurrency cap
Two related issues blocking real-world use of the kickoff parse:

1. Classifier calls take ~12s end-to-end on Featherless for the
   complex KickoffParse schema (Hermes-3-8B generating ~1.3KB of
   structured JSON). The 10s timeout was firing on most attempts,
   causing all 3 retries to time out and the empty-fallback to render
   with blank form values. Bumping the default
   classifier_timeout_s 10 → 30s gives generous headroom; measured
   p99 is ~13s, so 30s is comfortable.

2. Featherless caps concurrent connections per account (2 on free /
   lower paid tiers). Each turn flow can fire 4–5 calls (parse,
   scene-close detect, narrative stream, two state-update passes)
   plus the background significance worker. Without a gate, we'd
   exceed the cap and fail.

   Added a class-level ``asyncio.Semaphore`` to FeatherlessClient,
   shared across all instances, configured once in lifespan from
   ``Settings.featherless_max_concurrent`` (default 2). Both
   ``generate`` and ``stream`` acquire the semaphore for the duration
   of the call; the stream holds it until the async generator
   completes, so token streaming is correctly accounted for.

Verified live: 4/4 sequential kickoff parses for the same bot all
succeed with real parsed values (previously ~50% blank-fallback).
Full suite: 168 passed.
2026-04-26 15:15:14 -04:00
Joseph Doherty 5aab98e4d7 fix: classifier robustness — schema in prompt, retries, kickoff fallback
The kickoff parse-and-confirm route was 500-ing intermittently because
Hermes-3 + Featherless's response_format={"type":"json_object"} only
guarantees JSON output, NOT a particular schema. The model was inventing
its own field names (sceneTime, entities, settingDetails) instead of
the KickoffParse fields, causing Pydantic validation to fail on both
classify() retries.

Three changes:

1. Include the Pydantic JSON schema in the system prompt so the model
   knows exactly which keys to produce. Affects every classify() call
   (kickoff parse, turn parse, scene-close detect, significance,
   state-update, scene summarize). Strip ```json fences if the model
   wraps its output. Bump retries 2 → 3 (model is stochastic; one extra
   attempt closes most of the remaining gap).

2. parse_kickoff() now passes a default empty KickoffParse so the
   route degrades to a fillable form instead of 500 when the classifier
   ultimately fails. The confirm form is the human-in-the-loop; an
   empty form is strictly better UX than a stack trace.

3. Tests updated: bumped canned-failure arrays from 2 → 3 entries to
   match the new attempt count; renamed kickoff test from
   "raises_when_classifier_fails_twice" to
   "falls_back_to_empty_when_classifier_fails" reflecting the new
   degraded-but-usable behavior.

Verified live with all 3 sample bots (maya/eli/sam) — kickoff route
returns 200 across multiple attempts. Full suite: 168 passed.
2026-04-26 15:03:13 -04:00
Joseph Doherty 12502d6ec7 chore: add scripts/seed_sample_bots.py
Idempotent seeder for three sample bots (Maya — coworker slow-burn,
Eli — live-in partner, Sam — bartender / new connection). Each is a
distinct relational archetype to exercise the system from different
angles. Run from repo root:

    .venv/bin/python scripts/seed_sample_bots.py

Re-running skips ids that already exist. After seeding, walk each bot
through kickoff parse-and-confirm at /bots/<id>/kickoff.
2026-04-26 14:50:06 -04:00
Joseph Doherty 365dacc0d0 chore: post-Phase-1 cleanup — gitignore, packaging, backlog
- .gitignore: add *.egg-info/ so editable installs don't show in git status.
- pyproject.toml: add [build-system] and [tool.setuptools.packages.find]
  scoped to chat*, fixing pip install -e . which was failing on data/
  auto-discovery.
- CLAUDE.md: add Phase 1.5 cleanup backlog section under Phase 1 status,
  capturing the small follow-ups surfaced in implementer reviews
  (open_db refactor, regenerate SSE broadcast, you-activity purge,
  drawer edits for deferred fields, NICE trim order).
2026-04-26 14:39:10 -04:00
Joseph Doherty a302ed427a feat: error banners and first-run navigation flow 2026-04-26 14:33:28 -04:00
Joseph Doherty 0353d592cd feat: streaming UX with Stop, disconnect handling, send-lock 2026-04-26 14:27:39 -04:00
Joseph Doherty 330077afcf feat: transcript display formatting with markdown and OOC styling 2026-04-26 14:22:43 -04:00
Joseph Doherty 8390703b73 feat: nightly DB backups with 14-day retention 2026-04-26 14:18:57 -04:00
Joseph Doherty b9644fad31 feat: periodic snapshots with retention and cold-load fast-path 2026-04-26 14:15:17 -04:00
Joseph Doherty 82be8b3f51 feat: bot reset with hard confirm and event-driven purge 2026-04-26 14:07:56 -04:00
Joseph Doherty 46062973c2 feat: regenerate with edit-then-regenerate inline UX 2026-04-26 14:04:02 -04:00
Joseph Doherty aa0563b4fa feat: rewind with impact preview, pre-rewind snapshot, undo toast 2026-04-26 13:58:20 -04:00
Joseph Doherty b5175aefaa feat: per-POV summary and edge summary update on scene close 2026-04-26 13:53:12 -04:00
Joseph Doherty 0997562e75 feat: scene close on hard signals with manual override 2026-04-26 13:46:14 -04:00
Joseph Doherty db3005fc17 feat: drawer edits with manual_edit event capture 2026-04-26 13:40:40 -04:00
Joseph Doherty 5fc5b8ac23 feat: read-only drawer with scene, activity, edges, memories 2026-04-26 13:35:47 -04:00
Joseph Doherty 3995a8671b feat: FTS5 memory retrieval with witness filter and ranking boosts 2026-04-26 13:30:40 -04:00
Joseph Doherty eb4cdf9cbb feat: async significance pass with auto-pin on score 3 2026-04-26 13:27:25 -04:00
Joseph Doherty a45dabb6ae feat: per-turn memory writes with witness flags 2026-04-26 13:20:43 -04:00
Joseph Doherty e8d24a0875 feat: post-turn state-update pass per present entity 2026-04-26 13:17:07 -04:00
Joseph Doherty 9b45710cb1 feat: narrative streaming via SSE with assistant_turn event 2026-04-26 13:09:31 -04:00
Joseph Doherty 73d8b0c092 feat: prompt assembly with must/should/nice trim tiers 2026-04-26 13:00:00 -04:00
Joseph Doherty a0f5e818ec feat: turn input parser via classifier 2026-04-26 12:53:34 -04:00
Joseph Doherty 656c2558cb feat: per-chat SSE channel and pub/sub 2026-04-26 12:49:41 -04:00
Joseph Doherty e79f4d8d22 feat: chat shell page rendering 2026-04-26 12:39:15 -04:00
Joseph Doherty 0c08745194 feat: top-level nav and chat list view 2026-04-26 12:36:20 -04:00
Joseph Doherty fbb16c86b3 feat: kickoff parse-and-confirm flow with chat creation 2026-04-26 12:28:05 -04:00
Joseph Doherty e44e2bf93f feat: settings page with you-entity authoring 2026-04-26 12:22:00 -04:00
Joseph Doherty 44ea627a8a feat: bot authoring form with bot_authored event 2026-04-26 12:17:06 -04:00
Joseph Doherty a5339fc1d2 feat: kickoff prose parser via classifier 2026-04-26 12:09:17 -04:00
Joseph Doherty ec344064f1 feat: chats, chat_state, containers, scenes, activity tables 2026-04-26 12:03:26 -04:00
Joseph Doherty 30e6648122 feat: memory schema with witness flags and FTS5 index 2026-04-26 11:56:32 -04:00
Joseph Doherty bc97d425ef feat: directed edges with per-turn delta projector 2026-04-26 11:51:15 -04:00
Joseph Doherty 7e6c2985dd docs: fix Task 6 plan snippet: PRAGMA table_info name index is c[1] not c[0] 2026-04-26 11:48:30 -04:00
Joseph Doherty 5e6bbb586c feat: bot and you entity schemas with projector handlers 2026-04-26 11:46:19 -04:00
Joseph Doherty 517fe49aef feat: append-only event log with projector skeleton 2026-04-26 11:42:49 -04:00
Joseph Doherty c2aceffda1 feat: classifier wrapper with retry, timeout, schema-default fallback 2026-04-26 11:38:48 -04:00
Joseph Doherty e627356168 feat: LLMClient protocol with Featherless and mock implementations 2026-04-26 11:35:57 -04:00
Joseph Doherty 67517926aa feat: sqlite migration runner with meta version table 2026-04-26 11:32:32 -04:00
Joseph Doherty 01e6975d20 feat: config loader with toml + env override 2026-04-26 11:28:40 -04:00
Joseph Doherty 4a60171035 feat: project skeleton with health endpoint 2026-04-26 11:23:38 -04:00
133 changed files with 21557 additions and 1 deletions
+7
View File
@@ -2,3 +2,10 @@
# v1 runtime data (DB, backups, snapshots, exports, config with secrets)
data/
# Python
.venv/
__pycache__/
*.pyc
.pytest_cache/
*.egg-info/
+1
View File
@@ -0,0 +1 @@
3.12
+64
View File
@@ -50,6 +50,10 @@ The 3-entity cap is load-bearing: it makes the relationship graph fully enumerab
- **Snapshots**: periodic every 100 events / 30 min; pre-rewind always. 5 periodic retained; pre-rewind retained 14 days.
- **Streaming**: Stop button on streaming row; mid-stream disconnect commits partial with `truncated: true`; Send disabled mid-stream; multi-tab streaming via per-chat SSE channel.
- **Display**: lightweight markdown; `*action*` italic; OOC `((parens))` shown dimmed/italic, never sent to bot.
- **Multi-entity defaults (Phase 2)**: when `chat.guest_bot_id is None`, behavior matches Phase 1 single-bot 1:1. With a guest, all 3 entities are present in the prompt, witness writes, and state-update fan-out (6 directed pairs).
- **Addressee detection**: simple substring match (whole-word, case-insensitive) over the user turn's body. If both bot names match or neither does, the host gets the floor.
- **Interjection**: classifier-driven, conservative bias (default false on classifier failure / refusal / parse error). When the classifier returns true, the addressee speaks first, then the non-addressee may interject in a follow-up turn.
- **Per-POV summaries (multi-entity)**: each present witness with a memory store gets their own per-POV summary on scene close. The summary differs per bot based on persona + their edge to "you". The group node summary is updated alongside.
## Core concepts (vocabulary)
@@ -149,3 +153,63 @@ Don't jump phases. Phase 1 must work end-to-end before Phase 2 lands.
- Inference hosting (start with a cloud API, re-evaluate later)
- Character template format (during Phase 1)
- Multi-session / multi-character casts: **out of scope for v1**. Leave cheap schema hooks only.
## Phase 1 status
Phase 1 shipped end-to-end across **35 tasks** (T0T35). The single-bot core loop is functional: event log + projector, schema + migrations, settings/bot authoring, kickoff confirm, streaming turns, drawer rendering, regenerate/rewind, scene close + per-POV summaries, significance classifier, snapshots/backups, first-run navigation, and friendly 404/500 pages. **168 tests passing.**
Deferred to Phase 2: second bot, group node, scene configurations, witness filtering across multi-entity scenes, activity/containers, scene-transition compression. Phase 3: event queue + triggers, time skips, active threads. Phase 4: vector retrieval, branching, surgical delete + regenerate, impact-preview UI.
### Known v1 limitations (read before extending)
- **Drawer edits scope**: only affinity, significance, and pin can be hand-edited from the drawer. Other v1 fields (knowledge, summary text, traits) are deferred to Phase 1.5.
- **Cold-load snapshot path** is wired and unit-tested but rarely exercised in dev — long-running sessions are the only realistic trigger.
- **WAL sidecar files** (`-wal`, `-shm`) are not captured in nightly backups; the nightly snapshot is a fresh `.backup()` so this is fine for restore but worth knowing if you copy the db file by hand.
- **HTMX SSE event names** may need a version check if you bump the htmx CDN URL in `base.html` — the swap targets are name-coupled.
- **"You" activity rows** can linger after `bot_reset` (the reset purges the bot's chats and the bot's own activity row but not the "you" row that was associated with those chats). Cosmetic, fixed in Phase 1.5.
- **Projector replay is non-idempotent** for plain `INSERT` events. After appending, call `apply_event(conn, event)` for the new row only — calling `project(conn)` re-runs every handler from scratch and will trip uniqueness or duplicate inserts.
- **8-pin auto-cap eviction** is FIFO over the auto-pinned set only. Manual pins survive the eviction; this is by design (manual intent > auto-pin signal).
- **Regenerate (T29) does not broadcast `turn_html` over SSE** — the page must refresh to show the regenerated turn. Acceptable for v1 single-tab usage; Phase 1.5 should wire the SSE event.
- **First-run middleware** fires only on bare `/` and `/chats`. Sub-paths like `/chats/<id>` and `/chats/<id>/drawer` pass through (correct: HTMX partials should not page-redirect, and a deep-link to a missing chat should 404, not redirect mid-setup).
### Phase 1.5 cleanup backlog
All items shipped — see Phase 2.5 status below.
## Phase 2 status
Phase 2 shipped end-to-end across **13 tasks** (T36T48 wave). The multi-entity surface is functional: chats can host a guest bot, the prompt assembly is guest-aware, post-turn fans out across all directed pairs, and scene close writes a per-POV summary per present witness plus a group_node summary.
- **Multi-entity scene support**: chats can now have a guest bot (you + host + guest). The 3-entity cap holds. New event kinds: `guest_added`, `guest_removed`, `group_node_initialized`, `group_node_updated`. New table: `group_node` (members, summary, dynamic, threads).
- **Drawer guest UX**: add/remove guest from the drawer side panel. The "have they met?" prose seed is parsed by the `relationship_seed` classifier into inter-bot directed edges (host↔guest).
- **Multi-entity turn flow**: `post_turn` assembles narrative with the guest-aware prompt; writes memories for **all** present bot witnesses; runs state updates for **all** directed pairs (6 with 3 entities); detects interjections via classifier (default false; the addressee gets the floor first).
- **Per-POV scene close summaries**: each present witness with a memory store gets their own per-POV summary on close; `group_node` summary updated alongside.
- **Bot reset cascade**: resetting a bot now also clears `chats.guest_bot_id` references in other chats (root-cause fix for stale-guest references after T47).
### Phase 2.5 / 3 backlog
All items shipped — see Phase 2.5 status below.
## Phase 2.5 status
Phase 2.5 cleanup shipped end-to-end across 8 tasks (T68T75). Two CLAUDE.md backlogs (Phase 1.5 cleanup, Phase 2.5/3) are now empty; deferred follow-ups discovered during execution are tracked in a new "Phase 2.6 / 3 backlog" section below.
- **`open_db` with check_same_thread parameter (T68)**: refactored `chat/db/connection.py` so `chat/web/bots.py:get_conn` no longer duplicates the PRAGMA setup. Default behavior preserved.
- **`bot_reset` cross-chat cleanup (T69)**: now purges orphaned "you" activity rows. Note: this also fixed a latent FK constraint crash that was lurking in the projector — `activity.container_id` is FK-referenced and the prior code would have crashed on any reset of a bot whose chat had a non-NULL `container_id` "you" activity row. The bug was masked because no prior test seeded such a row.
- **LLM-merged group meta-summary (T70)**: replaces Phase 2 T45's naive concat with a classifier merge call. Falls back to the naive concat on classifier failure.
- **`prompt.py` polish (T71)**: witness role parametric (`host` vs `guest` derived from chat membership); single `ACTIVITIES:` block with bullet-level trim; NICE trim order kept with documented rationale (greedy cheapest-impact-first beats spec-listing order in practice).
- **Drawer polish (T72)**: deferred v1 edits (edge_trust slider, edge_summary textarea, memory pov_summary textarea, knowledge_facts add/remove) + first-meeting gate (Add-guest form disables prose textarea when host→guest edge already exists; "re-seed anyway" toggle re-enables) + witness flag inline-edit (per-memory checkboxes for [you, host, guest] flags). Two new `manual_edit` projector branches: `edge_knowledge_fact` and `memory_witness`.
- **Regenerate polish (T73)**: regenerate now broadcasts `turn_html_replace` over SSE (NEW event distinct from `turn_html` to avoid breaking the existing append-semantic consumer); regenerate covers interjection turns (re-detects + re-streams or supersedes); defensive stale-guest degrade removed.
- **Turn-flow polish + addressee service (T74)**: classifier-based addressee detection (substring helper kept as no-guest fast path); SignificanceJob enqueued for interjection memories; scene-close-on-cancel pinned with comment + regression test (close detection is genuinely user-prose-only); defensive stale-guest degrade removed.
### Phase 2.6 / 3 backlog
New follow-ups discovered during Phase 2.5 execution. None are blocking; pick up at any time.
- **Frontend handler for `turn_html_replace` SSE event (from T73.1 review)**: regenerate's backend broadcast lands, but no live tab swaps the regenerated turn until a JS handler is wired. The existing `turn_html` event uses HTMX `sse-swap` to append; `turn_html_replace` ships JSON with `supersedes_id` for replacement semantics. Phase 2.6 should wire the JS to swap the prior turn's DOM node in place.
- **Cancel/stop hook for in-flight regenerate streams (from T73 review)**: `post_turn` registers stream tasks in `_in_flight_tasks` so the user can stop them. Regenerate doesn't. A user clicking "Stop" mid-regenerate has no cancel hook today.
- **DRY: regenerate vs post_turn (from T73 review)**: recent-dialogue assembly and prior-edges block are duplicated between `chat/services/regenerate.py` and `chat/web/turns.py`. Extract to shared helpers analogous to `_gather_state_update_inputs`.
- **Sibling-discovery query optimization (from T73 review)**: `regenerate.py`'s sibling-assistant-turn lookup scans all non-superseded `assistant_turn` rows globally. Adding a `chat_id` predicate via JSON extraction (or a denormalized column) bounds the cost to per-chat scale.
- **`_witness_role_for` defensive coding (from T71 review)**: helper returns `"guest"` when `host_bot_id is None`, which is wrong for Phase-1 chats. Defensive: `return "host" if host_bot_id is None or speaker_bot_id == host_bot_id else "guest"`. Not exercised by current tests; harden as a precaution.
- **Confidence type tightening (from T74 review)**: `chat/services/addressee.py::AddresseeDecision.confidence` could be typed as `Literal["high","medium","low"]` for stricter validation. Currently `str` with a comment.
- **Scene-close-on-cancel UX revisit**: T74.3 pinned the existing behavior (close fires even on cancel). If real play-testing surfaces a regression, revisit.
View File
+134
View File
@@ -0,0 +1,134 @@
from __future__ import annotations
import logging
from contextlib import asynccontextmanager
from pathlib import Path
from fastapi import FastAPI, Request
from fastapi.responses import JSONResponse
from fastapi.staticfiles import StaticFiles
from fastapi.templating import Jinja2Templates
from starlette.exceptions import HTTPException as StarletteHTTPException
from chat.config import load_settings
from chat.db.connection import open_db
from chat.db.migrate import apply_migrations
from chat.eventlog.log import read_events
from chat.eventlog.projector import apply_event
from chat.services.background import BackgroundWorker
from chat.services.snapshot import latest_snapshot_path, restore_from_snapshot
# Trigger handler registration:
import chat.state.entities # noqa: F401
import chat.state.edges # noqa: F401
import chat.state.manual_edit # noqa: F401
import chat.state.memory # noqa: F401
import chat.state.world # noqa: F401
from chat.web.bots import router as bots_router
from chat.web.chat import router as chat_router
from chat.web.drawer import router as drawer_router
from chat.web.kickoff import router as kickoff_router
from chat.web.middleware import FirstRunRedirectMiddleware
from chat.web.nav import router as nav_router
from chat.web.settings import router as settings_router
from chat.web.sse import router as sse_router
from chat.web.turns import router as turns_router
log = logging.getLogger(__name__)
@asynccontextmanager
async def lifespan(app: FastAPI):
settings = load_settings()
settings.db_path.parent.mkdir(parents=True, exist_ok=True)
apply_migrations(settings.db_path)
# T31 cold-load fast-path: if a periodic snapshot exists, restore
# projected tables from it and replay only events past its
# ``last_event_id``. Migrations already ran above, so any new tables
# introduced after the snapshot was taken are present and empty —
# the replay-forward step refills them from the event log.
snapshot_path = latest_snapshot_path(settings.data_dir, kind="periodic")
if snapshot_path is not None:
with open_db(settings.db_path) as conn:
last_event_id = restore_from_snapshot(conn, snapshot_path)
for event in read_events(
conn, branch_id=1, after_id=last_event_id
):
apply_event(conn, event)
log.info(
"cold-load restored from %s, replayed events past id %d",
snapshot_path,
last_event_id,
)
app.state.settings = settings
# Cap concurrent Featherless connections to the account's limit
# (free / lower paid tiers cap at 2). Shared across all
# FeatherlessClient instances in the process.
from chat.llm.featherless import FeatherlessClient
FeatherlessClient.configure_concurrency(settings.featherless_max_concurrent)
# Background worker for the async significance pass (T22). Each job
# constructs a fresh FeatherlessClient via the factory; tests can
# disable enqueue by toggling ``app.state.background_worker.enabled``.
def _factory():
return FeatherlessClient(
api_key=settings.featherless_api_key,
base_url=settings.featherless_base_url,
)
worker = BackgroundWorker(settings, llm_client_factory=_factory)
await worker.start()
app.state.background_worker = worker
try:
yield
finally:
await worker.stop()
app = FastAPI(title="chat", lifespan=lifespan)
app.add_middleware(FirstRunRedirectMiddleware)
STATIC_DIR = Path(__file__).resolve().parent / "static"
app.mount("/static", StaticFiles(directory=str(STATIC_DIR)), name="static")
ERROR_TEMPLATES = Jinja2Templates(
directory=str(Path(__file__).resolve().parent / "templates")
)
@app.exception_handler(StarletteHTTPException)
async def http_exception_handler(request: Request, exc: StarletteHTTPException):
"""Render a friendly HTML page for 404/500; JSON for everything else."""
if exc.status_code in (404, 500):
return ERROR_TEMPLATES.TemplateResponse(
request,
"errors.html",
{
"status_code": exc.status_code,
"detail": exc.detail or "Something went wrong.",
"active_nav": "chats",
},
status_code=exc.status_code,
)
return JSONResponse(status_code=exc.status_code, content={"detail": exc.detail})
app.include_router(bots_router)
app.include_router(kickoff_router)
app.include_router(settings_router)
app.include_router(nav_router)
app.include_router(chat_router)
app.include_router(drawer_router)
app.include_router(sse_router)
app.include_router(turns_router)
@app.get("/health")
def health():
return {"status": "ok"}
+58
View File
@@ -0,0 +1,58 @@
from __future__ import annotations
import os
import tomllib
from pathlib import Path
from pydantic import BaseModel, Field
REPO_ROOT = Path(__file__).resolve().parent.parent
DEFAULT_CONFIG = REPO_ROOT / "data" / "config.toml"
DEFAULT_DB = REPO_ROOT / "data" / "chat.db"
class Settings(BaseModel):
featherless_api_key: str
featherless_base_url: str = "https://api.featherless.ai/v1"
narrative_model: str = "dphn/Dolphin-Mistral-24B-Venice-Edition"
classifier_model: str = "NousResearch/Hermes-3-Llama-3.1-8B"
classifier_fallbacks: list[str] = Field(
default_factory=lambda: [
"cognitivecomputations/dolphin-2.9.4-llama3-8b",
"mlabonne/Meta-Llama-3.1-8B-Instruct-abliterated",
]
)
ooc_marker: str = "(("
retrieval_k: int = 4
narrative_budget_hard: int = 8000
narrative_budget_soft: int = 6000
# Cap on each generated bot response. ~400 tokens ≈ 12 short paragraphs.
# Bump if you want longer scenes; drop to 200 for terse banter.
narrative_max_tokens: int = 400
# Sampling temperature for narrative generation. 0.7 = grounded /
# consistent; 0.85 = creative-but-in-character (default); 1.0 = wide
# variety, can drift; >1.0 = often off-the-rails.
narrative_temperature: float = 0.85
classifier_budget_hard: int = 4000
classifier_timeout_s: float = 30.0
# Featherless free tier and lower paid tiers cap concurrent connections.
# Set this to your account's max-concurrent-connections limit.
featherless_max_concurrent: int = 2
db_path: Path = DEFAULT_DB
data_dir: Path = REPO_ROOT / "data"
bind_host: str = "127.0.0.1"
bind_port: int = 8000
def load_settings() -> Settings:
config_path = Path(os.environ.get("CHAT_CONFIG_PATH", DEFAULT_CONFIG))
raw: dict = {}
if config_path.exists():
raw = tomllib.loads(config_path.read_text())
if "CHAT_DB_PATH" in os.environ:
raw["db_path"] = Path(os.environ["CHAT_DB_PATH"])
if "CHAT_DATA_DIR" in os.environ:
raw["data_dir"] = Path(os.environ["CHAT_DATA_DIR"])
elif "data_dir" not in raw and "db_path" in raw:
# T31: when ``CHAT_DB_PATH`` is overridden (typical in tests) but
# ``data_dir`` isn't, derive ``data_dir`` from the db's parent so
# snapshot/auxiliary files stay alongside the test db rather than
# leaking into the real repo data dir.
raw["data_dir"] = Path(raw["db_path"]).parent
return Settings(**raw)
View File
+17
View File
@@ -0,0 +1,17 @@
from __future__ import annotations
import sqlite3
from contextlib import contextmanager
from pathlib import Path
@contextmanager
def open_db(path: Path, *, check_same_thread: bool = True):
path.parent.mkdir(parents=True, exist_ok=True)
conn = sqlite3.connect(path, check_same_thread=check_same_thread)
conn.execute("PRAGMA journal_mode=WAL")
conn.execute("PRAGMA foreign_keys=ON")
try:
yield conn
conn.commit()
finally:
conn.close()
+26
View File
@@ -0,0 +1,26 @@
from __future__ import annotations
from pathlib import Path
from chat.db.connection import open_db
MIGRATIONS_DIR = Path(__file__).parent / "migrations"
def apply_migrations(db_path: Path) -> None:
with open_db(db_path) as conn:
conn.execute(
"CREATE TABLE IF NOT EXISTS meta (key TEXT PRIMARY KEY, value TEXT)"
)
cur = conn.execute("SELECT value FROM meta WHERE key = 'schema_version'")
row = cur.fetchone()
current = int(row[0]) if row else 0
for path in sorted(MIGRATIONS_DIR.glob("*.sql")):
version = int(path.stem.split("_", 1)[0])
if version <= current:
continue
sql = path.read_text()
conn.executescript(sql)
conn.execute(
"INSERT OR REPLACE INTO meta (key, value) VALUES ('schema_version', ?)",
(str(version),),
)
+2
View File
@@ -0,0 +1,2 @@
-- meta table is created by the migrate runner; this migration is a marker.
SELECT 1;
@@ -0,0 +1,8 @@
CREATE TABLE classifier_failures (
id INTEGER PRIMARY KEY,
kind TEXT NOT NULL,
model TEXT NOT NULL,
raw_text TEXT,
attempt_count INTEGER NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now'))
);
+10
View File
@@ -0,0 +1,10 @@
CREATE TABLE event_log (
id INTEGER PRIMARY KEY,
branch_id INTEGER NOT NULL DEFAULT 1,
ts TEXT NOT NULL DEFAULT (datetime('now')),
kind TEXT NOT NULL,
payload_json TEXT NOT NULL,
superseded_by INTEGER REFERENCES event_log(id),
hidden INTEGER NOT NULL DEFAULT 0
);
CREATE INDEX idx_event_log_branch_kind ON event_log(branch_id, kind);
+18
View File
@@ -0,0 +1,18 @@
CREATE TABLE bots (
id TEXT PRIMARY KEY,
name TEXT NOT NULL,
persona TEXT NOT NULL,
voice_samples_json TEXT NOT NULL DEFAULT '[]',
traits_json TEXT NOT NULL DEFAULT '[]',
backstory TEXT NOT NULL DEFAULT '',
initial_relationship_to_you TEXT NOT NULL DEFAULT '',
kickoff_prose TEXT NOT NULL DEFAULT '',
created_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE TABLE you_entity (
id INTEGER PRIMARY KEY CHECK (id = 1),
name TEXT NOT NULL,
pronouns TEXT NOT NULL DEFAULT '',
persona TEXT NOT NULL DEFAULT ''
);
+13
View File
@@ -0,0 +1,13 @@
CREATE TABLE edges (
id INTEGER PRIMARY KEY,
chat_id TEXT,
source_id TEXT NOT NULL,
target_id TEXT NOT NULL,
affinity INTEGER NOT NULL DEFAULT 50,
trust INTEGER NOT NULL DEFAULT 50,
summary TEXT NOT NULL DEFAULT '',
knowledge_json TEXT NOT NULL DEFAULT '[]',
last_interaction_chat_id TEXT,
last_interaction_at TEXT,
UNIQUE (source_id, target_id)
);
+35
View File
@@ -0,0 +1,35 @@
CREATE TABLE memories (
id INTEGER PRIMARY KEY,
owner_id TEXT NOT NULL,
chat_id TEXT NOT NULL,
scene_id INTEGER,
pov_summary TEXT NOT NULL,
witness_you INTEGER NOT NULL,
witness_host INTEGER NOT NULL,
witness_guest INTEGER NOT NULL,
chat_clock_at TEXT,
source TEXT,
reliability REAL NOT NULL DEFAULT 1.0,
significance INTEGER NOT NULL DEFAULT 1,
pinned INTEGER NOT NULL DEFAULT 0,
auto_pinned INTEGER NOT NULL DEFAULT 0,
created_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX idx_memories_owner ON memories(owner_id);
CREATE VIRTUAL TABLE memories_fts USING fts5(
pov_summary, content='memories', content_rowid='id'
);
CREATE TRIGGER memories_ai AFTER INSERT ON memories BEGIN
INSERT INTO memories_fts(rowid, pov_summary) VALUES (new.id, new.pov_summary);
END;
CREATE TRIGGER memories_au AFTER UPDATE ON memories BEGIN
INSERT INTO memories_fts(memories_fts, rowid, pov_summary)
VALUES('delete', old.id, old.pov_summary);
INSERT INTO memories_fts(rowid, pov_summary) VALUES (new.id, new.pov_summary);
END;
CREATE TRIGGER memories_ad AFTER DELETE ON memories BEGIN
INSERT INTO memories_fts(memories_fts, rowid, pov_summary)
VALUES('delete', old.id, old.pov_summary);
END;
+45
View File
@@ -0,0 +1,45 @@
CREATE TABLE chats (
id TEXT PRIMARY KEY,
host_bot_id TEXT NOT NULL,
guest_bot_id TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE TABLE chat_state (
chat_id TEXT PRIMARY KEY,
time TEXT NOT NULL,
weather TEXT NOT NULL DEFAULT '',
active_scene_id INTEGER,
narrative_anchor TEXT
);
CREATE TABLE containers (
id INTEGER PRIMARY KEY,
chat_id TEXT NOT NULL,
name TEXT NOT NULL,
type TEXT NOT NULL,
properties_json TEXT NOT NULL DEFAULT '{}',
parent_id INTEGER REFERENCES containers(id)
);
CREATE TABLE scenes (
id INTEGER PRIMARY KEY,
chat_id TEXT NOT NULL,
container_id INTEGER REFERENCES containers(id),
started_at TEXT NOT NULL,
ended_at TEXT,
significance INTEGER NOT NULL DEFAULT 0,
participants_json TEXT NOT NULL DEFAULT '[]'
);
CREATE TABLE activity (
entity_id TEXT PRIMARY KEY,
container_id INTEGER REFERENCES containers(id),
slot TEXT,
posture TEXT NOT NULL DEFAULT '',
action_json TEXT NOT NULL DEFAULT '{}',
attention TEXT NOT NULL DEFAULT '',
holding_json TEXT NOT NULL DEFAULT '[]',
status_json TEXT NOT NULL DEFAULT '{}',
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
+8
View File
@@ -0,0 +1,8 @@
CREATE TABLE group_node (
chat_id TEXT PRIMARY KEY,
members_json TEXT NOT NULL,
summary TEXT NOT NULL DEFAULT '',
dynamic TEXT NOT NULL DEFAULT '',
threads_json TEXT NOT NULL DEFAULT '[]',
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
View File
+77
View File
@@ -0,0 +1,77 @@
from __future__ import annotations
import json
from dataclasses import dataclass
from typing import Any, Iterator
from sqlite3 import Connection
@dataclass
class Event:
id: int
branch_id: int
ts: str
kind: str
payload: dict[str, Any]
superseded_by: int | None
hidden: bool
def append_event(conn: Connection, *, kind: str, payload: dict[str, Any], branch_id: int = 1) -> int:
cur = conn.execute(
"INSERT INTO event_log (branch_id, kind, payload_json) VALUES (?, ?, ?)",
(branch_id, kind, json.dumps(payload)),
)
return cur.lastrowid
def append_and_apply(
conn: Connection,
*,
kind: str,
payload: dict[str, Any],
branch_id: int = 1,
) -> int:
"""Append an event AND immediately apply just that event's handler.
Calling :func:`chat.eventlog.projector.project` after an append
re-runs every prior event, which is fine for idempotent inserts but
catastrophic for delta-shaped events like ``edge_update`` whose
handler is *not* replay-safe (each pass would re-add the same
``affinity_delta``). This helper runs only the brand-new event
through the registered handler, leaving prior state untouched.
No-ops cleanly when ``kind`` has no registered handler — useful for
transcript-only events like ``user_turn`` / ``assistant_turn`` where
callers may swap ``append_event`` for ``append_and_apply`` without
side effects.
"""
# Local import to avoid a circular dependency at module import: the
# projector imports from .log to define ``Event``.
from chat.eventlog.projector import apply_event
eid = append_event(conn, kind=kind, payload=payload, branch_id=branch_id)
event = Event(
id=eid,
branch_id=branch_id,
ts="",
kind=kind,
payload=payload,
superseded_by=None,
hidden=False,
)
apply_event(conn, event)
return eid
def read_events(conn: Connection, branch_id: int = 1, after_id: int = 0) -> Iterator[Event]:
cur = conn.execute(
"SELECT id, branch_id, ts, kind, payload_json, superseded_by, hidden "
"FROM event_log WHERE branch_id = ? AND id > ? AND hidden = 0 "
"AND superseded_by IS NULL ORDER BY id",
(branch_id, after_id),
)
for row in cur:
yield Event(
id=row[0], branch_id=row[1], ts=row[2], kind=row[3],
payload=json.loads(row[4]), superseded_by=row[5], hidden=bool(row[6]),
)
+27
View File
@@ -0,0 +1,27 @@
from __future__ import annotations
from collections.abc import Callable
from sqlite3 import Connection
from .log import Event, read_events
Handler = Callable[[Connection, Event], None]
_REGISTRY: dict[str, Handler] = {}
def on(kind: str):
def deco(fn: Handler) -> Handler:
_REGISTRY[kind] = fn
return fn
return deco
def project(conn: Connection, branch_id: int = 1) -> None:
for event in read_events(conn, branch_id=branch_id):
h = _REGISTRY.get(event.kind)
if h:
h(conn, event)
def apply_event(conn: Connection, event: Event) -> None:
h = _REGISTRY.get(event.kind)
if h:
h(conn, event)
View File
+62
View File
@@ -0,0 +1,62 @@
from __future__ import annotations
import json
import asyncio
from typing import TypeVar
from pydantic import BaseModel, ValidationError
from .client import LLMClient, Message
T = TypeVar("T", bound=BaseModel)
REFUSAL_PATTERNS = ("i can't", "i cannot", "i'm sorry, but", "as an ai")
def _strip_json_fences(text: str) -> str:
"""Strip ```json ... ``` markdown fences if the model wraps its JSON output."""
s = text.strip()
if s.startswith("```"):
# Drop the first fence line (which may be ``` or ```json)
s = s.split("\n", 1)[1] if "\n" in s else s[3:]
# Drop the trailing fence
if s.rstrip().endswith("```"):
s = s.rstrip()[:-3]
return s.strip()
async def classify(
client: LLMClient,
*,
model: str,
system: str,
user: str,
schema: type[T],
default: T | None = None,
timeout_s: float = 10.0,
) -> T:
schema_json = json.dumps(schema.model_json_schema(), indent=2)
schema_block = (
f"\n\nRespond with a single JSON object matching this exact schema. "
f"Use these field names exactly; do not invent your own keys:\n```json\n{schema_json}\n```"
)
msgs = [
Message(role="system", content=system + schema_block),
Message(role="user", content=user),
]
for attempt in range(3):
try:
text = await asyncio.wait_for(
client.generate(msgs, model=model, response_format={"type": "json_object"}),
timeout=timeout_s,
)
cleaned = _strip_json_fences(text)
if any(p in cleaned.lower()[:80] for p in REFUSAL_PATTERNS) and not cleaned.lstrip().startswith("{"):
raise ValueError("refusal-shaped response")
return schema.model_validate_json(cleaned)
except (ValidationError, ValueError, json.JSONDecodeError, asyncio.TimeoutError):
msgs[0] = Message(
role="system",
content=system + schema_block + "\n\nRespond with valid JSON ONLY. No prose, no markdown fences.",
)
continue
if default is None:
raise RuntimeError(f"classify failed for schema {schema.__name__} with no default")
return default
+14
View File
@@ -0,0 +1,14 @@
from __future__ import annotations
from dataclasses import dataclass
from typing import Protocol, AsyncIterator, Sequence
@dataclass
class Message:
role: str # "system" | "user" | "assistant"
content: str
class LLMClient(Protocol):
async def generate(self, messages: Sequence[Message], *, model: str, **params) -> str: ...
def stream(self, messages: Sequence[Message], *, model: str, **params) -> AsyncIterator[str]: ...
+55
View File
@@ -0,0 +1,55 @@
from __future__ import annotations
import asyncio
from typing import AsyncIterator, Sequence
from openai import AsyncOpenAI
from .client import Message
class FeatherlessClient:
"""Client for Featherless's OpenAI-compatible API.
Featherless caps concurrent connections per account (2 on free / lower
paid tiers). A class-level semaphore gates every ``generate`` and
``stream`` call so the orchestrator never exceeds the configured cap,
regardless of how many ``FeatherlessClient`` instances are alive.
Configure once at app startup via :meth:`configure_concurrency`. The
default is 2.
"""
_semaphore: asyncio.Semaphore | None = None
@classmethod
def configure_concurrency(cls, max_concurrent: int) -> None:
cls._semaphore = asyncio.Semaphore(max(1, int(max_concurrent)))
@classmethod
def _sem(cls) -> asyncio.Semaphore:
if cls._semaphore is None:
cls._semaphore = asyncio.Semaphore(2)
return cls._semaphore
def __init__(self, api_key: str, base_url: str = "https://api.featherless.ai/v1"):
self._client = AsyncOpenAI(api_key=api_key, base_url=base_url)
async def generate(self, messages: Sequence[Message], *, model: str, **params) -> str:
async with self._sem():
resp = await self._client.chat.completions.create(
model=model,
messages=[{"role": m.role, "content": m.content} for m in messages],
**params,
)
return resp.choices[0].message.content or ""
async def stream(self, messages: Sequence[Message], *, model: str, **params) -> AsyncIterator[str]:
async with self._sem():
stream = await self._client.chat.completions.create(
model=model,
messages=[{"role": m.role, "content": m.content} for m in messages],
stream=True,
**params,
)
async for chunk in stream:
delta = chunk.choices[0].delta.content or ""
if delta:
yield delta
+16
View File
@@ -0,0 +1,16 @@
from __future__ import annotations
from typing import AsyncIterator, Sequence
from .client import Message
class MockLLMClient:
def __init__(self, canned: list[str]):
self._canned = list(canned)
async def generate(self, messages: Sequence[Message], *, model: str, **params) -> str:
return self._canned.pop(0)
async def stream(self, messages: Sequence[Message], *, model: str, **params) -> AsyncIterator[str]:
text = self._canned.pop(0)
for ch in text:
yield ch
View File
+108
View File
@@ -0,0 +1,108 @@
"""Addressee classifier service (T74.1).
Phase 2 (T44) detected the addressee — host vs. guest — with a simple
case-insensitive whole-word substring match against the bots' names.
That worked for the obvious case ("BotB, what do you think?") but lost
the long tail: pronouns, paraphrases, indirect address, narrative
focus on a particular party. T74.1 swaps the substring helper for a
classifier call that reads the prose holistically.
The substring helper in :mod:`chat.web.turns` is kept as a fast-path
for the no-guest case (only one bot present means there is nothing to
classify) and as a non-breaking fallback for the regenerate path. The
multi-entity branch in :func:`chat.web.turns.post_turn` calls
:func:`detect_addressee` from this module.
Failure mode: classifier flake or low-confidence response degrades to
the host (the default speaker per Phase 2's host-keeps-the-floor
bias). The decision carries ``confidence`` and ``reason`` so callers
that want to log degraded decisions can distinguish a real "host" call
from a fallback.
"""
from __future__ import annotations
from pydantic import BaseModel
from chat.llm.classify import classify
from chat.llm.client import LLMClient
class AddresseeDecision(BaseModel):
"""Which present bot the user is addressing.
``addressee_id`` is the chosen bot's id. ``confidence`` is one of
``"high"`` / ``"medium"`` / ``"low"`` — callers may treat ``"low"``
as a soft fallback to the host. ``reason`` is a short free-form
string. The classifier-failure fallback uses ``reason="fallback"``
so it's distinguishable from a real low-confidence call.
"""
addressee_id: str
confidence: str = "medium" # "high" | "medium" | "low"
reason: str = ""
_SYSTEM = (
"Given a user's turn prose and the names of present bots, decide "
"which bot the user is addressing. If the user is speaking to no "
"specific bot (descriptive narration, action without dialogue), "
"default to the host. Output strict JSON matching the schema. "
"The addressee_id MUST be one of the ids supplied in the user "
"message — do not invent ids."
)
async def detect_addressee(
client: LLMClient,
*,
classifier_model: str,
user_prose: str,
host_id: str,
host_name: str,
guest_id: str | None,
guest_name: str | None,
timeout_s: float = 30.0,
) -> AddresseeDecision:
"""Classify which present bot the user is addressing.
Defaults to host on classifier failure or when the classifier picks
an id that isn't one of the supplied ids. The caller is expected to
only invoke this in the multi-entity case (a guest is present);
when no guest is present the substring fast-path in
:mod:`chat.web.turns` is used instead and this function is not
called.
"""
fallback = AddresseeDecision(
addressee_id=host_id, confidence="low", reason="fallback"
)
user = (
f"Host: {host_name} (id={host_id})\n"
+ (
f"Guest: {guest_name} (id={guest_id})\n"
if guest_id is not None
else ""
)
+ f"\nUser prose:\n{user_prose}"
)
decision = await classify(
client,
model=classifier_model,
system=_SYSTEM,
user=user,
schema=AddresseeDecision,
default=fallback,
timeout_s=timeout_s,
)
# Defensive: if the classifier returned an id outside the supplied
# set, treat it as a fallback to the host. This catches pathological
# outputs that pass schema validation but pick a phantom id.
valid_ids = {host_id}
if guest_id is not None:
valid_ids.add(guest_id)
if decision.addressee_id not in valid_ids:
return fallback
return decision
__all__ = ["AddresseeDecision", "detect_addressee"]
+262
View File
@@ -0,0 +1,262 @@
"""Async background worker for post-turn jobs (T22).
The turn flow records a ``memory_written`` event synchronously on the
request path so the timeline updates immediately. Significance scoring is
a separate classifier round-trip that we don't want to block on, so the
turn handler enqueues a :class:`SignificanceJob` here and the worker
drains the queue out-of-band.
A single :class:`BackgroundWorker` is started/stopped via FastAPI lifespan
in :mod:`chat.app`. The worker owns its own ``asyncio.Queue`` and runs
exactly one task that pulls jobs off the queue, calls
:func:`chat.services.significance.compute_significance`, and writes
``memory_significance_set`` (and on score 3, ``memory_pin_changed``)
events. Each job opens its own DB connection — workers and request
handlers don't share connections.
Failures inside ``_process`` are logged and swallowed: a flaky classifier
shouldn't take down the worker. Tests can disable enqueue() by setting
``BackgroundWorker.enabled = False`` (e.g. in the existing turn-flow
fixture, which doesn't have a usable LLM key for the lifespan-managed
factory).
"""
from __future__ import annotations
import asyncio
import logging
from dataclasses import dataclass
from typing import Callable
from chat.config import Settings
from chat.db.connection import open_db
from chat.eventlog.log import append_and_apply
from chat.llm.client import LLMClient
from chat.services.backup import (
prune_backups,
should_take_backup,
take_backup,
)
from chat.services.significance import compute_significance
from chat.services.snapshot import (
prune_periodic_snapshots,
should_take_periodic_snapshot,
take_snapshot,
)
# T32: tick-loop wake interval. 60s gives a single backup window per
# target hour with plenty of slack: should_take_backup's 23h freshness
# guard prevents back-to-back runs.
BACKUP_TICK_INTERVAL_SECONDS = 60.0
log = logging.getLogger(__name__)
@dataclass
class SignificanceJob:
"""One unit of work for the background worker.
``host_bot_id`` is the memory's owner — used both for the auto-pin
soft cap query and as the eventual scope for the soft-cap eviction.
"""
memory_id: int
narrative_text: str
prior_dialogue: list[dict]
host_bot_id: str
class BackgroundWorker:
"""asyncio.Queue-backed single-worker task.
Started on app startup; ``stop()`` enqueues a sentinel and awaits the
task so any in-flight job has a chance to finish. Pending jobs after
the sentinel are dropped on shutdown — Phase 1 simplification.
"""
def __init__(
self,
settings: Settings,
llm_client_factory: Callable[[], LLMClient],
*,
enabled: bool = True,
) -> None:
self._settings = settings
self._llm_client_factory = llm_client_factory
self._queue: asyncio.Queue[SignificanceJob | None] = asyncio.Queue()
self._task: asyncio.Task | None = None
# T32: nightly-backup tick loop runs alongside the job loop. The
# event is set by stop() to wake the loop early so shutdown is
# snappy even mid-tick.
self._tick_task: asyncio.Task | None = None
self._tick_stop: asyncio.Event = asyncio.Event()
self.enabled = enabled
async def start(self) -> None:
if self._task is not None:
return
self._task = asyncio.create_task(self._run())
self._tick_task = asyncio.create_task(self._tick_loop())
async def stop(self) -> None:
# Stop the tick loop first — it has no in-flight work to drain,
# so signalling early lets it exit while the job loop is still
# finishing its sentinel handoff.
self._tick_stop.set()
if self._tick_task is not None:
await self._tick_task
self._tick_task = None
if self._task is None:
return
await self._queue.put(None) # sentinel
await self._task
self._task = None
def enqueue(self, job: SignificanceJob) -> None:
if not self.enabled:
return
self._queue.put_nowait(job)
async def _run(self) -> None:
while True:
job = await self._queue.get()
if job is None:
return
try:
await self._process(job)
except Exception as exc: # noqa: BLE001 — worker must not die
log.exception("significance job failed: %s", exc)
async def _tick_loop(self) -> None:
"""Periodic-operations loop (T32 nightly backup).
Wakes every :data:`BACKUP_TICK_INTERVAL_SECONDS` seconds and
asks :func:`should_take_backup` whether a backup is due. The
scheduling decision lives in the backup module so we don't
duplicate the "is it 03:00?" logic here. Failures are caught
and logged so a flaky disk doesn't kill the loop — the next
tick will retry.
Wait uses :func:`asyncio.wait_for` on ``_tick_stop`` so that
:meth:`stop` can interrupt a sleeping tick instead of having to
wait the full interval.
"""
while not self._tick_stop.is_set():
try:
if should_take_backup(self._settings.data_dir):
take_backup(
db_path=self._settings.db_path,
data_dir=self._settings.data_dir,
)
prune_backups(self._settings.data_dir, keep=14)
log.info("nightly backup taken")
except Exception as exc: # noqa: BLE001 — never break the loop
log.exception("backup tick failed: %s", exc)
try:
await asyncio.wait_for(
self._tick_stop.wait(),
timeout=BACKUP_TICK_INTERVAL_SECONDS,
)
except asyncio.TimeoutError:
# Normal path: timed out waiting for stop, run another tick.
pass
async def _process(self, job: SignificanceJob) -> None:
client = self._llm_client_factory()
score = await compute_significance(
client,
model=self._settings.classifier_model,
narrative_text=job.narrative_text,
prior_dialogue=job.prior_dialogue,
)
with open_db(self._settings.db_path) as conn:
append_and_apply(
conn,
kind="memory_significance_set",
payload={
"memory_id": job.memory_id,
"significance": score,
},
)
if score >= 3:
_auto_pin_with_cap(
conn,
owner_id=job.host_bot_id,
memory_id=job.memory_id,
)
# T31: piggy-back the periodic snapshot check on the background
# worker so we don't need a separate timer task. The classifier
# pass already runs out-of-band, so snapshot I/O on the same
# worker is a natural fit. Each snapshot opens its own
# connection so we don't conflate the snapshot's read-only view
# with the significance-write transaction above. Failures are
# caught and logged: a flaky disk shouldn't take down the
# significance pipeline.
try:
with open_db(self._settings.db_path) as conn:
if should_take_periodic_snapshot(
conn, self._settings.data_dir
):
snapshot_path = take_snapshot(
conn,
data_dir=self._settings.data_dir,
kind="periodic",
)
prune_periodic_snapshots(
self._settings.data_dir, keep=5
)
log.info(
"periodic snapshot taken: %s", snapshot_path
)
except Exception as exc: # noqa: BLE001 — never break the worker
log.exception("periodic snapshot failed: %s", exc)
def _auto_pin_with_cap(
conn,
*,
owner_id: str,
memory_id: int,
cap: int = 8,
) -> None:
"""Auto-pin ``memory_id`` and evict the oldest auto-pin if over ``cap``.
Per §8.5: pivotal turns are auto-pinned, with a soft cap of 8 pins per
bot. When the cap is exceeded the oldest auto-pin is unpinned (manual
pins are never auto-evicted — we filter on ``auto_pinned = 1``).
"""
append_and_apply(
conn,
kind="memory_pin_changed",
payload={
"memory_id": memory_id,
"pinned": 1,
"auto_pinned": 1,
},
)
cur = conn.execute(
"SELECT COUNT(*) FROM memories WHERE owner_id = ? AND pinned = 1",
(owner_id,),
)
count = cur.fetchone()[0]
if count <= cap:
return
cur = conn.execute(
"SELECT id FROM memories "
"WHERE owner_id = ? AND pinned = 1 AND auto_pinned = 1 AND id != ? "
"ORDER BY created_at ASC, id ASC LIMIT 1",
(owner_id, memory_id),
)
row = cur.fetchone()
if row is None:
return
append_and_apply(
conn,
kind="memory_pin_changed",
payload={
"memory_id": row[0],
"pinned": 0,
"auto_pinned": 0,
},
)
+106
View File
@@ -0,0 +1,106 @@
"""Nightly DB backup service (T32, Requirements §12).
A simple in-process scheduler: at 03:00 local time daily, copy
``chat.db`` to ``data/backups/chat-<utc-timestamp>.db`` and prune to the
14 most recent. The BackgroundWorker tick loop calls
:func:`should_take_backup` every 60 seconds; when it returns True the
worker calls :func:`take_backup` then :func:`prune_backups`.
The launchd plist suggested in §12 can replace this later by invoking a
small script that calls :func:`take_backup` directly. For v1 the
in-process loop is enough — the daemon already runs continuously to
serve requests, so there's no extra moving part to install.
Backups capture the live ``.db`` file via :func:`shutil.copy2`. SQLite's
WAL mode means an in-flight transaction's pages might live in the
``-wal`` sidecar rather than the main file, but our codebase commits
every write transaction synchronously, so the .db alone is sufficient
for v1. A truly safe online backup would use
``sqlite3.Connection.backup()``; deferred.
"""
from __future__ import annotations
import shutil
from datetime import datetime, timezone
from pathlib import Path
# 03:00 local time per Requirements §12. Hardcoded for v1 — making this
# configurable via Settings is straightforward but not needed yet.
DEFAULT_BACKUP_HOUR = 3
# Retention window per Requirements §12 ("Last 14 retained").
DEFAULT_KEEP = 14
# Wake interval for should_take_backup's freshness check. We wake the
# tick loop every 60s, so a backup taken in the previous tick within the
# same target hour must NOT trigger another. 23h gives us a generous
# safety margin against scheduling jitter while still allowing a single
# backup per day.
FRESHNESS_HOURS = 23
def take_backup(*, db_path: Path, data_dir: Path) -> Path:
"""Copy ``db_path`` to ``data_dir/backups/chat-<utc-timestamp>.db``.
Returns the new file path. Creates the backup directory if missing.
Uses :func:`shutil.copy2` so the destination's mtime is preserved —
:func:`should_take_backup` reads mtime to gate fresh backups.
"""
backup_dir = data_dir / "backups"
backup_dir.mkdir(parents=True, exist_ok=True)
timestamp = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%SZ")
backup_path = backup_dir / f"chat-{timestamp}.db"
shutil.copy2(db_path, backup_path)
return backup_path
def prune_backups(data_dir: Path, *, keep: int = DEFAULT_KEEP) -> int:
"""Remove all but the most recent ``keep`` backup files.
Returns the number of files removed. Safe when the directory is
missing (returns 0). Sorting is by filename, which is the UTC
timestamp embedded in the name — lexicographic order matches
chronological order.
"""
backup_dir = data_dir / "backups"
if not backup_dir.exists():
return 0
files = sorted(backup_dir.glob("chat-*.db"))
to_remove = files[:-keep] if len(files) > keep else []
for f in to_remove:
f.unlink()
return len(to_remove)
def should_take_backup(
data_dir: Path, *, target_hour: int = DEFAULT_BACKUP_HOUR
) -> bool:
"""Decide whether a nightly backup is due.
Two conditions must hold:
* The current local hour matches ``target_hour``.
* No backup file in ``data_dir/backups/`` has an mtime within the
last :data:`FRESHNESS_HOURS` (23h). The 23h window prevents a
double-backup within the same target hour while still allowing
the next day's run to fire on time.
Local time (not UTC) is used for the hour comparison per the
requirements ("03:00 local time"). The filename embeds a UTC stamp
so file ordering remains unambiguous across DST transitions.
"""
now = datetime.now()
if now.hour != target_hour:
return False
backup_dir = data_dir / "backups"
if not backup_dir.exists():
return True
files = list(backup_dir.glob("chat-*.db"))
if not files:
return True
most_recent = max(files, key=lambda f: f.stat().st_mtime)
age_hours = (
datetime.now().timestamp() - most_recent.stat().st_mtime
) / 3600
return age_hours >= FRESHNESS_HOURS
+100
View File
@@ -0,0 +1,100 @@
"""Interjection classifier service (T39).
Per Requirements §6.2, when a guest is present and the addressee bot has
just spoken, the *non-addressee* bot may follow on with a brief
interjection beat. This service decides whether that interjection
fires. Conservative bias: most turns return ``should_interject=False``
— the addressee has the floor and an interjection is the exception.
Trigger ``True`` only when the silent witness's character, given their
persona and edges, would plausibly speak up: jealousy, surprise, strong
agreement worth voicing, correcting a factual falsehood, urgency.
T44 (turn flow) calls this and, on ``True``, generates the brief
follow-on response as the silent witness. Classifier failure falls back
to ``should_interject=False`` with ``reason="fallback"`` so the chat
keeps moving (§3.3 graceful-degradation rule); callers that care can
distinguish a real "no" from a degraded "no" by the reason string.
"""
from __future__ import annotations
from pydantic import BaseModel
from chat.llm.classify import classify
from chat.llm.client import LLMClient
class InterjectionDecision(BaseModel):
"""Whether the silent witness interjects, plus a short reason.
Defaults are a deliberate no-op: ``should_interject=False`` with an
empty reason. The classifier-failure fallback uses
``reason="fallback"`` so it's distinguishable from a real "no".
"""
should_interject: bool = False
reason: str = ""
_SYSTEM = (
"You decide whether a silent witness character interjects after the "
"addressee character finishes speaking. STRONGLY default to false — "
"the addressee has the floor and most turns should NOT have an "
"interjection. Only return true when the silent witness's character, "
"given their persona and edges, would plausibly speak up: jealousy, "
"surprise, strong agreement worth voicing, correcting a factual "
"falsehood, urgency. Output strict JSON matching the schema."
)
async def detect_interjection(
client: LLMClient,
*,
classifier_model: str,
addressee_name: str,
addressee_just_said: str,
silent_witness_name: str,
silent_witness_persona: str,
silent_witness_edge_to_addressee: dict, # {affinity, trust, summary}
silent_witness_edge_to_you: dict,
you_just_said: str,
timeout_s: float = 30.0,
) -> InterjectionDecision:
"""Decide whether the silent witness bot interjects after the addressee
finishes speaking.
The two ``silent_witness_edge_*`` dicts carry the silent witness's
directed edges toward the addressee and toward the user ("you"),
each shaped ``{affinity: int, trust: int, summary: str}``. Missing
keys fall back to a 50/50 baseline with an empty summary so this
function tolerates partially-populated edge state without raising.
"""
user = (
f"You said: {you_just_said}\n\n"
f"{addressee_name} just said: {addressee_just_said}\n\n"
f"Silent witness: {silent_witness_name}\n"
f"Persona: {silent_witness_persona}\n"
f"Edge {silent_witness_name} -> {addressee_name}: "
f"affinity={silent_witness_edge_to_addressee.get('affinity', 50)}, "
f"trust={silent_witness_edge_to_addressee.get('trust', 50)}, "
f"summary={silent_witness_edge_to_addressee.get('summary', '')}\n"
f"Edge {silent_witness_name} -> you: "
f"affinity={silent_witness_edge_to_you.get('affinity', 50)}, "
f"trust={silent_witness_edge_to_you.get('trust', 50)}, "
f"summary={silent_witness_edge_to_you.get('summary', '')}\n\n"
f"Should {silent_witness_name} interject?"
)
return await classify(
client,
model=classifier_model,
system=_SYSTEM,
user=user,
schema=InterjectionDecision,
default=InterjectionDecision(
should_interject=False, reason="fallback"
),
timeout_s=timeout_s,
)
__all__ = ["InterjectionDecision", "detect_interjection"]
+150
View File
@@ -0,0 +1,150 @@
"""Kickoff prose parser.
Service-layer function that converts a bot's authored kickoff prose into a
structured ``KickoffParse`` for the kickoff confirm-and-edit step (T13 will
wire this into the UI flow).
The classifier prompt includes only the bot context that's load-bearing for
parsing the opening scene: name, persona, the authored
``initial_relationship_to_you`` blurb, the ``you`` entity name, and the
kickoff prose itself. Other identity fields (traits, backstory, ...) are
intentionally left out — they would be noise for this extraction.
"""
from __future__ import annotations
from pydantic import BaseModel, Field
from chat.llm.classify import classify
from chat.llm.client import LLMClient
class ActivityShape(BaseModel):
"""Per-entity activity at scene start.
Maps onto Requirements §6.5: ``current_action.{verb,interruptible,
required_attention,expected_duration}`` plus posture, attention, holding.
``action_required_attention`` is left as a free-form string ("low" /
"medium" / "high" expected) rather than a Literal so the classifier has
room to vary phrasing in v1.
"""
posture: str
action_verb: str
action_interruptible: bool
action_required_attention: str # low | medium | high
action_expected_duration: str
attention: str = ""
holding: list[str] = Field(default_factory=list)
class KickoffParse(BaseModel):
"""Structured opening-scene state extracted from kickoff prose.
``container_properties`` is loose ``dict``: the classifier may emit
``moving`` / ``public`` / ``audible_range`` keys, but downstream
consumers (T13's confirm form) handle missing keys gracefully.
``initial_time_iso`` is stored as text — not validated as a datetime
here; ``chat_state.time`` stores it as a plain string.
"""
container_name: str
container_type: str
container_properties: dict
you_activity: ActivityShape
bot_activity: ActivityShape
initial_time_iso: str
edge_seed_summary: str
edge_seed_knowledge_facts: list[str]
_SYSTEM_PROMPT = (
"You are extracting structured scene state from a roleplay kickoff "
"scene description. The user provides bot context and a prose "
"description of the opening scene; you output JSON conforming to the "
"schema. Be concrete: pick a single container, single activity per "
"entity, and a sensible initial in-fiction time. Anything not stated "
"explicitly should be inferred reasonably from the prose."
)
def _build_user_prompt(
*,
bot_name: str,
bot_persona: str,
initial_relationship_to_you: str,
kickoff_prose: str,
you_name: str,
) -> str:
return (
f"BOT NAME: {bot_name}\n"
f"BOT PERSONA: {bot_persona}\n"
f"INITIAL RELATIONSHIP TO {you_name}: {initial_relationship_to_you}\n"
f"YOU NAME: {you_name}\n"
f"KICKOFF PROSE:\n{kickoff_prose}"
)
def _empty_activity() -> ActivityShape:
return ActivityShape(
posture="",
action_verb="",
action_interruptible=True,
action_required_attention="low",
action_expected_duration="brief",
)
def _empty_kickoff_parse() -> KickoffParse:
"""Default returned when the classifier can't produce a valid parse.
The user gets a mostly-empty confirm form they can fill in by hand
instead of a 500. ``initial_time_iso`` is left as the current UTC.
"""
from datetime import datetime, timezone
return KickoffParse(
container_name="",
container_type="",
container_properties={},
you_activity=_empty_activity(),
bot_activity=_empty_activity(),
initial_time_iso=datetime.now(timezone.utc).isoformat(timespec="seconds"),
edge_seed_summary="",
edge_seed_knowledge_facts=[],
)
async def parse_kickoff(
client: LLMClient,
*,
model: str,
bot_name: str,
bot_persona: str,
initial_relationship_to_you: str,
kickoff_prose: str,
you_name: str,
timeout_s: float = 10.0,
) -> KickoffParse:
"""Parse authored kickoff prose into a structured ``KickoffParse``.
Falls back to a mostly-empty default if the classifier fails — the
confirm-and-edit form is the human-in-the-loop, so a degraded form
that the user can fill in is preferable to a 500.
"""
user_prompt = _build_user_prompt(
bot_name=bot_name,
bot_persona=bot_persona,
initial_relationship_to_you=initial_relationship_to_you,
kickoff_prose=kickoff_prose,
you_name=you_name,
)
return await classify(
client,
model=model,
system=_SYSTEM_PROMPT,
user=user_prompt,
schema=KickoffParse,
default=_empty_kickoff_parse(),
timeout_s=timeout_s,
)
+178
View File
@@ -0,0 +1,178 @@
"""Per-turn memory writes (T21).
After ``assistant_turn`` lands, the turn flow records a ``memory_written``
event for each present POV owner. Phase 1 single-bot turns only have the
host bot as a memory-store owner — ``you`` doesn't have a memory store in
v1 — so we write exactly one row per turn.
Phase 1 simplifications (per plan §11.1, T27 will refine):
- ``pov_summary`` is the assistant's raw narrative text. T27 rewrites at
scene close into per-POV summary form.
- ``significance`` defaults to ``1`` (Notable). T22's async significance
pass overwrites via a follow-up event.
- Witness flags are hard-coded ``[you=1, host=1, guest=0]``. Phase 2 will
derive them from ``chat.guest_bot_id`` once a guest can be present.
"""
from __future__ import annotations
from sqlite3 import Connection
from chat.eventlog.log import append_and_apply
def record_turn_memory(
conn: Connection,
*,
chat_id: str,
host_bot_id: str,
narrative_text: str,
scene_id: int | None = None,
chat_clock_at: str | None = None,
source: str = "direct",
significance: int = 1,
) -> tuple[int, int | None]:
"""Append a ``memory_written`` event for the host bot's POV of this turn.
Uses :func:`chat.eventlog.log.append_and_apply` (not raw
:func:`append_event`) so the new memory row is projected immediately
without re-running prior non-idempotent handlers (e.g. ``edge_update``
deltas).
Returns ``(event_id, memory_id)``. ``event_id`` is the row id of the
just-appended ``memory_written`` event in ``event_log``. ``memory_id``
is the autoincrement PK of the corresponding ``memories`` row — these
are *different* numbers (event_log and memories use independent
rowid sequences) so callers needing to update significance or pin
state must use ``memory_id``. Falls back to ``None`` if the projected
row can't be located, which shouldn't happen but keeps the return
shape stable.
"""
payload: dict = {
"owner_id": host_bot_id,
"chat_id": chat_id,
"pov_summary": narrative_text,
"witness_you": 1,
"witness_host": 1,
"witness_guest": 0,
"source": source,
"reliability": 1.0,
"significance": significance,
"pinned": 0,
"auto_pinned": 0,
}
if scene_id is not None:
payload["scene_id"] = scene_id
if chat_clock_at is not None:
payload["chat_clock_at"] = chat_clock_at
event_id = append_and_apply(conn, kind="memory_written", payload=payload)
row = conn.execute(
"SELECT id FROM memories "
"WHERE owner_id = ? AND chat_id = ? "
"ORDER BY id DESC LIMIT 1",
(host_bot_id, chat_id),
).fetchone()
memory_id = row[0] if row else None
return event_id, memory_id
def _write_one_memory(
conn: Connection,
*,
owner_id: str,
chat_id: str,
narrative_text: str,
witness_you: int,
witness_host: int,
witness_guest: int,
scene_id: int | None,
chat_clock_at: str | None,
source: str,
significance: int,
) -> tuple[int, int | None]:
"""Append a single ``memory_written`` event for ``owner_id`` and return
``(event_id, memory_id)`` for the projected row."""
payload: dict = {
"owner_id": owner_id,
"chat_id": chat_id,
"pov_summary": narrative_text,
"witness_you": witness_you,
"witness_host": witness_host,
"witness_guest": witness_guest,
"source": source,
"reliability": 1.0,
"significance": significance,
"pinned": 0,
"auto_pinned": 0,
}
if scene_id is not None:
payload["scene_id"] = scene_id
if chat_clock_at is not None:
payload["chat_clock_at"] = chat_clock_at
event_id = append_and_apply(conn, kind="memory_written", payload=payload)
row = conn.execute(
"SELECT id FROM memories "
"WHERE owner_id = ? AND chat_id = ? "
"ORDER BY id DESC LIMIT 1",
(owner_id, chat_id),
).fetchone()
memory_id = row[0] if row else None
return event_id, memory_id
def record_turn_memory_for_present(
conn: Connection,
*,
chat_id: str,
host_bot_id: str,
guest_bot_id: str | None,
narrative_text: str,
scene_id: int | None = None,
chat_clock_at: str | None = None,
source: str = "direct",
significance: int = 1,
) -> dict[str, tuple[int, int | None]]:
"""Write a ``memory_written`` event for each present bot witness.
Host is always written. Guest is written iff ``guest_bot_id is not
None``. Witness flags are ``[you=1, host=1, guest=1]`` when a guest
is present, ``[you=1, host=1, guest=0]`` otherwise.
Returns a mapping ``{bot_id: (event_id, memory_id)}`` so callers can
look up the freshly-projected memory id per owner without re-querying
the database.
"""
witness_guest = 1 if guest_bot_id is not None else 0
result: dict[str, tuple[int, int | None]] = {}
result[host_bot_id] = _write_one_memory(
conn,
owner_id=host_bot_id,
chat_id=chat_id,
narrative_text=narrative_text,
witness_you=1,
witness_host=1,
witness_guest=witness_guest,
scene_id=scene_id,
chat_clock_at=chat_clock_at,
source=source,
significance=significance,
)
if guest_bot_id is not None:
result[guest_bot_id] = _write_one_memory(
conn,
owner_id=guest_bot_id,
chat_id=chat_id,
narrative_text=narrative_text,
witness_you=1,
witness_host=1,
witness_guest=1,
scene_id=scene_id,
chat_clock_at=chat_clock_at,
source=source,
significance=significance,
)
return result
+62
View File
@@ -0,0 +1,62 @@
"""Multi-entity state-update coordinator (T40).
Wraps single-pair compute_state_update to run state updates for ALL
directed pairs of present entities. With 3 present entities (you, host,
guest) that's 6 directed pairs. With 2 present (you, host) it's 2 pairs.
Calls run sequentially to respect Featherless's 2-connection cap (the
client-level semaphore would serialize them anyway, but doing it here
keeps the failure surface clean — a hung pair doesn't queue behind
itself).
"""
from __future__ import annotations
from chat.llm.client import LLMClient
from chat.services.state_update import StateUpdate, compute_state_update
async def compute_state_updates_for_present(
client: LLMClient,
*,
classifier_model: str,
present_ids: list[str],
present_names: dict[str, str],
personas: dict[str, str],
prior_edges: dict[tuple[str, str], dict],
recent_dialogue: list[dict],
timeout_s: float = 30.0,
) -> list[tuple[str, str, StateUpdate]]:
"""Run compute_state_update for every directed pair (src != tgt) over
``present_ids``. Returns list of ``(source_id, target_id, update)``
tuples in the natural iteration order over ``present_ids x present_ids``.
A single failing pair falls back to the schema-default StateUpdate
(zero deltas, empty facts) inside ``compute_state_update``; the batch
keeps going.
"""
out: list[tuple[str, str, StateUpdate]] = []
for src in present_ids:
for tgt in present_ids:
if src == tgt:
continue
edge = prior_edges.get((src, tgt), {})
update = await compute_state_update(
client,
model=classifier_model,
source_id=src,
target_id=tgt,
source_name=present_names.get(src, src),
source_persona=personas.get(src, "") or "",
target_name=present_names.get(tgt, tgt),
prior_affinity=int(edge.get("affinity", 50)),
prior_trust=int(edge.get("trust", 50)),
prior_summary=edge.get("summary", "") or "",
recent_dialogue=recent_dialogue,
timeout_s=timeout_s,
)
out.append((src, tgt, update))
return out
__all__ = ["compute_state_updates_for_present"]
+735
View File
@@ -0,0 +1,735 @@
"""Narrative-prompt assembly with must/should/nice trim tiers.
Implements Task 18 (Phase 1D). See Requirements §3.2 (token budgets and
trim tiers) and §6.3 (speaker prompt assembly order). The function
:func:`assemble_narrative_prompt` returns a list of
:class:`chat.llm.client.Message` objects ready to feed to
``LLMClient.generate``.
Trim policy when the assembled prompt exceeds the soft target:
- **MUST-include** (never trimmed): system / speaker identity, the
speaker→addressee edge, the activity snapshot for all present
entities, the current scene description, and the last 4 turns of
dialogue.
- **SHOULD-include** (trim when over budget): other edges of the
speaker. (Group nodes, active threads, and active events / props are
Phase 3 — skipped here.)
- **NICE-include** (trim first): retrieved memories beyond top-2,
dialogue turns beyond the last 4 (replaced with a one-line elision
placeholder), per-POV summary of the previous scene.
Token counting uses ``tiktoken.get_encoding("cl100k_base")`` per the
requirements. Mistral / Llama tokenizers diverge ~5%; we accept the
drift.
The function is intentionally deterministic (no LLM call) so it is
testable with synthetic state and so T29's regenerate flow can rebuild
prompts without re-running classifiers.
"""
from __future__ import annotations
from sqlite3 import Connection
import tiktoken
from chat.llm.client import Message
from chat.state.edges import get_edge, list_edges_for
from chat.state.entities import get_bot, get_you
from chat.state.group_node import get_group_node
from chat.state.memory import search_memories
from chat.state.world import (
active_scene,
get_activity,
get_chat,
get_container,
get_scene,
)
# Cache the encoder once at import-time. tiktoken's encoder load is
# non-trivial (~tens of ms) and the encoding is process-wide stable.
_ENCODER = tiktoken.get_encoding("cl100k_base")
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _count_tokens(text: str, encoding=_ENCODER) -> int:
"""Return the cl100k_base token count for ``text`` (0 for falsy)."""
if not text:
return 0
return len(encoding.encode(text))
def _build_speaker_identity(bot: dict) -> str:
"""Render the bot identity block. Skips empty optional fields."""
lines = [f"You are {bot['name']}."]
if bot.get("persona"):
lines.append("")
lines.append("PERSONA:")
lines.append(bot["persona"])
voice_samples = bot.get("voice_samples") or []
if voice_samples:
lines.append("")
lines.append("VOICE REFERENCE:")
lines.append("\n---\n".join(voice_samples))
traits = bot.get("traits") or []
if traits:
lines.append("")
lines.append(f"TRAITS: {', '.join(traits)}")
if bot.get("backstory"):
lines.append("")
lines.append("BACKSTORY:")
lines.append(bot["backstory"])
return "\n".join(lines)
def _build_edge_block(edge: dict | None, addressee_name: str) -> str | None:
"""Render the speaker → addressee edge. Returns None when no edge exists."""
if edge is None:
return None
lines = [f"YOUR EDGE TO {addressee_name}:"]
lines.append(f"- Affinity: {edge.get('affinity', 50)}/100")
lines.append(f"- Trust: {edge.get('trust', 50)}/100")
summary = edge.get("summary") or ""
if summary:
lines.append(f"- Summary: {summary}")
knowledge = edge.get("knowledge") or []
if knowledge:
lines.append(f"- What you know about {addressee_name}:")
for fact in knowledge:
lines.append(f" * {fact}")
return "\n".join(lines)
def _build_activity_block(activities: list[dict]) -> str | None:
"""Render the activity snapshot for all present entities."""
rendered: list[str] = []
for a in activities:
if a is None:
continue
label = a.get("_display_name") or a.get("entity_id", "?")
parts: list[str] = []
posture = a.get("posture") or ""
if posture:
parts.append(posture)
action = a.get("action") or {}
verb = action.get("verb") if isinstance(action, dict) else None
if verb:
parts.append(verb)
attention = a.get("attention") or ""
if attention:
parts.append(f"attention: {attention}")
holding = a.get("holding") or []
if holding:
parts.append(f"holding: {', '.join(holding)}")
if parts:
rendered.append(f"- {label}: " + ", ".join(parts))
else:
rendered.append(f"- {label}: (no activity recorded)")
if not rendered:
return None
return "ACTIVITIES:\n" + "\n".join(rendered)
def _build_scene_block(chat: dict, container: dict | None, scene: dict | None) -> str | None:
"""Render the current-scene block. Always present when chat exists."""
lines = ["CURRENT SCENE:"]
if container is not None:
lines.append(f"- Container: {container['name']} ({container['type']})")
chat_time = chat.get("time") if chat else None
if chat_time:
lines.append(f"- Time: {chat_time}")
if scene is not None and scene.get("started_at"):
lines.append(f"- Active scene started: {scene['started_at']}")
if len(lines) == 1:
return None
return "\n".join(lines)
def _format_dialogue_turn(turn: dict) -> str:
speaker = turn.get("speaker") or "?"
text = turn.get("text") or ""
return f"{speaker}: {text}"
def _build_dialogue_block(
recent: list[dict],
earlier_summary: str | None,
) -> str | None:
"""Render the recent-dialogue block. The ``recent`` list is the
*kept* tail of the dialogue (already trimmed to the last-N turns).
``earlier_summary``, when non-None, is rendered as the first line as
``earlier: <text>`` to flag elided context.
"""
if not recent and not earlier_summary:
return None
lines = ["RECENT DIALOGUE:"]
if earlier_summary:
lines.append(f"earlier: {earlier_summary}")
for turn in recent:
lines.append(_format_dialogue_turn(turn))
return "\n".join(lines)
def _build_memories_block(memory_summaries: list[str]) -> str | None:
if not memory_summaries:
return None
lines = ["RELEVANT MEMORIES:"]
for m in memory_summaries:
lines.append(f"- {m}")
return "\n".join(lines)
def _build_other_edges_block(edges: list[dict]) -> str | None:
"""Render edges to entities other than the addressee."""
if not edges:
return None
lines = ["OTHER EDGES:"]
for e in edges:
target = e.get("_display_name") or e.get("target_id", "?")
affinity = e.get("affinity", 50)
trust = e.get("trust", 50)
lines.append(f"- {target}: affinity {affinity}/100, trust {trust}/100")
summary = e.get("summary") or ""
if summary:
lines.append(f" summary: {summary}")
return "\n".join(lines)
def _build_previous_scene_block(pov_summary: str | None) -> str | None:
if not pov_summary:
return None
return "PREVIOUS SCENE SUMMARY:\n" + pov_summary
def _build_group_node_block(group_node: dict | None) -> str | None:
"""Render the group-node summary + dynamic as a SHOULD-tier block.
Used only in 3-entity scenes (you + host + guest). Returns None when
the row is missing or both summary and dynamic are empty.
"""
if not group_node:
return None
summary = (group_node.get("summary") or "").strip()
dynamic = (group_node.get("dynamic") or "").strip()
if not summary and not dynamic:
return None
lines = ["Group dynamic:"]
if summary:
lines.append(f"- Summary: {summary}")
if dynamic:
lines.append(f"- Dynamic: {dynamic}")
return "\n".join(lines)
def _closing_instruction(speaker_name: str, addressee_name: str) -> str:
return (
f"Continue the scene as {speaker_name}, in their voice, responding "
"naturally. Use *asterisks* for actions and quotes for dialogue. "
f"Stay in character. Do not narrate {addressee_name}'s actions or "
"thoughts. "
"Keep your response to a single beat — one or two short paragraphs "
"at most. Don't monologue; leave room for the other person to react."
)
def _join_blocks(blocks: list[str | None]) -> str:
"""Join non-empty blocks with double newlines."""
return "\n\n".join(b for b in blocks if b)
def _earlier_summary_placeholder(elided_count: int) -> str:
"""Phase 1 placeholder. Real summarization is a downstream concern."""
plural = "turn" if elided_count == 1 else "turns"
return f"{elided_count} earlier {plural} elided for brevity"
def _resolve_previous_scene_summary(
conn: Connection, chat_id: str, speaker_bot_id: str
) -> str | None:
"""Return ``pov_summary`` of the most recent ended scene, owned by
the speaker. None if no closed scene exists or no matching memory.
"""
row = conn.execute(
"SELECT id FROM scenes WHERE chat_id = ? AND ended_at IS NOT NULL "
"ORDER BY ended_at DESC LIMIT 1",
(chat_id,),
).fetchone()
if not row:
return None
scene_id = row[0]
mem = conn.execute(
"SELECT pov_summary FROM memories WHERE scene_id = ? AND owner_id = ? "
"ORDER BY id DESC LIMIT 1",
(scene_id, speaker_bot_id),
).fetchone()
if not mem:
return None
return mem[0]
def _witness_role_for(speaker_bot_id: str, host_bot_id: str | None) -> str:
"""Return the witness POV role for the speaker's memory query.
The host bot of a chat queries memories with ``witness_role="host"``;
the guest bot queries with ``witness_role="guest"``. Phase 2 T46
pinned the contract on ``search_memories``; this helper applies it
at the call site so a guest-as-speaker doesn't silently retrieve
memories under the wrong POV mask.
"""
return "host" if speaker_bot_id == host_bot_id else "guest"
def _resolve_addressee(
conn: Connection, addressee: str, you: dict | None
) -> tuple[str, str]:
"""Return ``(addressee_id, addressee_display_name)``.
The function is permissive: ``addressee="you"`` resolves to the
you-entity (display name is its authored name, falling back to
"you" if no entity exists yet). Other ids resolve as bot ids.
"""
if addressee == "you":
name = (you or {}).get("name") or "you"
return "you", name
bot = get_bot(conn, addressee)
if bot is not None:
return addressee, bot["name"]
return addressee, addressee
# ---------------------------------------------------------------------------
# Public API
# ---------------------------------------------------------------------------
def assemble_narrative_prompt(
conn: Connection,
*,
chat_id: str,
speaker_bot_id: str,
addressee: str = "you",
user_turn_prose: str | None = None,
recent_dialogue: list[dict] | None = None,
retrieved_memory_summaries: list[str] | None = None,
budget_soft: int = 6000,
budget_hard: int = 8000,
encoding_name: str = "cl100k_base",
guest_id: str | None = None,
) -> list[Message]:
"""Assemble the narrative prompt for ``speaker_bot_id`` to respond.
Returns a list of :class:`Message` objects: one ``system`` message
carrying the assembled context, optionally followed by a single
``user`` message containing ``user_turn_prose`` (when provided).
Trimming proceeds in tiers (NICE → SHOULD) once the total token
count exceeds ``budget_soft``; the function refuses to exceed
``budget_hard``. If the MUST-include block alone is already over
``budget_hard``, :class:`ValueError` is raised — the caller should
surface the failure rather than ship a malformed prompt.
"""
encoding = (
_ENCODER if encoding_name == "cl100k_base"
else tiktoken.get_encoding(encoding_name)
)
bot = get_bot(conn, speaker_bot_id)
if bot is None:
raise ValueError(f"speaker_bot_id {speaker_bot_id!r} not found")
chat = get_chat(conn, chat_id)
if chat is None:
raise ValueError(f"chat_id {chat_id!r} not found")
# Auto-detect guest from chat state when caller didn't pass one.
# Phase 1 chats have ``guest_bot_id is None``; the auto-detect is a
# no-op there and the function behaves exactly as before.
if guest_id is None:
guest_id = chat.get("guest_bot_id")
# A speaker addressing themself as guest doesn't add a third party.
if guest_id is not None and guest_id == speaker_bot_id:
guest_id = None
you = get_you(conn)
addressee_id, addressee_name = _resolve_addressee(conn, addressee, you)
# ---- Build all components as text strings ------------------------------
speaker_identity = _build_speaker_identity(bot)
edge_to_addressee = _build_edge_block(
get_edge(conn, speaker_bot_id, addressee_id),
addressee_name,
)
# Activity for present entities — single ACTIVITIES: block with up
# to three bullets (you, speaker, guest). The block itself is
# MUST-tier and survives all trims, but bullet-level trim drops
# bullets in the order guest -> you, keeping the speaker bullet
# (the speaker's own current activity is the load-bearing slice).
#
# T71.2 chose Option B from the polish plan: pre-truncate the
# bullets list at trim time before _build_activity_block runs,
# rather than introducing a granular tier mode in the trim
# machinery. The single-block render avoids the dual-ACTIVITIES:
# header that Phase 2 T43 introduced (read by some LLMs as a
# duplicate-section bug).
you_activity: dict | None = None
you_act = get_activity(conn, "you")
if you_act is not None:
you_activity = dict(you_act)
you_activity["_display_name"] = (you or {}).get("name") or "you"
speaker_activity: dict | None = None
bot_act = get_activity(conn, speaker_bot_id)
if bot_act is not None:
speaker_activity = dict(bot_act)
speaker_activity["_display_name"] = bot["name"]
guest_activity: dict | None = None
if guest_id is not None:
guest_act = get_activity(conn, guest_id)
if guest_act is not None:
guest_activity = dict(guest_act)
guest_bot = get_bot(conn, guest_id)
guest_activity["_display_name"] = (
guest_bot["name"] if guest_bot else guest_id
)
def _activity_block_for(
*, include_you: bool, include_guest: bool
) -> str | None:
"""Render the single ACTIVITIES: block with the requested bullets.
Speaker bullet is always included (it's the MUST-tier baseline);
``you`` and ``guest`` bullets are toggled by the caller during
trim. Returns None when no bullets remain.
"""
bullets: list[dict] = []
if include_you and you_activity is not None:
bullets.append(you_activity)
if speaker_activity is not None:
bullets.append(speaker_activity)
if include_guest and guest_activity is not None:
bullets.append(guest_activity)
return _build_activity_block(bullets)
# SHOULD-tier group-node block (Phase 2 / Task 43): rendered only
# when the group_node row is present AND it covers all three of
# you + host + guest (per the Task 43 spec).
group_node_block: str | None = None
if guest_id is not None:
gn = get_group_node(conn, chat_id)
if gn is not None:
members = set(gn.get("members") or [])
host_id = chat.get("host_bot_id")
required = {"you"}
if host_id is not None:
required.add(host_id)
required.add(guest_id)
if required.issubset(members):
group_node_block = _build_group_node_block(gn)
container = None
if chat.get("active_scene_id"):
scene = get_scene(conn, chat["active_scene_id"])
if scene and scene.get("container_id"):
container = get_container(conn, scene["container_id"])
else:
scene = active_scene(conn, chat_id)
if container is None and scene and scene.get("container_id"):
container = get_container(conn, scene["container_id"])
scene_block = _build_scene_block(chat, container, scene)
# Other edges: speaker → non-addressee.
all_outgoing = list_edges_for(conn, speaker_bot_id)
other_edges_raw = [e for e in all_outgoing if e.get("target_id") != addressee_id]
for e in other_edges_raw:
tid = e.get("target_id")
if tid == "you":
e["_display_name"] = (you or {}).get("name") or "you"
else:
tb = get_bot(conn, tid) if tid else None
e["_display_name"] = tb["name"] if tb else (tid or "?")
other_edges_block = _build_other_edges_block(other_edges_raw)
# Memories: caller override wins; otherwise FTS5 search keyed on the
# scene's container/posture as a coarse query proxy.
if retrieved_memory_summaries is not None:
memory_summaries = list(retrieved_memory_summaries)
else:
query = (container or {}).get("name") or chat.get("narrative_anchor") or ""
memory_summaries = []
if query:
try:
witness_role = _witness_role_for(
speaker_bot_id, chat.get("host_bot_id")
)
hits = search_memories(
conn, speaker_bot_id, witness_role, query, k=4
)
memory_summaries = [h["pov_summary"] for h in hits]
except Exception:
memory_summaries = []
# Dialogue: caller override only (no event_log read in Phase 1).
dialogue_full = list(recent_dialogue or [])
previous_scene_summary = _resolve_previous_scene_summary(
conn, chat_id, speaker_bot_id
)
closing = _closing_instruction(bot["name"], addressee_name)
# ---- Build the MUST core ----------------------------------------------
last4 = dialogue_full[-4:] if dialogue_full else []
must_dialogue_block = _build_dialogue_block(last4, earlier_summary=None)
# MUST-tier ACTIVITIES floor: the speaker bullet alone (you and
# guest bullets are dropped first under bullet-level trim before
# the block bottoms out at speaker-only).
must_activity_block = _activity_block_for(
include_you=False, include_guest=False
)
must_blocks: list[str | None] = [
speaker_identity,
edge_to_addressee,
scene_block,
must_activity_block,
must_dialogue_block,
closing,
]
must_text = _join_blocks(must_blocks)
must_tokens = _count_tokens(must_text, encoding)
if must_tokens > budget_hard:
raise ValueError(
f"MUST-include block ({must_tokens} tokens) exceeds budget_hard "
f"({budget_hard}). Cannot assemble prompt."
)
# ---- Stage SHOULD additions, then NICE additions -----------------------
# We carry a running "components" list and rebuild the body as we go
# so token accounting reflects join-overhead. Order in the final
# prompt follows §6.3: identity → edge → other edges → scene →
# activities → previous scene summary → memories → dialogue → close.
def assemble(
*,
include_other_edges: bool,
include_previous_scene: bool,
include_memories_top_k: int,
dialogue_keep: int,
include_you_activity: bool = True,
include_guest_activity: bool = True,
include_group_node: bool = True,
) -> tuple[str, int, list[dict]]:
# dialogue: keep the last `dialogue_keep` turns verbatim; older
# turns become an "earlier:" placeholder line.
kept_dialogue = (
dialogue_full[-dialogue_keep:] if dialogue_keep > 0 else []
)
elided = max(0, len(dialogue_full) - len(kept_dialogue))
earlier_summary = (
_earlier_summary_placeholder(elided) if elided > 0 else None
)
dialogue_block = _build_dialogue_block(kept_dialogue, earlier_summary)
memories_subset = memory_summaries[:include_memories_top_k]
memories_block = _build_memories_block(memories_subset)
prev_block = (
_build_previous_scene_block(previous_scene_summary)
if include_previous_scene else None
)
# Single ACTIVITIES: block, bullet-level trim (T71.2). Guest
# bullet drops first, then the you bullet; speaker bullet is the
# MUST-tier floor and always present when an activity row exists.
activity_block = _activity_block_for(
include_you=include_you_activity,
include_guest=include_guest_activity,
)
body = _join_blocks([
speaker_identity,
edge_to_addressee,
other_edges_block if include_other_edges else None,
scene_block,
activity_block,
group_node_block if include_group_node else None,
prev_block,
memories_block,
dialogue_block,
closing,
])
return body, _count_tokens(body, encoding), kept_dialogue
# Start with the MUST baseline: last 4 turns of dialogue, no
# SHOULD/NICE extras.
baseline_keep = min(4, len(dialogue_full))
# Try the most generous configuration first; trim greedily.
nice_dialogue_keep = len(dialogue_full) # all turns, no elision
nice_memories_k = min(4, len(memory_summaries))
include_prev = previous_scene_summary is not None
include_other = other_edges_block is not None
include_you_activity = you_activity is not None
include_guest_activity = guest_activity is not None
include_group_node = group_node_block is not None
def _build(*, prev: bool, mem_k: int, dlg: int, other: bool,
you_act: bool, guest_act: bool, group: bool) -> tuple[str, int]:
body, total, _ = assemble(
include_other_edges=other,
include_previous_scene=prev,
include_memories_top_k=mem_k,
dialogue_keep=dlg,
include_you_activity=you_act,
include_guest_activity=guest_act,
include_group_node=group,
)
return body, total
body, total = _build(
prev=include_prev, mem_k=nice_memories_k, dlg=nice_dialogue_keep,
other=include_other, you_act=include_you_activity,
guest_act=include_guest_activity, group=include_group_node,
)
# If under soft, we're done.
if total <= budget_soft:
return _emit(body, user_turn_prose)
# Drop NICE in order: previous scene → memories beyond top-2 →
# older dialogue turns (collapse to 4).
#
# T71.3 — order rationale: the §6.3 spec lists NICE-tier members
# with previous-scene LAST, which read as a literal trim order
# during T18 review. We deliberately keep the greedy order shown
# here (previous-scene FIRST) for two reasons:
#
# 1. Cheapest-impact-first: a per-POV previous-scene summary is
# a single short paragraph that loses very little narrative
# continuity when dropped, while the older dialogue turns it
# is competing with carry the speaker's last few beats — those
# ground the next response far more concretely.
# 2. Greedy lookahead is more expensive than the marginal
# narrative loss. Dropping previous-scene typically clears
# the soft-budget slack in one step; trying memories or
# dialogue first would routinely require multiple recompute
# passes through the assembler.
#
# The pin test test_nice_trim_order_documented locks this order so
# a future refactor can't quietly invert it without surfacing the
# decision.
if include_prev:
include_prev = False
body, total = _build(
prev=include_prev, mem_k=nice_memories_k, dlg=nice_dialogue_keep,
other=include_other, you_act=include_you_activity,
guest_act=include_guest_activity, group=include_group_node,
)
if total <= budget_soft:
return _emit(body, user_turn_prose)
if nice_memories_k > 2:
nice_memories_k = 2
body, total = _build(
prev=include_prev, mem_k=nice_memories_k, dlg=nice_dialogue_keep,
other=include_other, you_act=include_you_activity,
guest_act=include_guest_activity, group=include_group_node,
)
if total <= budget_soft:
return _emit(body, user_turn_prose)
if nice_dialogue_keep > baseline_keep:
nice_dialogue_keep = baseline_keep
body, total = _build(
prev=include_prev, mem_k=nice_memories_k, dlg=nice_dialogue_keep,
other=include_other, you_act=include_you_activity,
guest_act=include_guest_activity, group=include_group_node,
)
if total <= budget_soft:
return _emit(body, user_turn_prose)
# Drop more NICE until we're under hard: memories all the way to 0.
while nice_memories_k > 0 and total > budget_hard:
nice_memories_k = max(0, nice_memories_k - 1)
body, total = _build(
prev=include_prev, mem_k=nice_memories_k, dlg=nice_dialogue_keep,
other=include_other, you_act=include_you_activity,
guest_act=include_guest_activity, group=include_group_node,
)
# Drop SHOULD-tier extras in order:
# 1. guest activity bullet (T71.2: bullet-level trim within the
# single ACTIVITIES: block — guest goes first per Task 43 spec)
# 2. group node block
# 3. you activity bullet (still SHOULD-tier; speaker bullet is the
# MUST-tier floor and never dropped)
# 4. other edges
if include_guest_activity and total > budget_hard:
include_guest_activity = False
body, total = _build(
prev=include_prev, mem_k=nice_memories_k, dlg=nice_dialogue_keep,
other=include_other, you_act=include_you_activity,
guest_act=include_guest_activity, group=include_group_node,
)
if include_group_node and total > budget_hard:
include_group_node = False
body, total = _build(
prev=include_prev, mem_k=nice_memories_k, dlg=nice_dialogue_keep,
other=include_other, you_act=include_you_activity,
guest_act=include_guest_activity, group=include_group_node,
)
if include_you_activity and total > budget_hard:
include_you_activity = False
body, total = _build(
prev=include_prev, mem_k=nice_memories_k, dlg=nice_dialogue_keep,
other=include_other, you_act=include_you_activity,
guest_act=include_guest_activity, group=include_group_node,
)
if include_other and total > budget_hard:
include_other = False
body, total = _build(
prev=include_prev, mem_k=nice_memories_k, dlg=nice_dialogue_keep,
other=include_other, you_act=include_you_activity,
guest_act=include_guest_activity, group=include_group_node,
)
if total > budget_hard:
# We've stripped everything optional and we still overflow.
# MUST alone fits (we checked at the top), so this means our
# last-4 dialogue + must blocks together exceed hard. Fall back
# to the bare MUST core.
body = must_text
total = must_tokens
if total > budget_hard:
raise ValueError(
f"Prompt cannot fit budget_hard={budget_hard}; MUST core "
f"is {total} tokens"
)
return _emit(body, user_turn_prose)
def _emit(system_body: str, user_turn_prose: str | None) -> list[Message]:
msgs: list[Message] = [Message(role="system", content=system_body)]
if user_turn_prose is not None:
msgs.append(Message(role="user", content=user_turn_prose))
return msgs
__all__ = ["assemble_narrative_prompt"]
+623
View File
@@ -0,0 +1,623 @@
"""Regenerate flow (T29).
The user clicks "Regenerate" on the latest ``assistant_turn``. The UI
puts the prior ``user_turn`` into inline edit mode and submits to
:func:`regenerate_assistant_turn` either:
- with **no edit** — we re-run the narrative against the original user
prose and append a fresh ``assistant_turn`` superseding the old one;
- with **edited prose** — we additionally append a ``user_turn_edit``
event capturing the new prose, mark the original ``user_turn`` as
superseded by the edit, then run the narrative against the edited
prose.
Per Requirements §10.2 superseded events are *kept in the log* — the
display layer hides them. This is what makes rewinding to before a
regenerate cheap: we just clear ``superseded_by`` on the old row.
The supersede update is one of the rare "direct DB write" exceptions
documented in the plan: we manipulate metadata fields on the canonical
event_log row itself rather than projecting through a handler.
Phase 1 simplifications (per the plan's "bound it" guidance):
- Significance pass is *not* re-run on regenerate. The original score
remains attached to the prior memory. The state-update pass *is* re-run
so affinity/trust/knowledge reflect the new output.
- The route does not broadcast a fresh ``turn_html`` SSE event; T34
polishes UI swaps. The user refreshes the page to see the new turn.
*(T73.1 closed this gap — see Phase 2.5 changes below.)*
Phase 2 changes (T44):
- Multi-entity prompt assembly: ``guest_id`` is forwarded to the
prompt assembler so the regenerated narrative sees the same
guest-aware context the original turn did.
- Multi-witness memory write: ``record_turn_memory_for_present`` fans
out one ``memory_written`` event per witness when a guest is present.
- Multi-pair state-update: ``compute_state_updates_for_present`` emits
one ``edge_update`` per directed pair across present entities. With
three present that's six edges instead of two.
- Interjection regeneration is **deferred to Phase 2.5**. Regenerate
only re-streams the addressee turn for v2; ``detect_interjection``
is not invoked here. If the prior turn fired an interjection it
remains attached to the original assistant_turn (which is superseded
alongside the regenerated turn) — Phase 2.5 will revisit.
Phase 2.5 changes:
- T73.1: After the new ``assistant_turn`` lands we publish a
``turn_html_replace`` SSE event carrying the rendered HTML for the
regenerated turn plus the original assistant_turn's event_id as
``supersedes_id`` so connected tabs can swap the prior DOM node
in-place. We use a NEW event name (rather than re-using ``turn_html``)
because the existing HTMX ``sse-swap="turn_html"`` consumer expects a
raw-HTML body and an *append* semantic; ``turn_html_replace`` is a
JSON payload (sse.py auto-serialises when extra keys accompany
``data``) so the front-end JS can read ``supersedes_id`` and replace
the right node.
- T73.2: Interjection regeneration. When the original assistant_turn
group included an interjection beat we redo BOTH the primary and the
interjection — re-running ``detect_interjection`` against the new
primary text. If the classifier returns False this time we supersede
the original interjection without appending a replacement.
- T73.3: The defensive degrade-to-1:1 for stale ``guest_bot_id``
references was removed — Phase 2 T47 fixed the root cause (resets
clear the reference) so the guard is dead code.
"""
from __future__ import annotations
import json
from sqlite3 import Connection
from chat.config import Settings
from chat.eventlog.log import append_and_apply, append_event
from chat.services.interjection import detect_interjection
from chat.services.memory_write import record_turn_memory_for_present
from chat.services.multi_state_update import compute_state_updates_for_present
from chat.services.prompt import assemble_narrative_prompt
from chat.state.edges import get_edge
from chat.state.entities import get_bot, get_you
from chat.state.world import active_scene, get_chat
from chat.web.pubsub import publish
from chat.web.render import render_turn_html
async def regenerate_assistant_turn(
conn: Connection,
client,
*,
settings: Settings,
chat_id: str,
original_assistant_event_id: int,
edited_user_prose: str | None = None,
) -> str:
"""Regenerate the assistant turn linked to ``original_assistant_event_id``.
When ``edited_user_prose`` is provided the original user_turn is also
superseded by a fresh ``user_turn_edit`` event capturing the new
prose. Returns the new assistant text.
Raises :class:`ValueError` when the chat or the assistant_turn event
cannot be found — the FastAPI route translates this to 404.
"""
chat = get_chat(conn, chat_id)
if chat is None:
raise ValueError("chat not found")
host_bot_id = chat["host_bot_id"]
host_bot = get_bot(conn, host_bot_id) or {
"id": host_bot_id,
"name": "bot",
"persona": "",
}
# Phase 2: surface the guest (if any) so the prompt assembler and
# downstream multi-entity passes see the same shape post_turn does.
# Phase 2 T47 made bot_reset cascade-clear ``chat.guest_bot_id`` when
# the referenced bot is purged (verified by tests/test_reset.py), so
# we trust the column here: it's either a valid bot id or NULL.
guest_bot_id = chat.get("guest_bot_id")
guest_bot: dict | None = (
get_bot(conn, guest_bot_id) if guest_bot_id is not None else None
)
# 1. Locate the original assistant_turn event.
row = conn.execute(
"SELECT payload_json FROM event_log "
"WHERE id = ? AND kind = 'assistant_turn'",
(original_assistant_event_id,),
).fetchone()
if row is None:
raise ValueError("assistant_turn event not found")
original_assistant_payload = json.loads(row[0])
original_user_turn_id = original_assistant_payload.get("user_turn_id")
# 1a. Look up any sibling interjection beat in the same turn group
# (T73.2). The original group is (primary + optional interjection),
# both pinned to the same ``user_turn_id``. The interjection has a
# populated ``interjection_of`` field in its payload — its speaker is
# the silent witness (the bot that wasn't the primary addressee).
# Filter on ``superseded_by IS NULL`` so prior regenerates of this
# group don't reappear as siblings.
original_interjection_event_id: int | None = None
original_interjection_payload: dict | None = None
if original_user_turn_id is not None:
sibling_cur = conn.execute(
"SELECT id, payload_json FROM event_log "
"WHERE kind = 'assistant_turn' "
" AND id != ? "
" AND superseded_by IS NULL",
(original_assistant_event_id,),
)
for sib_id, sib_payload_json in sibling_cur.fetchall():
sib_payload = json.loads(sib_payload_json)
if sib_payload.get("user_turn_id") != original_user_turn_id:
continue
if not sib_payload.get("interjection_of"):
continue
original_interjection_event_id = sib_id
original_interjection_payload = sib_payload
break
# Phase 2 v2 regenerates only the addressee turn — preserve whichever
# bot the original turn was attributed to, falling back to the host
# for legacy rows that pre-date multi-entity support.
speaker_bot_id = original_assistant_payload.get("speaker_id") or host_bot_id
if speaker_bot_id == host_bot_id:
speaker_bot = host_bot
elif guest_bot is not None and speaker_bot_id == guest_bot.get("id"):
speaker_bot = guest_bot
else:
speaker_bot = get_bot(conn, speaker_bot_id) or host_bot
speaker_bot_id = speaker_bot.get("id", host_bot_id)
# 2. Determine the prose for the new prompt and (when edited) capture
# the user_turn_edit event up front so the new event ids exist before
# we link them from the assistant_turn payload.
new_user_event_id: int | None = None
if edited_user_prose is not None:
new_user_event_id = append_event(
conn,
kind="user_turn_edit",
payload={
"chat_id": chat_id,
"prose": edited_user_prose,
"supersedes_user_turn_id": original_user_turn_id,
},
)
if original_user_turn_id is not None:
conn.execute(
"UPDATE event_log SET superseded_by = ? WHERE id = ?",
(new_user_event_id, original_user_turn_id),
)
prose_for_prompt = edited_user_prose
else:
original_user_row = conn.execute(
"SELECT payload_json FROM event_log WHERE id = ?",
(original_user_turn_id,),
).fetchone() if original_user_turn_id is not None else None
if original_user_row is not None:
prose_for_prompt = json.loads(original_user_row[0]).get("prose", "")
else:
prose_for_prompt = ""
# 3. Build the recent-dialogue slice. Exclude the original
# assistant_turn explicitly (we haven't superseded it yet — that
# update lands at the end so the new event_id is known) and use the
# standard ``superseded_by IS NULL AND hidden = 0`` filter so any
# prior regenerates also drop out.
you_entity = get_you(conn) or {"name": "you", "persona": ""}
you_name = you_entity.get("name", "you")
cur = conn.execute(
"SELECT id, kind, payload_json FROM event_log "
"WHERE kind IN ('user_turn', 'user_turn_edit', 'assistant_turn') "
" AND id != ? "
" AND superseded_by IS NULL AND hidden = 0 "
"ORDER BY id DESC LIMIT 20",
(original_assistant_event_id,),
)
rows = list(reversed(cur.fetchall()))
recent: list[dict] = []
for _eid, kind, payload_json in rows:
p = json.loads(payload_json)
if p.get("chat_id") != chat_id:
continue
if kind in ("user_turn", "user_turn_edit"):
recent.append({"speaker": you_name, "text": p.get("prose", "")})
else:
spk = p.get("speaker_id", "bot")
spk_name = host_bot.get("name", "bot")
if spk == host_bot_id:
spk_name = host_bot.get("name", "bot")
elif guest_bot is not None and spk == guest_bot.get("id"):
spk_name = guest_bot.get("name", "bot")
recent.append({"speaker": spk_name, "text": p.get("text", "")})
# 4. Assemble the narrative prompt. ``recent`` already excludes the
# current user prose, which we pass through ``user_turn_prose``.
# Phase 2: forward ``guest_id`` so the prompt sees the third party.
messages = assemble_narrative_prompt(
conn,
chat_id=chat_id,
speaker_bot_id=speaker_bot_id,
user_turn_prose=prose_for_prompt or None,
recent_dialogue=recent,
budget_soft=settings.narrative_budget_soft,
budget_hard=settings.narrative_budget_hard,
guest_id=guest_bot_id,
)
# 5. Stream the new narrative.
accumulated: list[str] = []
async for chunk in client.stream(
messages,
model=settings.narrative_model,
max_tokens=settings.narrative_max_tokens,
temperature=settings.narrative_temperature,
):
accumulated.append(chunk)
await publish(
chat_id,
{"event": "token", "text": chunk, "speaker_id": speaker_bot_id},
)
new_text = "".join(accumulated)
# 6. Append the new assistant_turn event. ``user_turn_id`` points at
# the edit event when one was created, otherwise the original. The
# ``regenerated_from`` field is the back-pointer the UI uses for an
# "originally said …" affordance.
new_assistant_event_id = append_event(
conn,
kind="assistant_turn",
payload={
"chat_id": chat_id,
"speaker_id": speaker_bot_id,
"text": new_text,
"truncated": False,
"user_turn_id": (
new_user_event_id
if new_user_event_id is not None
else original_user_turn_id
),
"regenerated_from": original_assistant_event_id,
},
)
# 7. Mark the original assistant_turn as superseded by the new one.
conn.execute(
"UPDATE event_log SET superseded_by = ? WHERE id = ?",
(new_assistant_event_id, original_assistant_event_id),
)
# 7a. Broadcast a turn_html_replace SSE event so connected tabs can
# swap the prior assistant_turn DOM node in-place (T73.1, Phase 1.5
# backlog #2). Uses a separate event name from post_turn's
# ``turn_html`` (which is append-only) because regenerate is a
# *replace* operation — see module docstring for the rationale.
speaker_name_for_render = (
speaker_bot.get("name", "bot") if speaker_bot is not None else "bot"
)
new_turn_html = render_turn_html(
speaker_name_for_render, new_text, role="bot"
)
await publish(
chat_id,
{
"event": "turn_html_replace",
"data": new_turn_html,
"turn_id": new_assistant_event_id,
"supersedes_id": original_assistant_event_id,
},
)
# 8. Re-run downstream classifier passes (memory write + state update
# for every directed pair across present entities). Significance is
# intentionally skipped on regenerate (the prior score remains
# attached to the prior memory). Phase 2.5 will add interjection
# regeneration; v2 leaves any prior interjection beat in place.
scene = active_scene(conn, chat_id)
record_turn_memory_for_present(
conn,
chat_id=chat_id,
host_bot_id=host_bot_id,
guest_bot_id=guest_bot_id,
narrative_text=new_text,
scene_id=scene["id"] if scene else None,
chat_clock_at=chat.get("time"),
)
last_at = chat.get("time")
speaker_name = (
speaker_bot.get("name", "bot") if speaker_bot is not None else "bot"
)
recent_for_update = recent + [
{"speaker": speaker_name, "text": new_text}
]
# Build present-entity inputs for the multi-pair state-update pass.
# Host first preserves the Phase 1 directed-pair order (host->you,
# then you->host) so existing canned-response fixtures still line up.
present_ids: list[str] = [host_bot_id, "you"]
present_names: dict[str, str] = {
host_bot_id: host_bot.get("name", "bot"),
"you": you_name,
}
personas: dict[str, str] = {
host_bot_id: host_bot.get("persona") or "",
"you": you_entity.get("persona") or "",
}
if guest_bot is not None and guest_bot_id is not None:
present_ids.append(guest_bot_id)
present_names[guest_bot_id] = guest_bot.get("name", "bot")
personas[guest_bot_id] = guest_bot.get("persona") or ""
prior_edges: dict[tuple[str, str], dict] = {}
for src in present_ids:
for tgt in present_ids:
if src == tgt:
continue
edge = get_edge(conn, src, tgt) or {
"affinity": 50,
"trust": 50,
"summary": "",
}
prior_edges[(src, tgt)] = edge
state_updates = await compute_state_updates_for_present(
client,
classifier_model=settings.classifier_model,
present_ids=present_ids,
present_names=present_names,
personas=personas,
prior_edges=prior_edges,
recent_dialogue=recent_for_update,
timeout_s=settings.classifier_timeout_s,
)
for src_id, tgt_id, update in state_updates:
append_and_apply(
conn,
kind="edge_update",
payload={
"source_id": src_id,
"target_id": tgt_id,
"chat_id": chat_id,
"affinity_delta": update.affinity_delta,
"trust_delta": update.trust_delta,
"knowledge_facts": update.knowledge_facts,
"last_interaction_at": last_at,
"last_interaction_chat_id": chat_id,
},
)
# 9. Interjection regenerate branch (T73.2). When the original
# assistant_turn group included a follow-on interjection beat we need
# to revisit that beat against the regenerated primary. Three outcomes:
#
# - No original interjection: nothing to do; we already short-circuit
# above by leaving ``original_interjection_event_id`` as None.
# - Original interjection + classifier returns True: stream a fresh
# interjection from the silent witness, append it (with
# ``interjection_of`` linking to the new primary speaker), and
# supersede the original interjection's row. Also re-run memory
# + state-update so the second beat moves edges + writes memories.
# - Original interjection + classifier returns False: supersede the
# original interjection without appending a replacement. The
# regenerated group becomes "primary only" because the new primary
# no longer warrants a follow-on. No memory / state work needed
# for the absent beat.
#
# ``superseded_by`` on the original interjection's row points at the
# *new primary* in the no-replacement case (rather than NULL or a
# nonexistent id) so the row is consistently hidden by the standard
# ``superseded_by IS NULL`` timeline filter and the back-pointer
# leads somewhere meaningful for an "originally said …" affordance.
if original_interjection_event_id is not None and guest_bot is not None:
# Identify the silent witness from the original interjection's
# speaker_id (which is the bot that interjected last time). When
# we regenerate we keep the *same pair of present entities*, so
# the silent witness is whichever bot isn't the new primary
# speaker — derive it from present rather than reusing the prior
# speaker_id verbatim, in case the regenerated primary swapped
# who held the floor.
if speaker_bot_id == host_bot_id:
silent_witness = guest_bot
else:
silent_witness = host_bot
silent_witness_id = silent_witness.get("id")
edge_w_to_addr = get_edge(conn, silent_witness_id, speaker_bot_id) or {
"affinity": 50,
"trust": 50,
"summary": "",
}
edge_w_to_you = get_edge(conn, silent_witness_id, "you") or {
"affinity": 50,
"trust": 50,
"summary": "",
}
decision = await detect_interjection(
client,
classifier_model=settings.classifier_model,
addressee_name=speaker_bot.get("name", "bot"),
addressee_just_said=new_text,
silent_witness_name=silent_witness.get("name", "bot"),
silent_witness_persona=silent_witness.get("persona") or "",
silent_witness_edge_to_addressee=edge_w_to_addr,
silent_witness_edge_to_you=edge_w_to_you,
you_just_said=prose_for_prompt or "",
timeout_s=settings.classifier_timeout_s,
)
if decision.should_interject:
# Re-read recent so the just-appended primary is in the prompt.
interject_cur = conn.execute(
"SELECT id, kind, payload_json FROM event_log "
"WHERE kind IN ('user_turn', 'user_turn_edit', 'assistant_turn') "
" AND superseded_by IS NULL AND hidden = 0 "
"ORDER BY id DESC LIMIT 20",
)
interject_rows = list(reversed(interject_cur.fetchall()))
interject_recent: list[dict] = []
for _eid, kind, payload_json in interject_rows:
p = json.loads(payload_json)
if p.get("chat_id") != chat_id:
continue
if kind in ("user_turn", "user_turn_edit"):
interject_recent.append(
{"speaker": you_name, "text": p.get("prose", "")}
)
else:
spk = p.get("speaker_id", "bot")
if spk == host_bot_id:
spk_name = host_bot.get("name", "bot")
elif spk == guest_bot.get("id"):
spk_name = guest_bot.get("name", "bot")
else:
spk_name = "bot"
interject_recent.append(
{"speaker": spk_name, "text": p.get("text", "")}
)
if interject_recent and interject_recent[-1].get("speaker") == you_name:
interject_recent = interject_recent[:-1]
interject_messages = assemble_narrative_prompt(
conn,
chat_id=chat_id,
speaker_bot_id=silent_witness_id,
addressee=speaker_bot_id,
user_turn_prose=prose_for_prompt or None,
recent_dialogue=interject_recent,
budget_soft=settings.narrative_budget_soft,
budget_hard=settings.narrative_budget_hard,
guest_id=guest_bot_id,
)
interject_accumulated: list[str] = []
async for chunk in client.stream(
interject_messages,
model=settings.narrative_model,
max_tokens=settings.narrative_max_tokens,
temperature=settings.narrative_temperature,
):
interject_accumulated.append(chunk)
await publish(
chat_id,
{
"event": "token",
"text": chunk,
"speaker_id": silent_witness_id,
},
)
interject_text = "".join(interject_accumulated)
new_interjection_event_id = append_event(
conn,
kind="assistant_turn",
payload={
"chat_id": chat_id,
"speaker_id": silent_witness_id,
"text": interject_text,
"truncated": False,
"user_turn_id": (
new_user_event_id
if new_user_event_id is not None
else original_user_turn_id
),
"regenerated_from": original_interjection_event_id,
"interjection_of": speaker_bot_id,
},
)
# Supersede the original interjection by the new one.
conn.execute(
"UPDATE event_log SET superseded_by = ? WHERE id = ?",
(new_interjection_event_id, original_interjection_event_id),
)
# Broadcast a replace event so connected tabs swap the prior
# interjection node in-place (mirrors T73.1's primary swap).
interject_html = render_turn_html(
silent_witness.get("name", "bot"), interject_text, role="bot"
)
await publish(
chat_id,
{
"event": "turn_html_replace",
"data": interject_html,
"turn_id": new_interjection_event_id,
"supersedes_id": original_interjection_event_id,
},
)
# Memory write for the new interjection beat (one event per
# present witness).
record_turn_memory_for_present(
conn,
chat_id=chat_id,
host_bot_id=host_bot_id,
guest_bot_id=guest_bot_id,
narrative_text=interject_text,
scene_id=scene["id"] if scene else None,
chat_clock_at=chat.get("time"),
)
# Re-run the multi-pair state-update with the post-interjection
# dialogue tail so deltas land on the post-primary baseline.
recent_post_interject = recent_for_update + [
{
"speaker": silent_witness.get("name", "bot"),
"text": interject_text,
}
]
prior_edges_post: dict[tuple[str, str], dict] = {}
for src in present_ids:
for tgt in present_ids:
if src == tgt:
continue
edge = get_edge(conn, src, tgt) or {
"affinity": 50,
"trust": 50,
"summary": "",
}
prior_edges_post[(src, tgt)] = edge
state_updates_post = await compute_state_updates_for_present(
client,
classifier_model=settings.classifier_model,
present_ids=present_ids,
present_names=present_names,
personas=personas,
prior_edges=prior_edges_post,
recent_dialogue=recent_post_interject,
timeout_s=settings.classifier_timeout_s,
)
for src_id, tgt_id, update in state_updates_post:
append_and_apply(
conn,
kind="edge_update",
payload={
"source_id": src_id,
"target_id": tgt_id,
"chat_id": chat_id,
"affinity_delta": update.affinity_delta,
"trust_delta": update.trust_delta,
"knowledge_facts": update.knowledge_facts,
"last_interaction_at": last_at,
"last_interaction_chat_id": chat_id,
},
)
else:
# Classifier said "no follow-on this time" — supersede the
# original interjection without a replacement. Point the
# back-pointer at the new primary so the row is consistently
# hidden by the standard timeline filter.
conn.execute(
"UPDATE event_log SET superseded_by = ? WHERE id = ?",
(new_assistant_event_id, original_interjection_event_id),
)
return new_text
__all__ = ["regenerate_assistant_turn"]
+107
View File
@@ -0,0 +1,107 @@
"""Parse user-supplied "have they met?" prose into per-direction seed
content for two bots' edges (T38).
Per Requirements §5.2, when two bots first co-appear in a chat, the user
is offered a small drawer asking "Have they met before? If yes, write a
short prose seed describing how." That prose lands here and is parsed
into a :class:`RelationshipSeed` whose two halves populate the
``botA -> botB`` and ``botB -> botA`` edges respectively (summary,
initial knowledge facts, and small affinity/trust deltas around the
default 50/50 baseline).
The two directions can differ — A may know more about B than B knows
about A, or A may trust B less than the reverse — so the schema carries
both halves independently.
Empty/whitespace-only prose short-circuits to a default
``RelationshipSeed`` (all zeroes, empty strings); the caller treats
that as "they haven't met" and writes no edge content. The wrapper uses
:func:`chat.llm.classify.classify` with ``default=RelationshipSeed()``
so a flapping classifier degrades to the same no-op rather than
blocking the chat-creation flow (§3.3 graceful-degradation rule).
T42 (the inter-bot relationship drawer) calls this from the route layer.
"""
from __future__ import annotations
from pydantic import BaseModel, Field
from chat.llm.classify import classify
from chat.llm.client import LLMClient
class RelationshipSeed(BaseModel):
"""Structured per-direction seed for two bots' edges.
Defaults are a deliberate no-op: empty summaries, empty knowledge
lists, zero deltas. Both the empty-prose short-circuit and the
classifier-failure fallback return this default so the caller can
treat them identically.
"""
a_to_b_summary: str = ""
a_to_b_knowledge_facts: list[str] = Field(default_factory=list)
a_to_b_affinity_delta: int = 0 # signed, -10..+10 typical
a_to_b_trust_delta: int = 0
b_to_a_summary: str = ""
b_to_a_knowledge_facts: list[str] = Field(default_factory=list)
b_to_a_affinity_delta: int = 0
b_to_a_trust_delta: int = 0
_SYSTEM = (
"You parse a short prose seed describing how two characters know each "
"other into structured per-direction edge content. For each direction "
"(A -> B, B -> A) extract: summary (one sentence from that POV), "
"knowledge_facts (list of factual claims that direction can carry "
"into future scenes), affinity_delta (-10..+10 — small adjustments to "
"the default 50/50 baseline), trust_delta (-10..+10). Default deltas "
"to 0 when prose is neutral. The two directions can differ — A may "
"trust B more than B trusts A. Output strict JSON matching the schema."
)
async def seed_inter_bot_edges(
client: LLMClient,
*,
classifier_model: str,
bot_a_id: str,
bot_a_name: str,
bot_b_id: str,
bot_b_name: str,
relationship_prose: str,
timeout_s: float = 30.0,
) -> RelationshipSeed:
"""Parse user-supplied prose into structured edge content for both
directed pairs.
Empty/whitespace prose short-circuits to an empty
:class:`RelationshipSeed` (the caller treats this as "they haven't
met" and writes no edge content). Classifier failure also returns
the default — see module docstring for the rationale.
The ``bot_a_id`` / ``bot_b_id`` arguments are accepted for symmetry
with the caller (T42's drawer route uses them when emitting
``edge_update`` events); they're embedded in the prompt alongside
the names so the classifier can disambiguate when names collide.
"""
if not relationship_prose or not relationship_prose.strip():
return RelationshipSeed()
user = (
f"Bot A: {bot_a_name} (id={bot_a_id})\n"
f"Bot B: {bot_b_name} (id={bot_b_id})\n\n"
f"Prose seed:\n{relationship_prose.strip()}"
)
return await classify(
client,
model=classifier_model,
system=_SYSTEM,
user=user,
schema=RelationshipSeed,
default=RelationshipSeed(),
timeout_s=timeout_s,
)
__all__ = ["RelationshipSeed", "seed_inter_bot_edges"]
+23
View File
@@ -0,0 +1,23 @@
from __future__ import annotations
from sqlite3 import Connection
from chat.eventlog.log import append_and_apply
from chat.state.entities import get_bot
def reset_bot(conn: Connection, bot_id: str, *, confirm_name: str) -> None:
"""Reset a bot's runtime state via a ``bot_reset`` event.
Validates that ``confirm_name`` matches the bot's stored ``name``
exactly (case-sensitive, no trim). Raises:
- ``ValueError("bot {bot_id} not found")`` when the bot is missing.
- ``ValueError("confirm_name does not match bot name")`` on mismatch.
"""
bot = get_bot(conn, bot_id)
if bot is None:
raise ValueError(f"bot {bot_id} not found")
if confirm_name != bot["name"]:
raise ValueError("confirm_name does not match bot name")
append_and_apply(conn, kind="bot_reset", payload={"bot_id": bot_id})
+112
View File
@@ -0,0 +1,112 @@
"""Rewind service — truncate the event log past a chosen turn and re-project.
Per Requirements §10.1 and Plan Task 28, "rewind to here" must:
1. Take a snapshot of the current state so the user can recover (handed
off to :mod:`chat.services.snapshot`).
2. Truncate the event log past ``after_event_id`` — physical DELETE for
v1 simplicity; the spec says rewind should be a hard truncation, not
the soft ``hidden=1`` mechanism used by edits/regenerate.
3. Clear projected tables and re-project from the truncated log so live
state matches "what the world looked like at turn N". Without the
re-projection, projected tables would carry forward stale rows from
rewound events (e.g. an ``edge_update`` that bumped affinity past the
rewind point would still show in ``edges``).
Re-projection is a full replay rather than a "revert delta" because most
projector handlers are idempotent inserts, but the edge handler is a
delta-shaped accumulator — there's no clean way to invert a single
``edge_update`` against ``edges.affinity`` without replay. Wiping +
replaying is straightforward and correct.
"""
from __future__ import annotations
from pathlib import Path
from sqlite3 import Connection
from chat.db.connection import open_db
from chat.eventlog.projector import project
from chat.services.snapshot import take_snapshot
def compute_rewind_preview(
conn: Connection, after_event_id: int
) -> dict:
"""Return counts of each event kind that would be removed by rewinding.
Used by the preview modal so the user sees the impact (e.g. "this
will remove 8 events: 4 user_turn, 4 assistant_turn") before
confirming. Counts include hidden/superseded rows — they're still
physically deleted.
"""
cur = conn.execute(
"SELECT kind, COUNT(*) FROM event_log WHERE id > ? GROUP BY kind "
"ORDER BY kind",
(after_event_id,),
)
counts = {kind: count for kind, count in cur.fetchall()}
total = sum(counts.values())
return {
"after_event_id": after_event_id,
"total_events": total,
"by_kind": counts,
}
def execute_rewind(
*, db_path: Path, data_dir: Path, after_event_id: int
) -> Path:
"""Take a snapshot, truncate, and re-project. Returns the snapshot path.
The snapshot is taken inside the same connection scope as the
truncate + reproject so all three commit together — if any step
fails the connection's commit-on-exit is bypassed by the exception
and the database stays untouched. The snapshot file is on disk
regardless, which is the desired behaviour: even if the truncate
aborts, the user has a recovery point.
"""
with open_db(db_path) as conn:
# 1. Snapshot first — we want this on disk before any destructive
# operation runs.
snapshot_path = take_snapshot(
conn, data_dir=data_dir, kind="rewind"
)
# 2. Truncate the event log past the chosen id. Foreign keys are
# ON, but ``event_log.superseded_by`` self-references and the
# rows we're deleting are the only ones that could point
# forward — there's nothing to cascade.
conn.execute(
"DELETE FROM event_log WHERE id > ?", (after_event_id,)
)
# 3. Clear projected tables in topological order so FK ON DELETE
# constraints don't fire on referenced rows. ``activity`` and
# ``scenes`` reference ``containers``; ``chat_state`` references
# ``chats`` by id-convention only (no FK declared). ``memories``,
# ``edges``, ``bots``, ``you_entity``, and ``classifier_failures``
# have no incoming FKs from other projected tables.
#
# ``executescript`` is intentionally avoided so foreign_keys=ON
# stays in effect for each statement — executescript would
# implicitly commit and reset some pragmas on certain SQLite
# builds.
conn.execute("DELETE FROM memories")
conn.execute("DELETE FROM activity")
conn.execute("DELETE FROM scenes")
conn.execute("DELETE FROM containers")
conn.execute("DELETE FROM chat_state")
conn.execute("DELETE FROM chats")
conn.execute("DELETE FROM edges")
conn.execute("DELETE FROM bots")
conn.execute("DELETE FROM you_entity")
conn.execute("DELETE FROM classifier_failures")
# 4. Re-project from the truncated event log. Handler registry
# is module-level state populated by importing chat.state.* —
# callers (the route, tests) need to have those modules
# imported for this to do anything useful.
project(conn)
return snapshot_path
+100
View File
@@ -0,0 +1,100 @@
"""Scene-close hard-signal detection (T26).
A small classifier service that decides whether the user's prose narrates
a hard signal that should close the active scene. Hard signals (per
Requirements §7.2):
* Container change parsed from prose ("we drove to the park", "we stepped
outside").
* Explicit user pattern signaling end ("we're done here", "fade out",
"scene end").
NOT close signals:
* Brief activity changes within the same container ("I sit down").
* Future plans ("let's go to the park later").
The service returns a :class:`SceneCloseDecision`. The default on classifier
failure is ``should_close=False`` so the turn flow keeps moving — closing
on a misfire would be more disruptive than missing a real signal, and the
manual button in the drawer is always available as a fallback.
Phase 2/3 will introduce automatic re-opening with the new container; for
T26 the close is one-way and the next user turn operates without an active
scene (the prompt assembler already tolerates this).
"""
from __future__ import annotations
from pydantic import BaseModel
from chat.llm.classify import classify
from chat.llm.client import LLMClient
class SceneCloseDecision(BaseModel):
"""Classifier verdict for scene-close detection.
``new_container_hint`` is captured opportunistically when the close
signal is a container change, but T26 doesn't act on it — Phase 2/3
handles automatic re-opening at the new location.
"""
should_close: bool = False
reason: str = ""
new_container_hint: str = ""
_SYSTEM = (
"You decide whether a roleplay scene should close based on the user's "
"prose.\n"
"Close signals (return should_close=true):\n"
"- The prose narrates a CONTAINER CHANGE (moving to a different place, "
'e.g. "we drove to the park", "we stepped outside").\n'
"- The prose has an EXPLICIT USER PATTERN signaling end "
'("we\'re done here", "fade out", "scene end").\n'
"\n"
"DO NOT close on:\n"
"- Brief activity changes within the same place "
'(e.g. "I sit down" — same room).\n'
"- Future plans "
'("let\'s go to the park later" — not yet).\n'
"\n"
'Reply JSON: {"should_close": bool, "reason": str (short), '
'"new_container_hint": str (optional name)}.'
)
async def detect_scene_close(
client: LLMClient,
*,
model: str,
prose: str,
current_container_name: str,
timeout_s: float = 10.0,
) -> SceneCloseDecision:
"""Run the scene-close classifier on a single user turn.
The current container name is passed in so the prompt can reason about
"different place" relative to the active scene rather than guessing.
On classifier failure (parse error twice), the returned decision is the
safe ``should_close=False`` default.
"""
user = (
f"CURRENT CONTAINER: {current_container_name}\n"
f"\n"
f"PROSE:\n{prose}\n"
f"\n"
f"Decide whether to close the scene."
)
return await classify(
client,
model=model,
system=_SYSTEM,
user=user,
schema=SceneCloseDecision,
default=SceneCloseDecision(
should_close=False, reason="fallback", new_container_hint=""
),
timeout_s=timeout_s,
)
+425
View File
@@ -0,0 +1,425 @@
"""Per-POV scene summary and edge summary update on scene close (T27).
When a scene closes — either auto-detected by the hard-signal classifier
in T26 or fired by the manual close button on the drawer — we run a
single-shot classifier per present witness that produces three signals
in one pass:
* ``summary`` — a 2-4 sentence per-POV recap of the scene from this
witness's perspective. Different from omniscient narration; focuses on
what the witness noticed/felt/remembers.
* ``knowledge_facts`` — concrete new things this witness learned about
the user during the scene. Promoted to the directed edge's
``knowledge`` list via ``edge_update``.
* ``relationship_summary`` — a 1-2 sentence delta on how the
witness's relationship to the user shifted in this scene. v1
combines this with the prior edge summary by simple concatenation —
the LLM is asked to phrase ``relationship_summary`` as a merge-ready
fragment, so the result reads naturally without a second classifier
round-trip.
Phase 1 single-bot only the host bot is summarized; "you" doesn't have
a memory store in v1 so per-POV writes for the user are deferred. The
:func:`apply_scene_close_summary` driver is intentionally tolerant: if
no memories belong to the closed scene it silently skips the rewrite,
and a flapping classifier returns the empty default so the close flow
keeps moving.
"""
from __future__ import annotations
import json
from sqlite3 import Connection
from pydantic import BaseModel, Field
from chat.eventlog.log import append_and_apply
from chat.llm.classify import classify
from chat.llm.client import LLMClient
class ScenePOVSummary(BaseModel):
"""Classifier output: one witness's view of a closing scene.
Defaults are an inert no-op so a classifier failure is harmless —
callers can apply the result unconditionally and end up not
rewriting anything when the model misbehaves.
"""
summary: str = ""
knowledge_facts: list[str] = Field(default_factory=list)
relationship_summary: str = ""
_SYSTEM_TEMPLATE = (
"You are summarizing a roleplay scene from {bot_name}'s point of "
"view. Read the dialogue, then output JSON with exactly three "
"fields:\n"
"- summary: 2-4 sentences, in {bot_name}'s POV, of what happened "
"in the scene. This is NOT omniscient narration — focus on what "
"{bot_name} noticed, felt, and would remember.\n"
"- knowledge_facts: list of NEW factual things {bot_name} learned "
"about the user during this scene. Use specific stated content; do "
"not infer or interpret. Empty list is fine.\n"
"- relationship_summary: a SHORT (1-2 sentence) summary of how "
"{bot_name}'s relationship with the user changed or developed in "
"this scene. Phrase it so it reads as a continuation of the prior "
"summary; the caller will concatenate them.\n\n"
"Be specific. Avoid generic phrases."
)
def _format_dialogue(dialogue: list[dict]) -> str:
if not dialogue:
return "(no dialogue)"
return "\n".join(
f"{turn.get('speaker', '?')}: {turn.get('text', '')}"
for turn in dialogue
)
async def summarize_scene(
client: LLMClient,
*,
model: str,
bot_name: str,
bot_persona: str,
you_name: str,
prior_edge_summary: str,
dialogue: list[dict],
timeout_s: float = 10.0,
) -> ScenePOVSummary:
"""Run the per-POV summary classifier for one witness.
The signature mirrors :func:`compute_state_update` — passing the
bot's name and persona as separate fields lets the prompt address
the model directly ("YOU are {bot_name}") rather than handing it an
opaque id. ``prior_edge_summary`` is included so the classifier can
phrase ``relationship_summary`` as an additive fragment.
Returns the empty default on classifier failure (after one retry)
rather than raising, so the close pipeline keeps moving.
"""
system = _SYSTEM_TEMPLATE.format(bot_name=bot_name)
user = (
f"YOU are {bot_name}. {bot_persona or '(no persona on file)'}\n"
f"USER name: {you_name}\n"
f"PRIOR EDGE SUMMARY ({bot_name} -> {you_name}): "
f"{prior_edge_summary or '(empty)'}\n\n"
f"DIALOGUE:\n{_format_dialogue(dialogue)}\n\n"
f"Produce the JSON summary in {bot_name}'s POV."
)
return await classify(
client,
model=model,
system=system,
user=user,
schema=ScenePOVSummary,
default=ScenePOVSummary(),
timeout_s=timeout_s,
)
def _read_recent_dialogue(
conn: Connection, chat_id: str, *, limit: int = 50
) -> list[dict]:
"""Pull the last ``limit`` user/assistant turns for ``chat_id``.
Phase 1 ``user_turn`` / ``assistant_turn`` events don't carry a
``scene_id``, so we approximate the scene's transcript by taking
the most recent turns of the chat. Superseded and hidden rows are
filtered out so regenerated turns (T29) don't bleed into the
summary.
"""
cur = conn.execute(
"SELECT kind, payload_json FROM event_log "
"WHERE kind IN ('user_turn', 'assistant_turn') "
" AND superseded_by IS NULL AND hidden = 0 "
"ORDER BY id DESC LIMIT ?",
(limit,),
)
rows = list(reversed(cur.fetchall()))
out: list[dict] = []
for kind, payload_json in rows:
p = json.loads(payload_json)
if p.get("chat_id") != chat_id:
continue
if kind == "user_turn":
out.append({"speaker": "you", "text": p.get("prose", "")})
else:
out.append(
{
"speaker": p.get("speaker_id", "bot"),
"text": p.get("text", ""),
}
)
return out
async def _summarize_and_apply_for_witness(
conn: Connection,
client: LLMClient,
*,
classifier_model: str,
chat_id: str,
scene_id: int,
bot_id: str,
you_name: str,
dialogue: list[dict],
timeout_s: float,
) -> ScenePOVSummary:
"""Run :func:`summarize_scene` for one bot witness and apply the
three projected updates (memory pov_summary rewrite, edge summary
overwrite, edge knowledge_facts append).
Tolerant of missing pieces in the same way Phase 1 was: no memory
row -> skip the rewrite; no edge row -> skip the edge_summary write
(the empty-default classifier output simply yields no rewrites).
"""
from chat.state.edges import get_edge
from chat.state.entities import get_bot
bot = get_bot(conn, bot_id) or {"name": bot_id, "persona": ""}
edge_b2y = get_edge(conn, bot_id, "you")
prior_summary = (edge_b2y or {}).get("summary", "") or ""
pov = await summarize_scene(
client,
model=classifier_model,
bot_name=bot.get("name", bot_id),
bot_persona=bot.get("persona", "") or "",
you_name=you_name,
prior_edge_summary=prior_summary,
dialogue=dialogue,
timeout_s=timeout_s,
)
# Update memories belonging to the closed scene for this witness.
cur = conn.execute(
"SELECT id, pov_summary FROM memories "
"WHERE scene_id = ? AND owner_id = ?",
(scene_id, bot_id),
)
for memory_id, prior_pov in cur.fetchall():
if not pov.summary:
# Empty default -> skip the memory rewrite; the seeded
# per-turn pov_summary stays in place.
continue
append_and_apply(
conn,
kind="manual_edit",
payload={
"target_kind": "memory_pov_summary",
"target_id": int(memory_id),
"prior_value": prior_pov,
"new_value": pov.summary,
},
)
# Update this bot->you edge summary if we have an edge row and a
# non-empty relationship_summary to merge.
if edge_b2y is not None and pov.relationship_summary:
new_summary = (
f"{prior_summary} {pov.relationship_summary}".strip()
if prior_summary
else pov.relationship_summary
)
append_and_apply(
conn,
kind="manual_edit",
payload={
"target_kind": "edge_summary",
"target_id": {
"source_id": bot_id,
"target_id": "you",
},
"prior_value": prior_summary,
"new_value": new_summary,
},
)
# Append knowledge_facts to this bot->you edge if present.
if pov.knowledge_facts:
append_and_apply(
conn,
kind="edge_update",
payload={
"source_id": bot_id,
"target_id": "you",
"chat_id": chat_id,
"knowledge_facts": list(pov.knowledge_facts),
},
)
return pov
async def apply_scene_close_summary(
conn: Connection,
client: LLMClient,
*,
classifier_model: str,
chat_id: str,
scene_id: int,
host_bot_id: str,
timeout_s: float = 10.0,
) -> ScenePOVSummary:
"""Drive the per-POV summary pipeline after ``scene_closed``.
Phase 1 (single-bot) behavior — the host bot is summarized once and
the result drives memory + edge rewrites — is preserved exactly when
the chat has no guest. T45 extends this to fan out across each
present bot witness when a guest is also in the room:
1. Gather the closing scene's dialogue from the event_log.
2. For each present witness (host + guest if any), run
:func:`summarize_scene` once with that witness's persona and
their own prior ``bot -> you`` edge summary.
3. For each witness independently:
a. Rewrite each scene-bound memory's ``pov_summary`` via
``manual_edit`` (target_kind ``memory_pov_summary``).
b. Update that witness's ``bot -> you`` edge summary via
``manual_edit`` (target_kind ``edge_summary``). v2 combines
prior + classifier ``relationship_summary`` by simple
concatenation.
c. Append any ``knowledge_facts`` to the same edge via
``edge_update``.
4. If a ``group_node`` row exists for this chat, append a
``group_node_updated`` event whose ``summary`` is the naive
per-POV concat ``f"{name}: {summary}\\n\\n..."``. A true
LLM-merged group view is deferred to Phase 2.5; ``dynamic``
is left empty here for v2 (Phase 3 polishes it).
The host's :class:`ScenePOVSummary` is returned to preserve the
Phase 1 callers' contract.
"""
# Local imports to keep the module-level surface tight and avoid
# any chance of a circular dep through chat.state.*.
from chat.state.entities import get_bot, get_you
from chat.state.group_node import get_group_node
from chat.state.world import get_chat
you_entity = get_you(conn) or {"name": "you", "persona": ""}
you_name = you_entity.get("name", "you") or "you"
chat = get_chat(conn, chat_id) or {}
guest_bot_id = chat.get("guest_bot_id")
dialogue = _read_recent_dialogue(conn, chat_id)
host_pov = await _summarize_and_apply_for_witness(
conn,
client,
classifier_model=classifier_model,
chat_id=chat_id,
scene_id=scene_id,
bot_id=host_bot_id,
you_name=you_name,
dialogue=dialogue,
timeout_s=timeout_s,
)
guest_pov: ScenePOVSummary | None = None
if guest_bot_id is not None:
guest_pov = await _summarize_and_apply_for_witness(
conn,
client,
classifier_model=classifier_model,
chat_id=chat_id,
scene_id=scene_id,
bot_id=guest_bot_id,
you_name=you_name,
dialogue=dialogue,
timeout_s=timeout_s,
)
# Group node update: T70 runs a third classifier call to merge the
# two per-POV summaries into a coherent group-level view + a brief
# group-dynamic note. Falls back to the Phase 2 naive concat on
# classifier failure (see :func:`merge_group_summary`). Only fires
# when both POVs ran (i.e. the guest is present) and a group_node
# row exists for this chat.
if guest_pov is not None and get_group_node(conn, chat_id) is not None:
host_bot = get_bot(conn, host_bot_id) or {"name": host_bot_id}
guest_bot = get_bot(conn, guest_bot_id) or {"name": guest_bot_id}
host_name = host_bot.get("name", host_bot_id) or host_bot_id
guest_name = guest_bot.get("name", guest_bot_id) or guest_bot_id
merged = await merge_group_summary(
client,
classifier_model=classifier_model,
host_name=host_name,
host_pov_summary=host_pov.summary,
guest_name=guest_name,
guest_pov_summary=guest_pov.summary,
timeout_s=timeout_s,
)
append_and_apply(
conn,
kind="group_node_updated",
payload={
"chat_id": chat_id,
"summary": merged.summary,
"dynamic": merged.dynamic,
},
)
return host_pov
class GroupMetaSummary(BaseModel):
"""Classifier output: a merged group-level view of a closed scene.
Defaults are an empty no-op so callers can use the schema's default
as a sentinel; in practice :func:`merge_group_summary` builds an
explicit naive-concat fallback rather than returning these defaults
directly so existing Phase 2 behavior is preserved on classifier
failure.
"""
summary: str = ""
dynamic: str = ""
_GROUP_MERGE_SYSTEM = (
"Given two per-POV scene summaries from a 3-entity scene (you + "
"host + guest), produce a coherent group-level summary capturing "
"the shared events as both witnesses experienced them, plus a "
"brief 'dynamic' note describing the trio's group dynamic during "
"the scene. Output strict JSON matching schema."
)
async def merge_group_summary(
client: LLMClient,
*,
classifier_model: str,
host_name: str,
host_pov_summary: str,
guest_name: str,
guest_pov_summary: str,
timeout_s: float = 30.0,
) -> GroupMetaSummary:
"""Merge two per-POV scene summaries into a coherent group-level
summary + group-dynamic note. Falls back to the naive concat (the
existing behavior) on classifier failure."""
user = (
f"{host_name} (host) POV summary:\n{host_pov_summary}\n\n"
f"{guest_name} (guest) POV summary:\n{guest_pov_summary}"
)
fallback = GroupMetaSummary(
summary=(
f"{host_name}: {host_pov_summary}\n\n"
f"{guest_name}: {guest_pov_summary}"
),
dynamic="",
)
return await classify(
client,
model=classifier_model,
system=_GROUP_MERGE_SYSTEM,
user=user,
schema=GroupMetaSummary,
default=fallback,
timeout_s=timeout_s,
)
+75
View File
@@ -0,0 +1,75 @@
"""Turn-level significance scorer (T22).
Per Requirements §11.1, each turn is scored on a 0-3 scale:
- 0 = Routine: small talk, ordinary action.
- 1 = Notable: a specific detail or beat worth remembering.
- 2 = Significant: a scene-level moment, real disagreement, confided secret.
- 3 = Pivotal: a relationship-altering event (first kiss, betrayal, "I love
you").
The scorer is conservative: pivotal (3) requires a clear signal because the
auto-pin rule (§8.5) gives those memories permanent shelf space. The
classifier returns a strict-JSON ``SignificanceVerdict``; a malformed or
refusal-shaped response falls back to ``score=1`` (Notable) — a safe
middle-of-the-road default that won't trigger auto-pin.
"""
from __future__ import annotations
from pydantic import BaseModel, Field
from chat.llm.classify import classify
from chat.llm.client import LLMClient
class SignificanceVerdict(BaseModel):
score: int = Field(ge=0, le=3)
reason: str = ""
_SYSTEM = """You score the significance of a roleplay turn 0-3:
0 = Routine: small talk, ordinary action.
1 = Notable: a specific detail or beat worth remembering.
2 = Significant: a scene-level moment, real disagreement, confided secret.
3 = Pivotal: a relationship-altering event (first kiss, betrayal, "I love you").
Be conservative — pivotal (3) requires a clear signal. Reply with JSON: {"score": int 0-3, "reason": str}."""
async def compute_significance(
client: LLMClient,
*,
model: str,
narrative_text: str,
prior_dialogue: list[dict],
timeout_s: float = 10.0,
) -> int:
"""Score the significance of ``narrative_text`` (the just-written turn).
``prior_dialogue`` is a list of ``{"speaker", "text"}`` dicts ordered
oldest-first; the last 6 entries are stitched into the user prompt as
context so the classifier can recognize escalation. Returns an int in
``[0, 3]`` — clamped defensively in case the classifier slips a value
past the schema validator.
"""
user_prompt = "PRIOR DIALOGUE:\n"
for turn in prior_dialogue[-6:]:
speaker = turn.get("speaker", "?")
text = turn.get("text", "")
user_prompt += f"{speaker}: {text}\n"
user_prompt += (
f"\nNEW TURN:\n{narrative_text}\n\n"
"Score the significance of the NEW TURN."
)
result = await classify(
client,
model=model,
system=_SYSTEM,
user=user_prompt,
schema=SignificanceVerdict,
default=SignificanceVerdict(score=1, reason="fallback"),
timeout_s=timeout_s,
)
return max(0, min(3, result.score))
+245
View File
@@ -0,0 +1,245 @@
"""Snapshot service — write a JSON dump of all projected tables to disk.
Two snapshot kinds, both covered by this module:
* ``rewind`` (T28, Requirements §10.1): pre-rewind safety snapshot so the
user can recover if a rewind was a mistake. Retention: 14 days.
* ``periodic`` (T31, Requirements §10.4): full-state checkpoint taken
every 100 events OR every 30 minutes since the last one. Retention:
the most recent 5 are kept; older ones are pruned on write.
Both kinds live under ``data/snapshots/{kind}/`` with a UTC timestamp
filename so chronological listing matches creation order.
The dump captures the event log (so the original event sequence is
preserved verbatim), every projected table, and a top-level
``last_event_id`` recording the highest ``event_log.id`` at snapshot
time. The ``last_event_id`` is what the cold-load fast-path uses to
replay only events past the snapshot rather than the entire log.
The FTS shadow table ``memories_fts`` is intentionally skipped — it's a
virtual table maintained by the ``memories_ai/au/ad`` triggers, so it
rebuilds itself on a memories re-load. Snapshotting it would also fail
``PRAGMA table_info`` cleanly since FTS5 reports its columns differently.
"""
from __future__ import annotations
import json
import time
from datetime import datetime, timezone
from pathlib import Path
from sqlite3 import Connection
# Periodic snapshot triggers (Requirements §10.4): "every 100 events OR
# every 30 minutes since last snapshot". Module-level so tests can read
# them and so the values stay together with the policy that uses them.
EVENT_COUNT_THRESHOLD = 100
TIME_THRESHOLD_SECONDS = 30 * 60 # 30 minutes
# Order doesn't affect correctness for snapshotting (we read, not write),
# but listing tables explicitly keeps the snapshot stable across schema
# evolution: a new table won't silently change the dump shape until it's
# added here.
PROJECTED_TABLES = [
"bots",
"you_entity",
"edges",
"memories",
"memories_fts",
"chats",
"chat_state",
"containers",
"scenes",
"activity",
"classifier_failures",
]
def take_snapshot(
conn: Connection, *, data_dir: Path, kind: str = "rewind"
) -> Path:
"""Write a JSON dump of the event log and projected tables.
Returns the path to the written snapshot file. Creates parent
directories as needed. Filename is a UTC timestamp in
``YYYYMMDDTHHMMSSZ`` form so chronological listing matches creation
order.
The dump's top-level ``last_event_id`` is the highest ``event_log.id``
at snapshot time (0 if the log is empty). This is what the cold-load
fast-path uses to know which suffix of the log to replay.
"""
snapshot_dir = data_dir / "snapshots" / kind
snapshot_dir.mkdir(parents=True, exist_ok=True)
timestamp = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%SZ")
path = snapshot_dir / f"{timestamp}.json"
dump: dict = {}
# Record the high-water-mark id up front so cold-load can replay
# only events past it. ``MAX(id)`` is None on an empty log; treat
# that as 0 (i.e. "replay everything").
cur = conn.execute("SELECT MAX(id) FROM event_log")
max_id_row = cur.fetchone()
dump["last_event_id"] = max_id_row[0] if max_id_row[0] is not None else 0
# Event log: pull every column we care about. ``ts`` and the
# superseded/hidden flags are needed to faithfully reconstruct the
# log on restore.
cur = conn.execute(
"SELECT id, branch_id, ts, kind, payload_json, superseded_by, hidden "
"FROM event_log ORDER BY id"
)
dump["event_log"] = [
{
"id": r[0],
"branch_id": r[1],
"ts": r[2],
"kind": r[3],
"payload_json": r[4],
"superseded_by": r[5],
"hidden": r[6],
}
for r in cur.fetchall()
]
for table in PROJECTED_TABLES:
if table == "memories_fts":
# Virtual FTS5 table — rebuilt by triggers on insert, no need
# to snapshot it (and ``PRAGMA table_info`` reports its
# columns differently).
continue
cur = conn.execute(f"PRAGMA table_info({table})")
cols = [c[1] for c in cur.fetchall()]
if not cols:
# Table not present in this schema version — leave an empty
# list rather than raising, so older snapshots can survive.
dump[table] = []
continue
cur = conn.execute(f"SELECT {', '.join(cols)} FROM {table}")
dump[table] = [dict(zip(cols, row)) for row in cur.fetchall()]
# ``default=str`` covers Path-like or datetime values that might
# sneak through if a column ever stored them; the projected tables
# all use TEXT so this is mostly defensive.
path.write_text(json.dumps(dump, default=str))
return path
def latest_snapshot_path(data_dir: Path, kind: str = "periodic") -> Path | None:
"""Return the most recent snapshot file for ``kind``, or None if none exist.
Sorting by filename works because :func:`take_snapshot` uses a UTC
timestamp in ``YYYYMMDDTHHMMSSZ`` form — lexicographic order matches
chronological order.
"""
snapshot_dir = data_dir / "snapshots" / kind
if not snapshot_dir.exists():
return None
files = sorted(snapshot_dir.glob("*.json"))
return files[-1] if files else None
def should_take_periodic_snapshot(
conn: Connection, data_dir: Path
) -> bool:
"""Decide whether a periodic snapshot is due per Requirements §10.4.
The policy:
* No prior snapshot and at least one event in the log → take one.
* Time since last snapshot ≥ ``TIME_THRESHOLD_SECONDS`` → take one.
* New events since last snapshot's ``last_event_id`` ≥
``EVENT_COUNT_THRESHOLD`` → take one.
"Time since last snapshot" is measured by the file's mtime — we
don't trust the timestamp embedded in the filename for clock drift
reasons.
"""
latest = latest_snapshot_path(data_dir, kind="periodic")
if latest is None:
# No prior snapshot; take one if there are any events to capture.
cur = conn.execute("SELECT COUNT(*) FROM event_log")
return cur.fetchone()[0] > 0
age_seconds = time.time() - latest.stat().st_mtime
if age_seconds >= TIME_THRESHOLD_SECONDS:
return True
# Count events appended since the last snapshot was written. Reading
# ``last_event_id`` from the dump is cheap (a few KB at most for the
# header) but we still avoid loading the full file by parsing once.
last_dump = json.loads(latest.read_text())
last_event_id = last_dump.get("last_event_id", 0)
cur = conn.execute(
"SELECT COUNT(*) FROM event_log WHERE id > ?", (last_event_id,)
)
new_event_count = cur.fetchone()[0]
return new_event_count >= EVENT_COUNT_THRESHOLD
def prune_periodic_snapshots(data_dir: Path, keep: int = 5) -> int:
"""Delete all but the most recent ``keep`` periodic snapshots.
Returns the number of files removed. Safe to call when the directory
doesn't exist (returns 0). Sorting is by filename, which is the UTC
timestamp — same ordering :func:`latest_snapshot_path` uses.
"""
snapshot_dir = data_dir / "snapshots" / "periodic"
if not snapshot_dir.exists():
return 0
files = sorted(snapshot_dir.glob("*.json"))
to_remove = files[:-keep] if len(files) > keep else []
for f in to_remove:
f.unlink()
return len(to_remove)
def restore_from_snapshot(conn: Connection, snapshot_path: Path) -> int:
"""Restore projected tables from ``snapshot_path``.
Returns the snapshot's ``last_event_id`` so callers (the cold-load
fast-path in :func:`chat.app.lifespan`) know what suffix of the
event log still needs replaying.
Projected tables are cleared in the same FK-respecting order as
:func:`chat.services.rewind.execute_rewind`, then re-populated from
the dump. ``memories_fts`` is skipped — it's a virtual FTS5 table
that rebuilds itself when rows hit ``memories``. The event log
itself is *not* touched: cold-load assumes the on-disk log is the
source of truth and the snapshot is just a fast-forward to skip
re-projecting old events.
"""
dump = json.loads(snapshot_path.read_text())
# Same delete order as rewind: child tables before parents so FK
# ON DELETE doesn't fire on referenced rows.
conn.execute("DELETE FROM memories")
conn.execute("DELETE FROM activity")
conn.execute("DELETE FROM scenes")
conn.execute("DELETE FROM containers")
conn.execute("DELETE FROM chat_state")
conn.execute("DELETE FROM chats")
conn.execute("DELETE FROM edges")
conn.execute("DELETE FROM bots")
conn.execute("DELETE FROM you_entity")
conn.execute("DELETE FROM classifier_failures")
for table in PROJECTED_TABLES:
if table == "memories_fts":
# Rebuilt by triggers when memories rows are inserted below.
continue
rows = dump.get(table, [])
if not rows:
continue
cols = list(rows[0].keys())
placeholders = ", ".join("?" * len(cols))
col_list = ", ".join(cols)
for row in rows:
conn.execute(
f"INSERT INTO {table} ({col_list}) VALUES ({placeholders})",
tuple(row[c] for c in cols),
)
return dump.get("last_event_id", 0)
+144
View File
@@ -0,0 +1,144 @@
"""Post-turn state-update pass.
Per Requirements §3.4, after every utterance we run a classifier on each
present entity (silent witnesses included) to extract directed-edge
deltas — what changed in *source*'s view of *target*. The classifier
returns three signals:
- ``affinity_delta`` — signed change in how warmly source feels (typical
range -3..+3; the edge handler clamps the running total to 0..100).
- ``trust_delta`` — signed change in source's trust of target (same
shape).
- ``knowledge_facts`` — concrete things source learned about target
during this exchange. Stored verbatim and appended to ``edge.knowledge``.
The wrapper deliberately uses :func:`chat.llm.classify.classify` with a
``default=StateUpdate()`` so a flapping classifier never blocks the turn
flow — at worst the edge sits unchanged and the next turn tries again
(§3.3 "graceful degradation" rule).
"""
from __future__ import annotations
from pydantic import BaseModel, Field
from chat.llm.classify import classify
from chat.llm.client import LLMClient
class StateUpdate(BaseModel):
"""One directed-edge update from a single classifier call.
Defaults are deliberately a no-op (zero deltas, empty facts) so a
failing classifier produces a benign event rather than a disruption.
"""
affinity_delta: int = 0
trust_delta: int = 0
knowledge_facts: list[str] = Field(default_factory=list)
_SYSTEM_PROMPT = (
"You are reading a recent slice of dialogue from a roleplay scene. "
"You assess how SOURCE's view of TARGET shifted based on what was "
"said — including silent witnessing (SOURCE may not have spoken).\n\n"
"Output a JSON object with exactly three fields:\n"
"- affinity_delta: signed integer in [-3, 3]. How much warmer "
"(positive) or cooler (negative) SOURCE now feels toward TARGET.\n"
"- trust_delta: signed integer in [-3, 3]. How much more (positive) "
"or less (negative) SOURCE now trusts TARGET.\n"
"- knowledge_facts: list of short strings. New, concrete facts "
"SOURCE learned about TARGET in this exchange. Use TARGET's actual "
"stated content; do not infer or interpret. Empty list is fine.\n\n"
"Be conservative. Most turns produce small deltas (-1, 0, +1). "
"Reserve +/-2 or +/-3 for moments that materially shift the "
"relationship. Knowledge_facts should be specific things stated in "
"dialogue (e.g. \"works at the bakery\"), not interpretations "
"(\"seems lonely\")."
)
def _format_dialogue(recent_dialogue: list[dict]) -> str:
"""Render the recent-dialogue slice as plain ``Speaker: text`` lines."""
if not recent_dialogue:
return "(no dialogue yet)"
lines = []
for turn in recent_dialogue:
speaker = turn.get("speaker", "?")
text = turn.get("text", "")
lines.append(f"{speaker}: {text}")
return "\n".join(lines)
def _build_user_prompt(
*,
source_name: str,
source_persona: str,
target_name: str,
prior_affinity: int,
prior_trust: int,
prior_summary: str,
recent_dialogue: list[dict],
) -> str:
return (
f"SOURCE: {source_name}\n"
f"SOURCE_PERSONA: {source_persona or '(none)'}\n"
f"TARGET: {target_name}\n"
f"PRIOR_AFFINITY (0-100): {prior_affinity}\n"
f"PRIOR_TRUST (0-100): {prior_trust}\n"
f"PRIOR_SUMMARY: {prior_summary or '(none)'}\n\n"
f"RECENT_DIALOGUE:\n{_format_dialogue(recent_dialogue)}\n\n"
"How did SOURCE's view of TARGET shift? Respond with JSON only."
)
async def compute_state_update(
client: LLMClient,
*,
model: str,
source_id: str,
target_id: str,
source_name: str,
source_persona: str,
target_name: str,
prior_affinity: int,
prior_trust: int,
prior_summary: str,
recent_dialogue: list[dict],
timeout_s: float = 10.0,
) -> StateUpdate:
"""Run a classifier pass and return the directed-edge update.
On classifier failure (after retry) returns the schema default — a
no-op ``StateUpdate`` — so the turn flow can keep moving. The
``source_id`` / ``target_id`` arguments are accepted for symmetry
with the caller (T20's POST flow uses them when emitting the
``edge_update`` event); they're not currently embedded in the
prompt because the classifier reasons about names, not opaque ids.
"""
# ``source_id``/``target_id`` are kept on the signature even though
# the prompt only quotes the names: callers in turns.py thread the
# ids straight from this function's args into the appended event.
del source_id, target_id # silence unused-arg lint cleanly
user_prompt = _build_user_prompt(
source_name=source_name,
source_persona=source_persona,
target_name=target_name,
prior_affinity=prior_affinity,
prior_trust=prior_trust,
prior_summary=prior_summary,
recent_dialogue=recent_dialogue,
)
return await classify(
client,
model=model,
system=_SYSTEM_PROMPT,
user=user_prompt,
schema=StateUpdate,
default=StateUpdate(),
timeout_s=timeout_s,
)
+94
View File
@@ -0,0 +1,94 @@
"""Turn input parser.
Service-layer function that splits a user's authored turn into typed
segments — ``dialogue``, ``action``, or ``ooc`` (out-of-character).
Per Requirements §6.1 a turn is mixed prose with three conventions:
- ``*action*`` (single asterisks around prose) → action segment.
- Quoted text, or bare prose between the conventions → dialogue.
- ``((double parens))`` → OOC, the author talking to the system rather
than the bot. Downstream (T19) strips OOC from the prompt sent to the
bot but keeps it in the transcript display.
A regex-based splitter would brittle on edge cases (unclosed asterisks,
nested quotes, mixed punctuation), so v1 delegates the segmentation to
the classifier. The configurable ``Settings.ooc_marker`` is *not* read
here: the classifier figures OOC out from ``((`` ``))`` regardless of
config-time choice; marker-based stripping is a downstream concern.
"""
from __future__ import annotations
from pydantic import BaseModel
from chat.llm.classify import classify
from chat.llm.client import LLMClient
class TurnSegment(BaseModel):
"""One classified piece of a turn.
``kind`` is kept as a plain ``str`` (not a ``Literal``) so an
unexpected classifier output doesn't crash parsing — callers that
care about specific values can check defensively.
"""
kind: str # "dialogue" | "action" | "ooc"
text: str
class ParsedTurn(BaseModel):
"""A turn split into ordered, typed segments."""
segments: list[TurnSegment]
_SYSTEM_PROMPT = (
"You are splitting a roleplay turn into typed segments. The input "
"is mixed prose with three conventions:\n"
"- *text in single asterisks* is an ACTION segment.\n"
"- \"quoted text\" or bare prose between conventions is a DIALOGUE segment.\n"
"- ((text in double parens)) is an OOC (out-of-character) segment — "
"the author talking to the system, not the in-fiction bot.\n\n"
"Output a JSON object with shape "
'{"segments": [{"kind": "...", "text": "..."}, ...]} '
"where each ``kind`` is exactly one of: dialogue, action, ooc. "
"Preserve the original substring text as ``text``: do not rewrite, "
"translate, or normalize punctuation — strip only the marker "
"characters (asterisks, surrounding quotes, double parens) so "
"``text`` is the inner content. Emit segments in the order they "
"appear in the input."
)
async def parse_turn(
client: LLMClient,
*,
model: str,
prose: str,
timeout_s: float = 10.0,
) -> ParsedTurn:
"""Parse a user turn into typed segments.
Calls :func:`chat.llm.classify.classify` under the hood. Empty or
whitespace-only prose short-circuits to an empty ``ParsedTurn``
without an LLM call (the classifier would error on empty input
anyway, and the result is unambiguous).
Raises ``RuntimeError`` if the classifier fails twice — no default
is supplied, since the caller (T19's turn flow) is responsible for
surfacing the error to the user.
"""
if not prose.strip():
return ParsedTurn(segments=[])
user_prompt = f"INPUT:\n{prose}"
return await classify(
client,
model=model,
system=_SYSTEM_PROMPT,
user=user_prompt,
schema=ParsedTurn,
timeout_s=timeout_s,
)
View File
+84
View File
@@ -0,0 +1,84 @@
from __future__ import annotations
import json
from sqlite3 import Connection
from chat.eventlog.projector import on
from chat.eventlog.log import Event
def _clamp(value: int, lo: int = 0, hi: int = 100) -> int:
return max(lo, min(hi, value))
@on("edge_update")
def _apply_edge_update(conn: Connection, e: Event) -> None:
p = e.payload
source_id = p["source_id"]
target_id = p["target_id"]
chat_id = p.get("chat_id")
# Upsert: ensure a row exists with defaults, then apply deltas.
conn.execute(
"INSERT OR IGNORE INTO edges (chat_id, source_id, target_id) VALUES (?, ?, ?)",
(chat_id, source_id, target_id),
)
row = conn.execute(
"SELECT affinity, trust, knowledge_json, last_interaction_chat_id, last_interaction_at "
"FROM edges WHERE source_id = ? AND target_id = ?",
(source_id, target_id),
).fetchone()
affinity, trust, knowledge_json, last_chat_id, last_at = row
affinity_delta = int(p.get("affinity_delta", 0))
trust_delta = int(p.get("trust_delta", 0))
new_affinity = _clamp(affinity + affinity_delta)
new_trust = _clamp(trust + trust_delta)
new_facts = p.get("knowledge_facts") or []
if new_facts:
knowledge = json.loads(knowledge_json)
knowledge.extend(new_facts)
knowledge_json = json.dumps(knowledge)
payload_at = p.get("last_interaction_at")
payload_chat_id = p.get("last_interaction_chat_id")
if payload_at is not None:
last_at = payload_at
if payload_chat_id is not None:
last_chat_id = payload_chat_id
conn.execute(
"UPDATE edges SET affinity = ?, trust = ?, knowledge_json = ?, "
"last_interaction_chat_id = ?, last_interaction_at = ? "
"WHERE source_id = ? AND target_id = ?",
(new_affinity, new_trust, knowledge_json, last_chat_id, last_at,
source_id, target_id),
)
def get_edge(conn: Connection, source_id: str, target_id: str) -> dict | None:
row = conn.execute(
"SELECT * FROM edges WHERE source_id = ? AND target_id = ?",
(source_id, target_id),
).fetchone()
if not row:
return None
cols = [c[1] for c in conn.execute("PRAGMA table_info(edges)").fetchall()]
d = dict(zip(cols, row))
d["knowledge"] = json.loads(d.pop("knowledge_json"))
return d
def list_edges_for(conn: Connection, source_id: str) -> list[dict]:
cur = conn.execute(
"SELECT * FROM edges WHERE source_id = ? ORDER BY target_id",
(source_id,),
)
rows = cur.fetchall()
cols = [c[1] for c in conn.execute("PRAGMA table_info(edges)").fetchall()]
out: list[dict] = []
for row in rows:
d = dict(zip(cols, row))
d["knowledge"] = json.loads(d.pop("knowledge_json"))
out.append(d)
return out
+110
View File
@@ -0,0 +1,110 @@
from __future__ import annotations
import json
from sqlite3 import Connection
from chat.eventlog.projector import on
from chat.eventlog.log import Event
@on("bot_authored")
def _apply_bot_authored(conn: Connection, e: Event) -> None:
p = e.payload
conn.execute(
"INSERT OR REPLACE INTO bots "
"(id, name, persona, voice_samples_json, traits_json, backstory, "
" initial_relationship_to_you, kickoff_prose) "
"VALUES (?, ?, ?, ?, ?, ?, ?, ?)",
(p["id"], p["name"], p["persona"],
json.dumps(p.get("voice_samples", [])),
json.dumps(p.get("traits", [])),
p.get("backstory", ""),
p.get("initial_relationship_to_you", ""),
p.get("kickoff_prose", "")),
)
@on("you_authored")
def _apply_you_authored(conn: Connection, e: Event) -> None:
p = e.payload
conn.execute(
"INSERT OR REPLACE INTO you_entity (id, name, pronouns, persona) VALUES (1, ?, ?, ?)",
(p["name"], p.get("pronouns", ""), p.get("persona", "")),
)
@on("bot_reset")
def _apply_bot_reset(conn: Connection, e: Event) -> None:
"""Purge per-bot runtime state while preserving the bot's identity row.
Wipes chats hosted by this bot (with cascading chat-scoped tables),
memories owned by this bot, edges involving this bot, and the bot's own
activity row. The ``bots`` row itself is preserved so identity,
initial-relationship, and kickoff prose remain authored.
"""
bot_id = e.payload["bot_id"]
chat_ids = [
row[0]
for row in conn.execute(
"SELECT id FROM chats WHERE host_bot_id = ?", (bot_id,)
).fetchall()
]
# T69: purge orphaned "you" activity rows pointing at containers in this
# bot's chats BEFORE the containers/chats themselves are deleted, otherwise
# the subqueries find nothing and the FK constraint on activity.container_id
# blocks the container delete.
conn.execute(
"DELETE FROM activity WHERE entity_id = 'you' "
"AND container_id IN (SELECT id FROM containers WHERE chat_id IN ("
" SELECT id FROM chats WHERE host_bot_id = ?"
"))",
(bot_id,),
)
for chat_id in chat_ids:
conn.execute("DELETE FROM scenes WHERE chat_id = ?", (chat_id,))
conn.execute("DELETE FROM containers WHERE chat_id = ?", (chat_id,))
conn.execute("DELETE FROM chat_state WHERE chat_id = ?", (chat_id,))
conn.execute("DELETE FROM chats WHERE id = ?", (chat_id,))
# Activity for this bot's entity row (independent of chat_id since the
# ``activity`` table is keyed on entity_id).
conn.execute("DELETE FROM activity WHERE entity_id = ?", (bot_id,))
# Memories authored by this bot.
conn.execute("DELETE FROM memories WHERE owner_id = ?", (bot_id,))
# Edges in either direction involving this bot.
conn.execute(
"DELETE FROM edges WHERE source_id = ? OR target_id = ?",
(bot_id, bot_id),
)
# Phase 2 cascade: clear guest references in other bots' chats so the host
# doesn't see a stale guest_bot_id pointing at this (now-purged) bot.
conn.execute(
"UPDATE chats SET guest_bot_id = NULL WHERE guest_bot_id = ?",
(bot_id,),
)
# NOTE: bots row itself is preserved (identity, kickoff_prose intact).
def get_bot(conn: Connection, bot_id: str) -> dict | None:
row = conn.execute("SELECT * FROM bots WHERE id = ?", (bot_id,)).fetchone()
if not row:
return None
cols = [c[1] for c in conn.execute("PRAGMA table_info(bots)").fetchall()]
d = dict(zip(cols, row))
d["voice_samples"] = json.loads(d.pop("voice_samples_json"))
d["traits"] = json.loads(d.pop("traits_json"))
return d
def list_bots(conn: Connection) -> list[dict]:
cur = conn.execute("SELECT id, name FROM bots ORDER BY name")
return [{"id": r[0], "name": r[1]} for r in cur]
def get_you(conn: Connection) -> dict | None:
row = conn.execute("SELECT name, pronouns, persona FROM you_entity WHERE id = 1").fetchone()
if not row:
return None
return {"name": row[0], "pronouns": row[1], "persona": row[2]}
+50
View File
@@ -0,0 +1,50 @@
from __future__ import annotations
import json
from sqlite3 import Connection
from chat.eventlog.projector import on
from chat.eventlog.log import Event
@on("group_node_initialized")
def _apply_group_node_initialized(conn: Connection, e: Event) -> None:
p = e.payload
conn.execute(
"INSERT OR REPLACE INTO group_node "
"(chat_id, members_json, summary, dynamic, threads_json) "
"VALUES (?, ?, ?, ?, ?)",
(
p["chat_id"],
json.dumps(p["members"]),
p.get("summary", ""),
p.get("dynamic", ""),
json.dumps(p.get("threads", [])),
),
)
@on("group_node_updated")
def _apply_group_node_updated(conn: Connection, e: Event) -> None:
p = e.payload
conn.execute(
"UPDATE group_node SET summary = ?, dynamic = ?, updated_at = datetime('now') "
"WHERE chat_id = ?",
(p.get("summary", ""), p.get("dynamic", ""), p["chat_id"]),
)
def get_group_node(conn: Connection, chat_id: str) -> dict | None:
row = conn.execute(
"SELECT chat_id, members_json, summary, dynamic, threads_json, updated_at "
"FROM group_node WHERE chat_id = ?",
(chat_id,),
).fetchone()
if not row:
return None
return {
"chat_id": row[0],
"members": json.loads(row[1]),
"summary": row[2],
"dynamic": row[3],
"threads": json.loads(row[4]),
"updated_at": row[5],
}
+142
View File
@@ -0,0 +1,142 @@
"""Handler for ``manual_edit`` events (T25, §6.4 final paragraph).
A ``manual_edit`` event captures a user override of a projected field — its
payload snapshots both the prior value and the new value so any edit can
be reversed by emitting an inverse ``manual_edit`` later. This module
applies the new value to the appropriate target table; the snapshot of
``prior_value`` is taken by the route handler before this fires.
Phase 1 covers five target kinds:
- ``edge_affinity`` and ``edge_trust`` — slider edits on a specific edge,
clamped to 0..100.
- ``memory_significance`` — dropdown edit, clamped to 0..3.
- ``memory_pov_summary`` — textarea edit (string). Also reused by T27's
scene-close pipeline to rewrite per-turn raw narratives into a proper
per-POV scene summary.
- ``edge_summary`` — string overwrite of the directed edge's ``summary``
field. Driven by T27 from the classifier's ``relationship_summary``
output combined with the prior summary.
T72.1 (Phase 2.5) adds one list-shaped edit:
- ``edge_knowledge_fact`` — add/remove a single fact on an edge's
``knowledge_json`` list. Payload carries an ``action`` of ``"add"`` or
``"remove"`` and a ``fact`` string; remove matches the first occurrence
by string equality so the route handler doesn't have to track fact
indices across re-renders.
T72.3 adds a per-flag witness toggle:
- ``memory_witness`` — flip one of ``witness_you`` / ``witness_host`` /
``witness_guest`` on a memory row. Payload's ``new_value`` is a dict
``{"flag": "you"|"host"|"guest", "value": 0|1}`` and ``prior_value``
mirrors the same shape so an inverse edit can restore the flag.
Pin toggles intentionally use the existing ``memory_pin_changed`` event
(registered in :mod:`chat.state.memory`) rather than ``manual_edit`` so
the projection writes both ``pinned`` and ``auto_pinned`` atomically.
"""
from __future__ import annotations
import json
from sqlite3 import Connection
from chat.eventlog.log import Event
from chat.eventlog.projector import on
_VALID_WITNESS_FLAGS = {"you", "host", "guest"}
def _clamp(value: int, lo: int, hi: int) -> int:
return max(lo, min(hi, value))
@on("manual_edit")
def _apply_manual_edit(conn: Connection, e: Event) -> None:
p = e.payload
kind = p["target_kind"]
target_id = p["target_id"]
new_value = p["new_value"]
if kind == "edge_affinity":
conn.execute(
"UPDATE edges SET affinity = ? "
"WHERE source_id = ? AND target_id = ?",
(
_clamp(int(new_value), 0, 100),
target_id["source_id"],
target_id["target_id"],
),
)
elif kind == "edge_trust":
conn.execute(
"UPDATE edges SET trust = ? "
"WHERE source_id = ? AND target_id = ?",
(
_clamp(int(new_value), 0, 100),
target_id["source_id"],
target_id["target_id"],
),
)
elif kind == "memory_significance":
conn.execute(
"UPDATE memories SET significance = ? WHERE id = ?",
(_clamp(int(new_value), 0, 3), int(target_id)),
)
elif kind == "memory_pov_summary":
conn.execute(
"UPDATE memories SET pov_summary = ? WHERE id = ?",
(str(new_value), int(target_id)),
)
elif kind == "edge_summary":
# ``target_id`` here is a {"source_id", "target_id"} pair like
# the affinity/trust edits, since edges are keyed by the
# directed pair, not a single rowid.
conn.execute(
"UPDATE edges SET summary = ? "
"WHERE source_id = ? AND target_id = ?",
(
str(new_value),
target_id["source_id"],
target_id["target_id"],
),
)
elif kind == "edge_knowledge_fact":
# T72.1: add or remove a single fact on an edge's knowledge list.
# ``target_id`` is the {"source_id", "target_id"} edge pair;
# ``new_value`` carries ``{"action": "add"|"remove", "fact": str}``.
# Remove matches by string equality (first occurrence) so callers
# don't have to thread a fact_index through re-rendered drawers.
action = new_value["action"]
fact = str(new_value["fact"])
row = conn.execute(
"SELECT knowledge_json FROM edges "
"WHERE source_id = ? AND target_id = ?",
(target_id["source_id"], target_id["target_id"]),
).fetchone()
if row is not None:
knowledge = json.loads(row[0])
if action == "add":
knowledge.append(fact)
elif action == "remove" and fact in knowledge:
knowledge.remove(fact)
conn.execute(
"UPDATE edges SET knowledge_json = ? "
"WHERE source_id = ? AND target_id = ?",
(
json.dumps(knowledge),
target_id["source_id"],
target_id["target_id"],
),
)
elif kind == "memory_witness":
# T72.3: toggle one of the three witness flags on a memory row.
# ``new_value`` is the dict ``{"flag", "value"}``; ``prior_value``
# mirrors the same shape so an inverse edit restores the flag.
flag = new_value["flag"]
if flag in _VALID_WITNESS_FLAGS:
conn.execute(
f"UPDATE memories SET witness_{flag} = ? WHERE id = ?",
(1 if int(new_value["value"]) else 0, int(target_id)),
)
# Unknown target_kind: silently no-op for v1. Future kinds (activity
# fields, etc.) extend the dispatch above.
+166
View File
@@ -0,0 +1,166 @@
from __future__ import annotations
from sqlite3 import Connection
from chat.eventlog.projector import on
from chat.eventlog.log import Event
_VALID_WITNESS_ROLES = {"you", "host", "guest"}
def _row_to_dict(conn: Connection, row: tuple) -> dict:
cols = [c[1] for c in conn.execute("PRAGMA table_info(memories)").fetchall()]
return dict(zip(cols, row))
@on("memory_written")
def _apply_memory_written(conn: Connection, e: Event) -> None:
p = e.payload
conn.execute(
"INSERT INTO memories ("
"owner_id, chat_id, scene_id, pov_summary, "
"witness_you, witness_host, witness_guest, "
"chat_clock_at, source, reliability, significance, pinned, auto_pinned"
") VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)",
(
p["owner_id"],
p["chat_id"],
p.get("scene_id"),
p["pov_summary"],
int(p["witness_you"]),
int(p["witness_host"]),
int(p["witness_guest"]),
p.get("chat_clock_at"),
p.get("source", "direct"),
float(p.get("reliability", 1.0)),
int(p.get("significance", 1)),
int(p.get("pinned", 0)),
int(p.get("auto_pinned", 0)),
),
)
@on("memory_significance_set")
def _apply_memory_significance_set(conn: Connection, e: Event) -> None:
"""Update an existing memory's significance score (T22).
Emitted by the async significance worker after it scores the turn.
"""
p = e.payload
conn.execute(
"UPDATE memories SET significance = ? WHERE id = ?",
(int(p["significance"]), int(p["memory_id"])),
)
@on("memory_pin_changed")
def _apply_memory_pin_changed(conn: Connection, e: Event) -> None:
"""Toggle a memory's pin state (T22, §8.5).
Used both for auto-pinning a pivotal turn and for evicting the oldest
auto-pin when the per-owner soft cap is exceeded. Manual pins use the
same handler; the ``auto_pinned`` flag distinguishes them so the
eviction query can leave manual pins alone.
"""
p = e.payload
conn.execute(
"UPDATE memories SET pinned = ?, auto_pinned = ? WHERE id = ?",
(int(p["pinned"]), int(p["auto_pinned"]), int(p["memory_id"])),
)
def get_memory(conn: Connection, memory_id: int) -> dict | None:
row = conn.execute(
"SELECT * FROM memories WHERE id = ?", (memory_id,)
).fetchone()
if not row:
return None
return _row_to_dict(conn, row)
def get_pinned(conn: Connection, owner_id: str) -> list[dict]:
cur = conn.execute(
"SELECT * FROM memories WHERE owner_id = ? AND pinned = 1 "
"ORDER BY created_at DESC, id DESC",
(owner_id,),
)
rows = cur.fetchall()
cols = [c[1] for c in conn.execute("PRAGMA table_info(memories)").fetchall()]
return [dict(zip(cols, row)) for row in rows]
# Composite-score weights used by ``search_memories`` (T23, §8 retrieval).
# FTS5 BM25 ``rank`` is *more negative* for better matches, so subtracting a
# positive boost from it drives stronger candidates further down (i.e. earlier
# in an ascending sort). Hardcoded for v1 — tunable in a later pass.
_SIGNIFICANCE_WEIGHT = 0.3
_RECENCY_WEIGHT = 0.5
def search_memories(
conn: Connection,
owner_id: str,
witness_role: str,
query: str,
k: int = 4,
) -> list[dict]:
"""FTS5 search over pov_summary, scoped by owner and witness role.
witness_role must be one of {"you", "host", "guest"} per the witness flags
on each memory row. Returns up to ``k`` rows ranked by a composite score
that combines the FTS5 BM25 rank with two boosts (§8 retrieval rules):
* **significance boost** — ``0.3 * significance`` (0..3 per §11.1).
* **recency boost** — ``0.5 * (id / max_id)``, using the row id as a
monotonic recency proxy. Newer memories therefore tilt above older ones
when the BM25 rank and significance are otherwise tied.
BM25 returns negative scores (lower = better). Both boosts are subtracted
so that stronger candidates yield smaller composite scores; the result is
sorted ascending and truncated to ``k``. The unmodified ``fts_rank`` and a
debug-friendly ``composite_score`` are kept on each returned dict.
"""
if witness_role not in _VALID_WITNESS_ROLES:
raise ValueError(
f"witness_role must be one of {sorted(_VALID_WITNESS_ROLES)}, "
f"got {witness_role!r}"
)
if not query.strip():
return []
witness_col = f"witness_{witness_role}"
cols = [c[1] for c in conn.execute("PRAGMA table_info(memories)").fetchall()]
select_list = ", ".join(f"m.{c}" for c in cols)
# Over-fetch from FTS so the Python-side re-rank has room to reorder
# results that BM25 alone would have demoted past the top-k boundary.
over_fetch = max(k * 4, 20)
sql = (
f"SELECT {select_list}, memories_fts.rank AS fts_rank "
"FROM memories_fts "
"JOIN memories m ON m.id = memories_fts.rowid "
f"WHERE m.owner_id = ? AND m.{witness_col} = 1 "
"AND memories_fts MATCH ? "
"ORDER BY memories_fts.rank "
"LIMIT ?"
)
cur = conn.execute(sql, (owner_id, query, over_fetch))
rows = cur.fetchall()
if not rows:
return []
# Recency normalises against the current max id for this owner so the
# boost magnitude is bounded regardless of dataset size.
max_id_row = conn.execute(
"SELECT MAX(id) FROM memories WHERE owner_id = ?", (owner_id,)
).fetchone()
max_id = max_id_row[0] if max_id_row and max_id_row[0] else 1
result_cols = cols + ["fts_rank"]
enriched: list[dict] = []
for row in rows:
d = dict(zip(result_cols, row))
fts_rank = d.get("fts_rank") or 0.0
sig_boost = _SIGNIFICANCE_WEIGHT * (d.get("significance") or 0)
recency_boost = _RECENCY_WEIGHT * ((d.get("id") or 0) / max_id)
d["composite_score"] = fts_rank - sig_boost - recency_boost
enriched.append(d)
enriched.sort(key=lambda x: x["composite_score"])
return enriched[:k]
+220
View File
@@ -0,0 +1,220 @@
from __future__ import annotations
import json
from sqlite3 import Connection
from chat.eventlog.projector import on
from chat.eventlog.log import Event
def _row_to_dict(conn: Connection, table: str, row: tuple) -> dict:
cols = [c[1] for c in conn.execute(f"PRAGMA table_info({table})").fetchall()]
return dict(zip(cols, row))
@on("chat_created")
def _apply_chat_created(conn: Connection, e: Event) -> None:
p = e.payload
conn.execute(
"INSERT INTO chats (id, host_bot_id, guest_bot_id) VALUES (?, ?, ?)",
(p["id"], p["host_bot_id"], p.get("guest_bot_id")),
)
conn.execute(
"INSERT INTO chat_state (chat_id, time, weather, active_scene_id, narrative_anchor) "
"VALUES (?, ?, ?, NULL, ?)",
(
p["id"],
p["initial_time"],
p.get("weather", ""),
p.get("narrative_anchor", ""),
),
)
@on("guest_added")
def _apply_guest_added(conn: Connection, e: Event) -> None:
p = e.payload
conn.execute(
"UPDATE chats SET guest_bot_id = ? WHERE id = ?",
(p["guest_bot_id"], p["chat_id"]),
)
@on("guest_removed")
def _apply_guest_removed(conn: Connection, e: Event) -> None:
p = e.payload
conn.execute(
"UPDATE chats SET guest_bot_id = NULL WHERE id = ?",
(p["chat_id"],),
)
@on("container_created")
def _apply_container_created(conn: Connection, e: Event) -> None:
p = e.payload
conn.execute(
"INSERT INTO containers (chat_id, name, type, properties_json, parent_id) "
"VALUES (?, ?, ?, ?, ?)",
(
p["chat_id"],
p["name"],
p["type"],
json.dumps(p.get("properties", {})),
p.get("parent_id"),
),
)
@on("activity_change")
def _apply_activity_change(conn: Connection, e: Event) -> None:
p = e.payload
conn.execute(
"INSERT OR REPLACE INTO activity ("
"entity_id, container_id, slot, posture, action_json, "
"attention, holding_json, status_json, updated_at"
") VALUES (?, ?, ?, ?, ?, ?, ?, ?, datetime('now'))",
(
p["entity_id"],
p.get("container_id"),
p.get("slot"),
p.get("posture", ""),
json.dumps(p.get("action", {})),
p.get("attention", ""),
json.dumps(p.get("holding", [])),
json.dumps(p.get("status", {})),
),
)
@on("scene_opened")
def _apply_scene_opened(conn: Connection, e: Event) -> None:
p = e.payload
cur = conn.execute(
"INSERT INTO scenes (chat_id, container_id, started_at, ended_at, "
"significance, participants_json) VALUES (?, ?, ?, NULL, 0, ?)",
(
p["chat_id"],
p.get("container_id"),
p["started_at"],
json.dumps(p.get("participants", [])),
),
)
new_id = cur.lastrowid
conn.execute(
"UPDATE chat_state SET active_scene_id = ? WHERE chat_id = ?",
(new_id, p["chat_id"]),
)
@on("scene_closed")
def _apply_scene_closed(conn: Connection, e: Event) -> None:
p = e.payload
scene_id = p["scene_id"]
significance = int(p.get("significance", 0))
conn.execute(
"UPDATE scenes SET ended_at = ?, significance = ? WHERE id = ?",
(p["ended_at"], significance, scene_id),
)
row = conn.execute(
"SELECT chat_id FROM scenes WHERE id = ?", (scene_id,)
).fetchone()
if row is not None:
chat_id = row[0]
conn.execute(
"UPDATE chat_state SET active_scene_id = NULL WHERE chat_id = ?",
(chat_id,),
)
def _chat_select_columns() -> str:
return (
"c.id, c.host_bot_id, c.guest_bot_id, c.created_at, "
"s.time, s.weather, s.active_scene_id, s.narrative_anchor"
)
def _chat_row_to_dict(row: tuple) -> dict:
return {
"id": row[0],
"host_bot_id": row[1],
"guest_bot_id": row[2],
"created_at": row[3],
"time": row[4],
"weather": row[5],
"active_scene_id": row[6],
"narrative_anchor": row[7],
}
def get_chat(conn: Connection, chat_id: str) -> dict | None:
row = conn.execute(
f"SELECT {_chat_select_columns()} FROM chats c "
"JOIN chat_state s ON s.chat_id = c.id WHERE c.id = ?",
(chat_id,),
).fetchone()
if not row:
return None
return _chat_row_to_dict(row)
def list_chats(conn: Connection) -> list[dict]:
cur = conn.execute(
f"SELECT {_chat_select_columns()} FROM chats c "
"JOIN chat_state s ON s.chat_id = c.id ORDER BY c.id"
)
return [_chat_row_to_dict(row) for row in cur.fetchall()]
def get_container(conn: Connection, container_id: int) -> dict | None:
row = conn.execute(
"SELECT * FROM containers WHERE id = ?", (container_id,)
).fetchone()
if not row:
return None
d = _row_to_dict(conn, "containers", row)
d["properties"] = json.loads(d.pop("properties_json"))
return d
def find_container(conn: Connection, chat_id: str, name: str) -> dict | None:
row = conn.execute(
"SELECT * FROM containers WHERE chat_id = ? AND name = ?",
(chat_id, name),
).fetchone()
if not row:
return None
d = _row_to_dict(conn, "containers", row)
d["properties"] = json.loads(d.pop("properties_json"))
return d
def get_activity(conn: Connection, entity_id: str) -> dict | None:
row = conn.execute(
"SELECT * FROM activity WHERE entity_id = ?", (entity_id,)
).fetchone()
if not row:
return None
d = _row_to_dict(conn, "activity", row)
d["action"] = json.loads(d.pop("action_json"))
d["holding"] = json.loads(d.pop("holding_json"))
d["status"] = json.loads(d.pop("status_json"))
return d
def get_scene(conn: Connection, scene_id: int) -> dict | None:
row = conn.execute(
"SELECT * FROM scenes WHERE id = ?", (scene_id,)
).fetchone()
if not row:
return None
d = _row_to_dict(conn, "scenes", row)
d["participants"] = json.loads(d.pop("participants_json"))
return d
def active_scene(conn: Connection, chat_id: str) -> dict | None:
row = conn.execute(
"SELECT active_scene_id FROM chat_state WHERE chat_id = ?",
(chat_id,),
).fetchone()
if not row or row[0] is None:
return None
return get_scene(conn, row[0])
+125
View File
@@ -0,0 +1,125 @@
* { box-sizing: border-box; }
body {
font: 15px/1.5 system-ui, -apple-system, "Segoe UI", sans-serif;
margin: 0;
color: #1c1c1c;
background: #fafafa;
display: flex;
min-height: 100vh;
}
.rail {
width: 200px;
background: #1c1c1c;
color: #fff;
padding: 16px;
flex-shrink: 0;
}
.rail a { color: #fff; text-decoration: none; }
.rail-brand {
font-weight: 600;
display: block;
padding-bottom: 16px;
border-bottom: 1px solid #333;
margin-bottom: 16px;
}
.rail ul { list-style: none; padding: 0; margin: 0; }
.rail li { margin: 4px 0; }
.rail li a { display: block; padding: 6px 8px; border-radius: 3px; }
.rail li a.active { background: #333; }
.content {
flex: 1;
padding: 24px;
background: #fafafa;
overflow: auto;
}
.brand { font-weight: 600; text-decoration: none; color: inherit; }
.container { max-width: 720px; margin: 24px auto; padding: 0 16px; }
h1 { margin-top: 0; }
.page-header { display: flex; align-items: center; justify-content: space-between; }
.btn, button {
display: inline-block; padding: 8px 14px;
border: 1px solid #444; background: #1c1c1c; color: #fff;
border-radius: 4px; text-decoration: none; cursor: pointer;
font: inherit;
}
.bot-form label { display: block; margin-bottom: 14px; }
.bot-form label span { display: block; font-weight: 600; margin-bottom: 4px; }
.bot-form input[type=text], .bot-form textarea {
width: 100%; padding: 6px 8px; font: inherit;
border: 1px solid #ccc; border-radius: 3px; background: #fff;
}
.bot-form small { display: block; color: #666; margin-top: 2px; }
.bot-list { list-style: none; padding: 0; }
.bot-list li { padding: 8px 0; border-bottom: 1px solid #eee; }
.chat-list { list-style: none; padding: 0; margin: 0; }
.chat-row { border-bottom: 1px solid #eee; }
.chat-row a { display: block; padding: 12px 0; text-decoration: none; color: inherit; }
.chat-row a:hover { background: #f0f0f0; }
.chat-row-name { font-weight: 600; }
.chat-row-snippet { font-size: 14px; }
.chat-row-meta { font-size: 12px; }
.muted { color: #666; }
.error {
padding: 8px 12px; border: 1px solid #c33; background: #fdecea;
color: #a00; border-radius: 3px;
}
.success {
padding: 8px 12px; border: 1px solid #2d7a3a; background: #eafaf0;
color: #1f5c2a; border-radius: 3px;
}
code { font-family: ui-monospace, "SF Mono", Menlo, monospace; }
.chat-shell { display: flex; flex-direction: column; height: 100%; max-width: 760px; margin: 0 auto; }
.chat-header { display: flex; align-items: center; gap: 16px; border-bottom: 1px solid #e5e5e5; padding-bottom: 8px; margin-bottom: 16px; }
.chat-header h1 { margin: 0; flex: 1; }
.chat-meta { font-size: 13px; }
.drawer-toggle { padding: 4px 10px; border: 1px solid #ccc; background: #fff; color: #1c1c1c; border-radius: 3px; cursor: pointer; }
.timeline { flex: 1; overflow-y: auto; min-height: 200px; padding: 8px 0; }
.turn { margin: 12px 0; }
.turn strong { display: block; margin-bottom: 4px; }
.turn p { margin: 0 0 8px; }
.turn p:last-child { margin-bottom: 0; }
.turn-you strong { color: #1a73e8; }
.turn-bot strong { color: #1c1c1c; }
/* ``*action*`` — italic narration. */
.action { font-style: italic; color: #555; }
/* ``((ooc))`` — author-to-system aside. Dim, italic, smaller, set off
from surrounding prose so it doesn't read as in-fiction speech. */
.ooc {
font-style: italic;
font-size: 12px;
color: #999;
display: inline-block;
background: rgba(0, 0, 0, 0.04);
padding: 1px 4px;
border-radius: 3px;
}
.turn blockquote {
border-left: 3px solid #ccc;
padding-left: 12px;
margin: 8px 0;
color: #555;
}
.turn-input { display: flex; flex-direction: column; gap: 8px; padding-top: 12px; border-top: 1px solid #e5e5e5; }
.turn-input textarea { padding: 8px; font: inherit; border: 1px solid #ccc; border-radius: 3px; resize: vertical; }
.drawer { position: fixed; top: 0; right: 0; width: 360px; height: 100vh; background: #fff; border-left: 1px solid #e5e5e5; padding: 16px; overflow-y: auto; z-index: 10; }
.drawer[hidden] { display: none; }
.drawer-content { display: flex; flex-direction: column; gap: 16px; }
.drawer-header { display: flex; align-items: center; justify-content: space-between; padding-bottom: 8px; border-bottom: 1px solid #e5e5e5; }
.drawer-close { border: none; background: transparent; color: #1c1c1c; font-size: 24px; padding: 0 4px; cursor: pointer; }
.drawer-section h3 { margin: 0 0 8px; font-size: 14px; text-transform: uppercase; letter-spacing: 0.5px; color: #666; }
.activity-row, .edge-row { margin-bottom: 12px; }
.activity-row strong, .edge-row strong { display: block; }
.memory-list { list-style: none; padding: 0; margin: 0; }
.memory-list li { padding: 4px 0; font-size: 13px; }
.sig { display: inline-block; min-width: 16px; }
.sig-3 { color: #d4af37; }
/* Streaming UX (T34): typing indicator, Stop button, disconnect banner. */
.streaming { opacity: 0.85; }
.streaming-text:after {
content: "\025AE";
margin-left: 2px;
animation: blink 1s steps(2, start) infinite;
}
@keyframes blink { to { visibility: hidden; } }
.stop-streaming { background: #c33; border-color: #a00; margin-bottom: 8px; align-self: flex-start; }
.connection-lost { margin-bottom: 8px; }
+384
View File
@@ -0,0 +1,384 @@
<div class="drawer-content">
<header class="drawer-header">
<h2>{{ host_bot.name }}</h2>
<button class="drawer-close" type="button"
onclick="document.getElementById('drawer').setAttribute('hidden','')">&times;</button>
</header>
<section class="drawer-section">
<h3>Scene</h3>
{% if scene %}
<p>Started: {{ scene.started_at }}</p>
{% endif %}
{% if container %}
<p>Container: {{ container.name }} ({{ container.type }})</p>
{% else %}
<p class="muted">No active container.</p>
{% endif %}
<p>Time: {{ chat.time }}</p>
{% if scene %}
<form class="inline-edit"
hx-post="/chats/{{ chat.id }}/drawer/scene/close"
hx-target="#drawer" hx-swap="innerHTML">
<button type="submit">Close scene</button>
</form>
{% else %}
<p class="muted">No active scene.</p>
{% endif %}
</section>
<section class="drawer-section">
<h3>Activity</h3>
{% for label, act in [("you", you_activity), (host_bot.name, bot_activity)] %}
<div class="activity-row">
<strong>{{ label }}</strong>
{% if act %}
<p>{{ act.posture or "—" }} / {{ (act.action or {}).verb or "—" }}</p>
{% if act.attention %}<p class="muted">attention: {{ act.attention }}</p>{% endif %}
{% if act.holding %}<p class="muted">holding: {{ act.holding|join(", ") }}</p>{% endif %}
{% else %}
<p class="muted">No activity recorded.</p>
{% endif %}
</div>
{% endfor %}
</section>
{% if guest_bot %}
<section class="drawer-section">
<h3>Guest</h3>
<p><strong>{{ guest_bot.name }}</strong></p>
{% if guest_activity %}
<p>{{ guest_activity.posture or "—" }} / {{ (guest_activity.action or {}).verb or "—" }}</p>
{% if guest_activity.attention %}<p class="muted">attention: {{ guest_activity.attention }}</p>{% endif %}
{% if guest_activity.holding %}<p class="muted">holding: {{ guest_activity.holding|join(", ") }}</p>{% endif %}
{% else %}
<p class="muted">No activity recorded.</p>
{% endif %}
{% if edge_h2g %}
<div class="edge-row">
<strong>{{ host_bot.name }} &rarr; {{ guest_bot.name }}</strong>
<p>Affinity: {{ edge_h2g.affinity }}/100 &middot; Trust: {{ edge_h2g.trust }}/100</p>
{% if edge_h2g.knowledge %}
<details><summary>Knowledge ({{ edge_h2g.knowledge|length }})</summary>
<ul>{% for fact in edge_h2g.knowledge %}<li>{{ fact }}</li>{% endfor %}</ul>
</details>
{% endif %}
</div>
{% endif %}
{% if edge_g2h %}
<div class="edge-row">
<strong>{{ guest_bot.name }} &rarr; {{ host_bot.name }}</strong>
<p>Affinity: {{ edge_g2h.affinity }}/100 &middot; Trust: {{ edge_g2h.trust }}/100</p>
{% if edge_g2h.knowledge %}
<details><summary>Knowledge ({{ edge_g2h.knowledge|length }})</summary>
<ul>{% for fact in edge_g2h.knowledge %}<li>{{ fact }}</li>{% endfor %}</ul>
</details>
{% endif %}
</div>
{% endif %}
{% if edge_y2g %}
<div class="edge-row">
<strong>you &rarr; {{ guest_bot.name }}</strong>
<p>Affinity: {{ edge_y2g.affinity }}/100 &middot; Trust: {{ edge_y2g.trust }}/100</p>
</div>
{% endif %}
{% if edge_g2y %}
<div class="edge-row">
<strong>{{ guest_bot.name }} &rarr; you</strong>
<p>Affinity: {{ edge_g2y.affinity }}/100 &middot; Trust: {{ edge_g2y.trust }}/100</p>
</div>
{% endif %}
<form class="inline-edit"
hx-post="/chats/{{ chat.id }}/drawer/guest/remove"
hx-target="#drawer" hx-swap="innerHTML">
<button type="submit">Remove guest</button>
</form>
</section>
{% else %}
<section class="drawer-section">
<h3>Add guest</h3>
{% if available_guests %}
{% set first_guest_id = available_guests[0].id %}
{% set first_existing = existing_guest_edges.get(first_guest_id, False) %}
<form class="inline-edit add-guest-form"
hx-post="/chats/{{ chat.id }}/drawer/guest/add"
hx-target="#drawer" hx-swap="innerHTML">
<label>
Bot:
<select name="guest_bot_id" required class="add-guest-select">
{% for b in available_guests %}
<option value="{{ b.id }}"
data-existing-edge="{{ 'true' if existing_guest_edges.get(b.id) else 'false' }}">
{{ b.name }}{% if existing_guest_edges.get(b.id) %} (already met){% endif %}
</option>
{% endfor %}
</select>
</label>
<p class="muted add-guest-existing-note"
{% if not first_existing %}hidden{% endif %}>
they already know each other (edge exists from a prior chat)
</p>
<label class="add-guest-reseed-label"
{% if not first_existing %}hidden{% endif %}>
<input type="checkbox" name="reseed" value="1" class="add-guest-reseed">
re-seed anyway
</label>
<label>
Have they met before? Describe how (leave blank if not):
<textarea name="relationship_prose" rows="3"
class="add-guest-prose"
{% if first_existing %}disabled{% endif %}
placeholder="e.g. Old college friends who studied physics together."></textarea>
</label>
<button type="submit">Add guest</button>
</form>
<script>
(function () {
var form = document.currentScript.previousElementSibling;
while (form && !form.classList.contains('add-guest-form')) {
form = form.previousElementSibling;
}
if (!form) return;
var sel = form.querySelector('.add-guest-select');
var prose = form.querySelector('.add-guest-prose');
var reseed = form.querySelector('.add-guest-reseed');
var note = form.querySelector('.add-guest-existing-note');
var reseedLabel = form.querySelector('.add-guest-reseed-label');
function refresh() {
var opt = sel.options[sel.selectedIndex];
var existing = opt && opt.getAttribute('data-existing-edge') === 'true';
if (existing) {
note.removeAttribute('hidden');
reseedLabel.removeAttribute('hidden');
prose.disabled = !reseed.checked;
} else {
note.setAttribute('hidden', '');
reseedLabel.setAttribute('hidden', '');
reseed.checked = false;
prose.disabled = false;
}
}
sel.addEventListener('change', refresh);
reseed.addEventListener('change', refresh);
refresh();
})();
</script>
{% else %}
<p class="muted">No other bots authored yet.</p>
{% endif %}
</section>
{% endif %}
{% if group_node %}
<section class="drawer-section">
<h3>Group</h3>
{% if group_node.summary %}
<p>{{ group_node.summary }}</p>
{% else %}
<p class="muted">No group summary yet.</p>
{% endif %}
{% if group_node.dynamic %}
<p class="muted">Dynamic: {{ group_node.dynamic }}</p>
{% endif %}
</section>
{% endif %}
<section class="drawer-section">
<h3>Edges</h3>
{% if edge_b2y %}
<div class="edge-row">
<strong>{{ host_bot.name }} &rarr; you</strong>
<p>Affinity: {{ edge_b2y.affinity }}/100 &middot; Trust: {{ edge_b2y.trust }}/100</p>
<form class="inline-edit"
hx-post="/chats/{{ chat.id }}/drawer/edge/{{ host_bot.id }}/you/affinity"
hx-target="#drawer" hx-swap="innerHTML">
<label>
Affinity:
<input type="range" name="affinity" min="0" max="100"
value="{{ edge_b2y.affinity }}"
oninput="this.nextElementSibling.value = this.value">
<output>{{ edge_b2y.affinity }}</output>
</label>
<button type="submit">Save</button>
</form>
<form class="inline-edit"
hx-post="/chats/{{ chat.id }}/drawer/edge/trust"
hx-target="#drawer" hx-swap="innerHTML">
<input type="hidden" name="source_id" value="{{ host_bot.id }}">
<input type="hidden" name="target_id" value="you">
<label>
Trust:
<input type="range" name="new_value" min="0" max="100"
value="{{ edge_b2y.trust }}"
oninput="this.nextElementSibling.value = this.value">
<output>{{ edge_b2y.trust }}</output>
</label>
<button type="submit">Save</button>
</form>
<form class="inline-edit"
hx-post="/chats/{{ chat.id }}/drawer/edge/summary"
hx-target="#drawer" hx-swap="innerHTML">
<input type="hidden" name="source_id" value="{{ host_bot.id }}">
<input type="hidden" name="target_id" value="you">
<label>
Summary:
<textarea name="new_summary" rows="3" maxlength="2000">{{ edge_b2y.summary or "" }}</textarea>
</label>
<button type="submit">Save summary</button>
</form>
<details>
<summary>Knowledge ({{ (edge_b2y.knowledge or [])|length }})</summary>
{% if edge_b2y.knowledge %}
<ul>
{% for fact in edge_b2y.knowledge %}
<li>
{{ fact }}
<form class="inline-edit"
hx-post="/chats/{{ chat.id }}/drawer/edge/knowledge-facts"
hx-target="#drawer" hx-swap="innerHTML">
<input type="hidden" name="source_id" value="{{ host_bot.id }}">
<input type="hidden" name="target_id" value="you">
<input type="hidden" name="action" value="remove">
<input type="hidden" name="fact" value="{{ fact }}">
<button type="submit">Remove</button>
</form>
</li>
{% endfor %}
</ul>
{% endif %}
<form class="inline-edit"
hx-post="/chats/{{ chat.id }}/drawer/edge/knowledge-facts"
hx-target="#drawer" hx-swap="innerHTML">
<input type="hidden" name="source_id" value="{{ host_bot.id }}">
<input type="hidden" name="target_id" value="you">
<input type="hidden" name="action" value="add">
<label>
Add fact:
<input type="text" name="fact" maxlength="500" required>
</label>
<button type="submit">Add</button>
</form>
</details>
</div>
{% endif %}
{% if edge_y2b %}
<div class="edge-row">
<strong>you &rarr; {{ host_bot.name }}</strong>
<p>Affinity: {{ edge_y2b.affinity }}/100 &middot; Trust: {{ edge_y2b.trust }}/100</p>
<form class="inline-edit"
hx-post="/chats/{{ chat.id }}/drawer/edge/trust"
hx-target="#drawer" hx-swap="innerHTML">
<input type="hidden" name="source_id" value="you">
<input type="hidden" name="target_id" value="{{ host_bot.id }}">
<label>
Trust:
<input type="range" name="new_value" min="0" max="100"
value="{{ edge_y2b.trust }}"
oninput="this.nextElementSibling.value = this.value">
<output>{{ edge_y2b.trust }}</output>
</label>
<button type="submit">Save</button>
</form>
<form class="inline-edit"
hx-post="/chats/{{ chat.id }}/drawer/edge/summary"
hx-target="#drawer" hx-swap="innerHTML">
<input type="hidden" name="source_id" value="you">
<input type="hidden" name="target_id" value="{{ host_bot.id }}">
<label>
Summary:
<textarea name="new_summary" rows="3" maxlength="2000">{{ edge_y2b.summary or "" }}</textarea>
</label>
<button type="submit">Save summary</button>
</form>
</div>
{% endif %}
{% if not edge_b2y and not edge_y2b %}
<p class="muted">No edges yet.</p>
{% endif %}
</section>
<section class="drawer-section">
<h3>Pinned memories ({{ pinned|length }} / {{ pin_cap }})</h3>
{% if pinned %}
<ul class="memory-list">
{% for m in pinned %}
<li>
<span class="sig sig-{{ m.significance }}">{{ ['·','•','★','★★'][m.significance|default(0)] }}</span>
{{ m.pov_summary }}
<form class="inline-edit"
hx-post="/chats/{{ chat.id }}/drawer/memory/{{ m.id }}/pin"
hx-target="#drawer" hx-swap="innerHTML">
<input type="hidden" name="pinned" value="0">
<button type="submit">Unpin</button>
</form>
</li>
{% endfor %}
</ul>
{% else %}
<p class="muted">No pinned memories.</p>
{% endif %}
</section>
<section class="drawer-section">
<h3>Recent memories</h3>
{% if recent_memories %}
<ul class="memory-list">
{% for m in recent_memories %}
<li>
<span class="sig sig-{{ m.significance }}">{{ ['·','•','★','★★'][m.significance|default(0)] }}</span>
{{ m.pov_summary[:200] }}{% if m.pov_summary|length > 200 %}…{% endif %}
<form class="inline-edit"
hx-post="/chats/{{ chat.id }}/drawer/memory/{{ m.id }}/significance"
hx-target="#drawer" hx-swap="innerHTML">
<select name="significance">
{% for s in [0, 1, 2, 3] %}
<option value="{{ s }}" {% if m.significance == s %}selected{% endif %}>
{{ ['·','•','★','★★'][s] }} ({{ s }})
</option>
{% endfor %}
</select>
<button type="submit">Set</button>
</form>
<form class="inline-edit"
hx-post="/chats/{{ chat.id }}/drawer/memory/{{ m.id }}/pin"
hx-target="#drawer" hx-swap="innerHTML">
<input type="hidden" name="pinned" value="{{ 0 if m.pinned else 1 }}">
<button type="submit">{{ 'Unpin' if m.pinned else 'Pin' }}</button>
</form>
<div class="witness-row">
{% for flag in ['you', 'host', 'guest'] %}
{% set witnessed = m['witness_' ~ flag] %}
<form class="inline-edit"
hx-post="/chats/{{ chat.id }}/drawer/memory/witness"
hx-target="#drawer" hx-swap="innerHTML">
<input type="hidden" name="memory_id" value="{{ m.id }}">
<input type="hidden" name="flag" value="{{ flag }}">
<input type="hidden" name="new_value" value="{{ 0 if witnessed else 1 }}">
<label>
<input type="checkbox" {% if witnessed %}checked{% endif %}
onchange="this.form.requestSubmit()">
{{ flag }}
</label>
</form>
{% endfor %}
</div>
<details>
<summary>Edit POV summary</summary>
<form class="inline-edit"
hx-post="/chats/{{ chat.id }}/drawer/memory/pov-summary"
hx-target="#drawer" hx-swap="innerHTML">
<input type="hidden" name="memory_id" value="{{ m.id }}">
<textarea name="new_summary" rows="3" maxlength="2000">{{ m.pov_summary }}</textarea>
<button type="submit">Save</button>
</form>
</details>
</li>
{% endfor %}
</ul>
{% else %}
<p class="muted">No memories yet.</p>
{% endif %}
</section>
</div>
+13
View File
@@ -0,0 +1,13 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>{% block title %}chat{% endblock %}</title>
<link rel="stylesheet" href="/static/app.css">
<script src="https://unpkg.com/htmx.org@1.9.12" defer></script>
</head>
<body>
{% block body %}{% endblock %}
</body>
</html>
+57
View File
@@ -0,0 +1,57 @@
{% extends "layout.html" %}
{% block title %}New bot - chat{% endblock %}
{% block content %}
<h1>New bot</h1>
{% if error %}
<p class="error">{{ error }}</p>
{% endif %}
<form method="post" action="/bots/new" class="bot-form">
<label>
<span>id</span>
<input type="text" name="id" required value="{{ values.id|default('', true) }}">
<small>slug-like identifier (e.g. <code>bot_a</code>, <code>alice_office</code>)</small>
</label>
<label>
<span>name</span>
<input type="text" name="name" required value="{{ values.name|default('', true) }}">
</label>
<label>
<span>persona</span>
<textarea name="persona" rows="4" required>{{ values.persona|default('', true) }}</textarea>
<small>a short description, ~3-5 lines</small>
</label>
<label>
<span>voice samples</span>
<textarea name="voice_samples" rows="6">{{ values.voice_samples|default('', true) }}</textarea>
<small>1-3 samples, separated by a line containing only <code>---</code></small>
</label>
<label>
<span>traits</span>
<textarea name="traits" rows="3">{{ values.traits|default('', true) }}</textarea>
<small>comma- or newline-separated; 3-15 typical</small>
</label>
<label>
<span>backstory</span>
<textarea name="backstory" rows="6">{{ values.backstory|default('', true) }}</textarea>
<small>100-500 words target</small>
</label>
<label>
<span>initial relationship to you</span>
<textarea name="initial_relationship_to_you" rows="3" required>{{ values.initial_relationship_to_you|default('', true) }}</textarea>
</label>
<label>
<span>kickoff prose</span>
<textarea name="kickoff_prose" rows="4" required>{{ values.kickoff_prose|default('', true) }}</textarea>
<small>a short opening scene; parsed in the next step</small>
</label>
<button type="submit">Save bot</button>
</form>
{% endblock %}
+28
View File
@@ -0,0 +1,28 @@
{% extends "layout.html" %}
{% block title %}Bots - chat{% endblock %}
{% block content %}
<header class="page-header">
<h1>Bots</h1>
<a class="btn" href="/bots/new">+ New bot</a>
</header>
{% if bots %}
<ul class="bot-list">
{% for bot in bots %}
<li>
<a href="/bots/{{ bot.id }}">{{ bot.name }}</a>
<details class="bot-row-reset">
<summary>Reset</summary>
<form method="post" action="/bots/{{ bot.id }}/reset" class="inline-edit">
<label>Type "{{ bot.name }}" to confirm:
<input type="text" name="confirm_name" required>
</label>
<button type="submit">Reset bot</button>
</form>
</details>
</li>
{% endfor %}
</ul>
{% else %}
<p class="muted">No bots yet. <a href="/bots/new">Create your first bot.</a></p>
{% endif %}
{% endblock %}
+159
View File
@@ -0,0 +1,159 @@
{% extends "layout.html" %}
{% block title %}{{ host_bot.name }} - chat{% endblock %}
{% block content %}
<div class="chat-shell" data-chat-id="{{ chat.id }}"
hx-ext="sse"
sse-connect="/chats/{{ chat.id }}/events">
<header class="chat-header">
<h1>{{ host_bot.name }}</h1>
<div class="chat-meta muted">{{ chat.time }}</div>
<button class="drawer-toggle" type="button" aria-controls="drawer" aria-expanded="false">Drawer</button>
</header>
<section class="timeline" id="timeline"
sse-swap="turn_html"
hx-swap="beforeend">
{% if not turns %}
<p class="muted">No turns yet. Start typing below.</p>
{% else %}
{% for turn in turns %}
<div class="turn turn-{{ turn.role }}">
<strong>{{ turn.speaker }}</strong>
{{ turn.text|render_prose|safe }}
</div>
{% endfor %}
{% endif %}
</section>
<form class="turn-input" method="post" action="/chats/{{ chat.id }}/turns">
<textarea name="prose" rows="3" placeholder="What do you say or do?" required></textarea>
<button type="submit">Send</button>
</form>
<aside class="drawer" id="drawer" hidden
hx-get="/chats/{{ chat.id }}/drawer"
hx-trigger="revealed"
hx-swap="innerHTML">
<p class="muted">Loading drawer&hellip;</p>
</aside>
</div>
<script>
document.querySelector('.drawer-toggle')?.addEventListener('click', (e) => {
const drawer = document.getElementById('drawer');
const isHidden = drawer.hasAttribute('hidden');
if (isHidden) drawer.removeAttribute('hidden');
else drawer.setAttribute('hidden', '');
e.target.setAttribute('aria-expanded', String(isHidden));
});
</script>
<script>
// Streaming UX (T34): typing indicator, Stop button, send-lock,
// disconnect banner. Listens to the existing HTMX SSE channel for
// `token` (per-chunk) and `turn_html` (final swap) events. The
// mid-stream disconnect path is server-side: ``request.is_disconnected()``
// in T19 commits truncated; this script just shows the banner when
// the SSE EventSource fires `error` after the connection drops.
(function () {
const shell = document.querySelector('.chat-shell');
if (!shell) return;
const chatId = shell.dataset.chatId;
const form = shell.querySelector('.turn-input');
if (!form) return;
const textarea = form.querySelector('textarea[name="prose"]');
const sendBtn = form.querySelector('button[type="submit"]');
const timeline = document.getElementById('timeline');
let isStreaming = false;
let typingEl = null;
function ensureTypingEl() {
if (typingEl) return typingEl;
typingEl = document.createElement('div');
typingEl.className = 'turn turn-bot streaming';
typingEl.innerHTML = '<strong>...</strong><p class="streaming-text"></p>';
timeline.appendChild(typingEl);
return typingEl;
}
function unlock() {
isStreaming = false;
if (sendBtn) sendBtn.disabled = false;
if (textarea) {
textarea.readOnly = false;
textarea.value = '';
textarea.focus();
}
const stop = shell.querySelector('.stop-streaming');
if (stop) stop.remove();
}
function showBanner(msg) {
let banner = shell.querySelector('.connection-lost');
if (banner) return;
banner = document.createElement('div');
banner.className = 'connection-lost error';
banner.textContent = msg;
form.parentElement.insertBefore(banner, form);
}
// HTMX SSE extension dispatches `htmx:sseMessage` with detail.type
// (event name) and detail.data (payload string).
shell.addEventListener('htmx:sseMessage', (e) => {
const evt = e.detail.type;
const data = e.detail.data;
if (evt === 'token' && isStreaming) {
let parsed;
try { parsed = JSON.parse(data); } catch (_) { return; }
const el = ensureTypingEl();
el.querySelector('.streaming-text').textContent += (parsed.text || '');
} else if (evt === 'turn_html') {
// The server already pushes the final HTML via sse-swap on the
// timeline element; we just remove the typing placeholder and
// unlock the input. (Don't replace innerHTML here — HTMX has
// already done the append by the time this fires.)
if (typingEl) {
typingEl.remove();
typingEl = null;
}
unlock();
}
});
// SSE connection lost — show a banner and unlock so the user can
// retry. The server commits the partial as truncated when its
// request.is_disconnected() poll trips (T19).
shell.addEventListener('htmx:sseError', () => {
if (isStreaming) {
showBanner('connection lost — partial response saved');
unlock();
}
});
form.addEventListener('submit', () => {
isStreaming = true;
if (sendBtn) sendBtn.disabled = true;
// readOnly (not disabled) — disabled fields are excluded from the
// form submission, which would send prose="" and trigger the
// server's empty-prose 400.
if (textarea) textarea.readOnly = true;
if (!shell.querySelector('.stop-streaming')) {
const stopBtn = document.createElement('button');
stopBtn.type = 'button';
stopBtn.className = 'stop-streaming btn';
stopBtn.textContent = 'Stop';
stopBtn.addEventListener('click', async () => {
try {
await fetch('/chats/' + encodeURIComponent(chatId) + '/turns/cancel', {
method: 'POST',
});
} catch (_) {
// Network error on cancel is non-fatal — server will time out
// its own stream eventually and commit truncated.
}
});
form.parentElement.insertBefore(stopBtn, form);
}
});
})();
</script>
{% endblock %}
+26
View File
@@ -0,0 +1,26 @@
{% extends "layout.html" %}
{% block title %}Chats - chat{% endblock %}
{% block content %}
<header class="page-header">
<h1>Chats</h1>
<a class="btn" href="/bots/new">+ New bot</a>
</header>
{% if chats %}
<ul class="chat-list">
{% for chat in chats %}
<li class="chat-row">
<a href="/chats/{{ chat.id }}">
<div class="chat-row-name">{{ chat.host_bot_name }}</div>
<div class="chat-row-snippet muted">{{ chat.last_message_snippet or '—' }}</div>
<div class="chat-row-meta muted">
<span>{{ chat.time }}</span>
{% if chat.last_played_at %}<span>· {{ chat.last_played_at }}</span>{% endif %}
</div>
</a>
</li>
{% endfor %}
</ul>
{% else %}
<p class="muted">No chats yet. <a href="/bots/new">Create a bot</a> to start.</p>
{% endif %}
{% endblock %}
+9
View File
@@ -0,0 +1,9 @@
{% extends "layout.html" %}
{% block title %}Error - chat{% endblock %}
{% block content %}
<div class="error-page">
<h1>{{ status_code }}</h1>
<p>{{ detail }}</p>
<p><a href="/chats">Back to chats</a></p>
</div>
{% endblock %}
+118
View File
@@ -0,0 +1,118 @@
{% extends "layout.html" %}
{% block title %}Confirm kickoff - chat{% endblock %}
{% block content %}
<h1>Confirm kickoff</h1>
<p>Review and edit the parsed opening scene for <strong>{{ values.bot_name }}</strong>, then confirm to start the chat.</p>
<form method="post" action="/bots/{{ values.bot_id }}/kickoff" class="kickoff-form">
<fieldset>
<legend>Container</legend>
<label>
<span>name</span>
<input type="text" name="container_name" required value="{{ values.container_name|default('', true) }}">
</label>
<label>
<span>type</span>
<input type="text" name="container_type" required value="{{ values.container_type|default('', true) }}">
</label>
<label>
<span>properties (JSON)</span>
<textarea name="container_properties" rows="6">{{ values.container_properties|default('{}', true) }}</textarea>
<small>JSON object; invalid JSON falls back to <code>{}</code></small>
</label>
</fieldset>
<fieldset>
<legend>Initial in-fiction time</legend>
<label>
<span>initial_time_iso</span>
<input type="text" name="initial_time_iso" required value="{{ values.initial_time_iso|default('', true) }}">
<small>ISO 8601, e.g. <code>2026-04-26T20:00:00+00:00</code></small>
</label>
</fieldset>
<fieldset>
<legend>Your activity</legend>
<label>
<span>posture</span>
<input type="text" name="you_activity_posture" value="{{ values.you_activity_posture|default('', true) }}">
</label>
<label>
<span>action verb</span>
<input type="text" name="you_activity_action_verb" value="{{ values.you_activity_action_verb|default('', true) }}">
</label>
<label>
<span>interruptible</span>
<input type="checkbox" name="you_activity_action_interruptible"{% if values.you_activity_action_interruptible %} checked{% endif %}>
</label>
<label>
<span>required attention</span>
<input type="text" name="you_activity_action_required_attention" value="{{ values.you_activity_action_required_attention|default('low', true) }}">
<small>low / medium / high</small>
</label>
<label>
<span>expected duration</span>
<input type="text" name="you_activity_action_expected_duration" value="{{ values.you_activity_action_expected_duration|default('', true) }}">
</label>
<label>
<span>attention</span>
<input type="text" name="you_activity_attention" value="{{ values.you_activity_attention|default('', true) }}">
</label>
<label>
<span>holding (comma-separated)</span>
<input type="text" name="you_activity_holding" value="{{ values.you_activity_holding|default('', true) }}">
</label>
</fieldset>
<fieldset>
<legend>{{ values.bot_name }}'s activity</legend>
<label>
<span>posture</span>
<input type="text" name="bot_activity_posture" value="{{ values.bot_activity_posture|default('', true) }}">
</label>
<label>
<span>action verb</span>
<input type="text" name="bot_activity_action_verb" value="{{ values.bot_activity_action_verb|default('', true) }}">
</label>
<label>
<span>interruptible</span>
<input type="checkbox" name="bot_activity_action_interruptible"{% if values.bot_activity_action_interruptible %} checked{% endif %}>
</label>
<label>
<span>required attention</span>
<input type="text" name="bot_activity_action_required_attention" value="{{ values.bot_activity_action_required_attention|default('low', true) }}">
<small>low / medium / high</small>
</label>
<label>
<span>expected duration</span>
<input type="text" name="bot_activity_action_expected_duration" value="{{ values.bot_activity_action_expected_duration|default('', true) }}">
</label>
<label>
<span>attention</span>
<input type="text" name="bot_activity_attention" value="{{ values.bot_activity_attention|default('', true) }}">
</label>
<label>
<span>holding (comma-separated)</span>
<input type="text" name="bot_activity_holding" value="{{ values.bot_activity_holding|default('', true) }}">
</label>
</fieldset>
<fieldset>
<legend>Edge seed</legend>
<label>
<span>summary</span>
<textarea name="edge_seed_summary" rows="3">{{ values.edge_seed_summary|default('', true) }}</textarea>
</label>
<label>
<span>knowledge facts (one per line)</span>
<textarea name="edge_seed_knowledge_facts" rows="6">{{ values.edge_seed_knowledge_facts|default('', true) }}</textarea>
</label>
</fieldset>
<div class="actions">
<button type="submit">Confirm and start chat</button>
<a href="/bots">Cancel</a>
</div>
</form>
{% endblock %}
+14
View File
@@ -0,0 +1,14 @@
{% extends "base.html" %}
{% block body %}
<nav class="rail">
<a class="rail-brand" href="/chats">chat</a>
<ul>
<li><a href="/chats" class="{% if active_nav == 'chats' %}active{% endif %}">Chats</a></li>
<li><a href="/bots" class="{% if active_nav == 'bots' %}active{% endif %}">Bots</a></li>
<li><a href="/settings" class="{% if active_nav == 'settings' %}active{% endif %}">Settings</a></li>
</ul>
</nav>
<main class="content">
{% block content %}{% endblock %}
</main>
{% endblock %}
+29
View File
@@ -0,0 +1,29 @@
{% extends "layout.html" %}
{% block title %}Settings - chat{% endblock %}
{% block content %}
<h1>Settings</h1>
{% if saved %}
<p class="success">Settings saved.</p>
{% endif %}
<form method="post" action="/settings" class="bot-form">
<label>
<span>name</span>
<input type="text" name="name" required value="{{ values.name|default('', true) }}">
<small>required</small>
</label>
<label>
<span>pronouns</span>
<input type="text" name="pronouns" value="{{ values.pronouns|default('', true) }}">
<small>optional (e.g. they/them)</small>
</label>
<label>
<span>persona</span>
<textarea name="persona" rows="3">{{ values.persona|default('', true) }}</textarea>
<small>optional but recommended; a short description of you</small>
</label>
<button type="submit">Save settings</button>
</form>
{% endblock %}
View File
+132
View File
@@ -0,0 +1,132 @@
from __future__ import annotations
from pathlib import Path
from fastapi import APIRouter, Depends, Form, HTTPException, Request
from fastapi.responses import RedirectResponse, HTMLResponse
from fastapi.templating import Jinja2Templates
from chat.db.connection import open_db
from chat.eventlog.log import append_event
from chat.eventlog.projector import project
from chat.state.entities import list_bots
TEMPLATES = Jinja2Templates(directory=str(Path(__file__).resolve().parent.parent / "templates"))
router = APIRouter()
REQUIRED_FIELDS = ("id", "name", "persona", "initial_relationship_to_you", "kickoff_prose")
def get_conn(request: Request):
settings = request.app.state.settings
db_path: Path = settings.db_path
with open_db(db_path, check_same_thread=False) as conn:
yield conn
def _split_voice_samples(text: str) -> list[str]:
if not text or not text.strip():
return []
# Split on a line containing only "---" (with optional surrounding whitespace).
parts: list[str] = []
buf: list[str] = []
for line in text.splitlines():
if line.strip() == "---":
if buf:
parts.append("\n".join(buf).strip())
buf = []
continue
buf.append(line)
if buf:
parts.append("\n".join(buf).strip())
return [p for p in parts if p]
def _split_traits(text: str) -> list[str]:
if not text or not text.strip():
return []
items: list[str] = []
for line in text.splitlines():
line = line.strip()
if not line:
continue
if "," in line:
items.extend(p.strip() for p in line.split(","))
else:
items.append(line)
return [t for t in items if t]
@router.get("/bots", response_class=HTMLResponse)
async def bots_list(request: Request, conn=Depends(get_conn)):
bots = list_bots(conn)
return TEMPLATES.TemplateResponse(
request, "bot_list.html", {"bots": bots, "active_nav": "bots"}
)
@router.get("/bots/new", response_class=HTMLResponse)
async def bot_form(request: Request):
return TEMPLATES.TemplateResponse(
request, "bot_form.html", {"values": {}, "error": None, "active_nav": "bots"}
)
@router.post("/bots/new")
async def bot_create(
request: Request,
id: str = Form(""),
name: str = Form(""),
persona: str = Form(""),
voice_samples: str = Form(""),
traits: str = Form(""),
backstory: str = Form(""),
initial_relationship_to_you: str = Form(""),
kickoff_prose: str = Form(""),
conn=Depends(get_conn),
):
values = {
"id": id,
"name": name,
"persona": persona,
"voice_samples": voice_samples,
"traits": traits,
"backstory": backstory,
"initial_relationship_to_you": initial_relationship_to_you,
"kickoff_prose": kickoff_prose,
}
missing = [f for f in REQUIRED_FIELDS if not values[f].strip()]
if missing:
raise HTTPException(status_code=400, detail=f"missing required: {', '.join(missing)}")
payload = {
"id": id.strip(),
"name": name.strip(),
"persona": persona.strip(),
"voice_samples": _split_voice_samples(voice_samples),
"traits": _split_traits(traits),
"backstory": backstory.strip(),
"initial_relationship_to_you": initial_relationship_to_you.strip(),
"kickoff_prose": kickoff_prose.strip(),
}
append_event(conn, kind="bot_authored", payload=payload)
project(conn)
return RedirectResponse(url=f"/bots/{payload['id']}/kickoff", status_code=303)
@router.post("/bots/{bot_id}/reset")
async def reset_bot_route(
bot_id: str,
request: Request,
confirm_name: str = Form(""),
conn=Depends(get_conn),
):
from chat.services.reset import reset_bot
try:
reset_bot(conn, bot_id, confirm_name=confirm_name)
except ValueError as e:
msg = str(e).lower()
if "not found" in msg:
raise HTTPException(status_code=404, detail=str(e))
raise HTTPException(status_code=400, detail=str(e))
return RedirectResponse(url="/bots", status_code=303)
+71
View File
@@ -0,0 +1,71 @@
"""Chat detail (shell) page.
Renders ``/chats/<id>``: the title (host bot's name), a timeline placeholder,
the user-input form, and the drawer toggle. Turn handling lives in T19; this
module only sets up the structural shell.
"""
from __future__ import annotations
from pathlib import Path
from fastapi import APIRouter, Depends, HTTPException, Request
from fastapi.responses import HTMLResponse
from fastapi.templating import Jinja2Templates
from chat.state.entities import get_bot
from chat.state.world import get_chat
from chat.web.bots import get_conn
from chat.web.render import render_prose
from chat.web.turns import _read_recent_dialogue
TEMPLATES = Jinja2Templates(
directory=str(Path(__file__).resolve().parent.parent / "templates")
)
# Register the prose renderer as a Jinja filter so the chat-detail
# template can use ``{{ turn.text|render_prose|safe }}`` (Task 33).
# The renderer escapes user content internally; ``|safe`` is required
# because the output contains intentional ``<p>``/``<em>``/etc. tags.
TEMPLATES.env.filters["render_prose"] = render_prose
router = APIRouter()
@router.get("/chats/{chat_id}", response_class=HTMLResponse)
async def chat_detail(chat_id: str, request: Request, conn=Depends(get_conn)):
chat = get_chat(conn, chat_id)
if chat is None:
raise HTTPException(status_code=404, detail=f"chat not found: {chat_id}")
host_bot = get_bot(conn, chat["host_bot_id"])
if host_bot is None:
# Defensive: chat row references a bot that doesn't exist. Treat as 404
# rather than crashing the template render.
raise HTTPException(
status_code=404, detail=f"host bot not found: {chat['host_bot_id']}"
)
# T19: render the timeline from event_log. We pull both user_turn and
# assistant_turn events for this chat, in chronological order. Each row
# is shaped ``{"speaker": ..., "text": ...}`` and the template
# discriminates roles via the speaker id (the literal "you" vs. a bot id).
raw_turns = _read_recent_dialogue(conn, chat_id, limit=200)
turns: list[dict] = []
for t in raw_turns:
if t["speaker"] == "you":
turns.append({"role": "you", "speaker": "you", "text": t["text"]})
else:
bot = get_bot(conn, t["speaker"])
label = bot["name"] if bot else t["speaker"]
turns.append({"role": "bot", "speaker": label, "text": t["text"]})
return TEMPLATES.TemplateResponse(
request,
"chat.html",
{
"chat": chat,
"host_bot": host_bot,
"turns": turns,
"active_nav": "chats",
},
)
+841
View File
@@ -0,0 +1,841 @@
"""Chat drawer — read view (T24) and inline edits (T25, T72).
The GET endpoint renders an HTML partial showing the current scene +
container, per-entity activity, host <-> you edges, pinned memories with
an ``n / cap`` counter, and recent witnessed memories from the host's
POV with significance markers.
T25 adds three POST endpoints for the most useful inline edits, each
returning the refreshed drawer partial so HTMX can swap it in:
* affinity slider on an edge (emits ``manual_edit``);
* significance dropdown on a memory (emits ``manual_edit``);
* pin toggle on a memory (emits ``memory_pin_changed`` with
``auto_pinned=0`` so a manual pin is not subject to auto-eviction).
T72 (Phase 2.5) extends the inline-edit set to cover the remaining
§6.4 editable fields whose state-layer support already lands in the
``manual_edit`` projector: edge trust slider, edge summary textarea,
memory POV summary textarea, and per-edge knowledge-fact add/remove. It
also exposes a witness-flag toggle (``you/host/guest``) per memory row
and a "first-meeting gate" on the Add-guest form so an existing edge
isn't quietly overwritten by a re-seed.
Each ``manual_edit`` payload snapshots the prior value alongside the new
one so a later inverse edit can restore state (§6.4 final paragraph).
"""
from __future__ import annotations
from pathlib import Path
from fastapi import APIRouter, Depends, Form, HTTPException, Request
from fastapi.responses import HTMLResponse
from fastapi.templating import Jinja2Templates
from chat.eventlog.log import append_and_apply
from chat.services.relationship_seed import seed_inter_bot_edges
from chat.services.scene_summarize import apply_scene_close_summary
from chat.state.edges import get_edge
from chat.state.entities import get_bot, get_you, list_bots
from chat.state.group_node import get_group_node
from chat.state.memory import get_pinned
from chat.state.world import active_scene, get_activity, get_chat, get_container
from chat.web.bots import get_conn
from chat.web.kickoff import get_llm_client
TEMPLATES = Jinja2Templates(
directory=str(Path(__file__).resolve().parent.parent / "templates")
)
router = APIRouter()
# Soft cap on pinned memories per owner (§8.5). Surfaced in the drawer header
# as `pinned|length / pin_cap`; eviction logic itself lives in T22.
PIN_CAP = 8
# Recent-memories list is bounded to keep the drawer cheap to render.
RECENT_LIMIT = 10
# T72.1 caps on free-form textarea edits. Edge summaries and per-POV
# memory summaries are drawer-driven prose — bound them so a stray paste
# can't blow up the projected row size or the SSE drawer refresh payload.
EDGE_SUMMARY_MAX = 2000
MEMORY_POV_SUMMARY_MAX = 2000
KNOWLEDGE_FACT_MAX = 500
@router.get("/chats/{chat_id}/drawer", response_class=HTMLResponse)
async def drawer(chat_id: str, request: Request, conn=Depends(get_conn)):
chat = get_chat(conn, chat_id)
if chat is None:
raise HTTPException(status_code=404, detail=f"chat not found: {chat_id}")
host_bot = get_bot(conn, chat["host_bot_id"])
if host_bot is None:
raise HTTPException(
status_code=404, detail=f"host bot not found: {chat['host_bot_id']}"
)
you_entity = get_you(conn) or {"name": "you", "pronouns": "", "persona": ""}
scene = active_scene(conn, chat_id)
container = None
if scene and scene.get("container_id") is not None:
container = get_container(conn, scene["container_id"])
you_activity = get_activity(conn, "you")
bot_activity = get_activity(conn, chat["host_bot_id"])
edge_b2y = get_edge(conn, chat["host_bot_id"], "you")
edge_y2b = get_edge(conn, "you", chat["host_bot_id"])
# T42: guest + group context. Empty defaults keep the template happy
# when no guest is present (the relevant sections render conditionally).
guest_bot = None
guest_activity = None
edge_h2g = None
edge_g2h = None
edge_y2g = None
edge_g2y = None
available_guests: list[dict] = []
group_node = None
if chat.get("guest_bot_id"):
guest_bot_id = chat["guest_bot_id"]
guest_bot = get_bot(conn, guest_bot_id)
guest_activity = get_activity(conn, guest_bot_id)
edge_h2g = get_edge(conn, chat["host_bot_id"], guest_bot_id)
edge_g2h = get_edge(conn, guest_bot_id, chat["host_bot_id"])
edge_y2g = get_edge(conn, "you", guest_bot_id)
edge_g2y = get_edge(conn, guest_bot_id, "you")
else:
# Candidates for the "Add guest" dropdown — every authored bot
# except the host (and "you", which is implicit, never a bot row).
available_guests = [
b for b in list_bots(conn) if b["id"] != chat["host_bot_id"]
]
# T72.2 first-meeting gate: pre-compute whether a host->candidate edge
# already exists. Template renders the prose textarea disabled and the
# POST handler skips ``seed_inter_bot_edges`` (preserving the existing
# edge content) unless the user explicitly toggles "re-seed anyway".
existing_guest_edges = {
b["id"]: get_edge(conn, chat["host_bot_id"], b["id"]) is not None
for b in available_guests
}
group_node = get_group_node(conn, chat_id)
# Recent memories from host's POV (witness_host = 1), most recent first.
# Raw query keeps this read self-contained — no projector helper exposes
# "latest N for an owner" yet and the drawer is the only consumer. The
# three witness flags ride along so T72.3's per-row checkboxes can
# render the current state without a second query per memory.
recent_rows = conn.execute(
"""
SELECT id, pov_summary, significance, pinned, created_at,
witness_you, witness_host, witness_guest
FROM memories
WHERE owner_id = ? AND witness_host = 1
ORDER BY id DESC
LIMIT ?
""",
(chat["host_bot_id"], RECENT_LIMIT),
).fetchall()
recent_memories = [
{
"id": r[0],
"pov_summary": r[1],
"significance": r[2],
"pinned": r[3],
"created_at": r[4],
"witness_you": r[5],
"witness_host": r[6],
"witness_guest": r[7],
}
for r in recent_rows
]
pinned = get_pinned(conn, chat["host_bot_id"])
return TEMPLATES.TemplateResponse(
request,
"_drawer.html",
{
"chat": chat,
"host_bot": host_bot,
"you_entity": you_entity,
"scene": scene,
"container": container,
"you_activity": you_activity,
"bot_activity": bot_activity,
"edge_b2y": edge_b2y,
"edge_y2b": edge_y2b,
"guest_bot": guest_bot,
"guest_activity": guest_activity,
"edge_h2g": edge_h2g,
"edge_g2h": edge_g2h,
"edge_y2g": edge_y2g,
"edge_g2y": edge_g2y,
"available_guests": available_guests,
"existing_guest_edges": existing_guest_edges,
"group_node": group_node,
"recent_memories": recent_memories,
"pinned": pinned,
"pin_cap": PIN_CAP,
},
)
# --- T25 edit endpoints ---------------------------------------------------
#
# Each endpoint:
# 1. Loads the chat (404 if missing) and the target row (404 if missing).
# 2. Reads the prior value before mutating, so the event payload carries
# it for §6.4 reversibility.
# 3. Calls ``append_and_apply`` so the projected table updates atomically
# with the event log append; full reprojection would re-add deltas
# from earlier ``edge_update`` events.
# 4. Returns the refreshed drawer partial via ``await drawer(...)``, which
# HTMX swaps into ``#drawer``.
@router.post(
"/chats/{chat_id}/drawer/scene/close",
response_class=HTMLResponse,
)
async def close_scene_manual(
chat_id: str,
request: Request,
conn=Depends(get_conn),
client=Depends(get_llm_client),
):
"""Manual scene close from the drawer button.
Always available when there's an active scene; mirrors the auto-close
path in the turn flow but bypasses the hard-signal classifier. After
emitting ``scene_closed`` we run the T27 per-POV summary pipeline
(one classifier call) so the manual path produces the same memory /
edge updates as the auto path. Returns the refreshed drawer partial
so HTMX swaps it in. ``400`` when no scene is active — the button is
hidden in that state but a stale tab might still POST.
"""
chat = get_chat(conn, chat_id)
if chat is None:
raise HTTPException(status_code=404, detail=f"chat not found: {chat_id}")
scene = active_scene(conn, chat_id)
if scene is None:
raise HTTPException(
status_code=400, detail="no active scene to close"
)
append_and_apply(
conn,
kind="scene_closed",
payload={
"scene_id": scene["id"],
"ended_at": chat.get("time"),
# Significance defaults to 0; T22's significance worker
# operates on memories, not scenes.
"significance": 0,
},
)
settings = request.app.state.settings
await apply_scene_close_summary(
conn,
client,
classifier_model=settings.classifier_model,
chat_id=chat_id,
scene_id=scene["id"],
host_bot_id=chat["host_bot_id"],
timeout_s=settings.classifier_timeout_s,
)
return await drawer(chat_id, request, conn)
@router.post(
"/chats/{chat_id}/drawer/edge/{source_id}/{target_id}/affinity",
response_class=HTMLResponse,
)
async def edit_edge_affinity(
chat_id: str,
source_id: str,
target_id: str,
request: Request,
affinity: int = Form(...),
conn=Depends(get_conn),
):
chat = get_chat(conn, chat_id)
if chat is None:
raise HTTPException(status_code=404, detail=f"chat not found: {chat_id}")
edge = get_edge(conn, source_id, target_id)
if edge is None:
raise HTTPException(
status_code=404,
detail=f"edge not found: {source_id}->{target_id}",
)
prior = int(edge["affinity"])
new_value = max(0, min(100, int(affinity)))
append_and_apply(
conn,
kind="manual_edit",
payload={
"target_kind": "edge_affinity",
"target_id": {"source_id": source_id, "target_id": target_id},
"prior_value": prior,
"new_value": new_value,
},
)
return await drawer(chat_id, request, conn)
@router.post(
"/chats/{chat_id}/drawer/memory/{memory_id}/significance",
response_class=HTMLResponse,
)
async def edit_memory_significance(
chat_id: str,
memory_id: int,
request: Request,
significance: int = Form(...),
conn=Depends(get_conn),
):
chat = get_chat(conn, chat_id)
if chat is None:
raise HTTPException(status_code=404, detail=f"chat not found: {chat_id}")
row = conn.execute(
"SELECT significance FROM memories WHERE id = ?", (memory_id,)
).fetchone()
if row is None:
raise HTTPException(
status_code=404, detail=f"memory not found: {memory_id}"
)
prior = int(row[0])
new_value = max(0, min(3, int(significance)))
append_and_apply(
conn,
kind="manual_edit",
payload={
"target_kind": "memory_significance",
"target_id": int(memory_id),
"prior_value": prior,
"new_value": new_value,
},
)
return await drawer(chat_id, request, conn)
@router.post(
"/chats/{chat_id}/drawer/memory/{memory_id}/pin",
response_class=HTMLResponse,
)
async def toggle_memory_pin(
chat_id: str,
memory_id: int,
request: Request,
pinned: int = Form(...),
conn=Depends(get_conn),
):
chat = get_chat(conn, chat_id)
if chat is None:
raise HTTPException(status_code=404, detail=f"chat not found: {chat_id}")
row = conn.execute(
"SELECT pinned FROM memories WHERE id = ?", (memory_id,)
).fetchone()
if row is None:
raise HTTPException(
status_code=404, detail=f"memory not found: {memory_id}"
)
new_pinned = 1 if int(pinned) else 0
# Manual pin: ``auto_pinned=0`` so the §8.5 eviction query (which only
# touches auto-pinned rows) leaves this alone.
append_and_apply(
conn,
kind="memory_pin_changed",
payload={
"memory_id": int(memory_id),
"pinned": new_pinned,
"auto_pinned": 0,
},
)
return await drawer(chat_id, request, conn)
# --- T72.1 deferred v1 drawer edits --------------------------------------
#
# These four endpoints round out the §6.4 editable surface — the
# ``manual_edit`` projector already dispatches ``edge_trust``,
# ``edge_summary``, and ``memory_pov_summary`` (T25); ``edge_knowledge_fact``
# is a new dispatch branch added alongside this commit. Each route follows
# the T25 pattern: snapshot the prior value, append + apply ``manual_edit``,
# then re-render the drawer partial.
@router.post(
"/chats/{chat_id}/drawer/edge/trust",
response_class=HTMLResponse,
)
async def edit_edge_trust(
chat_id: str,
request: Request,
source_id: str = Form(...),
target_id: str = Form(...),
new_value: int = Form(...),
conn=Depends(get_conn),
):
chat = get_chat(conn, chat_id)
if chat is None:
raise HTTPException(status_code=404, detail=f"chat not found: {chat_id}")
if not 0 <= int(new_value) <= 100:
raise HTTPException(
status_code=400,
detail=f"trust must be in [0, 100], got {new_value}",
)
edge = get_edge(conn, source_id, target_id)
if edge is None:
raise HTTPException(
status_code=404,
detail=f"edge not found: {source_id}->{target_id}",
)
prior = int(edge["trust"])
append_and_apply(
conn,
kind="manual_edit",
payload={
"target_kind": "edge_trust",
"target_id": {"source_id": source_id, "target_id": target_id},
"prior_value": prior,
"new_value": int(new_value),
},
)
return await drawer(chat_id, request, conn)
@router.post(
"/chats/{chat_id}/drawer/edge/summary",
response_class=HTMLResponse,
)
async def edit_edge_summary(
chat_id: str,
request: Request,
source_id: str = Form(...),
target_id: str = Form(...),
new_summary: str = Form(...),
conn=Depends(get_conn),
):
chat = get_chat(conn, chat_id)
if chat is None:
raise HTTPException(status_code=404, detail=f"chat not found: {chat_id}")
if len(new_summary) > EDGE_SUMMARY_MAX:
raise HTTPException(
status_code=400,
detail=(
f"edge summary exceeds {EDGE_SUMMARY_MAX} chars "
f"(got {len(new_summary)})"
),
)
edge = get_edge(conn, source_id, target_id)
if edge is None:
raise HTTPException(
status_code=404,
detail=f"edge not found: {source_id}->{target_id}",
)
prior = edge.get("summary") or ""
append_and_apply(
conn,
kind="manual_edit",
payload={
"target_kind": "edge_summary",
"target_id": {"source_id": source_id, "target_id": target_id},
"prior_value": prior,
"new_value": new_summary,
},
)
return await drawer(chat_id, request, conn)
@router.post(
"/chats/{chat_id}/drawer/memory/pov-summary",
response_class=HTMLResponse,
)
async def edit_memory_pov_summary(
chat_id: str,
request: Request,
memory_id: int = Form(...),
new_summary: str = Form(...),
conn=Depends(get_conn),
):
chat = get_chat(conn, chat_id)
if chat is None:
raise HTTPException(status_code=404, detail=f"chat not found: {chat_id}")
if len(new_summary) > MEMORY_POV_SUMMARY_MAX:
raise HTTPException(
status_code=400,
detail=(
f"memory pov_summary exceeds {MEMORY_POV_SUMMARY_MAX} chars "
f"(got {len(new_summary)})"
),
)
# 404 when the memory either doesn't exist or belongs to a different
# chat — the drawer never surfaces cross-chat memories so editing one
# would be a path-traversal-style mistake.
row = conn.execute(
"SELECT pov_summary FROM memories WHERE id = ? AND chat_id = ?",
(int(memory_id), chat_id),
).fetchone()
if row is None:
raise HTTPException(
status_code=404,
detail=f"memory not found in chat: {memory_id}",
)
prior = row[0] or ""
append_and_apply(
conn,
kind="manual_edit",
payload={
"target_kind": "memory_pov_summary",
"target_id": int(memory_id),
"prior_value": prior,
"new_value": new_summary,
},
)
return await drawer(chat_id, request, conn)
@router.post(
"/chats/{chat_id}/drawer/edge/knowledge-facts",
response_class=HTMLResponse,
)
async def edit_edge_knowledge_facts(
chat_id: str,
request: Request,
source_id: str = Form(...),
target_id: str = Form(...),
action: str = Form(...),
fact: str = Form(...),
conn=Depends(get_conn),
):
"""Add or remove a single knowledge_fact on an edge.
Remove semantics are by string match (first occurrence) — the drawer
re-renders after every edit so threading a stable index through is
fragile when concurrent ``edge_update`` events can append more facts
between renders. The projector is a no-op when the fact isn't found,
keeping the route idempotent for stale form submissions.
"""
chat = get_chat(conn, chat_id)
if chat is None:
raise HTTPException(status_code=404, detail=f"chat not found: {chat_id}")
if action not in ("add", "remove"):
raise HTTPException(
status_code=400,
detail=f"action must be 'add' or 'remove', got {action!r}",
)
if len(fact) > KNOWLEDGE_FACT_MAX:
raise HTTPException(
status_code=400,
detail=(
f"fact exceeds {KNOWLEDGE_FACT_MAX} chars (got {len(fact)})"
),
)
if not fact.strip():
raise HTTPException(status_code=400, detail="fact must not be empty")
edge = get_edge(conn, source_id, target_id)
if edge is None:
raise HTTPException(
status_code=404,
detail=f"edge not found: {source_id}->{target_id}",
)
prior = list(edge.get("knowledge") or [])
append_and_apply(
conn,
kind="manual_edit",
payload={
"target_kind": "edge_knowledge_fact",
"target_id": {"source_id": source_id, "target_id": target_id},
"prior_value": prior,
"new_value": {"action": action, "fact": fact},
},
)
return await drawer(chat_id, request, conn)
# --- T72.3 witness flag inline-edit --------------------------------------
#
# Witness flags decide which entities can recall a memory (§7 retrieval).
# Editing them is rare but high-impact — flipping ``witness_guest`` from 0
# to 1 makes the memory available to the guest's prompt context. The route
# follows the T25 / T72.1 pattern: snapshot prior, append + apply
# ``manual_edit`` with a ``{flag, value}`` payload, refresh the partial.
_VALID_WITNESS_FLAGS = ("you", "host", "guest")
@router.post(
"/chats/{chat_id}/drawer/memory/witness",
response_class=HTMLResponse,
)
async def edit_memory_witness(
chat_id: str,
request: Request,
memory_id: int = Form(...),
flag: str = Form(...),
new_value: int = Form(...),
conn=Depends(get_conn),
):
chat = get_chat(conn, chat_id)
if chat is None:
raise HTTPException(status_code=404, detail=f"chat not found: {chat_id}")
if flag not in _VALID_WITNESS_FLAGS:
raise HTTPException(
status_code=400,
detail=(
f"flag must be one of {list(_VALID_WITNESS_FLAGS)}, "
f"got {flag!r}"
),
)
row = conn.execute(
f"SELECT witness_{flag} FROM memories "
"WHERE id = ? AND chat_id = ?",
(int(memory_id), chat_id),
).fetchone()
if row is None:
raise HTTPException(
status_code=404,
detail=f"memory not found in chat: {memory_id}",
)
prior_int = int(row[0])
new_int = 1 if int(new_value) else 0
append_and_apply(
conn,
kind="manual_edit",
payload={
"target_kind": "memory_witness",
"target_id": int(memory_id),
"prior_value": {"flag": flag, "value": prior_int},
"new_value": {"flag": flag, "value": new_int},
},
)
return await drawer(chat_id, request, conn)
# --- T42 guest add/remove -------------------------------------------------
#
# Adding a guest fans out into up to four events: a ``guest_added`` to flip
# ``chats.guest_bot_id``, two ``edge_update`` events seeded from the
# user-supplied prose (skipped when the prose is empty / the seed comes back
# default), and a ``group_node_initialized`` if no row exists yet — three
# entities now share the chat so the §8.4 group node becomes meaningful.
#
# Removing a guest first emits ``scene_closed`` for the active scene (so any
# host -> you scene closes cleanly with the guest still in scope) before
# clearing the guest_bot_id; per spec the next user message implicitly opens
# a fresh you+host scene via Phase 1's mid-chat reset behavior.
def _seed_is_default(seed) -> bool:
"""Treat a seed as a no-op when both summaries are empty AND both
delta pairs are zero AND both fact lists are empty.
"""
return (
not seed.a_to_b_summary
and not seed.b_to_a_summary
and seed.a_to_b_affinity_delta == 0
and seed.a_to_b_trust_delta == 0
and seed.b_to_a_affinity_delta == 0
and seed.b_to_a_trust_delta == 0
and not seed.a_to_b_knowledge_facts
and not seed.b_to_a_knowledge_facts
)
@router.post(
"/chats/{chat_id}/drawer/guest/add",
response_class=HTMLResponse,
)
async def add_guest(
chat_id: str,
request: Request,
guest_bot_id: str = Form(...),
relationship_prose: str = Form(""),
reseed: str = Form(""),
conn=Depends(get_conn),
client=Depends(get_llm_client),
):
chat = get_chat(conn, chat_id)
if chat is None:
raise HTTPException(status_code=404, detail=f"chat not found: {chat_id}")
if chat.get("guest_bot_id") is not None:
raise HTTPException(
status_code=400,
detail="a guest is already present in this chat",
)
if guest_bot_id == chat["host_bot_id"]:
raise HTTPException(
status_code=400, detail="guest must differ from host"
)
guest_bot = get_bot(conn, guest_bot_id)
if guest_bot is None:
raise HTTPException(
status_code=404, detail=f"guest bot not found: {guest_bot_id}"
)
host_bot = get_bot(conn, chat["host_bot_id"])
if host_bot is None:
raise HTTPException(
status_code=404,
detail=f"host bot not found: {chat['host_bot_id']}",
)
# T72.2 first-meeting gate: when an edge already exists from a prior
# chat, the textarea is rendered disabled. Submission without the
# explicit "re-seed anyway" toggle skips ``seed_inter_bot_edges``
# entirely so the existing edge content (affinity, trust, knowledge,
# summaries) survives. ``guest_added`` and ``group_node_initialized``
# still fire so the chat picks up the new participant.
existing_edge = (
get_edge(conn, chat["host_bot_id"], guest_bot_id) is not None
)
reseed_requested = reseed.lower() in ("1", "true", "on", "yes")
skip_seed = existing_edge and not reseed_requested
settings = request.app.state.settings
if skip_seed:
seed = None
else:
seed = await seed_inter_bot_edges(
client,
classifier_model=settings.classifier_model,
bot_a_id=chat["host_bot_id"],
bot_a_name=host_bot["name"],
bot_b_id=guest_bot_id,
bot_b_name=guest_bot["name"],
relationship_prose=relationship_prose,
timeout_s=settings.classifier_timeout_s,
)
append_and_apply(
conn,
kind="guest_added",
payload={"chat_id": chat_id, "guest_bot_id": guest_bot_id},
)
# Emit edge_update only when the seed carries content. Empty prose
# short-circuits inside ``seed_inter_bot_edges`` to a default seed,
# so this skips the two extra log entries on the no-prose path.
# NOTE: ``_apply_edge_update`` does not accept a ``summary`` field —
# per-direction summary is set via the per-pov scene-close path
# (T27), not direct edge_update. We therefore drop seed.*_summary
# here; the deltas + knowledge_facts are what materializes.
if seed is not None and not _seed_is_default(seed):
append_and_apply(
conn,
kind="edge_update",
payload={
"source_id": chat["host_bot_id"],
"target_id": guest_bot_id,
"chat_id": chat_id,
"affinity_delta": seed.a_to_b_affinity_delta,
"trust_delta": seed.a_to_b_trust_delta,
"knowledge_facts": seed.a_to_b_knowledge_facts,
"last_interaction_at": chat.get("time"),
"last_interaction_chat_id": chat_id,
},
)
append_and_apply(
conn,
kind="edge_update",
payload={
"source_id": guest_bot_id,
"target_id": chat["host_bot_id"],
"chat_id": chat_id,
"affinity_delta": seed.b_to_a_affinity_delta,
"trust_delta": seed.b_to_a_trust_delta,
"knowledge_facts": seed.b_to_a_knowledge_facts,
"last_interaction_at": chat.get("time"),
"last_interaction_chat_id": chat_id,
},
)
# Three entities now share the chat (you, host, guest) — initialize
# the group node row if Wave 1's reader doesn't see one yet.
if get_group_node(conn, chat_id) is None:
append_and_apply(
conn,
kind="group_node_initialized",
payload={
"chat_id": chat_id,
"members": ["you", chat["host_bot_id"], guest_bot_id],
"summary": "",
"dynamic": "",
"threads": [],
},
)
return await drawer(chat_id, request, conn)
@router.post(
"/chats/{chat_id}/drawer/guest/remove",
response_class=HTMLResponse,
)
async def remove_guest(
chat_id: str,
request: Request,
conn=Depends(get_conn),
):
chat = get_chat(conn, chat_id)
if chat is None:
raise HTTPException(status_code=404, detail=f"chat not found: {chat_id}")
if chat.get("guest_bot_id") is None:
raise HTTPException(
status_code=400, detail="no guest present in this chat"
)
# Close the active scene (if any) before flipping guest_bot_id so
# the scene record carries the guest as a participant.
scene = active_scene(conn, chat_id)
if scene is not None:
append_and_apply(
conn,
kind="scene_closed",
payload={
"scene_id": scene["id"],
"ended_at": chat.get("time"),
"significance": 0,
},
)
append_and_apply(
conn,
kind="guest_removed",
payload={"chat_id": chat_id},
)
return await drawer(chat_id, request, conn)
+286
View File
@@ -0,0 +1,286 @@
"""Kickoff parse-and-confirm flow.
After a bot is authored, the user lands on ``/bots/<id>/kickoff``. We call the
LLM-backed ``parse_kickoff`` to extract a structured opening scene from the
authored prose and render it as an editable form. On submit, the (possibly
edited) values are turned into a sequence of events that initialize the chat,
its container, the participants' activities, an open scene, and a seed edge.
"""
from __future__ import annotations
import json
from datetime import datetime
from pathlib import Path
from fastapi import APIRouter, Depends, Form, HTTPException, Request
from fastapi.responses import HTMLResponse, RedirectResponse
from fastapi.templating import Jinja2Templates
from chat.eventlog.log import append_event
from chat.eventlog.projector import project
from chat.llm.client import LLMClient
from chat.services.kickoff import parse_kickoff
from chat.state.entities import get_bot, get_you
from chat.web.bots import get_conn
TEMPLATES = Jinja2Templates(
directory=str(Path(__file__).resolve().parent.parent / "templates")
)
router = APIRouter()
def get_llm_client(request: Request) -> LLMClient:
"""Production LLM client. Tests override this via ``app.dependency_overrides``."""
settings = request.app.state.settings
from chat.llm.featherless import FeatherlessClient
return FeatherlessClient(
api_key=settings.featherless_api_key,
base_url=settings.featherless_base_url,
)
def _parse_holding(text: str) -> list[str]:
if not text or not text.strip():
return []
return [p.strip() for p in text.split(",") if p.strip()]
def _parse_facts(text: str) -> list[str]:
if not text or not text.strip():
return []
return [line.strip() for line in text.splitlines() if line.strip()]
def _parse_properties(text: str) -> dict:
"""Parse the container_properties textarea as JSON.
Returns ``{}`` on invalid JSON rather than raising — the form is editable
and a bad value should not block the user from confirming the rest.
"""
if not text or not text.strip():
return {}
try:
loaded = json.loads(text)
return loaded if isinstance(loaded, dict) else {}
except (json.JSONDecodeError, ValueError):
return {}
@router.get("/bots/{bot_id}/kickoff", response_class=HTMLResponse)
async def kickoff_get(
bot_id: str,
request: Request,
conn=Depends(get_conn),
llm=Depends(get_llm_client),
):
bot = get_bot(conn, bot_id)
if bot is None:
raise HTTPException(status_code=404, detail=f"bot not found: {bot_id}")
you = get_you(conn)
you_name = you["name"] if you else "You"
settings = request.app.state.settings
parsed = await parse_kickoff(
llm,
model=settings.classifier_model,
bot_name=bot["name"],
bot_persona=bot["persona"],
initial_relationship_to_you=bot.get("initial_relationship_to_you", ""),
kickoff_prose=bot.get("kickoff_prose", ""),
you_name=you_name,
timeout_s=settings.classifier_timeout_s,
)
# Render values onto the form. ``container_properties`` is shown as JSON;
# ``holding`` lists are rendered as comma-separated text; the seed
# knowledge facts are rendered one-per-line.
values = {
"bot_id": bot_id,
"bot_name": bot["name"],
"container_name": parsed.container_name,
"container_type": parsed.container_type,
"container_properties": json.dumps(parsed.container_properties, indent=2),
"initial_time_iso": parsed.initial_time_iso,
"you_activity_posture": parsed.you_activity.posture,
"you_activity_action_verb": parsed.you_activity.action_verb,
"you_activity_action_interruptible": parsed.you_activity.action_interruptible,
"you_activity_action_required_attention": parsed.you_activity.action_required_attention,
"you_activity_action_expected_duration": parsed.you_activity.action_expected_duration,
"you_activity_attention": parsed.you_activity.attention,
"you_activity_holding": ", ".join(parsed.you_activity.holding),
"bot_activity_posture": parsed.bot_activity.posture,
"bot_activity_action_verb": parsed.bot_activity.action_verb,
"bot_activity_action_interruptible": parsed.bot_activity.action_interruptible,
"bot_activity_action_required_attention": parsed.bot_activity.action_required_attention,
"bot_activity_action_expected_duration": parsed.bot_activity.action_expected_duration,
"bot_activity_attention": parsed.bot_activity.attention,
"bot_activity_holding": ", ".join(parsed.bot_activity.holding),
"edge_seed_summary": parsed.edge_seed_summary,
"edge_seed_knowledge_facts": "\n".join(parsed.edge_seed_knowledge_facts),
}
return TEMPLATES.TemplateResponse(
request, "kickoff_confirm.html", {"values": values, "active_nav": "bots"}
)
@router.post("/bots/{bot_id}/kickoff")
async def kickoff_post(
bot_id: str,
request: Request,
container_name: str = Form(""),
container_type: str = Form(""),
container_properties: str = Form(""),
initial_time_iso: str = Form(""),
you_activity_posture: str = Form(""),
you_activity_action_verb: str = Form(""),
you_activity_action_interruptible: str = Form(""),
you_activity_action_required_attention: str = Form("low"),
you_activity_action_expected_duration: str = Form(""),
you_activity_attention: str = Form(""),
you_activity_holding: str = Form(""),
bot_activity_posture: str = Form(""),
bot_activity_action_verb: str = Form(""),
bot_activity_action_interruptible: str = Form(""),
bot_activity_action_required_attention: str = Form("low"),
bot_activity_action_expected_duration: str = Form(""),
bot_activity_attention: str = Form(""),
bot_activity_holding: str = Form(""),
edge_seed_summary: str = Form(""),
edge_seed_knowledge_facts: str = Form(""),
conn=Depends(get_conn),
):
bot = get_bot(conn, bot_id)
if bot is None:
raise HTTPException(status_code=404, detail=f"bot not found: {bot_id}")
# Loose ISO 8601 validation. ``datetime.fromisoformat`` accepts the offset
# form ``2026-04-26T20:00:00+00:00`` we use; reject anything it can't parse.
if initial_time_iso.strip():
try:
datetime.fromisoformat(initial_time_iso.strip())
except ValueError:
raise HTTPException(
status_code=400,
detail=f"invalid initial_time_iso: {initial_time_iso!r}",
)
chat_id = f"chat_{bot_id}"
# Predict the next container id so we can reference it from later events
# without needing a mid-flow projection. Containers use AUTOINCREMENT-style
# rowid, so MAX(id)+1 is safe within this single-writer transaction.
next_container_row = conn.execute(
"SELECT COALESCE(MAX(id), 0) + 1 FROM containers"
).fetchone()
container_id = next_container_row[0]
# 1. chat_created
append_event(
conn,
kind="chat_created",
payload={
"id": chat_id,
"host_bot_id": bot_id,
"initial_time": initial_time_iso,
"narrative_anchor": "Day 1",
"weather": "",
},
)
# 2. container_created
append_event(
conn,
kind="container_created",
payload={
"chat_id": chat_id,
"name": container_name,
"type": container_type,
"properties": _parse_properties(container_properties),
"parent_id": None,
},
)
you_interruptible = bool(you_activity_action_interruptible)
bot_interruptible = bool(bot_activity_action_interruptible)
# 3. activity_change for "you"
append_event(
conn,
kind="activity_change",
payload={
"entity_id": "you",
"container_id": container_id,
"posture": you_activity_posture,
"action": {
"verb": you_activity_action_verb,
"interruptible": you_interruptible,
"required_attention": you_activity_action_required_attention,
"expected_duration": you_activity_action_expected_duration,
"started_at": initial_time_iso,
},
"attention": you_activity_attention,
"holding": _parse_holding(you_activity_holding),
"status": {},
},
)
# 4. activity_change for bot
append_event(
conn,
kind="activity_change",
payload={
"entity_id": bot_id,
"container_id": container_id,
"posture": bot_activity_posture,
"action": {
"verb": bot_activity_action_verb,
"interruptible": bot_interruptible,
"required_attention": bot_activity_action_required_attention,
"expected_duration": bot_activity_action_expected_duration,
"started_at": initial_time_iso,
},
"attention": bot_activity_attention,
"holding": _parse_holding(bot_activity_holding),
"status": {},
},
)
# 5. scene_opened
append_event(
conn,
kind="scene_opened",
payload={
"chat_id": chat_id,
"container_id": container_id,
"started_at": initial_time_iso,
"participants": ["you", bot_id],
},
)
# 6. edge_update (seed). The seed summary is preserved as the first
# knowledge fact prefixed with ``[summary] `` — proper summary writes happen
# at scene-close (T27).
facts = _parse_facts(edge_seed_knowledge_facts)
if edge_seed_summary.strip():
facts.insert(0, f"[summary] {edge_seed_summary.strip()}")
append_event(
conn,
kind="edge_update",
payload={
"source_id": bot_id,
"target_id": "you",
"chat_id": chat_id,
"knowledge_facts": facts,
},
)
# Project all events at once. ``bot_authored`` (already in log from prior
# POST) is idempotent (INSERT OR REPLACE); the new events project cleanly
# because they're being applied for the first time.
project(conn)
return RedirectResponse(url=f"/chats/{chat_id}", status_code=303)
+61
View File
@@ -0,0 +1,61 @@
from __future__ import annotations
from fastapi import Request
from fastapi.responses import RedirectResponse
from starlette.middleware.base import BaseHTTPMiddleware
from chat.db.connection import open_db
from chat.state.entities import get_you, list_bots
class FirstRunRedirectMiddleware(BaseHTTPMiddleware):
"""Redirect users through the first-run flow (per requirements §16.2).
Behavior on GET requests to landing routes (``/`` and ``/chats``):
- No ``you_entity`` → ``/settings``
- ``you_entity`` exists but no bots → ``/bots/new``
- Otherwise pass through to the underlying handler.
The middleware is a no-op for:
- Non-GET requests (POST/PUT writes proceed and surface their own errors).
- Static assets, health checks, and any path under ``/settings``,
``/bots``, ``/api``, ``/health``, ``/favicon`` — so the user can
actually complete setup once redirected.
- Sub-paths of ``/chats`` (e.g. ``/chats/<id>``, ``/chats/<id>/drawer``);
only the bare landing pages get the redirect treatment. Sub-resources
either 404 cleanly or are HTMX partials that should not page-redirect.
"""
SKIP_PREFIXES = (
"/static",
"/settings",
"/bots",
"/health",
"/favicon",
"/api",
)
async def dispatch(self, request: Request, call_next):
if request.method != "GET":
return await call_next(request)
path = request.url.path
if any(path.startswith(p) for p in self.SKIP_PREFIXES):
return await call_next(request)
# Only fire on the landing routes themselves.
if path != "/" and path != "/chats":
return await call_next(request)
settings = request.app.state.settings
with open_db(settings.db_path) as conn:
you = get_you(conn)
bots = list_bots(conn)
if you is None:
return RedirectResponse(url="/settings", status_code=303)
if not bots:
return RedirectResponse(url="/bots/new", status_code=303)
return await call_next(request)
+34
View File
@@ -0,0 +1,34 @@
from __future__ import annotations
from pathlib import Path
from fastapi import APIRouter, Depends, Request
from fastapi.responses import HTMLResponse, RedirectResponse
from fastapi.templating import Jinja2Templates
from chat.web.bots import get_conn
from chat.state.world import list_chats
from chat.state.entities import get_bot
TEMPLATES = Jinja2Templates(directory=str(Path(__file__).resolve().parent.parent / "templates"))
router = APIRouter()
@router.get("/", include_in_schema=False)
async def home():
return RedirectResponse(url="/chats", status_code=303)
@router.get("/chats", response_class=HTMLResponse)
async def chats_list(request: Request, conn=Depends(get_conn)):
chats = list_chats(conn)
# Annotate each chat with the host bot's name for display.
for ch in chats:
bot = get_bot(conn, ch["host_bot_id"])
ch["host_bot_name"] = bot["name"] if bot else ch["host_bot_id"]
# Last-message snippet and last-played-at are blank in v1; T19 fills them.
ch["last_message_snippet"] = ""
ch["last_played_at"] = None
return TEMPLATES.TemplateResponse(request, "chat_list.html", {
"chats": chats,
"active_nav": "chats",
})
+60
View File
@@ -0,0 +1,60 @@
"""In-process per-chat broadcast channel.
Each ``chat_id`` has a list of subscriber ``asyncio.Queue`` instances. T16
provides only the registry and fan-out mechanism; T19+ will publish events
(turn appends, streamed tokens, drawer updates, scene close, edge updates)
through this channel so all browser tabs viewing a chat stay in sync.
The registry is process-local: appropriate for a single-user local server.
"""
from __future__ import annotations
import asyncio
from collections import defaultdict
from typing import Any
# {chat_id: [queue, queue, ...]}
_subscribers: dict[str, list[asyncio.Queue]] = defaultdict(list)
_lock = asyncio.Lock()
async def subscribe(chat_id: str) -> asyncio.Queue:
"""Subscribe to a chat's broadcast channel.
Returns a fresh ``asyncio.Queue`` that will receive every event published
to ``chat_id`` while the subscription is active. Callers must invoke
:func:`unsubscribe` when finished (typically on client disconnect) to
avoid leaking queues into the registry.
"""
queue: asyncio.Queue = asyncio.Queue()
async with _lock:
_subscribers[chat_id].append(queue)
return queue
async def unsubscribe(chat_id: str, queue: asyncio.Queue) -> None:
"""Remove ``queue`` from the registry; remove the chat key if empty."""
async with _lock:
if chat_id in _subscribers:
if queue in _subscribers[chat_id]:
_subscribers[chat_id].remove(queue)
if not _subscribers[chat_id]:
del _subscribers[chat_id]
async def publish(chat_id: str, event: dict[str, Any]) -> None:
"""Fan-out ``event`` to every subscriber of ``chat_id``.
The same dict reference is enqueued to all subscribers. Callers should
treat published events as immutable. Queues are unbounded for v1.
"""
async with _lock:
queues = list(_subscribers.get(chat_id, []))
for q in queues:
await q.put(event)
def subscriber_count(chat_id: str) -> int:
"""Test helper. Returns the number of active subscribers for a chat."""
return len(_subscribers.get(chat_id, []))
+106
View File
@@ -0,0 +1,106 @@
"""Transcript display formatting (Task 33, Requirements §16.3).
Bot and user prose is rendered with **lightweight markdown**:
* ``*action*`` → ``<em class="action">…</em>`` — italic narration.
* ``**bold**`` → ``<strong>…</strong>`` — emphasis.
* ``((ooc))`` → ``<span class="ooc">((ooc))</span>`` — author-to-system
asides; visible to the reader, dimmed/italic in CSS, and stripped from
the prompt sent to the bot (see :func:`chat.web.turns._strip_ooc_for_prompt`).
* ``> line`` → ``<blockquote>line</blockquote>``.
* Double newline → paragraph break.
* Everything else is HTML-escaped and wrapped in ``<p>…</p>``.
No headings, code blocks, links, images, or tables — out of scope per
Requirements §16.3. The renderer is the single source of truth used by
both the chat-detail GET (initial timeline render, via Jinja filter) and
the per-turn SSE fragments emitted from :mod:`chat.web.turns`.
Order of operations matters:
1. ``html.escape`` the whole input first — every replacement below assumes
user-supplied ``<``/``>``/``&`` are already neutralised, so the wrapper
tags we add can never collide with an attacker-controlled tag.
2. OOC wrap before action/bold so its inner ``*`` are not interpreted.
3. Bold (``**``) before action (``*``) — the bold pattern is stricter and
would otherwise be partially consumed by the action regex.
4. Blockquote pass over already-escaped lines (so we match ``&gt;``).
5. Paragraph split on double newline.
"""
from __future__ import annotations
import html
import re
# ``((…))`` — non-greedy, allows newlines so a multi-line OOC aside still
# wraps cleanly. The inner ``[^)]*?`` keeps it from spanning across a
# closing-paren boundary.
_OOC_PATTERN = re.compile(r"\(\([^)]*?\)\)", re.DOTALL)
# ``**bold**`` — strict: no embedded asterisks or newlines. Must run
# *before* the single-asterisk action pattern, otherwise ``**x**`` would
# be partly consumed by ``*…*``.
_BOLD_PATTERN = re.compile(r"\*\*([^*\n]+)\*\*")
# ``*action*`` — single-asterisk italics; same restriction as bold.
_ACTION_PATTERN = re.compile(r"\*([^*\n]+)\*")
# ``> line`` at start of a line — note we match the *escaped* form
# ``&gt;`` because this pass runs after ``html.escape``.
_BLOCKQUOTE_PATTERN = re.compile(r"^&gt;\s?(.+)$", re.MULTILINE)
def render_prose(text: str) -> str:
"""Render prose to safe HTML.
Returns an empty string for empty/whitespace-only input so the caller
can append the result without producing stray ``<p></p>`` tags.
"""
if not text or not text.strip():
return ""
# Normalise CRLF so paragraph splitting on ``\n\n`` works for input
# pasted from Windows clients.
text = text.replace("\r\n", "\n").replace("\r", "\n")
escaped = html.escape(text)
# OOC first — the wrapped span survives subsequent passes.
escaped = _OOC_PATTERN.sub(
lambda m: f'<span class="ooc">{m.group(0)}</span>', escaped
)
# Bold strictly before action (regex precedence — see module docstring).
escaped = _BOLD_PATTERN.sub(r"<strong>\1</strong>", escaped)
escaped = _ACTION_PATTERN.sub(r'<em class="action">\1</em>', escaped)
# Blockquote on already-escaped ``&gt;`` markers.
escaped = _BLOCKQUOTE_PATTERN.sub(r"<blockquote>\1</blockquote>", escaped)
# Paragraph splitting — drop empty fragments so a trailing ``\n\n``
# doesn't yield an empty ``<p></p>`` block.
paragraphs = [p.strip() for p in escaped.split("\n\n") if p.strip()]
return "".join(f"<p>{p}</p>" for p in paragraphs)
def render_turn_html(speaker: str, text: str, role: str = "bot") -> str:
"""Render a full transcript turn as ``<div class="turn …">…</div>``.
Used by both the SSE fragment publisher in :mod:`chat.web.turns`
(per-turn live updates) and indirectly by the chat-detail Jinja
template (initial render, via the ``render_prose`` filter).
``role`` selects the CSS class (``turn-you`` vs ``turn-bot``); the
speaker label and role name are HTML-escaped defensively even though
they currently come from trusted server-side state.
"""
speaker_html = html.escape(speaker)
role_html = html.escape(role)
body_html = render_prose(text)
return (
f'<div class="turn turn-{role_html}">'
f"<strong>{speaker_html}</strong>"
f"{body_html}"
f"</div>"
)
+50
View File
@@ -0,0 +1,50 @@
from __future__ import annotations
from pathlib import Path
from fastapi import APIRouter, Depends, Form, HTTPException, Request
from fastapi.responses import HTMLResponse
from fastapi.templating import Jinja2Templates
from chat.eventlog.log import append_event
from chat.eventlog.projector import project
from chat.state.entities import get_you
from chat.web.bots import get_conn
TEMPLATES = Jinja2Templates(directory=str(Path(__file__).resolve().parent.parent / "templates"))
router = APIRouter()
@router.get("/settings", response_class=HTMLResponse)
async def settings_get(request: Request, conn=Depends(get_conn)):
you = get_you(conn) or {"name": "", "pronouns": "", "persona": ""}
return TEMPLATES.TemplateResponse(
request,
"settings.html",
{"values": you, "saved": False, "active_nav": "settings"},
)
@router.post("/settings", response_class=HTMLResponse)
async def settings_post(
request: Request,
name: str = Form(""),
pronouns: str = Form(""),
persona: str = Form(""),
conn=Depends(get_conn),
):
if not name.strip():
raise HTTPException(status_code=400, detail="name is required")
payload = {
"name": name.strip(),
"pronouns": pronouns.strip(),
"persona": persona.strip(),
}
append_event(conn, kind="you_authored", payload=payload)
project(conn)
return TEMPLATES.TemplateResponse(
request,
"settings.html",
{"values": payload, "saved": True, "active_nav": "settings"},
)
+88
View File
@@ -0,0 +1,88 @@
"""Server-Sent Events endpoint for per-chat live updates.
Each browser tab on ``/chats/<id>`` opens an SSE connection here. On connect:
1. We verify the chat exists (404 otherwise).
2. We subscribe to the chat's pub/sub channel.
3. We emit a ``snapshot`` event with the current state. T16 only provides a
stub payload (``{"chat_id": <id>, "ready": true}``) so the client can
confirm the channel is live; T19+ will populate it with real state.
4. We loop, awaiting events from the queue and yielding them as SSE frames.
A 15-second keepalive comment is emitted on idle to defeat intermediary
timeouts.
5. When the client disconnects we unsubscribe so the registry doesn't leak.
"""
from __future__ import annotations
import asyncio
import json
from fastapi import APIRouter, Depends, HTTPException, Request
from fastapi.responses import StreamingResponse
from chat.state.world import get_chat
from chat.web.bots import get_conn
from chat.web.pubsub import subscribe, unsubscribe
router = APIRouter()
# Heartbeat cadence. Long enough to avoid chattiness; short enough that most
# HTTP intermediaries won't close an idle connection.
_KEEPALIVE_SECONDS = 15.0
def _format_sse(event: str, data: dict | str) -> bytes:
"""Format a single SSE frame: ``event: <name>\\ndata: <body>\\n\\n``.
``data`` may be a dict (JSON-serialized) or a raw string. The string
form is used for HTMX SSE swaps where the payload is an HTML
fragment that the client splices into the DOM verbatim.
"""
if isinstance(data, str):
payload = data
else:
payload = json.dumps(data)
return f"event: {event}\ndata: {payload}\n\n".encode("utf-8")
@router.get("/chats/{chat_id}/events")
async def chat_events(chat_id: str, request: Request, conn=Depends(get_conn)):
chat = get_chat(conn, chat_id)
if chat is None:
raise HTTPException(status_code=404, detail="chat not found")
async def stream():
queue = await subscribe(chat_id)
try:
# Initial snapshot — T19 will fill in real state.
yield _format_sse("snapshot", {"chat_id": chat_id, "ready": True})
while True:
if await request.is_disconnected():
break
try:
event = await asyncio.wait_for(
queue.get(), timeout=_KEEPALIVE_SECONDS
)
except asyncio.TimeoutError:
# SSE comment line (per spec, lines starting with ":" are
# ignored by the client) — keeps the connection warm.
yield b": keepalive\n\n"
continue
# Allow publishers to set the SSE event name via "event" key;
# default to "message" if omitted. When the remaining payload
# is a single ``data`` string, send it verbatim — that lets
# turn-flow publishers ship pre-rendered HTML fragments that
# HTMX's SSE extension can swap into the DOM directly.
event = dict(event) # don't mutate the published dict
kind = event.pop("event", "message")
if set(event.keys()) == {"data"} and isinstance(
event["data"], str
):
yield _format_sse(kind, event["data"])
else:
yield _format_sse(kind, event)
finally:
await unsubscribe(chat_id, queue)
return StreamingResponse(stream(), media_type="text/event-stream")
+918
View File
@@ -0,0 +1,918 @@
"""POST ``/chats/<id>/turns`` — narrative turn flow with SSE streaming.
The turn flow strings together the pieces built in T17 (turn parser), T18
(prompt assembler), and T16 (SSE channel). Phase 2 (T44) extends it to
multi-entity scenes with optional guest support and a follow-on
interjection beat.
1. Parse the user's prose with the classifier into typed segments.
2. Append a ``user_turn`` event capturing both the original prose and the
parsed segments.
3. Append a placeholder ``assistant_turn_started`` marker so observers know
a response is in flight.
4. Detect the addressee (host vs. guest) from the prose using a simple
word-boundary substring match — see :func:`_detect_addressee_id`.
5. Build the narrative prompt for the addressee, dropping OOC segments
before they reach the bot (per Requirements §6.1 the OOC convention is
for the author to talk to the system, not to the in-fiction bot).
6. Stream tokens from the LLM, broadcasting each chunk over the chat's SSE
channel as a ``token`` event so any subscribed browser tab sees them
arrive in real time.
7. On stream complete, append an ``assistant_turn`` event with the full
text and ``truncated=False``. Then run a post-turn state-update pass
(Requirements §3.4): one classifier call per directed edge between
present entities, each producing an ``edge_update`` event with
affinity/trust/knowledge deltas.
8. When a guest is present, run the interjection classifier (§6.2). If it
fires we stream a second narrative as the silent witness, append a
second ``assistant_turn`` event linked to the same ``user_turn_id``,
and re-run memory + state-update for the interjector. The same
in-flight task covers both halves so cancel collapses both.
9. Scene-close detection runs after the (primary + optional interjection)
beats land so the close summary sees the full closing scene. T45's
guest-aware ``apply_scene_close_summary`` writes per-POV summaries for
each present witness.
10. Publish a ``turn_html`` event for each turn so HTMX's SSE extension
can append it to the timeline without a page reload.
11. Return ``204 No Content`` — the SSE channel is the real conveyor of
state, not the POST response body.
Errors during streaming flip the assistant_turn's ``truncated`` flag to
``True`` and we still commit what we received. ``asyncio.CancelledError``
is treated identically and re-raised after recording the partial turn.
A cancellation mid-interjection skips the interjector's state/memory
follow-up so we don't run classifiers against a half-formed beat.
"""
from __future__ import annotations
import asyncio
import html
import json
import re
from fastapi import APIRouter, Depends, Form, HTTPException, Request
from fastapi.responses import HTMLResponse, RedirectResponse, Response
from chat.eventlog.log import append_and_apply, append_event
from chat.services.addressee import detect_addressee
from chat.services.background import SignificanceJob
from chat.services.interjection import detect_interjection
from chat.services.memory_write import record_turn_memory_for_present
from chat.services.multi_state_update import compute_state_updates_for_present
from chat.services.prompt import assemble_narrative_prompt
from chat.services.rewind import compute_rewind_preview, execute_rewind
from chat.services.scene_close import detect_scene_close
from chat.services.scene_summarize import apply_scene_close_summary
from chat.services.turn_parse import ParsedTurn, parse_turn
from chat.state.edges import get_edge
from chat.state.entities import get_bot, get_you
from chat.state.world import active_scene, get_chat, get_container
from chat.web.bots import get_conn
from chat.web.kickoff import get_llm_client
from chat.web.pubsub import publish
from chat.web.render import render_turn_html as _render_turn_html
router = APIRouter()
# Module-level registry of in-flight streaming tasks, keyed by chat_id.
# The POST /chats/<id>/turns/cancel route looks up the task and calls
# .cancel(); the streaming coroutine in post_turn catches the resulting
# CancelledError, commits the partial as truncated, and unregisters.
# Single-process v1 only — sufficient for one user with multiple tabs.
_in_flight_tasks: dict[str, asyncio.Task] = {}
def _strip_ooc_for_prompt(parsed: ParsedTurn) -> str:
"""Concatenate non-OOC segments back to a prose string for the prompt.
OOC segments (``((double parens))``) are kept in the user_turn payload
for transcript display but stripped before assembly so the bot never
sees author-to-system messages.
"""
keep = [s.text for s in parsed.segments if s.kind != "ooc"]
return " ".join(keep).strip()
def _read_recent_dialogue(conn, chat_id: str, limit: int = 200) -> list[dict]:
"""Return user-side and assistant_turn events for ``chat_id``.
Includes ``user_turn``, ``user_turn_edit`` (T29 edited prose), and
``assistant_turn``. Ordered oldest-first; superseded/hidden rows are
skipped so regenerated turns (T29) drop out of the rendered timeline.
Each entry is shaped ``{"speaker": <id-or-"you">, "text": <prose>}``
for the prompt assembler and the chat-detail template.
"""
cur = conn.execute(
"SELECT id, kind, payload_json FROM event_log "
"WHERE kind IN ('user_turn', 'user_turn_edit', 'assistant_turn') "
" AND superseded_by IS NULL AND hidden = 0 "
"ORDER BY id DESC LIMIT ?",
(limit,),
)
rows = cur.fetchall()
rows.reverse() # back to chronological order
out: list[dict] = []
for _row_id, kind, payload_json in rows:
p = json.loads(payload_json)
if p.get("chat_id") != chat_id:
continue
if kind in ("user_turn", "user_turn_edit"):
# Edited prose substitutes for the original user_turn (the
# original is marked superseded_by and filtered above).
out.append({"speaker": "you", "text": p.get("prose", "")})
else:
out.append(
{
"speaker": p.get("speaker_id", "bot"),
"text": p.get("text", ""),
}
)
return out
def _detect_addressee_id(
prose: str, host_bot: dict, guest_bot: dict | None
) -> str:
"""Return the bot id of the addressee for ``prose``.
Phase 2 v1 uses a simple case-insensitive whole-word match. The host
is the default — addressee flips to guest only when the guest's name
appears in the prose AND the host's does not. If both names match
or neither matches, the host keeps the floor. This bias keeps the
primary speaker stable across ambiguous prose; the interjection
branch (later in the turn flow) is how the silent witness gets a word
in edgewise when warranted.
"""
if guest_bot is None:
return host_bot["id"]
host_name = host_bot.get("name") or ""
guest_name = guest_bot.get("name") or ""
host_match = bool(
host_name
and re.search(rf"\b{re.escape(host_name)}\b", prose, re.IGNORECASE)
)
guest_match = bool(
guest_name
and re.search(rf"\b{re.escape(guest_name)}\b", prose, re.IGNORECASE)
)
if guest_match and not host_match:
return guest_bot["id"]
return host_bot["id"]
def _gather_state_update_inputs(
conn,
*,
host_bot: dict,
guest_bot: dict | None,
you_entity: dict,
) -> tuple[list[str], dict[str, str], dict[str, str], dict[tuple[str, str], dict]]:
"""Collect ``(present_ids, present_names, personas, prior_edges)`` for
a multi-entity state-update pass.
Phase 2 v1 always pairs ``you`` with the host and (when present) the
guest. ``prior_edges`` falls back to the schema default 50/50 baseline
when no row exists yet — that mirrors the Phase 1 single-pair flow.
Order matters: the host comes first so the directed-pair iteration
in :func:`compute_state_updates_for_present` matches the Phase 1
sequence (host->you, then you->host). Existing tests pin the canned-
response queue to that order — keeping it stable means we don't
have to reshuffle test fixtures across the Phase 2 cutover.
"""
present_ids: list[str] = [host_bot["id"], "you"]
present_names: dict[str, str] = {
host_bot["id"]: host_bot["name"],
"you": you_entity.get("name") or "you",
}
personas: dict[str, str] = {
host_bot["id"]: host_bot.get("persona") or "",
"you": you_entity.get("persona") or "",
}
if guest_bot is not None:
present_ids.append(guest_bot["id"])
present_names[guest_bot["id"]] = guest_bot["name"]
personas[guest_bot["id"]] = guest_bot.get("persona") or ""
prior_edges: dict[tuple[str, str], dict] = {}
for src in present_ids:
for tgt in present_ids:
if src == tgt:
continue
edge = get_edge(conn, src, tgt) or {
"affinity": 50,
"trust": 50,
"summary": "",
}
prior_edges[(src, tgt)] = edge
return present_ids, present_names, personas, prior_edges
@router.post("/chats/{chat_id}/turns")
async def post_turn(
chat_id: str,
request: Request,
prose: str = Form(""),
conn=Depends(get_conn),
client=Depends(get_llm_client),
):
if not prose.strip():
raise HTTPException(status_code=400, detail="prose cannot be empty")
chat = get_chat(conn, chat_id)
if chat is None:
raise HTTPException(status_code=404, detail=f"chat not found: {chat_id}")
host_bot = get_bot(conn, chat["host_bot_id"])
if host_bot is None:
# Defensive: chat row references a missing bot.
raise HTTPException(
status_code=404,
detail=f"host bot not found: {chat['host_bot_id']}",
)
guest_bot = None
guest_bot_id = chat.get("guest_bot_id")
if guest_bot_id is not None:
# T47's bot_reset cascade clears guest_bot_id from any chat that
# referenced the deleted bot, so by the time we read it here it's
# either None or a live bot id. The previous defensive
# degrade-to-1:1 block (T44) was rendered dead by T47 and removed
# in T74.4 — get_bot now returns a real row.
guest_bot = get_bot(conn, guest_bot_id)
settings = request.app.state.settings
# 1. Parse turn (classifier).
parsed = await parse_turn(
client, model=settings.classifier_model, prose=prose
)
prompt_prose = _strip_ooc_for_prompt(parsed)
# 2. Append user_turn event.
user_turn_event_id = append_event(
conn,
kind="user_turn",
payload={
"chat_id": chat_id,
"prose": prose,
"segments": [s.model_dump() for s in parsed.segments],
},
)
# 3. Determine the addressee. Done before assistant_turn_started so the
# placeholder reflects the bot the user is actually talking to (host
# in 1:1, host-or-guest in multi-entity). T74.1 routes the multi-entity
# case through the addressee classifier; the no-guest case still uses
# the substring fast-path because there is nothing to classify when
# only one bot is present (and a classifier round-trip there would
# just be throughput overhead).
if guest_bot is None:
addressee_id = _detect_addressee_id(prose, host_bot, guest_bot)
else:
decision = await detect_addressee(
client,
classifier_model=settings.classifier_model,
user_prose=prose,
host_id=host_bot["id"],
host_name=host_bot["name"],
guest_id=guest_bot["id"],
guest_name=guest_bot["name"],
timeout_s=settings.classifier_timeout_s,
)
addressee_id = decision.addressee_id
addressee_bot = (
guest_bot if (guest_bot is not None and addressee_id == guest_bot["id"])
else host_bot
)
# 4. Append assistant_turn_started placeholder. ``user_turn``,
# ``assistant_turn_started``, and ``assistant_turn`` have no registered
# projector handlers — they live in the event_log purely for transcript
# rendering — so we don't call ``project`` here. (Re-projecting now would
# also re-run prior non-idempotent inserts like ``chat_created``.)
append_event(
conn,
kind="assistant_turn_started",
payload={
"chat_id": chat_id,
"speaker_id": addressee_bot["id"],
"user_turn_id": user_turn_event_id,
},
)
# 5. Build the narrative prompt for the addressee. ``guest_id`` is
# passed explicitly so the prompt assembler renders the guest's
# activity / group-node block when applicable. The assembler is
# tolerant of ``guest_id is None`` so this is a no-op for 1:1 chats.
recent = _read_recent_dialogue(conn, chat_id, limit=20)
# Drop the just-appended user turn from ``recent`` — it's passed as
# ``user_turn_prose`` to the assembler and would otherwise duplicate.
if recent and recent[-1].get("speaker") == "you":
recent = recent[:-1]
messages = assemble_narrative_prompt(
conn,
chat_id=chat_id,
speaker_bot_id=addressee_bot["id"],
user_turn_prose=prompt_prose if prompt_prose else None,
recent_dialogue=recent,
budget_soft=settings.narrative_budget_soft,
budget_hard=settings.narrative_budget_hard,
guest_id=guest_bot_id,
)
# 6. Stream and accumulate tokens. The stream runs as a Task so the
# /turns/cancel route can invoke ``Task.cancel()`` to abort it
# mid-stream. ``accumulated`` is a closure over the inner coroutine,
# so when the await on ``stream_task`` raises CancelledError below
# we still see whatever tokens were appended before cancellation.
primary_accumulated: list[str] = []
primary_truncated = False
cancelled = False
async def _stream_primary() -> None:
async for chunk in client.stream(
messages,
model=settings.narrative_model,
max_tokens=settings.narrative_max_tokens,
temperature=settings.narrative_temperature,
):
primary_accumulated.append(chunk)
await publish(
chat_id,
{
"event": "token",
"text": chunk,
"speaker_id": addressee_bot["id"],
},
)
stream_task = asyncio.create_task(_stream_primary())
_in_flight_tasks[chat_id] = stream_task
try:
await stream_task
except asyncio.CancelledError:
# Preserve the partial output before letting the cancellation
# propagate so the transcript reflects what the user actually saw.
primary_truncated = True
cancelled = True
except Exception:
# Surface as a truncated turn rather than losing the partial output.
primary_truncated = True
finally:
# Always unregister so a subsequent turn can register a fresh task.
_in_flight_tasks.pop(chat_id, None)
primary_text = "".join(primary_accumulated)
# 7. Append the assistant_turn with the final text. (See note above on
# why we skip ``project`` for these transcript-only event kinds.)
append_event(
conn,
kind="assistant_turn",
payload={
"chat_id": chat_id,
"speaker_id": addressee_bot["id"],
"text": primary_text,
"truncated": primary_truncated,
"user_turn_id": user_turn_event_id,
},
)
# 7a. Per-turn memory write (Plan §11.1, T21 / T41). With a guest
# present this fans out to one ``memory_written`` event per witness
# (host + guest); without a guest it preserves the Phase 1 single
# write keyed on the host. Witness flags are set inside the helper.
scene = active_scene(conn, chat_id)
memory_results = record_turn_memory_for_present(
conn,
chat_id=chat_id,
host_bot_id=host_bot["id"],
guest_bot_id=guest_bot_id,
narrative_text=primary_text,
scene_id=scene["id"] if scene else None,
chat_clock_at=chat.get("time"),
)
# 7b. Post-turn state-update pass (Requirements §3.4 / T40). All
# directed pairs over the present entities — 2 pairs for 1:1, 6 for
# 3-entity scenes. Run sequentially via the inner helper which honors
# the Featherless 2-conn cap.
you_entity = get_you(conn) or {"name": "you", "persona": ""}
last_at = chat.get("time")
recent_for_update = _read_recent_dialogue(conn, chat_id, limit=10)
present_ids, present_names, personas, prior_edges = (
_gather_state_update_inputs(
conn,
host_bot=host_bot,
guest_bot=guest_bot,
you_entity=you_entity,
)
)
state_updates = await compute_state_updates_for_present(
client,
classifier_model=settings.classifier_model,
present_ids=present_ids,
present_names=present_names,
personas=personas,
prior_edges=prior_edges,
recent_dialogue=recent_for_update,
timeout_s=settings.classifier_timeout_s,
)
for src_id, tgt_id, update in state_updates:
append_and_apply(
conn,
kind="edge_update",
payload={
"source_id": src_id,
"target_id": tgt_id,
"chat_id": chat_id,
"affinity_delta": update.affinity_delta,
"trust_delta": update.trust_delta,
"knowledge_facts": update.knowledge_facts,
"last_interaction_at": last_at,
"last_interaction_chat_id": chat_id,
},
)
# 7c. Enqueue the async significance pass (Plan §11.1, T22). The
# worker scores the just-written memory 0-3, updates significance,
# and auto-pins on score 3 with the §8.5 soft-cap eviction rule.
# Phase 2 picks the host's memory id as the canonical input — guest
# POV memories piggyback on the same significance score (the prose
# they record is identical for v2; per-POV rewrite happens at scene
# close in T45 and downstream-of-significance).
worker = getattr(request.app.state, "background_worker", None)
host_event_memory = memory_results.get(host_bot["id"])
host_memory_id = host_event_memory[1] if host_event_memory else None
if worker is not None and host_memory_id is not None:
worker.enqueue(
SignificanceJob(
memory_id=host_memory_id,
narrative_text=primary_text,
prior_dialogue=recent_for_update,
host_bot_id=host_bot["id"],
)
)
# 8. Interjection branch (T39 / T44). Only fires when the chat has a
# guest AND the addressee was the bot we *can* interject for (i.e.
# not the lone bot in a 1:1 chat). The silent witness is whichever
# bot didn't get the addressee slot. We only run this when the
# primary stream actually completed — a cancelled or errored primary
# short-circuits the follow-on so we don't classifier-spam against a
# half-formed beat.
interjection_text: str | None = None
interjection_speaker_id: str | None = None
interjection_truncated = False
if (
guest_bot is not None
and not cancelled
and not primary_truncated
and primary_text.strip()
):
# Identify the silent witness — the bot that is NOT the addressee.
if addressee_id == host_bot["id"]:
silent_witness = guest_bot
else:
silent_witness = host_bot
edge_w_to_addr = get_edge(
conn, silent_witness["id"], addressee_bot["id"]
) or {"affinity": 50, "trust": 50, "summary": ""}
edge_w_to_you = get_edge(conn, silent_witness["id"], "you") or {
"affinity": 50,
"trust": 50,
"summary": "",
}
decision = await detect_interjection(
client,
classifier_model=settings.classifier_model,
addressee_name=addressee_bot["name"],
addressee_just_said=primary_text,
silent_witness_name=silent_witness["name"],
silent_witness_persona=silent_witness.get("persona") or "",
silent_witness_edge_to_addressee=edge_w_to_addr,
silent_witness_edge_to_you=edge_w_to_you,
you_just_said=prose,
timeout_s=settings.classifier_timeout_s,
)
if decision.should_interject:
interjection_speaker_id = silent_witness["id"]
# Re-read recent_dialogue so the just-appended assistant_turn
# (the addressee's beat) is in the prompt context.
interject_recent = _read_recent_dialogue(conn, chat_id, limit=20)
if interject_recent and interject_recent[-1].get("speaker") == "you":
interject_recent = interject_recent[:-1]
interject_messages = assemble_narrative_prompt(
conn,
chat_id=chat_id,
speaker_bot_id=silent_witness["id"],
addressee=addressee_bot["id"],
user_turn_prose=prompt_prose if prompt_prose else None,
recent_dialogue=interject_recent,
budget_soft=settings.narrative_budget_soft,
budget_hard=settings.narrative_budget_hard,
guest_id=guest_bot_id,
)
interject_accumulated: list[str] = []
async def _stream_interjection() -> None:
async for chunk in client.stream(
interject_messages,
model=settings.narrative_model,
max_tokens=settings.narrative_max_tokens,
temperature=settings.narrative_temperature,
):
interject_accumulated.append(chunk)
await publish(
chat_id,
{
"event": "token",
"text": chunk,
"speaker_id": silent_witness["id"],
},
)
interject_task = asyncio.create_task(_stream_interjection())
_in_flight_tasks[chat_id] = interject_task
try:
await interject_task
except asyncio.CancelledError:
interjection_truncated = True
cancelled = True
except Exception:
interjection_truncated = True
finally:
_in_flight_tasks.pop(chat_id, None)
interjection_text = "".join(interject_accumulated)
append_event(
conn,
kind="assistant_turn",
payload={
"chat_id": chat_id,
"speaker_id": silent_witness["id"],
"text": interjection_text,
"truncated": interjection_truncated,
"user_turn_id": user_turn_event_id,
"interjection_of": addressee_bot["id"],
},
)
# Skip the downstream classifier passes if the interjection
# was cancelled mid-stream — we don't want to score a partial
# beat the user never got to read in full.
if not interjection_truncated:
# Re-run the multi-pair state update — the interjector
# adding their voice plausibly shifts edges for everyone
# in the room. Idempotent enough for v2 (deltas accumulate;
# no stale state). Re-read recent so the just-appended
# interjection turn is in scope.
recent_post_interject = _read_recent_dialogue(
conn, chat_id, limit=10
)
# Re-fetch prior edges so deltas land on the post-primary
# state rather than the pre-turn baseline.
_, _, _, prior_edges_post = _gather_state_update_inputs(
conn,
host_bot=host_bot,
guest_bot=guest_bot,
you_entity=you_entity,
)
state_updates_post = await compute_state_updates_for_present(
client,
classifier_model=settings.classifier_model,
present_ids=present_ids,
present_names=present_names,
personas=personas,
prior_edges=prior_edges_post,
recent_dialogue=recent_post_interject,
timeout_s=settings.classifier_timeout_s,
)
for src_id, tgt_id, update in state_updates_post:
append_and_apply(
conn,
kind="edge_update",
payload={
"source_id": src_id,
"target_id": tgt_id,
"chat_id": chat_id,
"affinity_delta": update.affinity_delta,
"trust_delta": update.trust_delta,
"knowledge_facts": update.knowledge_facts,
"last_interaction_at": last_at,
"last_interaction_chat_id": chat_id,
},
)
# Memory write for the interjection beat — a second pair
# of memory_written events (host + guest POVs).
interject_memory_results = record_turn_memory_for_present(
conn,
chat_id=chat_id,
host_bot_id=host_bot["id"],
guest_bot_id=guest_bot_id,
narrative_text=interjection_text,
scene_id=scene["id"] if scene else None,
chat_clock_at=chat.get("time"),
)
# T74.2: enqueue a significance pass for the interjection
# memory. Mirrors the primary-turn enqueue pattern above —
# we score on the host's memory id since the prose is
# identical across both POVs (per-POV rewrite happens at
# scene close in T45). Without this enqueue the
# interjection beat lands in memory but never gets scored,
# so it can never auto-pin even when it carries a pivotal
# moment.
interject_host_event = interject_memory_results.get(
host_bot["id"]
)
interject_host_memory_id = (
interject_host_event[1] if interject_host_event else None
)
if (
worker is not None
and interject_host_memory_id is not None
):
worker.enqueue(
SignificanceJob(
memory_id=interject_host_memory_id,
narrative_text=interjection_text,
prior_dialogue=recent_post_interject,
host_bot_id=host_bot["id"],
)
)
# 9. Scene-close detection (Plan §7.2, T26). Runs AFTER assistant_turn
# and the optional interjection so the bots' responses are part of
# the closing scene's final beat — closing before narrative would
# force the bot to speak "in no scene", which is awkward. Hard
# signals only in Phase 1: container change parsed from prose, or
# explicit "fade out" / "we're done here" patterns. On classifier
# failure the service returns ``should_close=False`` so the turn
# flow keeps moving; the manual close button in the drawer is the
# always-available fallback.
#
# Skip empty prose — no signal to classify and no point spending a
# round-trip. Skip when there's no active scene (e.g. after a prior
# close in the same chat) — we have nothing to close. T13 (kickoff)
# is the only scene-opener path in v1; Phase 2-3 will handle
# automatic re-opening with the next container.
#
# T74.3: this branch deliberately runs even when ``cancelled`` is
# True. Close detection consumes only the user's prose (which is
# fully appended to the event_log BEFORE streaming starts) and the
# current container name; it does NOT consume the bot's output.
# A user who types "we're done here, fade out" and then hits Stop
# mid-stream still meant to close the scene — the cancelled bot
# beat doesn't invalidate that intent. Pinned by
# test_cancelled_turn_still_closes_scene_when_user_prose_signals_close.
if scene is not None and prose.strip():
container = None
if scene.get("container_id") is not None:
container = get_container(conn, scene["container_id"])
container_name = container["name"] if container else "unknown"
decision = await detect_scene_close(
client,
model=settings.classifier_model,
prose=prose,
current_container_name=container_name,
)
if decision.should_close:
append_and_apply(
conn,
kind="scene_closed",
payload={
"scene_id": scene["id"],
"ended_at": chat.get("time"),
# T27 promotes the per-POV summary into ``edges.summary``
# but doesn't currently set scene significance — the
# async significance pass (T22) operates on memories.
"significance": 0,
},
)
# T27 / T45: per-POV summary + edge summary update + knowledge
# promotion for each present witness (host always; guest when
# present). Runs synchronously after the close so the next
# turn (or a subsequent GET /chats/<id>) sees the rewritten
# memories and edge summaries. Tolerates classifier failure
# (returns the empty default and skips the writes).
await apply_scene_close_summary(
conn,
client,
classifier_model=settings.classifier_model,
chat_id=chat_id,
scene_id=scene["id"],
host_bot_id=host_bot["id"],
timeout_s=settings.classifier_timeout_s,
)
# 10. Broadcast a JSON completion event (for JS consumers) and an HTML
# fragment event (for HTMX SSE swap-into-timeline). One pair per
# written assistant_turn so the timeline ends up with both the
# primary and the interjection beat in the right order.
await publish(
chat_id,
{
"event": "assistant_turn_complete",
"speaker_id": addressee_bot["id"],
"text": primary_text,
"truncated": primary_truncated,
},
)
primary_html = _render_turn_html(
addressee_bot["name"], primary_text, role="bot"
)
await publish(
chat_id, {"event": "turn_html", "data": primary_html}
)
if interjection_text is not None and interjection_speaker_id is not None:
# The interjector's display name is whichever bot wasn't the
# addressee — pull it from the in-scope variable directly.
interject_speaker_name = (
host_bot["name"]
if interjection_speaker_id == host_bot["id"]
else (guest_bot["name"] if guest_bot is not None else "bot")
)
await publish(
chat_id,
{
"event": "assistant_turn_complete",
"speaker_id": interjection_speaker_id,
"text": interjection_text,
"truncated": interjection_truncated,
},
)
interject_html = _render_turn_html(
interject_speaker_name, interjection_text, role="bot"
)
await publish(
chat_id, {"event": "turn_html", "data": interject_html}
)
if cancelled:
# Re-raise after the partial-turn has been recorded.
raise asyncio.CancelledError
return Response(status_code=204)
# ---------------------------------------------------------------------------
# Cancel route (Task 34).
#
# Fire-and-forget: the Stop button POSTs here, we mark the in-flight
# streaming Task as cancelled, and return 204 immediately. The cancel
# propagates into the streaming coroutine on its next await, the
# CancelledError handler in ``post_turn`` catches it, and the partial
# is committed with ``truncated=True``. No body is needed — the SSE
# channel is the conveyor of state. If no turn is in flight (or the
# task already completed), we 204 silently so the client can fire the
# Stop button without a precondition check.
# ---------------------------------------------------------------------------
@router.post("/chats/{chat_id}/turns/cancel")
async def cancel_turn(chat_id: str, request: Request):
task = _in_flight_tasks.get(chat_id)
if task is None or task.done():
return Response(status_code=204)
task.cancel()
return Response(status_code=204)
# ---------------------------------------------------------------------------
# Rewind routes (Task 28).
#
# Two endpoints: a GET that renders the impact-preview modal, and a POST
# that actually executes the rewind. The execution path opens its own
# database connection because the route's ``conn`` is closed when the
# dependency-injection scope exits — passing it to ``execute_rewind``
# would dangle.
# ---------------------------------------------------------------------------
@router.get(
"/chats/{chat_id}/rewind/preview/{event_id}",
response_class=HTMLResponse,
)
async def rewind_preview(
chat_id: str,
event_id: int,
request: Request,
conn=Depends(get_conn),
):
"""Render the rewind impact-preview modal as a small HTML fragment.
The HTMX form inside the fragment posts to the execute endpoint
below. v1 keeps the markup minimal — Task 35 polishes the modal.
"""
chat = get_chat(conn, chat_id)
if chat is None:
raise HTTPException(status_code=404, detail=f"chat not found: {chat_id}")
preview = compute_rewind_preview(conn, event_id)
items = "".join(
f"<li>{count} × {html.escape(kind)}</li>"
for kind, count in preview["by_kind"].items()
)
body = (
"<div class='rewind-modal'>"
f"<h3>Rewind to event {event_id}?</h3>"
f"<p>This will remove {preview['total_events']} events:</p>"
f"<ul>{items}</ul>"
f"<form hx-post='/chats/{html.escape(chat_id)}/rewind/{event_id}' "
"hx-target='body' hx-swap='innerHTML'>"
"<button type='submit'>Confirm Rewind</button>"
"</form>"
"</div>"
)
return HTMLResponse(body)
# ---------------------------------------------------------------------------
# Regenerate route (Task 29).
#
# A POST that re-streams the most recent assistant turn. The prior
# ``assistant_turn`` event is kept in the log but flagged
# ``superseded_by`` so the timeline filter in :func:`_read_recent_dialogue`
# hides it. When the user supplies ``prose`` the original ``user_turn``
# is also superseded by a fresh ``user_turn_edit`` event capturing the
# edit. Significance is *not* re-run on regenerate (per plan §11.1) but
# state-update + memory writes are.
# ---------------------------------------------------------------------------
@router.post("/chats/{chat_id}/turns/{event_id}/regenerate")
async def regenerate_turn(
chat_id: str,
event_id: int,
request: Request,
prose: str | None = Form(None),
conn=Depends(get_conn),
client=Depends(get_llm_client),
):
"""Regenerate the assistant turn referenced by ``event_id``.
``prose`` is optional. When provided (and non-empty) we capture a
``user_turn_edit`` event before re-streaming. Returns 204 on
success, 404 when the chat or assistant_turn event is missing. The
SSE channel emits per-token events as the new text arrives.
"""
chat = get_chat(conn, chat_id)
if chat is None:
raise HTTPException(status_code=404, detail=f"chat not found: {chat_id}")
settings = request.app.state.settings
# Local import keeps the module import graph flat (the service
# imports from ``state`` / ``services`` siblings already).
from chat.services.regenerate import regenerate_assistant_turn
edited_prose = prose if prose else None
try:
await regenerate_assistant_turn(
conn,
client,
settings=settings,
chat_id=chat_id,
original_assistant_event_id=event_id,
edited_user_prose=edited_prose,
)
except ValueError as e:
raise HTTPException(status_code=404, detail=str(e))
return Response(status_code=204)
@router.post("/chats/{chat_id}/rewind/{event_id}")
async def rewind_execute(
chat_id: str,
event_id: int,
request: Request,
conn=Depends(get_conn),
):
"""Execute the rewind: snapshot, truncate event_log, re-project.
Note: ``conn`` is only used to validate the chat exists. The actual
rewind opens its own connection inside ``execute_rewind`` because
we need it to commit independently and survive the route's
dependency teardown.
"""
chat = get_chat(conn, chat_id)
if chat is None:
raise HTTPException(status_code=404, detail=f"chat not found: {chat_id}")
settings = request.app.state.settings
execute_rewind(
db_path=settings.db_path,
data_dir=settings.data_dir,
after_event_id=event_id,
)
return RedirectResponse(url=f"/chats/{chat_id}", status_code=303)
+6
View File
@@ -0,0 +1,6 @@
# Copy this file to data/config.toml and fill in your API key.
featherless_api_key = "REPLACE_ME"
narrative_model = "dphn/Dolphin-Mistral-24B-Venice-Edition"
classifier_model = "NousResearch/Hermes-3-Llama-3.1-8B"
ooc_marker = "(("
retrieval_k = 4
@@ -884,7 +884,7 @@ def get_bot(conn: Connection, bot_id: str) -> dict | None:
row = conn.execute("SELECT * FROM bots WHERE id = ?", (bot_id,)).fetchone()
if not row:
return None
cols = [c[0] for c in conn.execute("PRAGMA table_info(bots)").fetchall()]
cols = [c[1] for c in conn.execute("PRAGMA table_info(bots)").fetchall()]
d = dict(zip(cols, row))
d["voice_samples"] = json.loads(d.pop("voice_samples_json"))
d["traits"] = json.loads(d.pop("traits_json"))
@@ -499,6 +499,8 @@ Written per witness when a scene closes. Different details, different interpreta
### Phase 2 — multi-entity
**Status: shipped 2026-04-26** — multi-entity scene support, guest add/remove drawer UX, guest-aware prompt assembly, multi-entity turn flow with interjection classifier, per-POV scene close summaries for every present witness, group_node initialization/update, and bot reset cascade clearing stale `chats.guest_bot_id` references all landed across the wave5 task series (see `CLAUDE.md` § "Phase 2 status" for the deliverable summary and follow-ups).
- Guest bot in chat (3-entity scene config).
- Interjection classifier call.
- Witness filtering across multiple owners.
@@ -0,0 +1,910 @@
# Roleplay Engine — Phase 2 Implementation Plan
> **For Claude:** REQUIRED SUB-SKILL: Use `superpowers-extended-cc:executing-plans` to implement this plan task-by-task. Use `superpowers-extended-cc:dispatching-parallel-agents` for the parallel waves below.
**Goal:** Add multi-entity (3-entity) scene support: guest bot can be added to a host's chat; up to 3 entities present simultaneously (you + host + guest); turn flow handles silent witnesses, interjections, per-pair edges, and per-witness memory; drawer reflects guest state; scene close writes per-POV summaries for each present witness.
**Architecture:** Builds on Phase 1's event-sourced architecture. New event kinds (`guest_added`, `guest_removed`, `group_node_initialized`) carry the multi-entity state changes; existing handlers (`edge_update`, `memory_written`) already accept any `source_id`/`target_id` and witness mask, so most schema work is additive. The "have they met?" first-co-appearance prompt runs once per `(botA, botB)` pair and seeds initial inter-bot edges via existing `edge_update` events.
**Tech Stack:** Same as Phase 1 (Python 3.11+, FastAPI, HTMX, SQLite, Featherless). No new dependencies.
**Source-of-truth references:**
- Phase 2 scope: requirements doc §13 "Phase 2 — multi-entity"
- Behavioral details: requirements doc §6.2 (turn-taking with interjection), §7.5 (guest leaves), §8.5 (memory ownership), §11.2 (per-POV summaries on close)
- Conventions: [../../CLAUDE.md](../../CLAUDE.md) §"Behavioral defaults"
- Phase 1 plan (style, TDD pattern): [2026-04-26-v1-phase1-implementation.md](2026-04-26-v1-phase1-implementation.md)
When a task says "see §X", that's the requirements doc unless stated otherwise.
---
## Pre-flight
**Branch:** Create `phase-2` from the latest `main` after Phase 1 has been merged. If Phase 1 is still in PR review, branch off `phase-1` directly:
```bash
# Option A: after main has phase-1 merged
git checkout main && git pull && git checkout -b phase-2
# Option B: continue from phase-1 directly
git checkout phase-1 && git pull && git checkout -b phase-2
```
**Schema baseline:** Phase 1 leaves the DB at version 7. Phase 2 adds **0008_group_node.sql**. No other migrations expected.
**Pinned non-negotiables (carried forward from Phase 1):**
- State changes go through the event log. Use `append_and_apply(conn, kind, payload)` for the live path; `apply_event` only after a fresh `append_event` returning the new id.
- Witness filter every memory read at SQL level (hard `WHERE` constraint; never a soft signal).
- Edges are directed; `botA → botB` and `botB → botA` are independent records.
- Per-POV scene summaries — never write omniscient narration.
- TDD: every task starts with a failing test.
- One commit per task minimum, more if it splits naturally.
**Verification before claiming done:** Use `superpowers-extended-cc:verification-before-completion` — run the test command, paste actual output. Don't assume green.
---
## Parallel-Execution Strategy
This plan is structured into **6 waves** of tasks. Within a wave, tasks are designed to touch disjoint files so they can be executed by parallel subagents safely. Between waves, the controller (you, the controlling Claude session) merges each subagent's commits and verifies the suite stays green before dispatching the next wave.
### How to dispatch a wave in parallel
Use the **Agent tool with `isolation: "worktree"`** so each subagent gets its own git worktree. The runtime cleans up the worktree automatically if no changes are made; otherwise it returns the path + branch for the controller to merge.
In a single message, dispatch all tasks in the wave:
```
Agent({
description: "Wave 1 — T36 group_node schema",
subagent_type: "general-purpose",
isolation: "worktree",
prompt: "<full task text from below>",
})
Agent({
description: "Wave 1 — T37 guest events",
subagent_type: "general-purpose",
isolation: "worktree",
prompt: "<full task text from below>",
})
Agent({
description: "Wave 1 — T38 relationship-seed service",
subagent_type: "general-purpose",
isolation: "worktree",
prompt: "<full task text from below>",
})
```
All three subagents start simultaneously, each working on a private worktree branched off `phase-2`. They cannot see each other's changes (no shared filesystem state) — that's the safety guarantee.
### After a wave completes
1. Each subagent returns its worktree path and commit SHA.
2. **Run a spec + quality reviewer subagent on each completed task** (same pattern as Phase 1 — see `superpowers-extended-cc:requesting-code-review`).
3. **Merge the wave into `phase-2`** in any order (file-disjointness guarantees no conflict). Use fast-forward if possible:
```bash
git checkout phase-2
for branch in <wave-1-branches>; do
git merge --no-ff "$branch" -m "merge: <task description>"
done
```
4. **Run the full test suite** on the merged `phase-2`. If it's red, the wave's mutual independence assumption was violated — bisect to find the offending pair, fix, re-merge.
5. **Push `phase-2` to gitea** so the work is durable before the next wave starts.
6. Optionally clean up worktrees: `git worktree remove .worktrees/<branch>`.
### Conflict prevention checklist (apply before dispatch)
For each parallel wave, verify the **Files** sections of all tasks in that wave have **no overlapping paths**. The waves below are designed to satisfy this; if you decide to add or merge tasks, re-check.
If a hot file (`chat/web/turns.py`, `chat/services/prompt.py`, `chat/templates/_drawer.html`) needs changes from multiple tasks, do **not** parallelize them — serialize within the wave or split into separate waves.
### Failure recovery
If one subagent in a parallel wave fails (test failures, blocked, infinite loop):
- **Do not block the wave on a failure.** Cancel the failed subagent, merge the others' successful work, and re-dispatch the failed task as a single follow-up.
- If a failure exposes a bad assumption shared by multiple tasks (e.g. an event-payload schema mismatch), pause the wave and revisit the plan.
### Why each wave is parallel-safe
| Wave | Tasks | Hot files touched | Disjoint? |
|------|-------|-------------------|-----------|
| 1 | T36, T37, T38 | new files only + `chat/state/world.py` (T37 only) | ✅ |
| 2 | T39, T40, T41 | new files only + `chat/services/memory_write.py` (T41 only) | ✅ |
| 3 | T42 | `chat/web/drawer.py`, `chat/templates/_drawer.html` | (single task) |
| 4a | T43, T45 | `chat/services/prompt.py` (T43), `chat/services/scene_summarize.py` (T45) | ✅ |
| 4b | T44 | `chat/web/turns.py`, `chat/services/regenerate.py` | (single task; depends on 4a) |
| 5 | T46, T47, T48 | new tests + `chat/state/entities.py` (T47) + docs (T48) | ✅ |
---
## Task overview
```
Wave 1 ─┬─ T36: group_node schema + handler
├─ T37: guest_added / guest_removed events
└─ T38: relationship-seed service ("have they met?")
Wave 2 ─┬─ T39: interjection classifier service
├─ T40: multi-entity state-update coordinator
└─ T41: multi-witness memory write helper
Wave 3 ─── T42: drawer guest support (add/remove + render guest state)
Wave 4a ─┬─ T43: multi-entity prompt assembly (extends assemble_narrative_prompt)
└─ T45: multi-entity per-POV summaries on scene close
Wave 4b ─── T44: multi-entity turn flow integration (post_turn rewrite)
Wave 5 ─┬─ T46: witness filter test coverage (cross-witness scenarios)
├─ T47: bot reset cascades to guest scenes
└─ T48: Phase 2 documentation update
```
Critical path: 6 sequential merge points. Total tasks: 13. Wall-clock parallelism advantage depends on subagent dispatch overhead, but in principle Wave 1's 3 tasks can run concurrently in ~the time of one task.
---
## Wave 1 — Foundation
These three tasks are **fully independent**: T36 adds new files only, T37 modifies `chat/state/world.py` (additive event handlers), T38 adds new files only. Dispatch all three in parallel.
### Task 36: Group node schema + handler
**Files:**
- Create: `chat/db/migrations/0008_group_node.sql`
- Create: `chat/state/group_node.py`
- Create: `tests/test_group_node.py`
**Spec:** Adds the `group_node` table (one row per chat, populated when all three entities are present in a scene) and a projector handler for the `group_node_initialized` event. The group node carries the shared summary, group dynamic, inside jokes, and active threads (Phase 3 will populate `active_threads`; for Phase 2, just `summary` and `dynamic` matter).
**Step 1: Write the failing test**
```python
# tests/test_group_node.py
from chat.db.migrate import apply_migrations
from chat.db.connection import open_db
from chat.eventlog.log import append_event
from chat.eventlog.projector import project
from chat.state.group_node import get_group_node
import chat.state.entities # noqa
import chat.state.world # noqa
import chat.state.group_node # noqa: F401 - registers handlers
def test_group_node_initialized_creates_row(tmp_path):
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
# Seed bot, you, chat (minimal world state — no scene yet)
append_event(conn, kind="bot_authored", payload={
"id": "bot_a", "name": "BotA", "persona": "...",
"voice_samples": [], "traits": [], "backstory": "",
"initial_relationship_to_you": "", "kickoff_prose": "",
})
append_event(conn, kind="chat_created", payload={
"id": "chat_bot_a", "host_bot_id": "bot_a",
"initial_time": "2026-04-26T20:00:00+00:00",
"narrative_anchor": "Day 1", "weather": "",
})
append_event(conn, kind="group_node_initialized", payload={
"chat_id": "chat_bot_a",
"members": ["you", "bot_a", "bot_b"],
"summary": "",
"dynamic": "",
})
project(conn)
gn = get_group_node(conn, "chat_bot_a")
assert gn is not None
assert gn["members"] == ["you", "bot_a", "bot_b"]
assert gn["summary"] == ""
```
**Step 2: Run test to verify it fails**
```bash
.venv/bin/pytest tests/test_group_node.py -v
```
Expected: `ModuleNotFoundError: No module named 'chat.state.group_node'`.
**Step 3: Write minimal implementation**
`chat/db/migrations/0008_group_node.sql`:
```sql
CREATE TABLE group_node (
chat_id TEXT PRIMARY KEY,
members_json TEXT NOT NULL,
summary TEXT NOT NULL DEFAULT '',
dynamic TEXT NOT NULL DEFAULT '',
threads_json TEXT NOT NULL DEFAULT '[]',
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
```
`chat/state/group_node.py`:
```python
from __future__ import annotations
import json
from sqlite3 import Connection
from chat.eventlog.projector import on
from chat.eventlog.log import Event
@on("group_node_initialized")
def _apply_group_node_initialized(conn: Connection, e: Event) -> None:
p = e.payload
conn.execute(
"INSERT OR REPLACE INTO group_node "
"(chat_id, members_json, summary, dynamic, threads_json) "
"VALUES (?, ?, ?, ?, ?)",
(
p["chat_id"],
json.dumps(p["members"]),
p.get("summary", ""),
p.get("dynamic", ""),
json.dumps(p.get("threads", [])),
),
)
@on("group_node_updated")
def _apply_group_node_updated(conn: Connection, e: Event) -> None:
"""T45 calls this on scene close to rewrite summary + dynamic."""
p = e.payload
conn.execute(
"UPDATE group_node SET summary = ?, dynamic = ?, updated_at = datetime('now') "
"WHERE chat_id = ?",
(p.get("summary", ""), p.get("dynamic", ""), p["chat_id"]),
)
def get_group_node(conn: Connection, chat_id: str) -> dict | None:
row = conn.execute(
"SELECT chat_id, members_json, summary, dynamic, threads_json, updated_at "
"FROM group_node WHERE chat_id = ?",
(chat_id,),
).fetchone()
if not row:
return None
return {
"chat_id": row[0],
"members": json.loads(row[1]),
"summary": row[2],
"dynamic": row[3],
"threads": json.loads(row[4]),
"updated_at": row[5],
}
```
**Step 4: Run test to verify it passes**
```bash
.venv/bin/pytest tests/test_group_node.py -v
```
Expected: 1 passed.
**Step 5: Commit**
```bash
git add chat/db/migrations/0008_group_node.sql chat/state/group_node.py tests/test_group_node.py
git commit -m "feat: group_node schema + projector handlers"
```
**Notes for the implementer:**
- Add a second test for `group_node_updated`: append init then update, assert `summary` and `dynamic` change but `members` stays.
- Add a test for `get_group_node` returning `None` on a missing chat_id.
- Schema version after migration: 8. The migration runner handles this automatically; no test assertion on schema_version.
---
### Task 37: Guest add / remove events + handlers
**Files:**
- Modify: `chat/state/world.py` (add `_apply_guest_added`, `_apply_guest_removed` handlers; both update `chats.guest_bot_id`)
- Create: `tests/test_guest_events.py`
**Spec:** Two new event kinds.
- `guest_added` payload: `{chat_id, guest_bot_id}`. Handler sets `chats.guest_bot_id = ?`.
- `guest_removed` payload: `{chat_id}`. Handler sets `chats.guest_bot_id = NULL`.
These are pure state mutations — no related side effects. The kickoff parse-and-confirm flow (T13, Phase 1) and the new T39 interjection / T42 drawer routes will append these events.
**Step 1: Write the failing test**
```python
# tests/test_guest_events.py
from chat.db.migrate import apply_migrations
from chat.db.connection import open_db
from chat.eventlog.log import append_event
from chat.eventlog.projector import project
from chat.state.world import get_chat
import chat.state.entities # noqa
import chat.state.world # noqa
def test_guest_added_sets_guest_bot_id(tmp_path):
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
# Seed bot, chat
append_event(conn, kind="bot_authored", payload={
"id": "bot_a", "name": "BotA", "persona": "...",
"voice_samples": [], "traits": [], "backstory": "",
"initial_relationship_to_you": "", "kickoff_prose": "",
})
append_event(conn, kind="bot_authored", payload={
"id": "bot_b", "name": "BotB", "persona": "...",
"voice_samples": [], "traits": [], "backstory": "",
"initial_relationship_to_you": "", "kickoff_prose": "",
})
append_event(conn, kind="chat_created", payload={
"id": "chat_bot_a", "host_bot_id": "bot_a",
"initial_time": "2026-04-26T20:00:00+00:00",
"narrative_anchor": "Day 1", "weather": "",
})
append_event(conn, kind="guest_added", payload={
"chat_id": "chat_bot_a", "guest_bot_id": "bot_b",
})
project(conn)
chat = get_chat(conn, "chat_bot_a")
assert chat["guest_bot_id"] == "bot_b"
def test_guest_removed_clears_guest_bot_id(tmp_path):
# similar: add then remove, assert guest_bot_id is None
...
```
**Step 3: Implementation**
In `chat/state/world.py`, add the two handlers next to `_apply_chat_created`:
```python
@on("guest_added")
def _apply_guest_added(conn: Connection, e: Event) -> None:
p = e.payload
conn.execute(
"UPDATE chats SET guest_bot_id = ? WHERE id = ?",
(p["guest_bot_id"], p["chat_id"]),
)
@on("guest_removed")
def _apply_guest_removed(conn: Connection, e: Event) -> None:
p = e.payload
conn.execute(
"UPDATE chats SET guest_bot_id = NULL WHERE id = ?",
(p["chat_id"],),
)
```
**Step 5: Commit**
```bash
git add chat/state/world.py tests/test_guest_events.py
git commit -m "feat: guest_added / guest_removed event handlers"
```
**Notes:**
- 2 tests minimum (added, removed). Optional third: idempotent re-add (overwrites cleanly).
- Don't add any UI here — T42 handles UI.
---
### Task 38: Relationship-seed service ("have they met?")
**Files:**
- Create: `chat/services/relationship_seed.py`
- Create: `tests/test_relationship_seed.py`
**Spec:** Per requirements §5.2: when two bots first co-appear in a chat, prompt the user with "Have they met before? If yes, write a short prose seed describing how." The seed is parsed via classifier into structured `botA ↔ botB` edge content (summary + initial knowledge facts).
This task adds the **service layer** only. T39 (interjection) doesn't touch this; T42 (drawer guest UI) calls it via a route added there. So at the service level, we just expose:
```python
async def seed_inter_bot_edges(
client: LLMClient,
*,
classifier_model: str,
bot_a_id: str,
bot_a_name: str,
bot_b_id: str,
bot_b_name: str,
relationship_prose: str, # user-supplied prose; empty = "they haven't met"
timeout_s: float = 30.0,
) -> RelationshipSeed:
"""Parse user-supplied prose into structured edge content for both
directed pairs (bot_a → bot_b and bot_b → bot_a). Return the
RelationshipSeed; caller is responsible for emitting two edge_update
events."""
```
`RelationshipSeed`:
```python
class RelationshipSeed(BaseModel):
a_to_b_summary: str = ""
a_to_b_knowledge_facts: list[str] = Field(default_factory=list)
a_to_b_affinity_delta: int = 0 # signed, -10..+10 typical
a_to_b_trust_delta: int = 0
b_to_a_summary: str = ""
b_to_a_knowledge_facts: list[str] = Field(default_factory=list)
b_to_a_affinity_delta: int = 0
b_to_a_trust_delta: int = 0
```
If `relationship_prose` is empty/whitespace, short-circuit and return an empty `RelationshipSeed` (they haven't met → fresh edges with default 50/50).
**Step 1: Failing test**
```python
import pytest, json
from chat.llm.mock import MockLLMClient
from chat.services.relationship_seed import seed_inter_bot_edges, RelationshipSeed
@pytest.mark.asyncio
async def test_seed_parses_canned_prose():
canned = json.dumps({
"a_to_b_summary": "BotA and BotB went to college together.",
"a_to_b_knowledge_facts": ["BotB has a younger brother."],
"a_to_b_affinity_delta": 5,
"a_to_b_trust_delta": 3,
"b_to_a_summary": "BotB sees BotA as the responsible one.",
"b_to_a_knowledge_facts": ["BotA was once a TA."],
"b_to_a_affinity_delta": 4,
"b_to_a_trust_delta": 5,
})
mock = MockLLMClient(canned=[canned])
seed = await seed_inter_bot_edges(
mock, classifier_model="x",
bot_a_id="bot_a", bot_a_name="BotA",
bot_b_id="bot_b", bot_b_name="BotB",
relationship_prose="They went to college together; BotB still sees BotA as the responsible one.",
)
assert "college" in seed.a_to_b_summary
assert seed.a_to_b_affinity_delta == 5
@pytest.mark.asyncio
async def test_seed_empty_prose_returns_empty():
mock = MockLLMClient(canned=[]) # never called
seed = await seed_inter_bot_edges(
mock, classifier_model="x",
bot_a_id="bot_a", bot_a_name="BotA",
bot_b_id="bot_b", bot_b_name="BotB",
relationship_prose="",
)
assert seed == RelationshipSeed()
```
**Step 3: Minimal impl**
Wraps `classify()` from `chat.llm.classify` with a `RelationshipSeed` schema and a system prompt explaining the task.
**Step 5: Commit**
```bash
git add chat/services/relationship_seed.py tests/test_relationship_seed.py
git commit -m "feat: relationship-seed service for first-co-appearance prompt"
```
---
## Wave 2 — Services
After Wave 1 merges, dispatch Wave 2 in parallel. T39 and T40 are new files; T41 modifies `chat/services/memory_write.py` (additive — adds a new function alongside existing `record_turn_memory`).
### Task 39: Interjection classifier service
**Files:**
- Create: `chat/services/interjection.py`
- Create: `tests/test_interjection.py`
**Spec:** Per requirements §6.2: when a guest is present and the addressee bot has just spoken, decide whether the *non-addressee* bot interjects. Classifier returns `{should_interject: bool, reason: str}`. Caller (T44 turn flow) generates the interjection beat as a brief follow-on response if `should_interject`.
**Public API:**
```python
class InterjectionDecision(BaseModel):
should_interject: bool = False
reason: str = ""
async def detect_interjection(
client: LLMClient,
*,
classifier_model: str,
addressee_name: str,
addressee_just_said: str,
silent_witness_name: str,
silent_witness_persona: str,
silent_witness_edge_to_addressee: dict, # {affinity, trust, summary}
silent_witness_edge_to_you: dict,
you_just_said: str,
timeout_s: float = 30.0,
) -> InterjectionDecision:
"""Decide whether the silent witness bot interjects after the addressee
finishes speaking. Conservative bias — most turns should NOT interject
(return False). Trigger only when the witness's character would
plausibly speak up: jealousy, surprise, agreement worth voicing,
correcting a falsehood, etc.
"""
```
Classifier system prompt should explicitly bias toward `should_interject=false` (per spec: "addressee gets the floor"; interjection is the exception).
**Tests:** 3 minimum.
1. Mock returns `{should_interject: true, reason: "..."}` → result is True.
2. Mock returns `{should_interject: false}` → result is False.
3. Classifier failure → fallback default (`should_interject=false`, `reason="fallback"`).
**Commit:** `feat: interjection classifier service`
---
### Task 40: Multi-entity state-update coordinator
**Files:**
- Create: `chat/services/multi_state_update.py`
- Create: `tests/test_multi_state_update.py`
**Spec:** Wraps the existing `chat.services.state_update.compute_state_update` (single-pair) into a coordinator that runs state updates for **all directed pairs of present entities**. With 3 entities (you, host, guest), that's 6 pairs:
```
you → host, host → you
you → guest, guest → you
host → guest, guest → host
```
Returns a list of `(source_id, target_id, StateUpdate)` tuples; caller (T44) emits one `edge_update` event per tuple via `append_and_apply`.
**Public API:**
```python
async def compute_state_updates_for_present(
client: LLMClient,
*,
classifier_model: str,
present_ids: list[str], # e.g. ["you", "bot_a", "bot_b"]
present_names: dict[str, str], # id -> display name
personas: dict[str, str], # id -> persona blob
prior_edges: dict[tuple[str, str], dict], # (src, tgt) -> {affinity, trust, summary}
recent_dialogue: list[dict], # [{speaker, text}, ...]
timeout_s: float = 30.0,
) -> list[tuple[str, str, StateUpdate]]:
"""Run compute_state_update for every directed pair where source != target.
Returns list of (source_id, target_id, update) tuples. Skips pairs
involving "you" with itself.
"""
```
Implementation: nested loops over `present_ids`, sequential calls to `compute_state_update` (parallel calls would exceed the Featherless 2-connection cap from the FeatherlessClient semaphore).
**Tests:** 3 minimum.
1. With 2 present (you, host) → returns 2 updates (existing 1A/2D parity).
2. With 3 present (you, host, guest) → returns 6 updates, one per directed non-self pair.
3. Failures in one pair don't kill the whole batch (per-pair `compute_state_update` already has a default fallback).
**Commit:** `feat: multi-entity state-update coordinator`
---
### Task 41: Multi-witness memory write helper
**Files:**
- Modify: `chat/services/memory_write.py` (add `record_turn_memory_for_present` alongside existing `record_turn_memory`; do NOT remove or change `record_turn_memory`)
- Add tests to: `tests/test_memory_write.py`
**Spec:** Currently Phase 1's `record_turn_memory(conn, *, chat_id, host_bot_id, narrative_text, ...)` writes a single memory event for the host bot's POV. With a guest present, we need:
- One memory in the host's store (witness mask `[1, 1, 1]` if you/host/guest present)
- One memory in the guest's store (same witness mask, owner = guest_bot_id)
"You" still doesn't have a memory store in v1 (per §5.4 / §11.2).
**New helper:**
```python
def record_turn_memory_for_present(
conn,
*,
chat_id: str,
host_bot_id: str,
guest_bot_id: str | None,
narrative_text: str,
scene_id: int | None = None,
chat_clock_at: str | None = None,
source: str = "direct",
significance: int = 1,
) -> dict[str, tuple[int, int]]:
"""Write a memory_written event for each present bot witness (host
always; guest if guest_bot_id is not None). Returns {bot_id:
(event_id, memory_id)}.
Witness mask is [1, 1, 1] when guest is present, [1, 1, 0] otherwise
(mirrors Phase 1 single-bot behavior when guest_bot_id is None).
"""
```
Implementation: appends one `memory_written` event per present bot, calling `append_and_apply` for each, and queries the resulting `memories.id` per owner+chat just like Phase 1's `record_turn_memory`.
**Tests:** 3 minimum, added to `tests/test_memory_write.py`:
1. With `guest_bot_id=None`, behaves identically to `record_turn_memory` (one memory for host, witness `[1, 1, 0]`).
2. With `guest_bot_id="bot_b"`, writes two memories — one each for host and guest, both with witness `[1, 1, 1]`.
3. Returned dict keys match `{host_bot_id, guest_bot_id}` (or just `{host_bot_id}` when no guest).
**Commit:** `feat: multi-witness memory write helper`
---
## Wave 3 — Drawer guest support (single task)
This wave is one task because all Phase 2 drawer work touches the same two files (`chat/web/drawer.py` and `chat/templates/_drawer.html`). Splitting would force serial execution with conflict resolution. Single-task wave runs alone.
### Task 42: Drawer guest support (add/remove + render)
**Files:**
- Modify: `chat/web/drawer.py` (add `POST /chats/{chat_id}/drawer/guest/add`, `POST /chats/{chat_id}/drawer/guest/remove`; extend `drawer` GET handler to query guest state when present)
- Modify: `chat/templates/_drawer.html` (render guest activity, guest edges, group node summary; add "Add guest" form and "Remove guest" button when applicable)
- Create: `tests/test_drawer_guest.py`
**Spec:**
**GET /chats/{chat_id}/drawer** (extend, don't replace):
- Read `chat["guest_bot_id"]` from the existing `get_chat` query.
- If guest present: also fetch `get_bot(conn, guest_bot_id)`, `get_activity(conn, guest_bot_id)`, edges in both `host ↔ guest` directions, edges in both `you ↔ guest` directions, and `get_group_node(conn, chat_id)`.
- Pass all of this to the template.
**Template changes:**
- New section "Guest" rendering guest's name, activity, and the four edges involving the guest.
- New section "Group" rendering `group_node.summary` and `group_node.dynamic` when present.
- "Add guest" button → expands form with: bot selector (dropdown of authored bots not currently in this chat) + relationship prose textarea (the "have they met?" prompt).
- "Remove guest" button visible when a guest is present.
**POST /chats/{chat_id}/drawer/guest/add** route:
1. Read form: `guest_bot_id`, `relationship_prose`.
2. 404 if chat or guest_bot is missing.
3. 400 if guest_bot_id == host_bot_id.
4. 400 if a guest is already present.
5. Call `seed_inter_bot_edges` (T38) with the prose. May produce empty seed if prose is blank.
6. Append events: `guest_added`, then up to 2 `edge_update` events (host ↔ guest deltas from the seed). Use `append_and_apply` for each.
7. If all 3 entities are now present and no `group_node` row exists for this chat, append `group_node_initialized` with members=[you, host, guest] and empty summary/dynamic.
8. Return refreshed drawer partial.
**POST /chats/{chat_id}/drawer/guest/remove** route:
1. 404 if chat missing; 400 if no guest present.
2. Append `scene_closed` for the active scene (per §7.5: removing the guest closes the current scene).
3. Append `guest_removed`.
4. (Per §7.5 the host's chat then implicitly opens a new scene with you+host. For Phase 2, leave that as a manual "next user message creates the new scene" — same as Phase 1 mid-chat reset semantics. Phase 3 may auto-open.)
5. Return refreshed drawer partial.
**Tests (`tests/test_drawer_guest.py`):** 6 minimum.
1. GET drawer with no guest → no "Guest" section in body.
2. POST add guest → 303-or-200 with refreshed drawer; chat.guest_bot_id is set; `group_node` row created; relationship-seed mock returns canned values; edges have the seeded values.
3. POST add guest with empty relationship_prose → guest added; `seed_inter_bot_edges` short-circuits; edges remain at default 50/50.
4. POST add guest when one is already present → 400.
5. POST remove guest → guest_bot_id NULL, scene_closed event written.
6. GET drawer with guest present → "Guest" section + group_node summary visible.
**Commit:** `feat: drawer guest add/remove + render`
**Notes for implementer:**
- The guest-bot-selector dropdown lists bots from `list_bots(conn)` minus the host. Don't filter for "bots not in any chat" — guests can be in multiple chats simultaneously (each chat has its own scene state).
- The "have they met?" prose textarea is the per-pair prompt. v1 only fires it on first co-appearance globally; for v2, fire it every time a `(host, guest)` pair has no existing `host → guest` edge. After the first add, the edge exists, so subsequent adds skip the prose (or render it disabled with "you've already met"). Treat this as Phase 2.5 polish if it gets fiddly — for T42 just always show the prose textarea, blank by default.
- The drawer route already uses `Depends(get_conn)` and templates; reuse the existing dependency and TEMPLATES instance.
---
## Wave 4a — Multi-entity prompt + scene close (parallel)
T43 and T45 touch different files (`prompt.py` and `scene_summarize.py`). Dispatch both in parallel.
### Task 43: Multi-entity prompt assembly
**Files:**
- Modify: `chat/services/prompt.py` (extend `assemble_narrative_prompt` to handle a `guest_id` parameter and fetch guest activity, guest edge, group node into the prompt blocks)
- Add tests to: `tests/test_prompt.py`
**Spec:** The current `assemble_narrative_prompt(conn, *, chat_id, speaker_bot_id, addressee="you", ...)` only handles you+host. Extend:
- Accept a `guest_id: str | None = None` parameter (auto-fetched from `chat.guest_bot_id` if not passed; explicit override for tests).
- When `guest_id` is provided:
- Activity block includes the guest's activity (`get_activity(conn, guest_id)`).
- If `speaker_bot_id == guest_id`, the addressee defaults to "you" but caller can override.
- "Speaker's other edges" SHOULD-tier block includes speaker → non-addressee (e.g., host → guest if speaker is host and addressee is you).
- MUST-tier identity block unchanged (still just speaker).
- Group-node summary becomes a SHOULD-tier block when all three are present (after MUST, before retrieved memories).
- Token budget tier ordering unchanged.
**Tests:** 4 minimum, added to `tests/test_prompt.py`:
1. With `guest_id=None`, output matches existing 2-entity behavior (regression).
2. With `guest_id="bot_b"` present and group_node populated, the assembled system message contains: speaker identity, guest activity, group_node summary, host→guest edge for the speaker.
3. Speaker is the guest (`speaker_bot_id == guest_id`), addressee="you" → guest's edges and group node correctly oriented.
4. Tight budget forces NICE-trim of guest activity → MUST blocks (speaker identity, edge_to_addressee, last 4 turns) survive.
**Commit:** `feat: multi-entity prompt assembly with guest activity, edges, group node`
---
### Task 45: Multi-entity per-POV summaries on scene close
**Files:**
- Modify: `chat/services/scene_summarize.py` (extend `apply_scene_close_summary` to write per-POV summaries for **each present witness** with a memory store, not just host)
- Modify: tests in `tests/test_per_pov_summary.py`
**Spec:** Phase 1's `apply_scene_close_summary` only summarizes from the host bot's POV. For Phase 2:
- Determine present witnesses with memory stores: host always; guest if `chat.guest_bot_id is not None`.
- For each, generate an independent per-POV summary via `summarize_scene` (the existing classifier wrapper). Each call uses **that bot's** persona, `you_name`, prior `bot → you` edge summary, and the same dialogue.
- Update each owner's memories of the closing scene with their per-POV summary.
- Update **all directed bot → you edges** with per-POV-derived `summary` content.
- If `group_node` exists for this chat, also append `group_node_updated` event with new `summary` and `dynamic` derived from the group view (run `summarize_scene` once with `bot_name="group"`, `bot_persona="all participants"` for a meta-summary). For v1 simplicity, the meta-summary can be naive concat of the host's per-POV summary + guest's per-POV summary; full LLM-merged group view is deferred to Phase 2.5.
**Tests:** 4 minimum, added to `tests/test_per_pov_summary.py`:
1. With no guest, behavior matches Phase 1 (regression test).
2. With guest, `apply_scene_close_summary` calls `summarize_scene` twice (one per bot witness) — assert mock called 2x.
3. After close, each bot's memories of the closed scene have their respective per-POV summary (different text).
4. With group_node present, after close `get_group_node(conn, chat_id).summary` is updated.
**Commit:** `feat: per-POV summaries on close for each present witness`
---
## Wave 4b — Turn flow integration (single task; depends on 4a)
T44 ties everything together. It modifies `chat/web/turns.py` (post_turn) and `chat/services/regenerate.py` to use the new multi-entity primitives. Must run after Wave 4a is merged so `assemble_narrative_prompt` accepts `guest_id` and `apply_scene_close_summary` handles guest.
### Task 44: Multi-entity turn flow
**Files:**
- Modify: `chat/web/turns.py` (rewrite `post_turn` to: parse turn → optionally close scene → assemble prompt with guest → narrative stream → write memories for ALL witnesses → state updates for ALL pairs → interjection check + interjection narrative if needed)
- Modify: `chat/services/regenerate.py` (mirror the changes — regenerated turn rebuilds with guest in scope)
- Modify: tests in `tests/test_turn_flow.py` (add multi-entity scenarios)
**Spec:** Refactored `post_turn` flow:
```
1. Validate prose (existing 400 check).
2. Look up chat, host_bot, guest_bot (None if no guest).
3. Parse turn (existing parse_turn).
4. Append user_turn event.
5. Append assistant_turn_started.
6. Detect scene close (existing path; runs even with guest).
7. (Recent dialogue read with multi-witness in mind — same query.)
8. Determine ADDRESSEE: simplest v2 heuristic — addressee is host unless
prose explicitly names guest_bot.name. Pass to assemble_narrative_prompt.
9. Assemble narrative prompt with speaker=addressee, guest_id passed.
10. Stream narrative; broadcast tokens; commit assistant_turn (existing).
11. Write memories: record_turn_memory_for_present(host, guest).
12. State updates: compute_state_updates_for_present, then append_and_apply
one edge_update per pair.
13. INTERJECTION CHECK (only if guest present and addressee != silent witness):
a. Call detect_interjection with the silent witness as candidate.
b. If should_interject: assemble narrative prompt with speaker=silent_witness,
addressee=host (or whoever just spoke), and instruct briefly.
c. Stream second narrative; broadcast as second turn_html; commit second
assistant_turn event.
d. Run state updates + memory writes for the interjection turn too
(smaller scope — just the interjector's outgoing edges + memories).
14. Scene close summary (existing path; now multi-witness via T45).
15. Broadcast turn_html for primary + interjection (if any).
16. Return 204.
```
**Addressee heuristic (Phase 2 v1):** simple substring match on bot names. If both names appear or neither: addressee defaults to host. Phase 2.5 / Phase 3 may improve with a classifier call.
**Cancel & truncated:** unchanged from Phase 1 — both halves of a streaming turn (primary + interjection) cancel together.
**`regenerate.py` changes:** parallel to `turns.py` — multi-entity prompt assembly + multi-witness memory + multi-pair state update. Interjection regeneration is deferred to Phase 2.5 (regenerate only the addressee's turn for v2).
**Tests added to `tests/test_turn_flow.py`:** 5 minimum.
1. Single-bot turn (no guest): full suite still passes (regression).
2. Multi-bot turn, no interjection: `post_turn` produces 1 user_turn + 1 assistant_turn + 6 edge_updates + 2 memory_written events. Mock interjection returns `should_interject=false`.
3. Multi-bot turn, with interjection: produces user_turn + 2 assistant_turns + 12 edge_updates + 4 memory_written events.
4. Multi-bot turn, scene close fires: `scene_closed` + multi-POV summaries written (per T45).
5. Addressee detection: prose `"BotB, what do you think?"` routes to BotB as speaker.
**Commit:** `feat: multi-entity turn flow with interjection support`
**Notes for implementer:**
- This task is the largest in Phase 2 by line count. Budget for ~150-300 lines of changes across `turns.py` and tests. The implementer should split commits if it helps clarity (one commit for primary turn, one for interjection, one for tests).
- Update the existing `_seed_chat` helper in `tests/test_turn_flow.py` to optionally seed a guest, and add `_seed_chat_with_guest` if cleaner.
- The fixture for the LLM mock now needs to provide canned responses for: parse_turn + scene_close_detect + narrative + state_updates×6 + interjection_decision + (optionally) interjection_narrative + state_updates×2 (interjection's outgoing only).
---
## Wave 5 — Polish (parallel)
Three independent tasks. Dispatch all three in parallel after Wave 4b merges.
### Task 46: Witness filter test coverage
**Files:**
- Create: `tests/test_witness_filter_multi.py`
**Spec:** Phase 1 tested witness filtering with single-bot scenarios. Phase 2 needs explicit tests for the cross-witness cases:
- Memory with witness `[1, 1, 0]`: visible to host, not guest (when guest queries from their POV).
- Memory with witness `[0, 1, 1]`: visible to host and guest, not "you".
- Secondhand-source memories: `source: "told_by:bot_a"`, witness flag for bot_b set, reliability < 1.0.
5 tests minimum.
**Commit:** `test: witness filter coverage for multi-entity scenarios`
---
### Task 47: Bot reset cascades to guest scenes
**Files:**
- Modify: `chat/state/entities.py` (`_apply_bot_reset` extended to also remove the bot's `guest_bot_id` references in OTHER chats: `UPDATE chats SET guest_bot_id = NULL WHERE guest_bot_id = ?`; remove the bot's activity row in those chats too)
- Modify: tests in `tests/test_reset.py` (add scenario: bot is guest in another's chat; reset clears the guest reference)
**Spec:** Currently `bot_reset` purges the bot's own chat state, memories, and edges. With Phase 2, a bot can be a guest in another bot's chat — that reference must also clear. Otherwise the host's chat sees a stale guest_bot_id pointing at a phantom bot.
Update `_apply_bot_reset` handler:
```python
# After existing purges:
conn.execute("UPDATE chats SET guest_bot_id = NULL WHERE guest_bot_id = ?", (bot_id,))
conn.execute("DELETE FROM activity WHERE entity_id = ?", (bot_id,)) # already there; covers all chats
```
(Activity is keyed by entity_id, so the existing line handles cross-chat activity rows already.)
**Tests:** 2 minimum, added to `tests/test_reset.py`.
1. BotB is guest in BotA's chat. Reset BotB. Assert `chat_bot_a.guest_bot_id` is NULL.
2. BotB has memories (witness flag set, owner=bot_b) from being guest in BotA's chat. Reset BotB. Assert those memories are gone.
**Commit:** `fix: bot_reset cascades to guest references in other chats`
---
### Task 48: Phase 2 documentation update
**Files:**
- Modify: `CLAUDE.md` (add "Phase 2 status" section; update "Behavioral defaults" with multi-entity additions; add to "Phase 1.5 / 2 cleanup backlog" any v2 follow-ups discovered during execution)
- Modify: `docs/plans/2026-04-26-v1-requirements-design.md` (mark Phase 2 deliverables as "shipped" in the appendix decisions log)
**Spec:** Documentation-only task. Run last in Phase 2 so it captures any deviations from the plan that emerged during execution. Reflect:
- Multi-entity scene support (you + host + guest).
- Interjection model (default false; explicit signals only).
- Per-POV summaries on close for all witnesses with memory stores.
- Group node populated on first 3-entity scene; updated on close.
- Phase 2 known limitations:
- "Meanwhile…" (scene config 4 — bot+bot without you) deferred to Phase 3.
- Interjection regeneration deferred (regenerate only acts on the addressee turn).
- Addressee detection is a simple name-match heuristic (no classifier call yet).
**Commit:** `docs: phase 2 status, behavioral defaults, deferred items`
---
## Wrap-up
After Wave 5 lands:
1. **Run full suite** on `phase-2`: should be ~210+ tests passing (168 from Phase 1 + ~45 new).
2. **Manual smoke**:
- Add a guest to one of the seeded bots' chats via the drawer.
- Verify "have they met?" prose seeds inter-bot edges.
- Play a few turns; verify host responds normally; verify guest occasionally interjects.
- Close the scene; check drawer for two distinct per-POV summaries.
- Remove guest mid-scene; check scene_closed fires.
- Reset a guest bot from another chat; verify guest_bot_id reference clears.
3. **Push `phase-2`** to gitea.
4. **Open PR** `phase-2 → main`.
5. **Phase 2.5 backlog candidates** (track in CLAUDE.md): interjection regenerate UI, classifier-based addressee detection, group-node LLM-merged meta-summary, drawer "first-meeting" gate vs "they already know each other" toggle, witness flag editing in drawer (currently read-only by spec).
---
## Notes for the controller running this plan
- **Don't dispatch Wave 4b until Wave 4a is merged AND tested green on `phase-2`.** Wave 4b's `turns.py` changes import the new `assemble_narrative_prompt` signature from Wave 4a's `prompt.py`; missing that produces import-time failures.
- **After each parallel wave**, the controller should run a code-review subagent (`subagent-driven-development` skill's two-stage review pattern) on each task before merging to `phase-2`. For purely mechanical tasks, a combined spec+quality review is acceptable.
- **If a parallel wave's merge produces a conflict**, the wave's file-disjointness assumption was violated. Bisect the affected pair, fix the offending task in a follow-up commit on `phase-2`, and proceed.
- **Token-spend rough estimate**: Phase 2 should be ~30-40% the size of Phase 1 (smaller scope; reuses Phase 1 patterns). Per-task token spend similar to Phase 1.
- **DO NOT modify Phase 1 code paths** unless explicitly required (e.g., Wave 5 T47 modifies `_apply_bot_reset` because the cascade is genuinely new behavior). The single-bot path must continue to work end-to-end after each wave.
@@ -0,0 +1,20 @@
{
"planPath": "docs/plans/2026-04-26-v2-phase2-implementation.md",
"tasks": [
{"id": 36, "subject": "T36: group_node schema + projector handlers", "status": "pending", "wave": 1, "parallelGroup": "wave-1"},
{"id": 37, "subject": "T37: guest_added / guest_removed event handlers", "status": "pending", "wave": 1, "parallelGroup": "wave-1"},
{"id": 38, "subject": "T38: relationship-seed service for first-co-appearance prompt", "status": "pending", "wave": 1, "parallelGroup": "wave-1"},
{"id": 39, "subject": "T39: interjection classifier service", "status": "pending", "wave": 2, "parallelGroup": "wave-2", "blockedBy": [37]},
{"id": 40, "subject": "T40: multi-entity state-update coordinator", "status": "pending", "wave": 2, "parallelGroup": "wave-2", "blockedBy": [37]},
{"id": 41, "subject": "T41: multi-witness memory write helper", "status": "pending", "wave": 2, "parallelGroup": "wave-2", "blockedBy": [37]},
{"id": 42, "subject": "T42: drawer guest add/remove + render", "status": "pending", "wave": 3, "parallelGroup": null, "blockedBy": [36, 37, 38]},
{"id": 43, "subject": "T43: multi-entity prompt assembly with guest activity, edges, group node", "status": "pending", "wave": 4, "parallelGroup": "wave-4a", "blockedBy": [36, 37]},
{"id": 45, "subject": "T45: per-POV summaries on close for each present witness", "status": "pending", "wave": 4, "parallelGroup": "wave-4a", "blockedBy": [36, 37]},
{"id": 44, "subject": "T44: multi-entity turn flow with interjection support", "status": "pending", "wave": 4, "parallelGroup": null, "blockedBy": [39, 40, 41, 43, 45]},
{"id": 46, "subject": "T46: witness filter test coverage for multi-entity scenarios", "status": "pending", "wave": 5, "parallelGroup": "wave-5", "blockedBy": [44]},
{"id": 47, "subject": "T47: bot_reset cascades to guest references in other chats", "status": "pending", "wave": 5, "parallelGroup": "wave-5", "blockedBy": [37]},
{"id": 48, "subject": "T48: Phase 2 documentation update", "status": "pending", "wave": 5, "parallelGroup": "wave-5", "blockedBy": [44]}
],
"lastUpdated": "2026-04-26T00:00:00Z",
"notes": "13 tasks across 6 waves (1, 2, 3, 4a, 4b, 5). Waves 1, 2, 4a, 5 are parallel-safe (file-disjoint within each). Waves 3 and 4b are single-task. Use Agent tool with isolation: 'worktree' to dispatch parallel tasks. Merge each wave's worktrees back into phase-2 before dispatching the next wave. See plan §Parallel-Execution Strategy for full guidance."
}
@@ -0,0 +1,596 @@
# Roleplay Engine — Phase 2.5 Cleanup Plan
> **For Claude:** REQUIRED SUB-SKILL: Use `superpowers-extended-cc:executing-plans` to implement this plan task-by-task. Use the parallel-dispatch pattern documented under "Parallel-Execution Strategy" for waves that fan out to multiple subagents.
**Goal:** Burn down the combined Phase 1.5 + Phase 2.5/3 backlog tracked in [`CLAUDE.md`](../../CLAUDE.md) §"Phase 1.5 cleanup backlog" and §"Phase 2.5 / 3 backlog". 15 follow-up items consolidated into 8 tasks (file-disjoint across waves) so several can run in parallel.
**Architecture:** No new architecture. Every change here is either a refactor (T68 `open_db`), a polish on an existing service/route (most tasks), or a UI affordance for state that already exists (T72 drawer edits, witness-flag editing). No new tables, no new event kinds, no schema migrations.
**Tech Stack:** Same as Phase 2. No new dependencies.
**Source-of-truth references:**
- Backlog list: [`CLAUDE.md`](../../CLAUDE.md) §"Phase 1.5 cleanup backlog" (5 items) + §"Phase 2.5 / 3 backlog" (10 items) = 15 items total.
- Conventions: [`CLAUDE.md`](../../CLAUDE.md) §"Behavioral defaults" + §"Phase 2 status".
- Phase 2 plan (style, TDD pattern, parallel-dispatch mechanics): [2026-04-26-v2-phase2-implementation.md](2026-04-26-v2-phase2-implementation.md).
- Phase 3 plan (in flight on a separate branch): [2026-04-26-v3-phase3-implementation.md](2026-04-26-v3-phase3-implementation.md).
When a task says "see §X", that's the requirements doc unless stated otherwise.
---
## Pre-flight
**Branch:** create `phase-2.5` from the latest `main` after Phase 2 has merged. If Phase 2 is still in PR review, branch off `phase-2` directly:
```bash
# Option A: after main has phase-2 merged
git checkout main && git pull && git checkout -b phase-2.5
# Option B: continue from phase-2 directly
git checkout phase-2 && git pull && git checkout -b phase-2.5
```
**Schema baseline:** Phase 2 leaves the DB at version 8. Phase 2.5 adds **no migrations**. Schema-version assertion in `tests/test_world.py` stays at 8.
**Relationship to Phase 3:** Phase 3 (`phase-3` branch, plan committed but not yet executed) uses task ids T49T67. Phase 2.5 uses **T68T75** to avoid collision regardless of merge order.
**Pinned non-negotiables (carried forward from Phases 1 + 2):**
- State changes go through the event log. Use `append_and_apply(conn, kind, payload)` for the live path; `apply_event` only after a fresh `append_event` returning the new id.
- Witness filter every memory read at SQL level (hard `WHERE` constraint; never a soft signal).
- Edges are directed; `botA → botB` and `botB → botA` are independent records.
- Per-POV scene summaries — never write omniscient narration.
- TDD: every task starts with a failing test (or, for refactors that preserve behavior, a regression test that pins the existing contract before any change).
- One commit per task minimum. Tasks that bundle 3+ small backlog items SHOULD split commits within the task — one commit per backlog item — so review can bisect cleanly.
**Verification before claiming done:** Use `superpowers-extended-cc:verification-before-completion` — run the test command, paste actual output. Don't assume green.
---
## Backlog item → task mapping
15 items consolidated into 8 tasks by **file ownership** (so each wave's tasks stay file-disjoint). Bundled tasks may split commits internally.
| # | Backlog item | Source | Task |
|---|--------------|--------|------|
| 1 | `open_db` refactor with `check_same_thread` parameter | Phase 1.5 | **T68** |
| 2 | Regenerate broadcasts `turn_html` over SSE | Phase 1.5 | **T73** |
| 3 | `bot_reset` purges orphaned "you" activity rows | Phase 1.5 | **T69** |
| 4 | Drawer edits for deferred v1 fields (edge_trust, edge_summary, memory pov_summary, knowledge_facts) | Phase 1.5 | **T72** |
| 5 | NICE trim order in prompt assembly | Phase 1.5 | **T71** |
| 6 | Interjection regenerate | Phase 2.5 | **T73** |
| 7 | Classifier-based addressee detection | Phase 2.5 | **T74** |
| 8 | LLM-merged group meta-summary | Phase 2.5 | **T70** |
| 9 | First-meeting gate (drawer "have they met?" toggle) | Phase 2.5 | **T72** |
| 10 | Witness flag editing in drawer | Phase 2.5 | **T72** |
| 11 | Significance for interjection memories | Phase 2.5 | **T74** |
| 12 | Stale guest reference defensive degrade removal | Phase 2.5 | **T73 + T74** (split by file) |
| 13 | Scene close on cancel review | Phase 2.5 | **T74** |
| 14 | Dual `ACTIVITIES:` block consolidation | Phase 2.5 | **T71** |
| 15 | Witness role hardcode in prompt assembly | Phase 2.5 | **T71** |
| — | Docs sweep — remove shipped items from CLAUDE.md | (this plan) | **T75** |
---
## Parallel-Execution Strategy
Same pattern as Phases 2 and 3. Five waves: parallel within each wave (file-disjoint), serial across waves. Cross-wave merges keep `phase-2.5` green between dispatches.
### How to dispatch a wave in parallel
Use the **Agent tool with `isolation: "worktree"`** so each subagent gets its own git worktree. (If the controlling session's working directory is **not** the chat repo, create worktrees manually with `git worktree add .worktrees/<wave>-<task> -b <wave>/<task> phase-2.5` from inside the chat repo and pass the worktree path explicitly into each subagent prompt — that is the pattern Phase 2 used.)
In a single message, dispatch all tasks in the wave:
```
Agent({
description: "Wave 1 — T68 open_db refactor",
subagent_type: "general-purpose",
isolation: "worktree",
prompt: "<full task text from below>",
})
Agent({ ...T69... })
Agent({ ...T70... })
```
### After a wave completes
1. Each subagent returns its worktree path and commit SHA(s).
2. **Run a spec + code-quality reviewer subagent on each completed task.** Combined review is acceptable for purely mechanical refactors (T68, T69); separate spec + quality reviewers for tasks that bundle multiple backlog items (T71, T72, T74).
3. **Merge the wave into `phase-2.5`** in any order (file-disjointness guarantees no conflict). Use `--no-ff`:
```bash
git checkout phase-2.5
for branch in <wave-branches>; do
git merge --no-ff "$branch" -m "merge: <task description>"
done
```
4. **Run the full test suite** on the merged `phase-2.5`. If it's red, the wave's mutual-independence assumption was violated — bisect the offending pair, fix, re-merge.
5. **Push `phase-2.5`** to gitea so the work is durable before the next wave starts.
6. Optionally clean up worktrees: `git worktree remove .worktrees/<branch>` and `git branch -D <branch>`.
### Conflict prevention checklist (apply before dispatch)
For each parallel wave, verify the **Files** sections of all tasks have **no overlapping paths**. The waves below are designed to satisfy this; if you decide to add or merge tasks, re-check.
The hot files in this plan are: `chat/web/turns.py`, `chat/services/regenerate.py`, `chat/web/drawer.py`, `chat/templates/_drawer.html`, `chat/services/prompt.py`. Each is owned by exactly one task in this plan.
### Failure recovery
If one subagent fails: cancel it, merge the others' successful work, re-dispatch the failed task as a single follow-up. Don't block the wave.
If a failure exposes a bad assumption shared by multiple tasks (e.g., a refactor that requires a wider blast radius than the plan accounted for), pause the wave and revisit.
### Why each wave is parallel-safe
| Wave | Tasks | Hot files touched | Disjoint? |
|------|-------|-------------------|-----------|
| 1 | T68, T69, T70 | `chat/db/connection.py` + `chat/web/bots.py` (T68); `chat/state/entities.py` (T69); `chat/services/scene_summarize.py` (T70) | ✅ |
| 2 | T71 | `chat/services/prompt.py` | (single task) |
| 3 | T72 | `chat/web/drawer.py` + `chat/templates/_drawer.html` | (single task) |
| 4 | T73, T74 | `chat/services/regenerate.py` (T73); `chat/web/turns.py` + new `chat/services/addressee.py` (T74) | ✅ |
| 5 | T75 | `CLAUDE.md` | (single task) |
---
## Task overview
```
Wave 1 ─┬─ T68: open_db refactor with check_same_thread param
├─ T69: bot_reset purges orphaned "you" activity rows
└─ T70: LLM-merged group meta-summary
Wave 2 ─── T71: prompt.py polish (NICE trim order + dual ACTIVITIES + witness role parametric)
Wave 3 ─── T72: drawer.py polish (deferred v1 edits + first-meeting gate + witness flag editing)
Wave 4 ─┬─ T73: regenerate.py polish (turn_html SSE + interjection regenerate + stale-guest cleanup)
└─ T74: turn-flow polish + addressee service (classifier addressee detection +
significance for interjection + scene close on cancel + stale-guest cleanup)
Wave 5 ─── T75: docs sweep — remove shipped items from CLAUDE.md backlogs
```
Critical path: 5 sequential merge points. Total tasks: 8. Wall-clock parallelism advantage: Waves 1 and 4 dispatch concurrently; Waves 2, 3, 5 are single-task by file constraint.
---
## Wave 1 — Independent small fixes (parallel)
Three tasks, fully file-disjoint.
### Task 68: `open_db` refactor with `check_same_thread` parameter
**Files:**
- Modify: `chat/db/connection.py` (extend `open_db(path, *, check_same_thread=True)` so callers can opt out of SQLite's main-thread requirement)
- Modify: `chat/web/bots.py` (use the new parameter in `get_conn` rather than hand-rolling its own context-manager body)
- Modify: tests in `tests/test_connection.py` (or wherever `open_db` is tested; add 1 test for the new parameter)
**Spec:** Currently `chat/web/bots.py:get_conn()` duplicates the body of `open_db` so it can pass `check_same_thread=False`. Extend `open_db` to accept this as a kwarg (default True, preserving existing behavior). Then have `get_conn` call `open_db(...)` directly. The PRAGMA setup (WAL, foreign_keys, synchronous, etc.) stays in one place.
**Step 1: failing test** — add a regression test that pins the existing contract:
```python
def test_open_db_default_uses_check_same_thread_true(tmp_path):
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
# Default is check_same_thread=True; calling from another thread should fail.
...
def test_open_db_can_disable_check_same_thread(tmp_path):
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db, check_same_thread=False) as conn:
# Same conn callable from another thread now.
...
```
**Step 3: implementation** — add `check_same_thread: bool = True` to `open_db`. Pass through to `sqlite3.connect`. Then in `chat/web/bots.py`, replace the duplicated context-manager body with `open_db(path, check_same_thread=False)`.
**Step 5: commit** — `refactor: open_db with check_same_thread parameter (T68)`.
**Notes for implementer:**
- This is a refactor — the full test suite must be GREEN before AND after. Run before to baseline, run after to confirm no regressions. Pay special attention to `tests/test_bots.py` if it exercises the `get_conn` path.
- Do NOT change the default. Existing callers don't pass `check_same_thread` and must continue to get `True`.
---
### Task 69: `bot_reset` purges orphaned "you" activity rows
**Files:**
- Modify: `chat/state/entities.py` (extend `_apply_bot_reset` with one more `DELETE` clause for "you" activity rows tied to chats that this bot hosted)
- Modify: tests in `tests/test_reset.py` (add 2 tests)
**Spec:** Currently `_apply_bot_reset` purges the bot's chats, the bot's own activity rows, the bot's memories, and edges involving the bot. Phase 2 T47 added a `chats.guest_bot_id` cascade. Still missing: when bot A's chats are deleted, "you"-owned activity rows that were associated with those chats' containers are not cleaned up. They linger as orphaned activity entries pointing at deleted containers.
The fix per the existing CLAUDE.md note:
```sql
DELETE FROM activity
WHERE entity_id = 'you'
AND container_id IN (SELECT id FROM containers WHERE chat_id IN (
SELECT id FROM chats WHERE host_bot_id = ?
));
```
Order matters: this `DELETE` must run BEFORE the `DELETE FROM containers` and `DELETE FROM chats` clauses — otherwise the subqueries return no rows. Verify ordering in the existing handler before placing the new line.
**Tests:** 2 added.
1. `test_reset_purges_orphaned_you_activity_rows`: seed bot_a, chat_bot_a, a container in chat_bot_a, and a "you" activity row pointing at that container. Reset bot_a. Assert `SELECT COUNT(*) FROM activity WHERE entity_id = 'you'` is 0.
2. `test_reset_does_not_purge_you_activity_in_other_chats`: seed bot_a + bot_b, both with chats and "you" activity in each. Reset bot_a. Assert "you" activity in chat_bot_a is gone, but "you" activity in chat_bot_b is preserved.
**Commit:** `fix: bot_reset purges orphaned 'you' activity rows (T69)`.
---
### Task 70: LLM-merged group meta-summary
**Files:**
- Modify: `chat/services/scene_summarize.py` (replace the naive `f"{host_name}: {host_summary}\n\n{guest_name}: {guest_summary}"` with an LLM-merged group view via a new classifier wrapper)
- Modify: tests in `tests/test_per_pov_summary.py` (replace the regression test for naive concat with one that asserts the merged text uses the classifier output; keep the existing per-POV memory tests intact)
**Spec:** Phase 2 T45 wrote a stub for `group_node.summary` that just concatenated the two per-POV summaries. Replace it with a small classifier call that produces a coherent group-level summary from both POVs.
Add a new helper at the bottom of `scene_summarize.py`:
```python
class GroupMetaSummary(BaseModel):
summary: str = ""
dynamic: str = ""
async def merge_group_summary(
client: LLMClient,
*,
classifier_model: str,
host_name: str,
host_pov_summary: str,
guest_name: str,
guest_pov_summary: str,
timeout_s: float = 30.0,
) -> GroupMetaSummary:
"""Merge two per-POV scene summaries into a coherent group-level
summary + group-dynamic note. Falls back to the naive concat on
classifier failure."""
```
System prompt: "Given two per-POV scene summaries from a 3-entity scene (you + host + guest), produce a coherent group-level summary capturing the shared events as both witnesses experienced them, plus a brief 'dynamic' note describing the trio's group dynamic during the scene." Output strict JSON matching schema. Default = `GroupMetaSummary(summary=f"{host_name}: {host_pov_summary}\n\n{guest_name}: {guest_pov_summary}", dynamic="")` (the existing naive concat preserved as fallback so a classifier failure doesn't degrade behavior).
In `apply_scene_close_summary`, replace the naive concat call site (the existing `summary=` kwarg of the `group_node_updated` event) with `await merge_group_summary(...)` and use its `.summary` and `.dynamic` outputs.
**Tests:** 3 in `tests/test_per_pov_summary.py`.
1. `test_group_summary_merges_per_pov_via_classifier_when_guest_present`: mock the classifier with `GroupMetaSummary(summary="merged summary", dynamic="warm rapport")`. Close a scene with guest. Assert `get_group_node(...).summary == "merged summary"` and `.dynamic == "warm rapport"`.
2. `test_group_summary_falls_back_to_naive_concat_on_classifier_failure`: mock classifier with bad JSON across all 3 retries. Close scene. Assert `summary` matches the old naive concat format. `dynamic` is empty.
3. `test_group_summary_skipped_when_no_guest`: no-guest path unchanged — `group_node_updated` not emitted at all (existing behavior).
**Commit:** `feat: LLM-merged group meta-summary (T70)`.
---
## Wave 2 — `prompt.py` polish (single task)
T71 bundles three prompt-assembly cleanups. All touch `chat/services/prompt.py`. Single task because the file is hot; the implementer SHOULD split into 3 commits within the task for clean review bisection.
### Task 71: prompt.py polish (NICE trim order + dual ACTIVITIES + witness role parametric)
**Files:**
- Modify: `chat/services/prompt.py`
- Modify: `tests/test_prompt.py` (add tests; preserve existing 10 tests)
**Spec:** Three independent cleanups bundled because the file is hot.
#### 71.1 — Witness role parametric (Phase 2.5 backlog #15)
`chat/services/prompt.py:436` (or wherever the call site is — verify) calls `search_memories(conn, speaker_bot_id, "host", query, k=4)` with `witness_role="host"` hardcoded. This is wrong when the speaker is the guest (the guest queries with `witness_role="guest"` should hit a different SQL filter).
Fix: derive the role from chat membership.
```python
def _witness_role_for(speaker_bot_id: str, host_bot_id: str) -> str:
return "host" if speaker_bot_id == host_bot_id else "guest"
```
Apply at the call site. The test contract is already pinned in `tests/test_witness_filter_multi.py` from Phase 2 T46 — those tests will continue to pass; this change unblocks guest-as-speaker in production.
**Commit:** `fix: witness role parametric in prompt assembly (T71.1)`.
#### 71.2 — Dual `ACTIVITIES:` block consolidation (Phase 2.5 backlog #14)
T43 (Phase 2) added a second `ACTIVITIES:` block to render guest activity separately from you+speaker activity (so the trim ladder could drop guest activity first under tight budget). Two consecutive `ACTIVITIES:` headers can read as a duplicate-section bug to the LLM.
Refactor to a single `ACTIVITIES:` block with three bullets (you, speaker, guest), where each bullet is independently trimmable: under tight budget, drop the guest bullet first, then the you bullet, keeping the speaker bullet (the speaker's own current activity is MUST-tier).
Implementation: the existing trim machinery uses block-level granularity. Extend it to bullet-level granularity for this block (one new helper or one new tier name like `MUST-bullet` / `SHOULD-bullet` / `NICE-bullet` — pick whichever is least disruptive).
**Commit:** `refactor: single ACTIVITIES: block with bullet-level trim (T71.2)`.
#### 71.3 — NICE trim order revisit (Phase 1.5 backlog #5)
Per T18 review: the NICE trim drops previous-scene first instead of last (the spec listing order was previous-scene last). Greedy-cuts heuristic vs. spec.
Revisit: review the trim ordering carefully. If real play surfaces a regression (the previous-scene block is genuinely important to bot continuity), reverse the NICE order so previous-scene drops last. If not, document the intentional deviation in a code comment and call it done.
**This is a judgment call.** Default action: leave the order as-is and add a comment explaining why (the heuristic is "drop the cheapest-impact thing first; greedy lookahead is more expensive than the marginal narrative loss"). If review feedback during execution disagrees, reverse the order.
**Commit:** `chore: document NICE trim order rationale (T71.3)` OR `fix: NICE trim order drops previous-scene last (T71.3)`.
#### Tests for T71
Add to `tests/test_prompt.py`:
1. `test_speaker_is_guest_uses_guest_witness_role`: speaker=guest_id. Patch `search_memories` to record its `witness_role` argument. Assert called with `"guest"`, not `"host"`.
2. `test_single_activities_block_with_three_bullets_when_3_entities`: 3-entity prompt. Assert exactly one `ACTIVITIES:` header present. Assert bullets for you, speaker, guest.
3. `test_tight_budget_drops_guest_activity_bullet_first`: 3-entity prompt with budget tight enough to force trim. Assert speaker activity bullet survives, guest activity bullet is dropped.
4. (Optional, depends on 71.3 outcome) `test_nice_trim_order_drops_previous_scene_last`: only add if you choose to fix the order.
**Verification gates:**
- `pytest tests/test_prompt.py -v` — 10 existing + 3-4 new all pass.
- `pytest tests/test_witness_filter_multi.py -v` — Phase 2 T46 tests still pass (proves the witness-role fix didn't break anything).
- Full suite green.
---
## Wave 3 — `drawer.py` polish (single task)
T72 bundles three drawer affordances. All touch `chat/web/drawer.py` and `chat/templates/_drawer.html`. Single task by file constraint; implementer SHOULD split into 3 commits.
### Task 72: drawer polish (deferred v1 edits + first-meeting gate + witness flag editing)
**Files:**
- Modify: `chat/web/drawer.py` (add 4-5 new POST routes for the deferred v1 edits + 1 GET extension for first-meeting gate + 1 POST for witness flag editing)
- Modify: `chat/templates/_drawer.html` (forms for each new edit affordance)
- Create: `tests/test_drawer_edits_extended.py` (new tests for the new routes; existing `tests/test_drawer_edits.py` and `tests/test_drawer_guest.py` stay unchanged)
**Spec:** Three independent backlog items.
#### 72.1 — Deferred v1 drawer edits (Phase 1.5 backlog #4)
The `manual_edit` projector already supports `target_kind` values for `edge_trust`, `edge_summary`, `memory_pov_summary`. These work end-to-end at the state layer; only the drawer routes are missing.
Add 4 new POST routes:
1. `POST /chats/{chat_id}/drawer/edge/trust` — form `{source_id, target_id, new_value}` (0100 int). Appends `manual_edit` with `target_kind="edge_trust"`, `prior_value=current_trust`, `new_value=...`. Validate range; 400 on out-of-bounds.
2. `POST /chats/{chat_id}/drawer/edge/summary` — form `{source_id, target_id, new_summary}` (text). Appends `manual_edit` with `target_kind="edge_summary"`. No validation beyond non-empty + reasonable length cap (e.g., 2000 chars).
3. `POST /chats/{chat_id}/drawer/memory/pov-summary` — form `{memory_id, new_summary}`. Appends `manual_edit` with `target_kind="memory_pov_summary"`. 404 if memory not in this chat or not owned by a present bot.
4. `POST /chats/{chat_id}/drawer/edge/knowledge-facts` — form `{source_id, target_id, action: 'add'|'remove', fact: str}`. Knowledge_facts needs a NEW dispatch branch in the `manual_edit` projector — add it as part of this task: `target_kind="edge_knowledge_fact"` with payload action + fact.
The existing drawer template has read-only renders for these fields. Replace with editable forms (textarea + slider + button).
Tests in `tests/test_drawer_edits_extended.py`:
- One test per route (4 tests minimum) asserting: the manual_edit event lands; the projected state changes; the response contains the updated drawer partial.
**Commit:** `feat: drawer edits for edge_trust / edge_summary / memory_pov_summary / knowledge_facts (T72.1)`.
#### 72.2 — First-meeting gate (Phase 2.5 backlog #9)
The "Add guest" form's `relationship_prose` textarea fires every time. In Phase 2 T42's notes: "fire it every time a `(host, guest)` pair has no existing `host → guest` edge."
Implement the gate: when the user opens the Add-guest form, check whether `get_edge(conn, host_bot_id, guest_bot_id)` already exists. If yes:
- Render the textarea disabled with the message "they already know each other (edge exists from a prior chat)" + a small "re-seed anyway" toggle that re-enables the textarea.
- If the user submits without toggling, skip the relationship-seed call (existing edge content stays).
- If the user toggles re-seed and submits prose, the existing flow runs — `seed_inter_bot_edges` produces deltas, two `edge_update` events fire on top of the existing edge content.
Tests:
1. `test_add_guest_form_disables_prose_when_edge_exists`: pre-seed a host→guest edge from a prior chat; render the form; assert the textarea has `disabled` attribute AND the "they already know each other" message is in the body.
2. `test_add_guest_with_existing_edge_skips_seed_call`: pre-seed edge; submit form without toggling re-seed; assert classifier mock was NOT called (count check on canned-response queue).
**Commit:** `feat: first-meeting gate on drawer Add-guest form (T72.2)`.
#### 72.3 — Witness flag editing (Phase 2.5 backlog #10)
Memories show witness flags `[you, host, guest]` read-only in the drawer. Add an inline-edit affordance: each flag becomes a checkbox; toggling submits a `manual_edit` event with `target_kind="memory_witness"`, payload `{memory_id, flag: 'you'|'host'|'guest', new_value: bool}`.
The `manual_edit` projector needs a new dispatch branch for `memory_witness` — same as the knowledge_facts branch in 72.1; do them together if cleaner.
Tests: 2.
1. `test_witness_flag_toggle_updates_memory_row`: seed memory with witness `[1, 1, 0]`. POST toggle on `guest` flag → 1. Project. Assert `memories.witness_guest = 1`.
2. `test_witness_flag_toggle_emits_manual_edit_event`: same setup; assert the manual_edit event has the right `target_kind` and `prior_value`/`new_value`.
**Commit:** `feat: drawer witness flag inline-edit (T72.3)`.
---
## Wave 4 — Turn-flow polish (parallel)
Two tasks, file-disjoint. T73 owns `chat/services/regenerate.py`; T74 owns `chat/web/turns.py` + adds a new addressee-detection service.
Each task bundles multiple backlog items. Implementer should split commits within each task.
### Task 73: `regenerate.py` polish
**Files:**
- Modify: `chat/services/regenerate.py`
- Modify: `tests/test_regenerate.py` (add tests; existing tests preserved)
**Spec:** Three regenerate-related backlog items.
#### 73.1 — Regenerate broadcasts `turn_html` over SSE (Phase 1.5 backlog #2)
After the new `assistant_turn` lands, broadcast a `turn_html` event over the chat's pub/sub channel — mirror the broadcast logic in `chat/web/turns.py:post_turn`. The existing `post_turn` does this via `publish(chat_id, {"event": "turn_html", "html": ...})` (or similar — verify). Use the same render path so connected tabs swap the regenerated turn live, no refresh required.
Test: `test_regenerate_broadcasts_turn_html_over_sse` — mock `publish` and assert it was called with the new `assistant_turn`'s rendered HTML.
**Commit:** `feat: regenerate broadcasts turn_html over SSE (T73.1)`.
#### 73.2 — Interjection regenerate (Phase 2.5 backlog #6)
Phase 2 T44 deferred interjection regenerate: regenerate currently only acts on the addressee turn. Extend so that when a turn group has both a primary `assistant_turn` and an `assistant_turn` flagged as `interjection_of=...`, regenerate redoes BOTH — the primary first, then the interjection (using the same interjection-decision classifier path as `post_turn`). The interjection branch may decide `should_interject=False` on the regenerate, in which case the previous interjection_turn is superseded but no new interjection is appended.
Test: `test_regenerate_with_interjection_redoes_both_turns` — seed a 3-entity scene with a prior primary + interjection; regenerate; assert two new assistant_turns land (or one new + a supersede-without-replace if the regenerated decision was "no interjection").
**Commit:** `feat: regenerate covers interjection turns (T73.2)`.
#### 73.3 — Stale-guest defensive degrade cleanup in regenerate.py (Phase 2.5 backlog #12, partial)
Phase 2 T44 added a defensive degrade-to-1:1 in `regenerate.py` when `chat.guest_bot_id` points at a deleted bot. T47 fixed the root cause (resets clear the reference). The defensive degrade is now dead code.
Remove the degrade block; let the function trust that `chat.guest_bot_id` is either valid or NULL. The corresponding existing test for the defensive degrade can be removed (the bot_reset cascade test in `tests/test_reset.py` already covers the root-cause behavior).
**Commit:** `chore: remove defensive stale-guest degrade in regenerate.py (T73.3)`.
#### Verification gates
- `pytest tests/test_regenerate.py -v` — existing + new all pass.
- Full suite green.
---
### Task 74: turn-flow polish + new addressee-detection service
**Files:**
- Modify: `chat/web/turns.py`
- Create: `chat/services/addressee.py` (new classifier wrapper for addressee detection)
- Create: `tests/test_addressee.py`
- Modify: `tests/test_turn_flow.py` (add tests; existing 8 tests preserved)
**Spec:** Four turn-flow backlog items.
#### 74.1 — Classifier-based addressee detection (Phase 2.5 backlog #7)
Phase 2 T44's `_detect_addressee_id` uses a substring whole-word regex match. This is brittle: bot names that are common English words (e.g., a bot named "Sam"), names appearing inside a quoted aside ("Did you see what Sam wrote in his letter?" — addressed to host, not Sam), or fuzzy references all break it.
Replace with a small classifier call. New module `chat/services/addressee.py`:
```python
class AddresseeDecision(BaseModel):
addressee_id: str # bot id, "you", or "host" as fallback
confidence: str = "medium" # "high" | "medium" | "low"
reason: str = ""
async def detect_addressee(
client: LLMClient,
*,
classifier_model: str,
user_prose: str,
host_id: str,
host_name: str,
guest_id: str | None,
guest_name: str | None,
timeout_s: float = 30.0,
) -> AddresseeDecision:
"""Classify which present bot the user is addressing in this turn.
Defaults to host on failure or low confidence."""
```
System prompt: "Given a user's turn prose and the names of present bots, decide which bot the user is addressing. If the user is speaking to no specific bot (descriptive narration, action without dialogue), default to the host. Output strict JSON."
Default fallback (classifier failure) = `AddresseeDecision(addressee_id=host_id, confidence="low", reason="fallback")`.
In `chat/web/turns.py`, replace `_detect_addressee_id` with a call to `detect_addressee`. Keep the substring helper as a low-confidence pre-filter for the no-guest case (no LLM call needed when only one bot is present — preserves throughput).
Tests:
- `tests/test_addressee.py` (new file): 3 tests — classifier returns guest, classifier returns host, classifier failure falls back to host.
- `tests/test_turn_flow.py`: update `test_addressee_detection_routes_to_named_bot` from Phase 2 T44 to use the new classifier path. (Existing test should keep passing with the new mock orchestration; canned-response queue may need an extra slot for the addressee decision.)
**Commit:** `feat: classifier-based addressee detection (T74.1)`.
#### 74.2 — Significance for interjection memories (Phase 2.5 backlog #11)
Phase 2 T44 noted: the interjection branch's `memory_written` event doesn't enqueue a `SignificanceJob`. Wire it in: after the interjection memory write (the `record_turn_memory_for_present` call in the interjection branch), enqueue a `SignificanceJob` with the interjection's host memory id (mirror the primary turn's enqueue at the end of the primary branch).
If both host and guest memory ids exist for the interjection (as they will when both are present), enqueue once for the host id (the existing pattern for primary turns — the score applies to both POVs since the prose is identical at the time of write).
Test: `test_interjection_enqueues_significance_job` — mock the worker; trigger an interjection; assert `SignificanceJob` was enqueued with the interjection memory id.
**Commit:** `fix: enqueue significance for interjection memories (T74.2)`.
#### 74.3 — Scene close on cancel review (Phase 2.5 backlog #13)
Phase 2 T44 review noted: when a primary turn is cancelled mid-stream, scene close still runs. Behavior may be intentional (close detection looks at user prose, not bot output) or wrong (a cancelled turn is incomplete; closing the scene on it is premature).
**Decision for this task:** review the call path. If the close detection truly only consults user prose AND the user prose is fully present at the moment of cancel (it is — user prose is appended before the stream starts), the existing behavior is correct: a cancelled turn doesn't invalidate the user's intent to close the scene. Document this in a code comment near the close-detection branch.
If a play-test surfaces a regression (e.g., a user cancels because the bot misread their close intent), revisit. Default: document and close as a no-op.
Test: `test_cancelled_turn_still_closes_scene_when_user_prose_signals_close` — pin the existing behavior so a future refactor doesn't quietly change it.
**Commit:** `chore: pin scene-close-on-cancel behavior + comment rationale (T74.3)`.
#### 74.4 — Stale-guest defensive degrade cleanup in turns.py (Phase 2.5 backlog #12, partial)
Same as T73.3 but for `chat/web/turns.py`: T44's defensive degrade-to-1:1 in `post_turn` (lines 235-242 per the T44 implementer note) is dead code now that T47 fixed the root cause. Remove it.
**Commit:** `chore: remove defensive stale-guest degrade in turns.py (T74.4)`.
#### Verification gates
- `pytest tests/test_addressee.py -v` — 3/3 new tests pass.
- `pytest tests/test_turn_flow.py -v` — existing 8 + new 2-3 all pass.
- `pytest tests/test_reset.py -v` — Phase 2 T47 root-cause cascade still green.
- Full suite green.
---
## Wave 5 — Docs sweep (single task)
### Task 75: Remove shipped items from CLAUDE.md backlogs
**Files:**
- Modify: `CLAUDE.md`
**Spec:** Walk through the 15 backlog items in `CLAUDE.md` §"Phase 1.5 cleanup backlog" and §"Phase 2.5 / 3 backlog". For each item shipped during Phases 2.5 (T68T74), remove it from the backlog list. Add a new section "Phase 2.5 status" near the existing "Phase 2 status" section listing what shipped:
- `open_db` refactor (T68).
- `bot_reset` purges orphaned "you" activity rows (T69).
- LLM-merged group meta-summary (T70).
- Prompt assembly polish: witness role parametric, single ACTIVITIES block, NICE trim documented (T71).
- Drawer edits for deferred v1 fields, first-meeting gate, witness flag editing (T72).
- Regenerate over SSE + interjection regenerate + stale-guest cleanup (T73).
- Classifier-based addressee detection + significance for interjection + scene-close-on-cancel pinned + stale-guest cleanup (T74).
If any task during execution chose NOT to ship a sub-item (e.g., T71.3 left NICE trim unchanged with a documented rationale), keep that sub-item in a "Phase 3.5+ deferred" section with the rationale. The goal is for the backlog list to reflect actual repo state, not aspirational scope.
If any new follow-ups were discovered during T68T74 reviews, add them to the appropriate backlog section.
**Commit:** `docs: phase 2.5 status, prune shipped backlog items (T75)`.
---
## Wrap-up
After Wave 5 lands:
1. **Run full suite** on `phase-2.5`: should be ~225+ tests passing (212 from Phase 2 + ~15 new across the 8 tasks).
2. **Manual smoke** (recommended before opening the PR):
- Drawer: edit edge_trust on a chat; verify the new value sticks after refresh.
- Drawer: edit edge_summary on a chat; refresh; verify.
- Drawer: toggle a memory's witness flag; refresh; verify.
- Drawer: open Add-guest form for a (host, guest) pair that already shares an edge; verify the gate disables the prose textarea.
- Drawer: open Add-guest form for a fresh pair; verify the textarea is enabled.
- Reset a bot; verify "you" activity rows for that bot's chats are gone (run `sqlite3 data/db.sqlite "SELECT * FROM activity WHERE entity_id='you'"` before/after).
- Multi-tab: open two tabs on the same chat; click Regenerate on one; verify the other tab sees the new turn live (no refresh).
- Trigger an interjection turn; check the worker queue or `significance_jobs` table; verify a job was enqueued for the interjection memory.
- Use a bot with a name that's a common word ("Sam"); ask "did you see what Sam wrote?" — verify host gets the floor (classifier addressee detection, not substring).
3. **Push `phase-2.5`** to gitea.
4. **Open PR** `phase-2.5 → main`.
5. **No new Phase 3+ backlog items expected** — if review surfaces any, add to CLAUDE.md.
---
## Notes for the controller running this plan
- **Don't dispatch Wave 4 until Wave 3 is merged AND tested green on `phase-2.5`.** T74 references the new addressee service path that's stand-alone, but the existing tests in `tests/test_turn_flow.py` may have shifted from Wave 3 if the drawer-test fixture interactions touch shared state. Verify green before fanning out.
- **After each parallel wave**, run a code-review subagent (`subagent-driven-development` skill's two-stage review pattern) on each task. For purely mechanical tasks (T68, T69), combined spec+quality is acceptable. For bundled tasks (T71, T72, T74), use separate spec + quality reviewers — the surface area is larger.
- **If Phase 3 (`phase-3` branch) is in flight in parallel**, T75 (the docs sweep) should land on `phase-2.5` only — Phase 3's docs sweep (T67) is independent. Both will resolve when the two branches merge to `main` in some order; expect a small CLAUDE.md merge to reconcile any overlapping backlog edits.
- **If a task's "split commits" guidance proves impractical** (e.g., bundling means a test pins 3 fixes at once), one consolidated commit is acceptable. The split is an aid for review bisection, not a hard rule.
- **Token-spend rough estimate**: Phase 2.5 should be ~50% the size of Phase 2 (smaller scope, all reuse). Per-task token spend similar to Phase 2's smaller tasks (T36, T37, T47).
- **DO NOT break existing v1 / v2 surface contracts.** Every test file that was green at the start of Phase 2.5 must stay green at the end. The `tests/test_witness_filter_multi.py` contracts pinned in Phase 2 T46 are particularly load-bearing for T71.1 — verify them after the witness-role parametric fix lands.
@@ -0,0 +1,15 @@
{
"planPath": "docs/plans/2026-04-26-v2.5-phase2.5-cleanup.md",
"tasks": [
{"id": 68, "subject": "T68: open_db refactor with check_same_thread parameter", "status": "pending", "wave": 1, "parallelGroup": "wave-1"},
{"id": 69, "subject": "T69: bot_reset purges orphaned 'you' activity rows", "status": "pending", "wave": 1, "parallelGroup": "wave-1"},
{"id": 70, "subject": "T70: LLM-merged group meta-summary", "status": "pending", "wave": 1, "parallelGroup": "wave-1"},
{"id": 71, "subject": "T71: prompt.py polish (NICE trim + dual ACTIVITIES + witness role)", "status": "pending", "wave": 2, "parallelGroup": null},
{"id": 72, "subject": "T72: drawer polish (deferred v1 edits + first-meeting gate + witness flag editing)", "status": "pending", "wave": 3, "parallelGroup": null},
{"id": 73, "subject": "T73: regenerate.py polish (turn_html SSE + interjection regenerate + stale-guest cleanup)", "status": "pending", "wave": 4, "parallelGroup": "wave-4", "blockedBy": [72]},
{"id": 74, "subject": "T74: turn-flow polish + addressee service (classifier addressee + significance interjection + scene close on cancel + stale-guest cleanup)", "status": "pending", "wave": 4, "parallelGroup": "wave-4", "blockedBy": [72]},
{"id": 75, "subject": "T75: docs sweep — remove shipped items from CLAUDE.md", "status": "pending", "wave": 5, "parallelGroup": null, "blockedBy": [73, 74]}
],
"lastUpdated": "2026-04-26T00:00:00Z",
"notes": "8 tasks across 5 waves consolidating 15 backlog items (5 from Phase 1.5, 10 from Phase 2.5/3). Waves 1 and 4 are parallel-safe (file-disjoint within each). Waves 2, 3, 5 are single-task by hot-file constraint (prompt.py, drawer.py, CLAUDE.md). Bundled tasks (T71, T72, T74) split into sub-commits per backlog item for clean review bisection. No schema migrations — schema baseline stays at version 8. Phase 3 plan uses T49-T67; this plan uses T68-T75 to avoid id collision regardless of merge order."
}
@@ -0,0 +1,891 @@
# Roleplay Engine — Phase 3 Implementation Plan
> **For Claude:** REQUIRED SUB-SKILL: Use `superpowers-extended-cc:executing-plans` to implement this plan task-by-task. Use the parallel-dispatch pattern documented under "Parallel-Execution Strategy" for waves that fan out to multiple subagents.
**Goal:** Add events with lifecycles, time skips (elision + jump), active threads, significance/retrieval refinements, and "Meanwhile…" scenes (host+guest with no "you" present). All scoped to a single chat; the cross-chat surface remains unchanged.
**Architecture:** Builds on Phase 2's event-sourced architecture and 3-entity scene support. New event kinds (`event_planned`, `event_started`, `event_completed`, `event_cancelled`, `event_expired`, `time_skip_elision`, `time_skip_jump`, `thread_opened`, `thread_updated`, `thread_closed`, `meanwhile_scene_started`, `meanwhile_scene_closed`, `synthesized_memories`) carry the new state changes. Two new tables (`events`, `threads`) hold lifecycle state. Existing handlers (`memory_written`, `edge_update`) gain new payload sources without changes — promotion logic lives in services, not in projector handlers.
**Tech Stack:** Same as Phase 2 (Python 3.11+, FastAPI, HTMX, SQLite, Featherless). No new dependencies.
**Source-of-truth references:**
- Phase 3 scope: requirements doc §13 "Phase 3 — events, skips, threads"
- Behavioral details: §4 (per-chat clocks), §6.3 (prompt assembly), §6.4 (drawer), §8.1 (retrieved-memory inputs), §9 ("Time, Skips, Events — Phase 3 surface"), §11 (significance & compression)
- Conventions: [../../CLAUDE.md](../../CLAUDE.md) §"Behavioral defaults" + §"Phase 2 status"
- Phase 2 plan (style, TDD pattern, parallel-dispatch mechanics): [2026-04-26-v2-phase2-implementation.md](2026-04-26-v2-phase2-implementation.md)
When a task says "see §X", that's the requirements doc unless stated otherwise.
---
## Pre-flight
**Branch:** create `phase-3` from the latest `main` after Phase 2 has merged. If Phase 2 is still in PR review, branch off `phase-2` directly:
```bash
# Option A: after main has phase-2 merged
git checkout main && git pull && git checkout -b phase-3
# Option B: continue from phase-2 directly
git checkout phase-2 && git pull && git checkout -b phase-3
```
**Schema baseline:** Phase 2 leaves the DB at version 8. Phase 3 adds two migrations: `0009_events.sql` and `0010_threads.sql`. No other migrations expected.
**Phase 2.5 backlog:** the items in CLAUDE.md §"Phase 2.5 / 3 backlog" are NOT scoped here — they should be cleaned up in a separate branch off `main` (suggested name `phase-2.5`) before or in parallel with Phase 3. None of them blocks Phase 3.
**Pinned non-negotiables (carried forward):**
- State changes go through the event log. Use `append_and_apply(conn, kind, payload)` for the live path; `apply_event` only after a fresh `append_event` returning the new id.
- Witness filter every memory read at SQL level (hard `WHERE` constraint; never a soft signal).
- Edges are directed; `botA → botB` and `botB → botA` are independent records.
- Per-POV scene summaries — never write omniscient narration. (Meanwhile scenes write per-POV summaries for both present bots; you receive a digest later, not during the scene.)
- TDD: every task starts with a failing test.
- One commit per task minimum, more if it splits naturally.
**Verification before claiming done:** Use `superpowers-extended-cc:verification-before-completion` — run the test command, paste actual output. Don't assume green.
---
## Parallel-Execution Strategy
Same pattern as Phase 2. Eight waves: parallel within each wave (file-disjoint), serial across waves. The controller (you, the controlling Claude session) merges each subagent's commits and verifies the suite stays green before dispatching the next wave.
### How to dispatch a wave in parallel
Use the **Agent tool with `isolation: "worktree"`** so each subagent gets its own git worktree. The runtime cleans up the worktree automatically if no changes are made; otherwise it returns the path + branch for the controller to merge. (If the controlling session's working directory is **not** the chat repo, create worktrees manually with `git worktree add .worktrees/<wave>-<task> -b <wave>/<task> phase-3` from inside the chat repo and pass the worktree path explicitly into each subagent prompt — that is the pattern Phase 2 used.)
In a single message, dispatch all tasks in the wave:
```
Agent({
description: "Wave 1 — T49 events table + handlers",
subagent_type: "general-purpose",
isolation: "worktree",
prompt: "<full task text from below>",
})
Agent({
description: "Wave 1 — T50 time_skip handlers",
subagent_type: "general-purpose",
isolation: "worktree",
prompt: "<full task text from below>",
})
Agent({
description: "Wave 1 — T51 threads table + handlers",
subagent_type: "general-purpose",
isolation: "worktree",
prompt: "<full task text from below>",
})
```
All subagents start simultaneously, each working on a private worktree branched off `phase-3`. They cannot see each other's changes (no shared filesystem state) — that's the safety guarantee.
### After a wave completes
1. Each subagent returns its worktree path and commit SHA.
2. **Run a spec + code-quality reviewer subagent on each completed task** (combined review is acceptable for purely mechanical schema/handler tasks; large or integration tasks like T62, T63 deserve separate spec + quality reviewers).
3. **Merge the wave into `phase-3`** in any order (file-disjointness guarantees no conflict). Use `--no-ff` so each task's history stays grouped:
```bash
git checkout phase-3
for branch in <wave-branches>; do
git merge --no-ff "$branch" -m "merge: <task description>"
done
```
4. **Run the full test suite** on the merged `phase-3`. If it's red, the wave's mutual-independence assumption was violated — bisect to find the offending pair, fix in a follow-up commit, re-merge.
5. **Push `phase-3`** to gitea so the work is durable before the next wave starts.
6. Optionally clean up worktrees: `git worktree remove .worktrees/<branch>` and `git branch -D <branch>`.
### Conflict prevention checklist (apply before dispatch)
For each parallel wave, verify the **Files** sections of all tasks have **no overlapping paths**. The waves below are designed to satisfy this; if you decide to add or merge tasks, re-check.
If a hot file (`chat/web/turns.py`, `chat/services/prompt.py`, `chat/web/drawer.py`, `chat/templates/_drawer.html`, `chat/services/regenerate.py`) needs changes from multiple tasks, do **not** parallelize them — serialize within the wave or split into separate waves.
### Failure recovery
If one subagent fails (test failures, blocked, infinite loop):
- **Do not block the wave on a failure.** Cancel the failed subagent, merge the others' successful work, and re-dispatch the failed task as a single follow-up.
- If a failure exposes a bad assumption shared by multiple tasks (e.g. an event-payload schema mismatch), pause the wave and revisit the plan.
### Why each wave is parallel-safe
| Wave | Tasks | Hot files touched | Disjoint? |
|------|-------|-------------------|-----------|
| 1 | T49, T50, T51 | new SQL migrations + new state modules; T50 also extends `chat/state/world.py` (additive) | ✅ |
| 2 | T52, T53, T54, T55 | new service modules only | ✅ |
| 3 | T56, T57, T58 | new service module (T56) + `chat/state/memory.py` retrieval extension (T57) + `chat/services/scene_summarize.py` (T58) | ✅ |
| 4 | T59 | `chat/web/drawer.py`, `chat/templates/_drawer.html` | (single task) |
| 5a | T60, T61 | `chat/services/prompt.py` (T60), `chat/web/turns.py` (T61) | ✅ |
| 5b | T62 | `chat/web/turns.py`, plus a new skip route module | (single task; depends on 5a) |
| 6 | T63, T64, T65 | meanwhile is tightly coupled — see Wave 6 sub-structure below | ⚠️ partial |
| 7 | T66, T67 | new test file + docs only | ✅ |
**Wave 6 sub-structure:** T63 is schema/state (new files); T64 is service + extends `chat/web/turns.py`; T65 is service + extends `chat/services/prompt.py`. T64 and T65 are file-disjoint relative to each other but both depend on T63's schema landing first. Dispatch as: T63 alone → merge → T64+T65 in parallel → merge.
---
## Task overview
```
Wave 1 ─┬─ T49: events table + lifecycle handlers
├─ T50: time_skip event kinds + handlers (advance chat clock)
└─ T51: threads table + open/update/close handlers
Wave 2 ─┬─ T52: event-lifecycle detection service (narrative → state changes)
├─ T53: skip narration service (elision + jump prose)
├─ T54: synthesized-memories service (jump skip "anything notable?")
└─ T55: thread-detection service (on scene close, identify open threads)
Wave 3 ─┬─ T56: event-completion promotion (inventory / edges / memories)
├─ T57: significance retrieval ranking refinements
└─ T58: scene compression keeps key quotes when significance ≥ 2
Wave 4 ─── T59: drawer additions — events panel, threads panel, skip controls
Wave 5a ─┬─ T60: prompt assembly includes active events + active threads
└─ T61: turn flow invokes event-detection + thread-update per turn
Wave 5b ─── T62: skip command surface (parse + route + jump UI prompt)
Wave 6 ─┬─ T63: meanwhile scene config — schema + state + scene-config-4 marker
└─ (after T63 merges)
├─ T64: meanwhile turn flow (host+guest, no "you")
└─ T65: meanwhile summary digest (briefs you on next active scene)
Wave 7 ─┬─ T66: cross-feature integration tests (events × skips × threads × meanwhile)
└─ T67: Phase 3 documentation update
```
Critical path: 8 sequential merge points (Waves 1, 2, 3, 4, 5a, 5b, 6a, 6b, 7). Total tasks: 19. Wall-clock parallelism advantage depends on subagent dispatch overhead, but in principle each wave's tasks can run concurrently in ~the time of one task.
---
## Wave 1 — Schema & state foundation
These three tasks are **fully independent**: each adds a new SQL migration + new state module. T50 also adds two handlers to `chat/state/world.py` (additive, alongside Phase 2's `_apply_guest_added`).
### Task 49: Events table + lifecycle handlers
**Files:**
- Create: `chat/db/migrations/0009_events.sql`
- Create: `chat/state/events.py`
- Create: `tests/test_events_state.py`
**Spec:** Adds the `events` table and projector handlers for the lifecycle: `event_planned`, `event_started`, `event_completed`, `event_cancelled`, `event_expired`. Each event row carries `chat_id`, `kind` (free-form domain-event tag like `"date_at_park"`), `status` (`planned|active|completed|cancelled|expired`), `props_json` (arbitrary blob), `planned_for` (ISO-8601 chat-clock string, optional), `started_at` / `completed_at` (chat-clock strings).
**Step 1: failing test** — see pattern in `tests/test_group_node.py` (Phase 2 T36). Three tests minimum:
1. `test_event_planned_creates_row`: append `event_planned` with `kind`, `props_json`, `planned_for`; project; assert `get_event(conn, event_id)` returns the row with `status="planned"`.
2. `test_event_started_then_completed_updates_status`: append `event_planned``event_started``event_completed`; assert `status` transitions and `completed_at` populated.
3. `test_event_cancelled_terminal`: append `event_planned``event_cancelled`; assert `status="cancelled"`. A subsequent `event_started` is ignored (handler no-op when status is terminal).
**Step 3: implementation** — `0009_events.sql`:
```sql
CREATE TABLE events (
id INTEGER PRIMARY KEY,
chat_id TEXT NOT NULL,
kind TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'planned',
props_json TEXT NOT NULL DEFAULT '{}',
planned_for TEXT,
started_at TEXT,
completed_at TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX events_chat_idx ON events(chat_id, status);
```
`chat/state/events.py`:
- `@on("event_planned")` inserts a new row with status `planned`. Payload provides a stable `event_id` (caller-allocated UUID) so the projector is idempotent.
- `@on("event_started")` updates status to `active` and sets `started_at` from payload (or current chat clock).
- `@on("event_completed")`, `@on("event_cancelled")`, `@on("event_expired")` each move to the named terminal state and stamp `completed_at` (the column doubles as "ended at").
- `get_event(conn, event_id)`, `list_active_events(conn, chat_id)`, `list_events_in_status(conn, chat_id, status)` readers.
- All handlers no-op when the row is already in a terminal state (idempotent re-projection safety).
**Step 5: commit** — `feat: events table + lifecycle handlers (T49)`.
**Notes for the implementer:**
- Use UUID-style ids (e.g., `f"evt_{uuid.uuid4().hex[:12]}"`) created by the caller; pass as `event_id` in payload. Don't auto-generate inside the projector.
- Schema version after this migration alone: 9. The full Phase 3 baseline is 10 (T51 adds 0010_threads.sql).
- `tests/test_world.py::test_schema_version_after_migration_is_8` will need to bump after Wave 1 merges — handle in the wave-merge step (mirrors Phase 2 T36's pattern).
---
### Task 50: Time-skip event kinds + chat-clock handlers
**Files:**
- Modify: `chat/state/world.py` (add `_apply_time_skip_elision`, `_apply_time_skip_jump`; both update `chats.time` and may reset `activity` rows)
- Create: `tests/test_time_skip_handlers.py`
**Spec:** Two new event kinds.
- `time_skip_elision` payload: `{chat_id, new_time}`. Handler updates `chats.time = ?`. Activity rows are NOT reset (the activity that was elided to its end-state is the resolution itself; the caller passes a follow-up `activity_changed` event when needed).
- `time_skip_jump` payload: `{chat_id, new_time, reset_activity: bool}`. Handler updates `chats.time = ?`; if `reset_activity` is true, deletes per-chat `activity` rows for the participants in that chat (a fresh landing state will be set by a follow-up `activity_changed` event from the skip service).
These are pure state mutations. T54 and T62 fire them via `append_and_apply`.
**Tests:** 3 minimum.
1. `test_elision_advances_chat_clock_only`: seed chat at time T0; append `time_skip_elision` with `new_time=T1`; project; assert `get_chat(...)["time"] == T1` and activity unchanged.
2. `test_jump_with_reset_clears_activity`: seed chat with one activity row; append `time_skip_jump` with `reset_activity=True`; assert chat clock advanced AND activity table empty for that chat.
3. `test_jump_without_reset_preserves_activity`: same seed; `reset_activity=False`; assert activity row still present and clock advanced.
**Implementation:** new handlers next to `_apply_chat_created` in `chat/state/world.py`. Use the same parameterized SQL patterns. Do NOT add UI here — T62 wires the skip command flow.
**Commit:** `feat: time_skip event handlers (T50)`.
---
### Task 51: Threads table + open/update/close handlers
**Files:**
- Create: `chat/db/migrations/0010_threads.sql`
- Create: `chat/state/threads.py`
- Create: `tests/test_threads_state.py`
**Spec:** Adds the `threads` table and projector handlers for `thread_opened`, `thread_updated`, `thread_closed`. A thread is a per-chat narrative continuity tag — open during scenes, surfaced to prompt assembly so successor scenes can reference unresolved arcs.
`0010_threads.sql`:
```sql
CREATE TABLE threads (
id INTEGER PRIMARY KEY,
chat_id TEXT NOT NULL,
title TEXT NOT NULL,
summary TEXT NOT NULL DEFAULT '',
status TEXT NOT NULL DEFAULT 'open', -- open | closed
opened_at TEXT NOT NULL DEFAULT (datetime('now')),
closed_at TEXT,
last_referenced_scene_id INTEGER,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX threads_chat_status_idx ON threads(chat_id, status);
```
`chat/state/threads.py`:
- `@on("thread_opened")` payload: `{thread_id, chat_id, title, summary?}`. Inserts a new row with `status='open'`.
- `@on("thread_updated")` payload: `{thread_id, summary, last_referenced_scene_id?}`. Updates summary + optional last-referenced-scene pointer.
- `@on("thread_closed")` payload: `{thread_id, closed_at?}`. Sets status='closed', stamps `closed_at`.
- Readers: `get_thread(conn, thread_id)`, `list_open_threads(conn, chat_id)`, `list_threads(conn, chat_id, status=None)`.
**Tests:** 3 minimum.
1. `test_thread_opened_creates_row`.
2. `test_thread_updated_changes_summary_and_last_referenced`.
3. `test_thread_closed_terminal`: subsequent `thread_updated` is ignored (matches the design's "closed threads are kept for replay but don't surface in prompt").
**Note:** the Phase 2 `group_node.threads_json` column was a Phase-3 placeholder and is NOT used as authoritative storage now — `threads` table is the source of truth. The drawer can choose to render either, but Phase 3 onward should treat the table as canonical and treat `group_node.threads_json` as a deprecated cache that we leave alone (or clear in the next migration).
**Commit:** `feat: threads table + projector handlers (T51)`.
---
## Wave 2 — Classifier services (parallel)
Four tasks, all new service modules — fully file-disjoint.
### Task 52: Event-lifecycle detection service
**Files:**
- Create: `chat/services/event_lifecycle.py`
- Create: `tests/test_event_lifecycle.py`
**Spec:** A classifier-wrapped service that inspects a freshly-narrated turn and decides whether any active events transitioned this turn (started, completed, cancelled). Returns a structured `EventLifecycleDecision` with one or more `EventTransition(event_id, new_status, reason)` items, or empty when nothing changed.
Schema:
```python
class EventTransition(BaseModel):
event_id: str
new_status: str # "active" | "completed" | "cancelled"
reason: str = ""
class EventLifecycleDecision(BaseModel):
transitions: list[EventTransition] = Field(default_factory=list)
```
Public API:
```python
async def detect_event_transitions(
client: LLMClient,
*,
classifier_model: str,
narrative_text: str,
active_events: list[dict], # [{id, kind, status, props}, ...] from list_active_events
timeout_s: float = 30.0,
) -> EventLifecycleDecision:
"""Decide whether any active events transitioned this turn. Conservative
bias — most turns return empty transitions. Trigger only when the
narrative text clearly resolves or starts a known active event.
"""
```
Caller (T61 turn flow) appends one `event_started` / `event_completed` / `event_cancelled` event per transition via `append_and_apply`.
**Tests:** 3 minimum — happy path with one transition, empty active_events short-circuits without classifier call, classifier failure returns empty default.
**Commit:** `feat: event-lifecycle detection service (T52)`.
---
### Task 53: Skip narration service
**Files:**
- Create: `chat/services/skip_narration.py`
- Create: `tests/test_skip_narration.py`
**Spec:** Generates the brief transition narration that bridges a time skip. Two flavors mirroring §9:
- **Elision:** "skip to when we arrive". Input: current activity ("walking to park"), expected end-state ("at the park, sitting on a bench"). Output: 1-2 sentence transition prose narrated from the host bot's POV. New chat-clock value is provided by the caller.
- **Jump:** "next morning". Input: time delta + landing-state hint (optional). Output: 2-3 sentences setting the scene at the new time.
Public API:
```python
async def narrate_skip(
client: LLMClient,
*,
narrative_model: str,
skip_kind: str, # "elision" | "jump"
speaker_bot: dict, # {id, name, persona}
you_name: str,
current_time: str,
new_time: str,
current_activity: str,
landing_state_hint: str = "",
timeout_s: float = 60.0,
) -> str:
"""Generate brief transition prose. Returns plain text, not JSON."""
```
Uses `client.generate(...)` (not `classify`) since output is free-form prose. Falls back to a deterministic template string on failure (e.g., `f"({new_time}: {landing_state_hint or current_activity}.)"`). The fallback ensures the skip flow never blocks even when the LLM is down.
**Tests:** 3 minimum — happy elision, happy jump, generation failure returns fallback string with the new time visible.
**Commit:** `feat: skip narration service (T53)`.
---
### Task 54: Synthesized-memories service
**Files:**
- Create: `chat/services/synthesized_memories.py`
- Create: `tests/test_synthesized_memories.py`
**Spec:** When the user does a jump skip ("a week later") they're prompted "anything notable happen?" If they answer with prose, this service parses that prose into 1-N synthesized memories per present bot. Each memory carries `source="synthesized"`, `reliability=0.7`, witness mask `[1, 1, 0]` or `[1, 1, 1]` per present set, and a one-sentence text body.
Schema:
```python
class SynthesizedMemory(BaseModel):
text: str
significance: int = 1 # 0..3, default 1
affinity_delta: int = 0
trust_delta: int = 0
class SynthesizedDigest(BaseModel):
memories: list[SynthesizedMemory] = Field(default_factory=list)
```
Public API:
```python
async def synthesize_memories(
client: LLMClient,
*,
classifier_model: str,
prose: str,
bot_name: str, # which witness's POV
bot_persona: str,
you_name: str,
timeout_s: float = 30.0,
) -> SynthesizedDigest:
"""Parse 'anything notable happen?' prose into structured memories
from a single bot's POV. Empty/whitespace prose short-circuits."""
```
Caller (T62 skip flow) calls this once per present bot (host always; guest if present), then writes via `record_turn_memory_for_present` with `source="synthesized"` and the synthesized text in place of narrative_text.
**Tests:** 3 minimum — happy path returns parseable memories, empty prose short-circuits, classifier failure returns empty digest.
**Commit:** `feat: synthesized-memories service for jump skips (T54)`.
---
### Task 55: Thread-detection service
**Files:**
- Create: `chat/services/thread_detection.py`
- Create: `tests/test_thread_detection.py`
**Spec:** On scene close, classify the scene transcript to detect open threads (unresolved arcs, dangling questions, promises made). Returns a list of `ThreadCandidate(title, summary, action: "open"|"update"|"close", existing_thread_id?)`.
The service receives the current set of open threads so it can decide to **update** an existing thread rather than open a duplicate. It can also signal **close** when the transcript clearly resolves an open thread.
Schema:
```python
class ThreadCandidate(BaseModel):
action: str # "open" | "update" | "close"
title: str = "" # required for "open"; ignored otherwise
summary: str = ""
existing_thread_id: str | None = None # required for "update"/"close"
class ThreadDetectionResult(BaseModel):
candidates: list[ThreadCandidate] = Field(default_factory=list)
```
Public API:
```python
async def detect_threads(
client: LLMClient,
*,
classifier_model: str,
scene_transcript: list[dict], # [{speaker, text}, ...]
open_threads: list[dict], # [{id, title, summary}, ...]
timeout_s: float = 30.0,
) -> ThreadDetectionResult:
"""Classify scene close into thread open/update/close candidates."""
```
Caller (T58 scene compression — added in Wave 3) loops over candidates and emits one `thread_opened`, `thread_updated`, or `thread_closed` event per candidate.
**Tests:** 3 minimum — opens a new thread, updates an existing thread (test asserts `existing_thread_id` is honored), classifier failure returns empty.
**Commit:** `feat: thread-detection service (T55)`.
---
## Wave 3 — Promotion & retrieval refinements
Three tasks. T56 is a new service module (event-completion promotion). T57 modifies `chat/state/memory.py` to add a significance-aware retrieval rank. T58 modifies `chat/services/scene_summarize.py` to integrate compression hints + the thread-detection service from T55. File-disjoint.
### Task 56: Event-completion promotion
**Files:**
- Create: `chat/services/event_promotion.py`
- Create: `tests/test_event_promotion.py`
**Spec:** When an event reaches `completed` (the only terminal state that promotes; cancelled/expired do NOT promote per §9 last paragraph), the orchestrator promotes any structured artifacts the event carried into the appropriate target store:
- `event.props.acquired_objects: list[str]` → append `inventory_added` events (Phase 4 schema; Phase 3 stub: just append a `manual_edit` with `target_kind="memory_pov_summary"` describing the acquisition into the host's memory).
- `event.props.knowledge_facts: list[{owner_id, target_id, fact}]` → append `edge_update` events with the facts on the named directed edge.
- `event.props.relationship_change: {summary, source_id, target_id}` → append `manual_edit` with `target_kind="edge_summary"` for that pair.
- Everything else stays in the closed event record (the projector kept the row; no further promotion).
Public API:
```python
def promote_completed_event(
conn,
*,
event_id: str,
chat_id: str,
chat_clock_at: str | None,
) -> dict:
"""Read the completed event's props_json and emit promotion events.
Returns a summary dict {inventory: int, knowledge: int, relationship: int}
of how many promotion events fired. No classifier calls — purely
structural. Skips if event status isn't 'completed'."""
```
This is **synchronous** (no async, no LLM). It reads a row, parses JSON, emits events via `append_and_apply`.
**Tests:** 4 minimum — empty props no-op, knowledge_facts produces edge_update events, relationship_change produces manual_edit, cancelled-event-doesn't-promote.
**Commit:** `feat: event-completion promotion service (T56)`.
---
### Task 57: Significance-aware retrieval ranking
**Files:**
- Modify: `chat/state/memory.py` (extend `search_memories(conn, owner_id, witness_role, query, k)` to add a significance bias to the rank ordering)
- Modify: `tests/test_memory_search.py` (or wherever the existing search tests live; add 2 tests)
**Spec:** Currently `search_memories` orders by FTS rank only. §11.1 says "Retrieval ranking: significance multiplier applied as `score × constant` to FTS / vector rank." Phase 3 implements this for FTS only (vector retrieval is Phase 4).
Change the SQL `ORDER BY` from `ORDER BY rank` to `ORDER BY (rank + significance * 0.5) DESC` (or whatever scaling produces sane results — this is a tuning knob, document the choice in a comment). The constant may need adjustment after manual play; surface it as a module-level constant `SIGNIFICANCE_RANK_BIAS`.
**Tests:** 2 added.
1. `test_higher_significance_outranks_equal_rank`: seed two memories with identical FTS-matching text but different significance scores; assert the higher-significance row appears first in results.
2. `test_significance_bias_is_constant_module_level`: verify the constant is accessible as `chat.state.memory.SIGNIFICANCE_RANK_BIAS` (so it's tunable without a code change in calling sites).
**Commit:** `feat: significance-aware retrieval ranking (T57)`.
---
### Task 58: Scene compression keeps key quotes when significance ≥ 2
**Files:**
- Modify: `chat/services/scene_summarize.py` (extend `apply_scene_close_summary` to also call `detect_threads` from T55 and emit thread events; extend the per-POV summary to include up to 3 verbatim "key quotes" from the closing scene when scene-max-significance ≥ 2)
- Modify: `tests/test_per_pov_summary.py` (add 3 tests for the new behavior)
**Spec:** §11.1 specifies "Compression: scenes with max-turn-significance ≥ 2 retain key quotes; ≤ 1 collapse fully into the per-POV summary." Implement this:
- Compute scene max significance from `memories.significance` rows in this scene.
- When max < 2: existing behavior unchanged (per-POV summary, no extra quotes).
- When max ≥ 2: include up to 3 verbatim quote spans (each ≤ 200 chars) in the per-POV summary text. Format: append `\n\nKey quotes:\n- "..."\n- "..."` to the summary. The `summarize_scene` classifier already produces the prose; the quote-selection step is a deterministic post-process that picks the top-3 highest-significance turn texts from the scene transcript (truncated).
Additionally, after writing per-POV summaries (existing behavior), call `detect_threads` (from T55) once per close. For each candidate emit the matching `thread_opened` / `thread_updated` / `thread_closed` event via `append_and_apply`. Failures fall back to no thread changes (existing memory + edge updates still land).
**Tests:** 3 added.
1. `test_low_significance_scene_omits_quotes`: max significance = 1; assert summary text contains no "Key quotes:" header.
2. `test_high_significance_scene_includes_top_3_quotes`: seed 4 memories with significance 3, 2, 1, 2; assert summary contains the top-3 (by significance) verbatim turn texts.
3. `test_thread_detection_emits_events`: stub `detect_threads` to return one `ThreadCandidate(action="open", ...)`; assert a `thread_opened` event landed.
**Commit:** `feat: significance-driven quote retention + thread emission on close (T58)`.
---
## Wave 4 — Drawer additions (single task)
This wave is one task because all Phase 3 drawer additions touch `chat/web/drawer.py` and `chat/templates/_drawer.html` together — splitting would force serial execution with conflicts.
### Task 59: Drawer events / threads / skip controls
**Files:**
- Modify: `chat/web/drawer.py` (extend `GET /chats/{chat_id}/drawer`; add `POST /chats/{chat_id}/drawer/event/plan`, `/drawer/event/cancel/{event_id}`, `/drawer/skip/elision`, `/drawer/skip/jump`, `/drawer/thread/close/{thread_id}`)
- Modify: `chat/templates/_drawer.html` (3 new sections: Events, Threads, Skip controls)
- Create: `tests/test_drawer_events_threads_skip.py`
**Spec:**
**GET extension:**
- `list_active_events(conn, chat_id)` → render in a new "Events" section.
- `list_open_threads(conn, chat_id)` → render in a new "Threads" section.
- A "Skip" subsection with two buttons: "Elision skip" (opens an inline form taking a `landing_state_hint`) and "Jump skip" (opens an inline form taking `target_time` ISO + optional `notable_prose` for the synthesized-memories prompt).
**POST routes:**
1. `POST /drawer/event/plan` — form `{kind, planned_for, props_json}` → 400-validates JSON, appends `event_planned`, returns refreshed drawer.
2. `POST /drawer/event/cancel/{event_id}` — appends `event_cancelled`, returns refreshed drawer.
3. `POST /drawer/skip/elision` — form `{landing_state_hint, new_time}` → calls `narrate_skip` (T53), appends `time_skip_elision` + an `assistant_turn` carrying the narration, returns refreshed drawer + chat partial.
4. `POST /drawer/skip/jump` — form `{new_time, notable_prose, reset_activity}` → calls `narrate_skip` for transition prose, calls `synthesize_memories` (T54) for each present bot, appends `time_skip_jump` + memories + transition turn, returns refreshed drawer + chat partial.
5. `POST /drawer/thread/close/{thread_id}` — appends `thread_closed`, returns refreshed drawer.
**Template additions:**
- "Events" section listing each active event by kind + planned_for + props.
- "Threads" section listing each open thread title + summary + a Close button.
- "Skip" controls under existing Activity section.
- Forms use HTMX (`hx-post`, `hx-target="#drawer"`, `hx-swap="innerHTML"`) consistent with Phase 2 drawer patterns.
**Tests (`tests/test_drawer_events_threads_skip.py`):** 6 minimum.
1. GET drawer with no events/threads → no Events/Threads sections rendered.
2. POST event/plan with valid form → event_planned event appended; drawer body now contains the event title.
3. POST event/cancel → event_cancelled appended; drawer no longer lists the event under "Active".
4. POST skip/elision → time_skip_elision appended, chat clock advanced, narration assistant_turn present in chat history.
5. POST skip/jump with notable_prose → time_skip_jump + N synthesized memory_written events; assert reliability=0.7 on those rows.
6. POST thread/close → thread_closed appended; thread no longer in open list.
**Commit:** `feat: drawer events / threads / skip controls (T59)`.
**Notes for implementer:**
- The existing `available_guests` dropdown helper from T42 is the reference for form-population patterns.
- For the Jump skip's `notable_prose` field, treat empty as "no synthesized memories" (just advance the clock) — the spec allows this.
- Validate `target_time` ISO format; 400 on parse failure. Do not allow target_time earlier than current chat clock.
---
## Wave 5a — Prompt + turn-flow integration (parallel)
T60 modifies `chat/services/prompt.py`. T61 modifies `chat/web/turns.py`. File-disjoint.
### Task 60: Prompt assembly includes active events + active threads
**Files:**
- Modify: `chat/services/prompt.py` (extend `assemble_narrative_prompt`)
- Modify: `tests/test_prompt.py` (add 3 tests)
**Spec:** Two new SHOULD-tier blocks added between the existing scene-context block and retrieved-memories block:
1. **Active events** — title `Active events:`. Lists each active event in this chat: `- {kind} (planned for {planned_for})` plus a one-line props excerpt (truncate to ~80 chars). Trim-tier SHOULD; drops before retrieved memories under tight budget.
2. **Active threads** — title `Open threads:`. Lists each open thread: `- {title}: {summary}` (summary truncated to ~120 chars). SHOULD-tier.
Both blocks are omitted entirely when their lists are empty (no header rendered).
Per Phase 2 T43's auto-detection precedent, the function reads `list_active_events(conn, chat_id)` and `list_open_threads(conn, chat_id)` itself; no new parameters.
**Tests:** 3 added.
1. `test_assemble_with_no_events_or_threads_omits_blocks` — regression; no events/threads → assembled prompt has neither block.
2. `test_assemble_with_active_events_renders_block` — seed one event_planned + event_started; assert "Active events:" header and event kind appear in prompt.
3. `test_assemble_with_open_thread_renders_block` — seed one thread_opened; assert "Open threads:" header and thread title appear.
**Commit:** `feat: prompt assembly renders active events + open threads (T60)`.
---
### Task 61: Turn flow invokes event-detection + thread-update per turn
**Files:**
- Modify: `chat/web/turns.py` (after the primary narrative + memory + state-update block, call `detect_event_transitions` from T52; emit `event_started`/`event_completed`/`event_cancelled` events accordingly)
- Modify: `chat/services/regenerate.py` (mirror — regenerate also re-detects event transitions for the regenerated turn)
- Modify: `tests/test_turn_flow.py` (add 3 tests)
**Spec:** After the existing post-turn classifier passes (memory write, state update, interjection check) and BEFORE scene-close detection, call `detect_event_transitions` with `narrative_text=primary_text` and `active_events=list_active_events(conn, chat_id)`.
For each `EventTransition` returned:
- `new_status="active"` → append `event_started` payload `{event_id, started_at: chat.time}`.
- `new_status="completed"` → append `event_completed` payload `{event_id, completed_at: chat.time}` AND THEN call `promote_completed_event` (T56) inline so promotion events emit synchronously after completion.
- `new_status="cancelled"` → append `event_cancelled`. Promotion is skipped.
Empty transitions list = no-op (most turns; no extra events written).
`regenerate.py` mirrors the same logic for the regenerated turn (existing event transitions from the superseded turn are NOT undone — that's a Phase 3.5 follow-up; document the limitation).
**Tests:** 3 added to `tests/test_turn_flow.py`.
1. `test_turn_with_event_transition_appends_started_event`: mock `detect_event_transitions` to return one transition; assert `event_started` lands in event log; canned-response queue matches.
2. `test_turn_with_event_completion_runs_promotion`: same mock returning `new_status="completed"`; seed a planned event with knowledge_facts in props; assert `event_completed` + `edge_update` (from promotion) both land.
3. `test_turn_with_no_active_events_skips_classifier`: no active events; assert `detect_event_transitions` is never called (its canned response slot would still be in the queue at end of test).
**Commit:** `feat: per-turn event-lifecycle detection + completion promotion (T61)`.
---
## Wave 5b — Skip command flow (single task)
Single task because it modifies `chat/web/turns.py` (which Wave 5a also touched). Run after Wave 5a is merged so the file's recent additions are stable.
### Task 62: Skip command surface
**Files:**
- Modify: `chat/web/turns.py` (extend `parse_turn` to detect natural-language skip commands like "skip to the park", "next morning", "a week later" and route to a skip-handling branch BEFORE the normal narrative flow)
- Create: `chat/web/skip.py` (new module hosting `process_elision_skip(...)` and `process_jump_skip(...)` controllers; called by both turns.py and the drawer skip routes from T59)
- Modify: `tests/test_turn_flow.py` (add 3 tests)
**Spec:** Currently `parse_turn` extracts the user's prose into structured fields (addressee inferred, etc.). Phase 3 adds detection of skip commands as a separate intent.
The classifier-based parse already produces an `intent` field (or similar — verify in code). Extend the schema with `intent="skip_elision"` and `intent="skip_jump"`. When intent is one of these, the turn flow short-circuits the normal narrative path and routes to:
- `process_elision_skip(conn, client, settings, *, chat_id, landing_state_hint=parsed.landing_state)` — calls `narrate_skip(skip_kind="elision")`, appends `time_skip_elision`, `assistant_turn` carrying narration, returns 204.
- `process_jump_skip(conn, client, settings, *, chat_id, target_time=parsed.target_time, notable_prose=parsed.notable_prose)` — appends `time_skip_jump`, calls `synthesize_memories` per present bot, appends synthesized `memory_written` events, calls `narrate_skip(skip_kind="jump")`, appends `assistant_turn` carrying transition prose, returns 204.
The drawer routes from T59 share these functions (don't duplicate the logic across drawer.py and turns.py).
For Phase 3's first cut, JUMP skip's `notable_prose` is NOT collected from natural-language ("a week later, anything notable?" requires a UI prompt). Two options:
- **(simpler)** Drawer-only entry for jump skip; natural-language jump short-circuits to drawer prompt.
- **(better UX)** Natural-language jump returns a 422 with an HTMX-swap that injects the "anything notable?" textarea into the chat surface; user submits prose to a follow-up `/chats/{chat_id}/skip/jump/confirm` endpoint.
Pick the simpler path for Phase 3 (drawer-only jump). Document the second option as a Phase 3.5 polish.
**Tests:** 3 added.
1. `test_elision_skip_via_natural_language` — user prose "skip to when we arrive at the park"; assert `time_skip_elision` event landed and chat clock advanced; an `assistant_turn` carrying transition prose was appended.
2. `test_jump_skip_via_natural_language_redirects_to_drawer` — user prose "next morning"; assert response is 422 with an HTMX swap pointing at the drawer's jump form (or whatever the chosen Phase 3 fallback is).
3. `test_skip_command_does_not_run_narrative_classifier` — same user prose as test 1; assert `assemble_narrative_prompt` was NOT called for a regular bot turn (the skip path bypasses it).
**Commit:** `feat: natural-language skip detection + skip command flow (T62)`.
---
## Wave 6 — Meanwhile scenes
Phase 3's capstone feature. Most ambitious: scene config 4 (host + guest, no "you"). Per §13 the cap stays at 2 bots in any scene; meanwhile is two-bot bot↔bot. "You" receives a digest later, not during.
Decomposed into 3 tasks. T63 lands first (schema + state); then T64 + T65 in parallel.
### Task 63: Meanwhile scene config — schema + state
**Files:**
- Create: `chat/db/migrations/0011_meanwhile_scenes.sql`
- Create: `chat/state/meanwhile.py`
- Create: `tests/test_meanwhile_state.py`
**Spec:** A meanwhile scene is a special kind of scene where `present_set = {host_bot_id, guest_bot_id}` (no "you"). The existing `scenes` table can carry it via a new `present_set_kind` column distinguishing `you_host`, `you_host_guest`, `host_guest`. Alternatively, `meanwhile_scenes` is a sidecar table — pick the lower-disruption option.
**Recommended:** add a `present_set_kind` column to `scenes` (default `'you_host'` for back-compat) via migration `0011_meanwhile_scenes.sql`:
```sql
ALTER TABLE scenes ADD COLUMN present_set_kind TEXT NOT NULL DEFAULT 'you_host';
ALTER TABLE scenes ADD COLUMN parent_scene_id INTEGER; -- the active you-scene this meanwhile branched off from
CREATE INDEX scenes_present_set_idx ON scenes(chat_id, present_set_kind, status);
```
New event kinds with `chat/state/meanwhile.py` handlers:
- `@on("meanwhile_scene_started")` payload: `{chat_id, scene_id, host_bot_id, guest_bot_id, parent_scene_id, started_at}`. Inserts a new scene row with `present_set_kind="host_guest"`, links to parent.
- `@on("meanwhile_scene_closed")` payload: `{scene_id, closed_at}`. Updates status to `closed`; subsequent per-POV summary writes for both bots happen via existing scene-close path (host + guest are the "present witnesses"; "you" is excluded).
Readers: `list_meanwhile_scenes(conn, chat_id, status='active')`, `get_parent_scene(conn, scene_id)`.
**Tests:** 3 minimum.
1. `test_meanwhile_started_creates_scene_with_correct_present_set_kind`.
2. `test_meanwhile_closed_marks_scene_closed`.
3. `test_active_you_scene_can_coexist_with_active_meanwhile_scene` (one chat, two active scenes — meanwhile + the main you-scene that spawned it).
**Commit:** `feat: meanwhile scene schema + state (T63)`.
---
### Task 64: Meanwhile turn flow
**Files:**
- Modify: `chat/web/turns.py` (add meanwhile-mode detection at the start of `post_turn`; if active meanwhile scene exists for this chat, route to `process_meanwhile_turn`)
- Create: `chat/web/meanwhile.py` (new module hosting `process_meanwhile_turn(...)` controller; mirrors post_turn but with no "you" in present_set)
- Modify: `chat/services/prompt.py` (small addition: when `present_set_kind="host_guest"`, exclude "you" from edges + activity blocks; addressee is always the other bot)
- Create: `tests/test_meanwhile_turn_flow.py`
**Spec:** A meanwhile scene runs entirely between two bots. The user can advance it manually via a meanwhile-mode chat surface (T65 wires the UI), but turn-flow logic is:
1. Read active meanwhile scene; identify `speaker_bot_id` (alternates each turn — start with host, then guest, etc.) and `addressee_bot_id` (the other one).
2. Assemble narrative prompt with `speaker_bot_id`, `addressee=addressee_bot.name`, `present_set_kind="host_guest"` (so "you" is omitted from edges/activities).
3. Stream narrative; commit `assistant_turn` event with `present_set_kind="host_guest"` and `meanwhile_scene_id` populated.
4. Memory writes: BOTH host and guest get a memory_written with witness `[0, 1, 1]` (you=0; you wasn't present). Use `record_turn_memory_for_present` adapted to the no-you case (or extend it with a `you_present: bool = True` parameter).
5. State updates: 2 directed pairs (host↔guest only). Skip you-related pairs.
6. Scene close detection: same path as regular scenes; on close, per-POV summaries fire for both bots; group_node updates if applicable.
Addressee-alternation: simple — each turn alternates speaker. (Phase 3.5 may add classifier-driven turn-taking with refusals.)
**Tests:** 4 minimum.
1. `test_meanwhile_turn_writes_memories_with_witness_0_1_1`.
2. `test_meanwhile_turn_emits_2_edge_updates_only` (host→guest, guest→host).
3. `test_meanwhile_turn_alternates_speaker` (turn 1: host speaks; turn 2: guest speaks).
4. `test_meanwhile_scene_close_writes_per_pov_for_both_bots_only` (no "you" memory; existing T45 path is hit but with `you_present=False`).
**Commit:** `feat: meanwhile turn flow (host+guest, no you) (T64)`.
---
### Task 65: Meanwhile summary digest
**Files:**
- Modify: `chat/services/scene_summarize.py` (when a meanwhile scene closes, generate ALSO a "you-facing digest" — a brief narrated summary that will surface to "you" the next time the main you-scene resumes)
- Modify: `chat/services/prompt.py` (when assembling for a regular you-scene and any closed-but-not-yet-surfaced meanwhile digests exist, include them as a SHOULD-tier block titled "Meanwhile while you were away:")
- Create: `chat/state/meanwhile_digest.py` (a small state module: `meanwhile_digest_pending` table; handlers for `meanwhile_digest_created` / `meanwhile_digest_consumed`)
- Modify: `tests/test_per_pov_summary.py` and `tests/test_prompt.py` (add tests)
**Spec:** When a meanwhile scene closes (T64's path), also append `meanwhile_digest_created` with `{chat_id, scene_id, summary}`. The summary is generated via a fresh `summarize_scene` call with `bot_persona="omniscient narrator briefing the absent player"`; output is a 2-3 sentence neutral summary of what happened.
When the next you-scene starts (or the prompt is assembled for the next active you-scene's turn), `assemble_narrative_prompt` queries `list_pending_meanwhile_digests(conn, chat_id)` and:
- Includes them as a SHOULD-tier block: `"Meanwhile while you were away:\n- {summary}\n- {summary}"`.
- After they're surfaced once, the caller (T64 in the post-meanwhile turn or the first you-turn after meanwhile-close) appends `meanwhile_digest_consumed` per digest, marking them as surfaced.
Migration `0011_meanwhile_scenes.sql` (T63) can include the `meanwhile_digest_pending` table OR T65 adds a thin `0012_meanwhile_digest.sql`. Pick lower-disruption — likely add to T63's migration for simplicity. Document the choice.
(If you choose to add the table in T65 via a new migration, add `0012_meanwhile_digest.sql`. The schema-version assertion bump in `tests/test_world.py` happens once after Wave 6 merges.)
**Tests:** 3 added.
1. `test_meanwhile_close_creates_digest`: close a meanwhile scene; assert `meanwhile_digest_pending` row exists with non-empty summary.
2. `test_pending_digest_renders_in_you_scene_prompt`: seed a pending digest; assemble prompt for a you-host scene; assert the "Meanwhile while you were away:" header and summary appear.
3. `test_consumed_digest_does_not_render_again`: append `meanwhile_digest_consumed`; reassemble prompt; digest no longer appears.
**Commit:** `feat: meanwhile summary digest surfaces to next you-scene (T65)`.
---
## Wave 7 — Polish (parallel)
Two independent tasks. New test file (T66) + docs only (T67). Dispatch in parallel after Wave 6 merges.
### Task 66: Cross-feature integration tests
**Files:**
- Create: `tests/test_phase3_integration.py`
**Spec:** Phase 3 introduces a lot of cross-feature interaction surfaces. This task adds tests that exercise multi-feature flows end-to-end:
1. Plan an event → play turns → event_started detected → event_completed detected → promotion fires → memory + edge updates land.
2. Open a thread on close → next scene's prompt includes the open thread → close thread via drawer → next scene's prompt no longer includes it.
3. Jump skip → synthesized memories land per present bot → next turn's prompt retrieves them via search.
4. Meanwhile scene → close → digest pending → first you-turn prompt includes digest → after that turn, digest is consumed.
5. Meanwhile while a regular you-scene is active → both scenes have memories; querying memories for either bot at the post-meanwhile main scene correctly returns both sets witness-filtered.
5 tests minimum.
**Commit:** `test: phase 3 cross-feature integration coverage (T66)`.
---
### Task 67: Phase 3 documentation update
**Files:**
- Modify: `CLAUDE.md` (add "Phase 3 status" section; update "Behavioral defaults"; add "Phase 3.5 / 4 backlog" with carry-overs from review feedback during execution)
- Modify: `docs/plans/2026-04-26-v1-requirements-design.md` (annotate §13 "Phase 3 — events, skips, threads" as **Status: shipped <date>**)
**Spec:** Documentation-only. Run last so it captures any deviations and review-noted follow-ups discovered during execution. Reflect:
- Events with full lifecycle (planned → active → completed/cancelled/expired).
- Time skips: elision (immediate end-state) + jump (synthesized memories from "anything notable?").
- Threads opened/updated/closed; surfaced in prompt assembly + drawer.
- Significance retrieval bias + key-quote retention at significance ≥ 2.
- Meanwhile scenes: bot+bot without "you"; per-POV summaries for both bots; you-facing digest on next you-scene.
- Phase 3 known limitations / 3.5 backlog candidates:
- Natural-language jump skip falls back to drawer form (no inline "anything notable?" prompt).
- Regenerate doesn't undo prior event transitions from the superseded turn.
- Meanwhile turn-taking is alternation (no classifier-driven refusals or initiative).
- Vector retrieval is still Phase 4.
**Commit:** `docs: phase 3 status, behavioral defaults, deferred items (T67)`.
---
## Wrap-up
After Wave 7 lands:
1. **Run full suite** on `phase-3`: should be ~260+ tests passing (212 from Phase 2 + ~50 new).
2. **Manual smoke** (recommended before opening the PR):
- Plan an event from the drawer; play turns until it completes; verify promotion landed (drawer shows updated edges / memories).
- Use elision and jump skips both via natural language and the drawer.
- Close a scene that opened a thread; verify the thread renders in the next scene's prompt.
- Trigger a meanwhile scene from the drawer; play 2 turns; close it; resume the main you-scene; verify the digest renders once and not again.
3. **Push `phase-3`** to gitea.
4. **Open PR** `phase-3 → main`.
5. **Phase 3.5 backlog candidates** (track in CLAUDE.md): inline natural-language jump prompt UI, regenerate-aware event-transition undo, classifier-driven meanwhile turn-taking, drawer surface for closed-event browsing, event template library (kind presets with default props).
---
## Notes for the controller running this plan
- **Don't dispatch Wave 5b until Wave 5a is merged AND green on `phase-3`.** Wave 5b's `turns.py` modifications layer on top of T61's recent additions; missing that produces merge conflicts or import-time failures.
- **Don't dispatch T64+T65 until T63 merges.** Both depend on the new `present_set_kind` column and the meanwhile event kinds.
- **After each parallel wave**, run a code-review subagent (`subagent-driven-development` skill's two-stage review pattern) on each task before merging to `phase-3`. For purely mechanical tasks (schema migrations, projector handlers), a combined spec+quality review is acceptable. For T62, T64, T65 (large or integration tasks), use separate spec + quality reviewers.
- **If a parallel wave's merge produces a conflict**, the wave's file-disjointness assumption was violated. Bisect the affected pair, fix the offending task in a follow-up commit on `phase-3`, and proceed.
- **Schema-version test bumps** happen at Wave 1 merge (8 → 10) and Wave 6 merge (10 → 11 or 12 depending on T65's migration choice). Update `tests/test_world.py` once per affected merge — same pattern as Phase 2 T36.
- **Token-spend rough estimate**: Phase 3 should be larger than Phase 2 (~1.5×) — events / skips / meanwhile each carry their own state + service + UI surfaces. Per-task token spend similar to Phase 2's larger tasks (T42, T44).
- **DO NOT modify Phase 1 / 2 code paths** unless explicitly required (e.g., T58 modifies `scene_summarize.py` because the new behavior is genuinely additive). Existing 1- and 2-entity flows must continue to work end-to-end after each wave.
@@ -0,0 +1,26 @@
{
"planPath": "docs/plans/2026-04-26-v3-phase3-implementation.md",
"tasks": [
{"id": 49, "subject": "T49: events table + lifecycle handlers", "status": "pending", "wave": 1, "parallelGroup": "wave-1"},
{"id": 50, "subject": "T50: time_skip event kinds + chat-clock handlers", "status": "pending", "wave": 1, "parallelGroup": "wave-1"},
{"id": 51, "subject": "T51: threads table + open/update/close handlers", "status": "pending", "wave": 1, "parallelGroup": "wave-1"},
{"id": 52, "subject": "T52: event-lifecycle detection service", "status": "pending", "wave": 2, "parallelGroup": "wave-2", "blockedBy": [49]},
{"id": 53, "subject": "T53: skip narration service (elision + jump)", "status": "pending", "wave": 2, "parallelGroup": "wave-2", "blockedBy": [50]},
{"id": 54, "subject": "T54: synthesized-memories service for jump skips", "status": "pending", "wave": 2, "parallelGroup": "wave-2", "blockedBy": [50]},
{"id": 55, "subject": "T55: thread-detection service", "status": "pending", "wave": 2, "parallelGroup": "wave-2", "blockedBy": [51]},
{"id": 56, "subject": "T56: event-completion promotion service", "status": "pending", "wave": 3, "parallelGroup": "wave-3", "blockedBy": [49, 52]},
{"id": 57, "subject": "T57: significance-aware retrieval ranking", "status": "pending", "wave": 3, "parallelGroup": "wave-3"},
{"id": 58, "subject": "T58: scene compression keeps key quotes + emits thread events", "status": "pending", "wave": 3, "parallelGroup": "wave-3", "blockedBy": [55]},
{"id": 59, "subject": "T59: drawer events / threads / skip controls", "status": "pending", "wave": 4, "parallelGroup": null, "blockedBy": [49, 50, 51, 53, 54]},
{"id": 60, "subject": "T60: prompt assembly includes active events + open threads", "status": "pending", "wave": 5, "parallelGroup": "wave-5a", "blockedBy": [49, 51]},
{"id": 61, "subject": "T61: turn flow invokes event-detection + completion promotion", "status": "pending", "wave": 5, "parallelGroup": "wave-5a", "blockedBy": [52, 56]},
{"id": 62, "subject": "T62: skip command surface (parse + route + jump UI)", "status": "pending", "wave": 5, "parallelGroup": null, "blockedBy": [50, 53, 54, 60, 61]},
{"id": 63, "subject": "T63: meanwhile scene config — schema + state", "status": "pending", "wave": 6, "parallelGroup": null},
{"id": 64, "subject": "T64: meanwhile turn flow (host+guest, no you)", "status": "pending", "wave": 6, "parallelGroup": "wave-6b", "blockedBy": [63]},
{"id": 65, "subject": "T65: meanwhile summary digest surfaces to next you-scene", "status": "pending", "wave": 6, "parallelGroup": "wave-6b", "blockedBy": [63]},
{"id": 66, "subject": "T66: cross-feature integration tests", "status": "pending", "wave": 7, "parallelGroup": "wave-7", "blockedBy": [62, 64, 65]},
{"id": 67, "subject": "T67: Phase 3 documentation update", "status": "pending", "wave": 7, "parallelGroup": "wave-7", "blockedBy": [62, 64, 65]}
],
"lastUpdated": "2026-04-26T00:00:00Z",
"notes": "19 tasks across 8 waves (1, 2, 3, 4, 5a, 5b, 6a, 6b, 7). Waves 1, 2, 3, 5a, and 7 are fully parallel-safe (file-disjoint within each). Waves 4, 5b, and 6a are single-task. Wave 6b is parallel after 6a (T63) merges. Use Agent tool with isolation: 'worktree' to dispatch parallel tasks. Merge each wave's worktrees back into phase-3 before dispatching the next wave. See plan §Parallel-Execution Strategy for full guidance. Schema baseline: Phase 2 ends at version 8; Phase 3 adds 0009_events.sql, 0010_threads.sql, 0011_meanwhile_scenes.sql (final version 11)."
}
+31
View File
@@ -0,0 +1,31 @@
[project]
name = "chat"
version = "0.1.0"
requires-python = ">=3.11"
dependencies = [
"fastapi>=0.110",
"uvicorn[standard]>=0.30",
"httpx>=0.27",
"pydantic>=2.6",
"pydantic-settings>=2.2",
"openai>=1.30",
"instructor>=1.3",
"tiktoken>=0.7",
"jinja2>=3.1",
"aiosqlite>=0.20",
"python-multipart>=0.0.9",
]
[project.optional-dependencies]
dev = ["pytest>=8", "pytest-asyncio>=0.23", "freezegun>=1.4"]
[build-system]
requires = ["setuptools>=61"]
build-backend = "setuptools.build_meta"
[tool.setuptools.packages.find]
include = ["chat*"]
[tool.pytest.ini_options]
pythonpath = ["."]
asyncio_mode = "auto"
+253
View File
@@ -0,0 +1,253 @@
"""Seed three sample bots via direct event-log append.
Idempotent: re-running skips bots whose ids already exist.
Run from the repo root:
.venv/bin/python scripts/seed_sample_bots.py
After running, walk each bot through kickoff parse-and-confirm at:
http://127.0.0.1:8000/bots/<id>/kickoff
"""
from __future__ import annotations
from chat.config import load_settings
from chat.db.connection import open_db
from chat.db.migrate import apply_migrations
from chat.eventlog.log import append_and_apply
from chat.state.entities import get_bot
# Trigger handler registration.
import chat.state.entities # noqa: F401
import chat.state.edges # noqa: F401
import chat.state.memory # noqa: F401
import chat.state.world # noqa: F401
import chat.state.manual_edit # noqa: F401
SAMPLES: list[dict] = [
{
"id": "maya",
"name": "Maya Chen",
"persona": (
"31, senior product designer at the same company you work for. "
"Sharp eye for what isn't said. Outwardly composed and dryly funny; "
"privately prone to overthinking and drafting texts she never sends. "
"Came out of a five-year relationship six months ago and is still "
"pretending she's fine."
),
"voice_samples": [
(
"\"You look like someone who needs water more than another coffee. "
"I'm just saying.\""
),
(
"She tilts her laptop screen toward you without looking up. "
"\"Tell me which one. Don't think about it. The first one your eye "
"lands on is the right one.\""
),
(
"A pause. \"I'm going to head out. ...Unless you want company on the "
"elevator. Which is a weird sentence I just said out loud.\""
),
],
"traits": [
"dry humor",
"observant",
"perfectionist",
"quick to deflect compliments",
"late-night texter",
"runs on cold brew",
"slow to trust",
"draws in margins when bored",
"hates small talk",
"secretly sentimental",
],
"backstory": (
"Grew up in Vancouver, only child of immigrant parents. Design school "
"in Toronto. Three years at this company; took the senior role eight "
"months ago after the previous lead left abruptly. Her father died of "
"a stroke last fall — she flew home for the funeral and was back at "
"her desk on Monday. She has not really talked to anyone about it, "
"including her mother, including her therapist, including her best "
"friend. She works too much. She knows she works too much."
),
"initial_relationship_to_you": (
"Coworkers for about eighteen months. Two desks over. You've been on "
"the same product team for the last year. She thinks you're one of the "
"very few people at the company who actually thinks before speaking, "
"which she finds annoying and also relieving. The two of you have "
"lunch sometimes. You stayed late together once before the big launch "
"and she doesn't remember exactly what was said but she remembers the "
"feeling of the empty office. She has not admitted anything to "
"herself. You probably haven't either."
),
"kickoff_prose": (
"It's 9:14 on a Thursday and you and Maya are the only people left on "
"the floor. The deck is due in the morning. She has her shoes off "
"under her desk. The kitchen lights flicker once and then steady. She "
"slides her chair back and rubs her eyes with the heels of her hands. "
"\"Okay,\" she says, to no one in particular, \"tell me honestly. "
"Slide eleven — does that read as ambitious or as desperate.\""
),
},
{
"id": "eli",
"name": "Eli Park",
"persona": (
"34, freelance illustrator. Quiet, tactile, generous with attention "
"but stingy with words. Bakes when stressed. Falls asleep on the "
"couch with his glasses on. Loves you in the kind of way that doesn't "
"need to be announced."
),
"voice_samples": [
(
"\"Hey.\" A pause, like he's deciding if it's worth saying. \"You "
"ate, right?\""
),
(
"He kisses the top of your head and keeps walking, not breaking "
"stride. \"Don't fall asleep on the bathroom floor again. That's "
"all I'm saying.\""
),
(
"\"You don't have to. I just.\" He looks at his hands. \"I just "
"like it when you're around when I'm working. It's stupid. It's "
"whatever.\""
),
],
"traits": [
"warm",
"present",
"distractible",
"terrible at confrontation",
"leaves coffee mugs in every room",
"draws on napkins",
"gets up at 6am to paint",
"owns far too many sweaters",
"never throws anything away",
"holds your hand without thinking about it",
],
"backstory": (
"Born and raised in Queens to Korean parents who ran a dry cleaner. "
"Older sister Lena died in a car accident when he was nineteen — the "
"year he left for art school. He won't talk about her on most days "
"but he keeps a small photo of her in his wallet, and on her birthday "
"he stops talking by 7pm and goes to bed early. He has been a "
"freelance illustrator for nine years. His work has appeared in The "
"New Yorker twice and he refuses to make this a personality trait. "
"He pays his bills on time. He loses his keys constantly."
),
"initial_relationship_to_you": (
"You've been together for four years, living together for two. He "
"proposed last summer, kind of — it was tentative and circular and "
"the question wasn't really a question, and you both laughed and "
"didn't really resolve it, and somewhere there is an unspent ring in "
"a sock drawer. You bicker about laundry and the right way to load a "
"dishwasher. He has seen you cry over genuinely stupid commercials. "
"You are each other's first call. You sleep on the left side."
),
"kickoff_prose": (
"Sunday morning, late. The blinds are still down. Eli is propped "
"against the headboard reading something on his phone, his glasses "
"pushed up into his hair. You've been awake for a while; he just "
"noticed. He sets the phone face-down on his chest and looks over at "
"you with the small private smile he only uses in this room. \"Hi,\" "
"he says, like it's a whole sentence."
),
},
{
"id": "sam",
"name": "Samira Reyes",
"persona": (
"28, bartender at a small cocktail bar near where you live, doing a "
"part-time master's in psychology she refuses to talk about. "
"Confident posture, careful words. Reads people fast and shares the "
"readings only when she likes them. Single by deliberate choice for "
"the last two years."
),
"voice_samples": [
(
"\"You're back.\" She says it without looking up from polishing "
"the glass. \"Same as last time, or are we trying something new "
"tonight.\""
),
(
"A long look. \"I'm going to ask you a question and you're not "
"going to answer it carefully. The first thing that comes into "
"your head. Ready.\""
),
(
"\"Don't tip me extra because we talked. I'm being serious. "
"That's a different transaction and I don't want it confused.\""
),
],
"traits": [
"observant",
"blunt",
"kind in unexpected ways",
"reads tarot for fun (doesn't believe in it)",
"drinks black coffee",
"runs at 5am",
"doesn't suffer fools",
"never forgets a face",
"occasional smoker when something is bothering her",
"owns three identical black t-shirts",
],
"backstory": (
"Born and raised in El Paso to a single mother who waitressed nights. "
"Came north five years ago for undergrad on a scholarship. Funded the "
"master's herself by bartending — she's careful about money in a way "
"that took being broke to learn. Her undergraduate thesis was on "
"attachment styles and she will not tell you what her own attachment "
"style is. Her mother passed away two years ago after a long illness, "
"and she went home for a month and came back different in ways she "
"can't articulate."
),
"initial_relationship_to_you": (
"You've talked at her bar maybe six times over the last month. She "
"knows your drink. The conversations have started lasting longer than "
"they should — you stay until close more often than you mean to. Last "
"week you walked her to her car at 1am because the lot is dim. "
"Nothing happened. You just talked, leaning on the hood of her old "
"Civic, longer than either of you intended. Neither of you has texted "
"the other since. Neither of you has stopped thinking about it."
),
"kickoff_prose": (
"It's 11:47 on a Tuesday — slow night. There's exactly one other "
"customer at the far end of the bar, finishing a beer he stopped "
"drinking ten minutes ago. Sam is wiping down the counter in long "
"unhurried passes. She glances up when the door chimes and the small "
"surprise on her face is gone before you'd swear it was there. "
"\"Look who it is,\" she says, even and unreadable, and pulls down a "
"glass without asking what you want."
),
},
]
def main() -> None:
settings = load_settings()
apply_migrations(settings.db_path)
created: list[str] = []
skipped: list[str] = []
with open_db(settings.db_path) as conn:
for spec in SAMPLES:
if get_bot(conn, spec["id"]) is not None:
skipped.append(spec["id"])
continue
append_and_apply(conn, kind="bot_authored", payload=spec)
created.append(spec["id"])
print(f"created: {created}")
print(f"skipped (already existed): {skipped}")
print()
print("Walk each new bot through kickoff parse-and-confirm:")
for bot_id in created:
print(f" http://127.0.0.1:8000/bots/{bot_id}/kickoff")
if __name__ == "__main__":
main()
View File
+99
View File
@@ -0,0 +1,99 @@
"""Addressee classifier service tests (T74.1).
Covers :func:`chat.services.addressee.detect_addressee`:
- Classifier picks the guest -> ``addressee_id == guest_id``.
- Classifier picks the host -> ``addressee_id == host_id``.
- Classifier flakes (3 bad-JSON responses, exhausting the built-in
retry budget in :func:`chat.llm.classify.classify`) -> fallback to
the host with ``reason="fallback"``.
"""
from __future__ import annotations
import json
import pytest
from chat.llm.mock import MockLLMClient
from chat.services.addressee import AddresseeDecision, detect_addressee
@pytest.mark.asyncio
async def test_classifier_picks_guest():
"""Classifier returns the guest id verbatim — caller propagates it."""
canned = [
json.dumps(
{
"addressee_id": "bot_b",
"confidence": "high",
"reason": "user named BotB",
}
)
]
client = MockLLMClient(canned=canned)
result = await detect_addressee(
client,
classifier_model="test-model",
user_prose="BotB, what do you think?",
host_id="bot_a",
host_name="BotA",
guest_id="bot_b",
guest_name="BotB",
)
assert isinstance(result, AddresseeDecision)
assert result.addressee_id == "bot_b"
assert result.confidence == "high"
@pytest.mark.asyncio
async def test_classifier_picks_host():
"""Classifier returns the host id — caller propagates it."""
canned = [
json.dumps(
{
"addressee_id": "bot_a",
"confidence": "medium",
"reason": "narration aimed at host",
}
)
]
client = MockLLMClient(canned=canned)
result = await detect_addressee(
client,
classifier_model="test-model",
user_prose="I lean back and stretch.",
host_id="bot_a",
host_name="BotA",
guest_id="bot_b",
guest_name="BotB",
)
assert result.addressee_id == "bot_a"
assert result.confidence == "medium"
@pytest.mark.asyncio
async def test_classifier_failure_falls_back_to_host():
"""Three bad-JSON responses exhaust the retry budget and the
classifier-failure fallback returns ``host_id`` with
``reason="fallback"``."""
canned = ["not json", "still not json", "garbage"]
client = MockLLMClient(canned=canned)
result = await detect_addressee(
client,
classifier_model="test-model",
user_prose="anything",
host_id="bot_a",
host_name="BotA",
guest_id="bot_b",
guest_name="BotB",
)
assert result.addressee_id == "bot_a"
assert result.reason == "fallback"
assert result.confidence == "low"
+92
View File
@@ -0,0 +1,92 @@
"""Tests for nightly DB backups (T32).
The backup service is intentionally simple: a flat ``data/backups/`` dir
containing timestamped copies of ``chat.db``, with retention of the most
recent 14. The scheduling decision (``should_take_backup``) is a pure
function of clock + filesystem state so it can be unit-tested without
spinning up the BackgroundWorker tick loop.
"""
from __future__ import annotations
from datetime import datetime
from unittest.mock import patch
from chat.services.backup import (
prune_backups,
should_take_backup,
take_backup,
)
def test_take_backup_creates_timestamped_copy(tmp_path):
db = tmp_path / "chat.db"
db.write_text("fake db contents")
backup_path = take_backup(db_path=db, data_dir=tmp_path / "data")
assert backup_path.exists()
assert backup_path.name.startswith("chat-")
assert backup_path.name.endswith(".db")
# Contents copied
assert backup_path.read_text() == "fake db contents"
# Located in data/backups/
assert backup_path.parent == tmp_path / "data" / "backups"
def test_prune_keeps_last_14(tmp_path):
backup_dir = tmp_path / "data" / "backups"
backup_dir.mkdir(parents=True)
# Create 17 dummy backup files spanning days 1..17 of Jan 2026.
# Filenames sort lexicographically by the embedded timestamp, so
# prune_backups should drop the three oldest.
for i in range(1, 18):
(backup_dir / f"chat-202601{i:02d}T000000Z.db").write_text(
f"backup {i}"
)
removed = prune_backups(tmp_path / "data", keep=14)
assert removed == 3
remaining = sorted(backup_dir.glob("chat-*.db"))
assert len(remaining) == 14
# Days 1, 2, 3 removed; day 4 is now the oldest retained backup.
assert remaining[0].name == "chat-20260104T000000Z.db"
def test_should_take_backup_when_no_prior_and_target_hour_matches(tmp_path):
from chat.services import backup as backup_mod
class FakeDateTime(datetime):
@classmethod
def now(cls, tz=None):
return datetime(2026, 4, 26, 3, 0, 0)
with patch.object(backup_mod, "datetime", FakeDateTime):
assert should_take_backup(tmp_path / "data") is True
def test_should_not_take_backup_outside_target_hour(tmp_path):
from chat.services import backup as backup_mod
class FakeDateTime(datetime):
@classmethod
def now(cls, tz=None):
return datetime(2026, 4, 26, 14, 0, 0)
with patch.object(backup_mod, "datetime", FakeDateTime):
assert should_take_backup(tmp_path / "data") is False
def test_should_not_take_backup_when_recent_backup_exists(tmp_path):
backup_dir = tmp_path / "data" / "backups"
backup_dir.mkdir(parents=True)
recent = backup_dir / "chat-recent.db"
recent.write_text("x")
# mtime defaults to "now" — within the 23h freshness window so
# should_take_backup must return False even at the target hour.
from chat.services import backup as backup_mod
class FakeDateTime(datetime):
@classmethod
def now(cls, tz=None):
return datetime(2026, 4, 26, 3, 0, 0)
with patch.object(backup_mod, "datetime", FakeDateTime):
assert should_take_backup(tmp_path / "data") is False
+132
View File
@@ -0,0 +1,132 @@
from __future__ import annotations
import pytest
from fastapi.testclient import TestClient
from chat.app import app
@pytest.fixture
def client(tmp_path, monkeypatch):
config_path = tmp_path / "config.toml"
config_path.write_text('featherless_api_key = "test"\n')
monkeypatch.setenv("CHAT_CONFIG_PATH", str(config_path))
monkeypatch.setenv("CHAT_DB_PATH", str(tmp_path / "test.db"))
with TestClient(app) as c:
yield c
def test_get_new_bot_form_renders(client):
response = client.get("/bots/new")
assert response.status_code == 200
body = response.text.lower()
assert "<form" in body
assert "name" in body
assert "persona" in body
assert "kickoff" in body
def test_post_new_bot_appends_event_and_redirects(client, tmp_path):
response = client.post(
"/bots/new",
data={
"id": "bot_a",
"name": "BotA",
"persona": "thoughtful, observant",
"voice_samples": "first sample\n---\nsecond sample",
"traits": "shy, quick to anger",
"backstory": "grew up in a small town",
"initial_relationship_to_you": "coworker",
"kickoff_prose": "you stay late at the office",
},
follow_redirects=False,
)
assert response.status_code == 303
assert response.headers["location"] == "/bots/bot_a/kickoff"
from chat.db.connection import open_db
from chat.state.entities import get_bot
with open_db(tmp_path / "test.db") as conn:
bot = get_bot(conn, "bot_a")
assert bot is not None
assert bot["name"] == "BotA"
assert bot["voice_samples"] == ["first sample", "second sample"]
assert bot["traits"] == ["shy", "quick to anger"]
assert bot["backstory"] == "grew up in a small town"
assert bot["initial_relationship_to_you"] == "coworker"
assert bot["kickoff_prose"] == "you stay late at the office"
# Confirm event was actually appended (state goes through event log).
cur = conn.execute(
"SELECT kind, payload_json FROM event_log WHERE kind = 'bot_authored'"
)
rows = cur.fetchall()
assert len(rows) == 1
def test_post_new_bot_rejects_missing_required(client):
response = client.post(
"/bots/new",
data={"id": "bot_b"},
follow_redirects=False,
)
assert response.status_code == 400
def test_get_bots_list_renders(client):
response = client.get("/bots")
assert response.status_code == 200
def test_post_new_bot_empty_traits_parses_to_empty_list(client, tmp_path):
response = client.post(
"/bots/new",
data={
"id": "bot_c",
"name": "BotC",
"persona": "stoic",
"voice_samples": "",
"traits": "",
"backstory": "",
"initial_relationship_to_you": "stranger",
"kickoff_prose": "the rain begins",
},
follow_redirects=False,
)
assert response.status_code == 303
from chat.db.connection import open_db
from chat.state.entities import get_bot
with open_db(tmp_path / "test.db") as conn:
bot = get_bot(conn, "bot_c")
assert bot is not None
assert bot["voice_samples"] == []
assert bot["traits"] == []
def test_post_new_bot_traits_split_by_newlines(client, tmp_path):
response = client.post(
"/bots/new",
data={
"id": "bot_d",
"name": "BotD",
"persona": "curious",
"voice_samples": "",
"traits": "calm\nthoughtful\nguarded",
"backstory": "",
"initial_relationship_to_you": "neighbor",
"kickoff_prose": "morning light",
},
follow_redirects=False,
)
assert response.status_code == 303
from chat.db.connection import open_db
from chat.state.entities import get_bot
with open_db(tmp_path / "test.db") as conn:
bot = get_bot(conn, "bot_d")
assert bot is not None
assert bot["traits"] == ["calm", "thoughtful", "guarded"]
+120
View File
@@ -0,0 +1,120 @@
from __future__ import annotations
from pathlib import Path
import pytest
from fastapi.testclient import TestClient
from chat.app import app
from chat.eventlog.log import append_event
from chat.eventlog.projector import project
@pytest.fixture
def client(tmp_path, monkeypatch):
config_path = tmp_path / "config.toml"
config_path.write_text('featherless_api_key = "test"\n')
monkeypatch.setenv("CHAT_CONFIG_PATH", str(config_path))
monkeypatch.setenv("CHAT_DB_PATH", str(tmp_path / "test.db"))
with TestClient(app) as c:
yield c
def _author_you(db_path: Path) -> None:
"""Author a ``you_entity`` so the first-run middleware doesn't redirect."""
from chat.db.connection import open_db
with open_db(db_path) as conn:
append_event(
conn,
kind="you_authored",
payload={"name": "Me", "pronouns": "", "persona": ""},
)
project(conn)
def _author_bot_and_chat(db_path: Path, bot_id: str = "bot_a") -> None:
"""Insert a you_entity, bot, and chat via the event log (skip kickoff route)."""
from chat.db.connection import open_db
with open_db(db_path) as conn:
append_event(
conn,
kind="you_authored",
payload={"name": "Me", "pronouns": "", "persona": ""},
)
append_event(
conn,
kind="bot_authored",
payload={
"id": bot_id,
"name": "BotA",
"persona": "thoughtful, observant",
"voice_samples": [],
"traits": ["shy"],
"backstory": "",
"initial_relationship_to_you": "coworker",
"kickoff_prose": "you stay late at the office; she's there too",
},
)
append_event(
conn,
kind="chat_created",
payload={
"id": f"chat_{bot_id}",
"host_bot_id": bot_id,
"initial_time": "2026-04-26T20:00:00+00:00",
"narrative_anchor": "Day 1",
"weather": "",
},
)
project(conn)
def test_root_redirects_to_chats_when_setup_complete(client, tmp_path):
# With both you_entity and a bot present, the first-run middleware
# passes through and the nav router sends "/" → "/chats".
_author_bot_and_chat(tmp_path / "test.db", "bot_a")
response = client.get("/", follow_redirects=False)
assert response.status_code == 303
assert response.headers["location"] == "/chats"
def test_chats_list_empty_state(client, tmp_path):
# Author you + a bot but NO chats — should render the empty-state
# chats list, not redirect.
_author_bot_and_chat(tmp_path / "test.db", "bot_a")
# Drop the chat row so we hit the empty-state branch (the helper
# creates a chat — undo it via a fresh seed without chat_created).
from chat.db.connection import open_db
with open_db(tmp_path / "test.db") as conn:
conn.execute("DELETE FROM chats")
conn.commit()
response = client.get("/chats")
assert response.status_code == 200
body = response.text.lower()
# Empty state should mention there are no chats yet.
assert "no chats yet" in body
def test_chats_list_renders_existing_chats(client, tmp_path):
_author_bot_and_chat(tmp_path / "test.db", "bot_a")
response = client.get("/chats")
assert response.status_code == 200
body = response.text
# The bot's display name should appear in the chat row.
assert "BotA" in body
# The chat's in-fiction time should appear in the meta.
assert "2026-04-26T20:00:00+00:00" in body
def test_existing_template_routes_still_work_with_new_layout(client):
# Smoke test the layout reshuffle didn't break the existing pages.
for path in ("/bots", "/bots/new", "/settings"):
response = client.get(path)
assert response.status_code == 200, f"{path} returned {response.status_code}"
body = response.text
# Each page should now show the persistent left-rail brand link.
assert 'class="rail"' in body or "rail-brand" in body
+83
View File
@@ -0,0 +1,83 @@
from __future__ import annotations
from pathlib import Path
import pytest
from fastapi.testclient import TestClient
from chat.app import app
from chat.db.connection import open_db
from chat.eventlog.log import append_event
from chat.eventlog.projector import project
@pytest.fixture
def client(tmp_path, monkeypatch):
cfg = tmp_path / "config.toml"
cfg.write_text('featherless_api_key = "test"\n')
monkeypatch.setenv("CHAT_CONFIG_PATH", str(cfg))
db = tmp_path / "test.db"
monkeypatch.setenv("CHAT_DB_PATH", str(db))
with TestClient(app) as c:
yield c
def _seed_chat(db_path: Path, bot_id: str = "bot_a", chat_id: str = "chat_bot_a") -> None:
"""Author a bot, create a chat with default state."""
with open_db(db_path) as conn:
append_event(
conn,
kind="bot_authored",
payload={
"id": bot_id,
"name": "BotA",
"persona": "...",
"voice_samples": [],
"traits": [],
"backstory": "",
"initial_relationship_to_you": "",
"kickoff_prose": "...",
},
)
append_event(
conn,
kind="chat_created",
payload={
"id": chat_id,
"host_bot_id": bot_id,
"initial_time": "2026-04-26T20:00:00+00:00",
"narrative_anchor": "Day 1",
"weather": "",
},
)
project(conn)
def test_get_chat_404_when_missing(client):
response = client.get("/chats/no_such_chat")
assert response.status_code == 404
def test_get_chat_renders_shell_with_host_bot_name(client, tmp_path):
_seed_chat(tmp_path / "test.db")
response = client.get("/chats/chat_bot_a")
assert response.status_code == 200
body = response.text
assert "BotA" in body
assert "<form" in body # turn input form
assert "drawer" in body.lower() # drawer present
assert "no turns yet" in body.lower() # empty timeline placeholder
def test_get_chat_includes_turn_post_action(client, tmp_path):
_seed_chat(tmp_path / "test.db")
response = client.get("/chats/chat_bot_a")
assert response.status_code == 200
assert 'action="/chats/chat_bot_a/turns"' in response.text
def test_get_chat_shows_chat_clock_time(client, tmp_path):
_seed_chat(tmp_path / "test.db")
response = client.get("/chats/chat_bot_a")
assert response.status_code == 200
assert "2026-04-26" in response.text
+24
View File
@@ -0,0 +1,24 @@
import pytest
from pydantic import BaseModel
from chat.llm.mock import MockLLMClient
from chat.llm.classify import classify
class Verdict(BaseModel):
score: int
reason: str
@pytest.mark.asyncio
async def test_classify_parses_valid_json():
mock = MockLLMClient(canned=['{"score": 2, "reason": "notable"}'])
result = await classify(mock, model="m", system="x", user="y", schema=Verdict)
assert result.score == 2
@pytest.mark.asyncio
async def test_classify_falls_back_on_unparseable_after_retry():
mock = MockLLMClient(canned=["nope", "still nope", "nope3"])
default = Verdict(score=1, reason="fallback")
result = await classify(mock, model="m", system="x", user="y", schema=Verdict, default=default)
assert result.reason == "fallback"
+26
View File
@@ -0,0 +1,26 @@
import os
from pathlib import Path
import pytest
from chat.config import load_settings
def test_load_settings_reads_toml(tmp_path, monkeypatch):
cfg = tmp_path / "config.toml"
cfg.write_text("""
featherless_api_key = "sk-test"
narrative_model = "dphn/Dolphin-Mistral-24B-Venice-Edition"
classifier_model = "NousResearch/Hermes-3-Llama-3.1-8B"
ooc_marker = "(("
retrieval_k = 4
""")
monkeypatch.setenv("CHAT_CONFIG_PATH", str(cfg))
s = load_settings()
assert s.featherless_api_key == "sk-test"
assert s.narrative_model.startswith("dphn/")
assert s.retrieval_k == 4
def test_chat_db_path_env_overrides_default(tmp_path, monkeypatch):
monkeypatch.setenv("CHAT_DB_PATH", str(tmp_path / "alt.db"))
monkeypatch.setenv("CHAT_CONFIG_PATH", str(tmp_path / "config.toml"))
(tmp_path / "config.toml").write_text('featherless_api_key = "x"\n')
s = load_settings()
assert s.db_path == tmp_path / "alt.db"
+190
View File
@@ -0,0 +1,190 @@
"""T25: drawer edits with manual_edit event capture.
Each editable field on the drawer is exposed as a small POST endpoint.
Edits emit either a ``manual_edit`` event (snapshotting the prior value
for §6.4 reversibility) or, for pin toggles, a ``memory_pin_changed``
event with ``auto_pinned=0`` so manual pins survive auto-eviction.
Phase 1 narrowed scope: affinity slider, significance dropdown, pin
toggle. Other §6.4 fields (activity, edge_summary, edge_trust, pov_summary,
knowledge_facts list) are deferred to a Phase 1.5 follow-up.
"""
from __future__ import annotations
import json
from pathlib import Path
import pytest
from fastapi.testclient import TestClient
from chat.app import app
from chat.db.connection import open_db
from chat.eventlog.log import append_event
from chat.eventlog.projector import project
@pytest.fixture
def client(tmp_path, monkeypatch):
cfg = tmp_path / "config.toml"
cfg.write_text('featherless_api_key = "test"\n')
monkeypatch.setenv("CHAT_CONFIG_PATH", str(cfg))
db = tmp_path / "test.db"
monkeypatch.setenv("CHAT_DB_PATH", str(db))
with TestClient(app) as c:
if hasattr(app.state, "background_worker"):
app.state.background_worker.enabled = False
yield c
def _seed(db: Path) -> None:
with open_db(db) as conn:
append_event(
conn,
kind="bot_authored",
payload={
"id": "bot_a",
"name": "BotA",
"persona": "...",
"voice_samples": [],
"traits": [],
"backstory": "",
"initial_relationship_to_you": "",
"kickoff_prose": "",
},
)
append_event(
conn,
kind="chat_created",
payload={
"id": "chat_bot_a",
"host_bot_id": "bot_a",
"initial_time": "2026-04-26T20:00:00+00:00",
"narrative_anchor": "Day 1",
"weather": "",
},
)
# Edge bot_a -> you with affinity_delta=0 to materialise the row at
# default 50/50.
append_event(
conn,
kind="edge_update",
payload={
"source_id": "bot_a",
"target_id": "you",
"chat_id": "chat_bot_a",
"affinity_delta": 0,
},
)
append_event(
conn,
kind="memory_written",
payload={
"owner_id": "bot_a",
"chat_id": "chat_bot_a",
"pov_summary": "A memory",
"witness_you": 1,
"witness_host": 1,
"witness_guest": 0,
"significance": 1,
},
)
project(conn)
def test_edit_edge_affinity_emits_manual_edit_and_updates(client, tmp_path):
_seed(tmp_path / "test.db")
response = client.post(
"/chats/chat_bot_a/drawer/edge/bot_a/you/affinity",
data={"affinity": "75"},
)
assert response.status_code == 200 # returns refreshed drawer partial
# Refresh shows the new affinity value.
assert "75" in response.text
with open_db(tmp_path / "test.db") as conn:
cur = conn.execute(
"SELECT payload_json FROM event_log WHERE kind = 'manual_edit'"
).fetchall()
assert len(cur) == 1
payload = json.loads(cur[0][0])
assert payload["target_kind"] == "edge_affinity"
assert payload["prior_value"] == 50
assert payload["new_value"] == 75
assert payload["target_id"]["source_id"] == "bot_a"
assert payload["target_id"]["target_id"] == "you"
from chat.state.edges import get_edge
edge = get_edge(conn, "bot_a", "you")
assert edge["affinity"] == 75
def test_edit_memory_significance_emits_event(client, tmp_path):
_seed(tmp_path / "test.db")
with open_db(tmp_path / "test.db") as conn:
memory_id = conn.execute("SELECT id FROM memories LIMIT 1").fetchone()[0]
response = client.post(
f"/chats/chat_bot_a/drawer/memory/{memory_id}/significance",
data={"significance": "3"},
)
assert response.status_code == 200
with open_db(tmp_path / "test.db") as conn:
sig = conn.execute(
"SELECT significance FROM memories WHERE id = ?", (memory_id,)
).fetchone()[0]
assert sig == 3
cur = conn.execute(
"SELECT payload_json FROM event_log WHERE kind = 'manual_edit'"
).fetchall()
assert len(cur) == 1
payload = json.loads(cur[0][0])
assert payload["target_kind"] == "memory_significance"
assert payload["prior_value"] == 1
assert payload["new_value"] == 3
assert payload["target_id"] == memory_id
def test_toggle_memory_pin_manual_emits_event_with_auto_pinned_0(client, tmp_path):
_seed(tmp_path / "test.db")
with open_db(tmp_path / "test.db") as conn:
memory_id = conn.execute("SELECT id FROM memories LIMIT 1").fetchone()[0]
response = client.post(
f"/chats/chat_bot_a/drawer/memory/{memory_id}/pin",
data={"pinned": "1"},
)
assert response.status_code == 200
with open_db(tmp_path / "test.db") as conn:
row = conn.execute(
"SELECT pinned, auto_pinned FROM memories WHERE id = ?", (memory_id,)
).fetchone()
assert row[0] == 1
assert row[1] == 0 # NOT auto-pinned (manual pin survives auto-eviction)
# The pin toggle uses memory_pin_changed (not manual_edit).
cur = conn.execute(
"SELECT payload_json FROM event_log "
"WHERE kind = 'memory_pin_changed' ORDER BY id DESC LIMIT 1"
).fetchone()
payload = json.loads(cur[0])
assert payload["pinned"] == 1
assert payload["auto_pinned"] == 0
assert payload["memory_id"] == memory_id
def test_edit_404_when_chat_missing(client):
response = client.post(
"/chats/no_such/drawer/edge/bot_a/you/affinity",
data={"affinity": "75"},
)
assert response.status_code == 404
def test_edit_404_when_target_missing(client, tmp_path):
_seed(tmp_path / "test.db")
response = client.post(
"/chats/chat_bot_a/drawer/memory/99999/significance",
data={"significance": "2"},
)
assert response.status_code == 404
+403
View File
@@ -0,0 +1,403 @@
"""T72: deferred v1 drawer edits + witness flag inline-edit.
T25 shipped affinity / significance / pin. T72.1 fills in the rest of the
§6.4 editable surface whose ``manual_edit`` projector dispatch was already
in place (or, for ``edge_knowledge_fact``, added alongside the route):
* ``POST /chats/{chat_id}/drawer/edge/trust`` slider 0..100.
* ``POST /chats/{chat_id}/drawer/edge/summary`` textarea, capped 2000.
* ``POST /chats/{chat_id}/drawer/memory/pov-summary`` textarea, capped.
* ``POST /chats/{chat_id}/drawer/edge/knowledge-facts`` add/remove fact.
T72.3 layers a witness-flag toggle on top:
* ``POST /chats/{chat_id}/drawer/memory/witness`` ``manual_edit`` with
``target_kind`` = ``memory_witness`` and a ``{flag, value}`` payload.
Each test asserts (a) the ``manual_edit`` event lands in the log,
(b) the projected table reflects the new value, and (c) the response is
the refreshed drawer partial.
"""
from __future__ import annotations
import json
from pathlib import Path
import pytest
from fastapi.testclient import TestClient
from chat.app import app
from chat.db.connection import open_db
from chat.eventlog.log import append_event
from chat.eventlog.projector import project
@pytest.fixture
def client(tmp_path, monkeypatch):
cfg = tmp_path / "config.toml"
cfg.write_text('featherless_api_key = "test"\n')
monkeypatch.setenv("CHAT_CONFIG_PATH", str(cfg))
db = tmp_path / "test.db"
monkeypatch.setenv("CHAT_DB_PATH", str(db))
with TestClient(app) as c:
if hasattr(app.state, "background_worker"):
app.state.background_worker.enabled = False
yield c
def _seed(db: Path) -> None:
"""Seed a chat with one host bot, one host->you edge with a fact and
summary already set, and one memory authored by ``bot_a`` witnessed by
all three roles. Tests reach into projected state to verify edits.
"""
with open_db(db) as conn:
append_event(
conn,
kind="bot_authored",
payload={
"id": "bot_a",
"name": "BotA",
"persona": "...",
"voice_samples": [],
"traits": [],
"backstory": "",
"initial_relationship_to_you": "",
"kickoff_prose": "",
},
)
append_event(
conn,
kind="chat_created",
payload={
"id": "chat_bot_a",
"host_bot_id": "bot_a",
"initial_time": "2026-04-26T20:00:00+00:00",
"narrative_anchor": "Day 1",
"weather": "",
},
)
# Materialise edge bot_a -> you with a knowledge_fact already on it
# so the remove path has something to consume.
append_event(
conn,
kind="edge_update",
payload={
"source_id": "bot_a",
"target_id": "you",
"chat_id": "chat_bot_a",
"affinity_delta": 0,
"knowledge_facts": ["studied physics together"],
},
)
append_event(
conn,
kind="memory_written",
payload={
"owner_id": "bot_a",
"chat_id": "chat_bot_a",
"pov_summary": "Original summary text.",
"witness_you": 1,
"witness_host": 1,
"witness_guest": 0,
"significance": 1,
},
)
project(conn)
# --- T72.1 tests ----------------------------------------------------------
def test_edit_edge_trust_emits_manual_edit_and_updates(client, tmp_path):
_seed(tmp_path / "test.db")
response = client.post(
"/chats/chat_bot_a/drawer/edge/trust",
data={"source_id": "bot_a", "target_id": "you", "new_value": "73"},
)
assert response.status_code == 200
# Refresh shows the new trust value somewhere in the partial.
assert "73" in response.text
with open_db(tmp_path / "test.db") as conn:
rows = conn.execute(
"SELECT payload_json FROM event_log WHERE kind = 'manual_edit'"
).fetchall()
assert len(rows) == 1
payload = json.loads(rows[0][0])
assert payload["target_kind"] == "edge_trust"
assert payload["prior_value"] == 50
assert payload["new_value"] == 73
assert payload["target_id"] == {
"source_id": "bot_a",
"target_id": "you",
}
from chat.state.edges import get_edge
edge = get_edge(conn, "bot_a", "you")
assert edge["trust"] == 73
def test_edit_edge_trust_400_on_out_of_range(client, tmp_path):
_seed(tmp_path / "test.db")
response = client.post(
"/chats/chat_bot_a/drawer/edge/trust",
data={"source_id": "bot_a", "target_id": "you", "new_value": "150"},
)
assert response.status_code == 400
def test_edit_edge_summary_emits_manual_edit_and_updates(client, tmp_path):
_seed(tmp_path / "test.db")
response = client.post(
"/chats/chat_bot_a/drawer/edge/summary",
data={
"source_id": "bot_a",
"target_id": "you",
"new_summary": "BotA respects you and shares lab notes.",
},
)
assert response.status_code == 200
with open_db(tmp_path / "test.db") as conn:
rows = conn.execute(
"SELECT payload_json FROM event_log WHERE kind = 'manual_edit'"
).fetchall()
assert len(rows) == 1
payload = json.loads(rows[0][0])
assert payload["target_kind"] == "edge_summary"
assert payload["new_value"].startswith("BotA respects")
assert payload["target_id"] == {
"source_id": "bot_a",
"target_id": "you",
}
summary = conn.execute(
"SELECT summary FROM edges "
"WHERE source_id = ? AND target_id = ?",
("bot_a", "you"),
).fetchone()[0]
assert "respects" in summary
# And the refreshed partial echoes the new summary back.
assert "respects" in response.text
def test_edit_edge_summary_400_on_overflow(client, tmp_path):
_seed(tmp_path / "test.db")
response = client.post(
"/chats/chat_bot_a/drawer/edge/summary",
data={
"source_id": "bot_a",
"target_id": "you",
"new_summary": "x" * 2001,
},
)
assert response.status_code == 400
def test_edit_memory_pov_summary_emits_manual_edit_and_updates(
client, tmp_path
):
_seed(tmp_path / "test.db")
with open_db(tmp_path / "test.db") as conn:
memory_id = conn.execute("SELECT id FROM memories LIMIT 1").fetchone()[0]
response = client.post(
"/chats/chat_bot_a/drawer/memory/pov-summary",
data={
"memory_id": str(memory_id),
"new_summary": "Cleaner per-POV restatement of the moment.",
},
)
assert response.status_code == 200
with open_db(tmp_path / "test.db") as conn:
rows = conn.execute(
"SELECT payload_json FROM event_log WHERE kind = 'manual_edit'"
).fetchall()
assert len(rows) == 1
payload = json.loads(rows[0][0])
assert payload["target_kind"] == "memory_pov_summary"
assert payload["prior_value"] == "Original summary text."
assert payload["new_value"].startswith("Cleaner per-POV")
assert payload["target_id"] == memory_id
pov = conn.execute(
"SELECT pov_summary FROM memories WHERE id = ?", (memory_id,)
).fetchone()[0]
assert pov.startswith("Cleaner per-POV")
assert "Cleaner per-POV" in response.text
def test_edit_memory_pov_summary_404_when_wrong_chat(client, tmp_path):
_seed(tmp_path / "test.db")
with open_db(tmp_path / "test.db") as conn:
memory_id = conn.execute("SELECT id FROM memories LIMIT 1").fetchone()[0]
# Re-home the memory to a different chat to confirm the route's
# cross-chat guard fires.
conn.execute(
"UPDATE memories SET chat_id = 'other_chat' WHERE id = ?",
(memory_id,),
)
conn.commit()
response = client.post(
"/chats/chat_bot_a/drawer/memory/pov-summary",
data={"memory_id": str(memory_id), "new_summary": "..."},
)
assert response.status_code == 404
def test_edit_edge_knowledge_facts_add_emits_event_and_appends(client, tmp_path):
_seed(tmp_path / "test.db")
response = client.post(
"/chats/chat_bot_a/drawer/edge/knowledge-facts",
data={
"source_id": "bot_a",
"target_id": "you",
"action": "add",
"fact": "lent you a textbook",
},
)
assert response.status_code == 200
with open_db(tmp_path / "test.db") as conn:
rows = conn.execute(
"SELECT payload_json FROM event_log WHERE kind = 'manual_edit'"
).fetchall()
assert len(rows) == 1
payload = json.loads(rows[0][0])
assert payload["target_kind"] == "edge_knowledge_fact"
assert payload["new_value"] == {
"action": "add",
"fact": "lent you a textbook",
}
# Prior value snapshots the entire knowledge list before the edit.
assert payload["prior_value"] == ["studied physics together"]
from chat.state.edges import get_edge
edge = get_edge(conn, "bot_a", "you")
assert "lent you a textbook" in edge["knowledge"]
assert "studied physics together" in edge["knowledge"]
assert "lent you a textbook" in response.text
def test_edit_edge_knowledge_facts_remove_drops_matching_fact(client, tmp_path):
_seed(tmp_path / "test.db")
response = client.post(
"/chats/chat_bot_a/drawer/edge/knowledge-facts",
data={
"source_id": "bot_a",
"target_id": "you",
"action": "remove",
"fact": "studied physics together",
},
)
assert response.status_code == 200
with open_db(tmp_path / "test.db") as conn:
from chat.state.edges import get_edge
edge = get_edge(conn, "bot_a", "you")
assert "studied physics together" not in edge["knowledge"]
rows = conn.execute(
"SELECT payload_json FROM event_log WHERE kind = 'manual_edit'"
).fetchall()
payload = json.loads(rows[0][0])
assert payload["target_kind"] == "edge_knowledge_fact"
assert payload["new_value"]["action"] == "remove"
def test_edit_edge_knowledge_facts_400_on_bad_action(client, tmp_path):
_seed(tmp_path / "test.db")
response = client.post(
"/chats/chat_bot_a/drawer/edge/knowledge-facts",
data={
"source_id": "bot_a",
"target_id": "you",
"action": "delete",
"fact": "x",
},
)
assert response.status_code == 400
# --- T72.3 tests (witness flag inline-edit) -------------------------------
def test_witness_flag_toggle_updates_memory_row(client, tmp_path):
"""Memory seeded with witness [you=1, host=1, guest=0]; toggling
``guest`` to 1 lands as ``witness_guest = 1`` after projection.
"""
_seed(tmp_path / "test.db")
with open_db(tmp_path / "test.db") as conn:
memory_id = conn.execute("SELECT id FROM memories LIMIT 1").fetchone()[0]
response = client.post(
"/chats/chat_bot_a/drawer/memory/witness",
data={
"memory_id": str(memory_id),
"flag": "guest",
"new_value": "1",
},
)
assert response.status_code == 200
with open_db(tmp_path / "test.db") as conn:
row = conn.execute(
"SELECT witness_you, witness_host, witness_guest "
"FROM memories WHERE id = ?",
(memory_id,),
).fetchone()
assert row == (1, 1, 1)
def test_witness_flag_toggle_emits_manual_edit_event(client, tmp_path):
_seed(tmp_path / "test.db")
with open_db(tmp_path / "test.db") as conn:
memory_id = conn.execute("SELECT id FROM memories LIMIT 1").fetchone()[0]
response = client.post(
"/chats/chat_bot_a/drawer/memory/witness",
data={
"memory_id": str(memory_id),
"flag": "guest",
"new_value": "1",
},
)
assert response.status_code == 200
with open_db(tmp_path / "test.db") as conn:
rows = conn.execute(
"SELECT payload_json FROM event_log WHERE kind = 'manual_edit'"
).fetchall()
assert len(rows) == 1
payload = json.loads(rows[0][0])
assert payload["target_kind"] == "memory_witness"
assert payload["target_id"] == memory_id
assert payload["prior_value"] == {"flag": "guest", "value": 0}
assert payload["new_value"] == {"flag": "guest", "value": 1}
def test_witness_flag_toggle_400_on_bad_flag(client, tmp_path):
_seed(tmp_path / "test.db")
with open_db(tmp_path / "test.db") as conn:
memory_id = conn.execute("SELECT id FROM memories LIMIT 1").fetchone()[0]
response = client.post(
"/chats/chat_bot_a/drawer/memory/witness",
data={
"memory_id": str(memory_id),
"flag": "narrator",
"new_value": "1",
},
)
assert response.status_code == 400
+478
View File
@@ -0,0 +1,478 @@
"""T42: drawer guest add/remove + render.
The drawer grows a "Guest" section (when a guest bot is present in the
chat), a "Group" section sourced from the ``group_node`` row, an
"Add guest" form (visible while no guest is present), and a "Remove
guest" button (visible while one is). The two new POST endpoints emit
``guest_added`` / ``guest_removed`` events plus ancillary updates:
* ``POST /chats/{chat_id}/drawer/guest/add`` runs the relationship-seed
classifier (T38) over the user-supplied prose and emits an
``edge_update`` per direction when the seed comes back non-default.
It also seeds a ``group_node_initialized`` row when none exists yet.
* ``POST /chats/{chat_id}/drawer/guest/remove`` first emits
``scene_closed`` for the active scene so the host -> you scene closes
cleanly before the guest leaves.
"""
from __future__ import annotations
import json
from pathlib import Path
import pytest
from fastapi.testclient import TestClient
from chat.app import app
from chat.db.connection import open_db
from chat.eventlog.log import append_event
from chat.eventlog.projector import project
from chat.llm.mock import MockLLMClient
@pytest.fixture
def client(tmp_path, monkeypatch):
cfg = tmp_path / "config.toml"
cfg.write_text('featherless_api_key = "test"\n')
monkeypatch.setenv("CHAT_CONFIG_PATH", str(cfg))
db = tmp_path / "test.db"
monkeypatch.setenv("CHAT_DB_PATH", str(db))
with TestClient(app) as c:
if hasattr(app.state, "background_worker"):
app.state.background_worker.enabled = False
yield c
def _bot_payload(bot_id: str, name: str) -> dict:
return {
"id": bot_id,
"name": name,
"persona": "...",
"voice_samples": [],
"traits": [],
"backstory": "",
"initial_relationship_to_you": "",
"kickoff_prose": "",
}
def _seed_chat(db: Path, *, with_scene: bool = True) -> None:
"""Seed a chat hosted by ``bot_a`` (with ``bot_b`` authored as a
candidate guest) and, by default, an open scene so the
``guest_removed`` flow has something to close.
"""
with open_db(db) as conn:
append_event(conn, kind="bot_authored", payload=_bot_payload("bot_a", "BotA"))
append_event(conn, kind="bot_authored", payload=_bot_payload("bot_b", "BotB"))
append_event(
conn,
kind="you_authored",
payload={"name": "Me", "pronouns": "they/them", "persona": ""},
)
append_event(
conn,
kind="chat_created",
payload={
"id": "chat_bot_a",
"host_bot_id": "bot_a",
"initial_time": "2026-04-26T20:00:00+00:00",
"narrative_anchor": "Day 1",
"weather": "",
},
)
if with_scene:
append_event(
conn,
kind="scene_opened",
payload={
"chat_id": "chat_bot_a",
"container_id": None,
"started_at": "2026-04-26T20:00:00+00:00",
"participants": ["you", "bot_a"],
},
)
project(conn)
def _override_llm(canned: list[str]):
"""Wire a ``MockLLMClient`` into the drawer's LLM dependency."""
from chat.web.kickoff import get_llm_client
app.dependency_overrides[get_llm_client] = lambda: MockLLMClient(
canned=list(canned)
)
def test_drawer_no_guest_omits_guest_section(client, tmp_path):
_seed_chat(tmp_path / "test.db")
response = client.get("/chats/chat_bot_a/drawer")
assert response.status_code == 200
body = response.text
# No guest-section header; the "Add guest" form should be visible instead.
assert "<h3>Guest</h3>" not in body
assert "Add guest" in body
def test_drawer_add_guest_seeds_edges_and_group_node(client, tmp_path):
_seed_chat(tmp_path / "test.db")
canned = json.dumps(
{
"a_to_b_summary": "old college friend",
"a_to_b_knowledge_facts": ["studied physics together"],
"a_to_b_affinity_delta": 4,
"a_to_b_trust_delta": -1,
"b_to_a_summary": "former roommate",
"b_to_a_knowledge_facts": ["lived together junior year"],
"b_to_a_affinity_delta": 3,
"b_to_a_trust_delta": 0,
}
)
_override_llm([canned])
try:
response = client.post(
"/chats/chat_bot_a/drawer/guest/add",
data={
"guest_bot_id": "bot_b",
"relationship_prose": (
"Alice and Bob met in college and studied physics together."
),
},
)
assert response.status_code == 200
finally:
app.dependency_overrides.clear()
with open_db(tmp_path / "test.db") as conn:
from chat.state.edges import get_edge
from chat.state.group_node import get_group_node
from chat.state.world import get_chat
chat = get_chat(conn, "chat_bot_a")
assert chat["guest_bot_id"] == "bot_b"
edge_a_to_b = get_edge(conn, "bot_a", "bot_b")
edge_b_to_a = get_edge(conn, "bot_b", "bot_a")
# Seed deltas applied around the 50/50 default.
assert edge_a_to_b["affinity"] == 54
assert edge_a_to_b["trust"] == 49
assert "studied physics together" in edge_a_to_b["knowledge"]
assert edge_b_to_a["affinity"] == 53
assert edge_b_to_a["trust"] == 50
assert "lived together junior year" in edge_b_to_a["knowledge"]
group = get_group_node(conn, "chat_bot_a")
assert group is not None
assert set(group["members"]) == {"you", "bot_a", "bot_b"}
def test_drawer_add_guest_empty_prose_skips_edge_update(client, tmp_path):
_seed_chat(tmp_path / "test.db")
# No canned responses: the seed function short-circuits on empty prose
# so no LLM call should happen.
_override_llm([])
try:
response = client.post(
"/chats/chat_bot_a/drawer/guest/add",
data={"guest_bot_id": "bot_b", "relationship_prose": " "},
)
assert response.status_code == 200
finally:
app.dependency_overrides.clear()
with open_db(tmp_path / "test.db") as conn:
from chat.state.world import get_chat
chat = get_chat(conn, "chat_bot_a")
assert chat["guest_bot_id"] == "bot_b"
# guest_added fires but no edge_update events between bot_a and bot_b.
added = conn.execute(
"SELECT COUNT(*) FROM event_log WHERE kind = 'guest_added'"
).fetchone()[0]
assert added == 1
edge_updates = conn.execute(
"SELECT payload_json FROM event_log WHERE kind = 'edge_update'"
).fetchall()
for (payload_json,) in edge_updates:
payload = json.loads(payload_json)
pair = {payload.get("source_id"), payload.get("target_id")}
assert pair != {"bot_a", "bot_b"}, (
"no edge_update should be emitted between host and guest "
"when prose is empty"
)
def test_drawer_add_guest_when_already_present_returns_400(client, tmp_path):
_seed_chat(tmp_path / "test.db")
# Pre-attach a guest directly via append_and_apply so we don't replay
# the prior chat_created (which would violate UNIQUE on chats.id).
from chat.eventlog.log import append_and_apply
with open_db(tmp_path / "test.db") as conn:
append_and_apply(
conn,
kind="bot_authored",
payload=_bot_payload("bot_c", "BotC"),
)
append_and_apply(
conn,
kind="guest_added",
payload={"chat_id": "chat_bot_a", "guest_bot_id": "bot_b"},
)
_override_llm([])
try:
response = client.post(
"/chats/chat_bot_a/drawer/guest/add",
data={"guest_bot_id": "bot_c", "relationship_prose": ""},
)
assert response.status_code == 400
finally:
app.dependency_overrides.clear()
def test_drawer_remove_guest_clears_and_closes_scene(client, tmp_path):
_seed_chat(tmp_path / "test.db")
from chat.eventlog.log import append_and_apply
with open_db(tmp_path / "test.db") as conn:
append_and_apply(
conn,
kind="guest_added",
payload={"chat_id": "chat_bot_a", "guest_bot_id": "bot_b"},
)
response = client.post("/chats/chat_bot_a/drawer/guest/remove")
assert response.status_code == 200
with open_db(tmp_path / "test.db") as conn:
from chat.state.world import active_scene, get_chat
chat = get_chat(conn, "chat_bot_a")
assert chat["guest_bot_id"] is None
assert active_scene(conn, "chat_bot_a") is None
kinds = [
row[0]
for row in conn.execute(
"SELECT kind FROM event_log ORDER BY id"
).fetchall()
]
# scene_closed must precede guest_removed in the log.
assert "scene_closed" in kinds
assert "guest_removed" in kinds
assert kinds.index("scene_closed") < kinds.index("guest_removed")
# --- T72.2 first-meeting gate ----------------------------------------------
def _seed_host_to_guest_edge(db: Path) -> None:
"""Materialise a bot_a -> bot_b edge so the gate's check fires."""
from chat.eventlog.log import append_and_apply
with open_db(db) as conn:
append_and_apply(
conn,
kind="edge_update",
payload={
"source_id": "bot_a",
"target_id": "bot_b",
"chat_id": "chat_bot_a",
"affinity_delta": 0,
"knowledge_facts": ["already met before"],
},
)
def test_add_guest_form_disables_prose_when_edge_exists(client, tmp_path):
"""When host->candidate edge already exists, the GET partial renders
the textarea disabled and surfaces the "already know each other"
message so the user knows submitting will skip the seed.
"""
_seed_chat(tmp_path / "test.db")
_seed_host_to_guest_edge(tmp_path / "test.db")
response = client.get("/chats/chat_bot_a/drawer")
assert response.status_code == 200
body = response.text
# Note + disabled state both present. The textarea sits next to the
# ``add-guest-prose`` class so we can match it specifically.
assert "already know each other" in body
assert 'class="add-guest-prose"' in body
# The textarea for the first (auto-selected) candidate should be
# disabled in the initial markup since an edge exists.
assert "disabled" in body.split('class="add-guest-prose"', 1)[1].split(">", 1)[0]
# And the option carries the ``data-existing-edge="true"`` attribute
# the inline JS uses to flip state on subsequent select changes.
assert 'data-existing-edge="true"' in body
def test_add_guest_with_existing_edge_skips_seed_call(client, tmp_path):
"""Submitting the Add-guest form WITHOUT toggling re-seed must skip
``seed_inter_bot_edges`` entirely. We assert this via an empty mock
queue: if the seed function had been called it would have consumed
a canned response (or raised because none was available).
"""
_seed_chat(tmp_path / "test.db")
_seed_host_to_guest_edge(tmp_path / "test.db")
# Empty queue: any classifier call would raise inside MockLLMClient.
canned_queue: list[str] = []
_override_llm(canned_queue)
try:
response = client.post(
"/chats/chat_bot_a/drawer/guest/add",
data={
"guest_bot_id": "bot_b",
"relationship_prose": "ignored prose",
# NO reseed flag — gate should suppress the seed call.
},
)
assert response.status_code == 200
finally:
app.dependency_overrides.clear()
with open_db(tmp_path / "test.db") as conn:
from chat.state.edges import get_edge
from chat.state.world import get_chat
chat = get_chat(conn, "chat_bot_a")
assert chat["guest_bot_id"] == "bot_b"
# The pre-seeded knowledge fact survives — proof the seed didn't run
# and overwrite the existing edge.
edge = get_edge(conn, "bot_a", "bot_b")
assert "already met before" in edge["knowledge"]
# Exactly one guest_added; no new edge_update events between
# bot_a and bot_b (the pre-seed edge_update from the test setup
# is the only edge_update on this pair).
added = conn.execute(
"SELECT COUNT(*) FROM event_log WHERE kind = 'guest_added'"
).fetchone()[0]
assert added == 1
edge_updates = conn.execute(
"SELECT payload_json FROM event_log WHERE kind = 'edge_update'"
).fetchall()
# Only the pre-seed edge_update from _seed_host_to_guest_edge.
ab_updates = [
json.loads(p[0])
for p in edge_updates
if {
json.loads(p[0]).get("source_id"),
json.loads(p[0]).get("target_id"),
}
== {"bot_a", "bot_b"}
]
assert len(ab_updates) == 1
assert ab_updates[0]["knowledge_facts"] == ["already met before"]
def test_add_guest_with_existing_edge_and_reseed_runs_seed(client, tmp_path):
"""Toggling ``re-seed anyway`` flips the gate off — the existing
flow runs (seed produces deltas, two ``edge_update`` events fire).
"""
_seed_chat(tmp_path / "test.db")
_seed_host_to_guest_edge(tmp_path / "test.db")
canned = json.dumps(
{
"a_to_b_summary": "reconnected",
"a_to_b_knowledge_facts": ["new fact"],
"a_to_b_affinity_delta": 2,
"a_to_b_trust_delta": 1,
"b_to_a_summary": "reconnected",
"b_to_a_knowledge_facts": [],
"b_to_a_affinity_delta": 1,
"b_to_a_trust_delta": 0,
}
)
_override_llm([canned])
try:
response = client.post(
"/chats/chat_bot_a/drawer/guest/add",
data={
"guest_bot_id": "bot_b",
"relationship_prose": "fresh prose",
"reseed": "1",
},
)
assert response.status_code == 200
finally:
app.dependency_overrides.clear()
with open_db(tmp_path / "test.db") as conn:
edge_updates = conn.execute(
"SELECT payload_json FROM event_log WHERE kind = 'edge_update'"
).fetchall()
# Pre-seed (1) + two from the re-seed = 3 edge_updates total.
ab_updates = [
json.loads(p[0])
for p in edge_updates
if {
json.loads(p[0]).get("source_id"),
json.loads(p[0]).get("target_id"),
}
== {"bot_a", "bot_b"}
]
assert len(ab_updates) == 3
def test_drawer_with_guest_renders_guest_and_group_sections(client, tmp_path):
_seed_chat(tmp_path / "test.db")
from chat.eventlog.log import append_and_apply
with open_db(tmp_path / "test.db") as conn:
append_and_apply(
conn,
kind="guest_added",
payload={"chat_id": "chat_bot_a", "guest_bot_id": "bot_b"},
)
# Activity for the guest so the section has content to render.
append_and_apply(
conn,
kind="activity_change",
payload={
"entity_id": "bot_b",
"posture": "leaning",
"action": {"verb": "smirking"},
"attention": "BotA",
},
)
# Edges in all four directions involving the guest.
for src, tgt in (("bot_a", "bot_b"), ("bot_b", "bot_a"), ("you", "bot_b"), ("bot_b", "you")):
append_and_apply(
conn,
kind="edge_update",
payload={
"source_id": src,
"target_id": tgt,
"chat_id": "chat_bot_a",
"affinity_delta": 1,
},
)
append_and_apply(
conn,
kind="group_node_initialized",
payload={
"chat_id": "chat_bot_a",
"members": ["you", "bot_a", "bot_b"],
"summary": "Three friends catching up over drinks.",
"dynamic": "warm and conspiratorial",
},
)
response = client.get("/chats/chat_bot_a/drawer")
assert response.status_code == 200
body = response.text
assert "<h3>Guest</h3>" in body
assert "BotB" in body
assert "smirking" in body
assert "<h3>Group</h3>" in body
assert "Three friends catching up over drinks." in body
assert "warm and conspiratorial" in body
# "Remove guest" button is visible when a guest is present.
assert "Remove guest" in body
+210
View File
@@ -0,0 +1,210 @@
from __future__ import annotations
from pathlib import Path
import pytest
from fastapi.testclient import TestClient
from chat.app import app
from chat.db.connection import open_db
from chat.eventlog.log import append_event
from chat.eventlog.projector import project
@pytest.fixture
def client(tmp_path, monkeypatch):
cfg = tmp_path / "config.toml"
cfg.write_text('featherless_api_key = "test"\n')
monkeypatch.setenv("CHAT_CONFIG_PATH", str(cfg))
db = tmp_path / "test.db"
monkeypatch.setenv("CHAT_DB_PATH", str(db))
with TestClient(app) as c:
# Disable background worker (we won't drive turns).
if hasattr(app.state, "background_worker"):
app.state.background_worker.enabled = False
yield c
def _seed(db: Path) -> None:
with open_db(db) as conn:
append_event(
conn,
kind="bot_authored",
payload={
"id": "bot_a",
"name": "BotA",
"persona": "...",
"voice_samples": [],
"traits": ["shy"],
"backstory": "",
"initial_relationship_to_you": "",
"kickoff_prose": "",
},
)
append_event(
conn,
kind="you_authored",
payload={
"name": "Me",
"pronouns": "they/them",
"persona": "engineer",
},
)
append_event(
conn,
kind="chat_created",
payload={
"id": "chat_bot_a",
"host_bot_id": "bot_a",
"initial_time": "2026-04-26T20:00:00+00:00",
"narrative_anchor": "Day 1",
"weather": "",
},
)
# Activity for both you and host bot.
append_event(
conn,
kind="activity_change",
payload={
"entity_id": "you",
"posture": "sitting",
"action": {"verb": "thinking"},
"attention": "the screen",
},
)
append_event(
conn,
kind="activity_change",
payload={
"entity_id": "bot_a",
"posture": "standing",
"action": {"verb": "looking out the window"},
},
)
# Edge host -> you.
append_event(
conn,
kind="edge_update",
payload={
"source_id": "bot_a",
"target_id": "you",
"chat_id": "chat_bot_a",
"affinity_delta": 5,
"trust_delta": 2,
"knowledge_facts": ["Me likes coffee"],
},
)
# A regular memory.
append_event(
conn,
kind="memory_written",
payload={
"owner_id": "bot_a",
"chat_id": "chat_bot_a",
"pov_summary": "Talked about her sister",
"witness_you": 1,
"witness_host": 1,
"witness_guest": 0,
"significance": 2,
},
)
# A pinned memory with significance 3 (★★).
append_event(
conn,
kind="memory_written",
payload={
"owner_id": "bot_a",
"chat_id": "chat_bot_a",
"pov_summary": "First kiss",
"witness_you": 1,
"witness_host": 1,
"witness_guest": 0,
"significance": 3,
"pinned": 1,
"auto_pinned": 1,
},
)
project(conn)
def test_drawer_404_when_chat_missing(client):
response = client.get("/chats/no_such/drawer")
assert response.status_code == 404
def test_drawer_renders_scene_and_activity(client, tmp_path):
_seed(tmp_path / "test.db")
response = client.get("/chats/chat_bot_a/drawer")
assert response.status_code == 200
body = response.text
# Scene/time anchor.
assert "2026-04-26" in body
# Activity verbs from both entities.
assert "thinking" in body
assert "looking out the window" in body
# Activity attention.
assert "the screen" in body
def test_drawer_renders_edges(client, tmp_path):
_seed(tmp_path / "test.db")
response = client.get("/chats/chat_bot_a/drawer")
assert response.status_code == 200
body = response.text
assert "BotA" in body
assert "you" in body
# Default affinity 50 + delta 5 = 55.
assert "55" in body
# Knowledge fact appears.
assert "Me likes coffee" in body
def test_drawer_renders_memories_with_significance_markers(client, tmp_path):
_seed(tmp_path / "test.db")
response = client.get("/chats/chat_bot_a/drawer")
assert response.status_code == 200
body = response.text
assert "Talked about her sister" in body
assert "First kiss" in body
# Pinned counter shows 1 / 8 (or 1/8).
assert "1 / 8" in body or "1/8" in body
# Significance star marker for the pinned, score-3 memory.
assert "" in body
def test_drawer_handles_no_state_gracefully(client, tmp_path):
db = tmp_path / "test.db"
with open_db(db) as conn:
# Just enough state for the chat to exist.
append_event(
conn,
kind="bot_authored",
payload={
"id": "bot_a",
"name": "BotA",
"persona": "...",
"voice_samples": [],
"traits": [],
"backstory": "",
"initial_relationship_to_you": "",
"kickoff_prose": "",
},
)
append_event(
conn,
kind="chat_created",
payload={
"id": "chat_bot_a",
"host_bot_id": "bot_a",
"initial_time": "2026-04-26T20:00:00+00:00",
"narrative_anchor": "Day 1",
"weather": "",
},
)
project(conn)
response = client.get("/chats/chat_bot_a/drawer")
assert response.status_code == 200
body = response.text
# Drawer renders gracefully with empty placeholders.
assert "No active container" in body or "Container:" not in body
assert "No edges yet" in body or "Edges" in body
+162
View File
@@ -0,0 +1,162 @@
from chat.db.migrate import apply_migrations
from chat.db.connection import open_db
from chat.eventlog.log import append_event
from chat.eventlog.projector import project
from chat.state.edges import get_edge, list_edges_for
import chat.state.entities # registers bot/you handlers
import chat.state.edges # registers edge_update handler
def _seed_entities(conn) -> None:
append_event(conn, kind="bot_authored", payload={
"id": "bot_a", "name": "BotA", "persona": "p",
"voice_samples": [], "traits": [],
"backstory": "", "initial_relationship_to_you": "",
"kickoff_prose": "",
})
append_event(conn, kind="you_authored", payload={
"name": "Me", "pronouns": "", "persona": "",
})
def test_edge_update_upsert_applies_first_delta(tmp_path):
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
_seed_entities(conn)
append_event(conn, kind="edge_update", payload={
"source_id": "bot_a", "target_id": "you",
"affinity_delta": 5,
})
project(conn)
edge = get_edge(conn, "bot_a", "you")
assert edge is not None
assert edge["affinity"] == 55
assert edge["trust"] == 50
assert edge["knowledge"] == []
assert edge["summary"] == ""
def test_edge_update_multiple_deltas_accumulate(tmp_path):
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
_seed_entities(conn)
append_event(conn, kind="edge_update", payload={
"source_id": "bot_a", "target_id": "you",
"affinity_delta": 5,
})
append_event(conn, kind="edge_update", payload={
"source_id": "bot_a", "target_id": "you",
"affinity_delta": -3,
"trust_delta": 2,
"knowledge_facts": ["she has a sister"],
})
project(conn)
edge = get_edge(conn, "bot_a", "you")
assert edge is not None
assert edge["affinity"] == 52
assert edge["trust"] == 52
assert edge["knowledge"] == ["she has a sister"]
def test_edge_update_clamps_affinity_at_max(tmp_path):
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
_seed_entities(conn)
append_event(conn, kind="edge_update", payload={
"source_id": "bot_a", "target_id": "you",
"affinity_delta": 5,
})
append_event(conn, kind="edge_update", payload={
"source_id": "bot_a", "target_id": "you",
"affinity_delta": 100,
})
project(conn)
edge = get_edge(conn, "bot_a", "you")
assert edge is not None
assert edge["affinity"] == 100
def test_edge_update_clamps_trust_at_min(tmp_path):
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
_seed_entities(conn)
append_event(conn, kind="edge_update", payload={
"source_id": "bot_a", "target_id": "you",
"trust_delta": -200,
})
project(conn)
edge = get_edge(conn, "bot_a", "you")
assert edge is not None
assert edge["trust"] == 0
def test_edges_are_directed_and_asymmetric(tmp_path):
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
_seed_entities(conn)
append_event(conn, kind="edge_update", payload={
"source_id": "bot_a", "target_id": "you",
"affinity_delta": 5,
})
append_event(conn, kind="edge_update", payload={
"source_id": "you", "target_id": "bot_a",
"affinity_delta": 10,
})
project(conn)
forward = get_edge(conn, "bot_a", "you")
reverse = get_edge(conn, "you", "bot_a")
assert forward is not None and reverse is not None
assert forward["affinity"] == 55
assert reverse["affinity"] == 60
# Independent rows
assert forward["affinity"] != reverse["affinity"]
def test_edge_update_bumps_last_interaction(tmp_path):
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
_seed_entities(conn)
append_event(conn, kind="edge_update", payload={
"source_id": "bot_a", "target_id": "you",
"affinity_delta": 1,
"last_interaction_at": "2026-04-26T10:00:00",
"last_interaction_chat_id": "chat_bot_a",
})
project(conn)
edge = get_edge(conn, "bot_a", "you")
assert edge is not None
assert edge["last_interaction_at"] == "2026-04-26T10:00:00"
assert edge["last_interaction_chat_id"] == "chat_bot_a"
def test_list_edges_for_returns_outgoing_only(tmp_path):
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
_seed_entities(conn)
append_event(conn, kind="bot_authored", payload={
"id": "bot_b", "name": "BotB", "persona": "p",
"voice_samples": [], "traits": [],
"backstory": "", "initial_relationship_to_you": "",
"kickoff_prose": "",
})
append_event(conn, kind="edge_update", payload={
"source_id": "bot_a", "target_id": "you", "affinity_delta": 1,
})
append_event(conn, kind="edge_update", payload={
"source_id": "bot_a", "target_id": "bot_b", "affinity_delta": 2,
})
append_event(conn, kind="edge_update", payload={
"source_id": "bot_b", "target_id": "bot_a", "affinity_delta": 3,
})
project(conn)
outgoing = list_edges_for(conn, "bot_a")
targets = [e["target_id"] for e in outgoing]
assert targets == sorted(targets)
assert set(targets) == {"you", "bot_b"}
+38
View File
@@ -0,0 +1,38 @@
from chat.db.migrate import apply_migrations
from chat.db.connection import open_db
from chat.eventlog.log import append_event
from chat.eventlog.projector import project
from chat.state.entities import get_bot, list_bots, get_you
import chat.state.entities # registers handlers
def test_bot_authored_creates_bot_row(tmp_path):
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
append_event(conn, kind="bot_authored", payload={
"id": "bot_a", "name": "BotA",
"persona": "...", "voice_samples": ["sample"], "traits": ["shy"],
"backstory": "...",
"initial_relationship_to_you": "coworker",
"kickoff_prose": "you stay late",
})
project(conn)
bot = get_bot(conn, "bot_a")
assert bot is not None
assert bot["name"] == "BotA"
assert bot["traits"] == ["shy"]
assert "bot_a" in [b["id"] for b in list_bots(conn)]
def test_you_authored_creates_you_singleton(tmp_path):
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
append_event(conn, kind="you_authored", payload={
"name": "Me", "pronouns": "they/them", "persona": "engineer",
})
project(conn)
you = get_you(conn)
assert you is not None
assert you["name"] == "Me"
+57
View File
@@ -0,0 +1,57 @@
from __future__ import annotations
import pytest
from fastapi.testclient import TestClient
from chat.app import app
@pytest.fixture
def client(tmp_path, monkeypatch):
cfg = tmp_path / "config.toml"
cfg.write_text('featherless_api_key = "test"\n')
monkeypatch.setenv("CHAT_CONFIG_PATH", str(cfg))
db = tmp_path / "test.db"
monkeypatch.setenv("CHAT_DB_PATH", str(db))
with TestClient(app) as c:
if hasattr(app.state, "background_worker"):
app.state.background_worker.enabled = False
yield c
def _setup_minimal_state(db_path):
"""Set up enough state so the first-run middleware doesn't redirect."""
from chat.db.connection import open_db
from chat.eventlog.log import append_event
from chat.eventlog.projector import project
with open_db(db_path) as conn:
append_event(
conn,
kind="you_authored",
payload={"name": "Me", "pronouns": "", "persona": ""},
)
append_event(
conn,
kind="bot_authored",
payload={
"id": "bot_a",
"name": "BotA",
"persona": "",
"voice_samples": [],
"traits": [],
"backstory": "",
"initial_relationship_to_you": "",
"kickoff_prose": "",
},
)
project(conn)
def test_404_renders_friendly_page_for_html(client, tmp_path):
_setup_minimal_state(tmp_path / "test.db")
response = client.get("/chats/no_such_chat")
assert response.status_code == 404
body = response.text
assert "404" in body
assert "back to" in body.lower()

Some files were not shown because too many files have changed in this diff Show More