Phase 3: events, time skips, threads, meanwhile scenes #4

2026-04-26T21:19:52-04:00

dohertj2 commented

2026-04-26 21:19:52 -04:00

Summary

Phase 3 adds events with full lifecycles, time skips (elision + jump), active threads, significance/retrieval refinements, and meanwhile scenes (host+guest with no "you"). All scoped per-chat; the cross-chat surface is unchanged. Built on Phase 2's event-sourced multi-entity architecture.

What shipped (19 tasks, 8 waves)

Wave 1 (parallel) — schema & state foundation: T49 events table + lifecycle handlers; T50 time_skip event handlers; T51 threads table + handlers
Wave 2 (parallel) — classifier services: T52 event-lifecycle detection; T53 skip narration (elision + jump); T54 synthesized memories for jump skips; T55 thread detection on close
Wave 3 (parallel) — promotion + retrieval: T56 event-completion promotion; T57 significance-aware retrieval ranking; T58 scene compression keeps key quotes ≥ 2 + thread emission on close
Wave 4 (single) — T59 drawer events panel + threads panel + skip controls (5 new POST routes)
Wave 5a (parallel) — T60 prompt assembly renders active events + open threads; T61 turn flow invokes event detection + completion promotion
Wave 5b (single) — T62 natural-language skip command surface (extracts shared skip controllers into chat/web/skip.py)
Wave 6a (single) — T63 meanwhile scene schema + state
Wave 6b (parallel after 6a) — T64 meanwhile turn flow (host+guest, no "you"); T65 meanwhile summary digest
Wave 7 (parallel) — T66 cross-feature integration tests; T67 documentation

Architecture notes

Schema migrations: 0009_events.sql, 0010_threads.sql, 0011_meanwhile_scenes.sql. Schema baseline now version 11.
Backward compatible: when no events/threads/meanwhile-scenes exist, all flows reduce exactly to Phase 2 behavior. All 247 Phase 2 tests pass unmodified.
Featherless 2-connection cap respected throughout.
Three new event-kind families: events (planned/started/completed/cancelled/expired), time_skip (elision/jump), threads (opened/updated/closed). Plus meanwhile_scene_started/closed and meanwhile_digest_created/consumed.

Test plan

Full pytest suite: 315 passing (247 Phase 2 + 68 Phase 3). 0 failures, 0 skips.
Each task verified in its own worktree before merge to phase-3.
Each task reviewed for spec compliance + code quality + security.
Cross-feature integration tests (T66) pin 5 multi-feature scenarios.
Manual smoke (recommend before merging this PR):
- Plan an event from the drawer; play turns until it completes; verify promotion landed (drawer shows updated edges/memories)
- Use elision and jump skips both via natural-language ('skip to when we arrive', 'next morning') and the drawer
- Close a scene that opened a thread; verify the thread renders in the next scene's prompt
- Trigger a meanwhile scene from the drawer; play 2 turns; close it; resume the main you-scene; verify the digest renders once
- Multi-tab regenerate with new `turn_html_replace` SSE event (note: frontend handler still pending — Phase 2.5 backlog)

Phase 3.5 / 4 backlog (tracked in CLAUDE.md)

Captured during reviews:

`narrate_skip` timeout_s not piped through (T53)
search_memories docstring missing SQL bias note (T57)
Scene close re-close suffix bloat risk + thread transcript scoping + swallowed exceptions + closed_at clock divergence + T58 test coverage gaps (T58)
Regenerate doesn't roll back lifecycle transitions from superseded turn + event-detection ordering asymmetry (T61)
Error-message prefix sniff for 404 vs 400 + skip command bypasses scene close (T62)
participants_json JSON-injection audit (T63)
Stop button cancellation route-level coverage + record_meanwhile_memory DRY (T64)
Cross-feature canned-queue brittleness (Wave 6b merge)
`consume_pending_meanwhile_digests` not wired into post_turn (T66) — meanwhile digests stay pending forever; Phase 3.5 should hook it into the first you-turn after meanwhile close
`_witness_role_for` defensive None (carry-over from Phase 2.5 T71)

Plan

`docs/plans/2026-04-26-v3-phase3-implementation.md` (committed in 3790547).

## Summary Phase 3 adds events with full lifecycles, time skips (elision + jump), active threads, significance/retrieval refinements, and meanwhile scenes (host+guest with no \"you\"). All scoped per-chat; the cross-chat surface is unchanged. Built on Phase 2's event-sourced multi-entity architecture. ## What shipped (19 tasks, 8 waves) - **Wave 1 (parallel)** — schema & state foundation: T49 events table + lifecycle handlers; T50 time_skip event handlers; T51 threads table + handlers - **Wave 2 (parallel)** — classifier services: T52 event-lifecycle detection; T53 skip narration (elision + jump); T54 synthesized memories for jump skips; T55 thread detection on close - **Wave 3 (parallel)** — promotion + retrieval: T56 event-completion promotion; T57 significance-aware retrieval ranking; T58 scene compression keeps key quotes ≥ 2 + thread emission on close - **Wave 4 (single)** — T59 drawer events panel + threads panel + skip controls (5 new POST routes) - **Wave 5a (parallel)** — T60 prompt assembly renders active events + open threads; T61 turn flow invokes event detection + completion promotion - **Wave 5b (single)** — T62 natural-language skip command surface (extracts shared skip controllers into chat/web/skip.py) - **Wave 6a (single)** — T63 meanwhile scene schema + state - **Wave 6b (parallel after 6a)** — T64 meanwhile turn flow (host+guest, no \"you\"); T65 meanwhile summary digest - **Wave 7 (parallel)** — T66 cross-feature integration tests; T67 documentation ## Architecture notes - Schema migrations: 0009_events.sql, 0010_threads.sql, 0011_meanwhile_scenes.sql. Schema baseline now version 11. - Backward compatible: when no events/threads/meanwhile-scenes exist, all flows reduce exactly to Phase 2 behavior. All 247 Phase 2 tests pass unmodified. - Featherless 2-connection cap respected throughout. - Three new event-kind families: events (planned/started/completed/cancelled/expired), time_skip (elision/jump), threads (opened/updated/closed). Plus meanwhile_scene_started/closed and meanwhile_digest_created/consumed. ## Test plan - [x] Full pytest suite: 315 passing (247 Phase 2 + 68 Phase 3). 0 failures, 0 skips. - [x] Each task verified in its own worktree before merge to phase-3. - [x] Each task reviewed for spec compliance + code quality + security. - [x] Cross-feature integration tests (T66) pin 5 multi-feature scenarios. - [ ] Manual smoke (recommend before merging this PR): - [ ] Plan an event from the drawer; play turns until it completes; verify promotion landed (drawer shows updated edges/memories) - [ ] Use elision and jump skips both via natural-language ('skip to when we arrive', 'next morning') and the drawer - [ ] Close a scene that opened a thread; verify the thread renders in the next scene's prompt - [ ] Trigger a meanwhile scene from the drawer; play 2 turns; close it; resume the main you-scene; verify the digest renders once - [ ] Multi-tab regenerate with new \`turn_html_replace\` SSE event (note: frontend handler still pending — Phase 2.5 backlog) ## Phase 3.5 / 4 backlog (tracked in CLAUDE.md) Captured during reviews: - \`narrate_skip\` timeout_s not piped through (T53) - search_memories docstring missing SQL bias note (T57) - Scene close re-close suffix bloat risk + thread transcript scoping + swallowed exceptions + closed_at clock divergence + T58 test coverage gaps (T58) - Regenerate doesn't roll back lifecycle transitions from superseded turn + event-detection ordering asymmetry (T61) - Error-message prefix sniff for 404 vs 400 + skip command bypasses scene close (T62) - participants_json JSON-injection audit (T63) - Stop button cancellation route-level coverage + record_meanwhile_memory DRY (T64) - Cross-feature canned-queue brittleness (Wave 6b merge) - **\`consume_pending_meanwhile_digests\` not wired into post_turn (T66)** — meanwhile digests stay pending forever; Phase 3.5 should hook it into the first you-turn after meanwhile close - \`_witness_role_for\` defensive None (carry-over from Phase 2.5 T71) ## Plan \`docs/plans/2026-04-26-v3-phase3-implementation.md\` (committed in 3790547).

dohertj2 added 41 commits 2026-04-26 21:19:53 -04:00

feat: events table + lifecycle handlers (T49) b6888ff36a

feat: time_skip event handlers (T50) ab2b494c21

feat: threads table + projector handlers (T51) 25bcbac055

merge: T49 events table + lifecycle handlers 5b7a195cf5

merge: T50 time_skip event handlers e26885b011

merge: T51 threads table + projector handlers 03ba34272b

test: bump schema_version assertion to 10 (0009 events + 0010 threads) da1f67fb6a

feat: event-lifecycle detection service (T52) 98250644ad

feat: synthesized-memories service for jump skips (T54) adbbd32873

feat: thread-detection service (T55) 7857da4112

feat: skip narration service (T53) c2144cd9df

merge: T52 event-lifecycle detection service afe940259a

merge: T53 skip narration service 2959e1ac2a

merge: T54 synthesized-memories service 959fe11410

merge: T55 thread-detection service a34931375c

feat: significance-aware retrieval ranking (T57) 5e6b29e0c5

feat: event-completion promotion service (T56) 021587b3df

feat: significance-driven quote retention + thread emission on close (T58) 343f305587

merge: T56 event-completion promotion service 88350d7d2e

merge: T57 significance-aware retrieval ranking 8aeadfd0e4

merge: T58 scene compression + thread emission on close 8efbcdf6c3

feat: drawer events / threads / skip controls (T59) 2d14197553

merge: T59 drawer events / threads / skip controls 83f94a4325

feat: prompt assembly renders active events + open threads (T60) 21c4ffa63c

feat: per-turn event-lifecycle detection + completion promotion (T61) b582567521

merge: T60 prompt assembly active events + open threads 3678bcaca6

merge: T61 per-turn event-lifecycle detection + completion promotion e236bcadcd

feat: natural-language skip detection + skip command flow (T62) a7eedb8037

Extend ParsedTurn with intent/landing_state_hint so the classifier can
flag skip-elision and skip-jump prose. The post_turn handler short-
circuits the regular narrative path when intent != "narrative":
elision runs through the shared controller in chat/web/skip.py;
jump returns 422 directing the user to the drawer's structured form
(simpler Phase 3 path — natural-language fiction-time delta parsing
is too fragile for v1 without a structured surface).

Extract the elision/jump logic that previously lived in drawer.py
into chat/web/skip.py so both the drawer T59 routes and the new
natural-language path share one canonical implementation. The drawer
routes become thin HTTP wrappers that translate ValueError to 400
and refresh the drawer partial; the existing drawer skip tests pass
unchanged.

The new natural-language elision derives ``new_time`` by bumping the
chat clock by 1 hour (Phase 3 stub) — the drawer's structured form
remains the path for picking a specific landing time.

merge: T62 natural-language skip command flow + shared skip controllers 819803da84

feat: meanwhile scene schema + state (T63) c463dc70b2

merge: T63 meanwhile scene schema + state c9d58b8229

feat: meanwhile summary digest surfaces to next you-scene (T65) a781732ee6

feat: meanwhile turn flow (host+guest, no you) (T64) cf43ba0993

merge: T64 meanwhile turn flow (host+guest, no you) 2c7aa68af9

merge: T65 meanwhile summary digest surfaces to next you-scene 0cd41636b3

test: feed meanwhile digest canned response after Wave 6b cross-feature merge dc35833534

docs: phase 3 status, behavioral defaults, deferred items (T67) af6c54dd05

test: phase 3 cross-feature integration coverage (T66) f865ac2ee2

merge: T66 phase 3 cross-feature integration coverage c3947bbb68

merge: T67 phase 3 documentation update 6709cf46a7