Commit Graph

228 Commits

Author SHA1 Message Date
Joseph Doherty a06f90a164 docs: add Phase 4.5 cleanup plan (all 24 backlog items)
16 tasks across 9 waves consolidating all 24 items in CLAUDE.md
Phase 4.5/5 backlog. Mix of:

- Wave 1 (parallel 6-way): trivial polish across 6 different files
- Wave 2 (single): schema migration 0014 (FK CASCADE + memories.event_id)
- Wave 3 (single): drawer bundle (event_id guard + html.escape + modal
  partial + bulk significance re-rate)
- Wave 4 (single): search UX (FTS snippet highlight + deep-link)
- Wave 5 (single): real embedding model swap (LLMClient.embed protocol)
- Wave 6 (single): branching read-side filter (riskiest — cross-cutting)
- Wave 7 (single): regenerate lifecycle rollback
- Wave 8 (single): sqlite-vec swap [ENVIRONMENTAL — may defer to Phase 5
  if Python rebuild / apsw not feasible]
- Wave 9 (parallel 3-way): structured fixture builder + integration tests + docs

Schema baseline 13 -> 14 (or 15 with T115). Big tasks (T112 real embed,
T113 branching filter, T114 lifecycle rollback) advance the engine
beyond Phase 4's metadata-only state. T115 environmental decision
captured in pre-flight; the other 13 tasks ship without it.

Uses task ids T103-T118 to avoid collision with prior phases.
2026-04-27 04:22:08 -04:00
dohertj2 df977fc985 Merge pull request 'Phase 4: vector retrieval, branching, drawer polish' (#6) from phase-4 into main 2026-04-27 04:10:25 -04:00
Joseph Doherty 51a12afbec merge: T102 phase 4 documentation update 2026-04-27 04:09:09 -04:00
Joseph Doherty fc3020a0ee merge: T101 phase 4 cross-feature integration tests 2026-04-27 04:09:09 -04:00
Joseph Doherty 228f9abb19 test: phase 4 cross-feature integration coverage (T101) 2026-04-27 04:08:25 -04:00
Joseph Doherty b6119879e5 docs: phase 4 status, behavioral defaults, deferred items (T102) 2026-04-27 03:56:45 -04:00
Joseph Doherty 3b4c7b9cef merge: T100 cross-chat search UX (top-bar + results page) 2026-04-27 03:48:06 -04:00
Joseph Doherty 36d75fa6e7 merge: T99 snapshot UX (manual trigger + list + restore + preview) 2026-04-27 03:48:06 -04:00
Joseph Doherty 0a2c5924f9 feat: cross-chat search UX (top-bar + results page) (T100)
Wires T93's `search_all_memories` service into a small read-only HTML
surface so users can find a memory across every chat in the database.

* `chat/web/search.py` (new): GET `/search?q=...` runs the FTS service
  with k=50, hydrates each row with bot name + scene timestamp, and
  renders `search.html`. Empty `q` short-circuits to no results so the
  top-bar form can submit even with an empty input.
* `chat/templates/search.html` (new): empty-state placeholder, results
  list with chat-level "Open chat" links (`/chats/{chat_id}` — memories
  don't carry an event_id today, so no per-turn anchor).
* `chat/templates/layout.html`: append a small `<form>` to the rail
  nav, additive only.
* `chat/app.py`: register `search_router` (additive import + include).
* `tests/test_search_ux.py`: 3 tests — multi-chat results, empty-query
  placeholder, chat link.
2026-04-27 03:46:52 -04:00
Joseph Doherty a5f0e69d44 feat: snapshot UX (manual trigger + list + restore + preview) (T99) 2026-04-27 03:46:49 -04:00
Joseph Doherty 3dbe1a01ff merge: T98 drawer Phase 4 bundle (branching + sig review + hide + delete + remaining edits) 2026-04-27 03:38:15 -04:00
Joseph Doherty 4546bc0d9c feat: drawer remaining v1 field edits (T98.5)
Audit of chat/state/manual_edit.py target_kind dispatch found two §6.4
fields without drawer affordances despite being already-projected text
columns: chat_state.narrative_anchor and chat_state.weather. Both land
via new manual_edit branches (target_kind chat_narrative_anchor and
chat_weather) plus paired drawer routes and Scene-section text inputs.

The container properties_json blob is intentionally deferred — bounded
JSON edits aren't wired through manual_edit and the drawer never
surfaces multiple containers at once, so v1 leaves it out.
2026-04-27 03:35:54 -04:00
Joseph Doherty c4fa11fe78 feat: drawer surgical delete with cascade preview (T98.4) 2026-04-27 03:29:07 -04:00
Joseph Doherty 461d441078 feat: drawer hide-from-view toggle + turn_hidden manual_edit branch (T98.3) 2026-04-27 03:27:59 -04:00
Joseph Doherty b25007eb44 feat: drawer significance review panel (T98.2) 2026-04-27 03:25:40 -04:00
Joseph Doherty d39d31479d feat: drawer branching UI (T98.1) 2026-04-27 03:24:02 -04:00
Joseph Doherty 7899c50b6c merge: T97 memory write hook + embedding worker + backfill + call-site wiring 2026-04-27 03:09:14 -04:00
Joseph Doherty 177e39d59c feat: wire embedding worker call sites in turns/meanwhile/skip/regenerate (T97.5) 2026-04-27 03:08:36 -04:00
Joseph Doherty d85ed8aaa6 feat: backfill_embeddings script for existing memories (T97.4) 2026-04-27 02:51:48 -04:00
Joseph Doherty 9c63d6b24c feat: app lifespan starts/stops EmbeddingWorker (T97.3) 2026-04-27 02:51:44 -04:00
Joseph Doherty 64a07aa87f feat: memory_write enqueues embedding job after each memory_written (T97.2) 2026-04-27 02:51:40 -04:00
Joseph Doherty 6674f9475c feat: embedding worker drains queue and emits embedding_indexed events (T97.1) 2026-04-27 02:51:36 -04:00
Joseph Doherty 50448b72f8 merge: T96 combined FTS + vector retrieval ranking via RRF 2026-04-27 02:44:03 -04:00
Joseph Doherty b8b4aed6d9 feat: combined FTS + vector retrieval ranking via RRF (T96) 2026-04-27 02:42:38 -04:00
Joseph Doherty 5ff107574c merge: T95 delete-impact computation service 2026-04-27 02:37:28 -04:00
Joseph Doherty 915d625d7f merge: T94 branching service 2026-04-27 02:37:28 -04:00
Joseph Doherty 28e13d416f feat: delete-impact computation service (preview without mutation) (T95) 2026-04-27 02:36:30 -04:00
Joseph Doherty 296e8fdddd feat: branching service (branch_from_event + switch + metadata) (T94) 2026-04-27 02:35:58 -04:00
Joseph Doherty 013b563f21 merge: T93 cross-chat search service 2026-04-27 02:32:53 -04:00
Joseph Doherty 62d5cdd826 merge: T92 pure-Python cosine vector search service 2026-04-27 02:32:53 -04:00
Joseph Doherty a25c166174 merge: T91 embedding generation service (pseudo-embedding) 2026-04-27 02:32:53 -04:00
Joseph Doherty 8f66e1123a feat: cross-chat search service (T93) 2026-04-27 02:31:31 -04:00
Joseph Doherty caa17b4174 feat: embedding generation service (Phase 4 pseudo-embedding) (T91) 2026-04-27 02:31:07 -04:00
Joseph Doherty c7cb0eb01e feat: pure-Python cosine vector search service (T92) 2026-04-27 02:31:06 -04:00
Joseph Doherty 1d6768e980 test: bump schema_version assertion to 13 (0012 embeddings + 0013 branches) 2026-04-27 02:28:11 -04:00
Joseph Doherty 8b086d4bb8 merge: T90 phase 3.6 carry-overs trio 2026-04-27 02:27:48 -04:00
Joseph Doherty 6c7ac8f69f merge: T89 branches table + projector handlers 2026-04-27 02:27:48 -04:00
Joseph Doherty fe34d4f4c0 merge: T88 embeddings table + projector handlers 2026-04-27 02:27:48 -04:00
Joseph Doherty 0d76a6b2d6 refactor: consolidate legacy record_turn_memory into unified API (T90.3)
The Phase 1 single-bot ``record_turn_memory`` lingered next to the
unified ``record_turn_memory_for_present`` introduced in T84. Only test
fixtures still called the legacy entry point.

- Remove ``record_turn_memory`` from ``chat/services/memory_write.py``.
- Update the two test_memory_write.py callers to use
  ``record_turn_memory_for_present(..., guest_bot_id=None)``, which
  produces the same ``[you=1, host=1, guest=0]`` witness mask.

The unified API returns ``dict[bot_id, (event_id, memory_id)]``; tests
extract the host entry. No production callers were affected.
2026-04-27 02:25:07 -04:00
Joseph Doherty cc71fb4d01 chore: clarify regenerate lifecycle warning wording (T90.2)
The warning said "lifecycle transitions from superseded turn ARE NOT
being rolled back". When regenerating an OLDER turn, the listed
transitions can include intervening-turn ones that legitimately stand
on their own — they weren't authored by the superseded turn itself.

Reword to "lifecycle transitions at-or-after turn <id>" so operators
reading logs aren't misled into thinking every listed event id was
emitted by the target turn. Cosmetic change to a single log message.

Test: extends test_regenerate_with_prior_lifecycle_logs_warning to
assert the new phrasing is present and the old phrasing is gone.
2026-04-27 02:23:55 -04:00
Joseph Doherty c06a32767b perf: read_recent_dialogue pushes chat-id filter into SQL (T90.1)
The previous implementation pulled the last N rows in SQL across all
chats and dropped foreign-chat rows in Python. With LIMIT N this could
return far fewer than N relevant rows when other chats had recent
activity. Push the chat_id filter into SQL via json_extract so LIMIT N
always returns N rows scoped to the requested chat.

Test: seeds two chats with 60 turns each interleaved; queries chat_a
with limit=50; asserts exactly 50 chat_a rows returned (was 0 prior to
the fix because chat_b's rows dominated the global tail).
2026-04-27 02:23:15 -04:00
Joseph Doherty 0ba374b790 feat: embeddings table + projector handlers (pure-Python cosine, T88) 2026-04-27 02:22:32 -04:00
Joseph Doherty 77f1636086 feat: branches table + projector handlers (T89) 2026-04-27 02:22:27 -04:00
Joseph Doherty bffd9a2f38 docs: add Phase 4 implementation plan (vector retrieval + branching + polish)
15 tasks across 8 waves landing the Phase 4 deliverables per
requirements doc §13 + §14:

- Vector retrieval via sqlite-vec (new external dependency)
- Branching UI (event log forks)
- Drawer-edit on every field (significance review, hide-from-view,
  surgical delete with cascade preview, branching affordances)
- Backup tooling (snapshot UX surface)
- Cross-chat search

Plus the 3 Phase 3.6 carry-over fixes (T90 bundle).

Wave structure:
- W1 (parallel 3-way): schema foundation + carry-overs
- W2 (parallel 3-way): embedding/search services
- W3 (parallel 2-way): branching + delete services
- W4 (single): combined retrieval ranking
- W5 (single): memory write hook + backfill
- W6 (single): drawer Phase 4 bundle (5 sub-features)
- W7 (parallel 2-way): snapshot UX + cross-chat search UX
- W8 (parallel 2-way): integration tests + docs

External dependency: sqlite-vec must be installed BEFORE Wave 1.
Embedding model choice (384-dim default) pinned in T91 before dispatch
since the migration hardcodes the dimension.

Schema baseline: 11 -> 13 (adds 0012_embeddings.sql + 0013_branches.sql).
Task ids T88-T102 to avoid collision with prior phases.
2026-04-27 02:03:08 -04:00
dohertj2 1b66a2821c Merge pull request 'Phase 3.5 cleanup: 17-item backlog burndown' (#5) from phase-3.5 into main 2026-04-27 01:56:28 -04:00
Joseph Doherty 74bb42397d merge: T87 phase 3.5 docs sweep — prune shipped backlog, capture phase 3.6 residuals 2026-04-26 22:46:26 -04:00
Joseph Doherty 3be8ed8915 docs: phase 3.5 status, prune shipped backlog items, capture phase 3.6 follow-ups (T87) 2026-04-26 22:45:59 -04:00
Joseph Doherty 097073ede5 merge: T86 frontend turn_html_replace SSE handler + event_id stamping 2026-04-26 22:42:40 -04:00
Joseph Doherty 4a2617565b merge: T85 JSON-build audit + meanwhile cancel route-level test 2026-04-26 22:42:40 -04:00
Joseph Doherty 73625c0ac4 merge: T84 unified record_turn_memory API with you_present kwarg 2026-04-26 22:42:40 -04:00