Commit Graph

253 Commits

Author SHA1 Message Date
Joseph Doherty e0a28abbcd feat: generate_embedding routes non-default models through client.embed (T112.3)
When model != DEFAULT_EMBEDDING_MODEL, generate_embedding now
calls client.embed(text, model=model) and wraps the returned
vector in an EmbeddingResult tagged with the requested model.
On any exception (NotImplementedError from providers without an
embeddings endpoint, transient network errors, etc.), the existing
T107 warning fires and the function falls back to the zero-vector
sentinel — callers detect model == 'fallback' and skip indexing.

Adds:
- MockLLMClient accepts a canned_embeddings queue mirroring
  the existing canned pattern. embed() pops from the front;
  empty queue raises IndexError so misconfigured tests fail
  loudly.
- Settings.embedding_model defaults to "pseudo-sha256-384"
  so existing zero-config installs keep Phase 4 behavior. The app
  lifespan now passes this through to EmbeddingWorker.model.

The public signature of generate_embedding is unchanged:
(client, *, text, model=DEFAULT_EMBEDDING_MODEL, dim=..., timeout_s=...).
2026-04-27 05:50:29 -04:00
Joseph Doherty ac6e74ab4c feat: FeatherlessClient.embed() against /v1/embeddings (T112.2)
Implements embed() on FeatherlessClient. Featherless's OpenAI-
compatible surface does NOT expose /v1/embeddings at the time of
writing, so this implementation raises NotImplementedError rather
than issuing a request that would 404. The
chat.services.embeddings.generate_embedding wrapper (T112.3)
catches the exception and degrades to the zero-vector fallback path
(plus the existing T107 warning) — misconfigured callers fail loudly
in logs while the request path keeps working.

If/when Featherless ships embeddings, swap the body for
self._client.embeddings.create(model=..., input=...) guarded by
the existing 2-conn semaphore (mirrors generate/stream). The Protocol
seam in T112.1 is already wired so no other code needs to change.

Adds tests/test_featherless.py pinning the NotImplementedError
contract.
2026-04-27 05:48:34 -04:00
Joseph Doherty 5f16bb575a feat: LLMClient Protocol gains embed() method (T112.1)
Adds async def embed(self, text: str, *, model: str) -> list[float]
to the LLMClient Protocol so Phase 4.5 can wire a real-embedding swap
without changing call sites. Protocol is structural — existing
implementations that don't use it remain compatible; downstream
implementations (FeatherlessClient, MockLLMClient) ship in T112.2 and
T112.3.
2026-04-27 05:47:55 -04:00
Joseph Doherty f05d1e0f21 merge: T111 search UX (FTS snippet + turn deep-link) 2026-04-27 05:42:48 -04:00
Joseph Doherty 9987da2c07 feat: cross-chat search deep-links to turn via memories.event_id (T111.2)
Add ``m.event_id`` (T109's nullable column from migration 0014) to
``search_all_memories``'s SELECT, propagate it through the route's
template context, and have ``search.html`` build result links as
``/chats/{chat_id}#turn-{event_id}`` — matching the ``id="turn-{event_id}"``
anchor that Phase 3.5 T86 stamps on each turn DOM node so the chat page
scrolls to the originating turn on load. Memory rows projected before
the 0014 migration ran read NULL ``event_id``; the template falls back
to a chat-level link in that case so we never emit ``#turn-None``.

Pre-existing tests that asserted on the bare ``href="/chats/{chat_id}"``
contract are updated to assert on the ``href="/chats/{chat_id}#turn-``
prefix to reflect the new deep-link.
2026-04-27 05:42:17 -04:00
Joseph Doherty fa87ab8c55 feat: cross-chat search FTS snippet highlighting (T111.1)
Replace the ``pov_summary`` column in ``search_all_memories``'s SELECT
with ``snippet(memories_fts, 0, '<mark>', '</mark>', '…', 32)`` so each
match in a result row is wrapped in ``<mark>`` for the search-results
UI. The original ``pov_summary`` is still returned alongside as a
non-highlighted fallback. Template renders ``r.snippet|safe`` — the only
HTML in the snippet output is the configured ``<mark>`` markers, so it
is safe to bypass Jinja's auto-escape.
2026-04-27 05:30:32 -04:00
Joseph Doherty fae6edef6b merge: T110 drawer Phase 4.5 bundle (event_id guard + html.escape + Jinja partial + bulk re-rate) 2026-04-27 05:26:03 -04:00
Joseph Doherty 2ab8fcbdf0 feat: drawer bulk significance re-rate per chat (T110.4)
The drawer's Significance review panel previously only supported
per-memory edits. Adds a bulk control: pick ``level_from`` and
``level_to``, and every memory in the chat at ``level_from`` is moved
to ``level_to``.

Implementation emits one ``manual_edit`` event per matching memory
(not a single bulk event) so the §6.4 per-row audit trail stays
intact — each affected memory carries its own ``prior_value -> new_value``
snapshot, so an inverse edit can restore an individual row without
needing to inspect a bulk payload's member list. Reuses the existing
``memory_significance`` ``manual_edit`` projector branch (T25), so no
state-layer changes are required.

The route rejects no-op submissions (``level_from == level_to``) with
400 to avoid padding the event log with empty edits, and clamps both
levels to 0..3 (matching ``edit_memory_significance``).

UI: a small ``<details>`` block in the Significance review section
with two number inputs and a submit button.

Test: tests/test_drawer_phase4.py::test_bulk_significance_re_rate_emits_manual_edit_per_memory.
2026-04-27 05:14:59 -04:00
Joseph Doherty 5d5c888acf refactor: drawer delete-impact modal extracted to Jinja partial (T110.3)
The modal HTML was assembled via raw f-string concatenation in
``delete_preview``. Move it to a dedicated Jinja2 partial
(``chat/templates/_delete_impact_modal.html``) and render via
``TEMPLATES.TemplateResponse``. Jinja2 autoescape now handles HTML
safety automatically — the explicit ``html.escape()`` calls added in
T110.2 (and the ``import html``) become redundant and are removed in
this commit.

Net behavioural change: attribute quoting style flips from single to
double quotes (Jinja default) — the existing T98.4 substring-based
assertions are unaffected, and the new T110.3 test pins the
double-quoted shape so future regressions surface.

Test: tests/test_drawer_phase4.py::test_delete_impact_modal_uses_jinja_partial.
2026-04-27 05:13:36 -04:00
Joseph Doherty a45a33534f fix: drawer delete-impact modal HTML escapes user-controllable fields (T110.2)
The delete-impact modal is built via raw f-string concatenation from the
ImpactReport — item.kind / item.description / report.notes ultimately
embed user-controllable content (turn prose, scene timestamps). A turn
with prose like "<script>alert(1)</script>" would reach the rendered
HTML verbatim. Currently safe (the fields embedded today are bounded
strings) but defense-in-depth — wrap with html.escape() so future
description changes can't smuggle markup through.

Test: tests/test_drawer_phase4.py::test_delete_impact_modal_escapes_user_controllable_strings.
2026-04-27 05:12:28 -04:00
Joseph Doherty f3827706df fix: drawer delete_turn guards event_id <= 0 (T110.1)
A stale tab or hand-crafted request posting event_id=0 to the surgical
delete route would compute after_event_id=-1 and silently truncate the
entire log. Now rejected with 400.

SQLite assigns event_log ids starting at 1, so any legitimate id is
always >= 1 — non-positive values can only indicate a client bug.

Test: tests/test_drawer_phase4.py::test_delete_turn_with_event_id_zero_returns_400.
2026-04-27 05:11:39 -04:00
Joseph Doherty 2afbb9fefc merge: T109 schema 0014 — memories.event_id column 2026-04-27 05:01:17 -04:00
Joseph Doherty 1f8b4d2078 feat: 0014 schema — embeddings FK CASCADE (deferred or applied) + memories.event_id column (T109) 2026-04-27 05:00:57 -04:00
Joseph Doherty 3f1a284acb merge: T108 scene-close-on-cancel strengthen test + rationale 2026-04-27 04:48:00 -04:00
Joseph Doherty 87f93f00b5 merge: T107 embeddings.py fallback warning 2026-04-27 04:48:00 -04:00
Joseph Doherty d1e2902655 merge: T106 search.py N+1 batching + k constant 2026-04-27 04:48:00 -04:00
Joseph Doherty 54dfa8d611 merge: T105 snapshots.py polish 2026-04-27 04:48:00 -04:00
Joseph Doherty 5d36d3456f merge: T104 memory.py DRY MAX(id) + fts_rank doc 2026-04-27 04:48:00 -04:00
Joseph Doherty 0e9421dcf7 merge: T103 branches polish (global-leak doc + unknown-name warning) 2026-04-27 04:48:00 -04:00
Joseph Doherty baffeb3a44 chore: scene-close-on-cancel — strengthen regression test + document rationale (T108)
Investigation surfaced a transactional bug in the cancel path: when the
primary stream raises asyncio.CancelledError mid-stream, post_turn
re-raises at end-of-function, and open_db's dependency teardown skips
conn.commit() — rolling back ALL post-cancel writes including the
scene_closed event. The existing T74.3 regression test only passes
because asyncio is not imported at module scope, so CancelledError
becomes NameError (caught by except Exception, leaves cancelled=False).
Documented in turns.py + test docstring; deferred for triage.
2026-04-27 04:47:26 -04:00
Joseph Doherty 29b7c90b29 chore: embeddings.py warns on fallback for non-default models (T107) 2026-04-27 04:47:17 -04:00
Joseph Doherty 64c9ca634a chore: snapshots.py polish — hoisted imports + strict kind + mtime doc (T105) 2026-04-27 04:47:14 -04:00
Joseph Doherty 374a76c867 chore: branches polish — global-leak docs + unknown-name warning (T103) 2026-04-27 04:34:32 -04:00
Joseph Doherty b65e1e1098 chore: memory.py DRY MAX(id) helper + document fts_rank=None contract (T104) 2026-04-27 04:34:28 -04:00
Joseph Doherty 996a16cfb5 perf: search.py N+1 batching + k constant extraction (T106) 2026-04-27 04:34:18 -04:00
Joseph Doherty a06f90a164 docs: add Phase 4.5 cleanup plan (all 24 backlog items)
16 tasks across 9 waves consolidating all 24 items in CLAUDE.md
Phase 4.5/5 backlog. Mix of:

- Wave 1 (parallel 6-way): trivial polish across 6 different files
- Wave 2 (single): schema migration 0014 (FK CASCADE + memories.event_id)
- Wave 3 (single): drawer bundle (event_id guard + html.escape + modal
  partial + bulk significance re-rate)
- Wave 4 (single): search UX (FTS snippet highlight + deep-link)
- Wave 5 (single): real embedding model swap (LLMClient.embed protocol)
- Wave 6 (single): branching read-side filter (riskiest — cross-cutting)
- Wave 7 (single): regenerate lifecycle rollback
- Wave 8 (single): sqlite-vec swap [ENVIRONMENTAL — may defer to Phase 5
  if Python rebuild / apsw not feasible]
- Wave 9 (parallel 3-way): structured fixture builder + integration tests + docs

Schema baseline 13 -> 14 (or 15 with T115). Big tasks (T112 real embed,
T113 branching filter, T114 lifecycle rollback) advance the engine
beyond Phase 4's metadata-only state. T115 environmental decision
captured in pre-flight; the other 13 tasks ship without it.

Uses task ids T103-T118 to avoid collision with prior phases.
2026-04-27 04:22:08 -04:00
dohertj2 df977fc985 Merge pull request 'Phase 4: vector retrieval, branching, drawer polish' (#6) from phase-4 into main 2026-04-27 04:10:25 -04:00
Joseph Doherty 51a12afbec merge: T102 phase 4 documentation update 2026-04-27 04:09:09 -04:00
Joseph Doherty fc3020a0ee merge: T101 phase 4 cross-feature integration tests 2026-04-27 04:09:09 -04:00
Joseph Doherty 228f9abb19 test: phase 4 cross-feature integration coverage (T101) 2026-04-27 04:08:25 -04:00
Joseph Doherty b6119879e5 docs: phase 4 status, behavioral defaults, deferred items (T102) 2026-04-27 03:56:45 -04:00
Joseph Doherty 3b4c7b9cef merge: T100 cross-chat search UX (top-bar + results page) 2026-04-27 03:48:06 -04:00
Joseph Doherty 36d75fa6e7 merge: T99 snapshot UX (manual trigger + list + restore + preview) 2026-04-27 03:48:06 -04:00
Joseph Doherty 0a2c5924f9 feat: cross-chat search UX (top-bar + results page) (T100)
Wires T93's `search_all_memories` service into a small read-only HTML
surface so users can find a memory across every chat in the database.

* `chat/web/search.py` (new): GET `/search?q=...` runs the FTS service
  with k=50, hydrates each row with bot name + scene timestamp, and
  renders `search.html`. Empty `q` short-circuits to no results so the
  top-bar form can submit even with an empty input.
* `chat/templates/search.html` (new): empty-state placeholder, results
  list with chat-level "Open chat" links (`/chats/{chat_id}` — memories
  don't carry an event_id today, so no per-turn anchor).
* `chat/templates/layout.html`: append a small `<form>` to the rail
  nav, additive only.
* `chat/app.py`: register `search_router` (additive import + include).
* `tests/test_search_ux.py`: 3 tests — multi-chat results, empty-query
  placeholder, chat link.
2026-04-27 03:46:52 -04:00
Joseph Doherty a5f0e69d44 feat: snapshot UX (manual trigger + list + restore + preview) (T99) 2026-04-27 03:46:49 -04:00
Joseph Doherty 3dbe1a01ff merge: T98 drawer Phase 4 bundle (branching + sig review + hide + delete + remaining edits) 2026-04-27 03:38:15 -04:00
Joseph Doherty 4546bc0d9c feat: drawer remaining v1 field edits (T98.5)
Audit of chat/state/manual_edit.py target_kind dispatch found two §6.4
fields without drawer affordances despite being already-projected text
columns: chat_state.narrative_anchor and chat_state.weather. Both land
via new manual_edit branches (target_kind chat_narrative_anchor and
chat_weather) plus paired drawer routes and Scene-section text inputs.

The container properties_json blob is intentionally deferred — bounded
JSON edits aren't wired through manual_edit and the drawer never
surfaces multiple containers at once, so v1 leaves it out.
2026-04-27 03:35:54 -04:00
Joseph Doherty c4fa11fe78 feat: drawer surgical delete with cascade preview (T98.4) 2026-04-27 03:29:07 -04:00
Joseph Doherty 461d441078 feat: drawer hide-from-view toggle + turn_hidden manual_edit branch (T98.3) 2026-04-27 03:27:59 -04:00
Joseph Doherty b25007eb44 feat: drawer significance review panel (T98.2) 2026-04-27 03:25:40 -04:00
Joseph Doherty d39d31479d feat: drawer branching UI (T98.1) 2026-04-27 03:24:02 -04:00
Joseph Doherty 7899c50b6c merge: T97 memory write hook + embedding worker + backfill + call-site wiring 2026-04-27 03:09:14 -04:00
Joseph Doherty 177e39d59c feat: wire embedding worker call sites in turns/meanwhile/skip/regenerate (T97.5) 2026-04-27 03:08:36 -04:00
Joseph Doherty d85ed8aaa6 feat: backfill_embeddings script for existing memories (T97.4) 2026-04-27 02:51:48 -04:00
Joseph Doherty 9c63d6b24c feat: app lifespan starts/stops EmbeddingWorker (T97.3) 2026-04-27 02:51:44 -04:00
Joseph Doherty 64a07aa87f feat: memory_write enqueues embedding job after each memory_written (T97.2) 2026-04-27 02:51:40 -04:00
Joseph Doherty 6674f9475c feat: embedding worker drains queue and emits embedding_indexed events (T97.1) 2026-04-27 02:51:36 -04:00
Joseph Doherty 50448b72f8 merge: T96 combined FTS + vector retrieval ranking via RRF 2026-04-27 02:44:03 -04:00
Joseph Doherty b8b4aed6d9 feat: combined FTS + vector retrieval ranking via RRF (T96) 2026-04-27 02:42:38 -04:00
Joseph Doherty 5ff107574c merge: T95 delete-impact computation service 2026-04-27 02:37:28 -04:00