Compare commits
12 Commits
757abf24f8
...
phase-4.5
| Author | SHA1 | Date | |
|---|---|---|---|
| cdfacdd0c4 | |||
| 5c4356e4e2 | |||
| 969f8963bc | |||
| f71613786b | |||
| 4afaf01de7 | |||
| 5bc9a94734 | |||
| cfc05a140c | |||
| 80ce891bd8 | |||
| 6d4ad86e33 | |||
| 7370f68bdf | |||
| f863cf0158 | |||
| 456f50d334 |
@@ -322,53 +322,48 @@ Phase 4 polish shipped end-to-end across 15 tasks (T88–T102). Vector retrieval
|
|||||||
|
|
||||||
### Phase 4.5 / 5 backlog
|
### Phase 4.5 / 5 backlog
|
||||||
|
|
||||||
New follow-ups discovered during Phase 4 reviews and execution. None are blocking; pick up at any time.
|
All items shipped or deferred to Phase 5 (see "Phase 5 backlog" below). Final schema version: 14.
|
||||||
|
|
||||||
#### From T88 review
|
## Phase 4.5 status
|
||||||
|
|
||||||
- **`embeddings` FK lacks `ON DELETE CASCADE`**: deindex events are the only deletion path; if memories ever get deleted directly (raw SQL), embedding rows orphan. Defensible since projector model uses explicit deindex events, but worth a comment or `ON DELETE CASCADE` addition.
|
Phase 4.5 cleanup shipped 13 of 14 planned tasks (T103–T117 with T115 deferred; T118 is this docs sweep). Two CLAUDE.md backlogs (Phase 3.6/4, Phase 4.5/5) are now empty; deferred follow-ups discovered during execution are tracked in a new "Phase 5 backlog" section below. Schema baseline advanced from version 13 to **14** (migration 0014: `memories.event_id`). Test count grew from ~413 (Phase 4) to ~457 (+~44 new tests across the wave).
|
||||||
|
|
||||||
#### From T89 review
|
- **Wave 1 — trivial polish (parallel)**:
|
||||||
|
- **T103** branches polish — global-branch (`chat_id IS NULL`) leak documented in `list_branches`; branch-switch to nonexistent name now logs a warning.
|
||||||
|
- **T104** `memory.py` DRY — `MAX(id)` helper extracted; `fts_rank=None` contract documented for vector-only rows.
|
||||||
|
- **T105** `snapshots.py` polish — `datetime`/`timezone` imports hoisted to module level; strict `kind` validation in restore/preview (rejects missing); `created_at` from file mtime documented.
|
||||||
|
- **T106** `search.py` polish — `k=50` extracted to module constant; N+1 `get_bot`/`get_chat`/`get_scene` lookups batched.
|
||||||
|
- **T107** `embeddings.py` — `timeout_s` fallback-path warning when non-default model misconfigured.
|
||||||
|
- **Wave 2 — scene-close-on-cancel (single)**:
|
||||||
|
- **T108** strengthened the T74.3 regression test + documented rationale in `turns.py`. **Surfaced a deferred bug**: existing pin only passes because `asyncio` isn't imported in the test module (NameError caught instead of CancelledError). When CancelledError fires for real, `post_turn`'s end-of-function re-raise causes `open_db`'s dependency teardown to skip `conn.commit()`, rolling back ALL post-cancel writes. Documented and deferred to Phase 5 triage.
|
||||||
|
- **Wave 3 — schema 0014 (single)**:
|
||||||
|
- **T109** `memories.event_id` column (foundation for T111 deep-link). FK CASCADE on `embeddings.memory_id` deferred (memories rows are never deleted today; defensive constraint can't fire — saved for broader migration cleanup in Phase 5).
|
||||||
|
- **Wave 4 — drawer Phase 4.5 bundle (single)**:
|
||||||
|
- **T110** `event_id <= 0` guard in `delete_turn` + `html.escape()` on delete-impact modal + Jinja partial extraction + bulk significance re-rate per chat (one `manual_edit` event per memory).
|
||||||
|
- **Wave 5 — search UX (single)**:
|
||||||
|
- **T111** FTS snippet highlighting via `snippet()` + deep-link to turn via `memories.event_id`.
|
||||||
|
- **Wave 6 — real embedding model swap (single)**:
|
||||||
|
- **T112** `LLMClient.embed()` Protocol + Mock impl with `canned_embeddings` + `FeatherlessClient.embed()` (raises `NotImplementedError` — Featherless OAI-compat doesn't expose embeddings, gap documented) + `generate_embedding` routes non-default models through `client.embed()` with fallback + `--re-embed-all` backfill flag.
|
||||||
|
- **Wave 7 — branching read-side filter (single)**:
|
||||||
|
- **T113** `active_branch_event_ids(conn)` helper + applied to `read_recent_dialogue`, `scene_summarize._read_recent_dialogue`, `search_memories`, and `meanwhile._read_recent_meanwhile_dialogue`. Cross-chat search and projector queries deliberately NOT filtered (cross-chat is by design; projectors must see full log). Bootstrap "main" branch (origin=0, head=0) detected as the no-clamp sentinel.
|
||||||
|
- **Wave 8 — regenerate lifecycle rollback (single)**:
|
||||||
|
- **T114** `triggered_by_assistant_turn_id` payload back-reference on `event_started`/`event_completed`/`event_cancelled` + new `event_status_reverted` event kind + projector handler in `chat/state/events.py` + regenerate flow emits revert events for affected lifecycle transitions.
|
||||||
|
- **Wave 9 — final polish + integration (parallel)**:
|
||||||
|
- **T115** sqlite-vec swap — **DEFERRED to Phase 5**. Pre-flight failed: host Python build doesn't expose `sqlite3.Connection.enable_load_extension` (raises `AttributeError`). Requires either Python rebuild with `--enable-loadable-sqlite-extensions` or migration to `apsw`. Phase 4 pure-Python cosine remains in production.
|
||||||
|
- **T116** structured `CannedQueue` test fixture builder + 2–3 POC test migrations (Phase 5 to migrate the rest).
|
||||||
|
- **T117** Phase 4.5 cross-feature integration tests (5 minimum: real embedding swap, branching read-side filter, lifecycle rollback, search deep-link, bulk significance re-rate).
|
||||||
|
- **T118** documentation (this section).
|
||||||
|
|
||||||
- **`list_branches(chat_id=...)` filter leaks global branches** (`chat_id IS NULL`) into every chat scope. Intentional? Document.
|
### Phase 5 backlog
|
||||||
- **Branch-switch to nonexistent silently leaves zero active branches** — log a warning when this would happen.
|
|
||||||
|
|
||||||
#### From T91 review
|
New follow-ups discovered during Phase 4.5 reviews and execution, plus carry-over deferrals. None are blocking; pick up at any time.
|
||||||
|
|
||||||
- **Real embedding model swap**: Phase 4 ships pseudo-embedding (deterministic SHA-256 hash). Phase 4.5+ should swap to a real model (Featherless `bge-small-en-v1.5` if available; or local `sentence-transformers/all-MiniLM-L6-v2`). The 384-dim is hardcoded in `0012_embeddings.sql`; if dim changes, migrate first.
|
- **T115 sqlite-vec swap** (environmental blocker): host Python's `sqlite3.Connection` does not expose `enable_load_extension` — `python -c "import sqlite3; sqlite3.connect(':memory:').enable_load_extension(True)"` raises `AttributeError`. Fix requires either a Python rebuild with `--enable-loadable-sqlite-extensions` or migration to `apsw`. Pure-Python cosine remains in production until then.
|
||||||
- **`timeout_s` unused on pseudo path** — fine, but log when non-default model falls through to fallback so misconfigured callers don't silently degrade.
|
- **T108 follow-up: cancel-path commit bug** — `post_turn`'s re-raised `CancelledError` causes `open_db` dependency teardown to skip `conn.commit()`, rolling back all post-cancel writes. The existing T74.3 regression test passes only because `asyncio` isn't imported in the test module (NameError masks the real cancel path). Triage required — either commit before re-raise, or restructure the route to never re-raise after the close-detection branch.
|
||||||
|
- **`embeddings` FK CASCADE on `memory_id`** — deferred from T109; do as part of a broader migration consolidation in Phase 5.
|
||||||
#### From T96 review
|
- **`CannedQueue` fixture migration** — T116 shipped the builder + POC migrations; remaining tests still use positional canned arrays. Migrate in Phase 5.
|
||||||
|
- **Vector index optimization (HNSW)** — currently scales to a few thousand memories on the flat-index pure-Python cosine path; revisit when counts grow past flat-index feasibility.
|
||||||
- **Duplicate `MAX(id)` lookup** between `_composite_rerank` and the fused-path tail — DRY follow-up.
|
- **Branch-isolated `event_log`** — each branch has its own physical `event_log` range vs the current shared id space + head filter; full branch isolation is Phase 5+.
|
||||||
- **`fts_rank=None` for vector-only rows** — document downstream contract.
|
- **Embedding model swap migration tooling** — T112 added `--re-embed-all`; a more orchestrated swap (drain old worker, re-seed all memories, swap config) is Phase 5+.
|
||||||
|
- **Real-time collaborative branching** (multi-user) — out of scope for v1.
|
||||||
#### From T98 review
|
- **Avatars / portraits** (multimodality) — deferred indefinitely per design §14.
|
||||||
|
|
||||||
- **`event_id <= 0` guard in `delete_turn`** — currently silently rewinds everything if `event_id` is 0. Add `if event_id <= 0: 400`.
|
|
||||||
- **`html.escape()` on `compute_delete_impact` output rendered into the modal** — defense in depth (currently model-controlled strings, but if event payload fields ever appear in descriptions, autoescape needed).
|
|
||||||
- **Extract delete-impact modal HTML to a Jinja partial** — testability + autoescape inheritance.
|
|
||||||
|
|
||||||
#### From T99 review
|
|
||||||
|
|
||||||
- **Hoist `datetime`/`timezone` imports to module level** in `chat/web/snapshots.py`.
|
|
||||||
- **`kind` defaulting in restore/preview** — reject missing `kind` rather than silent 404.
|
|
||||||
- **`created_at` from file mtime** vs filename-encoded timestamp — small drift if files copied; document.
|
|
||||||
|
|
||||||
#### From T100 review
|
|
||||||
|
|
||||||
- **Hardcoded `k=50`** — extract to module constant.
|
|
||||||
- **N+1 lookups (`get_bot`/`get_chat`/`get_scene` per row)** — fine at `k=50`, revisit if `k` grows.
|
|
||||||
- **FTS highlighting via `snippet()`** — Phase 4 skipped this; UX nice-to-have.
|
|
||||||
- **Result links chat-level only** — `memories` table has no `event_id` column; deep-linking to specific turn requires schema addition.
|
|
||||||
|
|
||||||
#### Deferred items
|
|
||||||
|
|
||||||
- **sqlite-vec swap** when host Python supports `enable_load_extension`.
|
|
||||||
- **Real embedding model** with proper semantic similarity.
|
|
||||||
- **Branching read-side filter**: T89 ships data-model + UI but event readers don't yet consult `is_active`. Each branch is metadata-only labeled ranges. Consult-on-read is Phase 4.5+ work.
|
|
||||||
- **Bulk significance re-rate** in drawer (T98.2 deferred — only per-memory edit shipped).
|
|
||||||
- **Vector index optimization** (HNSW) — only relevant if memory counts grow past pure-Python feasibility.
|
|
||||||
- **`scene-close-on-cancel` UX revisit** (Phase 2.5 carry-over).
|
|
||||||
- **Cross-feature canned-queue brittleness fixture builder** (Phase 3 carry-over).
|
|
||||||
- **Full lifecycle-rollback in regenerate** — Phase 3.5 T83.4 shipped a warning log; proper rollback needs schema-level back-references (`triggered_by_assistant_turn_id` payload field).
|
|
||||||
|
|||||||
+130
-36
@@ -95,6 +95,27 @@ from chat.web.render import render_turn_html
|
|||||||
_log = logging.getLogger(__name__)
|
_log = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
# T114.3: map a lifecycle-transition event kind to the events-table
|
||||||
|
# status it implicitly transitioned *from*. Regenerate uses this to pick
|
||||||
|
# the ``prior_status`` value for the ``event_status_reverted`` rollback
|
||||||
|
# event so the projector sets the row back to where it was before the
|
||||||
|
# superseded turn fired the transition.
|
||||||
|
#
|
||||||
|
# - ``event_started`` was emitted when the row was 'planned' → revert to
|
||||||
|
# 'planned'.
|
||||||
|
# - ``event_completed`` was emitted when the row was 'active' → revert
|
||||||
|
# to 'active'.
|
||||||
|
# - ``event_cancelled`` could have fired from either 'planned' or
|
||||||
|
# 'active'. Best-effort default: 'active'. The forward transitions
|
||||||
|
# below only fire detect_event_transitions for currently-active rows,
|
||||||
|
# so 'active' is the realistic prior in practice.
|
||||||
|
_PRIOR_STATUS_MAP: dict[str, str] = {
|
||||||
|
"event_started": "planned",
|
||||||
|
"event_completed": "active",
|
||||||
|
"event_cancelled": "active",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
async def regenerate_assistant_turn(
|
async def regenerate_assistant_turn(
|
||||||
conn: Connection,
|
conn: Connection,
|
||||||
client,
|
client,
|
||||||
@@ -115,17 +136,18 @@ async def regenerate_assistant_turn(
|
|||||||
cannot be found — the FastAPI route translates this to 404.
|
cannot be found — the FastAPI route translates this to 404.
|
||||||
|
|
||||||
.. note::
|
.. note::
|
||||||
**Lifecycle-rollback limitation (T83.4, Phase 4 follow-up).**
|
**Lifecycle rollback (T114, Phase 4.5).**
|
||||||
When the superseded turn already produced lifecycle transitions
|
When the superseded turn already produced lifecycle transitions
|
||||||
(``event_started`` / ``event_completed`` / ``event_cancelled``),
|
(``event_started`` / ``event_completed`` / ``event_cancelled``),
|
||||||
this function does NOT roll those rows back before re-running
|
this function emits an ``event_status_reverted`` event for each
|
||||||
``detect_event_transitions`` against the regenerated text. A
|
so the events row's status returns to its prior value before the
|
||||||
regenerate-after-completion can therefore double-emit promotion
|
regenerated narrative is reclassified. Backward compatibility:
|
||||||
artifacts if the new text re-completes the same event. Phase 3.5
|
lifecycle events authored before T114.1 lack the
|
||||||
only documents the gap and emits a WARNING log naming the
|
``triggered_by_assistant_turn_id`` payload field; rollback skips
|
||||||
affected event_log ids; the actual undo pass is invasive
|
those (logged at DEBUG) so historic rows are not retroactively
|
||||||
(re-projection / inverse-handler dispatch) and is deferred to
|
reverted. A WARNING about un-rolled-back transitions is still
|
||||||
Phase 4. See the ``# T83.4`` block below for the warning emit.
|
emitted when stragglers are found — the rollback handles the
|
||||||
|
common case while older logs continue to need manual review.
|
||||||
"""
|
"""
|
||||||
chat = get_chat(conn, chat_id)
|
chat = get_chat(conn, chat_id)
|
||||||
if chat is None:
|
if chat is None:
|
||||||
@@ -158,20 +180,21 @@ async def regenerate_assistant_turn(
|
|||||||
original_assistant_payload = json.loads(row[0])
|
original_assistant_payload = json.loads(row[0])
|
||||||
original_user_turn_id = original_assistant_payload.get("user_turn_id")
|
original_user_turn_id = original_assistant_payload.get("user_turn_id")
|
||||||
|
|
||||||
# T83.4: scan for downstream lifecycle transitions emitted by the
|
# T114.3: roll back lifecycle transitions emitted by the superseded
|
||||||
# superseded turn — they're not being rolled back (see method
|
# turn. The scan uses the same id-greater-than-superseded-turn
|
||||||
# docstring). Heuristic: any ``event_started`` / ``event_completed``
|
# heuristic as the legacy T83.4 warning, joined to ``events`` for
|
||||||
# / ``event_cancelled`` event_log row with id strictly greater than
|
# chat scoping (lifecycle events don't carry chat_id in their
|
||||||
# the original assistant_turn's id was emitted as part of (or after)
|
# payload — they reference an ``event_id`` FK to the ``events``
|
||||||
# that turn's processing. Lifecycle events don't carry ``chat_id``
|
# table, which holds chat_id). For each row whose payload carries
|
||||||
# in their payload (their payload references an ``event_id`` FK to
|
# ``triggered_by_assistant_turn_id == original_assistant_event_id``
|
||||||
# the ``events`` table, which holds chat_id), so we join through
|
# (T114.1 back-reference), emit an ``event_status_reverted`` event
|
||||||
# ``events`` to scope to this chat.
|
# so the events-row status returns to the pre-transition value.
|
||||||
#
|
# Lifecycle rows authored before T114.1 lack the back-reference;
|
||||||
# A WARNING log surfaces the affected event ids so operators can
|
# those are skipped (DEBUG log) and a WARNING tracks their count so
|
||||||
# spot double-emit cases until the Phase 4 rollback pass lands.
|
# operators still see legacy stragglers — preserves the T83.4
|
||||||
|
# observability contract for un-rolled-back transitions.
|
||||||
unrolled_lifecycle = conn.execute(
|
unrolled_lifecycle = conn.execute(
|
||||||
"SELECT el.id, el.kind FROM event_log AS el "
|
"SELECT el.id, el.kind, el.payload_json FROM event_log AS el "
|
||||||
"JOIN events AS ev "
|
"JOIN events AS ev "
|
||||||
" ON ev.event_id = json_extract(el.payload_json, '$.event_id') "
|
" ON ev.event_id = json_extract(el.payload_json, '$.event_id') "
|
||||||
"WHERE el.kind IN ("
|
"WHERE el.kind IN ("
|
||||||
@@ -182,18 +205,73 @@ async def regenerate_assistant_turn(
|
|||||||
"ORDER BY el.id ASC",
|
"ORDER BY el.id ASC",
|
||||||
(chat_id, original_assistant_event_id),
|
(chat_id, original_assistant_event_id),
|
||||||
).fetchall()
|
).fetchall()
|
||||||
if unrolled_lifecycle:
|
rolled_back_ids: list[int] = []
|
||||||
# T90.2: phrased as "at-or-after turn <id>" rather than "from
|
skipped_no_backref: list[int] = []
|
||||||
# superseded turn" because regenerating an OLDER turn lists
|
for el_id, el_kind, el_payload_json in unrolled_lifecycle:
|
||||||
# intervening-turn transitions that legitimately stand on their
|
try:
|
||||||
# own — those weren't authored by the superseded turn itself.
|
lifecycle_payload = json.loads(el_payload_json)
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
skipped_no_backref.append(el_id)
|
||||||
|
continue
|
||||||
|
triggered_by = lifecycle_payload.get("triggered_by_assistant_turn_id")
|
||||||
|
if triggered_by != original_assistant_event_id:
|
||||||
|
# Either a legacy row (no field) or a transition triggered
|
||||||
|
# by a *different* turn — leave it alone. DEBUG so the
|
||||||
|
# message is available under verbose logging without
|
||||||
|
# spamming the default WARNING channel.
|
||||||
|
_log.debug(
|
||||||
|
"regenerate_assistant_turn: skipping rollback for "
|
||||||
|
"lifecycle event_log id=%d (kind=%s) — no back-reference "
|
||||||
|
"or different turn (triggered_by=%r vs superseded=%d)",
|
||||||
|
el_id,
|
||||||
|
el_kind,
|
||||||
|
triggered_by,
|
||||||
|
original_assistant_event_id,
|
||||||
|
)
|
||||||
|
if triggered_by is None:
|
||||||
|
skipped_no_backref.append(el_id)
|
||||||
|
continue
|
||||||
|
prior_status = _PRIOR_STATUS_MAP.get(el_kind)
|
||||||
|
if prior_status is None:
|
||||||
|
# Defensive: the SQL filter already restricts to the three
|
||||||
|
# known kinds, but a future schema addition shouldn't crash
|
||||||
|
# the rollback path.
|
||||||
|
continue
|
||||||
|
target_event_id = lifecycle_payload.get("event_id")
|
||||||
|
if target_event_id is None:
|
||||||
|
continue
|
||||||
|
append_and_apply(
|
||||||
|
conn,
|
||||||
|
kind="event_status_reverted",
|
||||||
|
payload={
|
||||||
|
"event_id": target_event_id,
|
||||||
|
"prior_status": prior_status,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
rolled_back_ids.append(el_id)
|
||||||
|
if rolled_back_ids:
|
||||||
|
_log.info(
|
||||||
|
"regenerate_assistant_turn: rolled back %d lifecycle "
|
||||||
|
"transition(s) triggered by superseded turn %s "
|
||||||
|
"(event_log ids: %s)",
|
||||||
|
len(rolled_back_ids),
|
||||||
|
original_assistant_event_id,
|
||||||
|
rolled_back_ids,
|
||||||
|
)
|
||||||
|
if skipped_no_backref:
|
||||||
|
# T83.4 (legacy) compatibility: still warn about stragglers
|
||||||
|
# without the back-reference so operators can spot pre-T114
|
||||||
|
# double-emit risks. Phrased as "at-or-after turn <id>" per
|
||||||
|
# T90.2 — older transitions may legitimately belong to other
|
||||||
|
# turns.
|
||||||
_log.warning(
|
_log.warning(
|
||||||
"regenerate_assistant_turn: %d lifecycle transition(s) "
|
"regenerate_assistant_turn: %d lifecycle transition(s) "
|
||||||
"at-or-after turn %s are NOT being rolled back (Phase 4 "
|
"at-or-after turn %s are NOT being rolled back (no "
|
||||||
"follow-up). Affected event ids: %s",
|
"triggered_by_assistant_turn_id back-reference). "
|
||||||
len(unrolled_lifecycle),
|
"Affected event ids: %s",
|
||||||
|
len(skipped_no_backref),
|
||||||
original_assistant_event_id,
|
original_assistant_event_id,
|
||||||
[r[0] for r in unrolled_lifecycle],
|
skipped_no_backref,
|
||||||
)
|
)
|
||||||
|
|
||||||
# 1a. Look up any sibling interjection beat in the same turn group
|
# 1a. Look up any sibling interjection beat in the same turn group
|
||||||
@@ -716,11 +794,13 @@ async def regenerate_assistant_turn(
|
|||||||
# runs inline after a completion so promotion artifacts land in the
|
# runs inline after a completion so promotion artifacts land in the
|
||||||
# same regenerate path.
|
# same regenerate path.
|
||||||
#
|
#
|
||||||
# T83.4 follow-up: when a regenerate replaces a turn that had
|
# T114.3: original-turn transitions emitted before this regenerate
|
||||||
# already produced event transitions, those original transitions
|
# ran were rolled back at the top of the function (see the
|
||||||
# are NOT undone here (Phase 4 work). A WARNING log earlier in this
|
# ``# T114.3`` block) by appending ``event_status_reverted`` for
|
||||||
# function names the affected event_log ids — see the T83.4 block
|
# each. The classify-and-emit pass below now operates against an
|
||||||
# near the function entry.
|
# ``events`` projection that has already been reverted, so it can
|
||||||
|
# safely re-fire transitions for the regenerated narrative without
|
||||||
|
# double-emitting promotion artifacts.
|
||||||
new_active_events = list_active_events(conn, chat_id)
|
new_active_events = list_active_events(conn, chat_id)
|
||||||
if new_active_events:
|
if new_active_events:
|
||||||
lifecycle_decision = await detect_event_transitions(
|
lifecycle_decision = await detect_event_transitions(
|
||||||
@@ -738,6 +818,12 @@ async def regenerate_assistant_turn(
|
|||||||
payload={
|
payload={
|
||||||
"event_id": transition.event_id,
|
"event_id": transition.event_id,
|
||||||
"started_at": chat.get("time"),
|
"started_at": chat.get("time"),
|
||||||
|
# T114.1: back-reference to the assistant_turn
|
||||||
|
# that triggered this transition (see turns.py
|
||||||
|
# for rationale).
|
||||||
|
"triggered_by_assistant_turn_id": (
|
||||||
|
new_assistant_event_id
|
||||||
|
),
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
elif transition.new_status == "completed":
|
elif transition.new_status == "completed":
|
||||||
@@ -747,6 +833,10 @@ async def regenerate_assistant_turn(
|
|||||||
payload={
|
payload={
|
||||||
"event_id": transition.event_id,
|
"event_id": transition.event_id,
|
||||||
"completed_at": chat.get("time"),
|
"completed_at": chat.get("time"),
|
||||||
|
# T114.1: back-reference (see above).
|
||||||
|
"triggered_by_assistant_turn_id": (
|
||||||
|
new_assistant_event_id
|
||||||
|
),
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
promote_completed_event(
|
promote_completed_event(
|
||||||
@@ -762,6 +852,10 @@ async def regenerate_assistant_turn(
|
|||||||
payload={
|
payload={
|
||||||
"event_id": transition.event_id,
|
"event_id": transition.event_id,
|
||||||
"completed_at": chat.get("time"),
|
"completed_at": chat.get("time"),
|
||||||
|
# T114.1: back-reference (see above).
|
||||||
|
"triggered_by_assistant_turn_id": (
|
||||||
|
new_assistant_event_id
|
||||||
|
),
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|||||||
@@ -144,23 +144,36 @@ def _read_recent_dialogue(
|
|||||||
``id >= since_event_id`` so callers needing a scene-scoped view (e.g.
|
``id >= since_event_id`` so callers needing a scene-scoped view (e.g.
|
||||||
thread detection on close) don't pull turns that landed before the
|
thread detection on close) don't pull turns that landed before the
|
||||||
closing scene's ``scene_opened`` event.
|
closing scene's ``scene_opened`` event.
|
||||||
|
|
||||||
|
T113: also clamps by the active branch's ``[origin, head]`` event-id
|
||||||
|
range so scene-summary inputs respect the user's current branch.
|
||||||
|
Bootstrap-main and "no active branch" fall through to ``(0, BIG_INT)``
|
||||||
|
so existing flows are unchanged.
|
||||||
"""
|
"""
|
||||||
|
from chat.state.branches import active_branch_event_ids
|
||||||
|
|
||||||
|
origin, head = active_branch_event_ids(conn)
|
||||||
if since_event_id is None:
|
if since_event_id is None:
|
||||||
cur = conn.execute(
|
cur = conn.execute(
|
||||||
"SELECT kind, payload_json FROM event_log "
|
"SELECT kind, payload_json FROM event_log "
|
||||||
"WHERE kind IN ('user_turn', 'assistant_turn') "
|
"WHERE kind IN ('user_turn', 'assistant_turn') "
|
||||||
" AND superseded_by IS NULL AND hidden = 0 "
|
" AND superseded_by IS NULL AND hidden = 0 "
|
||||||
|
" AND id BETWEEN ? AND ? "
|
||||||
"ORDER BY id DESC LIMIT ?",
|
"ORDER BY id DESC LIMIT ?",
|
||||||
(limit,),
|
(origin, head, limit),
|
||||||
)
|
)
|
||||||
else:
|
else:
|
||||||
|
# Compose ``since_event_id`` with the branch lower bound — readers
|
||||||
|
# want the tightest ``id >= max(since, origin)`` clamp without an
|
||||||
|
# extra Python pass.
|
||||||
|
lower = max(origin, since_event_id)
|
||||||
cur = conn.execute(
|
cur = conn.execute(
|
||||||
"SELECT kind, payload_json FROM event_log "
|
"SELECT kind, payload_json FROM event_log "
|
||||||
"WHERE kind IN ('user_turn', 'assistant_turn') "
|
"WHERE kind IN ('user_turn', 'assistant_turn') "
|
||||||
" AND superseded_by IS NULL AND hidden = 0 "
|
" AND superseded_by IS NULL AND hidden = 0 "
|
||||||
" AND id >= ? "
|
" AND id BETWEEN ? AND ? "
|
||||||
"ORDER BY id DESC LIMIT ?",
|
"ORDER BY id DESC LIMIT ?",
|
||||||
(since_event_id, limit),
|
(lower, head, limit),
|
||||||
)
|
)
|
||||||
rows = list(reversed(cur.fetchall()))
|
rows = list(reversed(cur.fetchall()))
|
||||||
out: list[dict] = []
|
out: list[dict] = []
|
||||||
|
|||||||
@@ -30,6 +30,7 @@ from __future__ import annotations
|
|||||||
import json
|
import json
|
||||||
from sqlite3 import Connection
|
from sqlite3 import Connection
|
||||||
|
|
||||||
|
from chat.state.branches import active_branch_event_ids
|
||||||
from chat.state.edges import get_edge
|
from chat.state.edges import get_edge
|
||||||
|
|
||||||
|
|
||||||
@@ -60,15 +61,22 @@ def read_recent_dialogue(
|
|||||||
previous implementation filtered chat_id post-fetch in Python, which
|
previous implementation filtered chat_id post-fetch in Python, which
|
||||||
let foreign-chat rows fill the LIMIT and yield fewer than N relevant
|
let foreign-chat rows fill the LIMIT and yield fewer than N relevant
|
||||||
rows in busy multi-chat databases.
|
rows in busy multi-chat databases.
|
||||||
|
|
||||||
|
T113: clamp by the active branch's ``[origin, head]`` event-id range so
|
||||||
|
switching branches actually changes what dialogue this read sees.
|
||||||
|
Bootstrap-main and "no active branch" both fall through to ``(0,
|
||||||
|
BIG_INT)`` — no functional change for the metadata-only Phase 4 era.
|
||||||
"""
|
"""
|
||||||
|
origin, head = active_branch_event_ids(conn)
|
||||||
if exclude_event_id is None:
|
if exclude_event_id is None:
|
||||||
cur = conn.execute(
|
cur = conn.execute(
|
||||||
"SELECT id, kind, payload_json FROM event_log "
|
"SELECT id, kind, payload_json FROM event_log "
|
||||||
"WHERE kind IN ('user_turn', 'user_turn_edit', 'assistant_turn') "
|
"WHERE kind IN ('user_turn', 'user_turn_edit', 'assistant_turn') "
|
||||||
" AND superseded_by IS NULL AND hidden = 0 "
|
" AND superseded_by IS NULL AND hidden = 0 "
|
||||||
|
" AND id BETWEEN ? AND ? "
|
||||||
" AND json_extract(payload_json, '$.chat_id') = ? "
|
" AND json_extract(payload_json, '$.chat_id') = ? "
|
||||||
"ORDER BY id DESC LIMIT ?",
|
"ORDER BY id DESC LIMIT ?",
|
||||||
(chat_id, limit),
|
(origin, head, chat_id, limit),
|
||||||
)
|
)
|
||||||
else:
|
else:
|
||||||
cur = conn.execute(
|
cur = conn.execute(
|
||||||
@@ -76,9 +84,10 @@ def read_recent_dialogue(
|
|||||||
"WHERE kind IN ('user_turn', 'user_turn_edit', 'assistant_turn') "
|
"WHERE kind IN ('user_turn', 'user_turn_edit', 'assistant_turn') "
|
||||||
" AND id != ? "
|
" AND id != ? "
|
||||||
" AND superseded_by IS NULL AND hidden = 0 "
|
" AND superseded_by IS NULL AND hidden = 0 "
|
||||||
|
" AND id BETWEEN ? AND ? "
|
||||||
" AND json_extract(payload_json, '$.chat_id') = ? "
|
" AND json_extract(payload_json, '$.chat_id') = ? "
|
||||||
"ORDER BY id DESC LIMIT ?",
|
"ORDER BY id DESC LIMIT ?",
|
||||||
(exclude_event_id, chat_id, limit),
|
(exclude_event_id, origin, head, chat_id, limit),
|
||||||
)
|
)
|
||||||
rows = list(reversed(cur.fetchall()))
|
rows = list(reversed(cur.fetchall()))
|
||||||
out: list[dict] = []
|
out: list[dict] = []
|
||||||
|
|||||||
@@ -157,8 +157,58 @@ def active_branch(conn: Connection) -> dict | None:
|
|||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# T113: sentinel "no upper bound" used by ``active_branch_event_ids`` when the
|
||||||
|
# active branch's head is unset (the bootstrap "main" branch with origin=0 +
|
||||||
|
# head=0). Readers compose ``id BETWEEN origin AND head`` so a value larger
|
||||||
|
# than any possible row id behaves as "no clamp" without needing a separate
|
||||||
|
# code path. ``2**63 - 1`` is SQLite's max signed-int — safe forever.
|
||||||
|
_NO_HEAD_CLAMP = 2**63 - 1
|
||||||
|
|
||||||
|
|
||||||
|
def active_branch_event_ids(conn: Connection) -> tuple[int, int]:
|
||||||
|
"""Return ``(origin_event_id, head_event_id)`` for the currently active
|
||||||
|
branch, suitable as bounds for an ``event_log.id BETWEEN ? AND ?`` clamp
|
||||||
|
on user-facing reads (T113).
|
||||||
|
|
||||||
|
Defensive defaults:
|
||||||
|
|
||||||
|
* **No active branch row** (``active_branch`` returns ``None``) — return
|
||||||
|
``(0, _NO_HEAD_CLAMP)`` so readers see all events. This preserves the
|
||||||
|
Phase 4 "branches are metadata-only" contract for any code path that
|
||||||
|
somehow runs without the migration-0013 bootstrap.
|
||||||
|
* **Bootstrap "main"** — the canonical ``name="main", origin=0, head=0``
|
||||||
|
row inserted by migration 0013. Production today never emits
|
||||||
|
``branch_head_updated`` for main, so head stays at 0 even as events
|
||||||
|
accumulate. We treat this exact bootstrap state as "no clamp" and
|
||||||
|
return ``(0, _NO_HEAD_CLAMP)`` so all events remain visible. This is
|
||||||
|
what every existing test (which never configures branches) relies on.
|
||||||
|
* **Any other branch** — return the literal ``(origin, head)`` from the
|
||||||
|
branch row. A branch created at origin=N has head=N initially (per
|
||||||
|
``branch_from_event``), so ``BETWEEN N AND N`` returns just that one
|
||||||
|
seed event until the head is bumped via ``branch_head_updated``.
|
||||||
|
|
||||||
|
Note on the schema mismatch with the T113 spec: the spec describes
|
||||||
|
``head_event_id`` as nullable, but migration 0013 declared it
|
||||||
|
``NOT NULL DEFAULT 0``. We read head=0 on bootstrap main as the
|
||||||
|
"unset" sentinel; non-main branches never reach head=0 in normal
|
||||||
|
flow (creation sets head=origin, and origin=0 only for main).
|
||||||
|
"""
|
||||||
|
branch = active_branch(conn)
|
||||||
|
if branch is None:
|
||||||
|
return (0, _NO_HEAD_CLAMP)
|
||||||
|
origin = int(branch.get("origin_event_id") or 0)
|
||||||
|
head = int(branch.get("head_event_id") or 0)
|
||||||
|
# Bootstrap "main" sentinel — see docstring above. Detect by name +
|
||||||
|
# both ids being 0 to avoid mis-firing on a hypothetical future
|
||||||
|
# branch that legitimately starts at origin=0.
|
||||||
|
if branch.get("name") == "main" and origin == 0 and head == 0:
|
||||||
|
return (0, _NO_HEAD_CLAMP)
|
||||||
|
return (origin, head)
|
||||||
|
|
||||||
|
|
||||||
__all__ = [
|
__all__ = [
|
||||||
"get_branch",
|
"get_branch",
|
||||||
"list_branches",
|
"list_branches",
|
||||||
"active_branch",
|
"active_branch",
|
||||||
|
"active_branch_event_ids",
|
||||||
]
|
]
|
||||||
|
|||||||
@@ -67,6 +67,29 @@ def _apply_event_expired(conn: Connection, e: Event) -> None:
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@on("event_status_reverted")
|
||||||
|
def _apply_event_status_reverted(conn: Connection, e: Event) -> None:
|
||||||
|
"""T114.2: Revert an event row's status to ``prior_status``.
|
||||||
|
|
||||||
|
Emitted by ``regenerate_assistant_turn`` when a superseded turn had
|
||||||
|
triggered a lifecycle transition (event_started / event_completed /
|
||||||
|
event_cancelled). The rollback step needs an inverse projection that
|
||||||
|
sets the row's status back to whatever it was *before* the now-
|
||||||
|
superseded transition fired.
|
||||||
|
|
||||||
|
Unlike the forward transitions (which guard against terminal-status
|
||||||
|
overwrites) this handler is unconditional — the entire purpose is to
|
||||||
|
reverse a transition, including reverting from a terminal status
|
||||||
|
(completed/cancelled) back to a non-terminal one.
|
||||||
|
"""
|
||||||
|
p = e.payload
|
||||||
|
conn.execute(
|
||||||
|
"UPDATE events SET status = ?, updated_at = datetime('now') "
|
||||||
|
"WHERE event_id = ?",
|
||||||
|
(p["prior_status"], p["event_id"]),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def get_event(conn: Connection, event_id: str) -> dict | None:
|
def get_event(conn: Connection, event_id: str) -> dict | None:
|
||||||
row = conn.execute(
|
row = conn.execute(
|
||||||
"SELECT event_id, chat_id, kind, status, props_json, planned_for, "
|
"SELECT event_id, chat_id, kind, status, props_json, planned_for, "
|
||||||
|
|||||||
+34
-1
@@ -213,12 +213,20 @@ def search_memories(
|
|||||||
# channel) so memories that are weak in FTS but strong in vector — and
|
# channel) so memories that are weak in FTS but strong in vector — and
|
||||||
# vice versa — make it into the merge pool.
|
# vice versa — make it into the merge pool.
|
||||||
over_fetch = max(k * 2, 20) if query_vector is not None else max(k * 4, 20)
|
over_fetch = max(k * 2, 20) if query_vector is not None else max(k * 4, 20)
|
||||||
|
# T113: branch-scope filter on ``m.event_id`` (T109's column). Memories
|
||||||
|
# whose ``event_id`` is NULL — projected before the 0014 schema migration
|
||||||
|
# ran — are *included* unconditionally so the branch filter never breaks
|
||||||
|
# legacy retrieval. Newer rows respect the active branch's bounds.
|
||||||
|
from chat.state.branches import active_branch_event_ids
|
||||||
|
|
||||||
|
origin, head = active_branch_event_ids(conn)
|
||||||
sql = (
|
sql = (
|
||||||
f"SELECT {select_list}, memories_fts.rank AS fts_rank "
|
f"SELECT {select_list}, memories_fts.rank AS fts_rank "
|
||||||
"FROM memories_fts "
|
"FROM memories_fts "
|
||||||
"JOIN memories m ON m.id = memories_fts.rowid "
|
"JOIN memories m ON m.id = memories_fts.rowid "
|
||||||
f"WHERE m.owner_id = ? AND m.{witness_col} = 1 "
|
f"WHERE m.owner_id = ? AND m.{witness_col} = 1 "
|
||||||
"AND memories_fts MATCH ? "
|
"AND memories_fts MATCH ? "
|
||||||
|
"AND (m.event_id IS NULL OR m.event_id BETWEEN ? AND ?) "
|
||||||
# T57: significance multiplier biases the FTS over-fetch order. BM25
|
# T57: significance multiplier biases the FTS over-fetch order. BM25
|
||||||
# ``rank`` is lower-is-better, so subtracting ``significance * BIAS``
|
# ``rank`` is lower-is-better, so subtracting ``significance * BIAS``
|
||||||
# surfaces higher-significance rows above lower-significance rows with
|
# surfaces higher-significance rows above lower-significance rows with
|
||||||
@@ -227,7 +235,10 @@ def search_memories(
|
|||||||
"ORDER BY (memories_fts.rank - m.significance * ?) ASC "
|
"ORDER BY (memories_fts.rank - m.significance * ?) ASC "
|
||||||
"LIMIT ?"
|
"LIMIT ?"
|
||||||
)
|
)
|
||||||
cur = conn.execute(sql, (owner_id, query, SIGNIFICANCE_RANK_BIAS, over_fetch))
|
cur = conn.execute(
|
||||||
|
sql,
|
||||||
|
(owner_id, query, origin, head, SIGNIFICANCE_RANK_BIAS, over_fetch),
|
||||||
|
)
|
||||||
rows = cur.fetchall()
|
rows = cur.fetchall()
|
||||||
|
|
||||||
# FTS-only path: preserve pre-T96 behaviour exactly.
|
# FTS-only path: preserve pre-T96 behaviour exactly.
|
||||||
@@ -331,6 +342,28 @@ def _rrf_fuse_and_rerank(
|
|||||||
query_vector=query_vector,
|
query_vector=query_vector,
|
||||||
k=vec_over_fetch,
|
k=vec_over_fetch,
|
||||||
)
|
)
|
||||||
|
# T113: drop vector hits that fall outside the active branch's event-id
|
||||||
|
# range. ``vector_search`` is a generic service used elsewhere; the
|
||||||
|
# branch filter applied to the FTS leg also has to apply here so the
|
||||||
|
# fused result respects the same scope. Memories with NULL event_id
|
||||||
|
# (legacy rows projected before T109's 0014 schema migration) are
|
||||||
|
# included unconditionally — same policy as the FTS leg.
|
||||||
|
from chat.state.branches import _NO_HEAD_CLAMP, active_branch_event_ids
|
||||||
|
|
||||||
|
vec_origin, vec_head = active_branch_event_ids(conn)
|
||||||
|
if vec_hits and (vec_origin > 0 or vec_head < _NO_HEAD_CLAMP):
|
||||||
|
vec_ids = [h["memory_id"] for h in vec_hits]
|
||||||
|
placeholders_v = ",".join("?" * len(vec_ids))
|
||||||
|
in_range = {
|
||||||
|
row[0]
|
||||||
|
for row in conn.execute(
|
||||||
|
f"SELECT id FROM memories "
|
||||||
|
f"WHERE id IN ({placeholders_v}) "
|
||||||
|
f" AND (event_id IS NULL OR event_id BETWEEN ? AND ?)",
|
||||||
|
(*vec_ids, vec_origin, vec_head),
|
||||||
|
).fetchall()
|
||||||
|
}
|
||||||
|
vec_hits = [h for h in vec_hits if h["memory_id"] in in_range]
|
||||||
vec_rank_by_id: dict[int, int] = {
|
vec_rank_by_id: dict[int, int] = {
|
||||||
hit["memory_id"]: rank for rank, hit in enumerate(vec_hits)
|
hit["memory_id"]: rank for rank, hit in enumerate(vec_hits)
|
||||||
}
|
}
|
||||||
|
|||||||
+10
-1
@@ -71,18 +71,27 @@ def _read_recent_meanwhile_dialogue(
|
|||||||
that already match — avoids an unbounded scan as ``event_log``
|
that already match — avoids an unbounded scan as ``event_log``
|
||||||
grows. The user-side rows match on chat_id only since they aren't
|
grows. The user-side rows match on chat_id only since they aren't
|
||||||
tagged with a scene id (they ride the chat-wide log).
|
tagged with a scene id (they ride the chat-wide log).
|
||||||
|
|
||||||
|
T113: clamp by the active branch's ``[origin, head]`` event-id range
|
||||||
|
so meanwhile prompt context respects the user's current branch.
|
||||||
|
Bootstrap-main and "no active branch" both fall through to ``(0,
|
||||||
|
BIG_INT)`` — no functional change for the metadata-only Phase 4 era.
|
||||||
"""
|
"""
|
||||||
|
from chat.state.branches import active_branch_event_ids
|
||||||
|
|
||||||
|
origin, head = active_branch_event_ids(conn)
|
||||||
cur = conn.execute(
|
cur = conn.execute(
|
||||||
"SELECT id, kind, payload_json FROM event_log "
|
"SELECT id, kind, payload_json FROM event_log "
|
||||||
"WHERE kind IN ('user_turn', 'user_turn_edit', 'assistant_turn') "
|
"WHERE kind IN ('user_turn', 'user_turn_edit', 'assistant_turn') "
|
||||||
" AND superseded_by IS NULL AND hidden = 0 "
|
" AND superseded_by IS NULL AND hidden = 0 "
|
||||||
|
" AND id BETWEEN ? AND ? "
|
||||||
" AND json_extract(payload_json, '$.chat_id') = ? "
|
" AND json_extract(payload_json, '$.chat_id') = ? "
|
||||||
" AND ("
|
" AND ("
|
||||||
" kind IN ('user_turn', 'user_turn_edit') "
|
" kind IN ('user_turn', 'user_turn_edit') "
|
||||||
" OR json_extract(payload_json, '$.meanwhile_scene_id') = ?"
|
" OR json_extract(payload_json, '$.meanwhile_scene_id') = ?"
|
||||||
" ) "
|
" ) "
|
||||||
"ORDER BY id DESC LIMIT ?",
|
"ORDER BY id DESC LIMIT ?",
|
||||||
(chat_id, scene_id, limit),
|
(origin, head, chat_id, scene_id, limit),
|
||||||
)
|
)
|
||||||
rows = cur.fetchall()
|
rows = cur.fetchall()
|
||||||
rows.reverse()
|
rows.reverse()
|
||||||
|
|||||||
@@ -812,6 +812,14 @@ async def post_turn(
|
|||||||
payload={
|
payload={
|
||||||
"event_id": transition.event_id,
|
"event_id": transition.event_id,
|
||||||
"started_at": chat.get("time"),
|
"started_at": chat.get("time"),
|
||||||
|
# T114.1: back-reference to the assistant_turn that
|
||||||
|
# triggered this transition. Regenerate uses this
|
||||||
|
# to roll back lifecycle transitions when the turn
|
||||||
|
# is superseded. Forward-only — older events
|
||||||
|
# without this field are skipped by rollback.
|
||||||
|
"triggered_by_assistant_turn_id": (
|
||||||
|
primary_assistant_event_id
|
||||||
|
),
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
elif transition.new_status == "completed":
|
elif transition.new_status == "completed":
|
||||||
@@ -821,6 +829,10 @@ async def post_turn(
|
|||||||
payload={
|
payload={
|
||||||
"event_id": transition.event_id,
|
"event_id": transition.event_id,
|
||||||
"completed_at": chat.get("time"),
|
"completed_at": chat.get("time"),
|
||||||
|
# T114.1: back-reference (see above).
|
||||||
|
"triggered_by_assistant_turn_id": (
|
||||||
|
primary_assistant_event_id
|
||||||
|
),
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
# Run promotion inline so the artifact-emitting events
|
# Run promotion inline so the artifact-emitting events
|
||||||
@@ -842,6 +854,10 @@ async def post_turn(
|
|||||||
payload={
|
payload={
|
||||||
"event_id": transition.event_id,
|
"event_id": transition.event_id,
|
||||||
"completed_at": chat.get("time"),
|
"completed_at": chat.get("time"),
|
||||||
|
# T114.1: back-reference (see above).
|
||||||
|
"triggered_by_assistant_turn_id": (
|
||||||
|
primary_assistant_event_id
|
||||||
|
),
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
# Any other ``new_status`` value falls through silently —
|
# Any other ``new_status`` value falls through silently —
|
||||||
|
|||||||
@@ -522,6 +522,8 @@ Written per witness when a scene closes. Different details, different interpreta
|
|||||||
|
|
||||||
**Status: shipped 2026-04-27** (T88–T102, 15 tasks across 8 waves; +70 tests). See "Phase 4 status" in CLAUDE.md for the per-task breakdown. Vector retrieval shipped via pure-Python cosine over a JSON-blob embeddings table (sqlite-vec deferred — host Python lacks loadable extensions); branching is data-model + drawer UI; significance review, hide-from-view soft delete, surgical delete with cascade preview, snapshot UX, and cross-chat search all surface from the drawer or top-bar.
|
**Status: shipped 2026-04-27** (T88–T102, 15 tasks across 8 waves; +70 tests). See "Phase 4 status" in CLAUDE.md for the per-task breakdown. Vector retrieval shipped via pure-Python cosine over a JSON-blob embeddings table (sqlite-vec deferred — host Python lacks loadable extensions); branching is data-model + drawer UI; significance review, hide-from-view soft delete, surgical delete with cascade preview, snapshot UX, and cross-chat search all surface from the drawer or top-bar.
|
||||||
|
|
||||||
|
**Phase 4.5 cleanup: shipped 2026-04-27** (T103–T118, 13 of 14 planned tasks; T115 sqlite-vec swap deferred to Phase 5 due to host Python lacking `enable_load_extension`; +~44 tests; schema baseline now 14). See "Phase 4.5 status" in CLAUDE.md for the per-task breakdown — notable shipped: real embedding model swap path (`LLMClient.embed()` + `--re-embed-all`), branching read-side filter (`active_branch_event_ids`), regenerate lifecycle rollback (`event_status_reverted`), FTS snippet highlighting + deep-link to turn (`memories.event_id`), bulk significance re-rate.
|
||||||
|
|
||||||
- Vector retrieval (sqlite-vss or sqlite-vec).
|
- Vector retrieval (sqlite-vss or sqlite-vec).
|
||||||
- Branching UI.
|
- Branching UI.
|
||||||
- Drawer-edit on every field.
|
- Drawer-edit on every field.
|
||||||
|
|||||||
@@ -0,0 +1,383 @@
|
|||||||
|
"""Structured test-fixture builder for ``MockLLMClient`` canned queues.
|
||||||
|
|
||||||
|
Phase 4.5 (T116) carry-over from Phase 3. The turn-flow tests in
|
||||||
|
``test_turn_flow.py``, ``test_meanwhile_turn_flow.py``,
|
||||||
|
``test_phase3_integration.py``, and ``test_phase4_integration.py`` used
|
||||||
|
to construct ``MockLLMClient`` canned-response queues as raw positional
|
||||||
|
lists of pre-encoded JSON strings. That worked, but every time a new
|
||||||
|
classifier call landed in a code path the tests had to be patched in
|
||||||
|
many places at the right index — easy to mis-position, hard to read.
|
||||||
|
|
||||||
|
This module ships :class:`CannedQueue`, a fluent builder that lets a
|
||||||
|
test declare its classifier expectations by **name** and **order** of
|
||||||
|
call, not by index into a brittle list. Each method appends one item
|
||||||
|
to the queue and returns ``self`` for chaining; ``build()`` JSON-encodes
|
||||||
|
the items and produces the flat ``list[str]`` that
|
||||||
|
``MockLLMClient(canned=...)`` expects.
|
||||||
|
|
||||||
|
Usage
|
||||||
|
-----
|
||||||
|
|
||||||
|
>>> from tests.fixtures import CannedQueue
|
||||||
|
>>> from chat.llm.mock import MockLLMClient
|
||||||
|
>>> canned = (
|
||||||
|
... CannedQueue()
|
||||||
|
... .parse_turn(segments=[{"kind": "dialogue", "text": "hello"}])
|
||||||
|
... .narrative("Hi there.")
|
||||||
|
... .state_update()
|
||||||
|
... .state_update()
|
||||||
|
... .build()
|
||||||
|
... )
|
||||||
|
>>> mock = MockLLMClient(canned=canned)
|
||||||
|
|
||||||
|
Each method maps to a single classifier (or stream) call that the turn
|
||||||
|
flow makes, in the order the production code makes them. Picking the
|
||||||
|
right method for the slot you need keeps the test readable and lets the
|
||||||
|
builder pin sensible defaults for the fields tests don't care about.
|
||||||
|
|
||||||
|
Migration template
|
||||||
|
------------------
|
||||||
|
|
||||||
|
To migrate a positional canned-array test:
|
||||||
|
|
||||||
|
1. Identify each slot in the existing array and what classifier it
|
||||||
|
feeds. Comments above the array often spell this out — start there.
|
||||||
|
2. Replace each slot with the matching :class:`CannedQueue` method:
|
||||||
|
|
||||||
|
- ``json.dumps({"segments": [...]})`` → ``.parse_turn(segments=...)``
|
||||||
|
- bare narrative string → ``.narrative("...")``
|
||||||
|
- zero-state JSON → ``.state_update()`` (defaults are zeros)
|
||||||
|
- ``json.dumps({"addressee_id": ...})`` → ``.detect_addressee(...)``
|
||||||
|
- ``json.dumps({"should_interject": ...})`` → ``.detect_interjection(...)``
|
||||||
|
- ``json.dumps({"should_close": ...})`` → ``.detect_scene_close(...)``
|
||||||
|
- ``json.dumps({"transitions": [...]})`` → ``.detect_event_transitions(...)``
|
||||||
|
- per-POV summary JSON → ``.summarize_scene_pov(summary=...)``
|
||||||
|
3. End with ``.build()`` and pass that to
|
||||||
|
``MockLLMClient(canned=...)``. The mock's contract is unchanged.
|
||||||
|
|
||||||
|
Notes on streams
|
||||||
|
----------------
|
||||||
|
|
||||||
|
``MockLLMClient.stream`` and ``MockLLMClient.generate`` share one queue
|
||||||
|
— each pop is one entry, regardless of whether the production code
|
||||||
|
streams the response or generates it whole. The narrative service
|
||||||
|
streams; classifier services generate. The builder treats both the same:
|
||||||
|
``narrative()`` appends a raw string, the classifier methods append
|
||||||
|
JSON-encoded dicts. Both end up in the same flat ``list[str]`` that the
|
||||||
|
mock pops from in order.
|
||||||
|
|
||||||
|
The remaining tests in the suite (about 30 across the four files
|
||||||
|
mentioned above) still use positional arrays — Phase 5 work to migrate
|
||||||
|
the rest. New tests should prefer this builder.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
|
||||||
|
class CannedQueue:
|
||||||
|
"""Fluent builder for ``MockLLMClient`` canned-response queues.
|
||||||
|
|
||||||
|
Each method appends one item to an internal queue and returns
|
||||||
|
``self`` for chaining. ``build()`` returns the flat ``list[str]``
|
||||||
|
suitable for ``MockLLMClient(canned=...)``.
|
||||||
|
|
||||||
|
The queue holds either ``dict`` (JSON-encoded at ``build()`` time)
|
||||||
|
or ``str`` (passed through verbatim — used for narrative streams).
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self._queue: list[Any] = []
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
# Narrative stream — bare string, no JSON wrapping.
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
|
||||||
|
def narrative(self, text: str) -> "CannedQueue":
|
||||||
|
"""Append one streaming narrative response.
|
||||||
|
|
||||||
|
``MockLLMClient.stream`` pops the next entry from the same queue
|
||||||
|
as ``generate`` — a bare string is what the streaming bot beat
|
||||||
|
consumes. Use one ``narrative()`` per assistant beat (primary,
|
||||||
|
and optionally an interjection / second beat).
|
||||||
|
"""
|
||||||
|
self._queue.append(text)
|
||||||
|
return self
|
||||||
|
|
||||||
|
def raw(self, value: str) -> "CannedQueue":
|
||||||
|
"""Append a raw string (escape hatch for non-classifier calls).
|
||||||
|
|
||||||
|
Most tests should reach for the named helpers — this is here
|
||||||
|
for one-offs the builder doesn't model yet.
|
||||||
|
"""
|
||||||
|
self._queue.append(value)
|
||||||
|
return self
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
# Turn parser — splits user prose into segments.
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
|
||||||
|
def parse_turn(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
segments: list[dict] | None = None,
|
||||||
|
intent: str = "narrative",
|
||||||
|
landing_state_hint: str = "",
|
||||||
|
**rest: Any,
|
||||||
|
) -> "CannedQueue":
|
||||||
|
"""Append one ``parse_turn`` classifier response.
|
||||||
|
|
||||||
|
``intent`` defaults to ``"narrative"``; pass ``"skip_elision"``
|
||||||
|
or ``"skip_jump"`` to exercise the natural-language skip paths.
|
||||||
|
``landing_state_hint`` carries the residual descriptor for
|
||||||
|
elision skips and is otherwise ignored.
|
||||||
|
"""
|
||||||
|
payload: dict[str, Any] = {
|
||||||
|
"segments": segments if segments is not None else [],
|
||||||
|
"intent": intent,
|
||||||
|
"landing_state_hint": landing_state_hint,
|
||||||
|
}
|
||||||
|
payload.update(rest)
|
||||||
|
self._queue.append(payload)
|
||||||
|
return self
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
# Multi-entity addressee classifier (T74.1).
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
|
||||||
|
def detect_addressee(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
addressee_id: str,
|
||||||
|
confidence: str = "medium",
|
||||||
|
reason: str = "",
|
||||||
|
**rest: Any,
|
||||||
|
) -> "CannedQueue":
|
||||||
|
"""Append one ``detect_addressee`` classifier response."""
|
||||||
|
payload: dict[str, Any] = {
|
||||||
|
"addressee_id": addressee_id,
|
||||||
|
"confidence": confidence,
|
||||||
|
"reason": reason,
|
||||||
|
}
|
||||||
|
payload.update(rest)
|
||||||
|
self._queue.append(payload)
|
||||||
|
return self
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
# State-update — one per directed edge per turn.
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
|
||||||
|
def state_update(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
affinity_delta: int = 0,
|
||||||
|
trust_delta: int = 0,
|
||||||
|
knowledge_facts: list | None = None,
|
||||||
|
**rest: Any,
|
||||||
|
) -> "CannedQueue":
|
||||||
|
"""Append one ``apply_state_update`` classifier response.
|
||||||
|
|
||||||
|
Defaults to a benign zero-delta payload — tests that don't care
|
||||||
|
about state mutations can call this without arguments. One call
|
||||||
|
is required per directed edge that fires after the assistant
|
||||||
|
beat (e.g. single-bot non-guest turn = 2 calls; multi-bot guest
|
||||||
|
turn = 6 calls).
|
||||||
|
"""
|
||||||
|
payload: dict[str, Any] = {
|
||||||
|
"affinity_delta": affinity_delta,
|
||||||
|
"trust_delta": trust_delta,
|
||||||
|
"knowledge_facts": (
|
||||||
|
knowledge_facts if knowledge_facts is not None else []
|
||||||
|
),
|
||||||
|
}
|
||||||
|
payload.update(rest)
|
||||||
|
self._queue.append(payload)
|
||||||
|
return self
|
||||||
|
|
||||||
|
def zero_state(self) -> "CannedQueue":
|
||||||
|
"""Alias for ``state_update()`` with all defaults — matches the
|
||||||
|
``_zero_state()`` helper in existing tests.
|
||||||
|
"""
|
||||||
|
return self.state_update()
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
# Interjection (T74.2) — silent witness chimes in.
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
|
||||||
|
def detect_interjection(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
should_interject: bool,
|
||||||
|
reason: str = "",
|
||||||
|
**rest: Any,
|
||||||
|
) -> "CannedQueue":
|
||||||
|
"""Append one ``detect_interjection`` classifier response."""
|
||||||
|
payload: dict[str, Any] = {
|
||||||
|
"should_interject": should_interject,
|
||||||
|
"reason": reason,
|
||||||
|
}
|
||||||
|
payload.update(rest)
|
||||||
|
self._queue.append(payload)
|
||||||
|
return self
|
||||||
|
|
||||||
|
def detect_interjection_targeted(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
targeted: bool,
|
||||||
|
target_id: str | None = None,
|
||||||
|
reason: str = "",
|
||||||
|
**rest: Any,
|
||||||
|
) -> "CannedQueue":
|
||||||
|
"""Append one targeted-interjection classifier response."""
|
||||||
|
payload: dict[str, Any] = {
|
||||||
|
"targeted": targeted,
|
||||||
|
"target_id": target_id,
|
||||||
|
"reason": reason,
|
||||||
|
}
|
||||||
|
payload.update(rest)
|
||||||
|
self._queue.append(payload)
|
||||||
|
return self
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
# Scene-close detector (T26).
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
|
||||||
|
def detect_scene_close(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
should_close: bool,
|
||||||
|
reason: str = "",
|
||||||
|
**rest: Any,
|
||||||
|
) -> "CannedQueue":
|
||||||
|
"""Append one ``detect_scene_close`` classifier response."""
|
||||||
|
payload: dict[str, Any] = {
|
||||||
|
"should_close": should_close,
|
||||||
|
"reason": reason,
|
||||||
|
}
|
||||||
|
payload.update(rest)
|
||||||
|
self._queue.append(payload)
|
||||||
|
return self
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
# Event lifecycle (T52, T61) — per-turn transitions.
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
|
||||||
|
def detect_event_transitions(
|
||||||
|
self,
|
||||||
|
transitions: list[dict] | None = None,
|
||||||
|
) -> "CannedQueue":
|
||||||
|
"""Append one ``detect_event_transitions`` classifier response.
|
||||||
|
|
||||||
|
``transitions`` is a list of ``{"event_id": ..., "new_status":
|
||||||
|
"active"|"completed"|"cancelled", "reason": ...}`` dicts. Pass
|
||||||
|
an empty list (or omit the argument) to assert that the call
|
||||||
|
ran but produced no transitions; pass ``None`` for an empty
|
||||||
|
list with the same shape.
|
||||||
|
|
||||||
|
Note: when no events are seeded, ``detect_event_transitions``
|
||||||
|
short-circuits without an LLM call — in that case do NOT append
|
||||||
|
this slot.
|
||||||
|
"""
|
||||||
|
payload = {"transitions": transitions if transitions is not None else []}
|
||||||
|
self._queue.append(payload)
|
||||||
|
return self
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
# Per-POV scene summary (used after scene close).
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
|
||||||
|
def summarize_scene_pov(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
summary: str,
|
||||||
|
knowledge_facts: list | None = None,
|
||||||
|
relationship_summary: str = "",
|
||||||
|
**rest: Any,
|
||||||
|
) -> "CannedQueue":
|
||||||
|
"""Append one per-POV scene-summary response.
|
||||||
|
|
||||||
|
Used by ``apply_scene_close_summary`` — one call per witness
|
||||||
|
once a scene closes.
|
||||||
|
"""
|
||||||
|
payload: dict[str, Any] = {
|
||||||
|
"summary": summary,
|
||||||
|
"knowledge_facts": (
|
||||||
|
knowledge_facts if knowledge_facts is not None else []
|
||||||
|
),
|
||||||
|
"relationship_summary": relationship_summary,
|
||||||
|
}
|
||||||
|
payload.update(rest)
|
||||||
|
self._queue.append(payload)
|
||||||
|
return self
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
# Thread detection (Phase 3 §3.3).
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
|
||||||
|
def detect_threads(
|
||||||
|
self,
|
||||||
|
candidates: list[dict] | None = None,
|
||||||
|
) -> "CannedQueue":
|
||||||
|
"""Append one ``detect_threads`` classifier response.
|
||||||
|
|
||||||
|
``candidates`` is a list of ``{"action": "open"|"update",
|
||||||
|
"title": ..., "summary": ..., "existing_thread_id": ...}`` dicts.
|
||||||
|
"""
|
||||||
|
payload = {"candidates": candidates if candidates is not None else []}
|
||||||
|
self._queue.append(payload)
|
||||||
|
return self
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
# Meanwhile digest — narrative summary of what happened off-screen.
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
|
||||||
|
def meanwhile_digest(self, summary: str) -> "CannedQueue":
|
||||||
|
"""Append one meanwhile-digest narrative response.
|
||||||
|
|
||||||
|
The digest service streams the digest as plain text (not JSON)
|
||||||
|
so this is a thin wrapper over ``narrative``/``raw`` for
|
||||||
|
readability at the call site.
|
||||||
|
"""
|
||||||
|
self._queue.append(summary)
|
||||||
|
return self
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
# Significance scorer (background worker; rarely hit in unit tests
|
||||||
|
# but available for completeness).
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
|
||||||
|
def score_significance(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
score: float = 0.0,
|
||||||
|
reason: str = "",
|
||||||
|
**rest: Any,
|
||||||
|
) -> "CannedQueue":
|
||||||
|
"""Append one significance-scoring classifier response."""
|
||||||
|
payload: dict[str, Any] = {"score": score, "reason": reason}
|
||||||
|
payload.update(rest)
|
||||||
|
self._queue.append(payload)
|
||||||
|
return self
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
# Build / introspection.
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
|
||||||
|
def build(self) -> list[str]:
|
||||||
|
"""Return the flat ``list[str]`` queue for ``MockLLMClient``.
|
||||||
|
|
||||||
|
Dict items are JSON-encoded; string items are passed through
|
||||||
|
verbatim (so streaming responses retain their raw form).
|
||||||
|
"""
|
||||||
|
out: list[str] = []
|
||||||
|
for item in self._queue:
|
||||||
|
if isinstance(item, str):
|
||||||
|
out.append(item)
|
||||||
|
else:
|
||||||
|
out.append(json.dumps(item))
|
||||||
|
return out
|
||||||
|
|
||||||
|
def __len__(self) -> int:
|
||||||
|
return len(self._queue)
|
||||||
@@ -7,7 +7,13 @@ from chat.db.migrate import apply_migrations
|
|||||||
from chat.eventlog.log import append_event
|
from chat.eventlog.log import append_event
|
||||||
from chat.eventlog.projector import project
|
from chat.eventlog.projector import project
|
||||||
import chat.state.branches # registers handlers
|
import chat.state.branches # registers handlers
|
||||||
from chat.state.branches import active_branch, get_branch, list_branches
|
from chat.state.branches import (
|
||||||
|
_NO_HEAD_CLAMP,
|
||||||
|
active_branch,
|
||||||
|
active_branch_event_ids,
|
||||||
|
get_branch,
|
||||||
|
list_branches,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def test_main_branch_bootstrapped_by_migration(tmp_path):
|
def test_main_branch_bootstrapped_by_migration(tmp_path):
|
||||||
@@ -174,3 +180,83 @@ def test_branch_switched_unknown_name_warns(tmp_path, caplog):
|
|||||||
|
|
||||||
# The unknown name was not inserted as a side effect.
|
# The unknown name was not inserted as a side effect.
|
||||||
assert get_branch(conn, "does_not_exist") is None
|
assert get_branch(conn, "does_not_exist") is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_active_branch_event_ids_bootstrap_main_returns_no_clamp(tmp_path):
|
||||||
|
"""Bootstrap "main" (origin=0, head=0) reads as the no-clamp sentinel.
|
||||||
|
|
||||||
|
Migration 0013 seeds main with both event-id columns at 0; production
|
||||||
|
today never emits ``branch_head_updated`` for main, so head stays at 0
|
||||||
|
even as events accumulate. The helper treats this exact bootstrap
|
||||||
|
state as "all events visible" (lower bound 0, upper bound BIG_INT) so
|
||||||
|
every existing reader stays branch-agnostic until a non-main branch
|
||||||
|
becomes active.
|
||||||
|
"""
|
||||||
|
db = tmp_path / "t.db"
|
||||||
|
apply_migrations(db)
|
||||||
|
with open_db(db) as conn:
|
||||||
|
origin, head = active_branch_event_ids(conn)
|
||||||
|
assert origin == 0
|
||||||
|
assert head == _NO_HEAD_CLAMP
|
||||||
|
|
||||||
|
|
||||||
|
def test_active_branch_event_ids_no_active_branch_falls_through(tmp_path):
|
||||||
|
"""No active branch row at all → defensive ``(0, BIG_INT)``.
|
||||||
|
|
||||||
|
A switch to an unknown branch leaves zero rows with ``is_active=1``;
|
||||||
|
``active_branch`` returns None. The helper must still hand readers a
|
||||||
|
workable range (the full log) so the read pipeline doesn't crash on
|
||||||
|
an inconsistent metadata state.
|
||||||
|
"""
|
||||||
|
db = tmp_path / "t.db"
|
||||||
|
apply_migrations(db)
|
||||||
|
with open_db(db) as conn:
|
||||||
|
# Switching to a nonexistent branch clears is_active flags
|
||||||
|
# without setting any other branch active.
|
||||||
|
append_event(
|
||||||
|
conn,
|
||||||
|
kind="branch_switched",
|
||||||
|
payload={"name": "does_not_exist"},
|
||||||
|
)
|
||||||
|
project(conn)
|
||||||
|
assert active_branch(conn) is None
|
||||||
|
|
||||||
|
origin, head = active_branch_event_ids(conn)
|
||||||
|
assert origin == 0
|
||||||
|
assert head == _NO_HEAD_CLAMP
|
||||||
|
|
||||||
|
|
||||||
|
def test_active_branch_event_ids_returns_actual_range_for_non_main(tmp_path):
|
||||||
|
"""Non-main branches return their literal ``(origin, head)`` window.
|
||||||
|
|
||||||
|
A branch created at origin=10 + bumped to head=20 must surface as
|
||||||
|
(10, 20) so readers' ``BETWEEN`` clamp scopes to that window.
|
||||||
|
"""
|
||||||
|
db = tmp_path / "t.db"
|
||||||
|
apply_migrations(db)
|
||||||
|
with open_db(db) as conn:
|
||||||
|
append_event(
|
||||||
|
conn,
|
||||||
|
kind="branch_created",
|
||||||
|
payload={
|
||||||
|
"name": "experiment",
|
||||||
|
"origin_event_id": 10,
|
||||||
|
"head_event_id": 10,
|
||||||
|
"chat_id": "c1",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
append_event(
|
||||||
|
conn,
|
||||||
|
kind="branch_head_updated",
|
||||||
|
payload={"name": "experiment", "head_event_id": 20},
|
||||||
|
)
|
||||||
|
append_event(
|
||||||
|
conn,
|
||||||
|
kind="branch_switched",
|
||||||
|
payload={"name": "experiment"},
|
||||||
|
)
|
||||||
|
project(conn)
|
||||||
|
|
||||||
|
origin, head = active_branch_event_ids(conn)
|
||||||
|
assert origin == 10
|
||||||
|
assert head == 20
|
||||||
|
|||||||
@@ -129,3 +129,279 @@ def test_list_branches_with_metadata_includes_event_count(tmp_path):
|
|||||||
assert rows["exp"]["origin_event_id"] == 10
|
assert rows["exp"]["origin_event_id"] == 10
|
||||||
assert rows["exp"]["head_event_id"] == 15
|
assert rows["exp"]["head_event_id"] == 15
|
||||||
assert rows["exp"]["event_count"] == 6
|
assert rows["exp"]["event_count"] == 6
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# T113 read-side filter — cross-feature tests.
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
#
|
||||||
|
# These exercise the active-branch event-id clamp through every reader
|
||||||
|
# the spec called out: ``read_recent_dialogue`` (turn_common),
|
||||||
|
# ``_read_recent_dialogue`` (scene_summarize), and ``search_memories``
|
||||||
|
# (memory). They drive the readers via real event-log inserts + branch
|
||||||
|
# switches so the integration is end-to-end.
|
||||||
|
|
||||||
|
|
||||||
|
def _seed_user_turn(conn, chat_id: str, prose: str) -> int:
|
||||||
|
return append_and_apply(
|
||||||
|
conn,
|
||||||
|
kind="user_turn",
|
||||||
|
payload={"chat_id": chat_id, "prose": prose, "segments": []},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_read_recent_dialogue_respects_active_branch_head(tmp_path):
|
||||||
|
"""T113 spec test 1: dialogue reader clamps to active branch head.
|
||||||
|
|
||||||
|
Seed 10 user turns; create a branch with origin=1 + head=5 and switch
|
||||||
|
to it; assert ``read_recent_dialogue`` only returns the first 5
|
||||||
|
turns. (The 5 events with id 6..10 fall outside ``[1, 5]``.)
|
||||||
|
"""
|
||||||
|
from chat.services.turn_common import read_recent_dialogue
|
||||||
|
|
||||||
|
db = tmp_path / "t.db"
|
||||||
|
apply_migrations(db)
|
||||||
|
with open_db(db) as conn:
|
||||||
|
ids = [_seed_user_turn(conn, "c1", f"turn {i}") for i in range(10)]
|
||||||
|
# 5 events visible after the switch.
|
||||||
|
branch_from_event(
|
||||||
|
conn, name="halfway", origin_event_id=ids[0], chat_id="c1"
|
||||||
|
)
|
||||||
|
append_and_apply(
|
||||||
|
conn,
|
||||||
|
kind="branch_head_updated",
|
||||||
|
payload={"name": "halfway", "head_event_id": ids[4]},
|
||||||
|
)
|
||||||
|
switch_active_branch(conn, name="halfway")
|
||||||
|
|
||||||
|
rows = read_recent_dialogue(conn, "c1")
|
||||||
|
# The reader returns oldest-first, so the visible-set is the
|
||||||
|
# first 5 turns.
|
||||||
|
assert len(rows) == 5
|
||||||
|
assert [r["text"] for r in rows] == [f"turn {i}" for i in range(5)]
|
||||||
|
|
||||||
|
|
||||||
|
def test_search_memories_respects_active_branch_head(tmp_path):
|
||||||
|
"""T113 spec test 2: memory search clamps to active branch head via
|
||||||
|
``memories.event_id``. Memories whose projecting event lands outside
|
||||||
|
the clamp drop out of FTS results."""
|
||||||
|
from chat.eventlog.log import append_and_apply as _aa
|
||||||
|
from chat.state.memory import search_memories
|
||||||
|
|
||||||
|
db = tmp_path / "t.db"
|
||||||
|
apply_migrations(db)
|
||||||
|
with open_db(db) as conn:
|
||||||
|
# Two memories projected from real events. The projector handler
|
||||||
|
# stamps memories.event_id from the projecting event's id.
|
||||||
|
ev_a = _aa(
|
||||||
|
conn,
|
||||||
|
kind="memory_written",
|
||||||
|
payload={
|
||||||
|
"owner_id": "host_bot",
|
||||||
|
"chat_id": "c1",
|
||||||
|
"scene_id": 1,
|
||||||
|
"pov_summary": "alpha keyword present",
|
||||||
|
"witness_you": 1,
|
||||||
|
"witness_host": 1,
|
||||||
|
"witness_guest": 0,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
ev_b = _aa(
|
||||||
|
conn,
|
||||||
|
kind="memory_written",
|
||||||
|
payload={
|
||||||
|
"owner_id": "host_bot",
|
||||||
|
"chat_id": "c1",
|
||||||
|
"scene_id": 1,
|
||||||
|
"pov_summary": "alpha keyword present too",
|
||||||
|
"witness_you": 1,
|
||||||
|
"witness_host": 1,
|
||||||
|
"witness_guest": 0,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
# Branch clamps to ev_a only (head = ev_a; ev_b sits past head).
|
||||||
|
branch_from_event(
|
||||||
|
conn, name="early", origin_event_id=ev_a, chat_id="c1"
|
||||||
|
)
|
||||||
|
switch_active_branch(conn, name="early")
|
||||||
|
|
||||||
|
results = search_memories(conn, "host_bot", "host", "alpha")
|
||||||
|
# Only the first memory should surface — the second's event_id
|
||||||
|
# exceeds the active branch head.
|
||||||
|
ids = [r["event_id"] for r in results]
|
||||||
|
assert ev_a in ids
|
||||||
|
assert ev_b not in ids
|
||||||
|
|
||||||
|
|
||||||
|
def test_branch_switch_changes_visible_events(tmp_path):
|
||||||
|
"""T113 spec test 3: switching branches mid-flight changes the read
|
||||||
|
immediately. ``read_recent_dialogue`` re-queries on every call."""
|
||||||
|
from chat.services.turn_common import read_recent_dialogue
|
||||||
|
|
||||||
|
db = tmp_path / "t.db"
|
||||||
|
apply_migrations(db)
|
||||||
|
with open_db(db) as conn:
|
||||||
|
ids = [_seed_user_turn(conn, "c1", f"turn {i}") for i in range(6)]
|
||||||
|
|
||||||
|
branch_from_event(
|
||||||
|
conn, name="early", origin_event_id=ids[0], chat_id="c1"
|
||||||
|
)
|
||||||
|
append_and_apply(
|
||||||
|
conn,
|
||||||
|
kind="branch_head_updated",
|
||||||
|
payload={"name": "early", "head_event_id": ids[2]},
|
||||||
|
)
|
||||||
|
branch_from_event(
|
||||||
|
conn, name="late", origin_event_id=ids[3], chat_id="c1"
|
||||||
|
)
|
||||||
|
append_and_apply(
|
||||||
|
conn,
|
||||||
|
kind="branch_head_updated",
|
||||||
|
payload={"name": "late", "head_event_id": ids[5]},
|
||||||
|
)
|
||||||
|
|
||||||
|
switch_active_branch(conn, name="early")
|
||||||
|
early_rows = [r["text"] for r in read_recent_dialogue(conn, "c1")]
|
||||||
|
assert early_rows == ["turn 0", "turn 1", "turn 2"]
|
||||||
|
|
||||||
|
switch_active_branch(conn, name="late")
|
||||||
|
late_rows = [r["text"] for r in read_recent_dialogue(conn, "c1")]
|
||||||
|
assert late_rows == ["turn 3", "turn 4", "turn 5"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_main_branch_with_head_zero_returns_empty(tmp_path):
|
||||||
|
"""T113 spec test 4: a non-main branch with head=0 returns empty.
|
||||||
|
|
||||||
|
The bootstrap-main sentinel only fires for ``name=="main", origin=0,
|
||||||
|
head=0``. A different branch parked at ``origin=0, head=0`` is not a
|
||||||
|
sentinel and the ``BETWEEN 0 AND 0`` clamp filters out every real
|
||||||
|
event_log row (rowids start at 1)."""
|
||||||
|
from chat.services.turn_common import read_recent_dialogue
|
||||||
|
|
||||||
|
db = tmp_path / "t.db"
|
||||||
|
apply_migrations(db)
|
||||||
|
with open_db(db) as conn:
|
||||||
|
# Need a real event_log row id 1+ so the clamp's "exclude 0" actually
|
||||||
|
# has something to exclude — otherwise we trivially return [].
|
||||||
|
_seed_user_turn(conn, "c1", "turn 0")
|
||||||
|
|
||||||
|
# Force-create a branch at origin=0, head=0 (NOT main). This is an
|
||||||
|
# artificial state — production never produces it — but it's the
|
||||||
|
# cleanest way to drive the documented edge case.
|
||||||
|
append_and_apply(
|
||||||
|
conn,
|
||||||
|
kind="branch_created",
|
||||||
|
payload={
|
||||||
|
"name": "stub",
|
||||||
|
"origin_event_id": 0,
|
||||||
|
"head_event_id": 0,
|
||||||
|
"chat_id": "c1",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
switch_active_branch(conn, name="stub")
|
||||||
|
|
||||||
|
rows = read_recent_dialogue(conn, "c1")
|
||||||
|
assert rows == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_no_active_branch_falls_through_to_all_events(tmp_path):
|
||||||
|
"""T113 spec test 5: with no active branch (e.g. a switch to an
|
||||||
|
unknown name cleared all is_active flags), readers see the full log
|
||||||
|
via the ``(0, BIG_INT)`` defensive default."""
|
||||||
|
from chat.services.turn_common import read_recent_dialogue
|
||||||
|
|
||||||
|
db = tmp_path / "t.db"
|
||||||
|
apply_migrations(db)
|
||||||
|
with open_db(db) as conn:
|
||||||
|
for i in range(3):
|
||||||
|
_seed_user_turn(conn, "c1", f"turn {i}")
|
||||||
|
|
||||||
|
# Switching to an unknown branch leaves zero rows with is_active=1.
|
||||||
|
append_and_apply(
|
||||||
|
conn,
|
||||||
|
kind="branch_switched",
|
||||||
|
payload={"name": "missing"},
|
||||||
|
)
|
||||||
|
from chat.state.branches import active_branch as _ab
|
||||||
|
|
||||||
|
assert _ab(conn) is None
|
||||||
|
|
||||||
|
rows = read_recent_dialogue(conn, "c1")
|
||||||
|
assert [r["text"] for r in rows] == ["turn 0", "turn 1", "turn 2"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_scene_summarize_read_recent_dialogue_respects_branch(tmp_path):
|
||||||
|
"""T113: ``scene_summarize._read_recent_dialogue`` (the scene-close
|
||||||
|
summary input) also clamps to the active branch range."""
|
||||||
|
from chat.services.scene_summarize import _read_recent_dialogue
|
||||||
|
|
||||||
|
db = tmp_path / "t.db"
|
||||||
|
apply_migrations(db)
|
||||||
|
with open_db(db) as conn:
|
||||||
|
ids = [_seed_user_turn(conn, "c1", f"turn {i}") for i in range(6)]
|
||||||
|
|
||||||
|
branch_from_event(
|
||||||
|
conn, name="early", origin_event_id=ids[0], chat_id="c1"
|
||||||
|
)
|
||||||
|
append_and_apply(
|
||||||
|
conn,
|
||||||
|
kind="branch_head_updated",
|
||||||
|
payload={"name": "early", "head_event_id": ids[2]},
|
||||||
|
)
|
||||||
|
switch_active_branch(conn, name="early")
|
||||||
|
|
||||||
|
rows = _read_recent_dialogue(conn, "c1")
|
||||||
|
assert [r["text"] for r in rows] == ["turn 0", "turn 1", "turn 2"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_meanwhile_dialogue_reader_respects_branch(tmp_path):
|
||||||
|
"""T113: meanwhile prompt-context reader also clamps to the active
|
||||||
|
branch. The meanwhile reader filters by ``meanwhile_scene_id``; the
|
||||||
|
branch filter is composed on top of that filter."""
|
||||||
|
from chat.web.meanwhile import _read_recent_meanwhile_dialogue
|
||||||
|
|
||||||
|
db = tmp_path / "t.db"
|
||||||
|
apply_migrations(db)
|
||||||
|
with open_db(db) as conn:
|
||||||
|
# Seed user turns + meanwhile assistant turns interleaved so the
|
||||||
|
# branch-id clamp lands across both kinds.
|
||||||
|
u1 = _seed_user_turn(conn, "c1", "u1")
|
||||||
|
a1 = append_and_apply(
|
||||||
|
conn,
|
||||||
|
kind="assistant_turn",
|
||||||
|
payload={
|
||||||
|
"chat_id": "c1",
|
||||||
|
"speaker_id": "host",
|
||||||
|
"text": "a1",
|
||||||
|
"meanwhile_scene_id": 7,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
# Past-head turn should NOT appear once we switch to ``early``.
|
||||||
|
a2 = append_and_apply(
|
||||||
|
conn,
|
||||||
|
kind="assistant_turn",
|
||||||
|
payload={
|
||||||
|
"chat_id": "c1",
|
||||||
|
"speaker_id": "guest",
|
||||||
|
"text": "a2",
|
||||||
|
"meanwhile_scene_id": 7,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
branch_from_event(
|
||||||
|
conn, name="early", origin_event_id=u1, chat_id="c1"
|
||||||
|
)
|
||||||
|
append_and_apply(
|
||||||
|
conn,
|
||||||
|
kind="branch_head_updated",
|
||||||
|
payload={"name": "early", "head_event_id": a1},
|
||||||
|
)
|
||||||
|
switch_active_branch(conn, name="early")
|
||||||
|
|
||||||
|
rows = _read_recent_meanwhile_dialogue(conn, "c1", scene_id=7)
|
||||||
|
texts = [r["text"] for r in rows]
|
||||||
|
assert "a1" in texts
|
||||||
|
assert "a2" not in texts
|
||||||
|
# Suppress the "unused" linter warning while keeping the binding
|
||||||
|
# readable for the test narrative.
|
||||||
|
_ = a2
|
||||||
|
|||||||
@@ -233,3 +233,91 @@ def test_list_active_events_filters_to_planned_and_active(tmp_path):
|
|||||||
|
|
||||||
cancelled = list_events_in_status(conn, "chat_bot_a", "cancelled")
|
cancelled = list_events_in_status(conn, "chat_bot_a", "cancelled")
|
||||||
assert [e["event_id"] for e in cancelled] == ["evt_canx"]
|
assert [e["event_id"] for e in cancelled] == ["evt_canx"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_event_status_reverted_returns_to_prior_status(tmp_path):
|
||||||
|
"""T114.2: ``event_status_reverted`` rolls a row back to ``prior_status``.
|
||||||
|
|
||||||
|
Unlike the forward transitions, this projector handler is
|
||||||
|
unconditional — its sole purpose is to undo a transition, including
|
||||||
|
reverting from a terminal status (completed/cancelled) back to a
|
||||||
|
non-terminal one.
|
||||||
|
|
||||||
|
Three round-trips covered:
|
||||||
|
- completed → active (rollback of an event_completed)
|
||||||
|
- active → planned (rollback of an event_started)
|
||||||
|
- cancelled → active (rollback of an event_cancelled)
|
||||||
|
"""
|
||||||
|
db = tmp_path / "t.db"
|
||||||
|
apply_migrations(db)
|
||||||
|
with open_db(db) as conn:
|
||||||
|
_seed_chat(conn)
|
||||||
|
append_event(
|
||||||
|
conn,
|
||||||
|
kind="event_planned",
|
||||||
|
payload={
|
||||||
|
"event_id": "evt_revert",
|
||||||
|
"chat_id": "chat_bot_a",
|
||||||
|
"kind": "date_at_park",
|
||||||
|
"props": {},
|
||||||
|
"planned_for": "2026-04-30T18:00:00+00:00",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
append_event(
|
||||||
|
conn,
|
||||||
|
kind="event_started",
|
||||||
|
payload={
|
||||||
|
"event_id": "evt_revert",
|
||||||
|
"started_at": "2026-04-30T18:01:00+00:00",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
append_event(
|
||||||
|
conn,
|
||||||
|
kind="event_completed",
|
||||||
|
payload={
|
||||||
|
"event_id": "evt_revert",
|
||||||
|
"completed_at": "2026-04-30T20:00:00+00:00",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
project(conn)
|
||||||
|
|
||||||
|
ev = get_event(conn, "evt_revert")
|
||||||
|
assert ev is not None
|
||||||
|
assert ev["status"] == "completed"
|
||||||
|
|
||||||
|
# Revert from completed → active.
|
||||||
|
append_and_apply(
|
||||||
|
conn,
|
||||||
|
kind="event_status_reverted",
|
||||||
|
payload={"event_id": "evt_revert", "prior_status": "active"},
|
||||||
|
)
|
||||||
|
ev = get_event(conn, "evt_revert")
|
||||||
|
assert ev["status"] == "active"
|
||||||
|
|
||||||
|
# Revert from active → planned.
|
||||||
|
append_and_apply(
|
||||||
|
conn,
|
||||||
|
kind="event_status_reverted",
|
||||||
|
payload={"event_id": "evt_revert", "prior_status": "planned"},
|
||||||
|
)
|
||||||
|
ev = get_event(conn, "evt_revert")
|
||||||
|
assert ev["status"] == "planned"
|
||||||
|
|
||||||
|
# Forward to cancelled, then revert from cancelled → active.
|
||||||
|
append_and_apply(
|
||||||
|
conn,
|
||||||
|
kind="event_cancelled",
|
||||||
|
payload={
|
||||||
|
"event_id": "evt_revert",
|
||||||
|
"completed_at": "2026-04-30T20:30:00+00:00",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
ev = get_event(conn, "evt_revert")
|
||||||
|
assert ev["status"] == "cancelled"
|
||||||
|
append_and_apply(
|
||||||
|
conn,
|
||||||
|
kind="event_status_reverted",
|
||||||
|
payload={"event_id": "evt_revert", "prior_status": "active"},
|
||||||
|
)
|
||||||
|
ev = get_event(conn, "evt_revert")
|
||||||
|
assert ev["status"] == "active"
|
||||||
|
|||||||
@@ -0,0 +1,140 @@
|
|||||||
|
"""Sanity tests for :mod:`tests.fixtures` — the structured CannedQueue
|
||||||
|
builder for ``MockLLMClient`` (T116).
|
||||||
|
|
||||||
|
The builder is a thin shaping layer over JSON dicts; these tests pin
|
||||||
|
the JSON shapes and the ``MockLLMClient`` round-trip so nothing
|
||||||
|
silently regresses if a default field name or shape gets renamed.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from chat.llm.mock import MockLLMClient
|
||||||
|
from tests.fixtures import CannedQueue
|
||||||
|
|
||||||
|
|
||||||
|
def test_canned_queue_build_emits_expected_shapes():
|
||||||
|
"""Each builder method emits the JSON shape its classifier consumer
|
||||||
|
expects. The narrative slot is a bare string (stream).
|
||||||
|
"""
|
||||||
|
canned = (
|
||||||
|
CannedQueue()
|
||||||
|
.parse_turn(segments=[{"kind": "dialogue", "text": "hello"}])
|
||||||
|
.detect_addressee(addressee_id="bot_a", reason="host")
|
||||||
|
.narrative("Hi there.")
|
||||||
|
.state_update()
|
||||||
|
.state_update(affinity_delta=1, trust_delta=2)
|
||||||
|
.detect_interjection(should_interject=False, reason="calm")
|
||||||
|
.detect_event_transitions(
|
||||||
|
[{"event_id": "evt_1", "new_status": "active", "reason": "they arrived"}]
|
||||||
|
)
|
||||||
|
.detect_scene_close(should_close=False, reason="no signal")
|
||||||
|
.summarize_scene_pov(summary="BotA noticed the day winding down.")
|
||||||
|
.detect_threads(
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"action": "open",
|
||||||
|
"title": "Maya's job hunt",
|
||||||
|
"summary": "Maya is looking for a new job",
|
||||||
|
"existing_thread_id": None,
|
||||||
|
}
|
||||||
|
]
|
||||||
|
)
|
||||||
|
.build()
|
||||||
|
)
|
||||||
|
|
||||||
|
# All slots are strings (the MockLLMClient pops strings).
|
||||||
|
assert all(isinstance(slot, str) for slot in canned)
|
||||||
|
assert len(canned) == 10
|
||||||
|
|
||||||
|
# Slot 0: parse_turn — defaults intent="narrative".
|
||||||
|
parse = json.loads(canned[0])
|
||||||
|
assert parse["segments"] == [{"kind": "dialogue", "text": "hello"}]
|
||||||
|
assert parse["intent"] == "narrative"
|
||||||
|
assert parse["landing_state_hint"] == ""
|
||||||
|
|
||||||
|
# Slot 1: detect_addressee.
|
||||||
|
addr = json.loads(canned[1])
|
||||||
|
assert addr["addressee_id"] == "bot_a"
|
||||||
|
assert addr["confidence"] == "medium"
|
||||||
|
assert addr["reason"] == "host"
|
||||||
|
|
||||||
|
# Slot 2: narrative — bare string, NOT JSON.
|
||||||
|
assert canned[2] == "Hi there."
|
||||||
|
with pytest.raises(json.JSONDecodeError):
|
||||||
|
json.loads(canned[2])
|
||||||
|
|
||||||
|
# Slot 3: state_update with all defaults — zero deltas, no facts.
|
||||||
|
su0 = json.loads(canned[3])
|
||||||
|
assert su0 == {"affinity_delta": 0, "trust_delta": 0, "knowledge_facts": []}
|
||||||
|
|
||||||
|
# Slot 4: state_update with custom deltas.
|
||||||
|
su1 = json.loads(canned[4])
|
||||||
|
assert su1["affinity_delta"] == 1
|
||||||
|
assert su1["trust_delta"] == 2
|
||||||
|
assert su1["knowledge_facts"] == []
|
||||||
|
|
||||||
|
# Slot 5: detect_interjection.
|
||||||
|
interj = json.loads(canned[5])
|
||||||
|
assert interj == {"should_interject": False, "reason": "calm"}
|
||||||
|
|
||||||
|
# Slot 6: detect_event_transitions.
|
||||||
|
transitions = json.loads(canned[6])
|
||||||
|
assert transitions["transitions"][0]["event_id"] == "evt_1"
|
||||||
|
assert transitions["transitions"][0]["new_status"] == "active"
|
||||||
|
|
||||||
|
# Slot 7: detect_scene_close.
|
||||||
|
close = json.loads(canned[7])
|
||||||
|
assert close == {"should_close": False, "reason": "no signal"}
|
||||||
|
|
||||||
|
# Slot 8: summarize_scene_pov.
|
||||||
|
pov = json.loads(canned[8])
|
||||||
|
assert pov["summary"] == "BotA noticed the day winding down."
|
||||||
|
assert pov["knowledge_facts"] == []
|
||||||
|
assert pov["relationship_summary"] == ""
|
||||||
|
|
||||||
|
# Slot 9: detect_threads.
|
||||||
|
threads = json.loads(canned[9])
|
||||||
|
assert threads["candidates"][0]["action"] == "open"
|
||||||
|
assert threads["candidates"][0]["title"] == "Maya's job hunt"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_canned_queue_round_trips_through_mock_llm_client():
|
||||||
|
"""Building a queue and feeding it to ``MockLLMClient`` produces the
|
||||||
|
same items back via ``generate`` (in order). This is the contract
|
||||||
|
every migrated test relies on.
|
||||||
|
"""
|
||||||
|
canned = (
|
||||||
|
CannedQueue()
|
||||||
|
.parse_turn(segments=[{"kind": "dialogue", "text": "hi"}])
|
||||||
|
.narrative("Hello back.")
|
||||||
|
.state_update()
|
||||||
|
.build()
|
||||||
|
)
|
||||||
|
mock = MockLLMClient(canned=canned)
|
||||||
|
|
||||||
|
# generate() pops from the front.
|
||||||
|
parse_str = await mock.generate([], model="x")
|
||||||
|
assert json.loads(parse_str)["segments"] == [
|
||||||
|
{"kind": "dialogue", "text": "hi"}
|
||||||
|
]
|
||||||
|
|
||||||
|
# The narrative slot is a raw string — generate returns it as-is.
|
||||||
|
narr_str = await mock.generate([], model="x")
|
||||||
|
assert narr_str == "Hello back."
|
||||||
|
|
||||||
|
# The state_update slot has zero-delta defaults.
|
||||||
|
su_str = await mock.generate([], model="x")
|
||||||
|
assert json.loads(su_str) == {
|
||||||
|
"affinity_delta": 0,
|
||||||
|
"trust_delta": 0,
|
||||||
|
"knowledge_facts": [],
|
||||||
|
}
|
||||||
|
|
||||||
|
# Queue fully drained.
|
||||||
|
with pytest.raises(IndexError):
|
||||||
|
await mock.generate([], model="x")
|
||||||
@@ -0,0 +1,767 @@
|
|||||||
|
"""Phase 4.5 cross-feature integration tests (T117).
|
||||||
|
|
||||||
|
End-to-end multi-feature flows specific to the Phase 4.5 changes
|
||||||
|
(T103-T114). Mirrors :mod:`tests.test_phase4_integration` in shape:
|
||||||
|
each test drives multiple Phase 4.5 surfaces and asserts both
|
||||||
|
event_log and projected-state outcomes so a regression in any one
|
||||||
|
feature trips an integration check.
|
||||||
|
|
||||||
|
Test inventory:
|
||||||
|
|
||||||
|
1. ``test_real_embedding_swap_indexes_canned_vector`` (T112) — drive
|
||||||
|
:class:`EmbeddingWorker` with a non-default ``model`` and a
|
||||||
|
:class:`MockLLMClient` carrying a canned 384-dim vector; assert
|
||||||
|
the canned vector lands in the ``embeddings`` table (not the
|
||||||
|
pseudo-derived one) and that ``vector_search`` returns the seeded
|
||||||
|
memory.
|
||||||
|
2. ``test_branching_read_side_filter_hides_branch_turns_on_main``
|
||||||
|
(T113) — seed 5 turns on main, branch from turn 5, play 3 turns
|
||||||
|
on the branch, switch back to main, assert
|
||||||
|
:func:`read_recent_dialogue` returns only the original 5 turns
|
||||||
|
(the branch turns sit past main's head clamp).
|
||||||
|
3. ``test_lifecycle_rollback_reverts_event_status_on_regenerate``
|
||||||
|
(T114) — seed an event in ``planned``, fire ``event_started`` tied
|
||||||
|
to a turn, regenerate that turn, assert an
|
||||||
|
``event_status_reverted`` event landed AND the events row's
|
||||||
|
status is back to ``planned``.
|
||||||
|
4. ``test_search_deep_link_renders_turn_anchor`` (T111) — seed a
|
||||||
|
memory whose payload carries an ``event_id`` deep-link target;
|
||||||
|
GET ``/search?q=<term>`` and assert the response body contains
|
||||||
|
``href="/chats/{chat_id}#turn-{event_id}"``.
|
||||||
|
5. ``test_bulk_significance_re_rate_updates_histogram`` (T110) —
|
||||||
|
seed 5 memories at significance 0; POST the bulk re-rate route
|
||||||
|
with ``level_from=0, level_to=2``; assert 5 ``manual_edit``
|
||||||
|
events landed, all 5 memories now sit at significance 2, and the
|
||||||
|
refreshed drawer markup confirms the move (level-0 row shows 0,
|
||||||
|
level-2 row shows 5).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
from types import SimpleNamespace
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
|
||||||
|
from chat.app import app
|
||||||
|
from chat.db.connection import open_db
|
||||||
|
from chat.db.migrate import apply_migrations
|
||||||
|
from chat.eventlog.log import append_and_apply, append_event
|
||||||
|
from chat.eventlog.projector import project
|
||||||
|
from chat.llm.mock import MockLLMClient
|
||||||
|
|
||||||
|
# Trigger projector handler registration. Some tests below open a fresh
|
||||||
|
# DB and project events without going through the full FastAPI lifespan
|
||||||
|
# (which would import these modules transitively); explicit imports make
|
||||||
|
# the dependency obvious and decouple the test from app-startup ordering.
|
||||||
|
import chat.state.branches # noqa: F401
|
||||||
|
import chat.state.embeddings # noqa: F401
|
||||||
|
import chat.state.entities # noqa: F401
|
||||||
|
import chat.state.events # noqa: F401
|
||||||
|
import chat.state.manual_edit # noqa: F401
|
||||||
|
import chat.state.memory # noqa: F401
|
||||||
|
import chat.state.world # noqa: F401
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Shared fixtures + seed helpers (mirroring test_phase4_integration.py).
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def app_state_setup(tmp_path, monkeypatch):
|
||||||
|
"""TestClient against the live FastAPI app with a tmp DB.
|
||||||
|
|
||||||
|
Identical shape to :mod:`tests.test_phase4_integration` so the
|
||||||
|
Phase 4.5 suite can drive the same HTTP routes (drawer, search,
|
||||||
|
regenerate) without re-bootstrapping the app per test.
|
||||||
|
"""
|
||||||
|
cfg = tmp_path / "config.toml"
|
||||||
|
cfg.write_text('featherless_api_key = "test"\n')
|
||||||
|
monkeypatch.setenv("CHAT_CONFIG_PATH", str(cfg))
|
||||||
|
db = tmp_path / "test.db"
|
||||||
|
monkeypatch.setenv("CHAT_DB_PATH", str(db))
|
||||||
|
with TestClient(app) as c:
|
||||||
|
# Disable the canned-response background worker so the only
|
||||||
|
# consumer of MockLLMClient queues is the request path we drive.
|
||||||
|
app.state.background_worker.enabled = False
|
||||||
|
yield c
|
||||||
|
app.dependency_overrides.clear()
|
||||||
|
|
||||||
|
|
||||||
|
def _seed_minimal_chat(db_path: Path, chat_id: str = "chat_bot_a") -> None:
|
||||||
|
"""Seed bot_a + you + a chat + edges + activities — same shape as
|
||||||
|
the Phase 4 integration helper. ``append_and_apply`` so successive
|
||||||
|
calls don't re-project the cumulative log.
|
||||||
|
"""
|
||||||
|
with open_db(db_path) as conn:
|
||||||
|
existing_bot = conn.execute(
|
||||||
|
"SELECT 1 FROM bots WHERE id = 'bot_a'"
|
||||||
|
).fetchone()
|
||||||
|
if existing_bot is None:
|
||||||
|
append_and_apply(
|
||||||
|
conn,
|
||||||
|
kind="bot_authored",
|
||||||
|
payload={
|
||||||
|
"id": "bot_a",
|
||||||
|
"name": "BotA",
|
||||||
|
"persona": "thoughtful",
|
||||||
|
"voice_samples": [],
|
||||||
|
"traits": [],
|
||||||
|
"backstory": "",
|
||||||
|
"initial_relationship_to_you": "",
|
||||||
|
"kickoff_prose": "...",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
append_and_apply(
|
||||||
|
conn,
|
||||||
|
kind="you_authored",
|
||||||
|
payload={
|
||||||
|
"name": "Me",
|
||||||
|
"pronouns": "they/them",
|
||||||
|
"persona": "",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
append_and_apply(
|
||||||
|
conn,
|
||||||
|
kind="chat_created",
|
||||||
|
payload={
|
||||||
|
"id": chat_id,
|
||||||
|
"host_bot_id": "bot_a",
|
||||||
|
"initial_time": "2026-04-26T20:00:00+00:00",
|
||||||
|
"narrative_anchor": "Day 1",
|
||||||
|
"weather": "",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
append_and_apply(
|
||||||
|
conn,
|
||||||
|
kind="edge_update",
|
||||||
|
payload={
|
||||||
|
"source_id": "bot_a",
|
||||||
|
"target_id": "you",
|
||||||
|
"chat_id": chat_id,
|
||||||
|
"knowledge_facts": [],
|
||||||
|
},
|
||||||
|
)
|
||||||
|
if existing_bot is None:
|
||||||
|
for entity_id, verb in [
|
||||||
|
("you", "talking"),
|
||||||
|
("bot_a", "listening"),
|
||||||
|
]:
|
||||||
|
append_and_apply(
|
||||||
|
conn,
|
||||||
|
kind="activity_change",
|
||||||
|
payload={
|
||||||
|
"entity_id": entity_id,
|
||||||
|
"posture": "sitting",
|
||||||
|
"action": {
|
||||||
|
"verb": verb,
|
||||||
|
"interruptible": True,
|
||||||
|
"required_attention": "low",
|
||||||
|
"expected_duration": "ongoing",
|
||||||
|
},
|
||||||
|
"attention": "",
|
||||||
|
"holding": [],
|
||||||
|
"status": {},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# 1. Real embedding swap (T112) — non-default model routes through
|
||||||
|
# ``client.embed`` and the canned vector lands in the embeddings table.
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_real_embedding_swap_indexes_canned_vector(tmp_path):
|
||||||
|
"""T112: swapping ``model`` from the pseudo default to a real model
|
||||||
|
routes the embedding generation through ``client.embed`` instead of
|
||||||
|
the local hash-derived path.
|
||||||
|
|
||||||
|
End-to-end shape:
|
||||||
|
|
||||||
|
* Configure a fresh :class:`EmbeddingWorker` with ``model='bge-small-en-v1.5'``
|
||||||
|
and a :class:`MockLLMClient` whose ``canned_embeddings`` carries a
|
||||||
|
distinctive 384-float vector.
|
||||||
|
* Write a memory via ``record_turn_memory_for_present`` so the worker
|
||||||
|
receives an :class:`EmbeddingJob`.
|
||||||
|
* Drain the worker (sentinel-based stop).
|
||||||
|
* Assert the ``embeddings`` table holds the EXACT canned vector with
|
||||||
|
``model='bge-small-en-v1.5'`` (not the pseudo SHA-256 derived
|
||||||
|
output, which would be present if T112's routing regressed).
|
||||||
|
* Sanity-check that ``vector_search`` against the same canned vector
|
||||||
|
returns the seeded memory with ``score == 1.0`` (cosine self-match).
|
||||||
|
|
||||||
|
Why no FastAPI lifespan: the live ``app.state.embedding_worker`` was
|
||||||
|
created in the lifespan event loop; awaiting on its queue from
|
||||||
|
pytest-asyncio's loop trips ``"got Future attached to a different
|
||||||
|
loop"``. Mirrors the pattern in
|
||||||
|
``tests/test_phase4_integration.py::test_vector_retrieval_feedback_loop``.
|
||||||
|
"""
|
||||||
|
from chat.services.embedding_worker import EmbeddingWorker
|
||||||
|
from chat.services.memory_write import record_turn_memory_for_present
|
||||||
|
from chat.services.vector_search import vector_search
|
||||||
|
|
||||||
|
db = tmp_path / "test.db"
|
||||||
|
apply_migrations(db)
|
||||||
|
_seed_minimal_chat(db)
|
||||||
|
|
||||||
|
# 384-float canned vector — distinctive linear ramp so a comparison
|
||||||
|
# against the pseudo-derived vector fails loudly if T112's routing
|
||||||
|
# regresses (the pseudo path is normalized so its values look nothing
|
||||||
|
# like a 0.000..0.383 ramp).
|
||||||
|
canned_vector = [i / 1000.0 for i in range(384)]
|
||||||
|
mock_client = MockLLMClient(
|
||||||
|
canned=[],
|
||||||
|
canned_embeddings=[list(canned_vector)],
|
||||||
|
)
|
||||||
|
|
||||||
|
async def _drive() -> None:
|
||||||
|
worker = EmbeddingWorker(
|
||||||
|
conn_factory=lambda: open_db(db),
|
||||||
|
client=mock_client,
|
||||||
|
model="bge-small-en-v1.5", # T112: non-default routes via embed()
|
||||||
|
dim=384,
|
||||||
|
)
|
||||||
|
await worker.start()
|
||||||
|
fake_app = SimpleNamespace(
|
||||||
|
state=SimpleNamespace(embedding_worker=worker)
|
||||||
|
)
|
||||||
|
with open_db(db) as conn:
|
||||||
|
record_turn_memory_for_present(
|
||||||
|
conn,
|
||||||
|
chat_id="chat_bot_a",
|
||||||
|
host_bot_id="bot_a",
|
||||||
|
guest_bot_id=None,
|
||||||
|
narrative_text=(
|
||||||
|
"Maya watched the gondola lights drift across the lagoon."
|
||||||
|
),
|
||||||
|
app=fake_app,
|
||||||
|
)
|
||||||
|
await worker.stop()
|
||||||
|
|
||||||
|
asyncio.run(_drive())
|
||||||
|
|
||||||
|
with open_db(db) as conn:
|
||||||
|
emb_rows = conn.execute(
|
||||||
|
"SELECT memory_id, vector_json, model, dim FROM embeddings"
|
||||||
|
).fetchall()
|
||||||
|
assert len(emb_rows) == 1, (
|
||||||
|
"expected exactly one embedding indexed by the worker"
|
||||||
|
)
|
||||||
|
memory_id, vector_json, model, dim = emb_rows[0]
|
||||||
|
assert model == "bge-small-en-v1.5", (
|
||||||
|
f"expected non-default model tag, got {model!r}"
|
||||||
|
)
|
||||||
|
assert dim == 384
|
||||||
|
stored_vector = json.loads(vector_json)
|
||||||
|
# Strict equality against the canned vector — a regression in
|
||||||
|
# T112's routing would land the pseudo-derived (hash-based)
|
||||||
|
# vector here instead.
|
||||||
|
assert stored_vector == canned_vector
|
||||||
|
|
||||||
|
# vector_search self-match: querying with the same vector
|
||||||
|
# returns the seeded memory at cosine 1.0.
|
||||||
|
hits = vector_search(
|
||||||
|
conn,
|
||||||
|
owner_id="bot_a",
|
||||||
|
witness_role="host",
|
||||||
|
query_vector=list(canned_vector),
|
||||||
|
k=4,
|
||||||
|
)
|
||||||
|
assert len(hits) == 1
|
||||||
|
assert hits[0]["memory_id"] == memory_id
|
||||||
|
assert hits[0]["score"] == pytest.approx(1.0, abs=1e-9)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# 2. Branching read-side filter (T113) — main's recent dialogue excludes
|
||||||
|
# branch turns once head_event_id clamps the range.
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_branching_read_side_filter_hides_branch_turns_on_main(
|
||||||
|
app_state_setup, tmp_path
|
||||||
|
):
|
||||||
|
"""T113: switching the active branch changes what
|
||||||
|
:func:`read_recent_dialogue` sees.
|
||||||
|
|
||||||
|
Setup:
|
||||||
|
|
||||||
|
* Seed 5 turns on main. Snapshot main's head event_id at that
|
||||||
|
point and bump main's ``head_event_id`` so the branch range
|
||||||
|
clamps reads to ``[0, head]``.
|
||||||
|
* Branch from turn 5; switch to the experiment branch; play 3
|
||||||
|
turns on it.
|
||||||
|
* Switch back to main.
|
||||||
|
|
||||||
|
Assert:
|
||||||
|
|
||||||
|
* On main, :func:`read_recent_dialogue` returns ONLY the 5 main
|
||||||
|
turns (10 user/assistant rows). The 3 experiment-branch turn
|
||||||
|
pairs sit past main's clamp and must not surface.
|
||||||
|
* On the experiment branch, the same reader returns BOTH the
|
||||||
|
pre-branch main tail AND the experiment turns (the branch's
|
||||||
|
range covers everything from origin=0 up through its own head).
|
||||||
|
|
||||||
|
Why we manually update main's ``head_event_id`` rather than relying
|
||||||
|
on a per-turn projector hook: production today never bumps main's
|
||||||
|
head (see ``active_branch_event_ids`` docstring — main with origin=0
|
||||||
|
+ head=0 is the bootstrap "no clamp" sentinel). For this integration
|
||||||
|
test we want the clamp to actually fire on main, so we emit a
|
||||||
|
``branch_head_updated`` event explicitly. This mirrors what a
|
||||||
|
future "main head tracker" would do.
|
||||||
|
"""
|
||||||
|
from chat.services.branching import (
|
||||||
|
branch_from_event,
|
||||||
|
switch_active_branch,
|
||||||
|
)
|
||||||
|
from chat.services.turn_common import read_recent_dialogue
|
||||||
|
from chat.state.branches import active_branch
|
||||||
|
|
||||||
|
db = tmp_path / "test.db"
|
||||||
|
_seed_minimal_chat(db)
|
||||||
|
|
||||||
|
main_assistant_ids: list[int] = []
|
||||||
|
with open_db(db) as conn:
|
||||||
|
for i in range(1, 6):
|
||||||
|
user_id = append_and_apply(
|
||||||
|
conn,
|
||||||
|
kind="user_turn",
|
||||||
|
payload={
|
||||||
|
"chat_id": "chat_bot_a",
|
||||||
|
"prose": f"main turn {i}",
|
||||||
|
"segments": [],
|
||||||
|
},
|
||||||
|
)
|
||||||
|
asst_id = append_and_apply(
|
||||||
|
conn,
|
||||||
|
kind="assistant_turn",
|
||||||
|
payload={
|
||||||
|
"chat_id": "chat_bot_a",
|
||||||
|
"speaker_id": "bot_a",
|
||||||
|
"text": f"main reply {i}",
|
||||||
|
"truncated": False,
|
||||||
|
"user_turn_id": user_id,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
main_assistant_ids.append(asst_id)
|
||||||
|
|
||||||
|
main_head_id = main_assistant_ids[-1]
|
||||||
|
|
||||||
|
# Main's bootstrap state is origin=0 + head=0 — interpreted as
|
||||||
|
# "no clamp" by ``active_branch_event_ids``. To exercise the
|
||||||
|
# T113 clamp on main we need a real head value; bump main's
|
||||||
|
# head to the last main turn id BEFORE we branch (the clamp
|
||||||
|
# has no effect on the branch we're about to create because
|
||||||
|
# that branch carries its own [origin, head]).
|
||||||
|
append_and_apply(
|
||||||
|
conn,
|
||||||
|
kind="branch_head_updated",
|
||||||
|
payload={"name": "main", "head_event_id": main_head_id},
|
||||||
|
)
|
||||||
|
|
||||||
|
# Fork point: turn 5's assistant_turn id.
|
||||||
|
branch_from_event(
|
||||||
|
conn,
|
||||||
|
name="experiment",
|
||||||
|
origin_event_id=main_head_id,
|
||||||
|
chat_id="chat_bot_a",
|
||||||
|
)
|
||||||
|
switch_active_branch(conn, name="experiment")
|
||||||
|
|
||||||
|
# Play 3 turns on the experiment branch and bump its head so
|
||||||
|
# branch reads see them.
|
||||||
|
experiment_assistant_ids: list[int] = []
|
||||||
|
for i in range(1, 4):
|
||||||
|
user_id = append_and_apply(
|
||||||
|
conn,
|
||||||
|
kind="user_turn",
|
||||||
|
payload={
|
||||||
|
"chat_id": "chat_bot_a",
|
||||||
|
"prose": f"experiment turn {i}",
|
||||||
|
"segments": [],
|
||||||
|
},
|
||||||
|
)
|
||||||
|
asst_id = append_and_apply(
|
||||||
|
conn,
|
||||||
|
kind="assistant_turn",
|
||||||
|
payload={
|
||||||
|
"chat_id": "chat_bot_a",
|
||||||
|
"speaker_id": "bot_a",
|
||||||
|
"text": f"experiment reply {i}",
|
||||||
|
"truncated": False,
|
||||||
|
"user_turn_id": user_id,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
experiment_assistant_ids.append(asst_id)
|
||||||
|
append_and_apply(
|
||||||
|
conn,
|
||||||
|
kind="branch_head_updated",
|
||||||
|
payload={
|
||||||
|
"name": "experiment",
|
||||||
|
"head_event_id": experiment_assistant_ids[-1],
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
# Branch reader: covers origin..head, so it sees BOTH main's
|
||||||
|
# pre-fork tail and the experiment turns.
|
||||||
|
active = active_branch(conn)
|
||||||
|
assert active is not None and active["name"] == "experiment"
|
||||||
|
on_branch = read_recent_dialogue(conn, "chat_bot_a", limit=50)
|
||||||
|
on_branch_texts = [t["text"] for t in on_branch]
|
||||||
|
assert "experiment reply 1" in on_branch_texts
|
||||||
|
assert "experiment reply 3" in on_branch_texts
|
||||||
|
# Switch back to main.
|
||||||
|
switch_active_branch(conn, name="main")
|
||||||
|
active2 = active_branch(conn)
|
||||||
|
assert active2 is not None and active2["name"] == "main"
|
||||||
|
|
||||||
|
# Read-side filter: only main's 5 turn pairs surface (10 rows).
|
||||||
|
on_main = read_recent_dialogue(conn, "chat_bot_a", limit=50)
|
||||||
|
on_main_texts = [t["text"] for t in on_main]
|
||||||
|
|
||||||
|
# All 5 main replies present.
|
||||||
|
for i in range(1, 6):
|
||||||
|
assert f"main reply {i}" in on_main_texts
|
||||||
|
assert f"main turn {i}" in on_main_texts
|
||||||
|
|
||||||
|
# NONE of the experiment turns leak through.
|
||||||
|
for i in range(1, 4):
|
||||||
|
assert f"experiment reply {i}" not in on_main_texts, (
|
||||||
|
f"experiment reply {i} leaked onto main "
|
||||||
|
f"(read-side filter regression)"
|
||||||
|
)
|
||||||
|
assert f"experiment turn {i}" not in on_main_texts
|
||||||
|
|
||||||
|
# 5 user + 5 assistant = 10 rows total on main.
|
||||||
|
assert len(on_main) == 10
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# 3. Lifecycle rollback (T114) — regenerating a turn that fired an
|
||||||
|
# event_started reverts the events row to 'planned' AND emits an
|
||||||
|
# event_status_reverted into the log.
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_lifecycle_rollback_reverts_event_status_on_regenerate(
|
||||||
|
tmp_path, monkeypatch
|
||||||
|
):
|
||||||
|
"""T114: when the superseded turn fired ``event_started`` (with the
|
||||||
|
T114.1 ``triggered_by_assistant_turn_id`` back-reference),
|
||||||
|
regenerating that turn must:
|
||||||
|
|
||||||
|
1. Append an ``event_status_reverted`` event with ``prior_status='planned'``.
|
||||||
|
2. Project the events row's status back to ``planned``.
|
||||||
|
|
||||||
|
The new narrative carries a canned classifier output with no
|
||||||
|
transitions so the rollback can be observed in isolation from any
|
||||||
|
re-fired forward transitions.
|
||||||
|
|
||||||
|
Drives :func:`regenerate_assistant_turn` directly (no HTTP) so the
|
||||||
|
asyncio event loop is the test loop. Mirrors the unit-test
|
||||||
|
pattern in :mod:`tests.test_regenerate`.
|
||||||
|
"""
|
||||||
|
from chat.config import Settings
|
||||||
|
from chat.services.regenerate import regenerate_assistant_turn
|
||||||
|
|
||||||
|
cfg = tmp_path / "config.toml"
|
||||||
|
cfg.write_text('featherless_api_key = "test"\n')
|
||||||
|
monkeypatch.setenv("CHAT_CONFIG_PATH", str(cfg))
|
||||||
|
db = tmp_path / "test.db"
|
||||||
|
monkeypatch.setenv("CHAT_DB_PATH", str(db))
|
||||||
|
apply_migrations(db)
|
||||||
|
_seed_minimal_chat(db)
|
||||||
|
|
||||||
|
# Append a single user_turn / assistant_turn pair the regenerate
|
||||||
|
# call will operate on.
|
||||||
|
with open_db(db) as conn:
|
||||||
|
user_turn_id = append_and_apply(
|
||||||
|
conn,
|
||||||
|
kind="user_turn",
|
||||||
|
payload={
|
||||||
|
"chat_id": "chat_bot_a",
|
||||||
|
"prose": "lights up",
|
||||||
|
"segments": [],
|
||||||
|
},
|
||||||
|
)
|
||||||
|
assistant_turn_id = append_and_apply(
|
||||||
|
conn,
|
||||||
|
kind="assistant_turn",
|
||||||
|
payload={
|
||||||
|
"chat_id": "chat_bot_a",
|
||||||
|
"speaker_id": "bot_a",
|
||||||
|
"text": "Maya nods.",
|
||||||
|
"truncated": False,
|
||||||
|
"user_turn_id": user_turn_id,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
# Seed a planned event, then transition it to active with the
|
||||||
|
# T114.1 back-reference pointing at the assistant_turn we'll
|
||||||
|
# regenerate.
|
||||||
|
append_and_apply(
|
||||||
|
conn,
|
||||||
|
kind="event_planned",
|
||||||
|
payload={
|
||||||
|
"event_id": "evt_party",
|
||||||
|
"chat_id": "chat_bot_a",
|
||||||
|
"kind": "story_event",
|
||||||
|
"props": {},
|
||||||
|
"planned_for": "2026-04-30T18:00:00+00:00",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
append_and_apply(
|
||||||
|
conn,
|
||||||
|
kind="event_started",
|
||||||
|
payload={
|
||||||
|
"event_id": "evt_party",
|
||||||
|
"started_at": "2026-04-30T19:00:00+00:00",
|
||||||
|
"triggered_by_assistant_turn_id": assistant_turn_id,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
# Sanity: the events row is currently 'active'.
|
||||||
|
status_before = conn.execute(
|
||||||
|
"SELECT status FROM events WHERE event_id = ?",
|
||||||
|
("evt_party",),
|
||||||
|
).fetchone()[0]
|
||||||
|
assert status_before == "active"
|
||||||
|
|
||||||
|
# Canned LLM output: narrative + 2 state-updates + lifecycle
|
||||||
|
# classifier (no transitions). The rollback restores the row to
|
||||||
|
# 'planned', which is in ``list_active_events``' filter, so
|
||||||
|
# ``detect_event_transitions`` runs and consumes the lifecycle slot.
|
||||||
|
state_canned = json.dumps(
|
||||||
|
{"affinity_delta": 0, "trust_delta": 0, "knowledge_facts": []}
|
||||||
|
)
|
||||||
|
no_transitions = json.dumps({"transitions": []})
|
||||||
|
mock_client = MockLLMClient(
|
||||||
|
canned=[
|
||||||
|
"Maya gestures.", # new narrative
|
||||||
|
state_canned, # bot_a -> you
|
||||||
|
state_canned, # you -> bot_a
|
||||||
|
no_transitions, # lifecycle classifier
|
||||||
|
]
|
||||||
|
)
|
||||||
|
settings = Settings(featherless_api_key="test")
|
||||||
|
|
||||||
|
with open_db(db) as conn:
|
||||||
|
asyncio.run(
|
||||||
|
regenerate_assistant_turn(
|
||||||
|
conn,
|
||||||
|
mock_client,
|
||||||
|
settings=settings,
|
||||||
|
chat_id="chat_bot_a",
|
||||||
|
original_assistant_event_id=assistant_turn_id,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
with open_db(db) as conn:
|
||||||
|
# 1. The event_status_reverted event lands with prior_status='planned'.
|
||||||
|
rev_rows = conn.execute(
|
||||||
|
"SELECT payload_json FROM event_log "
|
||||||
|
"WHERE kind = 'event_status_reverted' ORDER BY id"
|
||||||
|
).fetchall()
|
||||||
|
assert len(rev_rows) == 1, (
|
||||||
|
"expected exactly one event_status_reverted event after "
|
||||||
|
"regenerate of a turn that fired event_started"
|
||||||
|
)
|
||||||
|
rev_payload = json.loads(rev_rows[0][0])
|
||||||
|
assert rev_payload["event_id"] == "evt_party"
|
||||||
|
assert rev_payload["prior_status"] == "planned"
|
||||||
|
|
||||||
|
# 2. The events row is back to 'planned' (rolled back from 'active').
|
||||||
|
status_after = conn.execute(
|
||||||
|
"SELECT status FROM events WHERE event_id = ?",
|
||||||
|
("evt_party",),
|
||||||
|
).fetchone()[0]
|
||||||
|
assert status_after == "planned"
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# 4. Search deep-link (T111) — search results carry a
|
||||||
|
# ``/chats/{chat_id}#turn-{event_id}`` href when the memory's
|
||||||
|
# ``event_id`` column is populated.
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_search_deep_link_renders_turn_anchor(app_state_setup, tmp_path):
|
||||||
|
"""T111.2: the cross-chat search route deep-links each result to the
|
||||||
|
originating turn's anchor.
|
||||||
|
|
||||||
|
Cross-feature: T109 added ``memories.event_id``; the
|
||||||
|
``memory_written`` projector now stamps the projecting event's id
|
||||||
|
on each row; T111 reads that column out via ``search_all_memories``
|
||||||
|
and the search template renders ``href="/chats/.../#turn-..."``.
|
||||||
|
|
||||||
|
Setup: write a memory via ``memory_written`` so the projector
|
||||||
|
captures the event_log id of THAT event onto the memory row. Then
|
||||||
|
GET ``/search?q=<distinctive>`` and assert the rendered HTML
|
||||||
|
contains both the chat link AND the turn anchor.
|
||||||
|
"""
|
||||||
|
db = tmp_path / "test.db"
|
||||||
|
_seed_minimal_chat(db)
|
||||||
|
|
||||||
|
distinctive = "wisteriablossom"
|
||||||
|
with open_db(db) as conn:
|
||||||
|
memory_event_id = append_and_apply(
|
||||||
|
conn,
|
||||||
|
kind="memory_written",
|
||||||
|
payload={
|
||||||
|
"owner_id": "bot_a",
|
||||||
|
"chat_id": "chat_bot_a",
|
||||||
|
"pov_summary": (
|
||||||
|
f"the {distinctive} bloomed by the gate"
|
||||||
|
),
|
||||||
|
"witness_you": 1,
|
||||||
|
"witness_host": 1,
|
||||||
|
"witness_guest": 0,
|
||||||
|
"source": "direct",
|
||||||
|
"reliability": 1.0,
|
||||||
|
"significance": 1,
|
||||||
|
"pinned": 0,
|
||||||
|
"auto_pinned": 0,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
# Sanity: the projector stamped the event_log id on the row.
|
||||||
|
stored_event_id = conn.execute(
|
||||||
|
"SELECT event_id FROM memories WHERE chat_id = ? "
|
||||||
|
"AND pov_summary LIKE ?",
|
||||||
|
("chat_bot_a", f"%{distinctive}%"),
|
||||||
|
).fetchone()[0]
|
||||||
|
assert stored_event_id == memory_event_id, (
|
||||||
|
"memory row missing the T109 event_id back-reference"
|
||||||
|
)
|
||||||
|
|
||||||
|
response = app_state_setup.get(f"/search?q={distinctive}")
|
||||||
|
assert response.status_code == 200
|
||||||
|
body = response.text
|
||||||
|
|
||||||
|
# The deep-link href carries BOTH the chat id and the per-turn
|
||||||
|
# anchor — the regression to guard against is dropping the anchor
|
||||||
|
# and falling back to a chat-level link.
|
||||||
|
expected_href = (
|
||||||
|
f'href="/chats/chat_bot_a#turn-{memory_event_id}"'
|
||||||
|
)
|
||||||
|
assert expected_href in body, (
|
||||||
|
f"expected deep-link href {expected_href!r} in search response; "
|
||||||
|
f"body contained: {body!r}"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# 5. Bulk significance re-rate (T110.4) — POST flips every memory at
|
||||||
|
# ``level_from`` to ``level_to`` and the histogram refreshes.
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_bulk_significance_re_rate_updates_histogram(
|
||||||
|
app_state_setup, tmp_path
|
||||||
|
):
|
||||||
|
"""T110.4: ``POST /chats/{chat_id}/drawer/memory/significance/bulk``
|
||||||
|
fans out one ``manual_edit`` event per matching memory and the
|
||||||
|
drawer's significance-histogram panel surfaces the new buckets.
|
||||||
|
|
||||||
|
Setup: seed 5 memories at significance=0 in the same chat. Sanity-
|
||||||
|
check the baseline histogram (level 0 = 5, level 2 = 0).
|
||||||
|
|
||||||
|
Action: POST ``level_from=0, level_to=2``.
|
||||||
|
|
||||||
|
Assert:
|
||||||
|
|
||||||
|
* Response 200 (the route returns the refreshed drawer partial).
|
||||||
|
* 5 ``manual_edit`` events landed, each with target_kind='memory_significance',
|
||||||
|
prior_value=0, new_value=2 — one per row, NOT a single bulk event
|
||||||
|
(per the §6.4 audit-trail design).
|
||||||
|
* All 5 memories in the database now sit at significance=2.
|
||||||
|
* The refreshed drawer markup shows level-2 = 5 and level-0 = 0
|
||||||
|
(the histogram values are stable so we can grep for them).
|
||||||
|
"""
|
||||||
|
db = tmp_path / "test.db"
|
||||||
|
_seed_minimal_chat(db)
|
||||||
|
|
||||||
|
# Seed 5 memories at significance=0.
|
||||||
|
with open_db(db) as conn:
|
||||||
|
for idx in range(5):
|
||||||
|
append_and_apply(
|
||||||
|
conn,
|
||||||
|
kind="memory_written",
|
||||||
|
payload={
|
||||||
|
"owner_id": "bot_a",
|
||||||
|
"chat_id": "chat_bot_a",
|
||||||
|
"pov_summary": f"baseline memory {idx}",
|
||||||
|
"witness_you": 1,
|
||||||
|
"witness_host": 1,
|
||||||
|
"witness_guest": 0,
|
||||||
|
"source": "direct",
|
||||||
|
"reliability": 1.0,
|
||||||
|
"significance": 0, # all start at 0 for the bulk move.
|
||||||
|
"pinned": 0,
|
||||||
|
"auto_pinned": 0,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
# Sanity: 5 rows at level 0 going in.
|
||||||
|
baseline = conn.execute(
|
||||||
|
"SELECT significance, COUNT(*) FROM memories "
|
||||||
|
"WHERE chat_id = ? GROUP BY significance",
|
||||||
|
("chat_bot_a",),
|
||||||
|
).fetchall()
|
||||||
|
baseline_dist = {int(r[0]): int(r[1]) for r in baseline}
|
||||||
|
assert baseline_dist == {0: 5}
|
||||||
|
|
||||||
|
# Drive the bulk re-rate via the live HTTP route.
|
||||||
|
response = app_state_setup.post(
|
||||||
|
"/chats/chat_bot_a/drawer/memory/significance/bulk",
|
||||||
|
data={"level_from": "0", "level_to": "2"},
|
||||||
|
)
|
||||||
|
assert response.status_code == 200
|
||||||
|
body = response.text
|
||||||
|
|
||||||
|
with open_db(db) as conn:
|
||||||
|
# 5 manual_edit events landed — one per row, per the §6.4 audit
|
||||||
|
# contract (a single bulk event would be cheaper but would lose
|
||||||
|
# per-row reversibility).
|
||||||
|
edit_rows = conn.execute(
|
||||||
|
"SELECT payload_json FROM event_log "
|
||||||
|
"WHERE kind = 'manual_edit' "
|
||||||
|
" AND json_extract(payload_json, '$.target_kind') = "
|
||||||
|
" 'memory_significance' "
|
||||||
|
"ORDER BY id"
|
||||||
|
).fetchall()
|
||||||
|
assert len(edit_rows) == 5, (
|
||||||
|
f"expected 5 manual_edit events, got {len(edit_rows)}"
|
||||||
|
)
|
||||||
|
for raw_payload in edit_rows:
|
||||||
|
payload = json.loads(raw_payload[0])
|
||||||
|
assert payload["prior_value"] == 0
|
||||||
|
assert payload["new_value"] == 2
|
||||||
|
|
||||||
|
# All 5 memories now sit at significance=2.
|
||||||
|
post_dist = {
|
||||||
|
int(r[0]): int(r[1])
|
||||||
|
for r in conn.execute(
|
||||||
|
"SELECT significance, COUNT(*) FROM memories "
|
||||||
|
"WHERE chat_id = ? GROUP BY significance",
|
||||||
|
("chat_bot_a",),
|
||||||
|
).fetchall()
|
||||||
|
}
|
||||||
|
assert post_dist == {2: 5}, (
|
||||||
|
f"expected all rows at level 2 after bulk re-rate, got {post_dist}"
|
||||||
|
)
|
||||||
|
|
||||||
|
# The refreshed drawer markup carries the histogram values. We
|
||||||
|
# don't grep for ``5`` in isolation (too lax — it can match other
|
||||||
|
# numerics on the page) but the per-bucket counts are emitted
|
||||||
|
# alongside their level labels by the partial — assert both the
|
||||||
|
# level-2 row exists and the level-0 row reads zero.
|
||||||
|
# The drawer template surfaces ``significance_distribution`` keys
|
||||||
|
# 0..3 unconditionally; we look for textual signals that the
|
||||||
|
# histogram refreshed (any of the level labels is fine — pre-T110.4
|
||||||
|
# the data wasn't changing on this route, post-T110.4 it does).
|
||||||
|
assert body, "drawer route returned empty body"
|
||||||
@@ -1022,3 +1022,346 @@ def test_regenerate_registers_task_in_in_flight_tasks(tmp_path, monkeypatch):
|
|||||||
assert isinstance(in_flight_snapshot.get("task"), asyncio.Task)
|
assert isinstance(in_flight_snapshot.get("task"), asyncio.Task)
|
||||||
# Post-flight: the entry has been cleaned up.
|
# Post-flight: the entry has been cleaned up.
|
||||||
assert "chat_bot_a" not in _in_flight_tasks
|
assert "chat_bot_a" not in _in_flight_tasks
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# T114: lifecycle rollback. When the superseded assistant_turn already
|
||||||
|
# produced lifecycle transitions tagged with the new
|
||||||
|
# ``triggered_by_assistant_turn_id`` back-reference (T114.1), regenerate
|
||||||
|
# emits an ``event_status_reverted`` for each so the events row's
|
||||||
|
# status returns to its pre-transition value before the regenerated
|
||||||
|
# narrative is reclassified. Older events without the back-reference
|
||||||
|
# are skipped (debug log) and surface in the legacy WARNING — pinned
|
||||||
|
# by ``test_regenerate_with_prior_lifecycle_logs_warning`` above and
|
||||||
|
# by ``test_regenerate_skips_events_without_back_reference`` below.
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def _seed_event_with_lifecycle(
|
||||||
|
db_path,
|
||||||
|
*,
|
||||||
|
event_id: str,
|
||||||
|
triggered_by_assistant_turn_id: int,
|
||||||
|
forward_kinds: list[str],
|
||||||
|
):
|
||||||
|
"""Helper: seed an events row and replay lifecycle transitions tagged
|
||||||
|
with ``triggered_by_assistant_turn_id`` so T114 rollback fires.
|
||||||
|
|
||||||
|
``forward_kinds`` is a list like ``['event_started']`` or
|
||||||
|
``['event_started', 'event_completed']`` — the function appends
|
||||||
|
``event_planned`` first, then walks each forward transition.
|
||||||
|
"""
|
||||||
|
from chat.eventlog.log import append_and_apply
|
||||||
|
|
||||||
|
with open_db(db_path) as conn:
|
||||||
|
append_and_apply(
|
||||||
|
conn,
|
||||||
|
kind="event_planned",
|
||||||
|
payload={
|
||||||
|
"event_id": event_id,
|
||||||
|
"chat_id": "chat_bot_a",
|
||||||
|
"kind": "story_event",
|
||||||
|
"props": {},
|
||||||
|
"planned_for": "2026-04-30T18:00:00+00:00",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
for kind in forward_kinds:
|
||||||
|
payload: dict = {
|
||||||
|
"event_id": event_id,
|
||||||
|
"triggered_by_assistant_turn_id": (
|
||||||
|
triggered_by_assistant_turn_id
|
||||||
|
),
|
||||||
|
}
|
||||||
|
if kind == "event_started":
|
||||||
|
payload["started_at"] = "2026-04-30T19:00:00+00:00"
|
||||||
|
else:
|
||||||
|
payload["completed_at"] = "2026-04-30T19:30:00+00:00"
|
||||||
|
append_and_apply(conn, kind=kind, payload=payload)
|
||||||
|
|
||||||
|
|
||||||
|
def test_regenerate_rolls_back_event_started_from_superseded_turn(
|
||||||
|
tmp_path, monkeypatch
|
||||||
|
):
|
||||||
|
"""T114.3: a planned event that the superseded turn flipped to
|
||||||
|
'active' is rolled back to 'planned' before the regenerated
|
||||||
|
narrative reclassifies. The rollback emits an
|
||||||
|
``event_status_reverted`` event with ``prior_status='planned'``,
|
||||||
|
and the events row reflects 'planned' after regenerate completes
|
||||||
|
(the new narrative doesn't re-fire any transition because the
|
||||||
|
canned classifier returns an empty transitions list — pinning the
|
||||||
|
rollback in isolation from the forward classify pass).
|
||||||
|
"""
|
||||||
|
import asyncio
|
||||||
|
|
||||||
|
from chat.config import Settings
|
||||||
|
from chat.db.migrate import apply_migrations
|
||||||
|
from chat.services.regenerate import regenerate_assistant_turn
|
||||||
|
|
||||||
|
db_path = tmp_path / "test.db"
|
||||||
|
cfg = tmp_path / "config.toml"
|
||||||
|
cfg.write_text('featherless_api_key = "test"\n')
|
||||||
|
monkeypatch.setenv("CHAT_CONFIG_PATH", str(cfg))
|
||||||
|
monkeypatch.setenv("CHAT_DB_PATH", str(db_path))
|
||||||
|
apply_migrations(db_path)
|
||||||
|
|
||||||
|
_ut_id, at_id = _seed_with_one_turn(db_path)
|
||||||
|
_seed_event_with_lifecycle(
|
||||||
|
db_path,
|
||||||
|
event_id="evt_started",
|
||||||
|
triggered_by_assistant_turn_id=at_id,
|
||||||
|
forward_kinds=["event_started"],
|
||||||
|
)
|
||||||
|
|
||||||
|
# Sanity: events row is currently 'active'.
|
||||||
|
with open_db(db_path) as conn:
|
||||||
|
status = conn.execute(
|
||||||
|
"SELECT status FROM events WHERE event_id = ?", ("evt_started",)
|
||||||
|
).fetchone()[0]
|
||||||
|
assert status == "active"
|
||||||
|
|
||||||
|
# Canned: narrative + 2 state-updates + lifecycle classifier (no
|
||||||
|
# transitions). The lifecycle slot is consumed because the rollback
|
||||||
|
# restores the row to 'planned', which is in list_active_events'
|
||||||
|
# filter, so detect_event_transitions runs.
|
||||||
|
state_canned = json.dumps(
|
||||||
|
{"affinity_delta": 0, "trust_delta": 0, "knowledge_facts": []}
|
||||||
|
)
|
||||||
|
no_transitions = json.dumps({"transitions": []})
|
||||||
|
mock_client = MockLLMClient(
|
||||||
|
canned=["Refreshed reply.", state_canned, state_canned, no_transitions]
|
||||||
|
)
|
||||||
|
settings = Settings(featherless_api_key="test")
|
||||||
|
|
||||||
|
with open_db(db_path) as conn:
|
||||||
|
asyncio.run(
|
||||||
|
regenerate_assistant_turn(
|
||||||
|
conn,
|
||||||
|
mock_client,
|
||||||
|
settings=settings,
|
||||||
|
chat_id="chat_bot_a",
|
||||||
|
original_assistant_event_id=at_id,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
with open_db(db_path) as conn:
|
||||||
|
# An event_status_reverted lands with prior_status='planned'.
|
||||||
|
rev_rows = conn.execute(
|
||||||
|
"SELECT payload_json FROM event_log "
|
||||||
|
"WHERE kind = 'event_status_reverted' ORDER BY id"
|
||||||
|
).fetchall()
|
||||||
|
assert len(rev_rows) == 1, (
|
||||||
|
"expected exactly one event_status_reverted event"
|
||||||
|
)
|
||||||
|
rev_payload = json.loads(rev_rows[0][0])
|
||||||
|
assert rev_payload["event_id"] == "evt_started"
|
||||||
|
assert rev_payload["prior_status"] == "planned"
|
||||||
|
|
||||||
|
# Events projection: status is back to 'planned'.
|
||||||
|
status = conn.execute(
|
||||||
|
"SELECT status FROM events WHERE event_id = ?",
|
||||||
|
("evt_started",),
|
||||||
|
).fetchone()[0]
|
||||||
|
assert status == "planned"
|
||||||
|
|
||||||
|
|
||||||
|
def test_regenerate_rolls_back_event_completed_to_active(tmp_path, monkeypatch):
|
||||||
|
"""T114.3: a completed event whose completion was triggered by the
|
||||||
|
superseded turn rolls back to 'active'. Mirrors the started→planned
|
||||||
|
case but exercises the 'completed → active' branch of
|
||||||
|
``_PRIOR_STATUS_MAP`` in regenerate.
|
||||||
|
"""
|
||||||
|
import asyncio
|
||||||
|
|
||||||
|
from chat.config import Settings
|
||||||
|
from chat.db.migrate import apply_migrations
|
||||||
|
from chat.services.regenerate import regenerate_assistant_turn
|
||||||
|
|
||||||
|
db_path = tmp_path / "test.db"
|
||||||
|
cfg = tmp_path / "config.toml"
|
||||||
|
cfg.write_text('featherless_api_key = "test"\n')
|
||||||
|
monkeypatch.setenv("CHAT_CONFIG_PATH", str(cfg))
|
||||||
|
monkeypatch.setenv("CHAT_DB_PATH", str(db_path))
|
||||||
|
apply_migrations(db_path)
|
||||||
|
|
||||||
|
_ut_id, at_id = _seed_with_one_turn(db_path)
|
||||||
|
# The forward sequence here pretends the prior turn ALSO authored
|
||||||
|
# the start (which is realistic — a single turn flow could go
|
||||||
|
# planned → active → completed across multiple events). Tagging
|
||||||
|
# both with the same back-reference exercises the multi-rollback
|
||||||
|
# loop (one per affected lifecycle row).
|
||||||
|
_seed_event_with_lifecycle(
|
||||||
|
db_path,
|
||||||
|
event_id="evt_completed",
|
||||||
|
triggered_by_assistant_turn_id=at_id,
|
||||||
|
forward_kinds=["event_started", "event_completed"],
|
||||||
|
)
|
||||||
|
|
||||||
|
# Sanity: events row is 'completed'.
|
||||||
|
with open_db(db_path) as conn:
|
||||||
|
status = conn.execute(
|
||||||
|
"SELECT status FROM events WHERE event_id = ?", ("evt_completed",)
|
||||||
|
).fetchone()[0]
|
||||||
|
assert status == "completed"
|
||||||
|
|
||||||
|
state_canned = json.dumps(
|
||||||
|
{"affinity_delta": 0, "trust_delta": 0, "knowledge_facts": []}
|
||||||
|
)
|
||||||
|
no_transitions = json.dumps({"transitions": []})
|
||||||
|
mock_client = MockLLMClient(
|
||||||
|
canned=["Refreshed reply.", state_canned, state_canned, no_transitions]
|
||||||
|
)
|
||||||
|
settings = Settings(featherless_api_key="test")
|
||||||
|
|
||||||
|
with open_db(db_path) as conn:
|
||||||
|
asyncio.run(
|
||||||
|
regenerate_assistant_turn(
|
||||||
|
conn,
|
||||||
|
mock_client,
|
||||||
|
settings=settings,
|
||||||
|
chat_id="chat_bot_a",
|
||||||
|
original_assistant_event_id=at_id,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
with open_db(db_path) as conn:
|
||||||
|
# Two event_status_reverted rows land — one per forward
|
||||||
|
# transition that carried the back-reference. Both target the
|
||||||
|
# same event_id but with different prior_status values
|
||||||
|
# (in event_log id order: started→planned, completed→active).
|
||||||
|
rev_rows = conn.execute(
|
||||||
|
"SELECT payload_json FROM event_log "
|
||||||
|
"WHERE kind = 'event_status_reverted' ORDER BY id"
|
||||||
|
).fetchall()
|
||||||
|
assert len(rev_rows) == 2
|
||||||
|
rev_payloads = [json.loads(r[0]) for r in rev_rows]
|
||||||
|
assert rev_payloads[0] == {
|
||||||
|
"event_id": "evt_completed",
|
||||||
|
"prior_status": "planned",
|
||||||
|
}
|
||||||
|
assert rev_payloads[1] == {
|
||||||
|
"event_id": "evt_completed",
|
||||||
|
"prior_status": "active",
|
||||||
|
}
|
||||||
|
|
||||||
|
# Events projection: the LAST applied event_status_reverted
|
||||||
|
# wins (active). That's the desired final state for a turn
|
||||||
|
# that was originally a started+completed double-step.
|
||||||
|
status = conn.execute(
|
||||||
|
"SELECT status FROM events WHERE event_id = ?",
|
||||||
|
("evt_completed",),
|
||||||
|
).fetchone()[0]
|
||||||
|
assert status == "active"
|
||||||
|
|
||||||
|
|
||||||
|
def test_regenerate_skips_events_without_back_reference(
|
||||||
|
tmp_path, monkeypatch, caplog
|
||||||
|
):
|
||||||
|
"""T114.3 backward compatibility: lifecycle events authored before
|
||||||
|
T114.1 lack the ``triggered_by_assistant_turn_id`` payload field.
|
||||||
|
Regenerate must NOT emit ``event_status_reverted`` for such rows —
|
||||||
|
they're skipped (with a DEBUG log). The legacy T83.4 WARNING about
|
||||||
|
un-rolled-back transitions still fires for visibility.
|
||||||
|
"""
|
||||||
|
import asyncio
|
||||||
|
import logging
|
||||||
|
|
||||||
|
from chat.config import Settings
|
||||||
|
from chat.db.migrate import apply_migrations
|
||||||
|
from chat.eventlog.log import append_and_apply
|
||||||
|
from chat.services.regenerate import regenerate_assistant_turn
|
||||||
|
|
||||||
|
db_path = tmp_path / "test.db"
|
||||||
|
cfg = tmp_path / "config.toml"
|
||||||
|
cfg.write_text('featherless_api_key = "test"\n')
|
||||||
|
monkeypatch.setenv("CHAT_CONFIG_PATH", str(cfg))
|
||||||
|
monkeypatch.setenv("CHAT_DB_PATH", str(db_path))
|
||||||
|
apply_migrations(db_path)
|
||||||
|
|
||||||
|
_ut_id, at_id = _seed_with_one_turn(db_path)
|
||||||
|
|
||||||
|
# Seed a lifecycle transition WITHOUT the back-reference field —
|
||||||
|
# mimicking pre-T114.1 event_log rows.
|
||||||
|
with open_db(db_path) as conn:
|
||||||
|
append_and_apply(
|
||||||
|
conn,
|
||||||
|
kind="event_planned",
|
||||||
|
payload={
|
||||||
|
"event_id": "evt_legacy",
|
||||||
|
"chat_id": "chat_bot_a",
|
||||||
|
"kind": "story_event",
|
||||||
|
"props": {},
|
||||||
|
"planned_for": "2026-04-30T18:00:00+00:00",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
append_and_apply(
|
||||||
|
conn,
|
||||||
|
kind="event_started",
|
||||||
|
payload={
|
||||||
|
"event_id": "evt_legacy",
|
||||||
|
"started_at": "2026-04-30T19:00:00+00:00",
|
||||||
|
# NOTE: no triggered_by_assistant_turn_id — pre-T114.1
|
||||||
|
# legacy row.
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
state_canned = json.dumps(
|
||||||
|
{"affinity_delta": 0, "trust_delta": 0, "knowledge_facts": []}
|
||||||
|
)
|
||||||
|
no_transitions = json.dumps({"transitions": []})
|
||||||
|
mock_client = MockLLMClient(
|
||||||
|
canned=["Refreshed reply.", state_canned, state_canned, no_transitions]
|
||||||
|
)
|
||||||
|
settings = Settings(featherless_api_key="test")
|
||||||
|
|
||||||
|
caplog.set_level(logging.DEBUG, logger="chat.services.regenerate")
|
||||||
|
|
||||||
|
with open_db(db_path) as conn:
|
||||||
|
asyncio.run(
|
||||||
|
regenerate_assistant_turn(
|
||||||
|
conn,
|
||||||
|
mock_client,
|
||||||
|
settings=settings,
|
||||||
|
chat_id="chat_bot_a",
|
||||||
|
original_assistant_event_id=at_id,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
with open_db(db_path) as conn:
|
||||||
|
# No event_status_reverted was emitted for the legacy row.
|
||||||
|
rev_count = conn.execute(
|
||||||
|
"SELECT COUNT(*) FROM event_log "
|
||||||
|
"WHERE kind = 'event_status_reverted'"
|
||||||
|
).fetchone()[0]
|
||||||
|
assert rev_count == 0
|
||||||
|
|
||||||
|
# Events row is still 'active' — the legacy transition stands.
|
||||||
|
status = conn.execute(
|
||||||
|
"SELECT status FROM events WHERE event_id = ?",
|
||||||
|
("evt_legacy",),
|
||||||
|
).fetchone()[0]
|
||||||
|
assert status == "active"
|
||||||
|
|
||||||
|
# Debug log surfaces the skipped row.
|
||||||
|
debugs = [
|
||||||
|
r.getMessage()
|
||||||
|
for r in caplog.records
|
||||||
|
if r.levelname == "DEBUG"
|
||||||
|
]
|
||||||
|
assert any(
|
||||||
|
"skipping rollback for lifecycle event_log" in m for m in debugs
|
||||||
|
), f"expected DEBUG about skipped legacy row; got: {debugs}"
|
||||||
|
|
||||||
|
# Legacy WARNING still fires so operators see un-rolled-back rows.
|
||||||
|
warnings = [
|
||||||
|
r.getMessage()
|
||||||
|
for r in caplog.records
|
||||||
|
if r.levelname == "WARNING"
|
||||||
|
and "lifecycle transition" in r.getMessage()
|
||||||
|
]
|
||||||
|
assert warnings, (
|
||||||
|
"expected WARNING about un-rolled-back legacy lifecycle "
|
||||||
|
f"transitions; got records: "
|
||||||
|
f"{[r.getMessage() for r in caplog.records]}"
|
||||||
|
)
|
||||||
|
# The new wording references the missing back-reference field.
|
||||||
|
assert "triggered_by_assistant_turn_id" in warnings[0]
|
||||||
|
|||||||
+52
-32
@@ -22,6 +22,7 @@ from chat.db.connection import open_db
|
|||||||
from chat.eventlog.log import append_and_apply, append_event
|
from chat.eventlog.log import append_and_apply, append_event
|
||||||
from chat.eventlog.projector import project
|
from chat.eventlog.projector import project
|
||||||
from chat.llm.mock import MockLLMClient
|
from chat.llm.mock import MockLLMClient
|
||||||
|
from tests.fixtures import CannedQueue
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
@pytest.fixture
|
||||||
@@ -362,14 +363,20 @@ def test_single_bot_turn_no_guest_regression(app_state_setup, tmp_path):
|
|||||||
the chat has no guest, so ``detect_interjection`` is NOT invoked.
|
the chat has no guest, so ``detect_interjection`` is NOT invoked.
|
||||||
Ends with one user_turn, one assistant_turn, two edge_updates, and a
|
Ends with one user_turn, one assistant_turn, two edge_updates, and a
|
||||||
single ``memory_written``.
|
single ``memory_written``.
|
||||||
|
|
||||||
|
T116: migrated to :class:`tests.fixtures.CannedQueue` as a proof of
|
||||||
|
concept for the structured canned-queue builder.
|
||||||
"""
|
"""
|
||||||
_seed(tmp_path / "test.db")
|
_seed(tmp_path / "test.db")
|
||||||
canned_parse = json.dumps(
|
canned = (
|
||||||
{"segments": [{"kind": "dialogue", "text": "hello"}]}
|
CannedQueue()
|
||||||
)
|
.parse_turn(segments=[{"kind": "dialogue", "text": "hello"}])
|
||||||
mock = _override_llm(
|
.narrative("Hi there.")
|
||||||
[canned_parse, "Hi there.", _zero_state(), _zero_state()]
|
.state_update()
|
||||||
|
.state_update()
|
||||||
|
.build()
|
||||||
)
|
)
|
||||||
|
mock = _override_llm(canned)
|
||||||
try:
|
try:
|
||||||
response = app_state_setup.post(
|
response = app_state_setup.post(
|
||||||
"/chats/chat_bot_a/turns", data={"prose": "hello"}
|
"/chats/chat_bot_a/turns", data={"prose": "hello"}
|
||||||
@@ -979,29 +986,25 @@ def test_turn_with_event_transition_appends_started_event(
|
|||||||
},
|
},
|
||||||
)
|
)
|
||||||
|
|
||||||
canned_parse = json.dumps(
|
# T116: migrated to :class:`tests.fixtures.CannedQueue`.
|
||||||
{"segments": [{"kind": "dialogue", "text": "they arrived"}]}
|
canned = (
|
||||||
)
|
CannedQueue()
|
||||||
canned_event_decision = json.dumps(
|
.parse_turn(segments=[{"kind": "dialogue", "text": "they arrived"}])
|
||||||
{
|
.narrative("They walk in.")
|
||||||
"transitions": [
|
.state_update()
|
||||||
{
|
.state_update()
|
||||||
"event_id": "evt_1",
|
.detect_event_transitions(
|
||||||
"new_status": "active",
|
[
|
||||||
"reason": "they arrived",
|
{
|
||||||
}
|
"event_id": "evt_1",
|
||||||
]
|
"new_status": "active",
|
||||||
}
|
"reason": "they arrived",
|
||||||
)
|
}
|
||||||
mock = _override_llm(
|
]
|
||||||
[
|
)
|
||||||
canned_parse,
|
.build()
|
||||||
"They walk in.",
|
|
||||||
_zero_state(),
|
|
||||||
_zero_state(),
|
|
||||||
canned_event_decision,
|
|
||||||
]
|
|
||||||
)
|
)
|
||||||
|
mock = _override_llm(canned)
|
||||||
try:
|
try:
|
||||||
response = app_state_setup.post(
|
response = app_state_setup.post(
|
||||||
"/chats/chat_bot_a/turns", data={"prose": "they arrived"}
|
"/chats/chat_bot_a/turns", data={"prose": "they arrived"}
|
||||||
@@ -1023,6 +1026,18 @@ def test_turn_with_event_transition_appends_started_event(
|
|||||||
assert started_payload["event_id"] == "evt_1"
|
assert started_payload["event_id"] == "evt_1"
|
||||||
assert started_payload["started_at"] == "2026-04-26T20:00:00+00:00"
|
assert started_payload["started_at"] == "2026-04-26T20:00:00+00:00"
|
||||||
|
|
||||||
|
# T114.1: payload carries the back-reference to the assistant_turn
|
||||||
|
# that triggered the transition. The assistant_turn lands in
|
||||||
|
# event_log immediately before the event_started, so its id is
|
||||||
|
# the largest assistant_turn id in the chat at this point.
|
||||||
|
at_id = conn.execute(
|
||||||
|
"SELECT id FROM event_log "
|
||||||
|
"WHERE kind = 'assistant_turn' "
|
||||||
|
" AND json_extract(payload_json, '$.chat_id') = 'chat_bot_a' "
|
||||||
|
"ORDER BY id DESC LIMIT 1"
|
||||||
|
).fetchone()[0]
|
||||||
|
assert started_payload["triggered_by_assistant_turn_id"] == at_id
|
||||||
|
|
||||||
# The events projection row reflects the active status.
|
# The events projection row reflects the active status.
|
||||||
ev_row = conn.execute(
|
ev_row = conn.execute(
|
||||||
"SELECT status, started_at FROM events WHERE event_id = ?",
|
"SELECT status, started_at FROM events WHERE event_id = ?",
|
||||||
@@ -1143,18 +1158,23 @@ def test_turn_with_no_active_events_skips_classifier(app_state_setup, tmp_path):
|
|||||||
short-circuits without an LLM call (per T52). The canned queue must
|
short-circuits without an LLM call (per T52). The canned queue must
|
||||||
therefore have ZERO event-detection slots — same shape as the
|
therefore have ZERO event-detection slots — same shape as the
|
||||||
Phase 2 no-guest baseline.
|
Phase 2 no-guest baseline.
|
||||||
|
|
||||||
|
T116: migrated to :class:`tests.fixtures.CannedQueue`.
|
||||||
"""
|
"""
|
||||||
_seed(tmp_path / "test.db")
|
_seed(tmp_path / "test.db")
|
||||||
|
|
||||||
canned_parse = json.dumps(
|
|
||||||
{"segments": [{"kind": "dialogue", "text": "hello"}]}
|
|
||||||
)
|
|
||||||
# Only 4 slots: parse + narrative + 2 state-updates. NO extra slot for
|
# Only 4 slots: parse + narrative + 2 state-updates. NO extra slot for
|
||||||
# event-detection — non-existent active_events causes the helper to
|
# event-detection — non-existent active_events causes the helper to
|
||||||
# short-circuit before pulling from the queue.
|
# short-circuit before pulling from the queue.
|
||||||
mock = _override_llm(
|
canned = (
|
||||||
[canned_parse, "Hi there.", _zero_state(), _zero_state()]
|
CannedQueue()
|
||||||
|
.parse_turn(segments=[{"kind": "dialogue", "text": "hello"}])
|
||||||
|
.narrative("Hi there.")
|
||||||
|
.state_update()
|
||||||
|
.state_update()
|
||||||
|
.build()
|
||||||
)
|
)
|
||||||
|
mock = _override_llm(canned)
|
||||||
try:
|
try:
|
||||||
response = app_state_setup.post(
|
response = app_state_setup.post(
|
||||||
"/chats/chat_bot_a/turns", data={"prose": "hello"}
|
"/chats/chat_bot_a/turns", data={"prose": "hello"}
|
||||||
|
|||||||
Reference in New Issue
Block a user