10 Commits

Author SHA1 Message Date
Joseph Doherty 789b9bd042 merge: T71 prompt.py polish (witness role + ACTIVITIES + NICE trim docs) 2026-04-26 17:18:02 -04:00
Joseph Doherty 73bb8c1f17 chore: document NICE trim order rationale (T71.3)
T18 review (Phase 1) noted the NICE-tier trim drops previous-scene
FIRST while §6.3 spec lists previous-scene LAST in the NICE tier
group. Decision: keep the existing greedy order (previous-scene
first), and document why.

Rationale (now in code at the trim ladder):
  1. Cheapest-impact-first — a per-POV previous-scene summary loses
     less narrative continuity than the older dialogue turns or
     memory hits it competes with.
  2. Greedy lookahead is more expensive than the marginal narrative
     loss. Dropping previous-scene typically clears the soft-budget
     slack in one step.

Test added: test_nice_trim_order_documented pins the observed order
(previous-scene -> memories -> dialogue) so a future refactor can't
silently invert it. Sized so that all-NICE config overflows soft but
dropping just previous-scene fits — proves memories and older
dialogue turns survive while previous-scene is the FIRST drop.
2026-04-26 17:16:02 -04:00
Joseph Doherty afd1a50958 refactor: single ACTIVITIES: block with bullet-level trim (T71.2)
Phase 2 T43 added a SECOND ACTIVITIES: block to render guest activity
separately from you+speaker. Two consecutive ACTIVITIES: headers can
read as a duplicate-section bug to the LLM and bloat the prompt.

Consolidate to a single ACTIVITIES: block whose body is composed from
up to three bullets (you, speaker, guest). The block itself is
MUST-tier (always renders); bullet-level trim drops bullets in the
order guest -> group node -> you -> other edges, with the speaker
bullet as the MUST-tier floor (the speaker's own current activity is
the load-bearing slice).

Implementation chose Option B from the polish plan: pre-truncate the
bullets list at trim time before _build_activity_block runs, rather
than introduce a granular tier mode in the trim machinery. Rationale
documented in code; the existing block-level trim ladder gains a
single new toggle (include_you_activity) and the SHOULD-tier
guest_activity_block is gone.

Tests:
- test_single_activities_block_with_three_bullets_when_3_entities:
  exactly one ACTIVITIES: header with all three entity bullets.
- test_tight_budget_drops_guest_activity_bullet_first: speaker bullet
  survives, guest bullet absent under tight budget.
- Existing test_assemble_with_tight_budget_drops_guest_activity_first
  still passes (asserts on bullet absence, not block-header absence).
2026-04-26 17:13:24 -04:00
Joseph Doherty 428438b223 fix: witness role parametric in prompt assembly (T71.1)
Phase 2 T46 pinned the witness mask contract on search_memories with a
witness_role parameter (host/guest/you). The prompt-assembly call site
in assemble_narrative_prompt was hardcoded to "host", which silently
returned the wrong rows when the speaker was the guest bot.

Derive the witness role from chat membership via a new private helper
_witness_role_for(speaker_bot_id, host_bot_id), and apply it at the
search_memories call. Behaviour is identical when the speaker is the
host (or when no guest is present); the fix is load-bearing only when
the guest bot is the speaker — exactly the scenario Phase 2 T43 added
support for.

Tests: pin both directions (host-as-speaker and guest-as-speaker) by
patching the imported search_memories reference and asserting the
witness_role argument the call site emits.
2026-04-26 17:11:20 -04:00
Joseph Doherty b13f3b4e47 merge: T70 LLM-merged group meta-summary 2026-04-26 17:09:16 -04:00
Joseph Doherty f701f9d7dd merge: T69 bot_reset purges orphaned 'you' activity rows 2026-04-26 17:09:16 -04:00
Joseph Doherty 1b9144442a merge: T68 open_db with check_same_thread parameter 2026-04-26 17:09:16 -04:00
Joseph Doherty 13c23fd898 feat: LLM-merged group meta-summary (T70) 2026-04-26 17:07:12 -04:00
Joseph Doherty c1e419e012 fix: bot_reset purges orphaned 'you' activity rows (T69) 2026-04-26 17:06:21 -04:00
Joseph Doherty 994728b5ed refactor: open_db with check_same_thread parameter (T68) 2026-04-26 17:05:29 -04:00
9 changed files with 896 additions and 67 deletions
+2 -2
View File
@@ -5,9 +5,9 @@ from pathlib import Path
@contextmanager @contextmanager
def open_db(path: Path): def open_db(path: Path, *, check_same_thread: bool = True):
path.parent.mkdir(parents=True, exist_ok=True) path.parent.mkdir(parents=True, exist_ok=True)
conn = sqlite3.connect(path) conn = sqlite3.connect(path, check_same_thread=check_same_thread)
conn.execute("PRAGMA journal_mode=WAL") conn.execute("PRAGMA journal_mode=WAL")
conn.execute("PRAGMA foreign_keys=ON") conn.execute("PRAGMA foreign_keys=ON")
try: try:
+130 -41
View File
@@ -273,6 +273,18 @@ def _resolve_previous_scene_summary(
return mem[0] return mem[0]
def _witness_role_for(speaker_bot_id: str, host_bot_id: str | None) -> str:
"""Return the witness POV role for the speaker's memory query.
The host bot of a chat queries memories with ``witness_role="host"``;
the guest bot queries with ``witness_role="guest"``. Phase 2 T46
pinned the contract on ``search_memories``; this helper applies it
at the call site so a guest-as-speaker doesn't silently retrieve
memories under the wrong POV mask.
"""
return "host" if speaker_bot_id == host_bot_id else "guest"
def _resolve_addressee( def _resolve_addressee(
conn: Connection, addressee: str, you: dict | None conn: Connection, addressee: str, you: dict | None
) -> tuple[str, str]: ) -> tuple[str, str]:
@@ -356,34 +368,57 @@ def assemble_narrative_prompt(
addressee_name, addressee_name,
) )
# Activity for present entities. Core (MUST): you + speaker bot. # Activity for present entities — single ACTIVITIES: block with up
# Phase 2 (SHOULD-tier): when a third party (guest) is present in # to three bullets (you, speaker, guest). The block itself is
# the chat, append their activity in a separate block so it can be # MUST-tier and survives all trims, but bullet-level trim drops
# trimmed independently under tight budget. # bullets in the order guest -> you, keeping the speaker bullet
activities: list[dict] = [] # (the speaker's own current activity is the load-bearing slice).
#
# T71.2 chose Option B from the polish plan: pre-truncate the
# bullets list at trim time before _build_activity_block runs,
# rather than introducing a granular tier mode in the trim
# machinery. The single-block render avoids the dual-ACTIVITIES:
# header that Phase 2 T43 introduced (read by some LLMs as a
# duplicate-section bug).
you_activity: dict | None = None
you_act = get_activity(conn, "you") you_act = get_activity(conn, "you")
if you_act is not None: if you_act is not None:
you_act = dict(you_act) you_activity = dict(you_act)
you_act["_display_name"] = (you or {}).get("name") or "you" you_activity["_display_name"] = (you or {}).get("name") or "you"
activities.append(you_act)
speaker_activity: dict | None = None
bot_act = get_activity(conn, speaker_bot_id) bot_act = get_activity(conn, speaker_bot_id)
if bot_act is not None: if bot_act is not None:
bot_act = dict(bot_act) speaker_activity = dict(bot_act)
bot_act["_display_name"] = bot["name"] speaker_activity["_display_name"] = bot["name"]
activities.append(bot_act)
activity_block = _build_activity_block(activities)
# SHOULD-tier guest activity extension (Phase 2 / Task 43). guest_activity: dict | None = None
guest_activity_block: str | None = None
if guest_id is not None: if guest_id is not None:
guest_act = get_activity(conn, guest_id) guest_act = get_activity(conn, guest_id)
if guest_act is not None: if guest_act is not None:
guest_act = dict(guest_act) guest_activity = dict(guest_act)
guest_bot = get_bot(conn, guest_id) guest_bot = get_bot(conn, guest_id)
guest_act["_display_name"] = ( guest_activity["_display_name"] = (
guest_bot["name"] if guest_bot else guest_id guest_bot["name"] if guest_bot else guest_id
) )
guest_activity_block = _build_activity_block([guest_act])
def _activity_block_for(
*, include_you: bool, include_guest: bool
) -> str | None:
"""Render the single ACTIVITIES: block with the requested bullets.
Speaker bullet is always included (it's the MUST-tier baseline);
``you`` and ``guest`` bullets are toggled by the caller during
trim. Returns None when no bullets remain.
"""
bullets: list[dict] = []
if include_you and you_activity is not None:
bullets.append(you_activity)
if speaker_activity is not None:
bullets.append(speaker_activity)
if include_guest and guest_activity is not None:
bullets.append(guest_activity)
return _build_activity_block(bullets)
# SHOULD-tier group-node block (Phase 2 / Task 43): rendered only # SHOULD-tier group-node block (Phase 2 / Task 43): rendered only
# when the group_node row is present AND it covers all three of # when the group_node row is present AND it covers all three of
@@ -433,7 +468,12 @@ def assemble_narrative_prompt(
memory_summaries = [] memory_summaries = []
if query: if query:
try: try:
hits = search_memories(conn, speaker_bot_id, "host", query, k=4) witness_role = _witness_role_for(
speaker_bot_id, chat.get("host_bot_id")
)
hits = search_memories(
conn, speaker_bot_id, witness_role, query, k=4
)
memory_summaries = [h["pov_summary"] for h in hits] memory_summaries = [h["pov_summary"] for h in hits]
except Exception: except Exception:
memory_summaries = [] memory_summaries = []
@@ -452,11 +492,18 @@ def assemble_narrative_prompt(
last4 = dialogue_full[-4:] if dialogue_full else [] last4 = dialogue_full[-4:] if dialogue_full else []
must_dialogue_block = _build_dialogue_block(last4, earlier_summary=None) must_dialogue_block = _build_dialogue_block(last4, earlier_summary=None)
# MUST-tier ACTIVITIES floor: the speaker bullet alone (you and
# guest bullets are dropped first under bullet-level trim before
# the block bottoms out at speaker-only).
must_activity_block = _activity_block_for(
include_you=False, include_guest=False
)
must_blocks: list[str | None] = [ must_blocks: list[str | None] = [
speaker_identity, speaker_identity,
edge_to_addressee, edge_to_addressee,
scene_block, scene_block,
activity_block, must_activity_block,
must_dialogue_block, must_dialogue_block,
closing, closing,
] ]
@@ -481,6 +528,7 @@ def assemble_narrative_prompt(
include_previous_scene: bool, include_previous_scene: bool,
include_memories_top_k: int, include_memories_top_k: int,
dialogue_keep: int, dialogue_keep: int,
include_you_activity: bool = True,
include_guest_activity: bool = True, include_guest_activity: bool = True,
include_group_node: bool = True, include_group_node: bool = True,
) -> tuple[str, int, list[dict]]: ) -> tuple[str, int, list[dict]]:
@@ -503,13 +551,20 @@ def assemble_narrative_prompt(
if include_previous_scene else None if include_previous_scene else None
) )
# Single ACTIVITIES: block, bullet-level trim (T71.2). Guest
# bullet drops first, then the you bullet; speaker bullet is the
# MUST-tier floor and always present when an activity row exists.
activity_block = _activity_block_for(
include_you=include_you_activity,
include_guest=include_guest_activity,
)
body = _join_blocks([ body = _join_blocks([
speaker_identity, speaker_identity,
edge_to_addressee, edge_to_addressee,
other_edges_block if include_other_edges else None, other_edges_block if include_other_edges else None,
scene_block, scene_block,
activity_block, activity_block,
guest_activity_block if include_guest_activity else None,
group_node_block if include_group_node else None, group_node_block if include_group_node else None,
prev_block, prev_block,
memories_block, memories_block,
@@ -527,16 +582,18 @@ def assemble_narrative_prompt(
nice_memories_k = min(4, len(memory_summaries)) nice_memories_k = min(4, len(memory_summaries))
include_prev = previous_scene_summary is not None include_prev = previous_scene_summary is not None
include_other = other_edges_block is not None include_other = other_edges_block is not None
include_guest_activity = guest_activity_block is not None include_you_activity = you_activity is not None
include_guest_activity = guest_activity is not None
include_group_node = group_node_block is not None include_group_node = group_node_block is not None
def _build(*, prev: bool, mem_k: int, dlg: int, other: bool, def _build(*, prev: bool, mem_k: int, dlg: int, other: bool,
guest_act: bool, group: bool) -> tuple[str, int]: you_act: bool, guest_act: bool, group: bool) -> tuple[str, int]:
body, total, _ = assemble( body, total, _ = assemble(
include_other_edges=other, include_other_edges=other,
include_previous_scene=prev, include_previous_scene=prev,
include_memories_top_k=mem_k, include_memories_top_k=mem_k,
dialogue_keep=dlg, dialogue_keep=dlg,
include_you_activity=you_act,
include_guest_activity=guest_act, include_guest_activity=guest_act,
include_group_node=group, include_group_node=group,
) )
@@ -544,8 +601,8 @@ def assemble_narrative_prompt(
body, total = _build( body, total = _build(
prev=include_prev, mem_k=nice_memories_k, dlg=nice_dialogue_keep, prev=include_prev, mem_k=nice_memories_k, dlg=nice_dialogue_keep,
other=include_other, guest_act=include_guest_activity, other=include_other, you_act=include_you_activity,
group=include_group_node, guest_act=include_guest_activity, group=include_group_node,
) )
# If under soft, we're done. # If under soft, we're done.
@@ -554,12 +611,32 @@ def assemble_narrative_prompt(
# Drop NICE in order: previous scene → memories beyond top-2 → # Drop NICE in order: previous scene → memories beyond top-2 →
# older dialogue turns (collapse to 4). # older dialogue turns (collapse to 4).
#
# T71.3 — order rationale: the §6.3 spec lists NICE-tier members
# with previous-scene LAST, which read as a literal trim order
# during T18 review. We deliberately keep the greedy order shown
# here (previous-scene FIRST) for two reasons:
#
# 1. Cheapest-impact-first: a per-POV previous-scene summary is
# a single short paragraph that loses very little narrative
# continuity when dropped, while the older dialogue turns it
# is competing with carry the speaker's last few beats — those
# ground the next response far more concretely.
# 2. Greedy lookahead is more expensive than the marginal
# narrative loss. Dropping previous-scene typically clears
# the soft-budget slack in one step; trying memories or
# dialogue first would routinely require multiple recompute
# passes through the assembler.
#
# The pin test test_nice_trim_order_documented locks this order so
# a future refactor can't quietly invert it without surfacing the
# decision.
if include_prev: if include_prev:
include_prev = False include_prev = False
body, total = _build( body, total = _build(
prev=include_prev, mem_k=nice_memories_k, dlg=nice_dialogue_keep, prev=include_prev, mem_k=nice_memories_k, dlg=nice_dialogue_keep,
other=include_other, guest_act=include_guest_activity, other=include_other, you_act=include_you_activity,
group=include_group_node, guest_act=include_guest_activity, group=include_group_node,
) )
if total <= budget_soft: if total <= budget_soft:
return _emit(body, user_turn_prose) return _emit(body, user_turn_prose)
@@ -568,8 +645,8 @@ def assemble_narrative_prompt(
nice_memories_k = 2 nice_memories_k = 2
body, total = _build( body, total = _build(
prev=include_prev, mem_k=nice_memories_k, dlg=nice_dialogue_keep, prev=include_prev, mem_k=nice_memories_k, dlg=nice_dialogue_keep,
other=include_other, guest_act=include_guest_activity, other=include_other, you_act=include_you_activity,
group=include_group_node, guest_act=include_guest_activity, group=include_group_node,
) )
if total <= budget_soft: if total <= budget_soft:
return _emit(body, user_turn_prose) return _emit(body, user_turn_prose)
@@ -578,8 +655,8 @@ def assemble_narrative_prompt(
nice_dialogue_keep = baseline_keep nice_dialogue_keep = baseline_keep
body, total = _build( body, total = _build(
prev=include_prev, mem_k=nice_memories_k, dlg=nice_dialogue_keep, prev=include_prev, mem_k=nice_memories_k, dlg=nice_dialogue_keep,
other=include_other, guest_act=include_guest_activity, other=include_other, you_act=include_you_activity,
group=include_group_node, guest_act=include_guest_activity, group=include_group_node,
) )
if total <= budget_soft: if total <= budget_soft:
return _emit(body, user_turn_prose) return _emit(body, user_turn_prose)
@@ -589,35 +666,47 @@ def assemble_narrative_prompt(
nice_memories_k = max(0, nice_memories_k - 1) nice_memories_k = max(0, nice_memories_k - 1)
body, total = _build( body, total = _build(
prev=include_prev, mem_k=nice_memories_k, dlg=nice_dialogue_keep, prev=include_prev, mem_k=nice_memories_k, dlg=nice_dialogue_keep,
other=include_other, guest_act=include_guest_activity, other=include_other, you_act=include_you_activity,
group=include_group_node, guest_act=include_guest_activity, group=include_group_node,
) )
# Drop SHOULD-tier blocks in order: guest activity → group node → # Drop SHOULD-tier extras in order:
# other edges. (Guest activity goes first per Task 43 spec — it's # 1. guest activity bullet (T71.2: bullet-level trim within the
# the most expendable additive context.) # single ACTIVITIES: block — guest goes first per Task 43 spec)
# 2. group node block
# 3. you activity bullet (still SHOULD-tier; speaker bullet is the
# MUST-tier floor and never dropped)
# 4. other edges
if include_guest_activity and total > budget_hard: if include_guest_activity and total > budget_hard:
include_guest_activity = False include_guest_activity = False
body, total = _build( body, total = _build(
prev=include_prev, mem_k=nice_memories_k, dlg=nice_dialogue_keep, prev=include_prev, mem_k=nice_memories_k, dlg=nice_dialogue_keep,
other=include_other, guest_act=include_guest_activity, other=include_other, you_act=include_you_activity,
group=include_group_node, guest_act=include_guest_activity, group=include_group_node,
) )
if include_group_node and total > budget_hard: if include_group_node and total > budget_hard:
include_group_node = False include_group_node = False
body, total = _build( body, total = _build(
prev=include_prev, mem_k=nice_memories_k, dlg=nice_dialogue_keep, prev=include_prev, mem_k=nice_memories_k, dlg=nice_dialogue_keep,
other=include_other, guest_act=include_guest_activity, other=include_other, you_act=include_you_activity,
group=include_group_node, guest_act=include_guest_activity, group=include_group_node,
)
if include_you_activity and total > budget_hard:
include_you_activity = False
body, total = _build(
prev=include_prev, mem_k=nice_memories_k, dlg=nice_dialogue_keep,
other=include_other, you_act=include_you_activity,
guest_act=include_guest_activity, group=include_group_node,
) )
if include_other and total > budget_hard: if include_other and total > budget_hard:
include_other = False include_other = False
body, total = _build( body, total = _build(
prev=include_prev, mem_k=nice_memories_k, dlg=nice_dialogue_keep, prev=include_prev, mem_k=nice_memories_k, dlg=nice_dialogue_keep,
other=include_other, guest_act=include_guest_activity, other=include_other, you_act=include_you_activity,
group=include_group_node, guest_act=include_guest_activity, group=include_group_node,
) )
if total > budget_hard: if total > budget_hard:
+74 -8
View File
@@ -334,26 +334,92 @@ async def apply_scene_close_summary(
timeout_s=timeout_s, timeout_s=timeout_s,
) )
# Group node update: naive per-POV concat for v2. Only fires when # Group node update: T70 runs a third classifier call to merge the
# both POVs ran (i.e. the guest is present) and a group_node row # two per-POV summaries into a coherent group-level view + a brief
# exists for this chat. # group-dynamic note. Falls back to the Phase 2 naive concat on
# classifier failure (see :func:`merge_group_summary`). Only fires
# when both POVs ran (i.e. the guest is present) and a group_node
# row exists for this chat.
if guest_pov is not None and get_group_node(conn, chat_id) is not None: if guest_pov is not None and get_group_node(conn, chat_id) is not None:
host_bot = get_bot(conn, host_bot_id) or {"name": host_bot_id} host_bot = get_bot(conn, host_bot_id) or {"name": host_bot_id}
guest_bot = get_bot(conn, guest_bot_id) or {"name": guest_bot_id} guest_bot = get_bot(conn, guest_bot_id) or {"name": guest_bot_id}
host_name = host_bot.get("name", host_bot_id) or host_bot_id host_name = host_bot.get("name", host_bot_id) or host_bot_id
guest_name = guest_bot.get("name", guest_bot_id) or guest_bot_id guest_name = guest_bot.get("name", guest_bot_id) or guest_bot_id
group_summary = ( merged = await merge_group_summary(
f"{host_name}: {host_pov.summary}\n\n" client,
f"{guest_name}: {guest_pov.summary}" classifier_model=classifier_model,
host_name=host_name,
host_pov_summary=host_pov.summary,
guest_name=guest_name,
guest_pov_summary=guest_pov.summary,
timeout_s=timeout_s,
) )
append_and_apply( append_and_apply(
conn, conn,
kind="group_node_updated", kind="group_node_updated",
payload={ payload={
"chat_id": chat_id, "chat_id": chat_id,
"summary": group_summary, "summary": merged.summary,
"dynamic": "", "dynamic": merged.dynamic,
}, },
) )
return host_pov return host_pov
class GroupMetaSummary(BaseModel):
"""Classifier output: a merged group-level view of a closed scene.
Defaults are an empty no-op so callers can use the schema's default
as a sentinel; in practice :func:`merge_group_summary` builds an
explicit naive-concat fallback rather than returning these defaults
directly so existing Phase 2 behavior is preserved on classifier
failure.
"""
summary: str = ""
dynamic: str = ""
_GROUP_MERGE_SYSTEM = (
"Given two per-POV scene summaries from a 3-entity scene (you + "
"host + guest), produce a coherent group-level summary capturing "
"the shared events as both witnesses experienced them, plus a "
"brief 'dynamic' note describing the trio's group dynamic during "
"the scene. Output strict JSON matching schema."
)
async def merge_group_summary(
client: LLMClient,
*,
classifier_model: str,
host_name: str,
host_pov_summary: str,
guest_name: str,
guest_pov_summary: str,
timeout_s: float = 30.0,
) -> GroupMetaSummary:
"""Merge two per-POV scene summaries into a coherent group-level
summary + group-dynamic note. Falls back to the naive concat (the
existing behavior) on classifier failure."""
user = (
f"{host_name} (host) POV summary:\n{host_pov_summary}\n\n"
f"{guest_name} (guest) POV summary:\n{guest_pov_summary}"
)
fallback = GroupMetaSummary(
summary=(
f"{host_name}: {host_pov_summary}\n\n"
f"{guest_name}: {guest_pov_summary}"
),
dynamic="",
)
return await classify(
client,
model=classifier_model,
system=_GROUP_MERGE_SYSTEM,
user=user,
schema=GroupMetaSummary,
default=fallback,
timeout_s=timeout_s,
)
+11 -2
View File
@@ -48,6 +48,17 @@ def _apply_bot_reset(conn: Connection, e: Event) -> None:
"SELECT id FROM chats WHERE host_bot_id = ?", (bot_id,) "SELECT id FROM chats WHERE host_bot_id = ?", (bot_id,)
).fetchall() ).fetchall()
] ]
# T69: purge orphaned "you" activity rows pointing at containers in this
# bot's chats BEFORE the containers/chats themselves are deleted, otherwise
# the subqueries find nothing and the FK constraint on activity.container_id
# blocks the container delete.
conn.execute(
"DELETE FROM activity WHERE entity_id = 'you' "
"AND container_id IN (SELECT id FROM containers WHERE chat_id IN ("
" SELECT id FROM chats WHERE host_bot_id = ?"
"))",
(bot_id,),
)
for chat_id in chat_ids: for chat_id in chat_ids:
conn.execute("DELETE FROM scenes WHERE chat_id = ?", (chat_id,)) conn.execute("DELETE FROM scenes WHERE chat_id = ?", (chat_id,))
conn.execute("DELETE FROM containers WHERE chat_id = ?", (chat_id,)) conn.execute("DELETE FROM containers WHERE chat_id = ?", (chat_id,))
@@ -74,8 +85,6 @@ def _apply_bot_reset(conn: Connection, e: Event) -> None:
(bot_id,), (bot_id,),
) )
# NOTE: bots row itself is preserved (identity, kickoff_prose intact). # NOTE: bots row itself is preserved (identity, kickoff_prose intact).
# NOTE: "you" activity (entity_id="you") may linger from a deleted chat;
# acceptable for v1 — Phase 1.5 cleanup if needed.
def get_bot(conn: Connection, bot_id: str) -> dict | None: def get_bot(conn: Connection, bot_id: str) -> dict | None:
+2 -9
View File
@@ -1,10 +1,10 @@
from __future__ import annotations from __future__ import annotations
import sqlite3
from pathlib import Path from pathlib import Path
from fastapi import APIRouter, Depends, Form, HTTPException, Request from fastapi import APIRouter, Depends, Form, HTTPException, Request
from fastapi.responses import RedirectResponse, HTMLResponse from fastapi.responses import RedirectResponse, HTMLResponse
from fastapi.templating import Jinja2Templates from fastapi.templating import Jinja2Templates
from chat.db.connection import open_db
from chat.eventlog.log import append_event from chat.eventlog.log import append_event
from chat.eventlog.projector import project from chat.eventlog.projector import project
from chat.state.entities import list_bots from chat.state.entities import list_bots
@@ -19,15 +19,8 @@ REQUIRED_FIELDS = ("id", "name", "persona", "initial_relationship_to_you", "kick
def get_conn(request: Request): def get_conn(request: Request):
settings = request.app.state.settings settings = request.app.state.settings
db_path: Path = settings.db_path db_path: Path = settings.db_path
db_path.parent.mkdir(parents=True, exist_ok=True) with open_db(db_path, check_same_thread=False) as conn:
conn = sqlite3.connect(db_path, check_same_thread=False)
conn.execute("PRAGMA journal_mode=WAL")
conn.execute("PRAGMA foreign_keys=ON")
try:
yield conn yield conn
conn.commit()
finally:
conn.close()
def _split_voice_samples(text: str) -> list[str]: def _split_voice_samples(text: str) -> list[str]:
+57
View File
@@ -0,0 +1,57 @@
from __future__ import annotations
import sqlite3
import threading
from chat.db.connection import open_db
def test_open_db_default_uses_check_same_thread_true(tmp_path):
"""Default open_db must reject cross-thread use (safe default)."""
db = tmp_path / "t.db"
captured: list[BaseException | None] = []
with open_db(db) as conn:
conn.execute("CREATE TABLE t (x INTEGER)")
def worker():
try:
conn.execute("SELECT 1").fetchall()
captured.append(None)
except BaseException as e: # noqa: BLE001
captured.append(e)
t = threading.Thread(target=worker)
t.start()
t.join()
assert len(captured) == 1
err = captured[0]
assert isinstance(err, sqlite3.ProgrammingError), (
f"expected sqlite3.ProgrammingError on cross-thread use, got {err!r}"
)
def test_open_db_can_disable_check_same_thread(tmp_path):
"""open_db(check_same_thread=False) must allow cross-thread use."""
db = tmp_path / "t.db"
captured: list[BaseException | None] = []
rows: list[object] = []
with open_db(db, check_same_thread=False) as conn:
conn.execute("CREATE TABLE t (x INTEGER)")
conn.execute("INSERT INTO t (x) VALUES (42)")
def worker():
try:
result = conn.execute("SELECT x FROM t").fetchall()
rows.extend(result)
captured.append(None)
except BaseException as e: # noqa: BLE001
captured.append(e)
t = threading.Thread(target=worker)
t.start()
t.join()
assert captured == [None], f"worker thread raised: {captured}"
assert rows == [(42,)]
+170 -5
View File
@@ -636,8 +636,10 @@ async def test_close_with_guest_updates_both_edges(tmp_path):
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_close_with_group_node_updates_group_summary(tmp_path): async def test_close_with_group_node_updates_group_summary(tmp_path):
"""When a group_node row exists, scene close emits group_node_updated """When a group_node row exists, scene close emits group_node_updated
with a non-empty summary that mentions both bots' names (v2 naive with a non-empty summary that mentions both bots' names. T70 swapped
concat of per-POV summaries).""" the Phase 2 naive concat for an LLM-merged summary; this regression
test feeds bad-JSON merge responses so the helper falls back to the
original naive-concat shape, preserving the original assertions."""
db = tmp_path / "t.db" db = tmp_path / "t.db"
apply_migrations(db) apply_migrations(db)
import chat.state.group_node # noqa: F401 -- register handlers import chat.state.group_node # noqa: F401 -- register handlers
@@ -660,7 +662,11 @@ async def test_close_with_group_node_updates_group_summary(tmp_path):
_seed_two_bot_scene(conn, with_group_node=True) _seed_two_bot_scene(conn, with_group_node=True)
project(conn) project(conn)
client = MockLLMClient(canned=[host_canned, guest_canned]) # 2 valid (host POV, guest POV) + 3 bad-JSON merge attempts ->
# merge_group_summary falls back to the naive concat default.
client = MockLLMClient(
canned=[host_canned, guest_canned, "bad1", "bad2", "bad3"]
)
await apply_scene_close_summary( await apply_scene_close_summary(
conn, conn,
client, client,
@@ -675,8 +681,167 @@ async def test_close_with_group_node_updates_group_summary(tmp_path):
gn = get_group_node(conn, "chat_bot_a") gn = get_group_node(conn, "chat_bot_a")
assert gn is not None assert gn is not None
assert gn["summary"] # non-empty assert gn["summary"] # non-empty
# Naive concat surfaces both bot names in the group summary. # Naive-concat fallback surfaces both bot names in the group summary.
assert "BotA" in gn["summary"] assert "BotA" in gn["summary"]
assert "BotB" in gn["summary"] assert "BotB" in gn["summary"]
# Phase 2 v2 keeps dynamic empty (Phase 3 polishes). # Naive-concat fallback keeps dynamic empty.
assert gn["dynamic"] == "" assert gn["dynamic"] == ""
# ---------------------------------------------------------------------------
# T70: LLM-merged group meta-summary on scene close.
# ---------------------------------------------------------------------------
@pytest.mark.asyncio
async def test_group_summary_merges_per_pov_via_classifier_when_guest_present(
tmp_path,
):
"""With a guest present and a group_node row, scene close runs the
merge classifier as a third call after the two per-POV summarize_scene
calls; its output drives the group_node summary + dynamic fields."""
db = tmp_path / "t.db"
apply_migrations(db)
import chat.state.group_node # noqa: F401 -- register handlers
host_canned = json.dumps(
{
"summary": "BotA appreciated the calm.",
"knowledge_facts": [],
"relationship_summary": "BotA felt steady.",
}
)
guest_canned = json.dumps(
{
"summary": "BotB found the room friendly.",
"knowledge_facts": [],
"relationship_summary": "BotB warmed up.",
}
)
merge_canned = json.dumps(
{"summary": "merged group view", "dynamic": "warm rapport"}
)
with open_db(db) as conn:
_seed_two_bot_scene(conn, with_group_node=True)
project(conn)
# Canned-queue layout matches the production call order in
# apply_scene_close_summary: host POV summarize_scene runs first,
# then guest POV summarize_scene, then merge_group_summary.
client = MockLLMClient(
canned=[host_canned, guest_canned, merge_canned]
)
await apply_scene_close_summary(
conn,
client,
classifier_model="x",
chat_id="chat_bot_a",
scene_id=1,
host_bot_id="bot_a",
)
# All three canned entries consumed -> classifier ran exactly 3x.
assert client._canned == []
from chat.state.group_node import get_group_node
gn = get_group_node(conn, "chat_bot_a")
assert gn is not None
assert gn["summary"] == "merged group view"
assert gn["dynamic"] == "warm rapport"
@pytest.mark.asyncio
async def test_group_summary_falls_back_to_naive_concat_on_classifier_failure(
tmp_path,
):
"""If the merge classifier flaps (bad JSON across all 3 retries), the
helper falls back to the original Phase 2 naive concat shape and
leaves dynamic empty."""
db = tmp_path / "t.db"
apply_migrations(db)
import chat.state.group_node # noqa: F401 -- register handlers
host_canned = json.dumps(
{
"summary": "BotA appreciated the calm.",
"knowledge_facts": [],
"relationship_summary": "BotA felt steady.",
}
)
guest_canned = json.dumps(
{
"summary": "BotB found the room friendly.",
"knowledge_facts": [],
"relationship_summary": "BotB warmed up.",
}
)
with open_db(db) as conn:
_seed_two_bot_scene(conn, with_group_node=True)
project(conn)
# 2 valid POV summaries + 3 bad-JSON merge attempts trip the
# classifier's retry-then-default path; the default is the naive
# concat fallback.
client = MockLLMClient(
canned=[host_canned, guest_canned, "bad1", "bad2", "bad3"]
)
await apply_scene_close_summary(
conn,
client,
classifier_model="x",
chat_id="chat_bot_a",
scene_id=1,
host_bot_id="bot_a",
)
from chat.state.group_node import get_group_node
gn = get_group_node(conn, "chat_bot_a")
assert gn is not None
expected = (
"BotA: BotA appreciated the calm.\n\n"
"BotB: BotB found the room friendly."
)
assert gn["summary"] == expected
assert gn["dynamic"] == ""
@pytest.mark.asyncio
async def test_group_summary_skipped_when_no_guest(tmp_path):
"""No-guest path: scene close does NOT invoke merge_group_summary
and emits no group_node_updated event. Confirms the existing
`if guest_bot_id is not None` gating at the call site."""
db = tmp_path / "t.db"
apply_migrations(db)
canned = json.dumps(
{
"summary": "BotA helped you talk through the deadline anxiety.",
"knowledge_facts": ["Deadline next Friday."],
"relationship_summary": "BotA leaned in supportively.",
}
)
with open_db(db) as conn:
_seed_single_bot_scene(conn)
project(conn)
# Only 1 canned entry; if merge_group_summary were called the
# MockLLMClient would IndexError on the empty queue.
client = MockLLMClient(canned=[canned])
await apply_scene_close_summary(
conn,
client,
classifier_model="x",
chat_id="chat_bot_a",
scene_id=1,
host_bot_id="bot_a",
)
# Exactly the host POV call consumed, nothing else.
assert client._canned == []
# No group_node_updated event was emitted.
rows = conn.execute(
"SELECT 1 FROM event_log WHERE kind = 'group_node_updated'"
).fetchall()
assert rows == []
+267
View File
@@ -452,6 +452,273 @@ def test_assemble_when_speaker_is_guest_orients_edges_correctly(tmp_path):
assert "68/100" in body assert "68/100" in body
def test_speaker_is_guest_uses_guest_witness_role(tmp_path, monkeypatch):
"""T71.1: when the guest is the speaker, ``search_memories`` is
called with ``witness_role="guest"``, not the previously-hardcoded
``"host"``. Pins the parametric witness role at the prompt call site
so guest-as-speaker honours the witness mask via Phase 2 T46.
"""
db = tmp_path / "t.db"
apply_migrations(db)
captured: dict = {}
def _fake_search(conn, owner_id, witness_role, query, k=4):
captured["owner_id"] = owner_id
captured["witness_role"] = witness_role
captured["query"] = query
return []
# Patch the imported reference inside the prompt module so the
# production call site uses the fake.
import chat.services.prompt as prompt_mod
monkeypatch.setattr(prompt_mod, "search_memories", _fake_search)
with open_db(db) as conn:
_seed_with_guest(conn)
# Guest as speaker — must request memories with witness_role="guest".
assemble_narrative_prompt(
conn,
chat_id="chat_bot_a",
speaker_bot_id="bot_b",
recent_dialogue=[],
# retrieved_memory_summaries=None forces the search path.
retrieved_memory_summaries=None,
)
assert captured["owner_id"] == "bot_b"
assert captured["witness_role"] == "guest"
def test_speaker_is_host_uses_host_witness_role(tmp_path, monkeypatch):
"""T71.1 (regression): host-as-speaker still queries with
``witness_role="host"``."""
db = tmp_path / "t.db"
apply_migrations(db)
captured: dict = {}
def _fake_search(conn, owner_id, witness_role, query, k=4):
captured["witness_role"] = witness_role
return []
import chat.services.prompt as prompt_mod
monkeypatch.setattr(prompt_mod, "search_memories", _fake_search)
with open_db(db) as conn:
_seed_with_guest(conn)
assemble_narrative_prompt(
conn,
chat_id="chat_bot_a",
speaker_bot_id="bot_a", # host as speaker
recent_dialogue=[],
retrieved_memory_summaries=None,
)
assert captured["witness_role"] == "host"
def test_single_activities_block_with_three_bullets_when_3_entities(tmp_path):
"""T71.2: with you + host + guest present, the assembled prompt
contains exactly ONE ``ACTIVITIES:`` header and bullets for all
three entities (no duplicate header from the prior dual-block
rendering).
"""
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
_seed_with_guest(conn)
msgs = assemble_narrative_prompt(
conn,
chat_id="chat_bot_a",
speaker_bot_id="bot_a",
recent_dialogue=[],
retrieved_memory_summaries=[],
)
body = msgs[0].content
# Exactly one ACTIVITIES: header.
assert body.count("ACTIVITIES:") == 1
# Bullets for all three entities (you=Sam, host=Aria, guest=Iris)
# — pin on the unique action verbs from the seed data.
assert "finishing emails" in body # you bullet
assert "pretending to work" in body # speaker (host) bullet
assert "smirking-distinctively" in body # guest bullet
def test_tight_budget_drops_guest_activity_bullet_first(tmp_path):
"""T71.2: under tight budget the speaker bullet survives but the
guest activity bullet is the first ACTIVITIES: bullet to drop. The
block as a whole stays present (it's MUST-tier); only its body
contracts.
"""
db = tmp_path / "t.db"
apply_migrations(db)
with open_db(db) as conn:
_seed_with_guest(conn)
dialogue = [
{"speaker": "you", "text": "line-16 hi there"},
{"speaker": "bot_a", "text": "line-17 hey"},
{"speaker": "you", "text": "line-18 quiet night"},
{"speaker": "bot_a", "text": "line-19 indeed"},
]
msgs = assemble_narrative_prompt(
conn,
chat_id="chat_bot_a",
speaker_bot_id="bot_a",
recent_dialogue=dialogue,
retrieved_memory_summaries=[],
budget_soft=250,
budget_hard=340,
)
body = msgs[0].content
# Speaker bullet survives (MUST-tier floor).
assert "pretending to work" in body
assert "ACTIVITIES:" in body
# Guest bullet is dropped first under budget pressure.
assert "smirking-distinctively" not in body
def test_nice_trim_order_documented(tmp_path):
"""T71.3: pin the NICE-tier trim order so a future refactor can't
quietly invert it.
Order under NICE pressure is:
1. previous-scene summary (dropped FIRST)
2. memories beyond top-2
3. older dialogue turns (collapsed to last-4)
We size the budget so that all-NICE-included is over soft, but
dropping ONLY previous-scene gets us back under soft. The observed
behaviour we pin: previous-scene gone, memories/dialogue intact.
"""
db = tmp_path / "t.db"
apply_migrations(db)
# Heavy previous-scene summary — large enough that dropping it
# alone clears the soft-budget overage. Defined out here so the
# marker is in scope for the assertions below.
prev_scene_blob = "PREVSCENE-MARKER " + ("filler " * 200)
with open_db(db) as conn:
# Append all events first, project once at the end (project is
# not idempotent — it replays every event in the log).
from chat.eventlog.log import append_event as _append
_append(conn, kind="bot_authored", payload={
"id": "bot_a",
"name": "Aria",
"persona": "reserved coworker who notices things",
"voice_samples": ["I — sorry, I didn't mean to."],
"traits": ["introverted"],
"backstory": "An archivist who joined the firm last spring.",
"initial_relationship_to_you": "coworker",
"kickoff_prose": "you stay late at the office",
})
_append(conn, kind="you_authored", payload={
"name": "Sam",
"pronouns": "they/them",
"persona": "tired analyst",
})
_append(conn, kind="chat_created", payload={
"id": "chat_bot_a",
"host_bot_id": "bot_a",
"guest_bot_id": None,
"initial_time": "2026-04-26T20:00:00+00:00",
"narrative_anchor": "Day 1 evening",
"weather": "clear",
})
_append(conn, kind="container_created", payload={
"chat_id": "chat_bot_a",
"name": "office bullpen",
"type": "workplace",
"properties": {"public": False, "moving": False, "audible_range": "room"},
})
_append(conn, kind="edge_update", payload={
"source_id": "bot_a",
"target_id": "you",
"affinity_delta": 12,
"trust_delta": 5,
"knowledge_facts": ["they work on the same floor"],
})
_append(conn, kind="activity_change", payload={
"entity_id": "you",
"container_id": 1,
"posture": "sitting at your desk",
"action": {"verb": "finishing emails"},
"attention": "the screen",
})
_append(conn, kind="activity_change", payload={
"entity_id": "bot_a",
"container_id": 1,
"posture": "sitting at her desk",
"action": {"verb": "pretending to work"},
"attention": "you, in glances",
})
_append(conn, kind="scene_opened", payload={
"chat_id": "chat_bot_a",
"container_id": 1,
"started_at": "2026-04-26T20:00:00+00:00",
"participants": ["you", "bot_a"],
})
# Close the seeded scene and write a per-POV summary memory so
# _resolve_previous_scene_summary returns a non-empty string.
_append(conn, kind="scene_closed", payload={
"scene_id": 1,
"ended_at": "2026-04-26T20:30:00+00:00",
"significance": 2,
})
_append(conn, kind="memory_written", payload={
"owner_id": "bot_a",
"chat_id": "chat_bot_a",
"scene_id": 1,
"pov_summary": prev_scene_blob,
"witness_you": 1,
"witness_host": 1,
"witness_guest": 0,
"source": "direct",
"reliability": 1.0,
"significance": 2,
})
project(conn)
# Six dialogue turns — last 4 plus 2 older. If older turns are
# dropped under NICE pressure, the unique markers for turns 0/1
# disappear; we'll assert they REMAIN to prove dialogue trim
# didn't fire.
dialogue = [
{"speaker": "you", "text": "DLG-OLD-00 hello"},
{"speaker": "bot_a", "text": "DLG-OLD-01 hi"},
{"speaker": "you", "text": "DLG-LAST-16 ok"},
{"speaker": "bot_a", "text": "DLG-LAST-17 sure"},
{"speaker": "you", "text": "DLG-LAST-18 night"},
{"speaker": "bot_a", "text": "DLG-LAST-19 indeed"},
]
# Four small memories — if "memories beyond top-2" trim fires,
# MEM-C/MEM-D disappear; we'll assert they REMAIN to prove
# memories trim didn't fire either.
memories = ["MEM-A short", "MEM-B short", "MEM-C short", "MEM-D short"]
# Soft tuned so the all-NICE config (with the heavy previous
# scene summary) overflows, but dropping just previous-scene
# fits comfortably. Hard set high so SHOULD-tier never trims.
msgs = assemble_narrative_prompt(
conn,
chat_id="chat_bot_a",
speaker_bot_id="bot_a",
recent_dialogue=dialogue,
retrieved_memory_summaries=memories,
budget_soft=400,
budget_hard=8000,
)
body = msgs[0].content
# Previous-scene summary was the FIRST NICE drop — its unique
# marker must be absent.
assert "PREVSCENE-MARKER" not in body
# Memories beyond top-2 stayed (proves memories trim did NOT fire).
assert "MEM-A" in body
assert "MEM-B" in body
assert "MEM-C" in body
assert "MEM-D" in body
# Older dialogue turns stayed (proves dialogue trim did NOT fire).
assert "DLG-OLD-00" in body
assert "DLG-OLD-01" in body
# Last-4 dialogue turns of course present.
assert "DLG-LAST-19" in body
def test_assemble_with_tight_budget_drops_guest_activity_first(tmp_path): def test_assemble_with_tight_budget_drops_guest_activity_first(tmp_path):
"""Under tight budget MUST blocks survive but SHOULD-tier guest """Under tight budget MUST blocks survive but SHOULD-tier guest
activity is dropped first.""" activity is dropped first."""
+183
View File
@@ -292,6 +292,189 @@ def test_reset_clears_guest_reference_in_other_chats(client, tmp_path):
).fetchone()[0] == 1 ).fetchone()[0] == 1
def test_reset_purges_orphaned_you_activity_rows(client, tmp_path):
"""T69: when a bot's chats are deleted, "you" activity rows tied to those
chats' containers should also be purged (otherwise they linger orphaned)."""
db = tmp_path / "test.db"
with open_db(db) as conn:
append_event(
conn,
kind="bot_authored",
payload={
"id": "bot_a",
"name": "BotA",
"persona": "thoughtful",
"voice_samples": [],
"traits": [],
"backstory": "",
"initial_relationship_to_you": "coworker",
"kickoff_prose": "",
},
)
append_event(
conn,
kind="chat_created",
payload={
"id": "chat_bot_a",
"host_bot_id": "bot_a",
"initial_time": "2026-04-26T20:00:00+00:00",
"narrative_anchor": "Day 1",
"weather": "",
},
)
append_event(
conn,
kind="container_created",
payload={
"chat_id": "chat_bot_a",
"name": "office",
"type": "workplace",
"properties": {},
},
)
append_event(
conn,
kind="activity_change",
payload={
"entity_id": "you",
"container_id": 1,
"posture": "standing",
"action": {"verb": "watching"},
},
)
project(conn)
# Sanity: the "you" activity row exists and points at the container.
assert conn.execute(
"SELECT COUNT(*) FROM activity WHERE entity_id = 'you'"
).fetchone()[0] == 1
response = client.post(
"/bots/bot_a/reset",
data={"confirm_name": "BotA"},
follow_redirects=False,
)
assert response.status_code == 303
with open_db(db) as conn:
# The orphaned "you" activity row tied to bot_a's purged container is gone.
assert conn.execute(
"SELECT COUNT(*) FROM activity WHERE entity_id = 'you'"
).fetchone()[0] == 0
def test_reset_does_not_purge_you_activity_in_other_chats(client, tmp_path):
"""T69: resetting bot_a must leave a "you" activity row pointing at
bot_b's container intact — only orphans from the reset bot's chats go."""
db = tmp_path / "test.db"
with open_db(db) as conn:
# bot_a + its chat + container.
append_event(
conn,
kind="bot_authored",
payload={
"id": "bot_a",
"name": "BotA",
"persona": "thoughtful",
"voice_samples": [],
"traits": [],
"backstory": "",
"initial_relationship_to_you": "coworker",
"kickoff_prose": "",
},
)
append_event(
conn,
kind="chat_created",
payload={
"id": "chat_bot_a",
"host_bot_id": "bot_a",
"initial_time": "2026-04-26T20:00:00+00:00",
"narrative_anchor": "Day 1",
"weather": "",
},
)
append_event(
conn,
kind="container_created",
payload={
"chat_id": "chat_bot_a",
"name": "office",
"type": "workplace",
"properties": {},
},
)
# bot_b + its chat + container.
append_event(
conn,
kind="bot_authored",
payload={
"id": "bot_b",
"name": "BotB",
"persona": "curious",
"voice_samples": [],
"traits": [],
"backstory": "",
"initial_relationship_to_you": "friend",
"kickoff_prose": "",
},
)
append_event(
conn,
kind="chat_created",
payload={
"id": "chat_bot_b",
"host_bot_id": "bot_b",
"initial_time": "2026-04-26T20:00:00+00:00",
"narrative_anchor": "Day 1",
"weather": "",
},
)
append_event(
conn,
kind="container_created",
payload={
"chat_id": "chat_bot_b",
"name": "kitchen",
"type": "home",
"properties": {},
},
)
# The activity table is keyed on entity_id (PRIMARY KEY), so only one
# "you" row exists at a time. Point it at bot_b's container so reset of
# bot_a should NOT touch it.
append_event(
conn,
kind="activity_change",
payload={
"entity_id": "you",
"container_id": 2, # kitchen, in chat_bot_b
"posture": "sitting",
"action": {"verb": "reading"},
},
)
project(conn)
# Sanity: the "you" activity row is in bot_b's container.
row = conn.execute(
"SELECT container_id FROM activity WHERE entity_id = 'you'"
).fetchone()
assert row is not None and row[0] == 2
response = client.post(
"/bots/bot_a/reset",
data={"confirm_name": "BotA"},
follow_redirects=False,
)
assert response.status_code == 303
with open_db(db) as conn:
# The "you" activity in bot_b's container is preserved.
row = conn.execute(
"SELECT container_id FROM activity WHERE entity_id = 'you'"
).fetchone()
assert row is not None
assert row[0] == 2
def test_reset_purges_guest_memories_from_other_chats(client, tmp_path): def test_reset_purges_guest_memories_from_other_chats(client, tmp_path):
db = tmp_path / "test.db" db = tmp_path / "test.db"
_seed_two_bots_with_guest_link( _seed_two_bots_with_guest_link(