Joseph Doherty
c874883a84
Replace the substring _detect_addressee_id helper with a classifier call for the multi-entity case. The substring helper is kept as a fast-path for the no-guest case (no LLM round-trip needed when only one bot is present, preserves throughput). - New service chat/services/addressee.py wrapping the existing classifier wrapper. AddresseeDecision carries addressee_id + confidence + reason; classifier failure falls back to the host with reason="fallback" (graceful-degradation, matches the relationship_seed / interjection pattern). - chat/web/turns.py post_turn now calls detect_addressee in the multi-entity branch; 1:1 keeps the substring path. - tests/test_addressee.py: 3 new tests (guest pick, host pick, classifier-failure fallback). - tests/test_turn_flow.py: existing multi-entity tests now feed a canned addressee response in the queue. The addressee-routing test is updated to assert classifier-driven routing rather than substring.
109 lines
3.7 KiB
Python
109 lines
3.7 KiB
Python
"""Addressee classifier service (T74.1).
|
|
|
|
Phase 2 (T44) detected the addressee — host vs. guest — with a simple
|
|
case-insensitive whole-word substring match against the bots' names.
|
|
That worked for the obvious case ("BotB, what do you think?") but lost
|
|
the long tail: pronouns, paraphrases, indirect address, narrative
|
|
focus on a particular party. T74.1 swaps the substring helper for a
|
|
classifier call that reads the prose holistically.
|
|
|
|
The substring helper in :mod:`chat.web.turns` is kept as a fast-path
|
|
for the no-guest case (only one bot present means there is nothing to
|
|
classify) and as a non-breaking fallback for the regenerate path. The
|
|
multi-entity branch in :func:`chat.web.turns.post_turn` calls
|
|
:func:`detect_addressee` from this module.
|
|
|
|
Failure mode: classifier flake or low-confidence response degrades to
|
|
the host (the default speaker per Phase 2's host-keeps-the-floor
|
|
bias). The decision carries ``confidence`` and ``reason`` so callers
|
|
that want to log degraded decisions can distinguish a real "host" call
|
|
from a fallback.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from pydantic import BaseModel
|
|
|
|
from chat.llm.classify import classify
|
|
from chat.llm.client import LLMClient
|
|
|
|
|
|
class AddresseeDecision(BaseModel):
|
|
"""Which present bot the user is addressing.
|
|
|
|
``addressee_id`` is the chosen bot's id. ``confidence`` is one of
|
|
``"high"`` / ``"medium"`` / ``"low"`` — callers may treat ``"low"``
|
|
as a soft fallback to the host. ``reason`` is a short free-form
|
|
string. The classifier-failure fallback uses ``reason="fallback"``
|
|
so it's distinguishable from a real low-confidence call.
|
|
"""
|
|
|
|
addressee_id: str
|
|
confidence: str = "medium" # "high" | "medium" | "low"
|
|
reason: str = ""
|
|
|
|
|
|
_SYSTEM = (
|
|
"Given a user's turn prose and the names of present bots, decide "
|
|
"which bot the user is addressing. If the user is speaking to no "
|
|
"specific bot (descriptive narration, action without dialogue), "
|
|
"default to the host. Output strict JSON matching the schema. "
|
|
"The addressee_id MUST be one of the ids supplied in the user "
|
|
"message — do not invent ids."
|
|
)
|
|
|
|
|
|
async def detect_addressee(
|
|
client: LLMClient,
|
|
*,
|
|
classifier_model: str,
|
|
user_prose: str,
|
|
host_id: str,
|
|
host_name: str,
|
|
guest_id: str | None,
|
|
guest_name: str | None,
|
|
timeout_s: float = 30.0,
|
|
) -> AddresseeDecision:
|
|
"""Classify which present bot the user is addressing.
|
|
|
|
Defaults to host on classifier failure or when the classifier picks
|
|
an id that isn't one of the supplied ids. The caller is expected to
|
|
only invoke this in the multi-entity case (a guest is present);
|
|
when no guest is present the substring fast-path in
|
|
:mod:`chat.web.turns` is used instead and this function is not
|
|
called.
|
|
"""
|
|
fallback = AddresseeDecision(
|
|
addressee_id=host_id, confidence="low", reason="fallback"
|
|
)
|
|
user = (
|
|
f"Host: {host_name} (id={host_id})\n"
|
|
+ (
|
|
f"Guest: {guest_name} (id={guest_id})\n"
|
|
if guest_id is not None
|
|
else ""
|
|
)
|
|
+ f"\nUser prose:\n{user_prose}"
|
|
)
|
|
decision = await classify(
|
|
client,
|
|
model=classifier_model,
|
|
system=_SYSTEM,
|
|
user=user,
|
|
schema=AddresseeDecision,
|
|
default=fallback,
|
|
timeout_s=timeout_s,
|
|
)
|
|
# Defensive: if the classifier returned an id outside the supplied
|
|
# set, treat it as a fallback to the host. This catches pathological
|
|
# outputs that pass schema validation but pick a phantom id.
|
|
valid_ids = {host_id}
|
|
if guest_id is not None:
|
|
valid_ids.add(guest_id)
|
|
if decision.addressee_id not in valid_ids:
|
|
return fallback
|
|
return decision
|
|
|
|
|
|
__all__ = ["AddresseeDecision", "detect_addressee"]
|