Joseph Doherty
5aab98e4d7
The kickoff parse-and-confirm route was 500-ing intermittently because
Hermes-3 + Featherless's response_format={"type":"json_object"} only
guarantees JSON output, NOT a particular schema. The model was inventing
its own field names (sceneTime, entities, settingDetails) instead of
the KickoffParse fields, causing Pydantic validation to fail on both
classify() retries.
Three changes:
1. Include the Pydantic JSON schema in the system prompt so the model
knows exactly which keys to produce. Affects every classify() call
(kickoff parse, turn parse, scene-close detect, significance,
state-update, scene summarize). Strip ```json fences if the model
wraps its output. Bump retries 2 → 3 (model is stochastic; one extra
attempt closes most of the remaining gap).
2. parse_kickoff() now passes a default empty KickoffParse so the
route degrades to a fillable form instead of 500 when the classifier
ultimately fails. The confirm form is the human-in-the-loop; an
empty form is strictly better UX than a stack trace.
3. Tests updated: bumped canned-failure arrays from 2 → 3 entries to
match the new attempt count; renamed kickoff test from
"raises_when_classifier_fails_twice" to
"falls_back_to_empty_when_classifier_fails" reflecting the new
degraded-but-usable behavior.
Verified live with all 3 sample bots (maya/eli/sam) — kickoff route
returns 200 across multiple attempts. Full suite: 168 passed.
63 lines
2.2 KiB
Python
63 lines
2.2 KiB
Python
from __future__ import annotations
|
|
import json
|
|
import asyncio
|
|
from typing import TypeVar
|
|
from pydantic import BaseModel, ValidationError
|
|
from .client import LLMClient, Message
|
|
|
|
T = TypeVar("T", bound=BaseModel)
|
|
|
|
REFUSAL_PATTERNS = ("i can't", "i cannot", "i'm sorry, but", "as an ai")
|
|
|
|
|
|
def _strip_json_fences(text: str) -> str:
|
|
"""Strip ```json ... ``` markdown fences if the model wraps its JSON output."""
|
|
s = text.strip()
|
|
if s.startswith("```"):
|
|
# Drop the first fence line (which may be ``` or ```json)
|
|
s = s.split("\n", 1)[1] if "\n" in s else s[3:]
|
|
# Drop the trailing fence
|
|
if s.rstrip().endswith("```"):
|
|
s = s.rstrip()[:-3]
|
|
return s.strip()
|
|
|
|
|
|
async def classify(
|
|
client: LLMClient,
|
|
*,
|
|
model: str,
|
|
system: str,
|
|
user: str,
|
|
schema: type[T],
|
|
default: T | None = None,
|
|
timeout_s: float = 10.0,
|
|
) -> T:
|
|
schema_json = json.dumps(schema.model_json_schema(), indent=2)
|
|
schema_block = (
|
|
f"\n\nRespond with a single JSON object matching this exact schema. "
|
|
f"Use these field names exactly; do not invent your own keys:\n```json\n{schema_json}\n```"
|
|
)
|
|
msgs = [
|
|
Message(role="system", content=system + schema_block),
|
|
Message(role="user", content=user),
|
|
]
|
|
for attempt in range(3):
|
|
try:
|
|
text = await asyncio.wait_for(
|
|
client.generate(msgs, model=model, response_format={"type": "json_object"}),
|
|
timeout=timeout_s,
|
|
)
|
|
cleaned = _strip_json_fences(text)
|
|
if any(p in cleaned.lower()[:80] for p in REFUSAL_PATTERNS) and not cleaned.lstrip().startswith("{"):
|
|
raise ValueError("refusal-shaped response")
|
|
return schema.model_validate_json(cleaned)
|
|
except (ValidationError, ValueError, json.JSONDecodeError, asyncio.TimeoutError):
|
|
msgs[0] = Message(
|
|
role="system",
|
|
content=system + schema_block + "\n\nRespond with valid JSON ONLY. No prose, no markdown fences.",
|
|
)
|
|
continue
|
|
if default is None:
|
|
raise RuntimeError(f"classify failed for schema {schema.__name__} with no default")
|
|
return default
|