feat: cap narrative response length + tune sampling

Bot replies were running long (4 paragraphs of action+dialogue beats
per turn) because we never set max_tokens on the narrative call. Three
tunable knobs now in Settings (set in data/config.toml to override):

- narrative_max_tokens: int = 400
  Hard cap on each generated response. ~400 tokens ≈ 1–2 short
  paragraphs. Drop to 200 for terse banter, bump to 800+ for longer
  scenes.

- narrative_temperature: float = 0.85
  Sampling temperature. 0.7 = grounded/consistent (slightly stiff),
  0.85 = creative-but-in-character (default), 1.0 = wide variety,
  >1.0 = often off-the-rails.

- prompt closing instruction now nudges: "Keep your response to a
  single beat — one or two short paragraphs at most. Don't monologue;
  leave room for the other person to react."

Both turns.py (post_turn) and regenerate.py forward the params to
client.stream(). FeatherlessClient already passes **params through to
the OpenAI-compat endpoint.

Note: temperature doesn't control length — that was a common
misconception. max_tokens is the actual length cap. Lower temperature
makes word choice more predictable (slightly stiffer voice), not
shorter. Both knobs are useful for different goals.
This commit is contained in:
Joseph Doherty
2026-04-26 15:28:08 -04:00
parent f0742dd4f9
commit d161e7b8e9
4 changed files with 18 additions and 3 deletions
+3 -1
View File
@@ -211,7 +211,9 @@ def _closing_instruction(speaker_name: str, addressee_name: str) -> str:
f"Continue the scene as {speaker_name}, in their voice, responding "
"naturally. Use *asterisks* for actions and quotes for dialogue. "
f"Stay in character. Do not narrate {addressee_name}'s actions or "
"thoughts."
"thoughts. "
"Keep your response to a single beat — one or two short paragraphs "
"at most. Don't monologue; leave room for the other person to react."
)
+4 -1
View File
@@ -156,7 +156,10 @@ async def regenerate_assistant_turn(
# 5. Stream the new narrative.
accumulated: list[str] = []
async for chunk in client.stream(
messages, model=settings.narrative_model
messages,
model=settings.narrative_model,
max_tokens=settings.narrative_max_tokens,
temperature=settings.narrative_temperature,
):
accumulated.append(chunk)
await publish(