From b83d16364a0ea499360f3b833207687c32d02d63 Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Mon, 18 May 2026 23:13:17 -0400 Subject: [PATCH] docs(notification-outbox): retarget S&F notification category to central --- .../requirements/Component-StoreAndForward.md | 22 +++++++++++-------- 1 file changed, 13 insertions(+), 9 deletions(-) diff --git a/docs/requirements/Component-StoreAndForward.md b/docs/requirements/Component-StoreAndForward.md index bbb5834..446eeef 100644 --- a/docs/requirements/Component-StoreAndForward.md +++ b/docs/requirements/Component-StoreAndForward.md @@ -13,7 +13,7 @@ Site clusters only. The central cluster does not buffer messages. - Buffer outbound messages when the target system is unavailable. - Manage three categories of buffered messages: - External system API calls. - - Email notifications. + - Notifications forwarded to the central cluster. - Cached database writes. - Retry delivery per message according to the configured retry policy. - Park messages that exhaust their retry limit (dead-letter). @@ -42,13 +42,16 @@ Attempt immediate delivery └── Max retries exhausted → Park message ``` +For notifications, "delivery" means forwarding the message to the central cluster via the Communication Layer; "success" is central's ack, on which the message is cleared. Notifications do not park — they are retried at the fixed forward interval until central acks. Parking applies only to the external-system-call and cached-database-write categories. + ## Retry Policy -Retry settings are defined on the **source entity** (not per-message): +For the external-system-call and cached-database-write categories, retry settings are defined on the **source entity** (not per-message): - **External systems**: Each external system definition includes max retry count and time between retries. -- **Notifications**: Email/SMTP configuration includes max retry count and time between retries. - **Cached database writes**: Each database connection definition includes max retry count and time between retries. +The **notification** category retries differently: it has no source-entity setting. The site→central forward uses a single fixed retry interval configured in the host `appsettings.json`. This interval concerns reaching the central cluster — it is infrastructure config, not a per-notification-list setting — so it applies uniformly to every buffered notification regardless of its target list. A buffered notification is retried until central acks it; it is not parked on a retry limit (central, once reachable, owns delivery, retry, and parking from that point on). + The retry interval is **fixed** (not exponential backoff). Fixed interval is sufficient for the expected use cases. **Note**: Only **transient failures** are eligible for store-and-forward buffering. For external system calls, transient failures are connection errors, timeouts, and HTTP 5xx responses. Permanent failures (HTTP 4xx) are returned directly to the calling script and are **not** queued for retry. This prevents the buffer from accumulating requests that will never succeed. @@ -79,8 +82,8 @@ There is **no maximum buffer size**. Messages accumulate in the buffer until del Each buffered message stores: - **Message ID**: Unique identifier. - **Category**: External system call, notification, or cached database write. -- **Target**: External system name, notification list name, or database connection name. -- **Payload**: Serialized message content (API method + parameters, email subject + body, SQL + parameters). +- **Target**: External system name, the central cluster (for notifications), or database connection name. +- **Payload**: Serialized message content (API method + parameters; notification list name + subject + body plus the locally generated `NotificationId` and source provenance; SQL + parameters). - **Retry Count**: Number of attempts so far. - **Created At**: Timestamp when the message was first queued. - **Last Attempt At**: Timestamp of the most recent delivery attempt. @@ -89,14 +92,15 @@ Each buffered message stores: ## Dependencies - **SQLite**: Local persistence on each node. -- **Communication Layer**: Application-level replication to standby node; remote query handling from central. +- **Communication Layer**: Application-level replication to standby node; remote query handling from central; carries buffered notifications to the central cluster (ClusterClient) and receives central's acks. - **External System Gateway**: Delivers external system API calls. -- **Notification Service**: Delivers email notifications. +- **Central–Site Communication**: The delivery target for the notification category — a buffered notification is forwarded to the central cluster over the Communication Layer and cleared on central's ack. - **Database Connections**: Delivers cached database writes. - **Site Event Logging**: Logs store-and-forward activity (queued, delivered, retried, parked). ## Interactions - **Site Runtime (Script Actors)**: Scripts submit messages to the buffer (external calls, notifications, cached DB writes). -- **Communication Layer**: Handles parked message queries/commands from central. -- **Health Monitoring**: Reports buffer depth metrics. +- **Communication Layer**: Handles parked message queries/commands from central; carries buffered notifications to the central cluster. +- **Notification Outbox**: The central destination for the notification category — central ingests each forwarded notification into the `Notifications` table and acks the site, on which the engine clears the buffered message. +- **Health Monitoring**: Reports buffer depth metrics, including the notification backlog covering the site→central forward leg.