dohertj2/scadalink-design
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

docs(notification-outbox): retarget S&F notification category to central

Browse Source
This commit is contained in:
Joseph Doherty
2026-05-18 23:13:17 -04:00
parent 2a7e7289b3
commit b83d16364a
1 changed files with 13 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files

22
docs/requirements/Component-StoreAndForward.md
View File

@@ -13,7 +13,7 @@ Site clusters only. The central cluster does not buffer messages.
- Buffer outbound messages when the target system is unavailable.
- Manage three categories of buffered messages:
- External system API calls.
- Email notifications.
- Notifications forwarded to the central cluster.
- Cached database writes.
- Retry delivery per message according to the configured retry policy.
- Park messages that exhaust their retry limit (dead-letter).
@@ -42,13 +42,16 @@ Attempt immediate delivery
└── Max retries exhausted → Park message
```
For notifications, "delivery" means forwarding the message to the central cluster via the Communication Layer; "success" is central's ack, on which the message is cleared. Notifications do not park — they are retried at the fixed forward interval until central acks. Parking applies only to the external-system-call and cached-database-write categories.
## Retry Policy
Retry settings are defined on the **source entity** (not per-message):
For the external-system-call and cached-database-write categories, retry settings are defined on the **source entity** (not per-message):
- **External systems**: Each external system definition includes max retry count and time between retries.
- **Notifications**: Email/SMTP configuration includes max retry count and time between retries.
- **Cached database writes**: Each database connection definition includes max retry count and time between retries.
The **notification** category retries differently: it has no source-entity setting. The site→central forward uses a single fixed retry interval configured in the host `appsettings.json`. This interval concerns reaching the central cluster — it is infrastructure config, not a per-notification-list setting — so it applies uniformly to every buffered notification regardless of its target list. A buffered notification is retried until central acks it; it is not parked on a retry limit (central, once reachable, owns delivery, retry, and parking from that point on).
The retry interval is **fixed** (not exponential backoff). Fixed interval is sufficient for the expected use cases.
**Note**: Only **transient failures** are eligible for store-and-forward buffering. For external system calls, transient failures are connection errors, timeouts, and HTTP 5xx responses. Permanent failures (HTTP 4xx) are returned directly to the calling script and are **not** queued for retry. This prevents the buffer from accumulating requests that will never succeed.
@@ -79,8 +82,8 @@ There is **no maximum buffer size**. Messages accumulate in the buffer until del
Each buffered message stores:
- **Message ID**: Unique identifier.
- **Category**: External system call, notification, or cached database write.
- **Target**: External system name, notification list name, or database connection name.
- **Payload**: Serialized message content (API method + parameters, email subject + body, SQL + parameters).
- **Target**: External system name, the central cluster (for notifications), or database connection name.
- **Payload**: Serialized message content (API method + parameters; notification list name + subject + body plus the locally generated `NotificationId` and source provenance; SQL + parameters).
- **Retry Count**: Number of attempts so far.
- **Created At**: Timestamp when the message was first queued.
- **Last Attempt At**: Timestamp of the most recent delivery attempt.
@@ -89,14 +92,15 @@ Each buffered message stores:
## Dependencies
- **SQLite**: Local persistence on each node.
- **Communication Layer**: Application-level replication to standby node; remote query handling from central.
- **Communication Layer**: Application-level replication to standby node; remote query handling from central; carries buffered notifications to the central cluster (ClusterClient) and receives central's acks.
- **External System Gateway**: Delivers external system API calls.
- **Notification Service**: Delivers email notifications.
- **Central–Site Communication**: The delivery target for the notification category — a buffered notification is forwarded to the central cluster over the Communication Layer and cleared on central's ack.
- **Database Connections**: Delivers cached database writes.
- **Site Event Logging**: Logs store-and-forward activity (queued, delivered, retried, parked).
## Interactions
- **Site Runtime (Script Actors)**: Scripts submit messages to the buffer (external calls, notifications, cached DB writes).
- **Communication Layer**: Handles parked message queries/commands from central.
- **Health Monitoring**: Reports buffer depth metrics.
- **Communication Layer**: Handles parked message queries/commands from central; carries buffered notifications to the central cluster.
- **Notification Outbox**: The central destination for the notification category — central ingests each forwarded notification into the `Notifications` table and acks the site, on which the engine clears the buffered message.
- **Health Monitoring**: Reports buffer depth metrics, including the notification backlog covering the site→central forward leg.