Files
scadalink-design/docs/plans/2026-03-16-external-system-gateway-refinement-design.md
Joseph Doherty 5fff1712a8 Refine External System Gateway: protocol, auth, timeouts, error classification
Specify HTTP/REST with JSON as the invocation protocol. Add API key and Basic Auth
as outbound authentication modes. Add per-system call timeouts. Classify errors by
HTTP status for store-and-forward decisions (5xx/transient → retry, 4xx → permanent
error to script). Document ADO.NET connection pooling for database connections.
Update Store-and-Forward to clarify transient-only buffering.
2026-03-16 07:57:00 -04:00

3.0 KiB

External System Gateway Refinement — Design

Date: 2026-03-16 Component: External System Gateway (Component-ExternalSystemGateway.md) Status: Approved

Problem

The External System Gateway doc lacked specification for the invocation protocol, authentication methods, call timeouts, error classification for store-and-forward decisions, and database connection management.

Decisions

Invocation Protocol

  • HTTP/REST only with JSON serialization. The ESG is an HTTP client with predefined endpoints.
  • Method definitions include HTTP method (GET/POST/PUT/DELETE) and relative path.
  • Parameters serialized as JSON body (POST/PUT) or query parameters (GET/DELETE).

Outbound Authentication

  • Two modes per external system definition:
    • API Key: Configurable header name and key value.
    • Basic Auth: Username and password, sent as standard HTTP Authorization header.

Call Timeouts

  • Per-system timeout — one timeout value applies to all methods on a given external system.
  • Defined in the external system definition.

Error Classification

  • Transient failures (connection errors, timeouts, HTTP 5xx): Routed to Store-and-Forward for retry. Script does not block.
  • Permanent failures (HTTP 4xx): No retry. Error returned synchronously to the calling script. Logged to Site Event Logging.
  • S&F buffer only accepts transient failures to avoid accumulating unrecoverable requests.

Permanent Failure Behavior

  • Synchronous error back to script, consistent with DCL write failure handling.

Database Connection Pooling

  • Standard ADO.NET connection pooling per named connection. No custom pool logic.
  • Pool tuning via connection string parameters if needed.

Serialization

  • JSON only, consistent with REST-only decision.

Affected Documents

Document Change
Component-ExternalSystemGateway.md Updated External System Definition fields (base URL, auth modes, timeout, HTTP method/path per method). Added 3 new sections: Invocation Protocol, Call Timeout & Error Handling, Database Connection Management.
Component-StoreAndForward.md Clarified that only transient failures are buffered; 4xx errors are not queued.

Alternatives Considered

  • SOAP support: Rejected — REST covers modern integrations; SOAP systems can be fronted by a thin REST wrapper.
  • OAuth2 client credentials: Rejected — adds token lifecycle complexity at every site for marginal benefit; can be handled by a gateway/proxy.
  • Per-method timeouts: Rejected — external systems tend to have consistent latency; per-system is the right granularity.
  • All failures retryable: Rejected — retrying 4xx errors pollutes the S&F buffer with requests that will never succeed.
  • Custom connection pooling: Rejected — ADO.NET pooling is battle-tested and handles this scenario natively.
  • XML serialization option: Rejected — JSON-only is consistent with REST-only; XML systems can use a wrapper.