# Phase 7: Integration Surfaces

**Date**: 2026-03-16
**Status**: Plan complete
**Goal**: External systems can call in and site scripts can call out.

---

## Scope

**Components**:
- Inbound API (full runtime)
- External System Gateway (full site-side execution)
- Notification Service (full site-side delivery)

**Features**:
- Inbound API: ASP.NET endpoint, X-API-Key auth, method routing, parameter validation, script execution on central, Route.To() cross-site calls, batch attribute operations, error handling, failures-only logging
- External System Gateway: HTTP/REST client, API key + Basic Auth, per-system timeout, dual call modes (Call/CachedCall), error classification, database access (synchronous + cached write), dedicated blocking I/O dispatcher
- Notification Service: SMTP with OAuth2 Client Credentials (M365) + Basic Auth, token lifecycle, BCC delivery plain text, timeout + max connections, error classification, S&F integration

---

## Prerequisites

| Phase | What must be complete |
|-------|-----------------------|
| Phase 1 | Security & Auth (API key storage in config DB), Configuration Database |
| Phase 2 | Template Engine (instance/template model for Route.To resolution) |
| Phase 3A | Cluster Infrastructure, Site Runtime (Instance Actor, Script Actor, Script Execution Actor) |
| Phase 3B | Communication Layer (integration routing pattern), Site Runtime (script runtime API framework, Script Execution Actor lifecycle, dedicated dispatcher) |
| Phase 3C | Store-and-Forward Engine (buffering, retry, parking, replication) |
| Phase 4 | API key management UI (Admin role) |
| Phase 5 | External system definition UI, DB connection definition UI, notification list UI, inbound API method definition UI |

---

## Requirements Checklist

### Section 5.1 — External System Definitions (runtime portion)
- [ ] `[5.1-1-rt]` Definitions are deployed uniformly to all sites — site-side can load deployed definitions
- [ ] `[5.1-2-rt]` Connection details (URL, auth, protocol) used at runtime for HTTP calls

### Section 5.2 — Site-to-External-System Communication
- [ ] `[5.2-1]` Sites communicate with external systems directly (not through central)
- [ ] `[5.2-2]` Scripts invoke external system methods by referencing predefined definitions

### Section 5.3 — Store-and-Forward for External Calls (integration portion)
- [ ] `[5.3-1-int]` If external system unavailable, message buffered locally at site
- [ ] `[5.3-2-int]` Retry per message — individual failed messages retry independently
- [ ] `[5.3-3-int]` Configurable retry settings: max retry count, fixed time between retries
- [ ] `[5.3-4-int]` After max retries exhausted, message is parked
- [ ] `[5.3-5-int]` No maximum buffer size

### Section 5.5 — Database Connections (runtime portion)
- [ ] `[5.5-1-rt]` Definitions deployed to sites and loadable at runtime
- [ ] `[5.5-2-rt]` Retry settings applied to cached writes

### Section 5.6 — Database Access Modes
- [ ] `[5.6-1]` Real-time (synchronous): `Database.Connection("name")` returns raw MS SQL client connection (ADO.NET)
- [ ] `[5.6-2]` Full ADO.NET control: queries, updates, transactions, stored procedures
- [ ] `[5.6-3]` Cached write (store-and-forward): `Database.CachedWrite("name", "sql", parameters)`
- [ ] `[5.6-4]` Cached entry stores: database connection name, SQL statement, parameter values
- [ ] `[5.6-5]` If database unavailable, write buffered locally and retried per retry settings
- [ ] `[5.6-6]` After max retries exhausted, cached write is parked

### Section 6.1 — Notification Lists (runtime portion)
- [ ] `[6.1-1-rt]` Notification lists deployed to sites and loadable at runtime
- [ ] `[6.1-2-rt]` List name resolves to recipients at site

### Section 6.2 — Email Support
- [ ] `[6.2-1]` Predefined support for sending email as notification delivery mechanism
- [ ] `[6.2-2]` SMTP settings defined centrally and deployed to all sites

### Section 6.3 — Script API
- [ ] `[6.3-1]` `Notify.To("list name").Send("subject", "message")` — simplified script API
- [ ] `[6.3-2]` Available to instance scripts, alarm on-trigger scripts, and shared scripts

### Section 6.4 — Store-and-Forward for Notifications
- [ ] `[6.4-1]` If email server unavailable, notifications buffered locally
- [ ] `[6.4-2]` Same retry pattern: configurable max retry count, fixed time between retries
- [ ] `[6.4-3]` After max retries exhausted, notification parked
- [ ] `[6.4-4]` No maximum buffer size

### Section 7.1 — Inbound API Purpose
- [ ] `[7.1-1]` Web API on central cluster for external systems to call in
- [ ] `[7.1-2]` Counterpart to outbound External System Gateway

### Section 7.2 — API Key Management (already in Phase 4; Phase 7 uses the keys)
- [ ] `[7.2-rt]` API keys stored in config DB and retrievable at runtime for validation

### Section 7.3 — Authentication
- [ ] `[7.3-1]` Inbound requests authenticated via API key (not LDAP/AD)
- [ ] `[7.3-2]` API key included with each request
- [ ] `[7.3-3]` Invalid or disabled keys rejected

### Section 7.4 — API Method Definitions (runtime portion)
- [ ] `[7.4-1-rt]` Method definitions loadable at runtime for routing and validation
- [ ] `[7.4-2-rt]` Approved API keys checked at runtime per method
- [ ] `[7.4-3-rt]` Parameter validation enforced at runtime (type checking against definitions)
- [ ] `[7.4-4-rt]` Return value serialized per return definition
- [ ] `[7.4-5-rt]` Timeout enforced per method (max execution time including routed calls)
- [ ] `[7.4-6-rt]` Implementation script executed on central cluster
- [ ] `[7.4-7-rt]` Route.To() routes calls to any instance at any site
- [ ] `[7.4-8-rt]` API scripts cannot call shared scripts directly (shared scripts are site-only)

### Section 7.5 — Availability
- [ ] `[7.5-1]` Inbound API hosted only on central cluster (active node)
- [ ] `[7.5-2]` On central failover, API becomes available on new active node

### Section 4.4 — Script Capabilities (Phase 7 portion)
- [ ] `[4.4-6]` `ExternalSystem.Call()` — synchronous request/response
- [ ] `[4.4-7]` `ExternalSystem.CachedCall()` — fire-and-forget with S&F on transient failure
- [ ] `[4.4-8]` Send notifications via `Notify.To().Send()`
- [ ] `[4.4-9]` `Database.Connection()` for raw ADO.NET access; `Database.CachedWrite()` for S&F delivery

---

## Design Constraints Checklist

| ID | Constraint | Source | Mapped WP |
|----|-----------|--------|-----------|
| KDD-ext-1 | External System Gateway: HTTP/REST only, JSON serialization, API key + Basic Auth | CLAUDE.md | WP-6 |
| KDD-ext-2 | Dual call modes: Call() synchronous and CachedCall() store-and-forward | CLAUDE.md | WP-7 |
| KDD-ext-3 | Error classification: HTTP 5xx/408/429/connection = transient; other 4xx = permanent | CLAUDE.md | WP-8 |
| KDD-ext-4 | Notification Service: SMTP with OAuth2 Client Credentials (M365) or Basic Auth. BCC delivery, plain text | CLAUDE.md | WP-10, WP-11 |
| KDD-ext-5 | Inbound API: POST /api/{methodName}, X-API-Key header, flat JSON, extended type system (Object/List) | CLAUDE.md | WP-1, WP-2, WP-3 |
| KDD-sf-4 | CachedCall idempotency is the caller's responsibility | CLAUDE.md | WP-7 |
| CD-ESG-1 | ESG acts as HTTP client; base URL + method relative path | Component-ESG | WP-6 |
| CD-ESG-2 | Request params serialized as JSON body (POST/PUT) or query params (GET/DELETE) | Component-ESG | WP-6 |
| CD-ESG-3 | Credentials (API key header or Basic Auth) attached to every request | Component-ESG | WP-6 |
| CD-ESG-4 | Per-system timeout applies to all method calls | Component-ESG | WP-8 |
| CD-ESG-5 | CachedCall: on transient failure, routed to S&F. Script does not block | Component-ESG | WP-7 |
| CD-ESG-6 | CachedCall: on permanent failure (4xx), error returned synchronously. No retry | Component-ESG | WP-7 |
| CD-ESG-7 | Dedicated dispatcher for Script Execution Actors to isolate blocking I/O | Component-ESG | WP-9 |
| CD-ESG-8 | Database connections use standard ADO.NET connection pooling per named connection | Component-ESG | WP-14 |
| CD-ESG-9 | Synchronous DB failures return error to calling script | Component-ESG | WP-14 |
| CD-NS-1 | Single email per Send() call, all recipients in BCC, from address in To | Component-NotificationService | WP-11 |
| CD-NS-2 | No per-recipient deduplication | Component-NotificationService | WP-11 |
| CD-NS-3 | Transient SMTP failures (connection refused, timeout, SMTP 4xx) → S&F. Script does not block | Component-NotificationService | WP-12 |
| CD-NS-4 | Permanent SMTP failures (5xx) → error returned synchronously. No retry | Component-NotificationService | WP-12 |
| CD-NS-5 | No application-level rate limiting; SMTP server throttling handled as transient failure | Component-NotificationService | WP-11 |
| CD-NS-6 | OAuth2 token lifecycle: fetch, cache, refresh on expiry | Component-NotificationService | WP-10 |
| CD-NS-7 | Connection timeout (default 30s) and max concurrent connections (default 5) | Component-NotificationService | WP-10 |
| CD-IA-1 | All calls are POST — RPC-style, not RESTful | Component-InboundAPI | WP-1 |
| CD-IA-2 | Parameters as top-level JSON fields in request body | Component-InboundAPI | WP-3 |
| CD-IA-3 | Success: 200 with return value JSON. Failure: 4xx/5xx with error object | Component-InboundAPI | WP-5 |
| CD-IA-4 | Only failures (500) logged centrally. Successful calls not logged | Component-InboundAPI | WP-5 |
| CD-IA-5 | No rate limiting | Component-InboundAPI | WP-1 |
| CD-IA-6 | Route.To() resolves instance site from config DB, routes via Communication Layer | Component-InboundAPI | WP-4 |
| CD-IA-7 | Route.To() calls are synchronous from API caller perspective — blocks until site responds or timeout | Component-InboundAPI | WP-4 |
| CD-IA-8 | No S&F buffering for inbound API calls — site unreachable = error to caller | Component-InboundAPI | WP-4 |
| CD-IA-9 | Route.To().GetAttributes/SetAttributes batch operations | Component-InboundAPI | WP-4 |
| CD-IA-10 | Database.Connection() available to Inbound API scripts for central DB access | Component-InboundAPI | WP-4 |

---

## Work Packages

### WP-1: Inbound API — ASP.NET Endpoint Registration (M)

**Description**: Register the `POST /api/{methodName}` ASP.NET Core endpoint on central. Wire into Host via `MapInboundAPI()`.

**Acceptance Criteria**:
- `POST /api/{methodName}` endpoint registered and reachable (`[7.1-1]`, `KDD-ext-5`, `CD-IA-1`)
- API hosted only on central cluster active node (`[7.5-1]`)
- On central failover, API becomes available on new active node (`[7.5-2]`)
- No rate limiting (`CD-IA-5`)
- Content-Type: application/json

**Complexity**: M
**Traces**: `[7.1-1]`, `[7.1-2]`, `[7.5-1]`, `[7.5-2]`, KDD-ext-5, CD-IA-1, CD-IA-5

---

### WP-2: Inbound API — X-API-Key Authentication (S)

**Description**: Extract and validate API key from the `X-API-Key` header. Check key exists, is enabled, and is approved for the method.

**Acceptance Criteria**:
- API key extracted from `X-API-Key` HTTP header (`[7.3-2]`, `KDD-ext-5`)
- Key validated against config DB: exists and is enabled (`[7.3-1]`, `[7.3-3]`, `[7.2-rt]`)
- Key checked against method's approved list (`[7.4-2-rt]`)
- Invalid key → 401 Unauthorized
- Valid key but not approved for method → 403 Forbidden
- Disabled key → 401 Unauthorized

**Complexity**: S
**Traces**: `[7.2-rt]`, `[7.3-1]`–`[7.3-3]`, `[7.4-2-rt]`, KDD-ext-5

---

### WP-3: Inbound API — Method Routing & Parameter Validation (M)

**Description**: Resolve method by name, validate and deserialize parameters against definitions (extended type system).

**Acceptance Criteria**:
- Method resolved by name from URL path segment (`[7.4-1-rt]`)
- Parameters validated against method definition: count, names, data types (`[7.4-3-rt]`, `CD-IA-2`)
- Extended type system (Object, List) supported for parameters (`KDD-ext-5`)
- Invalid parameters → 400 Bad Request
- Unknown method → 404 Not Found

**Complexity**: M
**Traces**: `[7.4-1-rt]`, `[7.4-3-rt]`, KDD-ext-5, CD-IA-2

---

### WP-4: Inbound API — Script Execution & Route.To() (L)

**Description**: Execute the method's C# implementation script on central. Implement Route.To() for cross-site calls and batch attribute operations.

**Acceptance Criteria**:
- Implementation script executed on central cluster (`[7.4-6-rt]`)
- Per-method timeout enforced (including routed calls to sites) (`[7.4-5-rt]`)
- `Route.To("instanceCode").Call("scriptName", params)` routes to site via Communication Layer (`[7.4-7-rt]`, `CD-IA-6`)
- Route.To() resolves instance site from config DB (`CD-IA-6`)
- Calls are synchronous from caller perspective — blocks until response or timeout (`CD-IA-7`)
- No S&F buffering — site unreachable returns error to caller (`CD-IA-8`)
- `Route.To().GetAttribute()` / `Route.To().GetAttributes()` batch read (`CD-IA-9`)
- `Route.To().SetAttribute()` / `Route.To().SetAttributes()` batch write (`CD-IA-9`)
- `Database.Connection("name")` available for central DB access (`CD-IA-10`)
- API scripts cannot call shared scripts directly (`[7.4-8-rt]`)
- Return value serialized per return definition (`[7.4-4-rt]`)

**Complexity**: L
**Traces**: `[7.4-4-rt]`–`[7.4-8-rt]`, CD-IA-6–CD-IA-10

---

### WP-5: Inbound API — Error Handling & Logging (S)

**Description**: Implement error response format and failures-only logging.

**Acceptance Criteria**:
- Success: 200 with return value JSON (`CD-IA-3`)
- Failure responses: error object with "error" and "code" fields (`CD-IA-3`)
- 401 for invalid/disabled key, 403 for unapproved key, 400 for invalid params, 500 for script failure
- Only 500 errors logged centrally (`CD-IA-4`)
- Successful calls not logged (audit log reserved for config changes) (`CD-IA-4`)
- Script execution errors: safe error message, no internal details exposed

**Complexity**: S
**Traces**: CD-IA-3, CD-IA-4

---

### WP-6: External System Gateway — HTTP/REST Client (M)

**Description**: Implement the HTTP client that executes external system API calls with JSON serialization.

**Acceptance Criteria**:
- HTTP/REST only, JSON serialization (`KDD-ext-1`, `CD-ESG-1`)
- Base URL from definition + method relative path (`CD-ESG-1`)
- Request params serialized as JSON body for POST/PUT, query params for GET/DELETE (`CD-ESG-2`)
- Response body deserialized from JSON into method return type
- Authentication: API key header or Basic Auth per system definition (`KDD-ext-1`, `CD-ESG-3`)
- Sites communicate directly with external systems (not through central) (`[5.2-1]`)
- Scripts invoke methods by referencing predefined definitions (`[5.2-2]`)
- Deployed definitions loaded at site runtime (`[5.1-1-rt]`, `[5.1-2-rt]`)

**Complexity**: M
**Traces**: `[5.1-1-rt]`, `[5.1-2-rt]`, `[5.2-1]`, `[5.2-2]`, KDD-ext-1, CD-ESG-1, CD-ESG-2, CD-ESG-3

---

### WP-7: External System Gateway — Dual Call Modes (M)

**Description**: Implement `ExternalSystem.Call()` (synchronous) and `ExternalSystem.CachedCall()` (S&F) in the script runtime API.

**Acceptance Criteria**:
- `ExternalSystem.Call("system", "method", params)` — synchronous. Script blocks until response or timeout. All failures return to script (`[4.4-6]`, `KDD-ext-2`)
- `ExternalSystem.CachedCall("system", "method", params)` — fire-and-forget. On transient failure, routed to S&F Engine. Script does not block (`[4.4-7]`, `KDD-ext-2`, `CD-ESG-5`)
- CachedCall on permanent failure (4xx except 408/429): error returned synchronously to script. No retry (`CD-ESG-6`)
- CachedCall on success: response discarded (fire-and-forget)
- CachedCall idempotency is caller's responsibility — documented (`KDD-sf-4`)
- S&F integration: buffered, retried per system's retry settings, parked after max retries (`[5.3-1-int]`–`[5.3-5-int]`)

**Complexity**: M
**Traces**: `[4.4-6]`, `[4.4-7]`, `[5.3-1-int]`–`[5.3-5-int]`, KDD-ext-2, KDD-sf-4, CD-ESG-5, CD-ESG-6

---

### WP-8: External System Gateway — Error Classification & Timeout (S)

**Description**: Implement transient/permanent error classification and per-system timeout enforcement.

**Acceptance Criteria**:
- Per-system timeout applied to all HTTP calls (`CD-ESG-4`)
- Transient failures: connection refused, timeout, HTTP 408, 429, 5xx (`KDD-ext-3`)
- Permanent failures: HTTP 4xx except 408/429 (`KDD-ext-3`)
- Transient → CachedCall buffers for retry; Call returns error to script
- Permanent → always returned to calling script regardless of call mode
- Permanent failures logged to Site Event Logging

**Complexity**: S
**Traces**: KDD-ext-3, CD-ESG-4

---

### WP-9: Blocking I/O Dispatcher Isolation (S)

**Description**: Configure a dedicated Akka.NET dispatcher for Script Execution Actors to isolate blocking I/O (HTTP calls, DB operations) from the default dispatcher.

**Acceptance Criteria**:
- Dedicated dispatcher configured for Script Execution Actors (`CD-ESG-7`)
- HTTP calls and database operations execute on dedicated dispatcher threads
- Default dispatcher (used by coordination actors) is not starved by blocking I/O

**Complexity**: S
**Traces**: CD-ESG-7

---

### WP-10: Notification Service — SMTP Client (M)

**Description**: Implement the SMTP client with support for OAuth2 Client Credentials (M365) and Basic Auth.

**Acceptance Criteria**:
- SMTP client supports OAuth2 Client Credentials flow: Tenant ID, Client ID, Client Secret (`KDD-ext-4`)
- OAuth2 token lifecycle: fetch on first use, cache, refresh on expiry (`CD-NS-6`)
- SMTP client supports Basic Auth (username/password) (`KDD-ext-4`)
- TLS modes: None, StartTLS, SSL
- Connection timeout configurable (default 30s) (`CD-NS-7`)
- Max concurrent connections configurable (default 5) (`CD-NS-7`)
- SMTP settings loaded from deployed configuration (`[6.2-1]`, `[6.2-2]`)
- Deployed notification lists loadable at runtime (`[6.1-1-rt]`, `[6.1-2-rt]`)

**Complexity**: M
**Traces**: `[6.1-1-rt]`, `[6.1-2-rt]`, `[6.2-1]`, `[6.2-2]`, KDD-ext-4, CD-NS-6, CD-NS-7

---

### WP-11: Notification Service — Email Delivery Behavior (S)

**Description**: Implement the email composition and delivery behavior: BCC, plain text, from address.

**Acceptance Criteria**:
- `Notify.To("listName").Send("subject", "message")` script API implemented (`[6.3-1]`)
- Available to instance scripts, alarm on-trigger scripts, and shared scripts (`[6.3-2]`, `[4.4-8]`)
- Single email per Send() call with all recipients in BCC (`CD-NS-1`, `KDD-ext-4`)
- From address placed in To field (`CD-NS-1`)
- Plain text only, no HTML (`KDD-ext-4`)
- No per-recipient deduplication (`CD-NS-2`)
- No application-level rate limiting (`CD-NS-5`)

**Complexity**: S
**Traces**: `[4.4-8]`, `[6.3-1]`, `[6.3-2]`, KDD-ext-4, CD-NS-1, CD-NS-2, CD-NS-5

---

### WP-12: Notification Service — Error Classification & S&F Integration (S)

**Description**: Implement transient/permanent SMTP error classification and integration with the Store-and-Forward Engine.

**Acceptance Criteria**:
- Transient failures (connection refused, timeout, SMTP 4xx): notification handed to S&F Engine. Script does not block (`CD-NS-3`, `[6.4-1]`)
- Permanent failures (SMTP 5xx): error returned synchronously to script. No retry (`CD-NS-4`)
- S&F retry per SMTP config retry settings: max retry count, fixed interval (`[6.4-2]`)
- After max retries exhausted, notification parked (`[6.4-3]`)
- No maximum buffer size (`[6.4-4]`)

**Complexity**: S
**Traces**: `[6.4-1]`–`[6.4-4]`, CD-NS-3, CD-NS-4

---

### WP-13: End-to-End Integration Tests (L)

**Description**: End-to-end tests covering the full integration paths.

**Acceptance Criteria**:
- **Test 1**: External system calls Inbound API → Route.To() → site script executes → response flows back (`[7.1-1]`, `[7.4-7-rt]`)
- **Test 2**: Script calls `ExternalSystem.Call()` → HTTP request → response returned to script (`[4.4-6]`)
- **Test 3**: Script calls `ExternalSystem.CachedCall()` → transient failure → S&F buffer → retry → success (`[4.4-7]`, `[5.3-1-int]`)
- **Test 4**: Script calls `Notify.To().Send()` → SMTP delivery succeeds (`[4.4-8]`, `[6.3-1]`)
- **Test 5**: Script calls `Notify.To().Send()` → SMTP unavailable → S&F buffer → retry → success (`[6.4-1]`)
- **Test 6**: Script calls `Database.Connection()` → ADO.NET query succeeds (`[4.4-9]`, `[5.6-1]`, `[5.6-2]`)
- **Test 7**: Script calls `Database.CachedWrite()` → DB unavailable → S&F buffer → retry → success (`[5.6-3]`–`[5.6-6]`)
- **Test 8**: CachedCall permanent failure (4xx) → error returned to script, not buffered
- **Test 9**: Inbound API → site unreachable → error returned to external caller (`CD-IA-8`)
- **Test 10**: Inbound API → disabled API key → 401 (`[7.3-3]`)
- **Test 11**: Batch attribute operations via Route.To().GetAttributes/SetAttributes (`CD-IA-9`)

**Complexity**: L
**Traces**: All requirements from this phase

---

### WP-14: Database Access — Connection() and CachedWrite() (M)

**Description**: Implement `Database.Connection()` for synchronous ADO.NET access and `Database.CachedWrite()` for S&F database writes in the script runtime API.

**Acceptance Criteria**:
- `Database.Connection("name")` returns raw ADO.NET SqlConnection (`[4.4-9]`, `[5.6-1]`)
- Full ADO.NET control: queries, updates, transactions, stored procedures (`[5.6-2]`)
- Standard ADO.NET connection pooling per named connection (`CD-ESG-8`)
- Synchronous DB failures return error to calling script (`CD-ESG-9`)
- `Database.CachedWrite("name", "sql", params)` submits to S&F Engine (`[5.6-3]`)
- Cached entry stores: connection name, SQL statement, parameter values (`[5.6-4]`)
- If DB unavailable, write buffered and retried per connection retry settings (`[5.6-5]`, `[5.5-2-rt]`)
- After max retries, cached write parked (`[5.6-6]`)
- Deployed database connection definitions loaded at site runtime (`[5.5-1-rt]`)

**Complexity**: M
**Traces**: `[4.4-9]`, `[5.5-1-rt]`, `[5.5-2-rt]`, `[5.6-1]`–`[5.6-6]`, CD-ESG-8, CD-ESG-9

---

## Test Strategy

### Unit Tests
- API key validation logic (valid, invalid, disabled, not approved for method)
- Method routing and parameter validation (type checking, missing params, extra params)
- Extended type system serialization/deserialization (Object, List)
- Error classification: transient vs. permanent for HTTP status codes
- Error classification: transient vs. permanent for SMTP status codes
- OAuth2 token fetch, cache, refresh-on-expiry logic
- BCC email composition
- Database.Connection() pooling behavior
- Database.CachedWrite() S&F submission

### Integration Tests
- Inbound API full flow: auth → route → method resolve → script execute → return
- External System Gateway: HTTP client against test REST server (infra/restapi)
- Notification Service: SMTP client against Mailpit test server (infra)
- Store-and-Forward integration: CachedCall → buffer → retry → deliver
- Database access: Connection() query against test MS SQL, CachedWrite() with simulated failure
- Route.To() cross-site call via Communication Layer

### Negative Tests
- `ExternalSystem.Call()` with unreachable system → error returned to script (no buffer)
- `ExternalSystem.CachedCall()` with permanent 4xx → error returned synchronously, not buffered
- Inbound API with invalid key → 401
- Inbound API with unapproved key → 403
- Inbound API with bad params → 400
- Inbound API script failure → 500 with safe error message, no internal details
- Inbound API script calls shared script → fails (shared scripts site-only)
- Inbound API route to unreachable site → error, no S&F
- `Notify.To().Send()` with SMTP 5xx → error returned to script, not buffered
- `Database.Connection()` with unreachable server → error returned to script

### Failover Tests
- Central failover → Inbound API becomes available on new active node
- Site failover → S&F buffer takeover for pending external calls, notifications, DB writes

---

## Verification Gate

Phase 7 is complete when:
1. All 14 work packages pass acceptance criteria
2. All unit, integration, negative, and failover tests pass
3. External systems can call `POST /api/{method}` with API key auth and receive scripted responses
4. Site scripts can call external systems via both `Call()` and `CachedCall()` modes
5. Notifications send via SMTP with both OAuth2 and Basic Auth
6. Database access works synchronously and via CachedWrite with S&F
7. Error classification correctly routes transient failures to S&F and permanent failures to the script
8. End-to-end integration paths verified (all 11 test scenarios in WP-13)

---

## Open Questions

| # | Question | Context | Impact | Status |
|---|----------|---------|--------|--------|
| Q12 | What Microsoft 365 tenant/app registration is available for SMTP OAuth2 testing? | Affects Notification Service OAuth2 implementation. | Phase 7. | Deferred — implement against Basic Auth first; OAuth2 tested when tenant available. |

(Existing question from questions.md — no new questions discovered.)

---

## Split-Section Verification

| Section | Phase 7 Bullets | Other Phase(s) | Other Phase Bullets |
|---------|----------------|-----------------|---------------------|
| 4.4 | `[4.4-6]`–`[4.4-9]` (external calls, notifications, DB access) | Phase 3B | `[4.4-1]`–`[4.4-5]`, `[4.4-10]` (read/write/call scripts) |
| 5.1 | `[5.1-1-rt]`, `[5.1-2-rt]` (runtime) | Phase 5 | `[5.1-1]`–`[5.1-5]` (definition UI) |
| 5.3 | `[5.3-1-int]`–`[5.3-5-int]` (integration) | Phase 3C | S&F Engine infrastructure |
| 5.5 | `[5.5-1-rt]`, `[5.5-2-rt]` (runtime) | Phase 5 | `[5.5-1]`–`[5.5-5]` (definition UI) |
| 6.1 | `[6.1-1-rt]`, `[6.1-2-rt]` (runtime) | Phase 5 | `[6.1-1]`–`[6.1-5]` (definition UI) |
| 7.4 | `[7.4-1-rt]`–`[7.4-8-rt]` (runtime) | Phase 5 | `[7.4-1]`–`[7.4-8]` (definition UI) |

---

## Orphan Check Result

**Forward check**: Every Requirements Checklist item and Design Constraints Checklist item maps to at least one work package with acceptance criteria that would fail if the requirement were not implemented. PASS.

**Reverse check**: Every work package traces back to at least one requirement or design constraint. No untraceable work. PASS.

**Split-section check**: All split sections verified above. Phase 7 covers runtime execution for all integration components. Definition management UI is in Phase 5. S&F infrastructure is in Phase 3C. Core script capabilities (read/write/call) are in Phase 3B. No unassigned bullets found. PASS.

**Negative requirement check**: The following negative requirements have explicit acceptance criteria:
- `[7.4-8-rt]` API scripts cannot call shared scripts directly → tested in WP-4 and negative tests
- `[4.4-10]` (Phase 3B) Scripts cannot access other instances → not this phase's concern
- `CD-IA-8` No S&F for inbound API calls → tested in WP-4 and WP-13 test 9
- `CD-NS-2` No per-recipient deduplication → verified in WP-11
- `CD-NS-5` No application-level rate limiting → verified in WP-11
- `CD-IA-5` No rate limiting on inbound API → verified in WP-1
- `CD-IA-4` Successful calls not logged → verified in WP-5

PASS.

**Codex MCP verification**: Skipped — external tool verification deferred.