Joseph Doherty
308 lines
24 KiB
Markdown
308 lines
24 KiB
Markdown
# Component: Inbound API
|
|
|
|
## Purpose
|
|
|
|
The Inbound API exposes a web API on the central cluster that external systems can call into. This is the reverse of the External System Gateway — where that component handles the SCADA system calling out to external systems, this component handles external systems calling in. It provides API key authentication, method-level authorization, and script-based method implementations.
|
|
|
|
## Location
|
|
|
|
Central cluster only (active node). Not available at site clusters.
|
|
|
|
## Responsibilities
|
|
|
|
- Host a web API endpoint on the central cluster.
|
|
- Authenticate inbound requests via API keys.
|
|
- Route requests to the appropriate API method definition.
|
|
- Enforce per-method API key authorization (only approved keys can call a given method).
|
|
- Execute the C# script implementation for the called method.
|
|
- Return structured responses to the caller.
|
|
- Failover: API becomes available on the new active node after central failover.
|
|
|
|
## API Key Management
|
|
|
|
### Storage
|
|
- API keys are stored in the **configuration database (MS SQL)**.
|
|
|
|
### Key Properties
|
|
- **Name/Label**: Human-readable identifier for the key (e.g., "MES-Production", "RecipeManager-Dev").
|
|
- **Key Value**: The secret key string used for authentication.
|
|
- **Enabled/Disabled Flag**: Keys can be disabled without deletion.
|
|
|
|
### Management
|
|
- Managed by users with the **Administrator** role via the Central UI.
|
|
- All key changes (create, enable/disable, delete) are audit logged.
|
|
|
|
## API Method Definition
|
|
|
|
### Properties
|
|
Each API method definition includes:
|
|
- **Method Name**: Unique identifier and URL path segment for the endpoint.
|
|
- **Approved API Keys**: List of API keys authorized to invoke this method. Requests from non-approved keys are rejected.
|
|
- **Parameter Definitions**: Ordered list of input parameters, each with:
|
|
- Parameter name.
|
|
- Data type — the **extended type system** (Boolean, Integer, Float, String, plus the nestable Object and List; see [Extended Type System](#extended-type-system)).
|
|
- Whether the parameter is required.
|
|
- **Return Value Definition**: Structure of the response, with:
|
|
- Field names and (extended-system) data types. Supports returning **lists of objects** and arbitrarily nested structures.
|
|
- **Implementation Script**: C# script that executes when the method is called. Stored **inline** in the method definition. Follows standard C# authoring patterns but has no template inheritance — it is a standalone script tied to this method.
|
|
- **Timeout**: Configurable per method. Defines the maximum time the method is allowed to execute (including any routed calls to sites) before returning a timeout error to the caller.
|
|
|
|
> **Authoring note — idempotent methods for at-least-once callers**: some external callers retry
|
|
> on their own connect-failure/timeout failover loop and cannot tell "the first attempt actually
|
|
> succeeded" from "the first attempt was lost" — e.g. the `DelmiaRecipeDownload` method backing
|
|
> `DelmiaNotifier` (see `docs/plans/2026-06-26-delmia-recipe-notifier-design.md`, "Delivery
|
|
> semantics: at-least-once"). Implementation scripts for such methods must be idempotent on their
|
|
> natural key so a duplicate delivery returns the same result without double-applying side effects.
|
|
|
|
### Management
|
|
- Managed by users with the **Designer** role via the Central UI.
|
|
- All method definition changes are audit logged.
|
|
|
|
## HTTP Contract
|
|
|
|
### URL Structure
|
|
- All API calls use `POST /api/{methodName}`.
|
|
- Method names map directly to URL path segments (e.g., method "GetProductionReport" → `POST /api/GetProductionReport`).
|
|
- All calls are POST — these are RPC-style script invocations, not RESTful resource operations.
|
|
|
|
### Authentication Header
|
|
- The API key token (`sbk_<keyId>_<secret>`) is accepted from **either** of two headers:
|
|
- `Authorization: Bearer sbk_<keyId>_<secret>` — the preferred form.
|
|
- `X-API-Key: sbk_<keyId>_<secret>` — alternate form for callers that cannot set the Authorization header.
|
|
- When both headers are present, `Authorization` takes precedence.
|
|
- Both forms are processed by the same `IApiKeyVerifier` from the shared `ZB.MOM.WW.Auth.ApiKeys` library; the verifier strips the optional `Bearer ` prefix and performs the same peppered-HMAC constant-time verification regardless of which header was used.
|
|
|
|
### Key rotation
|
|
|
|
The `sbk_<keyId>_<secret>` design already supports zero-downtime rotation — a caller can hold two valid keys at once, so there is no moment where the old key must be cut off before the new one works. The procedure:
|
|
|
|
1. **Create a new key** with the same scope set as the key being rotated — `security api-key create --name <name> --methods <methods>` (CLI `security api-key` group). Scopes are **copied at creation only**: the operator must pass the same `--methods` list the old key was approved for; the CLI does not read the old key's scopes for you, so scope drift between the pair is an operator error, not something the system detects.
|
|
2. **Deliver the new token to the caller and confirm cutover.** The response from `create` surfaces the one-time bearer token (`PrintCreatedKey` — it is shown once and cannot be retrieved again, so capture it immediately). Both the old and new keys verify successfully during this window — that overlap is intended, not a bug, and gives the caller time to redeploy with the new token without a synchronized flag-day cutover. Confirm the caller has actually switched by checking the audit log's Actor field (recorded per call), which shows which key ID is serving live traffic.
|
|
3. **Disable — don't delete — the old key** once cutover is confirmed: `security api-key update --key-id <old-key-id> --enabled false`. Disabling is reversible (`--enabled true` re-enables it) if the cutover needs to be rolled back, whereas deletion is not.
|
|
4. **Delete the old key after a quarantine period** (an operator-judgment window to be sure nothing is still presenting the old token): `security api-key delete --key-id <old-key-id>`.
|
|
|
|
### Request Format
|
|
- Content-Type: `application/json`.
|
|
- Parameters are top-level JSON fields in the request body matching the method's parameter definitions:
|
|
```json
|
|
{
|
|
"siteId": "SiteA",
|
|
"startDate": "2026-03-01",
|
|
"endDate": "2026-03-16"
|
|
}
|
|
```
|
|
|
|
### Response Format
|
|
- **Success (200)**: The response body is the method's return value as JSON, with fields matching the return value definition:
|
|
```json
|
|
{
|
|
"siteName": "Site Alpha",
|
|
"totalUnits": 14250,
|
|
"lines": [
|
|
{ "lineName": "Line-1", "units": 8200, "efficiency": 92.5 },
|
|
{ "lineName": "Line-2", "units": 6050, "efficiency": 88.1 }
|
|
]
|
|
}
|
|
```
|
|
- **Failure (4xx/5xx)**: The response body is an error object carrying a human-readable `error` and a stable, machine-readable `code`:
|
|
```json
|
|
{
|
|
"error": "Site unreachable",
|
|
"code": "SITE_UNREACHABLE"
|
|
}
|
|
```
|
|
- HTTP status codes distinguish success from failure — no envelope wrapper.
|
|
- **Error codes.** Every failure body carries a `code` an external integrator can branch on (the `error` message is for humans and may change). The two authorization negatives — unknown method and key-not-in-scope — deliberately share one code (`NOT_APPROVED`) and one body so a caller cannot enumerate which method names exist:
|
|
|
|
| `code` | HTTP status | Meaning |
|
|
|--------|-------------|---------|
|
|
| `UNAUTHORIZED` | 401 | Missing/invalid API key (every auth-stage failure maps here — no stage leak). |
|
|
| `NOT_APPROVED` | 403 | Method not found **or** key not in scope — indistinguishable by design. |
|
|
| `UNSUPPORTED_MEDIA_TYPE` | 415 | Request carried a body — declared by `Content-Length` or `Transfer-Encoding` (chunked) — with a non-JSON `Content-Type`. A body with **no** `Content-Type` is still parsed leniently as JSON, and a chunked (no `Content-Length`) body is treated identically to a fixed-length one. |
|
|
| `INVALID_JSON` | 400 | Request body was not valid JSON. |
|
|
| `VALIDATION_FAILED` | 400 | Parameter validation failed (missing required field, wrong type, undeclared field). |
|
|
| `BODY_TOO_LARGE` | 413 | Request body exceeded `MaxRequestBodyBytes`. |
|
|
| `STANDBY_NODE` | 503 | Request hit a standby central node; the inbound API serves only the active node. |
|
|
| `TIMEOUT` | 500 | The method script exceeded its execution timeout. |
|
|
| `SITE_UNREACHABLE` | 500 | The target site did not respond to the routed request within the routing timeout (no contact / down / partitioned) — typed from the routed Ask expiring (`AskTimeoutException`), never inferred from an error-message substring. A `Success=false` response means the site answered and is classified `SCRIPT_ERROR`. |
|
|
| `SCRIPT_COMPILE_FAILED` | 500 | The method's script failed to compile (returned consistently on every node until a compiling version is saved). |
|
|
| `SCRIPT_ERROR` | 500 | Catch-all for any other script-execution failure (details are logged centrally, never leaked to the caller). |
|
|
|
|
### Extended Type System
|
|
- API method parameter and return type definitions support an **extended type system** beyond the four template attribute types (Boolean, Integer, Float, String):
|
|
- **Object**: A named structure with typed fields. Supports nesting.
|
|
- **List**: An ordered collection of objects or primitive types.
|
|
- This allows complex request/response structures (e.g., an object containing properties and a list of nested objects).
|
|
- Template attributes retain the simpler four-type system. The extended types apply only to Inbound API method definitions and External System Gateway method definitions.
|
|
|
|
#### Type Definition Format & Nested Validation
|
|
|
|
- Parameter and return type definitions are persisted as **JSON Schema** (the canonical format produced by the Central UI schema builder; see the `MigrateParametersToJsonSchema` migration). An object declares its fields via `properties` (+ a `required` array); a list declares its element type via `items`. The legacy flat-array form (`[{name,type,required,itemType?}]`) is still accepted on read for transition safety.
|
|
- Validation is **recursive and type-aware** for the extended types (request parameters and script return values alike, via a single shared engine so the two cannot drift):
|
|
- **Object**: each declared field's value is validated against its declared (possibly nested) type; a missing required field and a present-but-wrong type are both reported.
|
|
- **List**: every element is validated against the declared element type (recursing into nested objects/lists). A list whose element type is left undeclared (`array` without `items`) is shape-checked only.
|
|
- **Scalars at any depth** are checked against the extended type.
|
|
- Errors are **path-qualified** (e.g. `order.items[2].quantity`) so the caller can locate the offending field.
|
|
- **Undeclared fields are rejected** at every level (consistent with the top-level "unexpected parameter" rejection): an object that declares its fields rejects any field not in its `properties`, so a typo'd field name surfaces as a `400`/error rather than being silently ignored. A bare object schema with no declared fields (`{"type":"object"}`) stays shape-only and accepts any fields.
|
|
- A JSON `null` value satisfies any declared type (a present-but-null field is allowed); only the **absence** of a required field is an error.
|
|
|
|
## Script Compilation & Hot-Reload
|
|
|
|
API method scripts are compiled at central startup — all method definitions are loaded from the configuration database and compiled into in-memory delegates.
|
|
|
|
### Update Workflow
|
|
|
|
- Updating a method via the CLI (`api-method update --id <N> --code '...'`) or Management API triggers immediate recompilation (`CompileAndRegister`). The updated script takes effect on the next API call — no node restart is required.
|
|
- Creating a new method after startup: if the method is created but not yet compiled, the first invocation triggers lazy (on-demand) compilation.
|
|
|
|
> **Compile failures on save are non-fatal but surfaced, and the DB row is authoritative.** The persisted `ApiMethod.Script` row is the single source of truth: on every request `ExecuteAsync` revision-checks the cached delegate against the freshly-fetched script and recompiles in place when it differs, so a saved-but-broken script does **not** leave a previously registered handler silently serving. The change still persists, so the Create/Update Management result carries a top-level **`compileWarning`** (null when the script compiled) listing the Roslyn/trust-model errors. On both **create** and **update**, once a non-compiling script is saved the method returns a compilation error (`HTTP 500`) **consistently on every central node** until a compiling version is saved — there is no node-divergent "old delegate still works here" state. This means a saved script that does not compile is **not silently a no-op** — the caller (CLI / Central UI) sees the warning, and the real diagnostics are also logged centrally.
|
|
|
|
### Direct SQL Warning
|
|
|
|
> **Direct SQL edits to API method scripts are now picked up automatically.** Because the DB row is authoritative and `ExecuteAsync` revision-checks the cached delegate against the freshly-fetched `ApiMethod.Script` on every request, a script changed directly in the database takes effect on the **next call** — no node restart and no Management round-trip is required. A broken direct edit surfaces the same way as a broken save: the method returns a `HTTP 500` compilation error consistently on all central nodes until a compiling version is stored. Editing via the CLI, Management API, or Central UI is still preferred because those paths validate the script up front and return the `compileWarning`, whereas a direct SQL edit is only diagnosed on the next request (and in the central log).
|
|
|
|
---
|
|
|
|
## API Call Logging
|
|
|
|
- **Every request — success or failure — emits one `ApiInbound.Completed` row** to `ICentralAuditWriter` from request middleware before the HTTP response is flushed. The row captures the API key **name** (never the key material), remote IP, user-agent, response status, duration, and the request/response bodies. Bodies are captured in full up to `AuditLog:InboundMaxBytes` (default 1 MiB); `PayloadTruncated = 1` only when that ceiling is hit. Header redaction and per-target body redactors still apply (see Component-AuditLog.md, Payload Capture Policy). This supersedes the earlier failures-only stance: operational API traffic is now part of the centralized audit log, so configuration changes and call activity share a single retention/query surface.
|
|
- Script execution errors (500 responses) remain captured on the same `ApiInbound.Completed` row (response status + error fields) rather than emitting a separate failure-only event.
|
|
- **Fail-soft semantics (fire-and-forget).** The audit write is **fire-and-forget** (`InboundAPI-018`): `WriteAsync` is called without `await` so the user-facing HTTP response is never blocked or delayed by the audit path. Asynchronous faults are observed via a `ContinueWith` continuation that logs at Warning and increments `CentralAuditWriteFailures` (see Health Monitoring #11) rather than being silently dropped. A failed audit append never turns a successful API call into an error returned to the caller.
|
|
- No rate limiting — this is a private API in a controlled industrial environment with a known set of callers. Misbehaving callers are handled operationally (disable the API key).
|
|
|
|
## Request Flow
|
|
|
|
```mermaid
|
|
%%{init: {'theme':'base', 'themeVariables': {'textColor':'#111111','lineColor':'#555555','edgeLabelBackground':'#ffffff','fontSize':'15px'}}}%%
|
|
flowchart TD
|
|
ext(["External System"])
|
|
api["Inbound API (Central)"]
|
|
s1["1. Extract API key from request"]
|
|
s2["2. Validate key exists and is enabled"]
|
|
s3["3. Resolve method by name"]
|
|
s4["4. Check API key is in method's approved list"]
|
|
s5["5. Validate and deserialize parameters"]
|
|
s6["6. Execute implementation script<br/>(subject to method timeout)"]
|
|
s7["7. Serialize return value"]
|
|
s8["8. Return response"]
|
|
|
|
ext --> api
|
|
api --> s1
|
|
s1 --> s2
|
|
s2 --> s3
|
|
s3 --> s4
|
|
s4 --> s5
|
|
s5 --> s6
|
|
s6 --> s7
|
|
s7 --> s8
|
|
|
|
classDef start fill:#d5e8d4,stroke:#82b366,color:#111111;
|
|
classDef proc fill:#dae8fc,stroke:#6c8ebf,color:#111111;
|
|
classDef dec fill:#fff2cc,stroke:#d6b656,color:#111111;
|
|
classDef warn fill:#ffe6cc,stroke:#d79b00,color:#111111;
|
|
classDef alt fill:#e1d5e7,stroke:#9673a6,color:#111111;
|
|
class ext start
|
|
class api proc
|
|
class s1,s2,s3,s4,s5,s7 dec
|
|
class s6 alt
|
|
class s8 warn
|
|
```
|
|
|
|
## Implementation Script Capabilities
|
|
|
|
The C# script that implements an API method executes on the central cluster. Unlike instance scripts at sites, inbound API scripts run on central and can interact with **any instance at any site** through a routing API.
|
|
|
|
Inbound API scripts **cannot** call shared scripts directly — shared scripts are deployed to sites only and execute inline in Script Actors. To execute logic on a site, use `Route.To().Call()`.
|
|
|
|
### Script Runtime API
|
|
|
|
#### Instance Routing
|
|
- `Route.To("instanceUniqueCode").Call("scriptName", parameters)` — Invoke a script on a specific instance at any site. Central routes the call to the appropriate site via the Communication Layer. The call reaches the target Instance Actor's Script Actor, which spawns a Script Execution Actor to execute the script. The return value flows back to the calling API script.
|
|
- `Route.To("instanceUniqueCode").GetAttribute("attributeName")` — Read a single attribute value from a specific instance at any site.
|
|
- `Route.To("instanceUniqueCode").GetAttributes("attr1", "attr2", ...)` — Read multiple attribute values in a **single call**, returned as a dictionary of name-value pairs.
|
|
- `Route.To("instanceUniqueCode").SetAttribute("attributeName", value)` — Write a single attribute value on a specific instance at any site.
|
|
- `Route.To("instanceUniqueCode").SetAttributes(dictionary)` — Write multiple attribute values in a **single call**, accepting a dictionary of name-value pairs.
|
|
- `Route.To("instanceUniqueCode").WaitForAttribute("attributeName", targetValue, timeout)` — Wait, event-driven, until an attribute on a specific instance at any site reaches `targetValue` (value-equality only across the wire), bounded by `timeout`. Returns `true` if matched within the timeout, `false` if it timed out. **The wait is bounded by its own `timeout`, not the generic method-level timeout** — this is the one routed call that may legitimately outlive the method timeout (the site enforces `timeout` and returns `false` when it elapses). A client disconnect still cancels the wait. This is the deliberate exception to the rule below that routed calls inherit the method-level timeout (see "Routing Behavior"): a long event-driven wait is the explicit reason `timeout` governs here.
|
|
|
|
#### Input/Output
|
|
- **Input parameters** are available as defined in the method definition.
|
|
- **Return value** construction matching the defined return structure.
|
|
|
|
#### Parameter Access
|
|
- `Parameters["key"]` — Raw dictionary access.
|
|
- `Parameters.Get<T>("key")` — Typed access (same API as site runtime scripts). See Site Runtime component for full type support.
|
|
|
|
#### Database Access
|
|
|
|
Inbound API scripts may read from and write to the configuration / machine-data
|
|
databases through the **curated, scoped `Database` helper** — `InboundDatabaseHelper`,
|
|
exposed as `InboundScriptContext.Database`. This is the "dedicated, scoped helper added
|
|
as an explicit design change" that the script trust model requires: scripts are **never**
|
|
handed a raw `SqlConnection`, and never reference `System.Data` (the `ForbiddenApiChecker`,
|
|
delegating to the shared `ScriptAnalysis` `ScriptTrustValidator` (#25), still statically
|
|
bans `System.IO`, `Process`, `Threading`, `Reflection`, and raw network access — that is
|
|
defence-in-depth static enforcement, not a true runtime sandbox).
|
|
|
|
The helper API (all asynchronous — scripts `await` them; bounded by the method timeout):
|
|
|
|
- `await Database.QuerySingleAsync<T>("connectionName", sql, parameters)` — first column of the first row as `T` (default if no rows).
|
|
- `await Database.QueryAsync("connectionName", sql, parameters)` — all rows as case-insensitive column→value dictionaries.
|
|
- `await Database.ExecuteAsync("connectionName", sql, parameters)` — run a write (INSERT/UPDATE/DELETE/DDL); returns rows affected. **Writes are permitted** — the move-in integration records results, not just reads them.
|
|
|
|
Containment rules (enforced, not advisory):
|
|
|
|
- **Named connections only.** `connectionName` selects one of the connections configured
|
|
on the central database gateway (`IDatabaseGateway`); a script cannot supply an arbitrary
|
|
connection string or reach a database the gateway is not configured for.
|
|
- **SQL-injection protection.** Statement text is authored by the (design-time) method
|
|
script, but every request-derived **value** is passed via `parameters` and bound as a
|
|
named `@`-prefixed SQL parameter — never string-concatenated into the command text.
|
|
Request input therefore reaches the database only through parameter binding.
|
|
- **Deadline-bound.** Calls use the async ADO.NET path end-to-end (no pool-thread blocking)
|
|
and honour the method deadline token, with a `CommandTimeout` backstop derived from the
|
|
method timeout, so a slow query is bounded by the method timeout.
|
|
|
|
For everything else, scripts interact with the system through the curated `Route` and
|
|
`Parameters` surfaces above.
|
|
|
|
### Routing Behavior
|
|
- The `Route.To()` helper resolves the instance's site assignment from the configuration database and routes the request to the correct site cluster via the Communication Layer.
|
|
- The call is **synchronous from the API caller's perspective** — the API method blocks until the site responds or the **method-level timeout** is reached.
|
|
- If the target site is unreachable or the call times out, the call fails and the API returns an error to the caller. No store-and-forward buffering is used for inbound API calls.
|
|
|
|
## Authentication Details
|
|
|
|
- The API key token is accepted from **either** the `Authorization: Bearer <token>` header (preferred) **or** the `X-API-Key: <token>` header (alternate). `Authorization` takes precedence when both are present.
|
|
- The token format is `sbk_<keyId>_<secret>`. Both header forms are verified by the same `IApiKeyVerifier` from `ZB.MOM.WW.Auth.ApiKeys`.
|
|
- The system validates:
|
|
1. The token is well-formed and the key ID exists in the key store.
|
|
2. The key is enabled (not revoked).
|
|
3. The key's provisioned scopes include the requested method name (case-sensitive).
|
|
- Failed authentication returns an appropriate HTTP error (401 Unauthorized for invalid/missing credentials; 403 Forbidden for a valid key not approved for the method). For enumeration-safety, both "method not found" and "key not in scope" return 403 with an identical body.
|
|
|
|
## Error Handling
|
|
|
|
- Invalid API key → 401 Unauthorized.
|
|
- Valid key but not approved for method → 403 Forbidden.
|
|
- Invalid parameters → 400 Bad Request.
|
|
- Script execution failure → 500 Internal Server Error (with safe error message, no internal details exposed).
|
|
- Script errors are logged in the central audit/event system.
|
|
|
|
## Dependencies
|
|
|
|
- **Configuration Database (MS SQL)**: Stores API keys and method definitions.
|
|
- **Communication Layer**: Routes requests to sites when method implementations need site data.
|
|
- **Security & Auth**: API key validation (separate from LDAP/AD — API uses key-based auth).
|
|
- **Configuration Database (via IAuditService)**: All API key and method definition changes are audit logged.
|
|
- **Script Analysis (#25)**: `ForbiddenApiChecker` delegates to `ScriptTrustValidator.FindViolations` for the authoritative forbidden-API verdict during script compilation and validation.
|
|
- **Audit Log (#23)**: Every inbound API request emits an `ApiInbound.Completed` row via `ICentralAuditWriter` from request middleware (non-blocking for the HTTP response). Request and response bodies are captured in full up to `AuditLog:InboundMaxBytes` (default 1 MiB) per the Audit Log Payload Capture Policy; redaction (headers + per-target body redactors) still applies before persistence.
|
|
- **Cluster Infrastructure**: API is hosted on the active central node and fails over with it.
|
|
|
|
## Interactions
|
|
|
|
- **External Systems**: Call the API with API keys.
|
|
- **Communication Layer**: API method scripts use this to reach sites.
|
|
- **Site Runtime (Instance Actors, Script Actors)**: Routed calls execute on site Instance Actors via their Script Actors.
|
|
- **Central UI**: Administrator manages API keys; Designer manages method definitions.
|
|
- **Configuration Database (via IAuditService)**: Configuration changes are audited.
|