dohertj2/ScadaBridge
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

feat(inbound-api): nested Object/List extended-type validation (#13)

Browse Source 
Object/List parameters and return values were shape-validated only (object vs
array), with no field-level/nested type checks — type-wrong nested data passed
inbound validation and failed only at script runtime. Add recursive type
validation (declared Object field types, List element type, scalars at any depth)
with path-qualified errors, symmetric across ParameterValidator and ReturnValueValidator.

Both validators now parse the canonical JSON Schema definition format (the
Central UI / MigrateParametersToJsonSchema output) via a shared recursive engine,
Commons.Types.InboundApi.InboundApiSchema, instead of the legacy flat
[{name,type}] array which they could not even deserialize from migrated rows.
The legacy flat-array form is still accepted on read for transition safety.
Undeclared fields are rejected at every level (consistent with the existing
top-level unexpected-parameter rejection); a present-but-null value satisfies
any type, only absence of a required field is an error.
This commit is contained in:
Joseph Doherty
2026-06-15 15:04:28 -04:00
parent 3032faac0d
commit 4b6187c853
8 changed files with 982 additions and 286 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/components/InboundAPI.md
+13 -13
View File
@@ -36,28 +36,28 @@ public class ApiMethod
public int Id { get; set; }
public string Name { get; set; } // route segment
public string Script { get; set; } // Roslyn C# script body
public string? ParameterDefinitions { get; set; } // JSON: List<ParameterDefinition>
public string? ReturnDefinition { get; set; } // JSON: List<ReturnFieldDefinition>
public string? ParameterDefinitions { get; set; } // JSON Schema (object) describing parameters
public string? ReturnDefinition { get; set; } // JSON Schema describing the return value
public int TimeoutSeconds { get; set; }
}
```
`ParameterDefinitions` and `ReturnDefinition` are stored as JSON strings to keep the schema simple; both are deserialized on every request by `ParameterValidator` and `ReturnValueValidator`.
`ParameterDefinitions` and `ReturnDefinition` are stored as JSON Schema strings (canonical form: `{"type":"object","properties":{…},"required":[…]}`, arrays via `"items"`); both are parsed on every request by `ParameterValidator` and `ReturnValueValidator` into a shared recursive `InboundApiSchema` (Commons). The legacy flat-array form (`[{name,type,required,itemType?}]`) is still accepted on read.
### Extended type system
Parameter and return field definitions share the same six-type vocabulary:
Parameter and return definitions share the same six-type vocabulary (JSON Schema type tokens in parentheses):
| Type | JSON shape | C# value after coercion |
|-----------|----------------------|-------------------------------------|
| `Boolean` | `true` / `false` | `bool` |
| `Integer` | number (whole) | `long` |
| `Float` | number | `double` |
| `String` | string | `string` |
| `Object` | JSON object | `Dictionary<string, object?>` |
| `List` | JSON array | `List<object?>` |
| Type | JSON Schema token | JSON shape | C# value after coercion |
|-----------|-------------------|------------------|-------------------------------|
| `Boolean` | `boolean` | `true` / `false` | `bool` |
| `Integer` | `integer` | number (whole) | `long` |
| `Float` | `number` | number | `double` |
| `String` | `string` | string | `string` |
| `Object` | `object` | JSON object | `Dictionary<string, object?>` |
| `List` | `array` | JSON array | `List<object?>` |
`Object` and `List` are validated for JSON shape only — field-level or element-level type constraints are the script's responsibility. Template attributes use only the four primitive types; the extended types apply here and in the External System Gateway.
`Object` and `List` are validated **recursively**: a declared object validates each field against its declared (nested) type and rejects undeclared fields; a list validates every element against the declared `items` type. Scalars are checked at any depth and errors are path-qualified (e.g. `order.items[2].quantity`). A bare `{"type":"object"}` / `{"type":"array"}` (no `properties` / `items`) stays shape-only. Template attributes use only the four primitive types; the extended types apply here and in the External System Gateway.
## Architecture
docs/plans/2026-06-15-stillpending-m2-implementation.md
+1
View File
@@ -102,6 +102,7 @@ Risk-first, migration-safe ordering. `#32` first (unblocks DB-backed verificatio
**Fix:** Recursive descent through the declared `Object` field schema / `List` element type, type-checking each level (scalars by extended-type, nested Object/List recursively). Reuse the existing extended-type system; keep error messages path-qualified (`field.sub[2].x`). Apply symmetrically in both validators.
**Tests:** `tests/.../InboundAPI.Tests` — valid nested payload passes; wrong scalar type at depth, wrong list element type, missing required nested field → rejected with path.
**DoD:** Nested type mismatches are caught at inbound validation, not at script runtime. (Satisfies the M4 cross-reference to this item.)
**Status: complete.** A shared recursive engine, `Commons.Types.InboundApi.InboundApiSchema` (parse + path-qualified `Validate`), backs both validators so they cannot drift. Key finding: the canonical persisted/authored format is **JSON Schema** (object `properties` + `required`, array `items`) — produced by the Central UI schema builder and the `MigrateParametersToJsonSchema` migration — but the validators still parsed the *legacy flat array* `[{name,type}]` and only shape-checked `Object`/`List`. They could not even consume a migrated JSON-Schema-object definition (the `Deserialize<List<…>>` would fail). Rewriting both to read `InboundApiSchema` fixes that latent format mismatch *and* delivers true nested validation; the legacy flat array is still accepted on read (case-insensitive keys) for transition safety. **Undeclared-field policy: reject at every level** (a declared object rejects any field not in its `properties`, consistent with the existing top-level `InboundAPI-010` "unexpected parameter" rejection); a bare `{"type":"object"}` with no declared fields stays shape-only. A present-but-null value satisfies any type; only the *absence* of a required field is an error.
### M2.7 — #20 + #21: return-type + argument-type compatibility checks
**Classification:** standard · **Files:** `src/.../TemplateEngine/Validation/SemanticValidator.cs:62-63,251-266,279-287,390-425`.
docs/requirements/Component-InboundAPI.md
+14 -2
View File
@@ -40,9 +40,10 @@ Each API method definition includes:
- **Approved API Keys**: List of API keys authorized to invoke this method. Requests from non-approved keys are rejected.
- **Parameter Definitions**: Ordered list of input parameters, each with:
- Parameter name.
- Data type (Boolean, Integer, Float, String — same fixed set as template attributes).
- Data type — the **extended type system** (Boolean, Integer, Float, String, plus the nestable Object and List; see [Extended Type System](#extended-type-system)).
- Whether the parameter is required.
- **Return Value Definition**: Structure of the response, with:
- Field names and data types. Supports returning **lists of objects**.
- Field names and (extended-system) data types. Supports returning **lists of objects** and arbitrarily nested structures.
- **Implementation Script**: C# script that executes when the method is called. Stored **inline** in the method definition. Follows standard C# authoring patterns but has no template inheritance — it is a standalone script tied to this method.
- **Timeout**: Configurable per method. Defines the maximum time the method is allowed to execute (including any routed calls to sites) before returning a timeout error to the caller.
@@ -99,6 +100,17 @@ Each API method definition includes:
- This allows complex request/response structures (e.g., an object containing properties and a list of nested objects).
- Template attributes retain the simpler four-type system. The extended types apply only to Inbound API method definitions and External System Gateway method definitions.
#### Type Definition Format & Nested Validation
- Parameter and return type definitions are persisted as **JSON Schema** (the canonical format produced by the Central UI schema builder; see the `MigrateParametersToJsonSchema` migration). An object declares its fields via `properties` (+ a `required` array); a list declares its element type via `items`. The legacy flat-array form (`[{name,type,required,itemType?}]`) is still accepted on read for transition safety.
- Validation is **recursive and type-aware** for the extended types (request parameters and script return values alike, via a single shared engine so the two cannot drift):
- **Object**: each declared field's value is validated against its declared (possibly nested) type; a missing required field and a present-but-wrong type are both reported.
- **List**: every element is validated against the declared element type (recursing into nested objects/lists). A list whose element type is left undeclared (`array` without `items`) is shape-checked only.
- **Scalars at any depth** are checked against the extended type.
- Errors are **path-qualified** (e.g. `order.items[2].quantity`) so the caller can locate the offending field.
- **Undeclared fields are rejected** at every level (consistent with the top-level "unexpected parameter" rejection): an object that declares its fields rejects any field not in its `properties`, so a typo'd field name surfaces as a `400`/error rather than being silently ignored. A bare object schema with no declared fields (`{"type":"object"}`) stays shape-only and accepts any fields.
- A JSON `null` value satisfies any declared type (a present-but-null field is allowed); only the **absence** of a required field is an error.
## Script Compilation & Hot-Reload
API method scripts are compiled at central startup — all method definitions are loaded from the configuration database and compiled into in-memory delegates.