Evidence doc gains the #55 section (root cause of no logs, 0.3.0 contents + SHA256, per-app commits, collector output per app: service.version, site.id, node.role and log records on OtOpcUa, HistorianGateway, ScadaBridge Site and MxAccessGateway). GAPS, current-state, SPEC, the Telemetry CLAUDE.md and the index row updated; infra/otel-collector gains otelidentity.py (Resource per signal + log samples). Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
4.6 KiB
Shared dev OpenTelemetry collector (zb-otel-collector)
One OpenTelemetry Collector (contrib)
that the family apps' OTLP exporters point at for dev/test. It runs as a single container on the
shared Docker host 10.100.0.35, next to the other shared fixtures (zb-shared-glauth, the
OtOpcUa driver fixtures). Added by scadaproj#13; evidence of the first end-to-end run:
components/observability/2026-11-12-otlp-end-to-end-evidence.md.
Scope: dev/test only. The receivers have no auth and no TLS. No wonder tier (DEV vd03, QA vq03/vq04) has a collector yet — that is scadaproj#54.
Endpoints
| Port | What | Use |
|---|---|---|
4317 |
OTLP gRPC receiver | The family default. Every app's OtlpExporter uses the gRPC protocol, so point apps here. |
4318 |
OTLP HTTP/protobuf receiver | For clients that only speak OTLP/HTTP (/v1/traces, /v1/metrics, /v1/logs). |
8889 |
Prometheus exporter | Every metric received over OTLP, re-exposed for scraping, with job=<service.namespace>/<service.name>, instance=<service.instance.id> and the identity labels (service_name, site_id, node_role, host_name). Series expire 15 min after the last update. |
13133 |
health_check |
curl -s http://10.100.0.35:13133/ → {"status":"Server available",...} |
Reachable from the Mac and from windev (10.100.0.48) — both verified.
Pointing an app at it
OTLP is opt-in and additive: Prometheus /metrics keeps working when it is switched on.
| App | Config keys (env form) |
|---|---|
| OtOpcUa | OtOpcUa__Telemetry__Exporter=Otlp, OtOpcUa__Telemetry__OtlpEndpoint=http://10.100.0.35:4317 |
| ScadaBridge | ScadaBridge__Telemetry__Exporter=Otlp, ScadaBridge__Telemetry__OtlpEndpoint=http://10.100.0.35:4317 |
| MxAccessGateway | MxGateway__Telemetry__Exporter=Otlp, MxGateway__Telemetry__OtlpEndpoint=http://10.100.0.35:4317 |
| HistorianGateway | Telemetry__Exporter=Otlp, Telemetry__OtlpEndpoint=http://10.100.0.35:4317 (root-level section) |
- Always include the scheme:
http://10.100.0.35:4317. Telemetry0.3.0(what the apps pin since 2026-11-12) rejects a scheme-lesshost:portat startup whenExporter=Otlp. - The SDK's default metric export interval is 60 s. For a quick check set
OTEL_METRIC_EXPORT_INTERVAL=10000(ms) on the process. - What travels: metrics, traces and logs (Telemetry
0.3.0+, scadaproj#55): withExporter=Otlpthe Serilog logs go to the same endpoint with the same Resource.<prefix>:Telemetry:ExportLogs=falsekeeps them local. Every app's Resource carriesservice.version(git SHA),site.idandnode.role; mxgw and HistorianGateway reportnode.role=standaloneand takesite.idfrom<prefix>:Telemetry:SiteId.
Reading what arrived
ssh dohertj2@10.100.0.35
docker logs --tail 20 zb-otel-collector # one debug line per received batch
cd ~/zb-otel-collector/data && python3 ../otelsum.py # per service: Resource, metric names, span names
python3 ../otelsum.py otopcua # one service
python3 ../otelidentity.py otopcua :12345 # Resource per signal + log samples, one process (instance-id suffix)
curl -s http://10.100.0.35:8889/metrics | grep 'job="ZB.MOM.WW/otopcua"' | head
data/{metrics,traces,logs}.jsonl are OTLP-JSON, one export request per line, rotated at 50 MB
(3 backups). otelsum.py is the summariser used for the evidence doc;
otelidentity.py prints the Resource each signal arrived with, the log count by severity
and sample log records, filtered to one process (scadaproj#55).
Deploy / update
# from this directory on the Mac
ssh dohertj2@10.100.0.35 'mkdir -p ~/zb-otel-collector/data && chmod 777 ~/zb-otel-collector/data'
scp docker-compose.yml config.yaml otelsum.py otelidentity.py dohertj2@10.100.0.35:zb-otel-collector/
ssh dohertj2@10.100.0.35 'cd ~/zb-otel-collector && docker compose up -d --force-recreate'
- The image runs as uid
10001, sodata/must be writable by it (chmod 777above, orsudo chown 10001:10001 data). - The image is pinned to an exact release (
0.162.0). Bump deliberately, re-run the health check and one app, and update this README.resource_to_telemetry_conversionis deprecated in this release; the config uses its replacementresource_constant_labels. - Host directory:
~/zb-otel-collector/(compose projectzb-otel-collector). It shares no network or volume with any other container. Tear down withdocker compose downthere.