Joseph Doherty
644f564c30
Unreplicated, the site-x survivor came up empty (by design: no LocalDb peer); replicated, kill and freeze each lost only the 26-30 s takeover window, every buffered message was delivered once, and the 4 duplicate ticks came from the frozen victim's 6 s zombie window. Central kill: 0 gap. Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi
30 lines
1.6 KiB
Bash
Executable File
30 lines
1.6 KiB
Bash
Executable File
#!/bin/bash
|
|
# sample.sh <seconds> [interval]: per-node data-path metrics + sink count, one line per interval.
|
|
DUR=${1:-120}; IV=${2:-5}; END=$(( $(date +%s) + DUR ))
|
|
m() { timeout 3 docker exec "scadabridge-env2-site-x-$1" bash -c 'exec 3<>/dev/tcp/localhost/8084 && printf "GET /metrics HTTP/1.0\r\nHost: localhost\r\n\r\n" >&3 && cat <&3' 2>/dev/null | python3 -c '
|
|
import sys,re
|
|
v={}
|
|
for l in sys.stdin:
|
|
if l.startswith("#"): continue
|
|
p=l.split()
|
|
if len(p)<2: continue
|
|
name=p[0]; val=float(p[1]) if re.match(r"^-?[0-9.e+]+$",p[1]) else 0
|
|
k=None
|
|
if name.startswith("scadabridge_site_script_runs_total"): k="runs"
|
|
elif name.startswith("scadabridge_site_data_connections_connected"): k="dc"
|
|
elif name.startswith("scadabridge_site_data_connections_total"): k="dct"
|
|
elif name.startswith("scadabridge_store_and_forward_queue_depth_items"): k="depth"
|
|
elif name.startswith("scadabridge_store_and_forward_activity_total"):
|
|
mm=re.search(r"(action)=\"([A-Za-z]+)\"",name); k="sf_"+(mm.group(2) if mm else "x")
|
|
if k: v[k]=v.get(k,0)+val
|
|
print(" ".join(f"{k}={int(x)}" for k,x in sorted(v.items())) or "DOWN")'; }
|
|
while [ "$(date +%s)" -lt "$END" ]; do
|
|
s=$(docker exec scadabridge-env2-sfsink sh -c 'wc -l < /data/received.jsonl; cat /data/mode 2>/dev/null || echo ok; tail -1 /data/received.jsonl' 2>/dev/null | python3 -c '
|
|
import sys,json
|
|
l=sys.stdin.read().splitlines()
|
|
t=json.loads(l[2])["body"]["tick"] if len(l)>2 and l[2] else "-"
|
|
print(f"sink_rows={l[0].strip()} mode={l[1].strip()} last_tick={t}")')
|
|
echo "$(date -u +%H:%M:%S) A[$(m a)] B[$(m b)] $s"
|
|
sleep "$IV"
|
|
done
|