Compare commits

..

22 Commits

Author SHA1 Message Date
Joseph Doherty e13152f340 test: remove redundant HostBuildingCollection workaround (shared lib no longer installs a global frozen logger) 2026-06-01 08:47:21 -04:00
Joseph Doherty deba5ed115 refactor(logging): correlation scope + redaction on shared ILogRedactor seam
Move the per-request correlation context and secret redaction off the MEL
mechanism onto the Serilog primitives the shared bootstrap consumes.

- GatewayRequestLoggingMiddlewareExtensions now pushes the correlation
  properties (SessionId / WorkerProcessId / CorrelationId / CommandMethod /
  ClientIdentity) via Serilog LogContext.PushProperty for the request lifetime
  and pops them on completion, replacing MEL ILogger.BeginScope. Header parsing
  and property names are unchanged; GatewayLogScope remains the data holder.
- Add GatewayLogRedactorAdapter : ILogRedactor delegating to the existing
  GatewayLogRedactor policy (mxgw_ bearer tokens / credential-bearing command
  values), registered as a singleton so the shared RedactionEnricher masks
  secrets on every event. Remove the now-dead GatewayLoggerExtensions MEL helper.
- Tests: add GatewayLogRedactorAdapterTests; serialize the four host-building
  test classes into one non-parallel collection (HostBuildingCollection) so the
  process-wide Serilog bootstrap logger is not frozen by two concurrent host
  builds racing in parallel collections.

The net48/x86 worker is untouched.
2026-06-01 08:06:28 -04:00
Joseph Doherty 4bf71a0b2c refactor(logging): adopt ZB.MOM.WW.Telemetry.Serilog bootstrap
Swap the gateway process logging from the default Microsoft.Extensions.Logging
provider onto the shared ZB.MOM.WW.Telemetry.Serilog two-stage bootstrap.

- Add a cross-repo ProjectReference to ZB.MOM.WW.Telemetry.Serilog (transitively
  brings the Telemetry core package); the referenced project resolves its own
  Directory.Build.props / Directory.Packages.props so it does not perturb this build.
- Replace MEL wiring in GatewayApplication with builder.AddZbSerilog(ServiceName=
  "mxgateway"; SiteId/NodeRole read from MxGateway:Telemetry when present) and add
  app.UseSerilogRequestLogging().
- Add a Serilog section (Console + daily rolling File sinks, MinimumLevel) to
  appsettings.json and a MinimumLevel override to appsettings.Development.json,
  replacing the old MEL Logging sections.

The net48/x86 worker is untouched. Correlation scope + redaction move to the
shared ILogRedactor seam in the follow-up commit.
2026-06-01 08:03:49 -04:00
Joseph Doherty b4a7bac4c0 scripts: add pack-clients.ps1 to pack/publish all 5 client packages 2026-05-28 17:12:08 -04:00
Joseph Doherty 6df373ae4c client/go: release docs and tag-go-module.ps1 helper 2026-05-28 17:07:25 -04:00
Joseph Doherty fe44e3c18a client/java: maven-publish wiring for Gitea Maven feed 2026-05-28 17:07:11 -04:00
Joseph Doherty 523f944f3e client/rust: Cargo metadata + Gitea alternative-registry config 2026-05-28 17:06:47 -04:00
Joseph Doherty c33f1e6047 client/python: PyPI metadata + Gitea feed install instructions 2026-05-28 17:06:01 -04:00
Joseph Doherty 92cc4688e6 client/go: avoid holding mutex across BrowseChildren RPC in Expand 2026-05-28 15:33:48 -04:00
Joseph Doherty a155554038 grpc: reuse GalaxyBrowseProjector.ResolveParentId from handler 2026-05-28 15:32:48 -04:00
Joseph Doherty 68f905a344 client/java: avoid holding monitor across BrowseChildren RPC in expand 2026-05-28 15:32:36 -04:00
Joseph Doherty 5abc222c72 galaxy: add by-name and by-path indexes to GalaxyHierarchyIndex 2026-05-28 15:31:56 -04:00
Joseph Doherty da3aa7b0b2 client/go: paginate DiscoverHierarchy across multi-page galaxies 2026-05-28 15:31:16 -04:00
Joseph Doherty f0ec068430 galaxy: add cycle guard to HasMatchingDescendant 2026-05-28 15:30:08 -04:00
Joseph Doherty 1a1d14a9fd client/python: add public browse_children_raw for API parity 2026-05-28 15:29:08 -04:00
Joseph Doherty b2448510ac client/java: add browseChildrenRejectsRepeatedPageToken test for parity 2026-05-28 15:17:52 -04:00
Joseph Doherty 75610e3f55 client/go: wrap browseChildren duplicate-page-token error in GatewayError 2026-05-28 15:17:10 -04:00
Joseph Doherty 5032166106 client/dotnet: assert failed expand leaves node unexpanded 2026-05-28 15:16:07 -04:00
Joseph Doherty 76a042d663 grpc: make page_token error strings RPC-name-agnostic 2026-05-28 15:15:40 -04:00
Joseph Doherty 4a19854eb9 docs: per-client High-level walker example using LazyBrowseNode
Add a "High-level walker" subsection under each client's "Browsing
lazily" section showing idiomatic use of LazyBrowseNode (browse +
expand, idempotency note, redeploy refresh pattern).
2026-05-28 14:34:19 -04:00
Joseph Doherty a4467e23ef client/python: make LazyBrowseNode.expand concurrency-safe 2026-05-28 14:32:35 -04:00
Joseph Doherty eacfeff9fb client/dotnet: make LazyBrowseNode.ExpandAsync thread-safe 2026-05-28 14:28:36 -04:00
40 changed files with 1831 additions and 163 deletions
+21
View File
@@ -0,0 +1,21 @@
<Project>
<PropertyGroup>
<!-- Shared package metadata for clients/dotnet/. Individual projects opt in via <IsPackable>true</IsPackable>. -->
<Authors>Joseph Doherty</Authors>
<Company>ZB MOM WW</Company>
<Copyright>Copyright (c) ZB MOM WW. All rights reserved.</Copyright>
<Product>MxAccessGateway Client</Product>
<RepositoryUrl>https://gitea.dohertylan.com/dohertj2/mxaccessgw</RepositoryUrl>
<RepositoryType>git</RepositoryType>
<PackageProjectUrl>https://gitea.dohertylan.com/dohertj2/mxaccessgw</PackageProjectUrl>
<PackageTags>mxaccess;mxgateway;grpc;client;archestra</PackageTags>
<PackageRequireLicenseAcceptance>false</PackageRequireLicenseAcceptance>
<!-- Versioning: bump per release. Symbols ship as snupkg. -->
<Version>0.1.0</Version>
<IncludeSymbols>true</IncludeSymbols>
<SymbolPackageFormat>snupkg</SymbolPackageFormat>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
<!-- Default: do NOT pack. Each project opts in. -->
<IsPackable>false</IsPackable>
</PropertyGroup>
</Project>
+49
View File
@@ -218,6 +218,32 @@ for (int i = 0; i < roots.Children.Count; i++)
}
```
#### High-level walker
For UI trees, the client provides a `LazyBrowseNode` walker that handles
sibling pagination and the `child_has_children` hint for you:
```csharp
await using GalaxyRepositoryClient repository = GalaxyRepositoryClient.Create(
new MxGatewayClientOptions { Endpoint = new Uri("http://localhost:5000"), ApiKey = apiKey });
IReadOnlyList<LazyBrowseNode> roots = await repository.BrowseAsync();
foreach (LazyBrowseNode root in roots)
{
if (root.HasChildrenHint)
{
await root.ExpandAsync();
}
foreach (LazyBrowseNode child in root.Children)
{
Console.WriteLine($"{child.Object.TagName} ({(child.HasChildrenHint ? "has children" : "leaf")})");
}
}
```
`ExpandAsync` is idempotent — calling it twice fires only one RPC,
and is safe under concurrent callers. To refresh after a Galaxy redeploy, call
`BrowseAsync` again from the root.
### Watching deploy events
`WatchDeployEventsAsync` opens the `WatchDeployEvents` server-streaming RPC. The
@@ -273,6 +299,29 @@ $env:MXGATEWAY_TEST_ITEM = 'Area001.Pump001.Speed'
dotnet run --project clients/dotnet/ZB.MOM.WW.MxGateway.Client.Cli -- smoke --endpoint $env:MXGATEWAY_ENDPOINT --api-key-env MXGATEWAY_API_KEY --item $env:MXGATEWAY_TEST_ITEM --json
```
## Installing as a NuGet Package
The client publishes to the internal Gitea NuGet feed at
`https://gitea.dohertylan.com/api/packages/dohertj2/nuget/index.json`.
Add the feed once:
````bash
dotnet nuget add source https://gitea.dohertylan.com/api/packages/dohertj2/nuget/index.json \
--name dohertj2-gitea \
--username <gitea-username> \
--password <gitea-token-or-password> \
--store-password-in-clear-text
````
Then add the package to your project:
````bash
dotnet add package ZB.MOM.WW.MxGateway.Client --version 0.1.0
````
The `ZB.MOM.WW.MxGateway.Contracts` package is pulled in transitively.
## Related Documentation
- [Client Packaging](../../docs/ClientPackaging.md)
@@ -106,6 +106,8 @@ public sealed class LazyBrowseNodeTests
new RpcException(new Status(StatusCode.NotFound, "Parent not found"))));
await Assert.ThrowsAsync<MxGatewayException>(async () => await roots[0].ExpandAsync());
Assert.False(roots[0].IsExpanded);
Assert.Empty(roots[0].Children);
}
/// <summary>
@@ -142,6 +144,37 @@ public sealed class LazyBrowseNodeTests
Assert.Equal("7:abc:2", transport.BrowseChildrenCalls[2].Request.PageToken);
}
/// <summary>
/// Verifies that ten concurrent ExpandAsync calls issue exactly one RPC, not ten.
/// </summary>
[Fact]
public async Task Expand_CalledConcurrently_OnlyFiresOneRpc()
{
FakeGalaxyRepositoryTransport transport = CreateTransport();
transport.BrowseChildrenReplies.Enqueue(BuildReply(
children: [BuildObject(1, "Plant", isArea: true)],
childHasChildren: [true],
cacheSequence: 7));
transport.BrowseChildrenReplies.Enqueue(BuildReply(
children: [BuildObject(2, "Mixer_001")],
childHasChildren: [false],
cacheSequence: 7));
await using GalaxyRepositoryClient client = CreateClient(transport);
IReadOnlyList<LazyBrowseNode> roots = await client.BrowseAsync();
// Fire ten concurrent expands of the same node.
Task[] tasks = Enumerable.Range(0, 10)
.Select(_ => roots[0].ExpandAsync())
.ToArray();
await Task.WhenAll(tasks);
Assert.True(roots[0].IsExpanded);
Assert.Single(roots[0].Children);
// 1 roots fetch + exactly 1 expand fetch = 2 total
Assert.Equal(2, transport.BrowseChildrenCalls.Count);
}
/// <summary>
/// Verifies that BrowseChildrenOptions filter fields are forwarded to the BrowseChildren request.
/// </summary>
@@ -13,6 +13,7 @@ public sealed class LazyBrowseNode
private readonly GalaxyRepositoryClient _client;
private readonly BrowseChildrenOptions _options;
private readonly List<LazyBrowseNode> _children = [];
private readonly SemaphoreSlim _expandLock = new(1, 1);
private bool _isExpanded;
internal LazyBrowseNode(
@@ -43,6 +44,10 @@ public sealed class LazyBrowseNode
/// Fetches direct children from the gateway and populates <see cref="Children"/>.
/// Idempotent: subsequent calls are no-ops.
/// </summary>
/// <remarks>
/// Thread-safe: concurrent callers see exactly one fetch; subsequent callers
/// (after the first completes) return immediately.
/// </remarks>
/// <param name="cancellationToken">Token to observe for cancellation.</param>
public async Task ExpandAsync(CancellationToken cancellationToken = default)
{
@@ -51,33 +56,46 @@ public sealed class LazyBrowseNode
return;
}
string pageToken = string.Empty;
HashSet<string> seenPageTokens = new(StringComparer.Ordinal);
do
await _expandLock.WaitAsync(cancellationToken).ConfigureAwait(false);
try
{
BrowseChildrenRequest request = GalaxyRepositoryClient.BuildBrowseChildrenRequest(_options);
request.ParentGobjectId = Object.GobjectId;
request.PageToken = pageToken;
BrowseChildrenReply reply = await _client
.BrowseChildrenRawAsync(request, cancellationToken)
.ConfigureAwait(false);
for (int i = 0; i < reply.Children.Count; i++)
if (_isExpanded)
{
bool hint = i < reply.ChildHasChildren.Count && reply.ChildHasChildren[i];
_children.Add(new LazyBrowseNode(_client, reply.Children[i], hint, _options));
return;
}
pageToken = reply.NextPageToken;
if (!string.IsNullOrWhiteSpace(pageToken) && !seenPageTokens.Add(pageToken))
string pageToken = string.Empty;
HashSet<string> seenPageTokens = new(StringComparer.Ordinal);
do
{
throw new MxGatewayException(
$"Galaxy BrowseChildren returned a repeated page token '{pageToken}'.");
BrowseChildrenRequest request = GalaxyRepositoryClient.BuildBrowseChildrenRequest(_options);
request.ParentGobjectId = Object.GobjectId;
request.PageToken = pageToken;
BrowseChildrenReply reply = await _client
.BrowseChildrenRawAsync(request, cancellationToken)
.ConfigureAwait(false);
for (int i = 0; i < reply.Children.Count; i++)
{
bool hint = i < reply.ChildHasChildren.Count && reply.ChildHasChildren[i];
_children.Add(new LazyBrowseNode(_client, reply.Children[i], hint, _options));
}
pageToken = reply.NextPageToken;
if (!string.IsNullOrWhiteSpace(pageToken) && !seenPageTokens.Add(pageToken))
{
throw new MxGatewayException(
$"Galaxy BrowseChildren returned a repeated page token '{pageToken}'.");
}
}
while (!string.IsNullOrWhiteSpace(pageToken));
_isExpanded = true;
}
finally
{
_expandLock.Release();
}
while (!string.IsNullOrWhiteSpace(pageToken));
_isExpanded = true;
}
}
@@ -7,9 +7,11 @@ namespace ZB.MOM.WW.MxGateway.Client;
/// </summary>
public static class MxGatewayClientContractInfo
{
/// <inheritdoc cref="GatewayContractInfo.GatewayProtocolVersion"/>
public const uint GatewayProtocolVersion =
GatewayContractInfo.GatewayProtocolVersion;
/// <inheritdoc cref="GatewayContractInfo.WorkerProtocolVersion"/>
public const uint WorkerProtocolVersion =
GatewayContractInfo.WorkerProtocolVersion;
}
@@ -16,4 +16,15 @@
<Nullable>enable</Nullable>
</PropertyGroup>
<PropertyGroup>
<IsPackable>true</IsPackable>
<PackageId>ZB.MOM.WW.MxGateway.Client</PackageId>
<Description>.NET 10 gRPC client for the MxAccessGateway service. Provides typed wrappers, retry, and a lazy-browse walker over the Galaxy Repository hierarchy.</Description>
<PackageReadmeFile>README.md</PackageReadmeFile>
</PropertyGroup>
<ItemGroup>
<None Include="..\README.md" Pack="true" PackagePath="\" />
</ItemGroup>
</Project>
+72
View File
@@ -143,6 +143,46 @@ for i, child := range reply.GetChildren() {
}
```
#### High-level walker
For UI trees, the client provides a `LazyBrowseNode` walker that handles
sibling pagination and the `child_has_children` hint for you:
```go
galaxy, err := mxgateway.DialGalaxy(ctx, mxgateway.Options{
Endpoint: "localhost:5000",
APIKey: os.Getenv("MXGATEWAY_API_KEY"),
Plaintext: true,
})
if err != nil {
log.Fatal(err)
}
defer galaxy.Close()
roots, err := galaxy.Browse(ctx, nil)
if err != nil {
log.Fatal(err)
}
for _, root := range roots {
if root.HasChildrenHint() {
if err := root.Expand(ctx); err != nil {
log.Fatal(err)
}
}
for _, child := range root.Children() {
kind := "leaf"
if child.HasChildrenHint() {
kind = "has children"
}
fmt.Printf("%s (%s)\n", child.Object().GetTagName(), kind)
}
}
```
`Expand` is idempotent — calling it twice fires only one RPC,
and is safe under concurrent callers. To refresh after a Galaxy redeploy, call
`Browse` again from the root.
### Watching deploy events
`WatchDeployEvents` opens a server-streaming subscription. The server emits a
@@ -235,6 +275,38 @@ $env:MXGATEWAY_TEST_ITEM = 'Area001.Tag.Value'
go run ./cmd/mxgw-go smoke -endpoint $env:MXGATEWAY_ENDPOINT -plaintext -api-key-env MXGATEWAY_API_KEY -item $env:MXGATEWAY_TEST_ITEM -json
```
## Installing the Go client
The module is resolved directly from the git repo — no package registry:
````bash
go get gitea.dohertylan.com/dohertj2/mxaccessgw/clients/go@v0.1.0
````
Then import:
````go
import "gitea.dohertylan.com/dohertj2/mxaccessgw/clients/go/mxgateway"
````
If your build environment cannot reach `gitea.dohertylan.com` directly,
configure `GOPROXY` to point at an internal proxy that fronts the Gitea
repo, or use `GONOSUMCHECK` + `GOPRIVATE` to bypass the checksum database
for the internal module path.
## Releasing a new version
Go modules in monorepo subdirectories use prefixed tags. To tag a release
from this repo:
````bash
pwsh scripts/tag-go-module.ps1 -Version v0.1.1 -Push
````
The script validates semver, refuses to tag with uncommitted tracked
changes, creates an annotated tag `clients/go/v0.1.1`, and (with `-Push`)
pushes it to origin.
## Related Documentation
- [Client Packaging](../../docs/ClientPackaging.md)
+113 -23
View File
@@ -18,6 +18,11 @@ import (
// browseChildrenPageSize is the per-request page size used by the lazy walker.
const browseChildrenPageSize = 500
// discoverHierarchyPageSize is the per-request page size used by DiscoverHierarchy.
// Mirrors the .NET client constant so large galaxies are not silently truncated
// by the server's default page cap.
const discoverHierarchyPageSize = 5000
// RawGalaxyRepositoryClient is the generated gRPC client interface for the
// Galaxy Repository service exposed for callers that need direct contract
// access.
@@ -155,16 +160,35 @@ func (c *GalaxyClient) GetLastDeployTime(ctx context.Context) (time.Time, bool,
// DiscoverHierarchy returns the deployed Galaxy object hierarchy with each
// object's dynamic attributes. The objects are returned in the order supplied
// by the server.
// by the server. The call pages over the server's NextPageToken until the
// server signals it has no more results, matching the .NET client.
func (c *GalaxyClient) DiscoverHierarchy(ctx context.Context) ([]*GalaxyObject, error) {
callCtx, cancel := c.callContext(ctx)
defer cancel()
reply, err := c.raw.DiscoverHierarchy(callCtx, &pb.DiscoverHierarchyRequest{})
if err != nil {
return nil, &GatewayError{Op: "galaxy discover hierarchy", Err: err}
var objects []*GalaxyObject
pageToken := ""
seen := map[string]struct{}{}
for {
callCtx, cancel := c.callContext(ctx)
reply, err := c.raw.DiscoverHierarchy(callCtx, &pb.DiscoverHierarchyRequest{
PageSize: discoverHierarchyPageSize,
PageToken: pageToken,
})
cancel()
if err != nil {
return nil, &GatewayError{Op: "galaxy discover hierarchy", Err: err}
}
objects = append(objects, reply.GetObjects()...)
pageToken = reply.GetNextPageToken()
if pageToken == "" {
return objects, nil
}
if _, dup := seen[pageToken]; dup {
return nil, &GatewayError{
Op: "galaxy discover hierarchy",
Err: fmt.Errorf("repeated page token %q", pageToken),
}
}
seen[pageToken] = struct{}{}
}
return reply.GetObjects(), nil
}
// WatchDeployEventsRaw starts the generated WatchDeployEvents stream for callers
@@ -249,15 +273,25 @@ func (c *GalaxyClient) Close() error {
// LazyBrowseNode is one node in a lazy Galaxy hierarchy walk produced by
// (*GalaxyClient).Browse. Children are not fetched until Expand is called.
// The node is safe for concurrent use; concurrent Expand calls collapse to a
// single RPC.
// The node is safe for concurrent use; concurrent Expand calls coalesce onto
// a single in-flight RPC and do not block snapshot accessors.
type LazyBrowseNode struct {
client *GalaxyClient
object *pb.GalaxyObject
hasChildrenHint bool
options BrowseChildrenOptions
mu sync.Mutex
// expandLock gates inspection and mutation of expand-coordination state
// (expanding, expandDone, expandErr). It is held only briefly; the BrowseChildren
// RPC itself runs outside this lock so concurrent readers and waiters are not blocked.
expandLock sync.Mutex
expanding bool
expandDone chan struct{}
expandErr error
// mu protects the children snapshot and isExpanded flag for concurrent
// Children() / IsExpanded() readers.
mu sync.RWMutex
children []*LazyBrowseNode
isExpanded bool
}
@@ -272,8 +306,8 @@ func (n *LazyBrowseNode) HasChildrenHint() bool { return n.hasChildrenHint }
// Children returns a snapshot copy of the currently-loaded child nodes. Returns
// an empty slice when Expand has not yet been called.
func (n *LazyBrowseNode) Children() []*LazyBrowseNode {
n.mu.Lock()
defer n.mu.Unlock()
n.mu.RLock()
defer n.mu.RUnlock()
out := make([]*LazyBrowseNode, len(n.children))
copy(out, n.children)
return out
@@ -281,28 +315,81 @@ func (n *LazyBrowseNode) Children() []*LazyBrowseNode {
// IsExpanded reports whether Expand has completed successfully on this node.
func (n *LazyBrowseNode) IsExpanded() bool {
n.mu.Lock()
defer n.mu.Unlock()
n.mu.RLock()
defer n.mu.RUnlock()
return n.isExpanded
}
// Expand fetches this node's direct children via BrowseChildren when they have
// not yet been loaded. Subsequent calls after a successful Expand are a no-op
// and do not issue another RPC.
//
// Expand is safe to call concurrently from multiple goroutines: callers that
// arrive while an expansion is in flight wait on the active RPC and share its
// result instead of issuing a second RPC. The RPC itself runs without holding
// the snapshot mutex, so concurrent Children() and IsExpanded() callers are
// not blocked for the duration of the network round trip.
//
// Failure semantics: a failed expansion surfaces the same error to every
// in-flight waiter, but the node is left in its pre-call state (isExpanded =
// false, no in-flight expansion). The next Expand call therefore retries with
// a fresh RPC; failures are not sticky.
func (n *LazyBrowseNode) Expand(ctx context.Context) error {
n.mu.Lock()
defer n.mu.Unlock()
// Fast path: already expanded.
n.mu.RLock()
if n.isExpanded {
n.mu.RUnlock()
return nil
}
n.mu.RUnlock()
// Either start a new expansion or wait on an existing one.
n.expandLock.Lock()
n.mu.RLock()
alreadyExpanded := n.isExpanded
n.mu.RUnlock()
if alreadyExpanded {
n.expandLock.Unlock()
return nil
}
if n.expanding {
done := n.expandDone
n.expandLock.Unlock()
select {
case <-done:
n.expandLock.Lock()
err := n.expandErr
n.expandLock.Unlock()
return err
case <-ctx.Done():
return ctx.Err()
}
}
n.expanding = true
n.expandDone = make(chan struct{})
done := n.expandDone
n.expandLock.Unlock()
// Issue the RPC outside any lock so concurrent readers/waiters are not blocked.
parentID := n.object.GetGobjectId()
children, err := n.client.browseChildrenInner(ctx, &parentID, n.options)
if err != nil {
return err
if err == nil {
n.mu.Lock()
n.children = children
n.isExpanded = true
n.mu.Unlock()
}
n.children = children
n.isExpanded = true
return nil
// Publish result to waiters and clear the in-flight marker so a failed
// expansion can be retried by the next Expand call.
n.expandLock.Lock()
n.expandErr = err
n.expanding = false
close(done)
n.expandLock.Unlock()
return err
}
// Browse returns the root nodes of the Galaxy hierarchy. The returned nodes
@@ -375,7 +462,10 @@ func (c *GalaxyClient) browseChildrenInner(
return nodes, nil
}
if _, dup := seen[pageToken]; dup {
return nil, fmt.Errorf("mxgateway: galaxy browse children returned repeated page token %q", pageToken)
return nil, &GatewayError{
Op: "galaxy browse children",
Err: fmt.Errorf("repeated page token %q", pageToken),
}
}
seen[pageToken] = struct{}{}
}
+136 -12
View File
@@ -4,6 +4,7 @@ import (
"context"
"errors"
"net"
"sync"
"testing"
"time"
@@ -146,6 +147,47 @@ func TestGalaxyDiscoverHierarchyReturnsObjects(t *testing.T) {
}
}
func TestGalaxyDiscoverHierarchyPaginatesAcrossMultiplePages(t *testing.T) {
page1 := &pb.DiscoverHierarchyReply{
Objects: []*pb.GalaxyObject{
{GobjectId: 1, TagName: "A"},
{GobjectId: 2, TagName: "B"},
},
NextPageToken: "page-2",
TotalObjectCount: 3,
}
page2 := &pb.DiscoverHierarchyReply{
Objects: []*pb.GalaxyObject{
{GobjectId: 3, TagName: "C"},
},
TotalObjectCount: 3,
}
fake := &fakeGalaxyServer{
discoverHierarchyReplies: []*pb.DiscoverHierarchyReply{page1, page2},
}
client, cleanup := newGalaxyBufconnClient(t, fake)
defer cleanup()
objs, err := client.DiscoverHierarchy(context.Background())
if err != nil {
t.Fatalf("DiscoverHierarchy: %v", err)
}
if got, want := len(objs), 3; got != want {
t.Fatalf("len(objs) = %d, want %d", got, want)
}
if len(fake.discoverHierarchyCalls) != 2 {
t.Fatalf("expected 2 RPC calls, got %d", len(fake.discoverHierarchyCalls))
}
if fake.discoverHierarchyCalls[0].GetPageSize() != discoverHierarchyPageSize {
t.Fatalf("first call PageSize = %d, want %d",
fake.discoverHierarchyCalls[0].GetPageSize(), discoverHierarchyPageSize)
}
if fake.discoverHierarchyCalls[1].GetPageToken() != "page-2" {
t.Fatalf("second call page token = %q, want %q",
fake.discoverHierarchyCalls[1].GetPageToken(), "page-2")
}
}
func TestGalaxyDialReturnsGatewayErrorOnRpcFailure(t *testing.T) {
fake := &fakeGalaxyServer{failTest: true}
client, cleanup := newGalaxyBufconnClient(t, fake)
@@ -372,18 +414,20 @@ func newGalaxyBufconnClient(t *testing.T, fake *fakeGalaxyServer) (*GalaxyClient
type fakeGalaxyServer struct {
pb.UnimplementedGalaxyRepositoryServer
testReply *pb.TestConnectionReply
testAuth string
failTest bool
deployReply *pb.GetLastDeployTimeReply
discoverReply *pb.DiscoverHierarchyReply
watchEvents []*pb.DeployEvent
watchRequest *pb.WatchDeployEventsRequest
watchSendInterval time.Duration
watchHoldOpen bool
browseChildrenCalls []*pb.BrowseChildrenRequest
browseChildrenReplies []*pb.BrowseChildrenReply
browseChildrenError error
testReply *pb.TestConnectionReply
testAuth string
failTest bool
deployReply *pb.GetLastDeployTimeReply
discoverReply *pb.DiscoverHierarchyReply
discoverHierarchyCalls []*pb.DiscoverHierarchyRequest
discoverHierarchyReplies []*pb.DiscoverHierarchyReply
watchEvents []*pb.DeployEvent
watchRequest *pb.WatchDeployEventsRequest
watchSendInterval time.Duration
watchHoldOpen bool
browseChildrenCalls []*pb.BrowseChildrenRequest
browseChildrenReplies []*pb.BrowseChildrenReply
browseChildrenError error
}
func (s *fakeGalaxyServer) TestConnection(ctx context.Context, req *pb.TestConnectionRequest) (*pb.TestConnectionReply, error) {
@@ -405,6 +449,12 @@ func (s *fakeGalaxyServer) GetLastDeployTime(ctx context.Context, req *pb.GetLas
}
func (s *fakeGalaxyServer) DiscoverHierarchy(ctx context.Context, req *pb.DiscoverHierarchyRequest) (*pb.DiscoverHierarchyReply, error) {
s.discoverHierarchyCalls = append(s.discoverHierarchyCalls, req)
if len(s.discoverHierarchyReplies) > 0 {
reply := s.discoverHierarchyReplies[0]
s.discoverHierarchyReplies = s.discoverHierarchyReplies[1:]
return reply, nil
}
if s.discoverReply != nil {
return s.discoverReply, nil
}
@@ -738,3 +788,77 @@ func TestGalaxyBrowseWithFilterForwardsToRequest(t *testing.T) {
t.Fatal("HistorizedOnly = false, want true")
}
}
func TestGalaxyBrowseExpandConcurrentCallersOnlyFireOneRpc(t *testing.T) {
fake := &fakeGalaxyServer{
browseChildrenReplies: []*pb.BrowseChildrenReply{
// roots
buildBrowseReply([]*pb.GalaxyObject{obj(1, "Plant", true)}, []bool{true}, 7),
// one expand: one child
buildBrowseReply([]*pb.GalaxyObject{obj(2, "Mixer", false)}, []bool{false}, 7),
},
}
client, cleanup := newGalaxyBufconnClient(t, fake)
defer cleanup()
ctx := context.Background()
roots, err := client.Browse(ctx, nil)
if err != nil {
t.Fatalf("Browse: %v", err)
}
var wg sync.WaitGroup
errs := make(chan error, 10)
for i := 0; i < 10; i++ {
wg.Add(1)
go func() {
defer wg.Done()
errs <- roots[0].Expand(ctx)
}()
}
wg.Wait()
close(errs)
for err := range errs {
if err != nil {
t.Fatalf("concurrent Expand: %v", err)
}
}
if !roots[0].IsExpanded() {
t.Fatal("IsExpanded() = false after 10 concurrent expands")
}
if got, want := len(roots[0].Children()), 1; got != want {
t.Fatalf("len(children) = %d, want %d", got, want)
}
// 1 roots fetch + exactly 1 expand fetch.
if got, want := len(fake.browseChildrenCalls), 2; got != want {
t.Fatalf("RPC count = %d, want %d", got, want)
}
}
func TestGalaxyBrowseChildrenRejectsRepeatedPageToken(t *testing.T) {
// Build a reply that carries a non-empty NextPageToken so browseChildrenInner
// will request a second page. Queue the same reply twice so the second response
// returns the same page token, triggering the duplicate-token guard.
page := buildBrowseReply(
[]*pb.GalaxyObject{obj(1, "Plant", true)},
[]bool{true},
1,
)
page.NextPageToken = "1:abc:1"
fake := &fakeGalaxyServer{
browseChildrenReplies: []*pb.BrowseChildrenReply{page, page},
}
client, cleanup := newGalaxyBufconnClient(t, fake)
defer cleanup()
_, err := client.Browse(context.Background(), nil)
if err == nil {
t.Fatal("Browse: error = nil, want repeated-page-token error")
}
var gwErr *GatewayError
if !errors.As(err, &gwErr) {
t.Fatalf("error type = %T, want *GatewayError; err = %v", err, err)
}
}
+61
View File
@@ -139,6 +139,36 @@ for (int i = 0; i < children.size(); i++) {
}
```
#### High-level walker
For UI trees, the client provides a `LazyBrowseNode` walker that handles
sibling pagination and the `child_has_children` hint for you:
```java
MxGatewayClientOptions options = MxGatewayClientOptions.builder()
.endpoint("localhost:5000")
.apiKey(System.getenv("MXGATEWAY_API_KEY"))
.plaintext(true)
.build();
try (GalaxyRepositoryClient galaxy = GalaxyRepositoryClient.connect(options)) {
List<LazyBrowseNode> roots = galaxy.browse();
for (LazyBrowseNode root : roots) {
if (root.hasChildrenHint()) {
root.expand();
}
for (LazyBrowseNode child : root.getChildren()) {
String kind = child.hasChildrenHint() ? "has children" : "leaf";
System.out.println(child.getObject().getTagName() + " (" + kind + ")");
}
}
}
```
`expand` is idempotent — calling it twice fires only one RPC,
and is safe under concurrent callers. To refresh after a Galaxy redeploy, call
`browse` again from the root.
### Watching deploy events
`GalaxyRepository.WatchDeployEvents` is a server-streaming RPC: the gateway
@@ -252,6 +282,37 @@ $env:MXGATEWAY_TEST_ITEM = 'TestObject.TestInt'
gradle :zb-mom-ww-mxgateway-cli:run --args="smoke --endpoint $env:MXGATEWAY_ENDPOINT --plaintext --api-key-env MXGATEWAY_API_KEY --item $env:MXGATEWAY_TEST_ITEM --json"
```
## Installing from the Gitea Maven repository
The client publishes to the internal Gitea Maven repository at
`https://gitea.dohertylan.com/api/packages/dohertj2/maven`.
In your consumer project's `build.gradle`:
````groovy
repositories {
maven {
url 'https://gitea.dohertylan.com/api/packages/dohertj2/maven'
credentials {
username = System.getenv('GITEA_USERNAME')
password = System.getenv('GITEA_TOKEN')
}
}
}
dependencies {
implementation 'com.zb.mom.ww.mxgateway:zb-mom-ww-mxgateway-client:0.1.0'
}
````
To publish a new version from this repo:
````bash
export GITEA_USERNAME=dohertj2
export GITEA_TOKEN=<your-gitea-token>
gradle :zb-mom-ww-mxgateway-client:publish
````
## Related Documentation
- [Client Packaging](../../docs/ClientPackaging.md)
+40
View File
@@ -37,4 +37,44 @@ subprojects {
testRuntimeOnly 'org.junit.platform:junit-platform-launcher'
}
}
pluginManager.withPlugin('maven-publish') {
publishing {
publications {
maven(MavenPublication) {
from components.java
pom {
url = 'https://gitea.dohertylan.com/dohertj2/mxaccessgw'
description = 'MxAccessGateway Java client'
scm {
url = 'https://gitea.dohertylan.com/dohertj2/mxaccessgw'
connection = 'scm:git:https://gitea.dohertylan.com/dohertj2/mxaccessgw.git'
}
developers {
developer {
id = 'dohertj2'
name = 'Joseph Doherty'
}
}
licenses {
license {
name = 'Proprietary'
distribution = 'repo'
}
}
}
}
}
repositories {
maven {
name = 'GiteaPackages'
url = 'https://gitea.dohertylan.com/api/packages/dohertj2/maven'
credentials {
username = System.getenv('GITEA_USERNAME') ?: ''
password = System.getenv('GITEA_TOKEN') ?: ''
}
}
}
}
}
}
@@ -1,6 +1,7 @@
plugins {
id 'java-library'
id 'com.google.protobuf'
id 'maven-publish'
}
dependencies {
@@ -30,6 +31,11 @@ sourceSets {
}
}
java {
withSourcesJar()
withJavadocJar()
}
protobuf {
protoc {
artifact = "com.google.protobuf:protoc:${protobufVersion}"
@@ -4,6 +4,9 @@ import galaxy_repository.v1.GalaxyRepositoryOuterClass.GalaxyObject;
import java.util.Collections;
import java.util.List;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.locks.ReentrantReadWriteLock;
/**
* One node in a lazy-loaded Galaxy browse tree. Holds the underlying
@@ -16,7 +19,14 @@ public final class LazyBrowseNode {
private final GalaxyObject object;
private final boolean hasChildrenHint;
private final BrowseChildrenOptions options;
private final Object lock = new Object();
// expandLock gates the start of a new expand AND the publish of the in-flight
// future. Readers (getChildren / isExpanded) use a separate read-write lock so
// they never block on the gRPC call.
private final Object expandLock = new Object();
private CompletableFuture<Void> inFlight;
private final ReentrantReadWriteLock readWriteLock = new ReentrantReadWriteLock();
private List<LazyBrowseNode> children = Collections.emptyList();
private boolean isExpanded;
@@ -43,15 +53,21 @@ public final class LazyBrowseNode {
/** @return a snapshot of direct children loaded by {@link #expand()}; empty until then. */
public List<LazyBrowseNode> getChildren() {
synchronized (lock) {
readWriteLock.readLock().lock();
try {
return List.copyOf(children);
} finally {
readWriteLock.readLock().unlock();
}
}
/** @return {@code true} after the first {@link #expand()} call completes. */
public boolean isExpanded() {
synchronized (lock) {
readWriteLock.readLock().lock();
try {
return isExpanded;
} finally {
readWriteLock.readLock().unlock();
}
}
@@ -59,17 +75,76 @@ public final class LazyBrowseNode {
* Fetches direct children from the gateway and populates {@link #getChildren()}.
* Idempotent: subsequent calls are no-ops and do not issue a second RPC.
*
* <p>Concurrent callers coalesce onto a single in-flight RPC: the first caller
* (the "leader") issues the gRPC call, while any other thread that calls
* {@code expand()} during that window blocks on the leader's future and sees
* the same result (or the same exception). On failure the in-flight slot is
* cleared so a subsequent call can retry.
*
* <p>Readers ({@link #getChildren()} / {@link #isExpanded()}) take a separate
* read lock and are never blocked for the duration of the RPC.
*
* @throws MxGatewayException on transport or protocol failure
*/
public void expand() {
synchronized (lock) {
if (isExpanded) {
if (isExpanded()) {
return;
}
CompletableFuture<Void> future;
boolean iAmTheLeader;
synchronized (expandLock) {
if (isExpanded()) {
return;
}
List<LazyBrowseNode> loaded =
client.browseChildrenInner(Integer.valueOf(object.getGobjectId()), options);
this.children = loaded;
this.isExpanded = true;
if (inFlight != null) {
future = inFlight;
iAmTheLeader = false;
} else {
future = new CompletableFuture<>();
inFlight = future;
iAmTheLeader = true;
}
}
if (iAmTheLeader) {
try {
List<LazyBrowseNode> loaded =
client.browseChildrenInner(object.getGobjectId(), options);
readWriteLock.writeLock().lock();
try {
this.children = loaded;
this.isExpanded = true;
} finally {
readWriteLock.writeLock().unlock();
}
synchronized (expandLock) {
inFlight = null;
}
future.complete(null);
} catch (RuntimeException ex) {
synchronized (expandLock) {
inFlight = null;
}
future.completeExceptionally(ex);
throw ex;
}
} else {
try {
future.get();
} catch (InterruptedException ie) {
Thread.currentThread().interrupt();
throw new MxGatewayException("Interrupted waiting for browse-children expand.", ie);
} catch (ExecutionException ee) {
Throwable cause = ee.getCause();
if (cause instanceof MxGatewayException me) {
throw me;
}
if (cause instanceof RuntimeException re) {
throw re;
}
throw new MxGatewayException("BrowseChildren expand failed.", cause);
}
}
}
}
@@ -40,9 +40,14 @@ import java.util.List;
import java.util.Optional;
import java.util.Queue;
import java.util.UUID;
import java.util.ArrayList;
import java.util.concurrent.CopyOnWriteArrayList;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicReference;
import org.junit.jupiter.api.Test;
@@ -203,6 +208,27 @@ final class GalaxyRepositoryClientTests {
}
}
@Test
void browseChildrenRejectsRepeatedPageToken() throws Exception {
// Queue the same BrowseChildrenReply twice with a non-empty NextPageToken.
// The client will request a second page and detect that the token repeats.
BrowseChildrenService service = new BrowseChildrenService();
BrowseChildrenReply repeatedReply = browseReply(
List.of(obj(1, "Plant", true)),
List.of(true),
1L,
"1:abc:1");
service.replies.add(repeatedReply);
service.replies.add(repeatedReply);
try (InProcessGalaxy g = InProcessGalaxy.start(service, new AtomicReference<>());
GalaxyRepositoryClient client = g.client("")) {
MxGatewayException error = assertThrows(MxGatewayException.class, client::browse);
assertTrue(error.getMessage().contains("repeated page token"));
}
}
@Test
void watchDeployEventsReceivesEventsInOrder() throws Exception {
DeployEvent first = DeployEvent.newBuilder()
@@ -445,6 +471,91 @@ final class GalaxyRepositoryClientTests {
}
}
@Test
void browseExpandConcurrentCallersOnlyFireOneRpc() throws Exception {
// Verifies that concurrent expand() calls coalesce onto a single in-flight
// BrowseChildren RPC and that readers (isExpanded/getChildren) are not
// blocked for the full RPC duration.
BrowseChildrenReply rootsReply = browseReply(
List.of(obj(1, "Plant", true)),
List.of(true),
7L,
"");
BrowseChildrenReply childrenReply = browseReply(
List.of(obj(2, "Mixer_001", false)),
List.of(false),
7L,
"");
// Gate the child fetch behind a latch so multiple expanders can pile up.
CountDownLatch release = new CountDownLatch(1);
AtomicInteger childCalls = new AtomicInteger();
BrowseChildrenService service = new BrowseChildrenService() {
@Override
public void browseChildren(
BrowseChildrenRequest request, StreamObserver<BrowseChildrenReply> responseObserver) {
calls.add(request);
BrowseChildrenReply reply;
if (!request.hasParentGobjectId()) {
reply = rootsReply;
} else {
// Block the leader until the followers have arrived.
try {
assertTrue(release.await(5, TimeUnit.SECONDS), "release latch never tripped");
} catch (InterruptedException ie) {
Thread.currentThread().interrupt();
responseObserver.onError(Status.CANCELLED.asRuntimeException());
return;
}
childCalls.incrementAndGet();
reply = childrenReply;
}
responseObserver.onNext(reply);
responseObserver.onCompleted();
}
};
try (InProcessGalaxy g = InProcessGalaxy.start(service, new AtomicReference<>());
GalaxyRepositoryClient client = g.client("")) {
List<LazyBrowseNode> roots = client.browse();
LazyBrowseNode root = roots.get(0);
int parallelism = 10;
ExecutorService pool = Executors.newFixedThreadPool(parallelism);
try {
CountDownLatch ready = new CountDownLatch(parallelism);
List<Future<Void>> futures = new ArrayList<>();
for (int i = 0; i < parallelism; i++) {
futures.add(pool.submit(() -> {
ready.countDown();
root.expand();
return null;
}));
}
// Wait for all callers to be in flight, then release the leader.
assertTrue(ready.await(5, TimeUnit.SECONDS), "expander threads did not start");
// Readers must not be blocked by an in-flight expand; this should not deadlock
// and should return the pre-expand state.
assertFalse(root.isExpanded());
assertEquals(0, root.getChildren().size());
release.countDown();
for (Future<Void> f : futures) {
f.get(10, TimeUnit.SECONDS);
}
} finally {
pool.shutdownNow();
}
assertTrue(root.isExpanded());
assertEquals(1, root.getChildren().size());
// Exactly one expand RPC was issued even though many callers raced.
assertEquals(1, childCalls.get());
// 1 roots fetch + exactly 1 expand fetch.
assertEquals(2, service.calls.size());
}
}
@Test
void browseWithFilterForwardsToRequest() throws Exception {
BrowseChildrenService service = new BrowseChildrenService();
@@ -486,7 +597,7 @@ final class GalaxyRepositoryClientTests {
return b.build();
}
private static final class BrowseChildrenService extends TestService {
private static class BrowseChildrenService extends TestService {
final List<BrowseChildrenRequest> calls =
Collections.synchronizedList(new CopyOnWriteArrayList<>());
final Queue<BrowseChildrenReply> replies = new ArrayDeque<>();
+37
View File
@@ -157,6 +157,30 @@ for child, has_children in zip(reply.children, reply.child_has_children):
print(child.tag_name, "expand=" + str(has_children))
```
#### High-level walker
For UI trees, the client provides a `LazyBrowseNode` walker that handles
sibling pagination and the `child_has_children` hint for you:
```python
async with await GalaxyRepositoryClient.connect(
endpoint="localhost:5000",
api_key="<gateway-api-key>",
plaintext=True,
) as galaxy:
roots = await galaxy.browse()
for root in roots:
if root.has_children_hint:
await root.expand()
for child in root.children:
kind = "has children" if child.has_children_hint else "leaf"
print(f"{child.object.tag_name} ({kind})")
```
`expand` is idempotent — calling it twice fires only one RPC,
and is safe under concurrent callers. To refresh after a Galaxy redeploy, call
`browse` again from the root.
### Watching deploy events
`GalaxyRepositoryClient.watch_deploy_events` opens a server-streaming
@@ -244,6 +268,19 @@ $env:MXGATEWAY_TEST_ITEM = 'Object.Attribute'
mxgw-py smoke --endpoint $env:MXGATEWAY_ENDPOINT --plaintext --api-key-env MXGATEWAY_API_KEY --item $env:MXGATEWAY_TEST_ITEM --json
```
## Installing from the Gitea PyPI Feed
The client publishes to the internal Gitea PyPI feed:
````bash
pip install \
--index-url https://gitea.dohertylan.com/api/packages/dohertj2/pypi/simple/ \
zb-mom-ww-mxaccess-gateway-client
````
If you need authentication (private feed), use `--extra-index-url` and either
a `~/.netrc` entry or `PIP_INDEX_URL=https://<user>:<token>@gitea.dohertylan.com/...`.
## Related Documentation
- [Client Packaging](../../docs/ClientPackaging.md)
+23
View File
@@ -13,12 +13,35 @@ dependencies = [
"grpcio>=1.80,<2",
"protobuf>=6.33,<7",
]
authors = [
{ name = "Joseph Doherty" },
]
license = { text = "Proprietary" }
keywords = ["mxaccess", "mxgateway", "grpc", "client", "archestra"]
classifiers = [
"Development Status :: 3 - Alpha",
"License :: Other/Proprietary License",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Topic :: System :: Distributed Computing",
"Topic :: Software Development :: Libraries :: Python Modules",
"Intended Audience :: Developers",
"Operating System :: OS Independent",
]
[project.urls]
Homepage = "https://gitea.dohertylan.com/dohertj2/mxaccessgw"
Repository = "https://gitea.dohertylan.com/dohertj2/mxaccessgw"
Issues = "https://gitea.dohertylan.com/dohertj2/mxaccessgw/issues"
[project.optional-dependencies]
dev = [
"grpcio-tools>=1.80,<2",
"pytest>=9,<10",
"pytest-asyncio>=1.3,<2",
"build>=1.2,<2",
"twine>=5,<6",
]
[project.scripts]
@@ -140,6 +140,22 @@ class GalaxyRepositoryClient:
)
seen_page_tokens.add(page_token)
async def browse_children_raw(
self, request: galaxy_pb.BrowseChildrenRequest
) -> galaxy_pb.BrowseChildrenReply:
"""Issue one BrowseChildren RPC and return the raw reply.
Lower-level escape hatch for callers that need direct page-token control
or do not want LazyBrowseNode wrapping. Most callers should use
:py:meth:`browse` and :py:meth:`LazyBrowseNode.expand` instead.
"""
return await self._unary(
"browse children",
self.raw_stub.BrowseChildren,
request,
)
async def browse(
self,
options: BrowseChildrenOptions | None = None,
@@ -292,6 +308,7 @@ class LazyBrowseNode:
self._options = options
self._children: list[LazyBrowseNode] = []
self._is_expanded = False
self._expand_lock = asyncio.Lock()
@property
def object(self) -> galaxy_pb.GalaxyObject:
@@ -317,14 +334,17 @@ class LazyBrowseNode:
"""Fetch direct children of this node; no-op on subsequent calls."""
if self._is_expanded:
return
new_children: list[LazyBrowseNode] = []
async for child in self._client._iter_browse_children(
parent_gobject_id=self._object.gobject_id,
options=self._options,
):
new_children.append(child)
self._children.extend(new_children)
self._is_expanded = True
async with self._expand_lock:
if self._is_expanded:
return
new_children: list[LazyBrowseNode] = []
async for child in self._client._iter_browse_children(
parent_gobject_id=self._object.gobject_id,
options=self._options,
):
new_children.append(child)
self._children.extend(new_children)
self._is_expanded = True
async def _canceling_iterator(call: Any) -> AsyncIterator[galaxy_pb.DeployEvent]:
+53 -1
View File
@@ -391,6 +391,28 @@ async def test_browse_expand_idempotent_no_second_rpc() -> None:
assert len(roots[0].children) == 1
@pytest.mark.asyncio
async def test_browse_expand_concurrent_callers_only_fire_one_rpc() -> None:
stub = FakeGalaxyStub()
stub.browse_children.replies = [
_build_browse_reply([_obj(1, "Plant", is_area=True)], [True], 7),
_build_browse_reply([_obj(2, "Mixer_001")], [False], 7),
]
client = await GalaxyRepositoryClient.connect(
ClientOptions(endpoint="fake", plaintext=True),
stub=stub,
)
roots = await client.browse()
# Ten concurrent expand calls on the same node should issue exactly one RPC.
await asyncio.gather(*(roots[0].expand() for _ in range(10)))
assert roots[0].is_expanded
assert len(roots[0].children) == 1
# 1 roots fetch + exactly 1 expand fetch = 2 total
assert len(stub.browse_children.requests) == 2
@pytest.mark.asyncio
async def test_browse_expand_unknown_parent_raises_mxgateway_error() -> None:
stub = FakeGalaxyStub()
@@ -485,6 +507,35 @@ async def test_browse_with_filter_forwards_to_request() -> None:
assert request.historized_only is True
@pytest.mark.asyncio
async def test_browse_children_raw_returns_reply_unwrapped() -> None:
"""browse_children_raw forwards the request to the stub and returns the raw reply."""
stub = FakeGalaxyStub()
expected = _build_browse_reply(
children=[_obj(1, "Plant", is_area=True)],
child_has_children=[True],
cache_sequence=42,
)
stub.browse_children.replies = [expected]
async with await GalaxyRepositoryClient.connect(
endpoint="fake",
plaintext=True,
stub=stub,
) as client:
request = galaxy_pb.BrowseChildrenRequest(
page_size=10,
tag_name_glob="Plant*",
)
reply = await client.browse_children_raw(request)
assert reply.cache_sequence == 42
assert len(reply.children) == 1
assert reply.children[0].tag_name == "Plant"
assert len(stub.browse_children.requests) == 1
assert stub.browse_children.requests[0].tag_name_glob == "Plant*"
class FakeGalaxyStub:
def __init__(self) -> None:
self.test_connection = FakeUnary([galaxy_pb.TestConnectionReply(ok=False)])
@@ -506,7 +557,8 @@ class FakeUnary:
def __init__(self, replies: list[Any]) -> None:
self.replies = replies
self.requests: list[Any] = []
self.exceptions: list[BaseException] = []
# None entries mean "no exception on this call"; aligns with the replies queue index-by-index.
self.exceptions: list[BaseException | None] = []
self.metadata: tuple[tuple[str, str], ...] | None = None
async def __call__(
+3
View File
@@ -17,3 +17,6 @@
# args through the GNU linker and reject `/STACK:`, are unaffected.
[target.'cfg(all(windows, target_env = "msvc"))']
rustflags = ["-C", "link-arg=/STACK:8388608"]
[registries.dohertj2-gitea]
index = "sparse+https://gitea.dohertylan.com/api/packages/dohertj2/cargo/"
+14 -2
View File
@@ -2,7 +2,16 @@
name = "zb-mom-ww-mxgateway-client"
version = "0.1.0"
edition = "2021"
publish = false
authors = ["Joseph Doherty"]
description = "Async Rust client for the MxAccessGateway gRPC service, including a lazy-browse walker over the Galaxy Repository hierarchy."
license = "Proprietary"
repository = "https://gitea.dohertylan.com/dohertj2/mxaccessgw"
homepage = "https://gitea.dohertylan.com/dohertj2/mxaccessgw"
documentation = "https://gitea.dohertylan.com/dohertj2/mxaccessgw"
readme = "README.md"
keywords = ["mxaccess", "mxgateway", "grpc", "client", "archestra"]
categories = ["api-bindings", "asynchronous"]
publish = ["dohertj2-gitea"]
build = "build.rs"
[workspace]
@@ -12,7 +21,10 @@ resolver = "2"
[workspace.package]
edition = "2021"
version = "0.1.0"
publish = false
authors = ["Joseph Doherty"]
license = "Proprietary"
repository = "https://gitea.dohertylan.com/dohertj2/mxaccessgw"
publish = ["dohertj2-gitea"]
[workspace.dependencies]
clap = { version = "4.5.53", features = ["derive"] }
+49
View File
@@ -157,6 +157,31 @@ for (child, has_children) in reply.children.iter().zip(reply.child_has_children.
}
```
#### High-level walker
For UI trees, the client provides a `LazyBrowseNode` walker that handles
sibling pagination and the `child_has_children` hint for you:
```rust
let mut client = GalaxyClient::connect(
ClientOptions::new("http://localhost:5000").with_api_key(ApiKey::new(api_key)),
).await?;
let roots = client.browse(None).await?;
for root in &roots {
if root.has_children_hint() {
root.expand().await?;
}
for child in root.children().await {
let kind = if child.has_children_hint() { "has children" } else { "leaf" };
println!("{} ({kind})", child.object().tag_name);
}
}
```
`expand` is idempotent — calling it twice fires only one RPC,
and is safe under concurrent callers. To refresh after a Galaxy redeploy, call
`browse` again from the root.
### Watching deploy events
`watch_deploy_events` opens the `WatchDeployEvents` server stream. The
@@ -211,3 +236,27 @@ cargo run -p mxgw-cli -- smoke --endpoint $env:MXGATEWAY_ENDPOINT --plaintext --
- [Client Proto Generation](../../docs/ClientProtoGeneration.md)
- [Rust Client Detailed Design](./RustClientDesign.md)
- [Rust Style Guide](../../docs/style-guides/RustStyleGuide.md)
## Installing from the Gitea Cargo registry
The crate publishes to the internal Gitea Cargo registry. Register the
registry once in your global `~/.cargo/config.toml`:
```toml
[registries.dohertj2-gitea]
index = "sparse+https://gitea.dohertylan.com/api/packages/dohertj2/cargo/"
```
Authentication: cargo reads credentials from `~/.cargo/credentials.toml`:
```toml
[registries.dohertj2-gitea]
token = "Bearer <your-gitea-token>"
```
Then add the dependency:
```toml
[dependencies]
zb-mom-ww-mxgateway-client = { version = "0.1.0", registry = "dohertj2-gitea" }
```
+1 -1
View File
@@ -2,7 +2,7 @@
name = "mxgw-cli"
version.workspace = true
edition.workspace = true
publish.workspace = true
publish = false
[[bin]]
name = "mxgw"
+312
View File
@@ -0,0 +1,312 @@
#Requires -Version 7
<#
.SYNOPSIS
Packs all MxAccessGateway clients into a single dist/ directory.
.DESCRIPTION
Runs each language client's native packaging command:
.NET -> dotnet pack (NuGet)
Python -> python -m build (sdist + wheel)
Rust -> cargo package (.crate)
Java -> gradle assemble + jars (jar + sources + javadoc + pom)
Go -> skipped; use scripts/tag-go-module.ps1
All artifacts land in -OutputDir (default: dist/).
With -Publish, each language pushes its package to the internal Gitea
feed. Requires GITEA_USERNAME and GITEA_TOKEN env vars.
.PARAMETER OutputDir
Where to drop the packed artifacts. Default: ./dist
.PARAMETER Languages
Subset of languages to pack. Default: all five.
Values: dotnet, python, rust, java, go
.PARAMETER Publish
After packing, upload to Gitea feeds. Requires:
GITEA_USERNAME
GITEA_TOKEN
Will refuse to publish if either is missing.
.PARAMETER SkipTests
Skip per-language regression tests before packing. Default: false.
.EXAMPLE
pwsh scripts/pack-clients.ps1
pwsh scripts/pack-clients.ps1 -Languages dotnet,python
pwsh scripts/pack-clients.ps1 -Publish
#>
[CmdletBinding()]
param(
[string]$OutputDir = (Join-Path $PSScriptRoot '..' 'dist'),
[string[]]$Languages = @('dotnet', 'python', 'rust', 'java', 'go'),
[switch]$Publish,
[switch]$SkipTests
)
Set-StrictMode -Version Latest
$ErrorActionPreference = 'Stop'
# Normalize comma-separated strings that shells may pass as a single element.
$validLanguages = @('dotnet', 'python', 'rust', 'java', 'go')
$Languages = @($Languages | ForEach-Object { $_ -split ',' } | ForEach-Object {
$_.Trim().ToLowerInvariant()
} | Where-Object { -not [string]::IsNullOrWhiteSpace($_) })
foreach ($lang in $Languages) {
if ($validLanguages -notcontains $lang) {
throw "Unsupported language '$lang'. Supported values: $($validLanguages -join ', ')."
}
}
if ($Languages.Count -eq 0) {
throw "At least one language is required. Supported values: $($validLanguages -join ', ')."
}
# Resolve absolute output dir
$OutputDir = [System.IO.Path]::GetFullPath($OutputDir)
$RepoRoot = [System.IO.Path]::GetFullPath((Join-Path $PSScriptRoot '..'))
if (-not (Test-Path $OutputDir)) {
New-Item -ItemType Directory -Path $OutputDir | Out-Null
}
if ($Publish) {
if ([string]::IsNullOrEmpty($env:GITEA_USERNAME)) {
throw 'Publish requires GITEA_USERNAME env var.'
}
if ([string]::IsNullOrEmpty($env:GITEA_TOKEN)) {
throw 'Publish requires GITEA_TOKEN env var.'
}
}
$GiteaNugetFeed = 'https://gitea.dohertylan.com/api/packages/dohertj2/nuget/index.json'
$GiteaPypiFeed = 'https://gitea.dohertylan.com/api/packages/dohertj2/pypi'
$JavaHome = '/Users/dohertj2/.local/jdks/jdk-21.0.11+10/Contents/Home'
function Write-Header {
param([string]$Text)
Write-Host ''
Write-Host '=== ' -NoNewline -ForegroundColor Cyan
Write-Host $Text -ForegroundColor Cyan
}
# -------- .NET --------
function Invoke-PackDotnet {
Write-Header '.NET'
if (-not $SkipTests) {
Write-Host 'Running .NET client tests...'
$testProject = Join-Path $RepoRoot 'clients/dotnet/ZB.MOM.WW.MxGateway.Client.Tests/ZB.MOM.WW.MxGateway.Client.Tests.csproj'
& dotnet test $testProject --no-restore
if ($LASTEXITCODE -ne 0) { throw '.NET tests failed.' }
}
Write-Host 'Packing ZB.MOM.WW.MxGateway.Contracts...'
& dotnet pack (Join-Path $RepoRoot 'src/ZB.MOM.WW.MxGateway.Contracts/ZB.MOM.WW.MxGateway.Contracts.csproj') `
-c Release -o $OutputDir
if ($LASTEXITCODE -ne 0) { throw '.NET Contracts pack failed.' }
Write-Host 'Packing ZB.MOM.WW.MxGateway.Client...'
& dotnet pack (Join-Path $RepoRoot 'clients/dotnet/ZB.MOM.WW.MxGateway.Client/ZB.MOM.WW.MxGateway.Client.csproj') `
-c Release -o $OutputDir
if ($LASTEXITCODE -ne 0) { throw '.NET Client pack failed.' }
Write-Host "Packed .NET artifacts -> $OutputDir" -ForegroundColor Green
if ($Publish) {
Write-Host 'Publishing .NET packages to Gitea...' -ForegroundColor Yellow
Get-ChildItem $OutputDir -Filter 'ZB.MOM.WW.MxGateway.*.nupkg' | ForEach-Object {
& dotnet nuget push $_.FullName --source $GiteaNugetFeed --api-key $env:GITEA_TOKEN
if ($LASTEXITCODE -ne 0) { throw "dotnet nuget push failed for '$($_.Name)'." }
}
}
}
# -------- Python --------
function Invoke-PackPython {
Write-Header 'Python'
# Use a persistent venv in /tmp so repeated runs skip reinstall.
$Venv = '/tmp/mxgw-py'
if (-not (Test-Path "$Venv/bin/python")) {
Write-Host "Creating Python venv at $Venv..."
& python3 -m venv $Venv
if ($LASTEXITCODE -ne 0) { throw 'python3 -m venv failed.' }
& "$Venv/bin/pip" install --quiet --upgrade pip
& "$Venv/bin/pip" install --quiet build twine
& "$Venv/bin/pip" install --quiet -e (Join-Path $RepoRoot 'clients/python[dev]')
}
if (-not $SkipTests) {
Write-Host 'Running Python tests...'
Push-Location (Join-Path $RepoRoot 'clients/python')
try {
& "$Venv/bin/python" -m pytest -q
if ($LASTEXITCODE -ne 0) { throw 'Python tests failed.' }
} finally { Pop-Location }
}
Write-Host 'Building Python sdist + wheel...'
& "$Venv/bin/python" -m build (Join-Path $RepoRoot 'clients/python') --outdir $OutputDir
if ($LASTEXITCODE -ne 0) { throw 'Python build failed.' }
Write-Host "Packed Python artifacts -> $OutputDir" -ForegroundColor Green
if ($Publish) {
Write-Host 'Publishing Python distribution to Gitea...' -ForegroundColor Yellow
$wheels = @(Get-ChildItem $OutputDir -Filter 'zb_mom_ww_mxaccess_gateway_client-*.whl')
$sdists = @(Get-ChildItem $OutputDir -Filter 'zb_mom_ww_mxaccess_gateway_client-*.tar.gz')
$files = ($wheels + $sdists) | ForEach-Object { $_.FullName }
& "$Venv/bin/python" -m twine upload `
--repository-url $GiteaPypiFeed `
-u $env:GITEA_USERNAME `
-p $env:GITEA_TOKEN `
@files
if ($LASTEXITCODE -ne 0) { throw 'twine upload failed.' }
}
}
# -------- Rust --------
function Invoke-PackRust {
Write-Header 'Rust'
$rustDir = Join-Path $RepoRoot 'clients/rust'
Push-Location $rustDir
try {
if (-not $SkipTests) {
Write-Host 'Running Rust tests...'
& cargo test --workspace
if ($LASTEXITCODE -ne 0) { throw 'Rust tests failed.' }
}
Write-Host 'Running cargo package...'
& cargo package --no-verify
if ($LASTEXITCODE -ne 0) { throw 'cargo package failed.' }
$packageDir = Join-Path $rustDir 'target/package'
$crates = @(Get-ChildItem $packageDir -Filter '*.crate')
if ($crates.Count -eq 0) {
throw 'cargo package produced no .crate files.'
}
foreach ($crate in $crates) {
Copy-Item $crate.FullName -Destination $OutputDir -Force
Write-Host " Copied $($crate.Name)"
}
} finally { Pop-Location }
Write-Host "Packed Rust artifacts -> $OutputDir" -ForegroundColor Green
if ($Publish) {
Write-Host 'Publishing Rust crate to Gitea...' -ForegroundColor Yellow
Push-Location (Join-Path $RepoRoot 'clients/rust')
try {
& cargo publish --no-verify --registry dohertj2-gitea
if ($LASTEXITCODE -ne 0) { throw 'cargo publish failed.' }
} finally { Pop-Location }
}
}
# -------- Java --------
function Invoke-PackJava {
Write-Header 'Java'
$env:JAVA_HOME = $JavaHome
$javaDir = Join-Path $RepoRoot 'clients/java'
Push-Location $javaDir
try {
if (-not $SkipTests) {
Write-Host 'Running Java tests...'
& gradle ':zb-mom-ww-mxgateway-client:test' --no-daemon
if ($LASTEXITCODE -ne 0) { throw 'Java tests failed.' }
}
Write-Host 'Assembling Java jars + pom...'
& gradle `
':zb-mom-ww-mxgateway-client:assemble' `
':zb-mom-ww-mxgateway-client:sourcesJar' `
':zb-mom-ww-mxgateway-client:javadocJar' `
':zb-mom-ww-mxgateway-client:generatePomFileForMavenPublication' `
--no-daemon
if ($LASTEXITCODE -ne 0) { throw 'Java assemble failed.' }
$libsDir = Join-Path $javaDir 'zb-mom-ww-mxgateway-client/build/libs'
$jars = @(Get-ChildItem $libsDir -Filter 'zb-mom-ww-mxgateway-client-*.jar')
if ($jars.Count -eq 0) {
throw "No jars found under '$libsDir'."
}
foreach ($jar in $jars) {
Copy-Item $jar.FullName -Destination $OutputDir -Force
Write-Host " Copied $($jar.Name)"
}
$pomSrc = Join-Path $javaDir 'zb-mom-ww-mxgateway-client/build/publications/maven/pom-default.xml'
if (Test-Path $pomSrc) {
# Derive the version from the jar filename (e.g. zb-mom-ww-mxgateway-client-0.1.0.jar).
$versionJar = $jars | Where-Object { $_.Name -notmatch '-(sources|javadoc)\.jar$' } | Select-Object -First 1
$version = if ($versionJar) {
[System.IO.Path]::GetFileNameWithoutExtension($versionJar.Name) -replace '^zb-mom-ww-mxgateway-client-', ''
} else {
'0.1.0'
}
$pomDest = Join-Path $OutputDir "zb-mom-ww-mxgateway-client-$version.pom"
Copy-Item $pomSrc -Destination $pomDest -Force
Write-Host " Copied pom -> $([System.IO.Path]::GetFileName($pomDest))"
} else {
Write-Warning "POM not found at '$pomSrc'; skipping."
}
} finally { Pop-Location }
Write-Host "Packed Java artifacts -> $OutputDir" -ForegroundColor Green
if ($Publish) {
Write-Host 'Publishing Java artifacts to Gitea Maven feed...' -ForegroundColor Yellow
Push-Location $javaDir
try {
& gradle ':zb-mom-ww-mxgateway-client:publish' --no-daemon
if ($LASTEXITCODE -ne 0) { throw 'gradle publish failed.' }
} finally { Pop-Location }
}
}
# -------- Go --------
function Invoke-PackGo {
Write-Header 'Go'
Write-Host 'Go modules are released by git-tagging — no artifact to pack.' -ForegroundColor Yellow
Write-Host 'To publish a Go release, run:' -ForegroundColor Yellow
Write-Host ' pwsh scripts/tag-go-module.ps1 -Version v0.1.0 -Push' -ForegroundColor Yellow
Write-Host '(skipping)' -ForegroundColor DarkGray
}
# -------- Dispatch --------
$wanted = @{}
foreach ($lang in $Languages) { $wanted[$lang.ToLower()] = $true }
if ($wanted.ContainsKey('dotnet')) { Invoke-PackDotnet }
if ($wanted.ContainsKey('python')) { Invoke-PackPython }
if ($wanted.ContainsKey('rust')) { Invoke-PackRust }
if ($wanted.ContainsKey('java')) { Invoke-PackJava }
if ($wanted.ContainsKey('go')) { Invoke-PackGo }
# -------- Summary --------
Write-Header 'Summary'
$artifacts = @(Get-ChildItem $OutputDir)
if ($artifacts.Count -eq 0) {
Write-Host ' (no artifacts)' -ForegroundColor DarkGray
} else {
foreach ($a in $artifacts) {
Write-Host (' {0,10} {1}' -f $a.Length, $a.Name)
}
}
Write-Host ''
Write-Host "All artifacts in: $OutputDir" -ForegroundColor Green
+62
View File
@@ -0,0 +1,62 @@
#Requires -Version 7
<#
.SYNOPSIS
Tags a release of the Go MxAccessGateway client module.
.DESCRIPTION
Go modules in monorepo subdirectories use prefixed tags
("clients/go/v0.1.0") so `go get <module>@v0.1.0` resolves correctly.
This script validates the version, creates the prefixed tag at HEAD,
and (optionally) pushes it.
.PARAMETER Version
Semver tag without the prefix, e.g. "v0.1.0".
.PARAMETER Push
When set, pushes the tag to origin after creation.
.EXAMPLE
pwsh scripts/tag-go-module.ps1 -Version v0.1.0
pwsh scripts/tag-go-module.ps1 -Version v0.1.1 -Push
#>
[CmdletBinding()]
param(
[Parameter(Mandatory = $true)]
[string]$Version,
[switch]$Push
)
Set-StrictMode -Version Latest
$ErrorActionPreference = 'Stop'
if ($Version -notmatch '^v\d+\.\d+\.\d+(-[A-Za-z0-9.-]+)?$') {
throw "Version '$Version' must match semver vX.Y.Z (optionally with -prerelease suffix)."
}
$tag = "clients/go/$Version"
Write-Host "Creating Go-module tag: $tag" -ForegroundColor Cyan
# Verify we're on a clean checkout — refuse to tag with uncommitted changes.
$status = (git status --porcelain) -join "`n"
if ($status -and -not ($status -match '^\?\?')) {
throw "Working tree has tracked changes. Commit or stash before tagging."
}
# Verify the tag doesn't already exist.
$existing = git tag --list $tag
if ($existing) {
throw "Tag '$tag' already exists. Use a new version."
}
git tag -a $tag -m "Go client release $Version"
Write-Host "Created tag: $tag" -ForegroundColor Green
if ($Push) {
git push origin $tag
Write-Host "Pushed tag to origin." -ForegroundColor Green
} else {
Write-Host "Tag not pushed. To publish, run: git push origin $tag" -ForegroundColor Yellow
}
@@ -8,10 +8,13 @@ namespace ZB.MOM.WW.MxGateway.Contracts;
/// </summary>
public static class GatewayContractInfo
{
/// <summary>Protocol version advertised to clients in <c>OpenSessionReply</c>.</summary>
public const uint GatewayProtocolVersion = 3;
/// <summary>Protocol version used to validate <c>WorkerEnvelope</c> framing on the gateway-worker pipe.</summary>
public const uint WorkerProtocolVersion = 1;
/// <summary>Default backend name identifying the MXAccess worker process type.</summary>
public const string DefaultBackendName = "mxaccess-worker";
/// <summary>
@@ -4,6 +4,24 @@
<TargetFrameworks>net10.0;net48</TargetFrameworks>
</PropertyGroup>
<PropertyGroup>
<IsPackable>true</IsPackable>
<PackageId>ZB.MOM.WW.MxGateway.Contracts</PackageId>
<Version>0.1.0</Version>
<Authors>Joseph Doherty</Authors>
<Company>ZB MOM WW</Company>
<Copyright>Copyright (c) ZB MOM WW. All rights reserved.</Copyright>
<Description>Protobuf contracts and gRPC stubs for the MxAccessGateway service. Multi-targets net10.0 and net48.</Description>
<RepositoryUrl>https://gitea.dohertylan.com/dohertj2/mxaccessgw</RepositoryUrl>
<RepositoryType>git</RepositoryType>
<PackageProjectUrl>https://gitea.dohertylan.com/dohertj2/mxaccessgw</PackageProjectUrl>
<PackageTags>mxaccess;mxgateway;grpc;contracts;protobuf</PackageTags>
<PackageRequireLicenseAcceptance>false</PackageRequireLicenseAcceptance>
<IncludeSymbols>true</IncludeSymbols>
<SymbolPackageFormat>snupkg</SymbolPackageFormat>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
</PropertyGroup>
<ItemGroup>
<Compile Remove="Generated\**\*.cs" />
<Protobuf Include="Protos\mxaccess_gateway.proto" ProtoRoot="Protos" OutputDir="Generated" GrpcOutputDir="Generated" GrpcServices="Both" />
@@ -0,0 +1,64 @@
using ZB.MOM.WW.Telemetry.Serilog;
namespace ZB.MOM.WW.MxGateway.Server.Diagnostics;
/// <summary>
/// Bridges the gateway's <see cref="GatewayLogRedactor"/> policy onto the shared
/// <see cref="ILogRedactor"/> seam consumed by <c>ZB.MOM.WW.Telemetry.Serilog</c>'s redaction
/// enricher. Applied to every Serilog log event before it reaches a sink, it masks the same
/// secrets the original MEL-scope path masked: API-key bearer tokens / client identities
/// (<c>mxgw_*</c>) and command values for credential-bearing MXAccess commands. All masking
/// decisions delegate to <see cref="GatewayLogRedactor"/> — this type adds no new policy.
/// </summary>
public sealed class GatewayLogRedactorAdapter : ILogRedactor
{
/// <summary>Property name carrying a client identity / authorization header value.</summary>
private const string ClientIdentityProperty = "ClientIdentity";
/// <summary>Property name carrying a raw authorization header value.</summary>
private const string AuthorizationProperty = "Authorization";
/// <summary>Property name carrying the MXAccess command method, used to gate value redaction.</summary>
private const string CommandMethodProperty = "CommandMethod";
/// <summary>Property name carrying a command payload value that may bear credentials.</summary>
private const string CommandValueProperty = "CommandValue";
/// <summary>
/// Masks any sensitive values in <paramref name="properties"/> in place using the shared
/// <see cref="GatewayLogRedactor"/> policy. Identity/authorization properties have their API-key
/// secret stripped; a command value is redacted when its associated command method bears
/// credentials.
/// </summary>
/// <param name="properties">The mutable log-event property dictionary for the current event.</param>
public void Redact(IDictionary<string, object?> properties)
{
ArgumentNullException.ThrowIfNull(properties);
RedactIdentity(properties, ClientIdentityProperty);
RedactIdentity(properties, AuthorizationProperty);
RedactCommandValue(properties);
}
private static void RedactIdentity(IDictionary<string, object?> properties, string propertyName)
{
if (properties.TryGetValue(propertyName, out object? value) && value is string identity)
{
properties[propertyName] = GatewayLogRedactor.RedactClientIdentity(identity);
}
}
private static void RedactCommandValue(IDictionary<string, object?> properties)
{
if (!properties.TryGetValue(CommandValueProperty, out object? value) || value is null)
{
return;
}
string? commandMethod = properties.TryGetValue(CommandMethodProperty, out object? method)
? method as string
: null;
properties[CommandValueProperty] = GatewayLogRedactor.RedactCommandValue(commandMethod, value);
}
}
@@ -1,20 +0,0 @@
using Microsoft.Extensions.Logging;
namespace ZB.MOM.WW.MxGateway.Server.Diagnostics;
public static class GatewayLoggerExtensions
{
/// <summary>Begins a gateway log scope with the specified scope properties.</summary>
/// <param name="logger">Logger used for diagnostic output.</param>
/// <param name="scope">Scope properties to apply.</param>
/// <returns>A disposable that ends the scope when disposed.</returns>
public static IDisposable? BeginGatewayScope(
this ILogger logger,
GatewayLogScope scope)
{
ArgumentNullException.ThrowIfNull(logger);
ArgumentNullException.ThrowIfNull(scope);
return logger.BeginScope(scope.ToDictionary());
}
}
@@ -1,4 +1,5 @@
using Microsoft.Extensions.Primitives;
using Serilog.Context;
namespace ZB.MOM.WW.MxGateway.Server.Diagnostics;
@@ -17,7 +18,12 @@ public static class GatewayRequestLoggingMiddlewareExtensions
/// <summary>Header name for the command method name.</summary>
public const string CommandMethodHeaderName = "x-command-method";
/// <summary>Adds gateway request logging scope middleware that reads correlation headers and redacts sensitive data.</summary>
/// <summary>
/// Adds gateway request logging middleware that reads the correlation headers and pushes them
/// as Serilog <see cref="LogContext"/> properties for the duration of the request. The pushed
/// properties (SessionId / WorkerProcessId / CorrelationId / CommandMethod / ClientIdentity)
/// are disposed when the request completes; the shared redaction enricher masks any secrets.
/// </summary>
/// <param name="app">Application builder.</param>
public static IApplicationBuilder UseGatewayRequestLoggingScope(this IApplicationBuilder app)
{
@@ -25,21 +31,56 @@ public static class GatewayRequestLoggingMiddlewareExtensions
return app.Use(async (context, next) =>
{
ILogger logger = context.RequestServices
.GetRequiredService<ILoggerFactory>()
.CreateLogger("MxGateway.Request");
using IDisposable? scope = logger.BeginGatewayScope(new GatewayLogScope(
GatewayLogScope scope = new(
SessionId: ReadHeader(context, SessionIdHeaderName),
WorkerProcessId: ReadInt32Header(context, WorkerProcessIdHeaderName),
CorrelationId: ReadUInt64Header(context, CorrelationIdHeaderName),
CommandMethod: ReadHeader(context, CommandMethodHeaderName),
ClientIdentity: ReadHeader(context, "authorization")));
ClientIdentity: ReadHeader(context, "authorization"));
using IDisposable correlationScope = PushCorrelationProperties(scope);
await next(context);
});
}
/// <summary>
/// Pushes the populated <paramref name="scope"/> properties onto the Serilog
/// <see cref="LogContext"/>, returning a single disposable that pops them all when the request
/// completes. Only the properties present in <see cref="GatewayLogScope.ToDictionary"/> (which
/// already applies the client-identity redaction policy) are pushed.
/// </summary>
/// <param name="scope">The correlation properties for the current request.</param>
/// <returns>A disposable that removes the pushed properties on disposal.</returns>
private static IDisposable PushCorrelationProperties(GatewayLogScope scope)
{
Stack<IDisposable> pushed = new();
foreach (KeyValuePair<string, object?> property in scope.ToDictionary())
{
pushed.Push(LogContext.PushProperty(property.Key, property.Value));
}
return new CorrelationPropertyScope(pushed);
}
/// <summary>
/// Disposes the pushed <see cref="LogContext"/> property bindings in reverse order, restoring
/// the ambient context to its pre-request state.
/// </summary>
private sealed class CorrelationPropertyScope(Stack<IDisposable> bindings) : IDisposable
{
private readonly Stack<IDisposable> _bindings = bindings;
public void Dispose()
{
while (_bindings.Count > 0)
{
_bindings.Pop().Dispose();
}
}
}
private static string? ReadHeader(HttpContext context, string headerName)
{
return context.Request.Headers.TryGetValue(headerName, out StringValues values)
@@ -62,7 +62,16 @@ public static class GalaxyBrowseProjector
return new GalaxyBrowseChildrenResult(page, hasChildren, filtered.Children.Count, filterSignature);
}
private static int ResolveParentId(GalaxyHierarchyCacheEntry entry, BrowseChildrenRequest request)
/// <summary>
/// Resolves the request's parent oneof to a gobject id, throwing
/// <see cref="RpcException"/> with <see cref="StatusCode.NotFound"/> when the
/// parent does not exist. Public so the gRPC handler can compute the same
/// parent id (needed for the page-token signature) without reimplementing the
/// resolution rules.
/// </summary>
/// <param name="entry">The Galaxy hierarchy cache entry to query.</param>
/// <param name="request">The browse-children request.</param>
public static int ResolveParentId(GalaxyHierarchyCacheEntry entry, BrowseChildrenRequest request)
{
switch (request.ParentCase)
{
@@ -80,9 +89,7 @@ public static class GalaxyBrowseProjector
return request.ParentGobjectId;
case BrowseChildrenRequest.ParentOneofCase.ParentTagName:
{
GalaxyObjectView? match = entry.Index.ObjectViews.FirstOrDefault(
view => string.Equals(view.Object.TagName, request.ParentTagName, StringComparison.OrdinalIgnoreCase));
if (match is null)
if (!entry.Index.ObjectViewsByTagName.TryGetValue(request.ParentTagName, out GalaxyObjectView? match))
{
throw new RpcException(new Status(StatusCode.NotFound, "BrowseChildren parent was not found."));
}
@@ -90,9 +97,7 @@ public static class GalaxyBrowseProjector
}
case BrowseChildrenRequest.ParentOneofCase.ParentContainedPath:
{
GalaxyObjectView? match = entry.Index.ObjectViews.FirstOrDefault(
view => string.Equals(view.ContainedPath, request.ParentContainedPath, StringComparison.OrdinalIgnoreCase));
if (match is null)
if (!entry.Index.ObjectViewsByContainedPath.TryGetValue(request.ParentContainedPath, out GalaxyObjectView? match))
{
throw new RpcException(new Status(StatusCode.NotFound, "BrowseChildren parent was not found."));
}
@@ -163,10 +168,17 @@ public static class GalaxyBrowseProjector
return false;
}
// Defend against pathological cycles in Galaxy data (e.g. a corrupt A→B→A chain).
// BuildContainedPath uses the same visited-id pattern; mirror it so this walk
// terminates even when ChildrenByParent forms a cycle.
HashSet<int> visited = new() { parent.Object.GobjectId };
Stack<GalaxyObjectView> stack = new();
foreach (GalaxyObjectView child in children)
{
stack.Push(child);
if (visited.Add(child.Object.GobjectId))
{
stack.Push(child);
}
}
while (stack.Count > 0)
{
@@ -180,7 +192,10 @@ public static class GalaxyBrowseProjector
{
foreach (GalaxyObjectView grandchild in grandchildren)
{
stack.Push(grandchild);
if (visited.Add(grandchild.Object.GobjectId))
{
stack.Push(grandchild);
}
}
}
}
@@ -8,12 +8,16 @@ public sealed class GalaxyHierarchyIndex
IReadOnlyList<GalaxyObjectView> objectViews,
IReadOnlyDictionary<int, GalaxyObjectView> objectViewsById,
IReadOnlyDictionary<string, GalaxyTagLookup> tagsByAddress,
IReadOnlyDictionary<int, IReadOnlyList<GalaxyObjectView>> childrenByParent)
IReadOnlyDictionary<int, IReadOnlyList<GalaxyObjectView>> childrenByParent,
IReadOnlyDictionary<string, GalaxyObjectView> objectViewsByTagName,
IReadOnlyDictionary<string, GalaxyObjectView> objectViewsByContainedPath)
{
ObjectViews = objectViews;
ObjectViewsById = objectViewsById;
TagsByAddress = tagsByAddress;
ChildrenByParent = childrenByParent;
ObjectViewsByTagName = objectViewsByTagName;
ObjectViewsByContainedPath = objectViewsByContainedPath;
}
/// <summary>Gets an empty Galaxy hierarchy index.</summary>
@@ -21,7 +25,9 @@ public sealed class GalaxyHierarchyIndex
Array.Empty<GalaxyObjectView>(),
new Dictionary<int, GalaxyObjectView>(),
new Dictionary<string, GalaxyTagLookup>(StringComparer.OrdinalIgnoreCase),
new Dictionary<int, IReadOnlyList<GalaxyObjectView>>());
new Dictionary<int, IReadOnlyList<GalaxyObjectView>>(),
new Dictionary<string, GalaxyObjectView>(StringComparer.OrdinalIgnoreCase),
new Dictionary<string, GalaxyObjectView>(StringComparer.OrdinalIgnoreCase));
/// <summary>Gets the object views.</summary>
public IReadOnlyList<GalaxyObjectView> ObjectViews { get; }
@@ -35,6 +41,12 @@ public sealed class GalaxyHierarchyIndex
/// <summary>Gets direct children grouped by parent gobject id. Root objects (no parent, or self-parented) live under key 0. Each list is sorted areas-first, then by display name (OrdinalIgnoreCase).</summary>
public IReadOnlyDictionary<int, IReadOnlyList<GalaxyObjectView>> ChildrenByParent { get; }
/// <summary>Gets object views indexed by <see cref="GalaxyObject.TagName"/> (OrdinalIgnoreCase). Lets browse/discover handlers resolve parents/roots by tag name in O(1) instead of scanning <see cref="ObjectViews"/>.</summary>
public IReadOnlyDictionary<string, GalaxyObjectView> ObjectViewsByTagName { get; }
/// <summary>Gets object views indexed by contained path (OrdinalIgnoreCase). Lets browse/discover handlers resolve parents/roots by path in O(1) instead of scanning <see cref="ObjectViews"/>.</summary>
public IReadOnlyDictionary<string, GalaxyObjectView> ObjectViewsByContainedPath { get; }
/// <summary>Builds a Galaxy hierarchy index from the given objects.</summary>
/// <param name="objects">The Galaxy objects to index.</param>
/// <returns>A new Galaxy hierarchy index.</returns>
@@ -54,6 +66,8 @@ public sealed class GalaxyHierarchyIndex
List<GalaxyObjectView> views = new(objects.Count);
Dictionary<int, GalaxyObjectView> viewsById = new();
Dictionary<string, GalaxyTagLookup> tagsByAddress = new(StringComparer.OrdinalIgnoreCase);
Dictionary<string, GalaxyObjectView> viewsByTagName = new(StringComparer.OrdinalIgnoreCase);
Dictionary<string, GalaxyObjectView> viewsByContainedPath = new(StringComparer.OrdinalIgnoreCase);
foreach (GalaxyObject obj in objects)
{
@@ -66,6 +80,12 @@ public sealed class GalaxyHierarchyIndex
if (!string.IsNullOrWhiteSpace(obj.TagName))
{
tagsByAddress.TryAdd(obj.TagName, new GalaxyTagLookup(obj, Attribute: null, path));
viewsByTagName.TryAdd(obj.TagName, view);
}
if (!string.IsNullOrWhiteSpace(path))
{
viewsByContainedPath.TryAdd(path, view);
}
foreach (GalaxyAttribute attribute in obj.Attributes)
@@ -109,7 +129,9 @@ public sealed class GalaxyHierarchyIndex
views,
viewsById,
tagsByAddress,
readOnlyChildren);
readOnlyChildren,
viewsByTagName,
viewsByContainedPath);
}
private static string BuildContainedPath(
@@ -103,7 +103,7 @@ public static class GalaxyHierarchyProjector
// ResolveRoot can throw RpcException(NotFound); run it before consulting the
// memo so a bad root surfaces consistently regardless of cache state.
IReadOnlyList<GalaxyObjectView> views = entry.Index.ObjectViews;
GalaxyObjectView? root = ResolveRoot(request, views);
GalaxyObjectView? root = ResolveRoot(request, entry.Index);
ConcurrentDictionary<string, IReadOnlyList<GalaxyObjectView>> memo =
FilteredViewCache.GetValue(entry, static _ => new ConcurrentDictionary<string, IReadOnlyList<GalaxyObjectView>>(StringComparer.Ordinal));
@@ -176,17 +176,17 @@ public static class GalaxyHierarchyProjector
private static GalaxyObjectView? ResolveRoot(
DiscoverHierarchyRequest request,
IReadOnlyList<GalaxyObjectView> views)
GalaxyHierarchyIndex index)
{
GalaxyObjectView? root = request.RootCase switch
{
DiscoverHierarchyRequest.RootOneofCase.None => null,
DiscoverHierarchyRequest.RootOneofCase.RootGobjectId => views.FirstOrDefault(
view => view.Object.GobjectId == request.RootGobjectId),
DiscoverHierarchyRequest.RootOneofCase.RootTagName => views.FirstOrDefault(
view => string.Equals(view.Object.TagName, request.RootTagName, StringComparison.OrdinalIgnoreCase)),
DiscoverHierarchyRequest.RootOneofCase.RootContainedPath => views.FirstOrDefault(
view => string.Equals(view.ContainedPath, request.RootContainedPath, StringComparison.OrdinalIgnoreCase)),
DiscoverHierarchyRequest.RootOneofCase.RootGobjectId =>
index.ObjectViewsById.TryGetValue(request.RootGobjectId, out GalaxyObjectView? byId) ? byId : null,
DiscoverHierarchyRequest.RootOneofCase.RootTagName =>
index.ObjectViewsByTagName.TryGetValue(request.RootTagName, out GalaxyObjectView? byTag) ? byTag : null,
DiscoverHierarchyRequest.RootOneofCase.RootContainedPath =>
index.ObjectViewsByContainedPath.TryGetValue(request.RootContainedPath, out GalaxyObjectView? byPath) ? byPath : null,
_ => null,
};
@@ -1,4 +1,5 @@
using Microsoft.AspNetCore.Hosting.StaticWebAssets;
using Serilog;
using ZB.MOM.WW.MxGateway.Contracts;
using ZB.MOM.WW.MxGateway.Server.Alarms;
using ZB.MOM.WW.MxGateway.Server.Configuration;
@@ -11,6 +12,7 @@ using ZB.MOM.WW.MxGateway.Server.Security.Authentication;
using ZB.MOM.WW.MxGateway.Server.Security.Authorization;
using ZB.MOM.WW.MxGateway.Server.Sessions;
using ZB.MOM.WW.MxGateway.Server.Workers;
using ZB.MOM.WW.Telemetry.Serilog;
namespace ZB.MOM.WW.MxGateway.Server;
@@ -31,7 +33,10 @@ public static class GatewayApplication
WebApplicationBuilder builder = CreateBuilder(args);
WebApplication app = builder.Build();
// Push the per-request correlation properties (via Serilog LogContext) before the
// request-logging middleware emits its completion event, so those properties appear on it.
app.UseGatewayRequestLoggingScope();
app.UseSerilogRequestLogging();
app.UseStaticFiles();
app.UseAuthentication();
app.UseAuthorization();
@@ -55,6 +60,8 @@ public static class GatewayApplication
});
StaticWebAssetsLoader.UseStaticWebAssets(builder.Environment, builder.Configuration);
ConfigureSerilog(builder);
builder.Services.AddGatewayConfiguration();
builder.Services.AddSqliteAuthStore();
builder.Services.AddGatewayGrpcAuthorization();
@@ -72,6 +79,30 @@ public static class GatewayApplication
return builder;
}
/// <summary>
/// Replaces the default Microsoft.Extensions.Logging provider with the shared
/// <c>ZB.MOM.WW.Telemetry.Serilog</c> bootstrap (<see cref="ZbSerilogExtensions.AddZbSerilog"/>).
/// Sinks and minimum level come from the <c>Serilog</c> configuration section; identity
/// (<c>SiteId</c>/<c>NodeRole</c>) is read from <c>MxGateway:Telemetry</c> when present.
/// Also registers the project's <see cref="ILogRedactor"/> adapter so the shared redaction
/// enricher masks gateway secrets on every event.
/// </summary>
/// <param name="builder">The web application builder being configured.</param>
private static void ConfigureSerilog(WebApplicationBuilder builder)
{
string? siteId = builder.Configuration["MxGateway:Telemetry:SiteId"];
string? nodeRole = builder.Configuration["MxGateway:Telemetry:NodeRole"];
builder.Services.AddSingleton<ILogRedactor, GatewayLogRedactorAdapter>();
builder.AddZbSerilog(options =>
{
options.ServiceName = "mxgateway";
options.SiteId = string.IsNullOrWhiteSpace(siteId) ? null : siteId;
options.NodeRole = string.IsNullOrWhiteSpace(nodeRole) ? null : nodeRole;
});
}
private static string ResolveContentRootPath()
{
string? configuredContentRootPath = Environment.GetEnvironmentVariable("ASPNETCORE_CONTENTROOT");
@@ -128,8 +128,11 @@ public sealed class GalaxyRepositoryGrpcService(
IReadOnlyList<string> browseSubtrees = ResolveBrowseSubtrees();
// Resolve the parent id once so the page-token signature can include it
// and the projector sees the same resolved id when memoizing.
int parentId = ResolveParentIdForToken(entry, request);
// and the projector sees the same resolved id when memoizing. The projector
// re-resolves internally; with the by-name/by-path indexes on
// GalaxyHierarchyIndex that second call is O(1), so the redundancy is cheap
// and keeps the projector self-contained.
int parentId = GalaxyDb.GalaxyBrowseProjector.ResolveParentId(entry, request);
string filterSignature = GalaxyDb.GalaxyBrowseProjector.ComputeFilterSignature(
request, browseSubtrees, parentId);
PageToken pageToken = ParsePageToken(request.PageToken, entry.Sequence, filterSignature);
@@ -283,32 +286,6 @@ public sealed class GalaxyRepositoryGrpcService(
return Math.Min(pageSize, MaxDiscoverPageSize);
}
// Lightweight parent resolver used only for signature computation. Re-throws
// NotFound consistently with the projector so the error surface matches.
private static int ResolveParentIdForToken(
GalaxyDb.GalaxyHierarchyCacheEntry entry,
BrowseChildrenRequest request)
{
return request.ParentCase switch
{
BrowseChildrenRequest.ParentOneofCase.None => 0,
BrowseChildrenRequest.ParentOneofCase.ParentGobjectId =>
request.ParentGobjectId == 0 ? 0
: entry.Index.ObjectViewsById.ContainsKey(request.ParentGobjectId)
? request.ParentGobjectId
: throw new RpcException(new Status(StatusCode.NotFound, "BrowseChildren parent was not found.")),
BrowseChildrenRequest.ParentOneofCase.ParentTagName =>
entry.Index.ObjectViews.FirstOrDefault(
v => string.Equals(v.Object.TagName, request.ParentTagName, StringComparison.OrdinalIgnoreCase))?.Object.GobjectId
?? throw new RpcException(new Status(StatusCode.NotFound, "BrowseChildren parent was not found.")),
BrowseChildrenRequest.ParentOneofCase.ParentContainedPath =>
entry.Index.ObjectViews.FirstOrDefault(
v => string.Equals(v.ContainedPath, request.ParentContainedPath, StringComparison.OrdinalIgnoreCase))?.Object.GobjectId
?? throw new RpcException(new Status(StatusCode.NotFound, "BrowseChildren parent was not found.")),
_ => 0,
};
}
private IReadOnlyList<string> ResolveBrowseSubtrees()
{
ApiKeyConstraints constraints = identityAccessor.Current?.EffectiveConstraints ?? ApiKeyConstraints.Empty;
@@ -348,21 +325,21 @@ public sealed class GalaxyRepositoryGrpcService(
{
throw new RpcException(new Status(
StatusCode.InvalidArgument,
"DiscoverHierarchy page_token is invalid."));
"page_token is invalid."));
}
if (sequence != currentSequence)
{
throw new RpcException(new Status(
StatusCode.InvalidArgument,
"DiscoverHierarchy page_token is stale."));
"page_token is stale."));
}
if (!string.Equals(parts[1], currentFilterSignature, StringComparison.Ordinal))
{
throw new RpcException(new Status(
StatusCode.InvalidArgument,
"DiscoverHierarchy page_token does not match the current filters."));
"page_token does not match the current filters."));
}
return new PageToken(sequence, parts[1], offset);
@@ -15,6 +15,13 @@
<ItemGroup>
<ProjectReference Include="..\ZB.MOM.WW.MxGateway.Contracts\ZB.MOM.WW.MxGateway.Contracts.csproj" />
<!--
Shared structured-logging bootstrap (ZB.MOM.WW.Telemetry.Serilog) lives in the sibling
scadaproj workspace. Cross-repo ProjectReference: the referenced project resolves its own
Directory.Build.props / Directory.Packages.props from its own tree, so it does not perturb
this repo's build settings. It transitively brings the ZB.MOM.WW.Telemetry core package.
-->
<ProjectReference Include="..\..\..\scadaproj\ZB.MOM.WW.Telemetry\src\ZB.MOM.WW.Telemetry.Serilog\ZB.MOM.WW.Telemetry.Serilog.csproj" />
</ItemGroup>
</Project>
@@ -1,8 +1,10 @@
{
"Logging": {
"LogLevel": {
"Serilog": {
"MinimumLevel": {
"Default": "Information",
"Microsoft.AspNetCore": "Warning"
"Override": {
"Microsoft.AspNetCore": "Warning"
}
}
}
}
@@ -1,9 +1,30 @@
{
"Logging": {
"LogLevel": {
"Serilog": {
"Using": [
"Serilog.Sinks.Console",
"Serilog.Sinks.File"
],
"MinimumLevel": {
"Default": "Information",
"Microsoft.AspNetCore": "Warning"
}
"Override": {
"Microsoft.AspNetCore": "Warning"
}
},
"WriteTo": [
{
"Name": "Console",
"Args": {
"outputTemplate": "[{Timestamp:HH:mm:ss} {Level:u3}] [{NodeRole}/{NodeHostname}] {Message:lj} {Properties:j}{NewLine}{Exception}"
}
},
{
"Name": "File",
"Args": {
"path": "logs/mxgateway-.log",
"rollingInterval": "Day"
}
}
]
},
"AllowedHosts": "*",
"MxGateway": {
@@ -0,0 +1,92 @@
using ZB.MOM.WW.MxGateway.Server.Diagnostics;
using ZB.MOM.WW.Telemetry.Serilog;
namespace ZB.MOM.WW.MxGateway.Tests.Diagnostics;
/// <summary>
/// Pins that <see cref="GatewayLogRedactorAdapter"/> applies the gateway's redaction policy through
/// the shared <see cref="ILogRedactor"/> seam — the same secrets the former MEL-scope path masked
/// must still be masked once events flow through the Serilog redaction enricher.
/// </summary>
public sealed class GatewayLogRedactorAdapterTests
{
private readonly ILogRedactor _redactor = new GatewayLogRedactorAdapter();
/// <summary>Verifies the client identity property has its API-key secret stripped in place.</summary>
[Fact]
public void Redact_StripsApiKeySecretFromClientIdentity()
{
Dictionary<string, object?> properties = new()
{
["ClientIdentity"] = "Bearer mxgw_operator01_super-secret",
};
_redactor.Redact(properties);
Assert.Equal("Bearer mxgw_operator01_[redacted]", properties["ClientIdentity"]);
Assert.DoesNotContain("super-secret", (string?)properties["ClientIdentity"]);
}
/// <summary>Verifies a raw authorization header property is redacted too.</summary>
[Fact]
public void Redact_StripsApiKeySecretFromAuthorizationProperty()
{
Dictionary<string, object?> properties = new()
{
["Authorization"] = "Bearer mxgw_admin_top-secret",
};
_redactor.Redact(properties);
Assert.Equal("Bearer mxgw_admin_[redacted]", properties["Authorization"]);
}
/// <summary>Verifies a command value is redacted for a credential-bearing command method.</summary>
[Fact]
public void Redact_RedactsCommandValueForCredentialBearingCommand()
{
Dictionary<string, object?> properties = new()
{
["CommandMethod"] = "WriteSecured",
["CommandValue"] = "credential-bearing-value",
};
_redactor.Redact(properties);
Assert.Equal(GatewayLogRedactor.RedactedValue, properties["CommandValue"]);
}
/// <summary>Verifies a command value is redacted by default (value logging disabled) for any command.</summary>
[Fact]
public void Redact_RedactsCommandValueByDefault()
{
Dictionary<string, object?> properties = new()
{
["CommandMethod"] = "Write",
["CommandValue"] = "plaintext-tag-value",
};
_redactor.Redact(properties);
Assert.Equal(GatewayLogRedactor.RedactedValue, properties["CommandValue"]);
}
/// <summary>Verifies non-sensitive properties are left untouched.</summary>
[Fact]
public void Redact_LeavesNonSensitivePropertiesUnchanged()
{
Dictionary<string, object?> properties = new()
{
["SessionId"] = "session-1",
["CorrelationId"] = (ulong)99,
["ClientIdentity"] = "Bearer plain-token-no-marker",
};
_redactor.Redact(properties);
Assert.Equal("session-1", properties["SessionId"]);
Assert.Equal((ulong)99, properties["CorrelationId"]);
// No mxgw_ marker — identity passes through unchanged.
Assert.Equal("Bearer plain-token-no-marker", properties["ClientIdentity"]);
}
}
@@ -223,6 +223,77 @@ public sealed class GalaxyBrowseProjectorTests
Assert.Equal("Plant.Line_A", result.Children[0].TagName);
}
/// <summary>
/// Verifies <see cref="GalaxyBrowseProjector"/> terminates when the Galaxy data
/// contains a cyclic parent chain (A→B→C→A). Without the visited-id guard in
/// <c>HasMatchingDescendant</c>, the depth-first walk would loop forever; the
/// 5-second xUnit timeout asserts termination.
/// </summary>
[Fact(Timeout = 5000)]
public async Task Project_CyclicDescendants_DoesNotInfiniteLoop()
{
await Task.Yield();
// Construct a 3-node cycle: A(10)→B(11)→C(12)→A. Each node's ParentGobjectId
// points to the next, so GalaxyHierarchyIndex.ChildrenByParent has
// [12] = [A], [10] = [B], [11] = [C].
// None of them are historized, so HistorizedOnly=true forces the projector to
// call HasMatchingDescendant on every direct child, exercising the cycle walk.
GalaxyObject a = new()
{
GobjectId = 10,
ParentGobjectId = 12,
ContainedName = "A",
BrowseName = "A",
TagName = "A",
};
GalaxyObject b = new()
{
GobjectId = 11,
ParentGobjectId = 10,
ContainedName = "B",
BrowseName = "B",
TagName = "B",
};
GalaxyObject c = new()
{
GobjectId = 12,
ParentGobjectId = 11,
ContainedName = "C",
BrowseName = "C",
TagName = "C",
};
IReadOnlyList<GalaxyObject> objects = new[] { a, b, c };
GalaxyHierarchyCacheEntry entry = GalaxyHierarchyCacheEntry.Empty with
{
Status = GalaxyCacheStatus.Healthy,
Sequence = 1,
LastSuccessAt = DateTimeOffset.UtcNow,
Objects = objects,
Index = GalaxyHierarchyIndex.Build(objects),
DashboardSummary = DashboardGalaxySummary.Unknown with
{
Status = DashboardGalaxyStatus.Healthy,
ObjectCount = objects.Count,
},
ObjectCount = objects.Count,
};
// Browse children of A (id=10). Its direct child B fails HistorizedOnly, so the
// projector falls back to HasMatchingDescendant(B), which walks B→C→A→B…
// without the visited-id guard. With the guard, the walk terminates and returns
// an empty page (no historized descendants exist anywhere in the cycle).
GalaxyBrowseChildrenResult result = GalaxyBrowseProjector.ProjectChildren(
entry,
new BrowseChildrenRequest { ParentGobjectId = 10, HistorizedOnly = true },
browseSubtreeGlobs: null,
offset: 0,
pageSize: 10);
Assert.Empty(result.Children);
Assert.Equal(0, result.TotalChildCount);
}
private static GalaxyHierarchyCacheEntry CreateEntry()
{
IReadOnlyList<GalaxyObject> objects = CreateObjects();
@@ -75,6 +75,47 @@ public sealed class GalaxyHierarchyIndexTests
}
}
/// <summary>Verifies <see cref="GalaxyHierarchyIndex.ObjectViewsByTagName"/> is OrdinalIgnoreCase and supports O(1) lookups.</summary>
[Fact]
public void ObjectViewsByTagName_IsCaseInsensitive_AndLookupsAreO1()
{
GalaxyObject root = new() { GobjectId = 1, ParentGobjectId = 0, IsArea = true, ContainedName = "Plant", BrowseName = "Plant", TagName = "Plant" };
GalaxyObject mixer = new() { GobjectId = 2, ParentGobjectId = 1, ContainedName = "Mixer_001", BrowseName = "Mixer_001", TagName = "Plant.Mixer_001" };
GalaxyHierarchyIndex index = GalaxyHierarchyIndex.Build([root, mixer]);
Assert.True(index.ObjectViewsByTagName.TryGetValue("Plant.Mixer_001", out GalaxyObjectView? exact));
Assert.NotNull(exact);
Assert.Equal(2, exact!.Object.GobjectId);
// Case-insensitive lookup must hit the same entry.
Assert.True(index.ObjectViewsByTagName.TryGetValue("plant.mixer_001", out GalaxyObjectView? lower));
Assert.NotNull(lower);
Assert.Same(exact, lower);
Assert.False(index.ObjectViewsByTagName.ContainsKey("Plant.Missing"));
}
/// <summary>Verifies <see cref="GalaxyHierarchyIndex.ObjectViewsByContainedPath"/> is OrdinalIgnoreCase.</summary>
[Fact]
public void ObjectViewsByContainedPath_IsCaseInsensitive()
{
GalaxyObject root = new() { GobjectId = 1, ParentGobjectId = 0, IsArea = true, ContainedName = "Plant", BrowseName = "Plant", TagName = "Plant" };
GalaxyObject lineA = new() { GobjectId = 2, ParentGobjectId = 1, IsArea = true, ContainedName = "Line_A", BrowseName = "Line_A", TagName = "Plant.Line_A" };
GalaxyHierarchyIndex index = GalaxyHierarchyIndex.Build([root, lineA]);
Assert.True(index.ObjectViewsByContainedPath.TryGetValue("Plant/Line_A", out GalaxyObjectView? exact));
Assert.NotNull(exact);
Assert.Equal(2, exact!.Object.GobjectId);
Assert.True(index.ObjectViewsByContainedPath.TryGetValue("plant/line_a", out GalaxyObjectView? lower));
Assert.NotNull(lower);
Assert.Same(exact, lower);
Assert.False(index.ObjectViewsByContainedPath.ContainsKey("Plant/Missing"));
}
/// <summary>Verifies children sort areas-first, then by display name (case-insensitive).</summary>
[Fact]
public void ChildrenByParent_SortsAreasFirstThenByDisplayName()