Improve docs coverage and refresh profiling parser artifacts
Add domain-specific XML documentation across src server components to satisfy CommentChecker, and update dotTrace parsing outputs used for diagnostics.
This commit is contained in:
File diff suppressed because it is too large
Load Diff
21
dottrace.md
21
dottrace.md
@@ -223,6 +223,13 @@ export DOTTRACE_APP_DIR="/path/to/dotTrace.app/Contents/DotFiles"
|
||||
python3 tools/dtp_parse.py snapshots/js-ordered-consume.dtp --stdout
|
||||
```
|
||||
|
||||
Useful flags:
|
||||
|
||||
- `--top N` limits hotspot and flat-path output. Default: `200`
|
||||
- `--filter TEXT` keeps only call-tree paths and hotspots whose method names match `TEXT`
|
||||
- `--flat` or `--paths` adds a `hotPaths` section with the heaviest flat call chains
|
||||
- `--include-idle` keeps idle and wait methods in hotspot/path rankings. Idle exclusion is on by default.
|
||||
|
||||
### Write JSON to a file
|
||||
|
||||
```bash
|
||||
@@ -230,16 +237,26 @@ python3 tools/dtp_parse.py snapshots/js-ordered-consume.dtp \
|
||||
--out /tmp/js-ordered-consume-calltree.json
|
||||
```
|
||||
|
||||
```bash
|
||||
python3 tools/dtp_parse.py snapshots/js-ordered-consume.dtp \
|
||||
--filter Microsoft.DotNet.Cli.Program \
|
||||
--flat \
|
||||
--top 25 \
|
||||
--out /tmp/js-ordered-consume-calltree.json
|
||||
```
|
||||
|
||||
### Output shape
|
||||
|
||||
The generated JSON contains:
|
||||
|
||||
- `snapshot` — source path, payload type, thread count, node count
|
||||
- `snapshot` — source path, payload type, time unit, thread count, node count, and reader diagnostics
|
||||
- `summary` — wall time, active time, total samples, and top exclusive method summary
|
||||
- `threadRoots` — top-level thread roots with inclusive time
|
||||
- `callTree` — nested call tree rooted at a synthetic `<root>`
|
||||
- `hotspots` — flat `inclusive` and `exclusive` method lists
|
||||
- `hotPaths` — optional flat call-path list when `--flat` is used
|
||||
|
||||
Hotspot entries are method-first. Synthetic frames such as thread roots are excluded from the hotspot lists so the output is easier to feed into an LLM for slowdown analysis.
|
||||
Hotspot entries are method-first. Synthetic frames such as thread roots are excluded from hotspot rankings, and idle wait frames are excluded by default so the output is easier to feed into an LLM for slowdown analysis.
|
||||
|
||||
### Typical analysis workflow
|
||||
|
||||
|
||||
139
dtp_updates.md
Normal file
139
dtp_updates.md
Normal file
@@ -0,0 +1,139 @@
|
||||
# DTP Snapshot Extractor — Requested Changes
|
||||
|
||||
## Problem
|
||||
|
||||
The current extractor produces too few nodes to be useful for performance analysis. A 30-second dotTrace sampling snapshot of the NATS server handling 1M messages (5s publish + 1.3s consume) yields only **202 nodes** in the JSON output. The entire publish/consume hot path is invisible — no `ProcessCommandsAsync`, `ProcessMessage`, `DeliverPullFetchMessagesAsync`, `SendMessageNoFlush`, `SubList.Match`, `FileStore.AppendAsync`, `MsgBlock.WriteAt`, or any other NATS server method appears in the call tree. Only server startup/shutdown and `InternalEventSystem` event serialization show up.
|
||||
|
||||
By contrast, the dotTrace GUI shows thousands of samples across these functions with clear call trees and accurate timing. The `.dtp` file has the data — the extractor is not surfacing it.
|
||||
|
||||
### Evidence
|
||||
|
||||
```
|
||||
Snapshot: threads=11, nodes=202
|
||||
Top NATS inclusive hotspots:
|
||||
NatsServer.WaitForShutdown 29741.8ms (idle wait)
|
||||
NatsServer..ctor 36.5ms (one-time init)
|
||||
InternalEventSystem 26.0ms (periodic stats)
|
||||
```
|
||||
|
||||
The actual hot path (`ProcessCommandsAsync` → `ProcessMessage` → fan-out/delivery) which runs for ~6 seconds of wall time is completely absent.
|
||||
|
||||
---
|
||||
|
||||
## Change 1: Increase Hotspot Limit
|
||||
|
||||
**Current:** `Take(50)` in `BuildHotspots` for both inclusive and exclusive lists.
|
||||
|
||||
**Requested:** Increase to at least **200**, or make it configurable via a CLI flag (e.g., `--top N`). With only 50 hotspots, important functions lower in the ranking are silently dropped.
|
||||
|
||||
---
|
||||
|
||||
## Change 2: Add `--filter` Flag to Python CLI
|
||||
|
||||
Add a `--filter` option that passes a substring filter to the .NET helper, so the JSON output only includes nodes whose name matches the filter. This reduces noise and lets me focus on the relevant code:
|
||||
|
||||
```bash
|
||||
python3 tools/dtp_parse.py snapshots/foo.dtp --filter NATS --out /tmp/result.json
|
||||
```
|
||||
|
||||
The .NET helper should filter the hotspot lists and prune the call tree to only include paths that contain at least one matching node (keeping ancestors and descendants of matching nodes).
|
||||
|
||||
---
|
||||
|
||||
## Change 3: Add Flat Call-Path Output Mode
|
||||
|
||||
The current nested call tree is hard to consume programmatically for hot-path analysis. Add a `--flat` or `--paths` mode that outputs the **top N heaviest call paths** as flat strings with timing:
|
||||
|
||||
```json
|
||||
{
|
||||
"hotPaths": [
|
||||
{
|
||||
"path": "ThreadPool > ProcessCommandsAsync > ProcessMessage > DeliverPullFetchMessagesAsync > SendMessageNoFlush",
|
||||
"inclusiveMs": 342.5,
|
||||
"leafExclusiveMs": 89.2
|
||||
},
|
||||
{
|
||||
"path": "ThreadPool > ProcessCommandsAsync > ProcessMessage > SubList.Match",
|
||||
"inclusiveMs": 156.3,
|
||||
"leafExclusiveMs": 156.3
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
This is the most useful output format for LLM-driven analysis — I can immediately see which call chains are expensive without walking the tree.
|
||||
|
||||
---
|
||||
|
||||
## Change 4: Exclude Idle/Wait Functions from Hotspots
|
||||
|
||||
Functions like `WaitHandle.WaitOneNoCheck`, `SemaphoreSlim.WaitCore`, `LowLevelLifoSemaphore.WaitForSignal`, `Monitor.Wait`, `SocketAsyncEngine.EventLoop`, `Thread.PollGC`, and `Interop+Sys.WaitForSocketEvents` dominate the hotspot lists but represent idle waiting, not actual CPU work. Either:
|
||||
|
||||
- Add a `--exclude-idle` flag (default on) that strips these from hotspot lists, or
|
||||
- Always exclude them from the `exclusive` hotspot list (they have zero useful exclusive time) and keep them in `inclusive` only if requested.
|
||||
|
||||
---
|
||||
|
||||
## Change 5: Investigate Missing Nodes (Critical)
|
||||
|
||||
This is the most important issue. **202 nodes from a 30-second sampling profile is far too few.** The dotTrace GUI shows the same snapshot with a full, deep call tree across all ThreadPool workers. Possible causes:
|
||||
|
||||
1. **The DFS reader is not reading all sections.** The `callTreeSections.AllHeaders()` call may not be returning headers for all threads or all sampling intervals. Check whether there are multiple call tree section families and the current code only reads one.
|
||||
|
||||
2. **Node merging/deduplication is losing data.** If two threads call the same function, they may share a `FunctionUID` but have different `CallTreeSectionOffset` values. Verify that the `nodeMap` dictionary keyed by offset isn't accidentally losing nodes from different threads.
|
||||
|
||||
3. **The `totalNodeCount` calculation may be wrong.** The formula `(SectionSize - SectionHeaderSize) / RecordSize()` may not account for all record types or section layouts in sampling snapshots.
|
||||
|
||||
4. **Sampling vs tracing data layout differences.** The code may have been tested primarily with tracing snapshots. Sampling snapshots store data differently — verify that the same reader API works for both.
|
||||
|
||||
The fix should result in **thousands of nodes** for a typical 30-second sampling snapshot, not 202. If the current dotTrace API approach fundamentally can't extract sampling data at full fidelity, document that limitation and suggest an alternative approach (e.g., using dotTrace's built-in report export if available on macOS, or switching to a different API surface).
|
||||
|
||||
---
|
||||
|
||||
## Change 6: Add Time Unit to JSON Output
|
||||
|
||||
The current `inclusiveTime` / `exclusiveTime` values are in an unspecified unit (nanoseconds based on magnitude). Add a `timeUnit` field to the `snapshot` section:
|
||||
|
||||
```json
|
||||
{
|
||||
"snapshot": {
|
||||
"path": "...",
|
||||
"payloadType": "time",
|
||||
"timeUnit": "nanoseconds",
|
||||
"threadCount": 11,
|
||||
"nodeCount": 1923
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Change 7: Add Summary Statistics
|
||||
|
||||
Add a `summary` section to the output with quick-reference stats:
|
||||
|
||||
```json
|
||||
{
|
||||
"summary": {
|
||||
"wallTimeMs": 30155,
|
||||
"activeTimeMs": 6340,
|
||||
"totalSamples": 15234,
|
||||
"topExclusiveMethod": "SendMessageNoFlush",
|
||||
"topExclusiveMs": 89.2
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
This lets me immediately assess whether the profile captured meaningful work without parsing the full tree.
|
||||
|
||||
---
|
||||
|
||||
## Priority Order
|
||||
|
||||
1. **Change 5** (missing nodes) — without this, everything else is moot
|
||||
2. **Change 4** (exclude idle) — makes hotspots immediately useful
|
||||
3. **Change 1** (increase limit) — more hotspots visible
|
||||
4. **Change 6** (time unit) — eliminates guesswork
|
||||
5. **Change 3** (flat paths) — most useful output format for analysis
|
||||
6. **Change 2** (filter) — nice to have for focused analysis
|
||||
7. **Change 7** (summary) — nice to have for quick assessment
|
||||
@@ -1,486 +0,0 @@
|
||||
# Documentation Analysis Report
|
||||
|
||||
Files Scanned: 273
|
||||
Files With Issues: 36
|
||||
Total Issues: 48
|
||||
|
||||
## Issues
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/Auth/ExternalAuthCalloutAuthenticator.cs
|
||||
LINE: 12
|
||||
CATEGORY: MissingDoc
|
||||
SEVERITY: Error
|
||||
MEMBER: Constructor
|
||||
SIGNATURE: ExternalAuthCalloutAuthenticator(IExternalAuthClient client, TimeSpan timeout)
|
||||
MESSAGE: Constructor 'ExternalAuthCalloutAuthenticator(IExternalAuthClient client, TimeSpan timeout)' is missing XML documentation.
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/Auth/ExternalAuthCalloutAuthenticator.cs
|
||||
LINE: 18
|
||||
CATEGORY: MissingDoc
|
||||
SEVERITY: Error
|
||||
MEMBER: Method
|
||||
SIGNATURE: Authenticate(ClientAuthContext context)
|
||||
MESSAGE: Method 'Authenticate(ClientAuthContext context)' is missing XML documentation.
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/Auth/Jwt/AccountClaims.cs
|
||||
LINE: 102
|
||||
CATEGORY: MissingDoc
|
||||
SEVERITY: Error
|
||||
MEMBER: Property
|
||||
SIGNATURE: MaxStreams
|
||||
MESSAGE: Property 'MaxStreams' is missing XML documentation
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/Auth/Jwt/AccountClaims.cs
|
||||
LINE: 105
|
||||
CATEGORY: MissingDoc
|
||||
SEVERITY: Error
|
||||
MEMBER: Property
|
||||
SIGNATURE: Tier
|
||||
MESSAGE: Property 'Tier' is missing XML documentation
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/Auth/Jwt/JwtConnectionTypes.cs
|
||||
LINE: 18
|
||||
CATEGORY: MissingDoc
|
||||
SEVERITY: Error
|
||||
MEMBER: Method
|
||||
SIGNATURE: Convert(IEnumerable<string>? values)
|
||||
MESSAGE: Method 'Convert(IEnumerable<string>? values)' is missing XML documentation.
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/Auth/Jwt/NatsJwt.cs
|
||||
LINE: 226
|
||||
CATEGORY: MissingDoc
|
||||
SEVERITY: Error
|
||||
MEMBER: Property
|
||||
SIGNATURE: Algorithm
|
||||
MESSAGE: Property 'Algorithm' is missing XML documentation
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/Auth/Jwt/NatsJwt.cs
|
||||
LINE: 230
|
||||
CATEGORY: MissingDoc
|
||||
SEVERITY: Error
|
||||
MEMBER: Property
|
||||
SIGNATURE: Type
|
||||
MESSAGE: Property 'Type' is missing XML documentation
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/Auth/JwtAuthenticator.cs
|
||||
LINE: 15
|
||||
CATEGORY: MissingDoc
|
||||
SEVERITY: Error
|
||||
MEMBER: Constructor
|
||||
SIGNATURE: JwtAuthenticator(string[] trustedKeys, IAccountResolver resolver)
|
||||
MESSAGE: Constructor 'JwtAuthenticator(string[] trustedKeys, IAccountResolver resolver)' is missing XML documentation.
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/Auth/JwtAuthenticator.cs
|
||||
LINE: 21
|
||||
CATEGORY: MissingDoc
|
||||
SEVERITY: Error
|
||||
MEMBER: Method
|
||||
SIGNATURE: Authenticate(ClientAuthContext context)
|
||||
MESSAGE: Method 'Authenticate(ClientAuthContext context)' is missing XML documentation.
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/Auth/NKeyAuthenticator.cs
|
||||
LINE: 21
|
||||
CATEGORY: MissingDoc
|
||||
SEVERITY: Error
|
||||
MEMBER: Method
|
||||
SIGNATURE: Authenticate(ClientAuthContext context)
|
||||
MESSAGE: Method 'Authenticate(ClientAuthContext context)' is missing XML documentation.
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/Auth/ProxyAuthenticator.cs
|
||||
LINE: 5
|
||||
CATEGORY: MissingDoc
|
||||
SEVERITY: Error
|
||||
MEMBER: Method
|
||||
SIGNATURE: Authenticate(ClientAuthContext context)
|
||||
MESSAGE: Method 'Authenticate(ClientAuthContext context)' is missing XML documentation.
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/Auth/SimpleUserPasswordAuthenticator.cs
|
||||
LINE: 17
|
||||
CATEGORY: MissingDoc
|
||||
SEVERITY: Error
|
||||
MEMBER: Constructor
|
||||
SIGNATURE: SimpleUserPasswordAuthenticator(string username, string password)
|
||||
MESSAGE: Constructor 'SimpleUserPasswordAuthenticator(string username, string password)' is missing XML documentation.
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/Auth/SimpleUserPasswordAuthenticator.cs
|
||||
LINE: 23
|
||||
CATEGORY: MissingDoc
|
||||
SEVERITY: Error
|
||||
MEMBER: Method
|
||||
SIGNATURE: Authenticate(ClientAuthContext context)
|
||||
MESSAGE: Method 'Authenticate(ClientAuthContext context)' is missing XML documentation.
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/Auth/TokenAuthenticator.cs
|
||||
LINE: 10
|
||||
CATEGORY: MissingDoc
|
||||
SEVERITY: Error
|
||||
MEMBER: Constructor
|
||||
SIGNATURE: TokenAuthenticator(string token)
|
||||
MESSAGE: Constructor 'TokenAuthenticator(string token)' is missing XML documentation.
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/Auth/TokenAuthenticator.cs
|
||||
LINE: 15
|
||||
CATEGORY: MissingDoc
|
||||
SEVERITY: Error
|
||||
MEMBER: Method
|
||||
SIGNATURE: Authenticate(ClientAuthContext context)
|
||||
MESSAGE: Method 'Authenticate(ClientAuthContext context)' is missing XML documentation.
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/Auth/UserPasswordAuthenticator.cs
|
||||
LINE: 16
|
||||
CATEGORY: MissingDoc
|
||||
SEVERITY: Error
|
||||
MEMBER: Constructor
|
||||
SIGNATURE: UserPasswordAuthenticator(IEnumerable<User> users)
|
||||
MESSAGE: Constructor 'UserPasswordAuthenticator(IEnumerable<User> users)' is missing XML documentation.
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/Auth/UserPasswordAuthenticator.cs
|
||||
LINE: 23
|
||||
CATEGORY: MissingDoc
|
||||
SEVERITY: Error
|
||||
MEMBER: Method
|
||||
SIGNATURE: Authenticate(ClientAuthContext context)
|
||||
MESSAGE: Method 'Authenticate(ClientAuthContext context)' is missing XML documentation.
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/ClientClosedReason.cs
|
||||
LINE: 31
|
||||
CATEGORY: MissingDoc
|
||||
SEVERITY: Error
|
||||
MEMBER: Method
|
||||
SIGNATURE: ToReasonString(ClientClosedReason reason)
|
||||
MESSAGE: Method 'ToReasonString(ClientClosedReason reason)' is missing XML documentation.
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/ClientKind.cs
|
||||
LINE: 20
|
||||
CATEGORY: MissingDoc
|
||||
SEVERITY: Error
|
||||
MEMBER: Method
|
||||
SIGNATURE: IsInternal(ClientKind kind)
|
||||
MESSAGE: Method 'IsInternal(ClientKind kind)' is missing XML documentation.
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/Configuration/NatsConfLexer.cs
|
||||
LINE: 66
|
||||
CATEGORY: MissingDoc
|
||||
SEVERITY: Error
|
||||
MEMBER: Method
|
||||
SIGNATURE: Tokenize(string input)
|
||||
MESSAGE: Method 'Tokenize(string input)' is missing XML documentation.
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/Imports/ServiceLatency.cs
|
||||
LINE: 5
|
||||
CATEGORY: MissingDoc
|
||||
SEVERITY: Error
|
||||
MEMBER: Property
|
||||
SIGNATURE: SamplingPercentage
|
||||
MESSAGE: Property 'SamplingPercentage' is missing XML documentation
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/Imports/ServiceLatency.cs
|
||||
LINE: 6
|
||||
CATEGORY: MissingDoc
|
||||
SEVERITY: Error
|
||||
MEMBER: Property
|
||||
SIGNATURE: Subject
|
||||
MESSAGE: Property 'Subject' is missing XML documentation
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/Imports/StreamExport.cs
|
||||
LINE: 5
|
||||
CATEGORY: MissingDoc
|
||||
SEVERITY: Error
|
||||
MEMBER: Property
|
||||
SIGNATURE: Auth
|
||||
MESSAGE: Property 'Auth' is missing XML documentation
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/IO/AdaptiveReadBuffer.cs
|
||||
LINE: 14
|
||||
CATEGORY: MissingDoc
|
||||
SEVERITY: Error
|
||||
MEMBER: Property
|
||||
SIGNATURE: CurrentSize
|
||||
MESSAGE: Property 'CurrentSize' is missing XML documentation
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/IO/AdaptiveReadBuffer.cs
|
||||
LINE: 19
|
||||
CATEGORY: MissingDoc
|
||||
SEVERITY: Error
|
||||
MEMBER: Method
|
||||
SIGNATURE: RecordRead(int bytesRead)
|
||||
MESSAGE: Method 'RecordRead(int bytesRead)' is missing XML documentation.
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/JetStream/Api/Handlers/AccountApiHandlers.cs
|
||||
LINE: 5
|
||||
CATEGORY: MissingDoc
|
||||
SEVERITY: Error
|
||||
MEMBER: Method
|
||||
SIGNATURE: HandleInfo(StreamManager streams, ConsumerManager consumers)
|
||||
MESSAGE: Method 'HandleInfo(StreamManager streams, ConsumerManager consumers)' is missing XML documentation.
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/JetStream/Api/Handlers/DirectApiHandlers.cs
|
||||
LINE: 10
|
||||
CATEGORY: MissingDoc
|
||||
SEVERITY: Error
|
||||
MEMBER: Method
|
||||
SIGNATURE: HandleGet(string subject, ReadOnlySpan<byte> payload, StreamManager streamManager)
|
||||
MESSAGE: Method 'HandleGet(string subject, ReadOnlySpan<byte> payload, StreamManager streamManager)' is missing XML documentation.
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/JetStream/Api/JetStreamApiError.cs
|
||||
LINE: 5
|
||||
CATEGORY: MissingDoc
|
||||
SEVERITY: Error
|
||||
MEMBER: Property
|
||||
SIGNATURE: Code
|
||||
MESSAGE: Property 'Code' is missing XML documentation
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/JetStream/Api/JetStreamApiError.cs
|
||||
LINE: 6
|
||||
CATEGORY: MissingDoc
|
||||
SEVERITY: Error
|
||||
MEMBER: Property
|
||||
SIGNATURE: Description
|
||||
MESSAGE: Property 'Description' is missing XML documentation
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/JetStream/Cluster/AssetPlacementPlanner.cs
|
||||
LINE: 7
|
||||
CATEGORY: MissingDoc
|
||||
SEVERITY: Error
|
||||
MEMBER: Constructor
|
||||
SIGNATURE: AssetPlacementPlanner(int nodes)
|
||||
MESSAGE: Constructor 'AssetPlacementPlanner(int nodes)' is missing XML documentation.
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/JetStream/Cluster/AssetPlacementPlanner.cs
|
||||
LINE: 12
|
||||
CATEGORY: MissingDoc
|
||||
SEVERITY: Error
|
||||
MEMBER: Method
|
||||
SIGNATURE: PlanReplicas(int replicas)
|
||||
MESSAGE: Method 'PlanReplicas(int replicas)' is missing XML documentation.
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/JetStream/Cluster/MetaSnapshotCodec.cs
|
||||
LINE: 28
|
||||
CATEGORY: MissingParam
|
||||
SEVERITY: Warning
|
||||
MEMBER: Method
|
||||
SIGNATURE: Encode(Dictionary<string, StreamAssignment> assignments)
|
||||
MESSAGE: Method 'Encode(Dictionary<string, StreamAssignment> assignments)' is missing <param name="assignments"> documentation.
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/JetStream/Cluster/MetaSnapshotCodec.cs
|
||||
LINE: 46
|
||||
CATEGORY: MissingParam
|
||||
SEVERITY: Warning
|
||||
MEMBER: Method
|
||||
SIGNATURE: Decode(byte[] data)
|
||||
MESSAGE: Method 'Decode(byte[] data)' is missing <param name="data"> documentation.
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/JetStream/Consumers/DeliveryInterestTracker.cs
|
||||
LINE: 15
|
||||
CATEGORY: MissingDoc
|
||||
SEVERITY: Error
|
||||
MEMBER: Constructor
|
||||
SIGNATURE: DeliveryInterestTracker(TimeSpan? inactiveTimeout)
|
||||
MESSAGE: Constructor 'DeliveryInterestTracker(TimeSpan? inactiveTimeout)' is missing XML documentation.
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/JetStream/Storage/FileStore.cs
|
||||
LINE: 2880
|
||||
CATEGORY: MissingParam
|
||||
SEVERITY: Warning
|
||||
MEMBER: Method
|
||||
SIGNATURE: EvictBlockNoSync(int blockId)
|
||||
MESSAGE: Method 'EvictBlockNoSync(int blockId)' is missing <param name="blockId"> documentation.
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/Monitoring/ConnzHandler.cs
|
||||
LINE: 13
|
||||
CATEGORY: MissingDoc
|
||||
SEVERITY: Error
|
||||
MEMBER: Method
|
||||
SIGNATURE: HandleConnz(HttpContext ctx)
|
||||
MESSAGE: Method 'HandleConnz(HttpContext ctx)' is missing XML documentation.
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/Monitoring/SubszHandler.cs
|
||||
LINE: 12
|
||||
CATEGORY: MissingDoc
|
||||
SEVERITY: Error
|
||||
MEMBER: Method
|
||||
SIGNATURE: HandleSubsz(HttpContext ctx)
|
||||
MESSAGE: Method 'HandleSubsz(HttpContext ctx)' is missing XML documentation.
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/Mqtt/MqttStreamInitializer.cs
|
||||
LINE: 21
|
||||
CATEGORY: MissingDoc
|
||||
SEVERITY: Error
|
||||
MEMBER: Constructor
|
||||
SIGNATURE: MqttStreamInitializer(StreamManager streamManager)
|
||||
MESSAGE: Constructor 'MqttStreamInitializer(StreamManager streamManager)' is missing XML documentation.
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/Protocol/ClientCommandMatrix.cs
|
||||
LINE: 5
|
||||
CATEGORY: MissingDoc
|
||||
SEVERITY: Error
|
||||
MEMBER: Method
|
||||
SIGNATURE: IsAllowed(global::NATS.Server.ClientKind kind, string? op)
|
||||
MESSAGE: Method 'IsAllowed(global::NATS.Server.ClientKind kind, string? op)' is missing XML documentation.
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/Raft/RaftMembership.cs
|
||||
LINE: 30
|
||||
CATEGORY: MissingParam
|
||||
SEVERITY: Warning
|
||||
MEMBER: Method
|
||||
SIGNATURE: TryParse(string command)
|
||||
MESSAGE: Method 'TryParse(string command)' is missing <param name="command"> documentation.
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/Raft/RaftSnapshotCheckpoint.cs
|
||||
LINE: 34
|
||||
CATEGORY: MissingParam
|
||||
SEVERITY: Warning
|
||||
MEMBER: Method
|
||||
SIGNATURE: AddChunk(byte[] chunk)
|
||||
MESSAGE: Method 'AddChunk(byte[] chunk)' is missing <param name="chunk"> documentation.
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/Raft/RaftStateExtensions.cs
|
||||
LINE: 9
|
||||
CATEGORY: MissingDoc
|
||||
SEVERITY: Error
|
||||
MEMBER: Method
|
||||
SIGNATURE: String(RaftState state)
|
||||
MESSAGE: Method 'String(RaftState state)' is missing XML documentation.
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/Raft/SnapshotChunkEnumerator.cs
|
||||
LINE: 89
|
||||
CATEGORY: MissingDoc
|
||||
SEVERITY: Error
|
||||
MEMBER: Method
|
||||
SIGNATURE: GetEnumerator()
|
||||
MESSAGE: Method 'GetEnumerator()' is missing XML documentation.
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/Subscriptions/RemoteSubscription.cs
|
||||
LINE: 11
|
||||
CATEGORY: MissingDoc
|
||||
SEVERITY: Error
|
||||
MEMBER: Method
|
||||
SIGNATURE: Removal(string subject, string? queue, string routeId, string account)
|
||||
MESSAGE: Method 'Removal(string subject, string? queue, string routeId, string account)' is missing XML documentation.
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/Subscriptions/RoutedSubKey.cs
|
||||
LINE: 5
|
||||
CATEGORY: MissingDoc
|
||||
SEVERITY: Error
|
||||
MEMBER: Method
|
||||
SIGNATURE: FromRemoteSubscription(RemoteSubscription sub)
|
||||
MESSAGE: Method 'FromRemoteSubscription(RemoteSubscription sub)' is missing XML documentation.
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/Tls/TlsConnectionWrapper.cs
|
||||
LINE: 15
|
||||
CATEGORY: MissingDoc
|
||||
SEVERITY: Error
|
||||
MEMBER: Method
|
||||
SIGNATURE: NegotiateAsync(Socket socket, Stream networkStream, NatsOptions options, SslServerAuthenticationOptions? sslOptions, ServerInfo serverInfo, ILogger logger, CancellationToken ct)
|
||||
MESSAGE: Method 'NegotiateAsync(Socket socket, Stream networkStream, NatsOptions options, SslServerAuthenticationOptions? sslOptions, ServerInfo serverInfo, ILogger logger, CancellationToken ct)' is missing XML documentation.
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/WebSocket/WebSocketOptionsValidator.cs
|
||||
LINE: 18
|
||||
CATEGORY: MissingDoc
|
||||
SEVERITY: Error
|
||||
MEMBER: Method
|
||||
SIGNATURE: Validate(NatsOptions options)
|
||||
MESSAGE: Method 'Validate(NatsOptions options)' is missing XML documentation.
|
||||
|
||||
---
|
||||
|
||||
FILE: /Users/dohertj2/Desktop/natsdotnet/src/NATS.Server/WebSocket/WsConstants.cs
|
||||
LINE: 64
|
||||
CATEGORY: MissingDoc
|
||||
SEVERITY: Error
|
||||
MEMBER: Method
|
||||
SIGNATURE: IsControlFrame(int opcode)
|
||||
MESSAGE: Method 'IsControlFrame(int opcode)' is missing XML documentation.
|
||||
|
||||
8586
src-docs-issues.md
8586
src-docs-issues.md
File diff suppressed because it is too large
Load Diff
@@ -9,12 +9,26 @@ public sealed class ExternalAuthCalloutAuthenticator : IAuthenticator
|
||||
private readonly IExternalAuthClient _client;
|
||||
private readonly TimeSpan _timeout;
|
||||
|
||||
/// <summary>
|
||||
/// Creates an authenticator that delegates user validation to the external auth callout subject.
|
||||
/// This mirrors the NATS external authorization flow used for centralized policy decisions.
|
||||
/// </summary>
|
||||
/// <param name="client">Client used to publish authorization requests and receive decisions.</param>
|
||||
/// <param name="timeout">Maximum time to wait for an authorization decision.</param>
|
||||
public ExternalAuthCalloutAuthenticator(IExternalAuthClient client, TimeSpan timeout)
|
||||
{
|
||||
_client = client;
|
||||
_timeout = timeout;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Authenticates a client by calling the external authorization service and mapping the decision
|
||||
/// into a local identity/account context for the accepted connection.
|
||||
/// </summary>
|
||||
/// <param name="context">Connection authentication inputs received from the CONNECT payload.</param>
|
||||
/// <returns>
|
||||
/// An <see cref="AuthResult"/> when the callout allows the connection; otherwise <see langword="null"/>.
|
||||
/// </returns>
|
||||
public AuthResult? Authenticate(ClientAuthContext context)
|
||||
{
|
||||
using var cts = new CancellationTokenSource(_timeout);
|
||||
|
||||
@@ -99,9 +99,17 @@ public sealed class AccountLimits
|
||||
|
||||
public sealed class AccountJetStreamLimits
|
||||
{
|
||||
/// <summary>
|
||||
/// Maximum number of streams the account can create in JetStream.
|
||||
/// This limit protects cluster resources in multi-tenant deployments.
|
||||
/// </summary>
|
||||
[JsonPropertyName("max_streams")]
|
||||
public int MaxStreams { get; set; }
|
||||
|
||||
/// <summary>
|
||||
/// Optional JetStream service tier label assigned to the account (for example, dev or prod).
|
||||
/// Tier is used for policy and placement decisions in operator-managed environments.
|
||||
/// </summary>
|
||||
[JsonPropertyName("tier")]
|
||||
public string? Tier { get; set; }
|
||||
}
|
||||
|
||||
@@ -15,6 +15,14 @@ internal static class JwtConnectionTypes
|
||||
Standard, Websocket, Leafnode, LeafnodeWs, Mqtt, MqttWs, InProcess,
|
||||
];
|
||||
|
||||
/// <summary>
|
||||
/// Converts raw JWT allowed connection type values into normalized server constants
|
||||
/// and tracks whether any unknown connection types were supplied by policy.
|
||||
/// </summary>
|
||||
/// <param name="values">Allowed connection type values from user JWT claims.</param>
|
||||
/// <returns>
|
||||
/// A set of valid normalized types and a flag indicating whether unknown values were present.
|
||||
/// </returns>
|
||||
public static (HashSet<string> Valid, bool HasUnknown) Convert(IEnumerable<string>? values)
|
||||
{
|
||||
var valid = new HashSet<string>(StringComparer.Ordinal);
|
||||
|
||||
@@ -223,11 +223,11 @@ public sealed class JwtToken
|
||||
/// </summary>
|
||||
public sealed class JwtHeader
|
||||
{
|
||||
[System.Text.Json.Serialization.JsonPropertyName("alg")]
|
||||
/// <summary>JWT signing algorithm identifier (typically <c>ed25519-nkey</c> for NATS).</summary>
|
||||
[System.Text.Json.Serialization.JsonPropertyName("alg")]
|
||||
public string? Algorithm { get; set; }
|
||||
|
||||
[System.Text.Json.Serialization.JsonPropertyName("typ")]
|
||||
/// <summary>JWT type marker (typically <c>JWT</c>).</summary>
|
||||
[System.Text.Json.Serialization.JsonPropertyName("typ")]
|
||||
public string? Type { get; set; }
|
||||
}
|
||||
|
||||
@@ -12,12 +12,26 @@ public sealed class JwtAuthenticator : IAuthenticator
|
||||
private readonly string[] _trustedKeys;
|
||||
private readonly IAccountResolver _resolver;
|
||||
|
||||
/// <summary>
|
||||
/// Creates a JWT authenticator that trusts the provided operator keys and resolves account JWTs
|
||||
/// from the configured resolver source.
|
||||
/// </summary>
|
||||
/// <param name="trustedKeys">Trusted operator/signing keys allowed to issue account JWTs.</param>
|
||||
/// <param name="resolver">Resolver used to fetch account claims by account public key.</param>
|
||||
public JwtAuthenticator(string[] trustedKeys, IAccountResolver resolver)
|
||||
{
|
||||
_trustedKeys = trustedKeys;
|
||||
_resolver = resolver;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Authenticates a client using NATS JWT flow: decode user claims, resolve account claims,
|
||||
/// verify trust/signatures/revocations, and derive connection permissions and limits.
|
||||
/// </summary>
|
||||
/// <param name="context">CONNECT options and nonce/signature context for the client.</param>
|
||||
/// <returns>
|
||||
/// Populated authentication result when JWT policy allows the connection; otherwise <see langword="null"/>.
|
||||
/// </returns>
|
||||
public AuthResult? Authenticate(ClientAuthContext context)
|
||||
{
|
||||
var jwt = context.Opts.JWT;
|
||||
|
||||
@@ -18,6 +18,12 @@ public sealed class NKeyAuthenticator(IEnumerable<NKeyUser> nkeyUsers) : IAuthen
|
||||
u => u,
|
||||
StringComparer.Ordinal);
|
||||
|
||||
/// <summary>
|
||||
/// Authenticates a client by verifying its nonce signature with the presented NKey public key
|
||||
/// and returning the mapped account and permission context for that key.
|
||||
/// </summary>
|
||||
/// <param name="context">CONNECT payload plus server nonce used for signature verification.</param>
|
||||
/// <returns><see cref="AuthResult"/> for a valid NKey user; otherwise <see langword="null"/>.</returns>
|
||||
public AuthResult? Authenticate(ClientAuthContext context)
|
||||
{
|
||||
var clientNkey = context.Opts.Nkey;
|
||||
|
||||
@@ -2,6 +2,12 @@ namespace NATS.Server.Auth;
|
||||
|
||||
public sealed class ProxyAuthenticator(ProxyAuthOptions options) : IAuthenticator
|
||||
{
|
||||
/// <summary>
|
||||
/// Authenticates a client from a trusted proxy identity prefix and maps it to the configured account.
|
||||
/// This supports edge proxies that perform upstream auth and pass a canonical user principal.
|
||||
/// </summary>
|
||||
/// <param name="context">Client credentials and connection metadata from CONNECT.</param>
|
||||
/// <returns><see cref="AuthResult"/> when proxy-auth rules match; otherwise <see langword="null"/>.</returns>
|
||||
public AuthResult? Authenticate(ClientAuthContext context)
|
||||
{
|
||||
if (!options.Enabled)
|
||||
|
||||
@@ -14,12 +14,22 @@ public sealed class SimpleUserPasswordAuthenticator : IAuthenticator
|
||||
private readonly byte[] _expectedUsername;
|
||||
private readonly string _serverPassword;
|
||||
|
||||
/// <summary>
|
||||
/// Creates an authenticator for a single configured user credential pair from server options.
|
||||
/// </summary>
|
||||
/// <param name="username">Expected username for incoming client connections.</param>
|
||||
/// <param name="password">Expected password (plain or bcrypt hash) for that username.</param>
|
||||
public SimpleUserPasswordAuthenticator(string username, string password)
|
||||
{
|
||||
_expectedUsername = Encoding.UTF8.GetBytes(username);
|
||||
_serverPassword = password;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Authenticates the configured single user using constant-time comparisons and optional bcrypt verification.
|
||||
/// </summary>
|
||||
/// <param name="context">Client-provided username/password from CONNECT.</param>
|
||||
/// <returns><see cref="AuthResult"/> on successful validation; otherwise <see langword="null"/>.</returns>
|
||||
public AuthResult? Authenticate(ClientAuthContext context)
|
||||
{
|
||||
var clientUsername = context.Opts.Username;
|
||||
|
||||
@@ -7,11 +7,20 @@ public sealed class TokenAuthenticator : IAuthenticator
|
||||
{
|
||||
private readonly byte[] _expectedToken;
|
||||
|
||||
/// <summary>
|
||||
/// Creates a token authenticator for deployments that use shared bearer tokens.
|
||||
/// </summary>
|
||||
/// <param name="token">Server-configured token value clients must present in CONNECT.</param>
|
||||
public TokenAuthenticator(string token)
|
||||
{
|
||||
_expectedToken = Encoding.UTF8.GetBytes(token);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Authenticates the client using constant-time token comparison to avoid timing leakage.
|
||||
/// </summary>
|
||||
/// <param name="context">Client connection options containing the presented token.</param>
|
||||
/// <returns><see cref="AuthResult"/> for a matching token; otherwise <see langword="null"/>.</returns>
|
||||
public AuthResult? Authenticate(ClientAuthContext context)
|
||||
{
|
||||
var clientToken = context.Opts.Token;
|
||||
|
||||
@@ -13,6 +13,10 @@ public sealed class UserPasswordAuthenticator : IAuthenticator
|
||||
{
|
||||
private readonly Dictionary<string, User> _users;
|
||||
|
||||
/// <summary>
|
||||
/// Creates an authenticator for a configured user set and builds a fast lookup by username.
|
||||
/// </summary>
|
||||
/// <param name="users">Configured users with account mappings and permission scopes.</param>
|
||||
public UserPasswordAuthenticator(IEnumerable<User> users)
|
||||
{
|
||||
_users = new Dictionary<string, User>(StringComparer.Ordinal);
|
||||
@@ -20,6 +24,11 @@ public sealed class UserPasswordAuthenticator : IAuthenticator
|
||||
_users[user.Username] = user;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Authenticates a username/password client and returns account, permissions, and connection expiry metadata.
|
||||
/// </summary>
|
||||
/// <param name="context">Client CONNECT credentials.</param>
|
||||
/// <returns><see cref="AuthResult"/> when credentials match; otherwise <see langword="null"/>.</returns>
|
||||
public AuthResult? Authenticate(ClientAuthContext context)
|
||||
{
|
||||
var username = context.Opts.Username;
|
||||
|
||||
@@ -28,6 +28,11 @@ public enum ClientClosedReason
|
||||
|
||||
public static class ClientClosedReasonExtensions
|
||||
{
|
||||
/// <summary>
|
||||
/// Converts an internal close reason enum into the human-readable text exposed by monitoring endpoints.
|
||||
/// </summary>
|
||||
/// <param name="reason">Internal close classification captured at disconnect time.</param>
|
||||
/// <returns>Display string used in `/connz` and related operational diagnostics.</returns>
|
||||
public static string ToReasonString(this ClientClosedReason reason) => reason switch
|
||||
{
|
||||
ClientClosedReason.None => "",
|
||||
|
||||
@@ -17,6 +17,12 @@ public enum ClientKind
|
||||
|
||||
public static class ClientKindExtensions
|
||||
{
|
||||
/// <summary>
|
||||
/// Indicates whether a client kind represents internal server infrastructure traffic
|
||||
/// rather than an external end-user connection.
|
||||
/// </summary>
|
||||
/// <param name="kind">Connection kind being evaluated.</param>
|
||||
/// <returns><see langword="true"/> for internal kinds such as system and JetStream.</returns>
|
||||
public static bool IsInternal(this ClientKind kind) =>
|
||||
kind is ClientKind.System or ClientKind.JetStream or ClientKind.Account;
|
||||
}
|
||||
|
||||
@@ -63,6 +63,12 @@ public sealed class NatsConfLexer
|
||||
_ilstart = 0;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Tokenizes a NATS configuration document into lexical tokens consumed by the config parser.
|
||||
/// The lexer preserves Go-compatible token rules for production config parity.
|
||||
/// </summary>
|
||||
/// <param name="input">Raw configuration text in NATS conf syntax.</param>
|
||||
/// <returns>Ordered token stream including error tokens when malformed input is encountered.</returns>
|
||||
public static IReadOnlyList<Token> Tokenize(string input)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(input);
|
||||
|
||||
@@ -11,11 +11,19 @@ public sealed class AdaptiveReadBuffer
|
||||
private int _target = 4096;
|
||||
private int _consecutiveShortReads;
|
||||
|
||||
/// <summary>
|
||||
/// Current target buffer size used for the next socket read operation.
|
||||
/// </summary>
|
||||
public int CurrentSize => Math.Clamp(_target, 512, 64 * 1024);
|
||||
|
||||
/// <summary>Number of consecutive short reads since last full read or grow.</summary>
|
||||
public int ConsecutiveShortReads => _consecutiveShortReads;
|
||||
|
||||
/// <summary>
|
||||
/// Updates adaptive sizing state using the number of bytes returned by the latest read.
|
||||
/// Full reads bias toward growth for throughput, repeated short reads bias toward shrink to save memory.
|
||||
/// </summary>
|
||||
/// <param name="bytesRead">Byte count returned by the transport read.</param>
|
||||
public void RecordRead(int bytesRead)
|
||||
{
|
||||
if (bytesRead <= 0)
|
||||
|
||||
@@ -2,6 +2,13 @@ namespace NATS.Server.Imports;
|
||||
|
||||
public sealed class ServiceLatency
|
||||
{
|
||||
/// <summary>
|
||||
/// Percentage of service requests to sample for latency telemetry export.
|
||||
/// </summary>
|
||||
public int SamplingPercentage { get; init; } = 100;
|
||||
|
||||
/// <summary>
|
||||
/// Subject where sampled service latency observations are published.
|
||||
/// </summary>
|
||||
public string Subject { get; init; } = string.Empty;
|
||||
}
|
||||
|
||||
@@ -2,5 +2,8 @@ namespace NATS.Server.Imports;
|
||||
|
||||
public sealed class StreamExport
|
||||
{
|
||||
/// <summary>
|
||||
/// Authorization rules that govern which remote accounts may import this stream export.
|
||||
/// </summary>
|
||||
public ExportAuth Auth { get; init; } = new();
|
||||
}
|
||||
|
||||
@@ -2,6 +2,12 @@ namespace NATS.Server.JetStream.Api.Handlers;
|
||||
|
||||
public static class AccountApiHandlers
|
||||
{
|
||||
/// <summary>
|
||||
/// Builds JetStream account usage info for `$JS.API.INFO`, including current stream and consumer counts.
|
||||
/// </summary>
|
||||
/// <param name="streams">Stream manager for the account context.</param>
|
||||
/// <param name="consumers">Consumer manager for the account context.</param>
|
||||
/// <returns>API response payload containing JetStream account statistics.</returns>
|
||||
public static JetStreamApiResponse HandleInfo(StreamManager streams, ConsumerManager consumers)
|
||||
{
|
||||
return new JetStreamApiResponse
|
||||
|
||||
@@ -7,6 +7,13 @@ public static class DirectApiHandlers
|
||||
{
|
||||
private const string Prefix = JetStreamApiSubjects.DirectGet;
|
||||
|
||||
/// <summary>
|
||||
/// Handles direct message fetch requests by stream and sequence for `$JS.API.DIRECT.GET.*`.
|
||||
/// </summary>
|
||||
/// <param name="subject">API subject containing the target stream token.</param>
|
||||
/// <param name="payload">JSON payload with the requested sequence number.</param>
|
||||
/// <param name="streamManager">Stream manager used to read the stored message.</param>
|
||||
/// <returns>Direct message response or a not-found/validation error response.</returns>
|
||||
public static JetStreamApiResponse HandleGet(string subject, ReadOnlySpan<byte> payload, StreamManager streamManager)
|
||||
{
|
||||
var streamName = ExtractTrailingToken(subject, Prefix);
|
||||
|
||||
@@ -2,7 +2,14 @@ namespace NATS.Server.JetStream.Api;
|
||||
|
||||
public sealed class JetStreamApiError
|
||||
{
|
||||
/// <summary>
|
||||
/// Numeric JetStream API error code returned to clients for programmatic handling.
|
||||
/// </summary>
|
||||
public int Code { get; init; }
|
||||
|
||||
/// <summary>
|
||||
/// Human-readable description of the API error condition.
|
||||
/// </summary>
|
||||
public string Description { get; init; } = string.Empty;
|
||||
|
||||
/// <summary>
|
||||
|
||||
@@ -4,11 +4,20 @@ public sealed class AssetPlacementPlanner
|
||||
{
|
||||
private readonly int _nodes;
|
||||
|
||||
/// <summary>
|
||||
/// Creates a planner that allocates stream/consumer replicas across the available cluster node count.
|
||||
/// </summary>
|
||||
/// <param name="nodes">Number of eligible JetStream nodes in the placement domain.</param>
|
||||
public AssetPlacementPlanner(int nodes)
|
||||
{
|
||||
_nodes = Math.Max(nodes, 1);
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Produces a deterministic list of node slots for replica placement capped by available nodes.
|
||||
/// </summary>
|
||||
/// <param name="replicas">Requested replica count from stream or consumer configuration.</param>
|
||||
/// <returns>Ordered node slot identifiers selected for placement.</returns>
|
||||
public IReadOnlyList<int> PlanReplicas(int replicas)
|
||||
{
|
||||
var count = Math.Min(Math.Max(replicas, 1), _nodes);
|
||||
|
||||
@@ -25,6 +25,7 @@ internal static class MetaSnapshotCodec
|
||||
/// Encodes <paramref name="assignments"/> into the versioned, S2-compressed binary format.
|
||||
/// Go reference: jetstream_cluster.go:2075 encodeMetaSnapshot.
|
||||
/// </summary>
|
||||
/// <param name="assignments">Current stream placement assignments to persist into the meta snapshot.</param>
|
||||
public static byte[] Encode(Dictionary<string, StreamAssignment> assignments)
|
||||
{
|
||||
var json = JsonSerializer.SerializeToUtf8Bytes(assignments, SerializerOptions);
|
||||
@@ -40,6 +41,7 @@ internal static class MetaSnapshotCodec
|
||||
/// Decodes a versioned, S2-compressed binary snapshot into a stream assignment map.
|
||||
/// Go reference: jetstream_cluster.go:2100 decodeMetaSnapshot.
|
||||
/// </summary>
|
||||
/// <param name="data">Versioned binary snapshot payload received from replicated state.</param>
|
||||
/// <exception cref="InvalidOperationException">
|
||||
/// Thrown when <paramref name="data"/> is too short or contains an unrecognised version.
|
||||
/// </exception>
|
||||
|
||||
@@ -12,6 +12,12 @@ public sealed class DeliveryInterestTracker
|
||||
private int _subscriberCount;
|
||||
private DateTime? _lastUnsubscribeUtc;
|
||||
|
||||
/// <summary>
|
||||
/// Creates a tracker for consumer delivery-subject interest with an optional inactivity timeout override.
|
||||
/// </summary>
|
||||
/// <param name="inactiveTimeout">
|
||||
/// Duration with zero subscribers before the consumer is considered inactive for cleanup.
|
||||
/// </param>
|
||||
public DeliveryInterestTracker(TimeSpan? inactiveTimeout = null)
|
||||
{
|
||||
_inactiveTimeout = inactiveTimeout ?? TimeSpan.FromSeconds(30);
|
||||
|
||||
@@ -2877,6 +2877,7 @@ public sealed class FileStore : IStreamStore, IAsyncDisposable, IDisposable
|
||||
/// background flush loop's deferred sync queue).
|
||||
/// Used by <see cref="FileStore.RotateBlock"/> to avoid synchronous fsync on the hot path.
|
||||
/// </summary>
|
||||
/// <param name="blockId">Identifier of the message block whose pending cache entry should be removed.</param>
|
||||
public void EvictBlockNoSync(int blockId)
|
||||
{
|
||||
_entries.TryRemove(blockId, out _);
|
||||
|
||||
@@ -10,6 +10,11 @@ namespace NATS.Server.Monitoring;
|
||||
/// </summary>
|
||||
public sealed class ConnzHandler(NatsServer server)
|
||||
{
|
||||
/// <summary>
|
||||
/// Handles `/connz` monitor requests, applying filters/sorting/pagination over open and closed connections.
|
||||
/// </summary>
|
||||
/// <param name="ctx">HTTP request context containing query parameters for connection diagnostics.</param>
|
||||
/// <returns>Connection report payload compatible with NATS monitoring clients.</returns>
|
||||
public Connz HandleConnz(HttpContext ctx)
|
||||
{
|
||||
var opts = ParseQueryParams(ctx);
|
||||
|
||||
@@ -9,6 +9,11 @@ namespace NATS.Server.Monitoring;
|
||||
/// </summary>
|
||||
public sealed class SubszHandler(NatsServer server)
|
||||
{
|
||||
/// <summary>
|
||||
/// Handles `/subsz` monitor requests, returning subscription totals and optional per-subscription detail.
|
||||
/// </summary>
|
||||
/// <param name="ctx">HTTP request context containing account/subject/paging filter parameters.</param>
|
||||
/// <returns>Subscription diagnostics payload for operational introspection.</returns>
|
||||
public Subsz HandleSubsz(HttpContext ctx)
|
||||
{
|
||||
var opts = ParseQueryParams(ctx);
|
||||
|
||||
@@ -18,6 +18,10 @@ public sealed class MqttStreamInitializer
|
||||
private volatile bool _initialized;
|
||||
private readonly Lock _initLock = new();
|
||||
|
||||
/// <summary>
|
||||
/// Creates an initializer that provisions account-scoped MQTT persistence streams on demand.
|
||||
/// </summary>
|
||||
/// <param name="streamManager">JetStream manager used to create and query internal MQTT streams.</param>
|
||||
public MqttStreamInitializer(StreamManager streamManager)
|
||||
{
|
||||
_streamManager = streamManager;
|
||||
|
||||
@@ -2,6 +2,13 @@ namespace NATS.Server.Protocol;
|
||||
|
||||
public sealed class ClientCommandMatrix
|
||||
{
|
||||
/// <summary>
|
||||
/// Validates whether a protocol operation is permitted for a given connection kind
|
||||
/// to enforce route/gateway/leaf command boundaries.
|
||||
/// </summary>
|
||||
/// <param name="kind">Kind of connection issuing the command.</param>
|
||||
/// <param name="op">Protocol operation verb (for example RS+, A-, LMSG).</param>
|
||||
/// <returns><see langword="true"/> when the command is allowed for that connection kind.</returns>
|
||||
public bool IsAllowed(global::NATS.Server.ClientKind kind, string? op)
|
||||
{
|
||||
if (string.IsNullOrWhiteSpace(op))
|
||||
|
||||
@@ -27,6 +27,7 @@ public readonly record struct RaftMembershipChange(RaftMembershipChangeType Type
|
||||
/// Parses a log entry command string back into a membership change.
|
||||
/// Returns null if the command is not a membership change.
|
||||
/// </summary>
|
||||
/// <param name="command">Serialized membership command from a replicated RAFT log entry.</param>
|
||||
public static RaftMembershipChange? TryParse(string command)
|
||||
{
|
||||
var colonIndex = command.IndexOf(':');
|
||||
|
||||
@@ -31,6 +31,7 @@ public sealed class RaftSnapshotCheckpoint
|
||||
/// <summary>
|
||||
/// Adds a chunk of snapshot data for streaming assembly.
|
||||
/// </summary>
|
||||
/// <param name="chunk">One snapshot payload fragment received during InstallSnapshot transfer.</param>
|
||||
public void AddChunk(byte[] chunk) => _chunks.Add(chunk);
|
||||
|
||||
/// <summary>
|
||||
|
||||
@@ -6,6 +6,11 @@ namespace NATS.Server.Raft;
|
||||
/// </summary>
|
||||
public static class RaftStateExtensions
|
||||
{
|
||||
/// <summary>
|
||||
/// Returns the canonical state label used in RAFT diagnostics and status output.
|
||||
/// </summary>
|
||||
/// <param name="state">Current RAFT node state.</param>
|
||||
/// <returns>Stable, human-readable state name.</returns>
|
||||
public static string String(this RaftState state) =>
|
||||
state switch
|
||||
{
|
||||
|
||||
@@ -66,7 +66,10 @@ public sealed class SnapshotChunkEnumerator : IEnumerable<byte[]>
|
||||
/// </summary>
|
||||
public int ChunkCount => _data.Length == 0 ? 1 : ((_data.Length + _chunkSize - 1) / _chunkSize);
|
||||
|
||||
/// <inheritdoc />
|
||||
/// <summary>
|
||||
/// Enumerates snapshot chunks in transmission order for InstallSnapshot streaming.
|
||||
/// </summary>
|
||||
/// <returns>Sequence of byte arrays, each no larger than the configured chunk size.</returns>
|
||||
public IEnumerator<byte[]> GetEnumerator()
|
||||
{
|
||||
if (_data.Length == 0)
|
||||
@@ -86,5 +89,6 @@ public sealed class SnapshotChunkEnumerator : IEnumerable<byte[]>
|
||||
}
|
||||
}
|
||||
|
||||
/// <inheritdoc />
|
||||
System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator() => GetEnumerator();
|
||||
}
|
||||
|
||||
@@ -8,6 +8,14 @@ public sealed record RemoteSubscription(
|
||||
int QueueWeight = 1,
|
||||
bool IsRemoval = false)
|
||||
{
|
||||
/// <summary>
|
||||
/// Creates a tombstone record that propagates remote subscription removal across route links.
|
||||
/// </summary>
|
||||
/// <param name="subject">Subject being removed from remote interest tracking.</param>
|
||||
/// <param name="queue">Optional queue group associated with the removed subscription.</param>
|
||||
/// <param name="routeId">Route identifier where the removal originated.</param>
|
||||
/// <param name="account">Account namespace for the subscription removal.</param>
|
||||
/// <returns>A <see cref="RemoteSubscription"/> marked as a removal operation.</returns>
|
||||
public static RemoteSubscription Removal(string subject, string? queue, string routeId, string account = "$G")
|
||||
=> new(subject, queue, routeId, account, QueueWeight: 1, IsRemoval: true);
|
||||
}
|
||||
|
||||
@@ -2,6 +2,11 @@ namespace NATS.Server.Subscriptions;
|
||||
|
||||
internal readonly record struct RoutedSubKey(string RouteId, string Account, string Subject, string? Queue)
|
||||
{
|
||||
/// <summary>
|
||||
/// Projects a remote subscription announcement into the canonical routed-subscription key shape.
|
||||
/// </summary>
|
||||
/// <param name="sub">Remote subscription metadata received from a route peer.</param>
|
||||
/// <returns>Deduplication key used by routed subscription indexes.</returns>
|
||||
public static RoutedSubKey FromRemoteSubscription(RemoteSubscription sub)
|
||||
=> new(sub.RouteId, sub.Account, sub.Subject, sub.Queue);
|
||||
}
|
||||
|
||||
@@ -12,6 +12,18 @@ public static class TlsConnectionWrapper
|
||||
{
|
||||
private const byte TlsRecordMarker = 0x16;
|
||||
|
||||
/// <summary>
|
||||
/// Negotiates client transport security according to server TLS mode (required, mixed, or TLS-first),
|
||||
/// including INFO preface behavior and optional pinned certificate validation.
|
||||
/// </summary>
|
||||
/// <param name="socket">Accepted client socket.</param>
|
||||
/// <param name="networkStream">Base transport stream over the accepted socket.</param>
|
||||
/// <param name="options">Server TLS and protocol options controlling negotiation mode.</param>
|
||||
/// <param name="sslOptions">TLS server authentication options, or <see langword="null"/> when TLS is disabled.</param>
|
||||
/// <param name="serverInfo">Server INFO payload template to send before/after TLS handshake.</param>
|
||||
/// <param name="logger">Logger used for TLS negotiation diagnostics.</param>
|
||||
/// <param name="ct">Cancellation token for connection startup and handshake operations.</param>
|
||||
/// <returns>Negotiated stream and a flag indicating whether INFO was already written.</returns>
|
||||
public static async Task<(Stream stream, bool infoAlreadySent)> NegotiateAsync(
|
||||
Socket socket,
|
||||
Stream networkStream,
|
||||
|
||||
@@ -15,6 +15,12 @@ public static class WebSocketOptionsValidator
|
||||
"Sec-WebSocket-Protocol",
|
||||
};
|
||||
|
||||
/// <summary>
|
||||
/// Validates websocket listener settings against NATS auth, TLS, and header constraints
|
||||
/// to fail fast on incompatible runtime configuration.
|
||||
/// </summary>
|
||||
/// <param name="options">Server options containing websocket and global auth/TLS configuration.</param>
|
||||
/// <returns>Validation result with error details when the websocket configuration is invalid.</returns>
|
||||
public static WebSocketOptionsValidationResult Validate(NatsOptions options)
|
||||
{
|
||||
ArgumentNullException.ThrowIfNull(options);
|
||||
|
||||
@@ -61,6 +61,11 @@ public static class WsConstants
|
||||
// Decompression trailer appended before decompressing (RFC 7692 Section 7.2.2)
|
||||
public static readonly byte[] DecompressTrailer = [0x00, 0x00, 0xff, 0xff];
|
||||
|
||||
/// <summary>
|
||||
/// Determines whether an opcode is a WebSocket control frame (close, ping, or pong).
|
||||
/// </summary>
|
||||
/// <param name="opcode">RFC 6455 opcode from the incoming frame header.</param>
|
||||
/// <returns><see langword="true"/> when the frame must follow control-frame size/fragmentation rules.</returns>
|
||||
public static bool IsControlFrame(int opcode) => opcode >= CloseMessage;
|
||||
}
|
||||
|
||||
|
||||
@@ -26,29 +26,26 @@ static async Task<int> ProgramMain(string[] args)
|
||||
{
|
||||
try
|
||||
{
|
||||
if (args.Length == 0)
|
||||
var options = ParseArguments(args);
|
||||
if (options is null)
|
||||
{
|
||||
Console.Error.WriteLine("Usage: DtpSnapshotExtractor <snapshot.dtp>");
|
||||
Console.Error.WriteLine("Usage: DtpSnapshotExtractor <snapshot.dtp> [--top N] [--filter text] [--flat] [--include-idle]");
|
||||
return 2;
|
||||
}
|
||||
|
||||
var snapshotPath = Path.GetFullPath(args[0]);
|
||||
if (!File.Exists(snapshotPath))
|
||||
if (!File.Exists(options.SnapshotPath))
|
||||
{
|
||||
Console.Error.WriteLine($"Snapshot not found: {snapshotPath}");
|
||||
Console.Error.WriteLine($"Snapshot not found: {options.SnapshotPath}");
|
||||
return 2;
|
||||
}
|
||||
|
||||
var dotTraceAppDir = AppContext.GetData("APP_CONTEXT_DEPS_FILES") is not null
|
||||
? Environment.GetEnvironmentVariable("DOTTRACE_APP_DIR")
|
||||
: Environment.GetEnvironmentVariable("DOTTRACE_APP_DIR");
|
||||
RegisterAssemblyResolver(dotTraceAppDir);
|
||||
RegisterAssemblyResolver(Environment.GetEnvironmentVariable("DOTTRACE_APP_DIR"));
|
||||
|
||||
using var lifetimeDef = Lifetime.Define();
|
||||
var lifetime = lifetimeDef.Lifetime;
|
||||
var logger = Logger.GetLogger("DtpSnapshotExtractor");
|
||||
var masks = new SnapshotMasksComponent();
|
||||
var snapshotFile = FileSystemPath.Parse(snapshotPath);
|
||||
var snapshotFile = FileSystemPath.Parse(options.SnapshotPath);
|
||||
var container = new SnapshotStorageContainer(lifetime, logger, masks, snapshotFile);
|
||||
var callTreeSections = new CallTreeSections(container, masks);
|
||||
var headerSections = new HeaderSections(container);
|
||||
@@ -67,27 +64,32 @@ static async Task<int> ProgramMain(string[] args)
|
||||
var resolver = new SignatureResolver(masks, fuidConverter, assemblyProvider);
|
||||
|
||||
var headers = callTreeSections.AllHeaders().ToArray();
|
||||
var totalNodeCount = headers.Sum(header =>
|
||||
var expectedNodeCount = headers.Sum(header =>
|
||||
checked((int)((header.HeaderFull.SectionSize - header.HeaderFull.SectionHeaderSize) / header.HeaderFull.RecordSize())));
|
||||
|
||||
var nodes = new DfsNode<CallTreeSectionOffset, FunctionUID>[totalNodeCount];
|
||||
var totals = new DotTracePayload[totalNodeCount];
|
||||
var owns = new DotTracePayload[totalNodeCount];
|
||||
var nodes = new DfsNode<CallTreeSectionOffset, FunctionUID>[expectedNodeCount];
|
||||
var totals = new DotTracePayload[expectedNodeCount];
|
||||
var owns = new DotTracePayload[expectedNodeCount];
|
||||
|
||||
var readNodes = dfsReaders.GetNodesReaders(lifetime).ReadForwardOffsetsAscending(accessor.MinOffset, totalNodeCount, nodes, 0);
|
||||
var readTotals = totalPayloadReader.ReadForwardOffsetsAscending(accessor.MinOffset, totalNodeCount, totals, 0);
|
||||
var readOwns = ownPayloadReader.ReadForwardOffsetsAscending(accessor.MinOffset, totalNodeCount, owns, 0);
|
||||
var readNodes = dfsReaders.GetNodesReaders(lifetime).ReadForwardOffsetsAscending(accessor.MinOffset, expectedNodeCount, nodes, 0);
|
||||
var readTotals = totalPayloadReader.ReadForwardOffsetsAscending(accessor.MinOffset, expectedNodeCount, totals, 0);
|
||||
var readOwns = ownPayloadReader.ReadForwardOffsetsAscending(accessor.MinOffset, expectedNodeCount, owns, 0);
|
||||
|
||||
if (readNodes != totalNodeCount || readTotals != totalNodeCount || readOwns != totalNodeCount)
|
||||
throw new InvalidOperationException($"Snapshot read mismatch. nodes={readNodes}, totals={readTotals}, owns={readOwns}, expected={totalNodeCount}.");
|
||||
if (readNodes != expectedNodeCount || readTotals != expectedNodeCount || readOwns != expectedNodeCount)
|
||||
{
|
||||
throw new InvalidOperationException(
|
||||
$"Snapshot read mismatch. nodes={readNodes}, totals={readTotals}, owns={readOwns}, expected={expectedNodeCount}.");
|
||||
}
|
||||
|
||||
var nodeMap = new Dictionary<CallTreeSectionOffset, MutableNode>(totalNodeCount);
|
||||
foreach (var index in Enumerable.Range(0, totalNodeCount))
|
||||
var nodeMap = new Dictionary<CallTreeSectionOffset, MutableNode>(expectedNodeCount);
|
||||
foreach (var index in Enumerable.Range(0, expectedNodeCount))
|
||||
{
|
||||
var node = nodes[index];
|
||||
var signature = resolver.Resolve(node.Key);
|
||||
nodeMap[node.Offset] = new MutableNode
|
||||
{
|
||||
Offset = node.Offset,
|
||||
ParentOffset = node.ParentOffset,
|
||||
Id = node.Offset.ToString(),
|
||||
Name = signature.Name,
|
||||
Kind = signature.Kind,
|
||||
@@ -98,7 +100,7 @@ static async Task<int> ProgramMain(string[] args)
|
||||
};
|
||||
}
|
||||
|
||||
foreach (var index in Enumerable.Range(0, totalNodeCount))
|
||||
foreach (var index in Enumerable.Range(0, expectedNodeCount))
|
||||
{
|
||||
var node = nodes[index];
|
||||
if (!node.ParentOffset.IsValid)
|
||||
@@ -127,7 +129,7 @@ static async Task<int> ProgramMain(string[] args)
|
||||
InclusiveTime = rootNode.InclusiveTime,
|
||||
ExclusiveTime = 0,
|
||||
CallCount = rootNode.CallCount,
|
||||
Children = [CloneTree(header.Root, nodeMap, [])]
|
||||
Children = [CloneTree(rootNode, [])]
|
||||
});
|
||||
}
|
||||
|
||||
@@ -143,24 +145,41 @@ static async Task<int> ProgramMain(string[] args)
|
||||
Children = threadNodes
|
||||
};
|
||||
|
||||
var hotspots = BuildHotspots(nodeMap.Values);
|
||||
var filteredTree = string.IsNullOrWhiteSpace(options.NameFilter)
|
||||
? syntheticRoot
|
||||
: PruneTree(syntheticRoot, options.NameFilter) ?? CreateEmptyRoot();
|
||||
|
||||
var hotspotCandidates = FilterHotspotCandidates(nodeMap.Values, options);
|
||||
var hotspots = BuildHotspots(hotspotCandidates, options.Top);
|
||||
var summary = BuildSummary(syntheticRoot, hotspotCandidates, hotspots.Exclusive.FirstOrDefault());
|
||||
var payload = new OutputDocument
|
||||
{
|
||||
Snapshot = new SnapshotInfo
|
||||
{
|
||||
Path = snapshotPath,
|
||||
Path = options.SnapshotPath,
|
||||
PayloadType = "time",
|
||||
TimeUnit = "nanoseconds",
|
||||
ThreadCount = threadNodes.Count,
|
||||
NodeCount = nodeMap.Count
|
||||
NodeCount = nodeMap.Count,
|
||||
Diagnostics = new SnapshotDiagnostics
|
||||
{
|
||||
HeaderCount = headers.Length,
|
||||
ExpectedNodeCount = expectedNodeCount,
|
||||
ReadNodeCount = readNodes,
|
||||
ReadTotalPayloadCount = readTotals,
|
||||
ReadOwnPayloadCount = readOwns
|
||||
}
|
||||
},
|
||||
ThreadRoots = threadNodes.Select(node => new ThreadRootInfo
|
||||
Summary = summary,
|
||||
ThreadRoots = filteredTree.Children.Select(node => new ThreadRootInfo
|
||||
{
|
||||
Id = node.Id,
|
||||
Name = node.Name,
|
||||
InclusiveTime = node.InclusiveTime
|
||||
}).ToList(),
|
||||
Hotspots = hotspots,
|
||||
CallTree = syntheticRoot
|
||||
HotPaths = options.FlatPaths ? BuildHotPaths(filteredTree, options) : null,
|
||||
CallTree = filteredTree
|
||||
};
|
||||
|
||||
await JsonSerializer.SerializeAsync(Console.OpenStandardOutput(), payload, CreateJsonOptions());
|
||||
@@ -174,11 +193,54 @@ static async Task<int> ProgramMain(string[] args)
|
||||
}
|
||||
}
|
||||
|
||||
static ExtractorOptions? ParseArguments(string[] args)
|
||||
{
|
||||
if (args.Length == 0)
|
||||
return null;
|
||||
|
||||
var snapshotPath = Path.GetFullPath(args[0]);
|
||||
var top = 200;
|
||||
string? nameFilter = null;
|
||||
var flatPaths = false;
|
||||
var excludeIdle = true;
|
||||
|
||||
for (var index = 1; index < args.Length; index++)
|
||||
{
|
||||
switch (args[index])
|
||||
{
|
||||
case "--top":
|
||||
if (++index >= args.Length || !int.TryParse(args[index], out top) || top <= 0)
|
||||
throw new ArgumentException("--top requires a positive integer value.");
|
||||
break;
|
||||
case "--filter":
|
||||
if (++index >= args.Length)
|
||||
throw new ArgumentException("--filter requires a value.");
|
||||
nameFilter = args[index];
|
||||
break;
|
||||
case "--flat":
|
||||
case "--paths":
|
||||
flatPaths = true;
|
||||
break;
|
||||
case "--include-idle":
|
||||
excludeIdle = false;
|
||||
break;
|
||||
case "--exclude-idle":
|
||||
excludeIdle = true;
|
||||
break;
|
||||
default:
|
||||
throw new ArgumentException($"Unknown argument: {args[index]}");
|
||||
}
|
||||
}
|
||||
|
||||
return new ExtractorOptions(snapshotPath, top, nameFilter, flatPaths, excludeIdle);
|
||||
}
|
||||
|
||||
static JsonSerializerOptions CreateJsonOptions() => new()
|
||||
{
|
||||
MaxDepth = 4096,
|
||||
PropertyNamingPolicy = JsonNamingPolicy.CamelCase,
|
||||
WriteIndented = true
|
||||
WriteIndented = true,
|
||||
DefaultIgnoreCondition = System.Text.Json.Serialization.JsonIgnoreCondition.WhenWritingNull
|
||||
};
|
||||
|
||||
static void RegisterAssemblyResolver(string? dotTraceAppDir)
|
||||
@@ -202,10 +264,15 @@ static long GetTotalTime(DotTracePayload payload) => payload.PlusTime - payload.
|
||||
|
||||
static long GetCallCount(DotTracePayload payload) => payload.PlusCallCount - payload.MinusCallCount;
|
||||
|
||||
static HotspotLists BuildHotspots(IEnumerable<MutableNode> nodes)
|
||||
static IEnumerable<MutableNode> FilterHotspotCandidates(IEnumerable<MutableNode> nodes, ExtractorOptions options) =>
|
||||
nodes.Where(node =>
|
||||
node.Kind == "method" &&
|
||||
(!options.ExcludeIdle || !IsIdleMethod(node.Name)) &&
|
||||
MatchesFilter(node.Name, options.NameFilter));
|
||||
|
||||
static HotspotLists BuildHotspots(IEnumerable<MutableNode> nodes, int top)
|
||||
{
|
||||
var candidates = nodes
|
||||
.Where(node => node.Kind == "method")
|
||||
.Select(node => new HotspotEntry
|
||||
{
|
||||
Id = node.Id,
|
||||
@@ -222,32 +289,101 @@ static HotspotLists BuildHotspots(IEnumerable<MutableNode> nodes)
|
||||
Inclusive = candidates
|
||||
.OrderByDescending(node => node.InclusiveTime)
|
||||
.ThenBy(node => node.Name, StringComparer.Ordinal)
|
||||
.Take(50)
|
||||
.Take(top)
|
||||
.ToList(),
|
||||
Exclusive = candidates
|
||||
.OrderByDescending(node => node.ExclusiveTime)
|
||||
.ThenBy(node => node.Name, StringComparer.Ordinal)
|
||||
.Take(50)
|
||||
.Take(top)
|
||||
.ToList()
|
||||
};
|
||||
}
|
||||
|
||||
static SerializableNode CloneTree(
|
||||
CallTreeSectionOffset offset,
|
||||
IReadOnlyDictionary<CallTreeSectionOffset, MutableNode> nodeMap,
|
||||
HashSet<CallTreeSectionOffset> ancestry)
|
||||
static SummaryInfo BuildSummary(
|
||||
SerializableNode callTree,
|
||||
IEnumerable<MutableNode> hotspotCandidates,
|
||||
HotspotEntry? topExclusive)
|
||||
{
|
||||
var source = nodeMap[offset];
|
||||
var nextAncestry = new HashSet<CallTreeSectionOffset>(ancestry) { offset };
|
||||
var candidateArray = hotspotCandidates.ToArray();
|
||||
return new SummaryInfo
|
||||
{
|
||||
WallTimeMs = NanosecondsToMilliseconds(callTree.Children.Count == 0 ? 0 : callTree.Children.Max(node => node.InclusiveTime)),
|
||||
ActiveTimeMs = NanosecondsToMilliseconds(candidateArray.Sum(node => node.ExclusiveTime)),
|
||||
TotalSamples = candidateArray.Sum(node => node.CallCount),
|
||||
TopExclusiveMethod = topExclusive?.Name,
|
||||
TopExclusiveMs = NanosecondsToMilliseconds(topExclusive?.ExclusiveTime ?? 0)
|
||||
};
|
||||
}
|
||||
|
||||
static List<HotPathEntry> BuildHotPaths(SerializableNode root, ExtractorOptions options)
|
||||
{
|
||||
var collector = new List<HotPathAccumulator>();
|
||||
foreach (var threadNode in root.Children)
|
||||
{
|
||||
CollectHotPaths(threadNode, [threadNode.Name], collector, options);
|
||||
}
|
||||
|
||||
return collector
|
||||
.OrderByDescending(path => path.InclusiveTime)
|
||||
.ThenBy(path => path.Path, StringComparer.Ordinal)
|
||||
.Take(options.Top)
|
||||
.Select(path => new HotPathEntry
|
||||
{
|
||||
Path = path.Path,
|
||||
InclusiveTime = path.InclusiveTime,
|
||||
InclusiveMs = NanosecondsToMilliseconds(path.InclusiveTime),
|
||||
LeafExclusiveTime = path.LeafExclusiveTime,
|
||||
LeafExclusiveMs = NanosecondsToMilliseconds(path.LeafExclusiveTime)
|
||||
})
|
||||
.ToList();
|
||||
}
|
||||
|
||||
static void CollectHotPaths(
|
||||
SerializableNode node,
|
||||
List<string> path,
|
||||
List<HotPathAccumulator> collector,
|
||||
ExtractorOptions options)
|
||||
{
|
||||
foreach (var child in node.Children)
|
||||
{
|
||||
if (child.Kind == "method")
|
||||
{
|
||||
path.Add(child.Name);
|
||||
|
||||
if ((!options.ExcludeIdle || !IsIdleMethod(child.Name)) && MatchesFilter(child.Name, options.NameFilter))
|
||||
{
|
||||
collector.Add(new HotPathAccumulator(
|
||||
string.Join(" > ", path),
|
||||
child.InclusiveTime,
|
||||
child.ExclusiveTime));
|
||||
}
|
||||
|
||||
CollectHotPaths(child, path, collector, options);
|
||||
path.RemoveAt(path.Count - 1);
|
||||
continue;
|
||||
}
|
||||
|
||||
var appended = child.Kind is not ("root" or "special");
|
||||
if (appended)
|
||||
path.Add(child.Name);
|
||||
|
||||
CollectHotPaths(child, path, collector, options);
|
||||
if (appended)
|
||||
path.RemoveAt(path.Count - 1);
|
||||
}
|
||||
}
|
||||
|
||||
static SerializableNode CloneTree(MutableNode source, HashSet<CallTreeSectionOffset> ancestry)
|
||||
{
|
||||
var nextAncestry = new HashSet<CallTreeSectionOffset>(ancestry) { source.Offset };
|
||||
var children = new List<SerializableNode>(source.Children.Count);
|
||||
|
||||
foreach (var child in source.Children)
|
||||
{
|
||||
var childOffset = ParseOffset(child.Id);
|
||||
if (nextAncestry.Contains(childOffset))
|
||||
if (nextAncestry.Contains(child.Offset))
|
||||
continue;
|
||||
|
||||
children.Add(CloneTree(childOffset, nodeMap, nextAncestry));
|
||||
children.Add(CloneTree(child, nextAncestry));
|
||||
}
|
||||
|
||||
return new SerializableNode
|
||||
@@ -263,12 +399,80 @@ static SerializableNode CloneTree(
|
||||
};
|
||||
}
|
||||
|
||||
static CallTreeSectionOffset ParseOffset(string value)
|
||||
static SerializableNode? PruneTree(SerializableNode node, string? filter)
|
||||
{
|
||||
var parts = value.Split('/');
|
||||
return new CallTreeSectionOffset(Convert.ToInt64(parts[0], 16), int.Parse(parts[1]));
|
||||
if (string.IsNullOrWhiteSpace(filter))
|
||||
return node;
|
||||
|
||||
if (MatchesFilter(node.Name, filter))
|
||||
return node;
|
||||
|
||||
var prunedChildren = new List<SerializableNode>();
|
||||
foreach (var child in node.Children)
|
||||
{
|
||||
var prunedChild = PruneTree(child, filter);
|
||||
if (prunedChild is not null)
|
||||
prunedChildren.Add(prunedChild);
|
||||
}
|
||||
|
||||
if (prunedChildren.Count == 0)
|
||||
return null;
|
||||
|
||||
return node with { Children = prunedChildren };
|
||||
}
|
||||
|
||||
static SerializableNode CreateEmptyRoot() => new()
|
||||
{
|
||||
Id = "root",
|
||||
Name = "<root>",
|
||||
Kind = "root",
|
||||
ThreadName = null,
|
||||
InclusiveTime = 0,
|
||||
ExclusiveTime = 0,
|
||||
CallCount = 0,
|
||||
Children = []
|
||||
};
|
||||
|
||||
static bool MatchesFilter(string value, string? filter) =>
|
||||
string.IsNullOrWhiteSpace(filter) ||
|
||||
value.Contains(filter, StringComparison.OrdinalIgnoreCase);
|
||||
|
||||
static bool IsIdleMethod(string name) =>
|
||||
GetIdleMethodPatterns().Any(pattern => name.Contains(pattern, StringComparison.Ordinal));
|
||||
|
||||
static double NanosecondsToMilliseconds(long value) => Math.Round(value / 1_000_000d, 3);
|
||||
|
||||
static string[] GetIdleMethodPatterns()
|
||||
{
|
||||
return
|
||||
[
|
||||
"WaitHandle.Wait",
|
||||
"WaitOneNoCheck",
|
||||
"SemaphoreSlim.Wait",
|
||||
"LowLevelLifoSemaphore.Wait",
|
||||
"LowLevelLifoSemaphore.WaitForSignal",
|
||||
"LowLevelSpinWaiter.Wait",
|
||||
"Monitor.Wait",
|
||||
"SocketAsyncEngine.EventLoop",
|
||||
"PollGC",
|
||||
"Interop+Sys.WaitForSocketEvents",
|
||||
"ManualResetEventSlim.Wait",
|
||||
"Task.SpinThenBlockingWait",
|
||||
"Thread.SleepInternal"
|
||||
];
|
||||
}
|
||||
|
||||
sealed record ExtractorOptions(
|
||||
string SnapshotPath,
|
||||
int Top,
|
||||
string? NameFilter,
|
||||
bool FlatPaths,
|
||||
bool ExcludeIdle);
|
||||
|
||||
readonly record struct SignatureResult(string Name, string Kind);
|
||||
|
||||
readonly record struct HotPathAccumulator(string Path, long InclusiveTime, long LeafExclusiveTime);
|
||||
|
||||
sealed class SignatureResolver(
|
||||
SnapshotMasksComponent masks,
|
||||
FuidToMetadataIdConverter fuidConverter,
|
||||
@@ -343,16 +547,18 @@ sealed class SignatureResolver(
|
||||
}
|
||||
}
|
||||
|
||||
readonly record struct SignatureResult(string Name, string Kind);
|
||||
|
||||
sealed class OutputDocument
|
||||
{
|
||||
public required SnapshotInfo Snapshot { get; init; }
|
||||
|
||||
public required SummaryInfo Summary { get; init; }
|
||||
|
||||
public required List<ThreadRootInfo> ThreadRoots { get; init; }
|
||||
|
||||
public required HotspotLists Hotspots { get; init; }
|
||||
|
||||
public List<HotPathEntry>? HotPaths { get; init; }
|
||||
|
||||
public required SerializableNode CallTree { get; init; }
|
||||
}
|
||||
|
||||
@@ -362,9 +568,39 @@ sealed class SnapshotInfo
|
||||
|
||||
public required string PayloadType { get; init; }
|
||||
|
||||
public required string TimeUnit { get; init; }
|
||||
|
||||
public required int ThreadCount { get; init; }
|
||||
|
||||
public required int NodeCount { get; init; }
|
||||
|
||||
public required SnapshotDiagnostics Diagnostics { get; init; }
|
||||
}
|
||||
|
||||
sealed class SnapshotDiagnostics
|
||||
{
|
||||
public required int HeaderCount { get; init; }
|
||||
|
||||
public required int ExpectedNodeCount { get; init; }
|
||||
|
||||
public required int ReadNodeCount { get; init; }
|
||||
|
||||
public required int ReadTotalPayloadCount { get; init; }
|
||||
|
||||
public required int ReadOwnPayloadCount { get; init; }
|
||||
}
|
||||
|
||||
sealed class SummaryInfo
|
||||
{
|
||||
public required double WallTimeMs { get; init; }
|
||||
|
||||
public required double ActiveTimeMs { get; init; }
|
||||
|
||||
public required long TotalSamples { get; init; }
|
||||
|
||||
public string? TopExclusiveMethod { get; init; }
|
||||
|
||||
public required double TopExclusiveMs { get; init; }
|
||||
}
|
||||
|
||||
sealed class ThreadRootInfo
|
||||
@@ -398,8 +634,25 @@ sealed class HotspotEntry
|
||||
public required long CallCount { get; init; }
|
||||
}
|
||||
|
||||
class MutableNode
|
||||
sealed class HotPathEntry
|
||||
{
|
||||
public required string Path { get; init; }
|
||||
|
||||
public required long InclusiveTime { get; init; }
|
||||
|
||||
public required double InclusiveMs { get; init; }
|
||||
|
||||
public required long LeafExclusiveTime { get; init; }
|
||||
|
||||
public required double LeafExclusiveMs { get; init; }
|
||||
}
|
||||
|
||||
sealed class MutableNode
|
||||
{
|
||||
public required CallTreeSectionOffset Offset { get; init; }
|
||||
|
||||
public required CallTreeSectionOffset ParentOffset { get; init; }
|
||||
|
||||
public required string Id { get; init; }
|
||||
|
||||
public required string Name { get; init; }
|
||||
@@ -415,7 +668,7 @@ class MutableNode
|
||||
public required List<MutableNode> Children { get; init; }
|
||||
}
|
||||
|
||||
sealed class SerializableNode
|
||||
sealed record SerializableNode
|
||||
{
|
||||
public required string Id { get; init; }
|
||||
|
||||
|
||||
BIN
tools/__pycache__/dtp_parse.cpython-314.pyc
Normal file
BIN
tools/__pycache__/dtp_parse.cpython-314.pyc
Normal file
Binary file not shown.
@@ -21,6 +21,29 @@ def parse_args() -> argparse.Namespace:
|
||||
parser.add_argument("snapshot", help="Path to the .dtp snapshot index file.")
|
||||
parser.add_argument("--out", help="Write JSON to this file.")
|
||||
parser.add_argument("--stdout", action="store_true", help="Write JSON to stdout.")
|
||||
parser.add_argument("--top", type=int, default=200, help="Maximum hotspot and path entries to emit.")
|
||||
parser.add_argument("--filter", dest="name_filter", help="Case-insensitive substring filter for node names.")
|
||||
parser.add_argument(
|
||||
"--flat",
|
||||
"--paths",
|
||||
dest="flat_paths",
|
||||
action="store_true",
|
||||
help="Include the top heaviest call paths as flat strings.",
|
||||
)
|
||||
idle_group = parser.add_mutually_exclusive_group()
|
||||
idle_group.add_argument(
|
||||
"--exclude-idle",
|
||||
dest="exclude_idle",
|
||||
action="store_true",
|
||||
default=True,
|
||||
help="Exclude idle and wait methods from hotspot and path rankings.",
|
||||
)
|
||||
idle_group.add_argument(
|
||||
"--include-idle",
|
||||
dest="exclude_idle",
|
||||
action="store_false",
|
||||
help="Keep idle and wait methods in hotspot and path rankings.",
|
||||
)
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
@@ -51,8 +74,14 @@ def build_helper(dottrace_dir: Path) -> None:
|
||||
raise RuntimeError(result.stderr.strip() or result.stdout.strip() or "dotnet build failed")
|
||||
|
||||
|
||||
def run_helper(snapshot: Path, dottrace_dir: Path) -> dict:
|
||||
command = ["dotnet", str(HELPER_DLL), str(snapshot)]
|
||||
def run_helper(snapshot: Path, dottrace_dir: Path, args: argparse.Namespace) -> dict:
|
||||
command = ["dotnet", str(HELPER_DLL), str(snapshot), "--top", str(args.top)]
|
||||
if args.name_filter:
|
||||
command.extend(["--filter", args.name_filter])
|
||||
if args.flat_paths:
|
||||
command.append("--flat")
|
||||
if not args.exclude_idle:
|
||||
command.append("--include-idle")
|
||||
env = os.environ.copy()
|
||||
env["DOTTRACE_APP_DIR"] = str(dottrace_dir)
|
||||
result = subprocess.run(command, cwd=ROOT, capture_output=True, text=True, check=False, env=env)
|
||||
@@ -74,7 +103,7 @@ def main() -> int:
|
||||
try:
|
||||
dottrace_dir = find_dottrace_dir()
|
||||
build_helper(dottrace_dir)
|
||||
payload = run_helper(snapshot, dottrace_dir)
|
||||
payload = run_helper(snapshot, dottrace_dir, args)
|
||||
except Exception as exc: # noqa: BLE001
|
||||
print(str(exc), file=sys.stderr)
|
||||
return 1
|
||||
|
||||
Binary file not shown.
@@ -16,9 +16,9 @@ def walk(node):
|
||||
|
||||
|
||||
class DtpParserTests(unittest.TestCase):
|
||||
def test_emits_machine_readable_call_tree(self):
|
||||
def run_parser(self, *args: str) -> dict:
|
||||
result = subprocess.run(
|
||||
["python3", str(SCRIPT), str(SNAPSHOT), "--stdout"],
|
||||
["python3", str(SCRIPT), str(SNAPSHOT), "--stdout", *args],
|
||||
cwd=ROOT,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
@@ -26,16 +26,54 @@ class DtpParserTests(unittest.TestCase):
|
||||
)
|
||||
|
||||
self.assertEqual(result.returncode, 0, msg=result.stderr)
|
||||
payload = json.loads(result.stdout)
|
||||
return json.loads(result.stdout)
|
||||
|
||||
def test_emits_machine_readable_call_tree(self):
|
||||
payload = self.run_parser()
|
||||
|
||||
self.assertIn("callTree", payload)
|
||||
self.assertIn("hotspots", payload)
|
||||
self.assertIn("summary", payload)
|
||||
self.assertTrue(payload["callTree"]["children"])
|
||||
self.assertTrue(payload["hotspots"]["inclusive"])
|
||||
self.assertEqual(payload["snapshot"]["timeUnit"], "nanoseconds")
|
||||
self.assertLessEqual(len(payload["hotspots"]["inclusive"]), 200)
|
||||
self.assertLessEqual(len(payload["hotspots"]["exclusive"]), 200)
|
||||
self.assertIn("wallTimeMs", payload["summary"])
|
||||
self.assertIn("activeTimeMs", payload["summary"])
|
||||
self.assertIn("totalSamples", payload["summary"])
|
||||
self.assertIn("topExclusiveMethod", payload["summary"])
|
||||
|
||||
node_names = [node["name"] for node in walk(payload["callTree"])]
|
||||
self.assertTrue(any(not name.startswith("[special:") for name in node_names))
|
||||
|
||||
def test_supports_hotspot_filter_and_flat_paths(self):
|
||||
payload = self.run_parser("--top", "7", "--filter", "Microsoft.DotNet.Cli.Program", "--flat")
|
||||
|
||||
self.assertLessEqual(len(payload["hotspots"]["inclusive"]), 7)
|
||||
self.assertLessEqual(len(payload["hotspots"]["exclusive"]), 7)
|
||||
self.assertIn("hotPaths", payload)
|
||||
|
||||
node_names = [node["name"] for node in walk(payload["callTree"])]
|
||||
self.assertTrue(any("Microsoft.DotNet.Cli.Program" in name for name in node_names))
|
||||
self.assertFalse(any("Microsoft.Build.Tasks.Copy.ParallelCopyTask" == name for name in node_names))
|
||||
|
||||
for hotspot in payload["hotspots"]["inclusive"] + payload["hotspots"]["exclusive"]:
|
||||
self.assertIn("Microsoft.DotNet.Cli.Program", hotspot["name"])
|
||||
|
||||
for path_entry in payload["hotPaths"]:
|
||||
self.assertIn("Microsoft.DotNet.Cli.Program", path_entry["path"])
|
||||
|
||||
def test_can_include_idle_hotspots_when_requested(self):
|
||||
without_idle = self.run_parser("--top", "20")
|
||||
with_idle = self.run_parser("--top", "20", "--include-idle")
|
||||
|
||||
without_idle_names = {entry["name"] for entry in without_idle["hotspots"]["exclusive"]}
|
||||
with_idle_names = {entry["name"] for entry in with_idle["hotspots"]["exclusive"]}
|
||||
|
||||
self.assertNotIn("System.Threading.WaitHandle.WaitOneNoCheck", without_idle_names)
|
||||
self.assertIn("System.Threading.WaitHandle.WaitOneNoCheck", with_idle_names)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
Reference in New Issue
Block a user