docs: add feature audit script design
Automated PortTracker CLI command (feature audit) using Roslyn syntax tree analysis to classify 3394 unknown features into verified/stub/n_a/deferred.
This commit is contained in:
Joseph Doherty
154
docs/plans/2026-02-27-feature-audit-script-design.md
Normal file
154
docs/plans/2026-02-27-feature-audit-script-design.md
Normal file
@@ -0,0 +1,154 @@
|
||||
# Feature Audit Script Design
|
||||
|
||||
**Date:** 2026-02-27
|
||||
**Status:** Approved
|
||||
|
||||
## Problem
|
||||
|
||||
3394 features in module 8 (`server`) are marked `unknown`. The existing plan (`2026-02-27-feature-status-audit-plan.md`) describes a manual 68-batch process of inspecting .NET source and classifying each feature. This design automates that process.
|
||||
|
||||
## Solution
|
||||
|
||||
A new PortTracker CLI command `feature audit` that uses Roslyn syntax tree analysis to parse .NET source files, build a method index, and classify all unknown features automatically.
|
||||
|
||||
## Command Interface
|
||||
|
||||
```
|
||||
dotnet run --project tools/NatsNet.PortTracker -- feature audit \
|
||||
--source dotnet/src/ZB.MOM.NatsNet.Server/ \
|
||||
--output reports/audit-results.csv \
|
||||
--db porting.db \
|
||||
[--module 8] \
|
||||
[--execute]
|
||||
```
|
||||
|
||||
| Flag | Default | Description |
|
||||
|------|---------|-------------|
|
||||
| `--source` | `dotnet/src/ZB.MOM.NatsNet.Server/` | .NET source directory to parse |
|
||||
| `--output` | `reports/audit-results.csv` | CSV report output path |
|
||||
| `--db` | `porting.db` | SQLite database (inherited from root) |
|
||||
| `--module` | *(all)* | Restrict to a specific module ID |
|
||||
| `--execute` | `false` | Apply DB updates (default: dry-run) |
|
||||
|
||||
## Architecture
|
||||
|
||||
### Component 1: Source Indexer (`Audit/SourceIndexer.cs`)
|
||||
|
||||
Parses all `.cs` files under the source directory into Roslyn syntax trees and builds a lookup index.
|
||||
|
||||
**Process:**
|
||||
1. Recursively glob `**/*.cs` (skip `obj/`, `bin/`)
|
||||
2. Parse each file with `CSharpSyntaxTree.ParseText()`
|
||||
3. Walk syntax trees for `ClassDeclarationSyntax` and `StructDeclarationSyntax`
|
||||
4. Extract all method, property, and constructor declarations
|
||||
5. Build dictionary: `Dictionary<(string className, string memberName), List<MethodInfo>>`
|
||||
|
||||
**`MethodInfo`:**
|
||||
- `FilePath` — source file path
|
||||
- `LineNumber` — starting line
|
||||
- `BodyLineCount` — lines in method body (excluding braces)
|
||||
- `IsStub` — body is `throw new NotImplementedException(...)` or empty
|
||||
- `IsPartial` — body has some logic AND a `NotImplementedException` throw
|
||||
- `StatementCount` — number of meaningful statements
|
||||
|
||||
**Partial class handling:** Same class name across multiple files produces multiple entries in the index. Lookup checks all of them — a feature is matched if the method exists in ANY file for that class.
|
||||
|
||||
**Name matching:** Case-insensitive comparison for both class and method names. Handles `dotnet_class` values that contain commas (e.g. `ClosedRingBuffer,ClosedClient`) by splitting and checking each.
|
||||
|
||||
### Component 2: Feature Classifier (`Audit/FeatureClassifier.cs`)
|
||||
|
||||
Classifies each feature using the source index. Priority order (first match wins):
|
||||
|
||||
**1. N/A Lookup Table**
|
||||
|
||||
Checked first against `(go_file, go_method)` or `dotnet_class` patterns:
|
||||
|
||||
| Pattern | Reason |
|
||||
|---------|--------|
|
||||
| Go logging functions (`Noticef`, `Debugf`, `Tracef`, `Warnf`, `Errorf`, `Fatalf`) | .NET uses Microsoft.Extensions.Logging |
|
||||
| Go signal handling (`HandleSignals`, `processSignal`) | .NET uses IHostApplicationLifetime |
|
||||
| Go HTTP handler setup (`Statz`, `Varz`, `Connz`, etc.) | .NET uses ASP.NET middleware |
|
||||
|
||||
Table is extensible — add entries as new patterns are identified.
|
||||
|
||||
**2. Method Not Found** -> `deferred`
|
||||
- `dotnet_class` not found in source index, OR
|
||||
- `dotnet_method` not found within the class
|
||||
|
||||
**3. Stub Detection** -> `stub`
|
||||
- Body is solely `throw new NotImplementedException(...)` (expression-bodied or block)
|
||||
- Body is empty (no statements)
|
||||
- Body has logic but also contains `NotImplementedException` (partial implementation)
|
||||
|
||||
**4. Verified** -> `verified`
|
||||
- Method exists with 1+ meaningful statements that are not `NotImplementedException` throws
|
||||
|
||||
### Component 3: Audit Command (`Commands/AuditCommand.cs`)
|
||||
|
||||
Orchestrates the audit:
|
||||
|
||||
1. Query `SELECT id, dotnet_class, dotnet_method, go_file, go_method FROM features WHERE status = 'unknown'` (optionally filtered by module)
|
||||
2. Build source index via `SourceIndexer`
|
||||
3. Classify each feature via `FeatureClassifier`
|
||||
4. Write CSV report
|
||||
5. Print console summary
|
||||
6. If `--execute`: update DB in a single transaction per status group
|
||||
|
||||
### DB Update Strategy
|
||||
|
||||
- Group features by `(new_status, notes)` tuple
|
||||
- One `UPDATE features SET status = @s, notes = @n WHERE id IN (...)` per group
|
||||
- All groups in a single transaction
|
||||
- For `n_a` features: set `notes` to the reason from the lookup table
|
||||
|
||||
## Output
|
||||
|
||||
### CSV Report
|
||||
|
||||
```csv
|
||||
id,dotnet_class,dotnet_method,go_file,go_method,old_status,new_status,reason
|
||||
150,ServiceRespType,String,server/accounts.go,String,unknown,verified,Method found with 3 statements
|
||||
151,Account,NewAccount,server/accounts.go,NewAccount,unknown,stub,Body is throw NotImplementedException
|
||||
```
|
||||
|
||||
### Console Summary
|
||||
|
||||
```
|
||||
Feature Status Audit Results
|
||||
=============================
|
||||
Source: dotnet/src/ZB.MOM.NatsNet.Server/ (142 files, 4821 methods indexed)
|
||||
Features audited: 3394
|
||||
|
||||
verified: NNNN
|
||||
stub: NNNN
|
||||
n_a: NNNN
|
||||
deferred: NNNN
|
||||
|
||||
Dry-run mode. Add --execute to apply changes.
|
||||
Report: reports/audit-results.csv
|
||||
```
|
||||
|
||||
## Dependencies
|
||||
|
||||
New NuGet package required:
|
||||
|
||||
```xml
|
||||
<PackageReference Include="Microsoft.CodeAnalysis.CSharp" Version="4.13.0" />
|
||||
```
|
||||
|
||||
## Files to Create/Modify
|
||||
|
||||
| File | Action |
|
||||
|------|--------|
|
||||
| `tools/NatsNet.PortTracker/NatsNet.PortTracker.csproj` | Add Roslyn package reference |
|
||||
| `tools/NatsNet.PortTracker/Audit/SourceIndexer.cs` | New — Roslyn source parsing and indexing |
|
||||
| `tools/NatsNet.PortTracker/Audit/FeatureClassifier.cs` | New — classification heuristics |
|
||||
| `tools/NatsNet.PortTracker/Commands/AuditCommand.cs` | New — CLI command wiring |
|
||||
| `tools/NatsNet.PortTracker/Program.cs` | Add `AuditCommand.Create()` to root command |
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- No semantic analysis (full compilation) — syntax trees are sufficient
|
||||
- No Go source parsing — we only inspect .NET source
|
||||
- No unit test reclassification — separate effort
|
||||
- No code changes to the server project — classification only
|
||||
@@ -1,6 +1,6 @@
|
||||
# NATS .NET Porting Status Report
|
||||
|
||||
Generated: 2026-02-27 09:50:38 UTC
|
||||
Generated: 2026-02-27 10:09:38 UTC
|
||||
|
||||
## Modules (12 total)
|
||||
|
||||
|
||||
35
reports/report_e7f2597.md
Normal file
35
reports/report_e7f2597.md
Normal file
@@ -0,0 +1,35 @@
|
||||
# NATS .NET Porting Status Report
|
||||
|
||||
Generated: 2026-02-27 10:09:38 UTC
|
||||
|
||||
## Modules (12 total)
|
||||
|
||||
| Status | Count |
|
||||
|--------|-------|
|
||||
| verified | 12 |
|
||||
|
||||
## Features (3673 total)
|
||||
|
||||
| Status | Count |
|
||||
|--------|-------|
|
||||
| unknown | 3394 |
|
||||
| verified | 279 |
|
||||
|
||||
## Unit Tests (3257 total)
|
||||
|
||||
| Status | Count |
|
||||
|--------|-------|
|
||||
| deferred | 2680 |
|
||||
| n_a | 187 |
|
||||
| verified | 390 |
|
||||
|
||||
## Library Mappings (36 total)
|
||||
|
||||
| Status | Count |
|
||||
|--------|-------|
|
||||
| mapped | 36 |
|
||||
|
||||
|
||||
## Overall Progress
|
||||
|
||||
**868/6942 items complete (12.5%)**
|
||||
Reference in New Issue
Block a user