# Porting Tracker Design A system for tracking the port of NATS server from Go to .NET 10 C#. Consists of an SQLite database, a Go AST analyzer for initial codebase decomposition, a .NET CLI tool for ongoing management, and 7 phase instruction guides. ## Context - **Source**: `golang/nats-server/` — ~130K LOC across 109 non-test Go files, 85 test files (~217K test LOC) - **Target**: .NET 10 C# solution following conventions defined in `documentation_rules.md` - **Dependencies**: 10+ Go external packages (klauspost/compress, nats-io/jwt, nats-io/nkeys, nats-io/nuid, golang.org/x/crypto, etc.) ## Architecture Two tools + one database: 1. **Go AST Analyzer** (`tools/go-analyzer/`) — Parses Go source using `go/ast`, `go/parser`, `go/token`, and `golang.org/x/tools/go/callgraph` to extract packages, functions, types, call graphs, LOC counts. Populates the SQLite DB. Used during Phases 1-2 only. 2. **.NET PortTracker CLI** (`tools/NatsNet.PortTracker/`) — Manages the DB after initial population: update status, map .NET items, run reports, export markdown. Used during all phases. 3. **SQLite Database** (`porting.db`) — Single source of truth for porting progress. ## Database Schema ### modules Tracks top-level Go packages/logical groupings. ```sql CREATE TABLE modules ( id INTEGER PRIMARY KEY AUTOINCREMENT, name TEXT NOT NULL, description TEXT, go_package TEXT, go_file TEXT, go_line_start INTEGER, go_line_count INTEGER, status TEXT NOT NULL DEFAULT 'not_started' CHECK (status IN ('not_started', 'stub', 'complete', 'verified', 'n_a')), dotnet_project TEXT, dotnet_namespace TEXT, dotnet_class TEXT, notes TEXT, created_at DATETIME DEFAULT CURRENT_TIMESTAMP, updated_at DATETIME DEFAULT CURRENT_TIMESTAMP ); ``` ### features Tracks methods/functions/features within a module. ```sql CREATE TABLE features ( id INTEGER PRIMARY KEY AUTOINCREMENT, module_id INTEGER NOT NULL REFERENCES modules(id), name TEXT NOT NULL, description TEXT, go_file TEXT, go_class TEXT, go_method TEXT, go_line_number INTEGER, go_line_count INTEGER, status TEXT NOT NULL DEFAULT 'not_started' CHECK (status IN ('not_started', 'stub', 'complete', 'verified', 'n_a')), dotnet_project TEXT, dotnet_class TEXT, dotnet_method TEXT, notes TEXT, created_at DATETIME DEFAULT CURRENT_TIMESTAMP, updated_at DATETIME DEFAULT CURRENT_TIMESTAMP ); ``` ### unit_tests Tracks test functions. ```sql CREATE TABLE unit_tests ( id INTEGER PRIMARY KEY AUTOINCREMENT, module_id INTEGER NOT NULL REFERENCES modules(id), feature_id INTEGER REFERENCES features(id), name TEXT NOT NULL, description TEXT, go_file TEXT, go_class TEXT, go_method TEXT, go_line_number INTEGER, go_line_count INTEGER, status TEXT NOT NULL DEFAULT 'not_started' CHECK (status IN ('not_started', 'stub', 'complete', 'verified', 'n_a')), dotnet_project TEXT, dotnet_class TEXT, dotnet_method TEXT, notes TEXT, created_at DATETIME DEFAULT CURRENT_TIMESTAMP, updated_at DATETIME DEFAULT CURRENT_TIMESTAMP ); ``` ### dependencies Polymorphic dependency relationships between any tracked items. Used to determine porting order (port dependencies first). ```sql CREATE TABLE dependencies ( id INTEGER PRIMARY KEY AUTOINCREMENT, source_type TEXT NOT NULL CHECK (source_type IN ('module', 'feature', 'unit_test')), source_id INTEGER NOT NULL, target_type TEXT NOT NULL CHECK (target_type IN ('module', 'feature', 'unit_test')), target_id INTEGER NOT NULL, dependency_kind TEXT DEFAULT 'calls', created_at DATETIME DEFAULT CURRENT_TIMESTAMP, UNIQUE (source_type, source_id, target_type, target_id) ); ``` ### library_mappings Maps Go imports to .NET equivalents. ```sql CREATE TABLE library_mappings ( id INTEGER PRIMARY KEY AUTOINCREMENT, go_import_path TEXT NOT NULL UNIQUE, go_library_name TEXT, go_usage_description TEXT, dotnet_package TEXT, dotnet_namespace TEXT, dotnet_usage_notes TEXT, status TEXT NOT NULL DEFAULT 'not_mapped' CHECK (status IN ('not_mapped', 'mapped', 'verified')), created_at DATETIME DEFAULT CURRENT_TIMESTAMP, updated_at DATETIME DEFAULT CURRENT_TIMESTAMP ); ``` ### Indexes ```sql CREATE INDEX idx_features_module ON features(module_id); CREATE INDEX idx_features_status ON features(status); CREATE INDEX idx_unit_tests_module ON unit_tests(module_id); CREATE INDEX idx_unit_tests_feature ON unit_tests(feature_id); CREATE INDEX idx_unit_tests_status ON unit_tests(status); CREATE INDEX idx_deps_source ON dependencies(source_type, source_id); CREATE INDEX idx_deps_target ON dependencies(target_type, target_id); CREATE INDEX idx_library_status ON library_mappings(status); CREATE INDEX idx_modules_status ON modules(status); ``` ## Go AST Analyzer Located at `tools/go-analyzer/`. A Go program that: 1. **Parses all `.go` files** in `golang/nats-server/server/` and subdirectories using `go/ast` and `go/parser` 2. **Extracts per-file**: package name, struct types, functions/methods with receiver types, line numbers, LOC counts 3. **Builds call graph**: using `golang.org/x/tools/go/callgraph/cha` (Class Hierarchy Analysis) for whole-program analysis 4. **Extracts test functions**: parses `*_test.go` files separately, links to tested functions by name convention and call graph 5. **Extracts imports**: all imported packages per file, distinguishing stdlib vs external 6. **Writes to SQLite**: directly populates `modules`, `features`, `unit_tests`, `dependencies`, and partially populates `library_mappings` (go-side fields only) ### Module Grouping Strategy - Each Go file in `server/` that represents a major subsystem maps to a module (e.g., `jetstream.go` + `jetstream_api.go` + `jetstream_cluster.go` -> "jetstream") - Subdirectories (`stree/`, `pse/`, `certidp/`, etc.) each become their own module - Groupings are auto-generated then user-adjustable via the PortTracker CLI ### Usage ```bash cd tools/go-analyzer go run . --source ../../golang/nats-server --db ../../porting.db ``` ## .NET PortTracker CLI Located at `tools/NatsNet.PortTracker/`. A .NET 10 console app using `System.CommandLine` and `Microsoft.Data.Sqlite`. ### Command Structure ``` porttracker ├── init # Create/reset the DB schema ├── module │ ├── list [--status ] # List modules with filters │ ├── show # Show module detail + features + deps │ ├── update --status # Update status │ ├── map --project

--class # Map to .NET │ └── set-na --reason # Mark as N/A with reason ├── feature │ ├── list [--module ] [--status ] │ ├── show │ ├── update --status │ ├── map --project

--class --method │ └── set-na --reason ├── test │ ├── list [--module ] [--status ] │ ├── show │ ├── update --status │ └── map --project

--class --method ├── library │ ├── list [--status ] │ ├── map --package --namespace --notes │ └── suggest # Show unmapped libraries ├── dependency │ ├── show # Show deps for an item │ ├── blocked # Items with unported dependencies │ └── ready # Items whose deps are all ported ├── report │ ├── summary # Overall progress stats │ ├── phase # Progress for a specific phase │ ├── export --format md # Export markdown status report │ └── blocking # Critical path / blocking chains └── phase ├── list # Show all phases and status └── check # Run verification for phase N ``` ### Design Decisions - **Raw SQL over EF Core**: Simpler, fewer dependencies. This is a tools app, not a production service. - **Status stored as text**: Readable in DB tools and queries. - **Batch operations**: Commands accept `--all-in-module ` for bulk status updates. - **DB location**: `porting.db` at repo root, overridable with `--db` flag. ## Phase Instructions Seven markdown guides in `docs/plans/phases/`. Each is a standalone step-by-step document. ### Phase 1: Go Codebase Decomposition **Objective**: Parse the Go source into the SQLite DB — modules, features, unit tests, and call dependencies. **Steps**: 1. Run `porttracker init` to create the database 2. Build and run the Go AST analyzer: `cd tools/go-analyzer && go run . --source ../../golang/nats-server --db ../../porting.db` 3. Review auto-generated module groupings: `porttracker module list` 4. Adjust module groupings as needed via manual SQL or future CLI commands 5. Spot-check feature extraction against Go source files 6. Verify test function extraction: `porttracker test list` 7. Review dependency graph: `porttracker dependency show module ` for key modules **Completion criteria**: All Go source files accounted for. All public functions extracted. Test functions linked to features. Call dependencies populated. ### Phase 2: Verification of Captured Items **Objective**: Ensure nothing was missed and dependencies are accurate. **Steps**: 1. Run `porttracker report summary` — compare counts against baselines: - File count: `find golang/nats-server/server -name "*.go" ! -name "*_test.go" | wc -l` - Function count: `grep -r "^func " golang/nats-server/server/ --include="*.go" --exclude="*_test.go" | wc -l` - Test count: `grep -r "^func Test" golang/nats-server/server/ --include="*_test.go" | wc -l` 2. Cross-check: every `*_test.go` function appears in `unit_tests` 3. Cross-check: every non-test `.go` file has a corresponding module 4. Check for orphaned nodes: `porttracker dependency blocked` — items with no path to a root 5. Identify and resolve circular dependencies 6. Manual review of the 10 largest modules for completeness **Completion criteria**: Counts match baselines. No orphaned nodes. No missing files. Dependency graph is valid. ### Phase 3: Library Mapping **Objective**: Map every Go external dependency to its .NET equivalent. **Steps**: 1. Run `porttracker library suggest` to list all unmapped Go imports 2. For each Go import, determine the .NET equivalent: - **Go stdlib** -> .NET BCL (e.g., `encoding/json` -> `System.Text.Json`, `sync` -> `System.Threading`, `net/http` -> `Microsoft.AspNetCore` or `System.Net.Http`) - **External packages** (e.g., `klauspost/compress` -> evaluate `System.IO.Compression` or a NuGet package) - **NATS-specific** (e.g., `nats-io/nkeys` -> .NET NKeys library or custom implementation) 3. Record each mapping: `porttracker library map --package --namespace --notes ` 4. Flag libraries with no clear .NET equivalent — these need design decisions **Completion criteria**: `porttracker library list --status not_mapped` returns empty. All ambiguous cases documented. ### Phase 4: .NET Solution Design **Objective**: Design the target .NET 10 solution structure and map every Go item to its .NET counterpart. **Steps**: 1. Define solution structure following .NET conventions: - `src/NATS.Server/` — main server library - `src/NATS.Server.Host/` — host/entry point - Additional projects as needed per module grouping - `tests/NATS.Server.Tests/` — unit tests - `tests/NATS.Server.IntegrationTests/` — integration tests 2. For each module: `porttracker module map --project --class ` 3. For each feature: `porttracker feature map --project --class --method ` 4. For each test: `porttracker test map --project --class --method ` 5. For items not applicable in .NET (e.g., Go-specific platform code, custom logger replaced by Serilog): `porttracker module set-na --reason "Replaced by Serilog"` (or feature/test equivalent) **Completion criteria**: Every item has a .NET mapping or is set to N/A with a reason. ### Phase 5: Mapping Verification **Objective**: Verify all Go items are mapped to .NET targets. **Steps**: 1. Run `porttracker report summary` — confirm 0 unmapped items (excluding N/A) 2. Review all N/A items — each must have a justification: `porttracker module list --status n_a`, `porttracker feature list --status n_a` 3. Verify .NET naming follows conventions: - PascalCase for classes and methods - Proper namespace hierarchy - No collisions (two features mapped to the same .NET class+method) 4. Generate full mapping report: `porttracker report export --format md` 5. Review the exported report for completeness **Completion criteria**: Zero unmapped items. All N/A items justified. Naming conventions validated. No collisions. ### Phase 6: Initial Porting **Objective**: Port Go code to .NET, working through the dependency graph bottom-up. **Steps**: 1. Run `porttracker dependency ready` to find items with all dependencies already ported 2. For each ready item: a. `porttracker feature update --status stub` — when creating the class/method skeleton b. Implement the logic, referencing the Go source at the line numbers stored in the DB c. `porttracker feature update --status complete` — when implementation is done d. Run the corresponding unit tests for that feature 3. After completing items, re-run `porttracker dependency ready` to find newly unblocked items 4. Periodically run `porttracker report summary` to track overall progress 5. For batch scaffolding: `porttracker feature update --all-in-module --status stub` **DB update workflow**: ```bash # Starting a feature porttracker feature update 42 --status stub # Feature implemented porttracker feature update 42 --status complete # Corresponding test implemented porttracker test update 87 --status complete # Check what's unblocked now porttracker dependency ready ``` **Completion criteria**: All non-N/A items at status `complete` or better. ### Phase 7: Verification of Ported Items **Objective**: Verify all ported code works correctly through targeted testing. **Steps**: 1. For each module, identify its mapped tests: `porttracker test list --module ` 2. Run targeted tests per module using `dotnet test --filter`: ```bash dotnet test --filter "FullyQualifiedName~NATS.Server.Tests.Protocol" ``` 3. For each passing module, update status: `porttracker module update --status verified` `porttracker feature update --all-in-module --status verified` 4. For failures: investigate, fix, and re-run the targeted tests 5. Run integration tests for cross-module behavior 6. Compare behavior: run Go server and .NET server with same workload, verify equivalent behavior 7. Final report: `porttracker report summary` — all items `verified` or `n_a` **Completion criteria**: All targeted tests pass. All items at `verified` or `n_a`. Behavioral equivalence confirmed for key scenarios. ## Project Layout ``` natsnet/ ├── golang/nats-server/ # Existing Go source (reference) ├── tools/ │ ├── go-analyzer/ # Go AST analyzer (Phases 1-2) │ │ ├── go.mod │ │ ├── main.go │ │ ├── analyzer.go │ │ ├── grouper.go │ │ └── sqlite.go │ └── NatsNet.PortTracker/ # .NET CLI tool (all phases) │ ├── NatsNet.PortTracker.csproj │ ├── Program.cs │ ├── Commands/ │ ├── Data/ │ └── Reporting/ ├── docs/ │ └── plans/ │ └── phases/ │ ├── phase-1-decomposition.md │ ├── phase-2-verification.md │ ├── phase-3-library-mapping.md │ ├── phase-4-dotnet-design.md │ ├── phase-5-mapping-verification.md │ ├── phase-6-porting.md │ └── phase-7-porting-verification.md ├── porting.db # SQLite database (gitignored) ├── porting-schema.sql # DDL reference (committed) └── documentation_rules.md # Existing ``` ### Notes - `porting.db` is `.gitignore`d (contains local state); `porting-schema.sql` is committed - `porttracker init` creates the DB from the embedded schema - Go analyzer writes directly to SQLite — no intermediate format - Phase docs reference `porttracker` commands by name - Run the .NET tool from anywhere: `dotnet run --project tools/NatsNet.PortTracker/`