docs: add phase 1-3 instruction guides
This commit is contained in:
291
docs/plans/phases/phase-1-decomposition.md
Normal file
291
docs/plans/phases/phase-1-decomposition.md
Normal file
@@ -0,0 +1,291 @@
|
|||||||
|
# Phase 1: Go Codebase Decomposition
|
||||||
|
|
||||||
|
## Objective
|
||||||
|
|
||||||
|
Parse the Go NATS server source code into a structured SQLite database, extracting
|
||||||
|
modules, features (functions/methods), unit tests, external library imports, and
|
||||||
|
inter-module dependencies. This database becomes the single source of truth that
|
||||||
|
drives all subsequent porting phases.
|
||||||
|
|
||||||
|
## Prerequisites
|
||||||
|
|
||||||
|
| Requirement | Version / Notes |
|
||||||
|
|---|---|
|
||||||
|
| Go | 1.25+ (required by `tools/go-analyzer/go.mod`) |
|
||||||
|
| .NET SDK | 10.0+ |
|
||||||
|
| SQLite3 CLI | 3.x (optional, for manual inspection) |
|
||||||
|
| CGO | Must be enabled (`CGO_ENABLED=1`); the Go analyzer uses `github.com/mattn/go-sqlite3` |
|
||||||
|
| Go source | Cloned at `golang/nats-server/` relative to the repo root |
|
||||||
|
|
||||||
|
Verify prerequisites before starting:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
go version # should print go1.25 or later
|
||||||
|
dotnet --version # should print 10.x
|
||||||
|
sqlite3 --version # optional, any 3.x
|
||||||
|
echo $CGO_ENABLED # should print 1 (or be unset; we set it explicitly below)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Steps
|
||||||
|
|
||||||
|
### Step 1: Initialize the porting database
|
||||||
|
|
||||||
|
Create a fresh SQLite database with the porting tracker schema. This creates
|
||||||
|
`porting.db` in the repository root with all tables, indexes, and triggers.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
dotnet run --project tools/NatsNet.PortTracker -- init --db porting.db --schema porting-schema.sql
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected output:
|
||||||
|
|
||||||
|
```
|
||||||
|
Database initialized at porting.db
|
||||||
|
```
|
||||||
|
|
||||||
|
If the database already exists, this command is idempotent -- it applies
|
||||||
|
`CREATE TABLE IF NOT EXISTS` statements and will not destroy existing data.
|
||||||
|
|
||||||
|
### Step 2: Build the Go analyzer
|
||||||
|
|
||||||
|
The Go analyzer is a standalone tool that uses `go/ast` to parse Go source files
|
||||||
|
and writes results directly into the SQLite database.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd tools/go-analyzer && CGO_ENABLED=1 go build -o go-analyzer . && cd ../..
|
||||||
|
```
|
||||||
|
|
||||||
|
This produces the binary `tools/go-analyzer/go-analyzer`. If the build fails,
|
||||||
|
see the Troubleshooting section below.
|
||||||
|
|
||||||
|
### Step 3: Run the Go analyzer
|
||||||
|
|
||||||
|
Point the analyzer at the NATS server source and the porting database:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./tools/go-analyzer/go-analyzer \
|
||||||
|
--source golang/nats-server \
|
||||||
|
--db porting.db \
|
||||||
|
--schema porting-schema.sql
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected output (counts will vary with the NATS server version):
|
||||||
|
|
||||||
|
```
|
||||||
|
Analysis complete:
|
||||||
|
Modules: <N>
|
||||||
|
Features: <N>
|
||||||
|
Unit Tests: <N>
|
||||||
|
Dependencies: <N>
|
||||||
|
Imports: <N>
|
||||||
|
```
|
||||||
|
|
||||||
|
The analyzer does the following:
|
||||||
|
1. Walks `golang/nats-server/server/` for all `.go` files (skipping `configs/`
|
||||||
|
and `testdata/` directories).
|
||||||
|
2. Groups files into logical modules by directory.
|
||||||
|
3. Parses each non-test file, extracting every `func` and method as a feature.
|
||||||
|
4. Parses each `_test.go` file, extracting `Test*` and `Benchmark*` functions.
|
||||||
|
5. Infers module-level dependencies from cross-package imports.
|
||||||
|
6. Collects all import paths and classifies them as stdlib or external.
|
||||||
|
7. Writes modules, features, unit_tests, dependencies, and library_mappings to
|
||||||
|
the database.
|
||||||
|
|
||||||
|
### Step 4: Review module groupings
|
||||||
|
|
||||||
|
List all modules that were created to confirm the grouping makes sense:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
dotnet run --project tools/NatsNet.PortTracker -- module list --db porting.db
|
||||||
|
```
|
||||||
|
|
||||||
|
This shows every module with its ID, name, status, Go package, and line count.
|
||||||
|
Verify that the major areas of the NATS server are represented (e.g., core,
|
||||||
|
jetstream, client, auth, protocol, route, gateway, leafnode, mqtt, websocket,
|
||||||
|
monitoring, logging, errors, subscriptions, tls, events, raft, config, accounts).
|
||||||
|
|
||||||
|
### Step 5: Spot-check individual modules
|
||||||
|
|
||||||
|
Inspect a few modules to verify that features and tests were correctly extracted:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Check the first module
|
||||||
|
dotnet run --project tools/NatsNet.PortTracker -- module show 1 --db porting.db
|
||||||
|
|
||||||
|
# Check a few more
|
||||||
|
dotnet run --project tools/NatsNet.PortTracker -- module show 2 --db porting.db
|
||||||
|
dotnet run --project tools/NatsNet.PortTracker -- module show 3 --db porting.db
|
||||||
|
```
|
||||||
|
|
||||||
|
For each module, confirm that:
|
||||||
|
- Features list functions and methods from the corresponding Go files.
|
||||||
|
- Each feature has a `go_file`, `go_method`, and `go_line_number`.
|
||||||
|
- Tests are listed under the module and have `go_file` and `go_method` populated.
|
||||||
|
- Dependencies point to other valid modules.
|
||||||
|
|
||||||
|
### Step 6: Verify test extraction
|
||||||
|
|
||||||
|
List all extracted tests to confirm test files were parsed:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
dotnet run --project tools/NatsNet.PortTracker -- test list --db porting.db
|
||||||
|
```
|
||||||
|
|
||||||
|
Spot-check a few individual tests for detail:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
dotnet run --project tools/NatsNet.PortTracker -- test show 1 --db porting.db
|
||||||
|
```
|
||||||
|
|
||||||
|
Verify that each test has a module assignment and that the `feature_id` link is
|
||||||
|
populated where the analyzer could infer the connection from naming conventions
|
||||||
|
(e.g., `TestConnect` links to a feature named `Connect`).
|
||||||
|
|
||||||
|
### Step 7: Review inter-module dependencies
|
||||||
|
|
||||||
|
Check the dependency graph for a representative module:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
dotnet run --project tools/NatsNet.PortTracker -- dependency show module 1 --db porting.db
|
||||||
|
```
|
||||||
|
|
||||||
|
Also view the full blocked-items report to see the overall dependency shape:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
dotnet run --project tools/NatsNet.PortTracker -- dependency blocked --db porting.db
|
||||||
|
```
|
||||||
|
|
||||||
|
And check which items have no unported dependencies and are ready to start:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
dotnet run --project tools/NatsNet.PortTracker -- dependency ready --db porting.db
|
||||||
|
```
|
||||||
|
|
||||||
|
### Step 8: Review extracted library mappings
|
||||||
|
|
||||||
|
List all external Go libraries that were detected:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
dotnet run --project tools/NatsNet.PortTracker -- library list --db porting.db
|
||||||
|
```
|
||||||
|
|
||||||
|
All entries should have status `not_mapped` at this point. Library mapping is
|
||||||
|
handled in Phase 3.
|
||||||
|
|
||||||
|
### Step 9: Generate a baseline summary report
|
||||||
|
|
||||||
|
Create a summary snapshot to use as a reference for Phase 2 verification:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
dotnet run --project tools/NatsNet.PortTracker -- report summary --db porting.db
|
||||||
|
```
|
||||||
|
|
||||||
|
Optionally export a full markdown report for archival:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
dotnet run --project tools/NatsNet.PortTracker -- report export --format md --output docs/reports/phase-1-baseline.md --db porting.db
|
||||||
|
```
|
||||||
|
|
||||||
|
## Completion Criteria
|
||||||
|
|
||||||
|
Phase 1 is complete when ALL of the following are true:
|
||||||
|
|
||||||
|
- [ ] `porting.db` exists and contains data in all five tables (modules, features,
|
||||||
|
unit_tests, dependencies, library_mappings).
|
||||||
|
- [ ] All Go source files under `golang/nats-server/server/` are accounted for
|
||||||
|
(no files silently skipped without a logged warning).
|
||||||
|
- [ ] All public and private functions/methods are extracted as features.
|
||||||
|
- [ ] All `Test*` and `Benchmark*` functions are extracted as unit_tests.
|
||||||
|
- [ ] Test-to-feature links are populated where naming conventions allow inference.
|
||||||
|
- [ ] Module-level dependencies are recorded in the dependencies table.
|
||||||
|
- [ ] External import paths are recorded in the library_mappings table.
|
||||||
|
- [ ] `phase check 1` shows all checklist items marked `[x]`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
dotnet run --project tools/NatsNet.PortTracker -- phase check 1 --db porting.db
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected:
|
||||||
|
|
||||||
|
```
|
||||||
|
Phase 1: Analysis & Schema
|
||||||
|
Run Go AST analyzer, populate DB schema, map libraries
|
||||||
|
|
||||||
|
Phase 1 Checklist:
|
||||||
|
[x] Modules populated: <N>
|
||||||
|
[x] Features populated: <N>
|
||||||
|
[x] Unit tests populated: <N>
|
||||||
|
[x] Dependencies mapped: <N>
|
||||||
|
[x] Libraries identified: <N>
|
||||||
|
[ ] All libraries mapped: 0/<N>
|
||||||
|
```
|
||||||
|
|
||||||
|
Note: The "All libraries mapped" item will be unchecked -- that is expected.
|
||||||
|
Library mapping is the concern of Phase 3.
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
### CGO_ENABLED not set or build fails with "gcc not found"
|
||||||
|
|
||||||
|
The `go-sqlite3` driver requires C compilation. Make sure you have a C compiler
|
||||||
|
installed and CGO is enabled:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# macOS -- Xcode command line tools
|
||||||
|
xcode-select --install
|
||||||
|
|
||||||
|
# Then build with explicit CGO
|
||||||
|
CGO_ENABLED=1 go build -o go-analyzer .
|
||||||
|
```
|
||||||
|
|
||||||
|
### "cannot find package" errors during Go build
|
||||||
|
|
||||||
|
The analyzer depends on `github.com/mattn/go-sqlite3`. Run:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd tools/go-analyzer
|
||||||
|
go mod download
|
||||||
|
go mod verify
|
||||||
|
```
|
||||||
|
|
||||||
|
### Wrong source path
|
||||||
|
|
||||||
|
The `--source` flag must point to the root of the cloned nats-server repository
|
||||||
|
(the directory that contains the `server/` subdirectory). If you see
|
||||||
|
"discovering files: no such file or directory", verify:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ls golang/nats-server/server/
|
||||||
|
```
|
||||||
|
|
||||||
|
### Database locked errors
|
||||||
|
|
||||||
|
If you run the analyzer while another process has `porting.db` open, SQLite may
|
||||||
|
report a lock error. Close any other connections (including `sqlite3` CLI
|
||||||
|
sessions) and retry. The schema enables WAL mode to reduce lock contention:
|
||||||
|
|
||||||
|
```sql
|
||||||
|
PRAGMA journal_mode=WAL;
|
||||||
|
```
|
||||||
|
|
||||||
|
### Analyzer prints warnings but continues
|
||||||
|
|
||||||
|
Warnings like "Warning: skipping server/foo.go: <parse error>" mean an individual
|
||||||
|
file could not be parsed. The analyzer continues with remaining files. Investigate
|
||||||
|
any warnings -- they may indicate a Go version mismatch or syntax not yet
|
||||||
|
supported by the `go/ast` parser at your Go version.
|
||||||
|
|
||||||
|
### Empty database after analyzer runs
|
||||||
|
|
||||||
|
If the analyzer prints zeros for all counts, verify that:
|
||||||
|
1. The `--source` path is correct and contains Go files.
|
||||||
|
2. The `--schema` path points to a valid `porting-schema.sql`.
|
||||||
|
3. The `--db` path is writable.
|
||||||
|
|
||||||
|
You can inspect the database directly:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sqlite3 porting.db "SELECT COUNT(*) FROM modules;"
|
||||||
|
sqlite3 porting.db "SELECT COUNT(*) FROM features;"
|
||||||
|
sqlite3 porting.db "SELECT COUNT(*) FROM unit_tests;"
|
||||||
|
```
|
||||||
315
docs/plans/phases/phase-2-verification.md
Normal file
315
docs/plans/phases/phase-2-verification.md
Normal file
@@ -0,0 +1,315 @@
|
|||||||
|
# Phase 2: Verification of Captured Items
|
||||||
|
|
||||||
|
## Objective
|
||||||
|
|
||||||
|
Verify that the Phase 1 decomposition captured every Go source file, function,
|
||||||
|
test, and dependency accurately. Compare database counts against independent
|
||||||
|
baselines derived directly from the filesystem. Identify and fix any gaps before
|
||||||
|
proceeding to library mapping and porting.
|
||||||
|
|
||||||
|
## Prerequisites
|
||||||
|
|
||||||
|
- Phase 1 is complete (`porting.db` is populated).
|
||||||
|
- The Go source at `golang/nats-server/` has not changed since the Phase 1
|
||||||
|
analyzer run. If the source was updated, re-run the Phase 1 analyzer first.
|
||||||
|
- `dotnet`, `sqlite3`, `find`, `grep`, and `wc` are available on your PATH.
|
||||||
|
|
||||||
|
## Steps
|
||||||
|
|
||||||
|
### Step 1: Generate the summary report
|
||||||
|
|
||||||
|
Start with a high-level view of what the database contains:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
dotnet run --project tools/NatsNet.PortTracker -- report summary --db porting.db
|
||||||
|
```
|
||||||
|
|
||||||
|
Record the counts for modules, features, unit tests, and library mappings. These
|
||||||
|
are the numbers you will verify in subsequent steps.
|
||||||
|
|
||||||
|
### Step 2: Count Go source files on disk
|
||||||
|
|
||||||
|
Count non-test `.go` files under the server directory (the scope of the analyzer):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
find golang/nats-server/server -name "*.go" ! -name "*_test.go" ! -path "*/configs/*" ! -path "*/testdata/*" | wc -l
|
||||||
|
```
|
||||||
|
|
||||||
|
This should produce approximately 109 files. Compare this count against the
|
||||||
|
number of distinct `go_file` values in the features table:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sqlite3 porting.db "SELECT COUNT(DISTINCT go_file) FROM features;"
|
||||||
|
```
|
||||||
|
|
||||||
|
If the database count is lower, some source files may have been skipped. Check
|
||||||
|
the analyzer stderr output for warnings, or list the missing files:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sqlite3 porting.db "SELECT DISTINCT go_file FROM features ORDER BY go_file;" > /tmp/db_files.txt
|
||||||
|
find golang/nats-server/server -name "*.go" ! -name "*_test.go" ! -path "*/configs/*" ! -path "*/testdata/*" -exec realpath --relative-to=golang/nats-server {} \; | sort > /tmp/disk_files.txt
|
||||||
|
diff /tmp/db_files.txt /tmp/disk_files.txt
|
||||||
|
```
|
||||||
|
|
||||||
|
### Step 3: Count Go test files on disk
|
||||||
|
|
||||||
|
```bash
|
||||||
|
find golang/nats-server/server -name "*_test.go" ! -path "*/configs/*" ! -path "*/testdata/*" | wc -l
|
||||||
|
```
|
||||||
|
|
||||||
|
This should produce approximately 85 files. Compare against distinct test files
|
||||||
|
in the database:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sqlite3 porting.db "SELECT COUNT(DISTINCT go_file) FROM unit_tests;"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Step 4: Compare function counts
|
||||||
|
|
||||||
|
Count all exported and unexported functions in source files on disk:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
grep -r "^func " golang/nats-server/server/ --include="*.go" --exclude="*_test.go" | grep -v "/configs/" | grep -v "/testdata/" | wc -l
|
||||||
|
```
|
||||||
|
|
||||||
|
Compare against the features count from the database:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sqlite3 porting.db "SELECT COUNT(*) FROM features;"
|
||||||
|
```
|
||||||
|
|
||||||
|
The numbers should be close. Small discrepancies can occur because:
|
||||||
|
- The `grep` approach counts lines starting with `func` which may miss functions
|
||||||
|
with preceding comments on the same line or multi-line signatures.
|
||||||
|
- The AST parser used by the analyzer is more accurate; it finds all `func`
|
||||||
|
declarations regardless of formatting.
|
||||||
|
|
||||||
|
If the database count is significantly lower (more than 5% off), investigate.
|
||||||
|
|
||||||
|
### Step 5: Compare test function counts
|
||||||
|
|
||||||
|
Count test functions on disk:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
grep -r "^func Test" golang/nats-server/server/ --include="*_test.go" | wc -l
|
||||||
|
```
|
||||||
|
|
||||||
|
Also count benchmarks:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
grep -r "^func Benchmark" golang/nats-server/server/ --include="*_test.go" | wc -l
|
||||||
|
```
|
||||||
|
|
||||||
|
Compare the combined total against the unit_tests table:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sqlite3 porting.db "SELECT COUNT(*) FROM unit_tests;"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Step 6: Run the phase check command
|
||||||
|
|
||||||
|
The PortTracker has a built-in Phase 1 checklist that verifies all tables are
|
||||||
|
populated:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
dotnet run --project tools/NatsNet.PortTracker -- phase check 1 --db porting.db
|
||||||
|
```
|
||||||
|
|
||||||
|
All items except "All libraries mapped" should show `[x]`.
|
||||||
|
|
||||||
|
### Step 7: Check for orphaned items
|
||||||
|
|
||||||
|
Look for features that are not linked to any module (should be zero):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sqlite3 porting.db "SELECT COUNT(*) FROM features WHERE module_id NOT IN (SELECT id FROM modules);"
|
||||||
|
```
|
||||||
|
|
||||||
|
Look for tests that are not linked to any module (should be zero):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sqlite3 porting.db "SELECT COUNT(*) FROM unit_tests WHERE module_id NOT IN (SELECT id FROM modules);"
|
||||||
|
```
|
||||||
|
|
||||||
|
Look for test-to-feature links that point to non-existent features:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sqlite3 porting.db "SELECT COUNT(*) FROM unit_tests WHERE feature_id IS NOT NULL AND feature_id NOT IN (SELECT id FROM features);"
|
||||||
|
```
|
||||||
|
|
||||||
|
Look for dependencies that reference non-existent source or target items:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sqlite3 porting.db "
|
||||||
|
SELECT COUNT(*) FROM dependencies
|
||||||
|
WHERE (source_type = 'module' AND source_id NOT IN (SELECT id FROM modules))
|
||||||
|
OR (target_type = 'module' AND target_id NOT IN (SELECT id FROM modules))
|
||||||
|
OR (source_type = 'feature' AND source_id NOT IN (SELECT id FROM features))
|
||||||
|
OR (target_type = 'feature' AND target_id NOT IN (SELECT id FROM features))
|
||||||
|
OR (source_type = 'unit_test' AND source_id NOT IN (SELECT id FROM unit_tests))
|
||||||
|
OR (target_type = 'unit_test' AND target_id NOT IN (SELECT id FROM unit_tests));
|
||||||
|
"
|
||||||
|
```
|
||||||
|
|
||||||
|
All of these queries should return 0.
|
||||||
|
|
||||||
|
### Step 8: Review the largest modules
|
||||||
|
|
||||||
|
The largest modules are the most likely to have issues. List modules sorted by
|
||||||
|
feature count:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sqlite3 porting.db "
|
||||||
|
SELECT m.id, m.name, m.go_line_count,
|
||||||
|
COUNT(f.id) as feature_count
|
||||||
|
FROM modules m
|
||||||
|
LEFT JOIN features f ON f.module_id = m.id
|
||||||
|
GROUP BY m.id
|
||||||
|
ORDER BY feature_count DESC
|
||||||
|
LIMIT 10;
|
||||||
|
"
|
||||||
|
```
|
||||||
|
|
||||||
|
For each of the top 3 modules, do a manual spot-check:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
dotnet run --project tools/NatsNet.PortTracker -- module show <id> --db porting.db
|
||||||
|
```
|
||||||
|
|
||||||
|
Scroll through the features list and verify that the functions look correct
|
||||||
|
(check a few against the actual Go source file).
|
||||||
|
|
||||||
|
### Step 9: Validate the dependency graph
|
||||||
|
|
||||||
|
Check for any circular module dependencies (modules that depend on each other):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sqlite3 porting.db "
|
||||||
|
SELECT d1.source_id, d1.target_id
|
||||||
|
FROM dependencies d1
|
||||||
|
JOIN dependencies d2
|
||||||
|
ON d1.source_type = d2.target_type AND d1.source_id = d2.target_id
|
||||||
|
AND d1.target_type = d2.source_type AND d1.target_id = d2.source_id
|
||||||
|
WHERE d1.source_type = 'module' AND d1.target_type = 'module';
|
||||||
|
"
|
||||||
|
```
|
||||||
|
|
||||||
|
Circular dependencies are not necessarily wrong (Go packages can have them via
|
||||||
|
interfaces), but they should be reviewed.
|
||||||
|
|
||||||
|
Check which items are blocked by unported dependencies:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
dotnet run --project tools/NatsNet.PortTracker -- dependency blocked --db porting.db
|
||||||
|
```
|
||||||
|
|
||||||
|
And confirm that at least some items are ready to port (have no unported deps):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
dotnet run --project tools/NatsNet.PortTracker -- dependency ready --db porting.db
|
||||||
|
```
|
||||||
|
|
||||||
|
### Step 10: Verify library import completeness
|
||||||
|
|
||||||
|
Ensure every external import found in the source is tracked:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sqlite3 porting.db "SELECT COUNT(*) FROM library_mappings;"
|
||||||
|
```
|
||||||
|
|
||||||
|
Cross-check against a manual count of unique non-stdlib imports:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
grep -rh "\"" golang/nats-server/server/ --include="*.go" | \
|
||||||
|
grep -oP '"\K[^"]+' | \
|
||||||
|
grep '\.' | \
|
||||||
|
sort -u | \
|
||||||
|
wc -l
|
||||||
|
```
|
||||||
|
|
||||||
|
This is an approximate check. The AST-based analyzer is more accurate than grep
|
||||||
|
for import extraction, but the numbers should be in the same ballpark.
|
||||||
|
|
||||||
|
### Step 11: Export a verification snapshot
|
||||||
|
|
||||||
|
Save the current state as a markdown report for your records:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
dotnet run --project tools/NatsNet.PortTracker -- report export \
|
||||||
|
--format md \
|
||||||
|
--output docs/reports/phase-2-verification.md \
|
||||||
|
--db porting.db
|
||||||
|
```
|
||||||
|
|
||||||
|
## Completion Criteria
|
||||||
|
|
||||||
|
Phase 2 is complete when ALL of the following are true:
|
||||||
|
|
||||||
|
- [ ] Source file counts on disk match distinct `go_file` counts in the database
|
||||||
|
(within a small margin for intentionally excluded directories).
|
||||||
|
- [ ] Feature counts from `grep` are within 5% of the database count (AST is the
|
||||||
|
authoritative source).
|
||||||
|
- [ ] Test function counts from `grep` match the database count closely.
|
||||||
|
- [ ] No orphaned features (all linked to valid modules).
|
||||||
|
- [ ] No orphaned tests (all linked to valid modules).
|
||||||
|
- [ ] No broken test-to-feature links.
|
||||||
|
- [ ] No dangling dependency references.
|
||||||
|
- [ ] Dependency graph is reviewed -- circular deps (if any) are acknowledged.
|
||||||
|
- [ ] `dependency ready` returns at least one item (the graph has valid roots).
|
||||||
|
- [ ] Library mappings table contains all external imports.
|
||||||
|
- [ ] `phase check 1` passes with all items except "All libraries mapped" checked.
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
### File count mismatch is large
|
||||||
|
|
||||||
|
If the disk file count exceeds the database count by more than a few files,
|
||||||
|
re-run the analyzer with stderr visible:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./tools/go-analyzer/go-analyzer \
|
||||||
|
--source golang/nats-server \
|
||||||
|
--db porting.db \
|
||||||
|
--schema porting-schema.sql 2>&1 | tee /tmp/analyzer.log
|
||||||
|
```
|
||||||
|
|
||||||
|
Search for warnings:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
grep "Warning" /tmp/analyzer.log
|
||||||
|
```
|
||||||
|
|
||||||
|
Common causes:
|
||||||
|
- Files with build tags that prevent parsing (e.g., `//go:build ignore`).
|
||||||
|
- Files in excluded directories (`configs/`, `testdata/`).
|
||||||
|
- Syntax errors in Go files that the parser cannot handle.
|
||||||
|
|
||||||
|
### Feature count is significantly different
|
||||||
|
|
||||||
|
The AST parser counts every `func` declaration, including unexported helper
|
||||||
|
functions. The `grep` baseline only matches lines starting with `func `. If
|
||||||
|
features that have multiline signatures like:
|
||||||
|
|
||||||
|
```go
|
||||||
|
func (s *Server) handleConnection(
|
||||||
|
conn net.Conn,
|
||||||
|
) {
|
||||||
|
```
|
||||||
|
|
||||||
|
...they will be missed by grep but found by the AST parser. Trust the database
|
||||||
|
count as authoritative.
|
||||||
|
|
||||||
|
### Orphaned records found
|
||||||
|
|
||||||
|
If orphaned records exist, the analyzer may have a bug or the database was
|
||||||
|
partially populated from a prior run. The safest fix is to:
|
||||||
|
|
||||||
|
1. Delete the database: `rm porting.db`
|
||||||
|
2. Re-run Phase 1 from Step 1.
|
||||||
|
|
||||||
|
### Tests not linked to features
|
||||||
|
|
||||||
|
The analyzer uses naming conventions to link tests to features (e.g.,
|
||||||
|
`TestConnect` maps to a feature containing `Connect`). If many tests show
|
||||||
|
`feature_id = NULL`, this is expected for tests whose names do not follow the
|
||||||
|
convention. These links can be manually added later if needed.
|
||||||
283
docs/plans/phases/phase-3-library-mapping.md
Normal file
283
docs/plans/phases/phase-3-library-mapping.md
Normal file
@@ -0,0 +1,283 @@
|
|||||||
|
# Phase 3: Library Mapping
|
||||||
|
|
||||||
|
## Objective
|
||||||
|
|
||||||
|
Map every external Go dependency detected by the analyzer to its .NET equivalent.
|
||||||
|
This includes Go standard library packages, well-known third-party libraries, and
|
||||||
|
NATS ecosystem packages. When Phase 3 is complete, the `library suggest` command
|
||||||
|
returns an empty list and every import has a documented .NET migration path.
|
||||||
|
|
||||||
|
## Prerequisites
|
||||||
|
|
||||||
|
- Phases 1 and 2 are complete (database is populated and verified).
|
||||||
|
- Familiarity with both the Go standard library and .NET BCL / NuGet ecosystem.
|
||||||
|
- A working `dotnet` CLI for running PortTracker commands.
|
||||||
|
|
||||||
|
## Steps
|
||||||
|
|
||||||
|
### Step 1: List all unmapped libraries
|
||||||
|
|
||||||
|
Start by seeing everything that needs attention:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
dotnet run --project tools/NatsNet.PortTracker -- library suggest --db porting.db
|
||||||
|
```
|
||||||
|
|
||||||
|
This shows all library_mappings entries with status `not_mapped`, sorted by import
|
||||||
|
path. The output includes the Go import path, the library name, and a usage
|
||||||
|
description.
|
||||||
|
|
||||||
|
If the list is empty, all libraries are already mapped and Phase 3 is complete.
|
||||||
|
|
||||||
|
### Step 2: Review the full library list
|
||||||
|
|
||||||
|
To see both mapped and unmapped libraries in one view:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
dotnet run --project tools/NatsNet.PortTracker -- library list --db porting.db
|
||||||
|
```
|
||||||
|
|
||||||
|
You can also filter by status:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
dotnet run --project tools/NatsNet.PortTracker -- library list --status not_mapped --db porting.db
|
||||||
|
dotnet run --project tools/NatsNet.PortTracker -- library list --status mapped --db porting.db
|
||||||
|
```
|
||||||
|
|
||||||
|
### Step 3: Map each library
|
||||||
|
|
||||||
|
For each unmapped library, determine the appropriate .NET equivalent using the
|
||||||
|
reference table below, then record the mapping:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
dotnet run --project tools/NatsNet.PortTracker -- library map <id> \
|
||||||
|
--package "<NuGet package or BCL>" \
|
||||||
|
--namespace "<.NET namespace>" \
|
||||||
|
--notes "<migration notes>" \
|
||||||
|
--db porting.db
|
||||||
|
```
|
||||||
|
|
||||||
|
**Example -- mapping `encoding/json`:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
dotnet run --project tools/NatsNet.PortTracker -- library map 1 \
|
||||||
|
--package "System.Text.Json" \
|
||||||
|
--namespace "System.Text.Json" \
|
||||||
|
--notes "Use JsonSerializer. Consider source generators for AOT." \
|
||||||
|
--db porting.db
|
||||||
|
```
|
||||||
|
|
||||||
|
**Example -- mapping `github.com/klauspost/compress`:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
dotnet run --project tools/NatsNet.PortTracker -- library map 12 \
|
||||||
|
--package "System.IO.Compression" \
|
||||||
|
--namespace "System.IO.Compression" \
|
||||||
|
--notes "S2/Snappy codec needs evaluation; may need custom impl or IronSnappy NuGet." \
|
||||||
|
--db porting.db
|
||||||
|
```
|
||||||
|
|
||||||
|
Repeat for every entry in the `library suggest` output.
|
||||||
|
|
||||||
|
### Step 4: Handle libraries that need custom implementations
|
||||||
|
|
||||||
|
Some Go libraries have no direct .NET equivalent and will require custom code.
|
||||||
|
For these, record the mapping with a descriptive note:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
dotnet run --project tools/NatsNet.PortTracker -- library map <id> \
|
||||||
|
--package "Custom" \
|
||||||
|
--namespace "NatsNet.Internal" \
|
||||||
|
--notes "No direct equivalent; requires custom implementation. See <details>." \
|
||||||
|
--db porting.db
|
||||||
|
```
|
||||||
|
|
||||||
|
### Step 5: Verify all libraries are mapped
|
||||||
|
|
||||||
|
Run the suggest command again -- it should return an empty list:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
dotnet run --project tools/NatsNet.PortTracker -- library suggest --db porting.db
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected output:
|
||||||
|
|
||||||
|
```
|
||||||
|
All libraries have been mapped!
|
||||||
|
```
|
||||||
|
|
||||||
|
Also verify via the full list:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
dotnet run --project tools/NatsNet.PortTracker -- library list --db porting.db
|
||||||
|
```
|
||||||
|
|
||||||
|
Every entry should show status `mapped` (or `verified` if you have already
|
||||||
|
validated the mapping in code).
|
||||||
|
|
||||||
|
### Step 6: Run the phase check
|
||||||
|
|
||||||
|
Confirm Phase 1 now shows full completion including library mappings:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
dotnet run --project tools/NatsNet.PortTracker -- phase check 1 --db porting.db
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected:
|
||||||
|
|
||||||
|
```
|
||||||
|
Phase 1 Checklist:
|
||||||
|
[x] Modules populated: <N>
|
||||||
|
[x] Features populated: <N>
|
||||||
|
[x] Unit tests populated: <N>
|
||||||
|
[x] Dependencies mapped: <N>
|
||||||
|
[x] Libraries identified: <N>
|
||||||
|
[x] All libraries mapped: <N>/<N>
|
||||||
|
```
|
||||||
|
|
||||||
|
### Step 7: Export the mapping report
|
||||||
|
|
||||||
|
Save the complete library mapping state for reference during porting:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
dotnet run --project tools/NatsNet.PortTracker -- report export \
|
||||||
|
--format md \
|
||||||
|
--output docs/reports/phase-3-library-mapping.md \
|
||||||
|
--db porting.db
|
||||||
|
```
|
||||||
|
|
||||||
|
## Common Go to .NET Mappings Reference
|
||||||
|
|
||||||
|
Use this table as a starting point when determining .NET equivalents. Adapt based
|
||||||
|
on the specific usage patterns found in the NATS server source.
|
||||||
|
|
||||||
|
### Go Standard Library
|
||||||
|
|
||||||
|
| Go Package | .NET Equivalent | Notes |
|
||||||
|
|---|---|---|
|
||||||
|
| `encoding/json` | `System.Text.Json` | Use `JsonSerializer`; consider source generators for performance |
|
||||||
|
| `encoding/binary` | `System.Buffers.Binary.BinaryPrimitives` | For endian-aware reads/writes |
|
||||||
|
| `encoding/base64` | `System.Convert` | `Convert.ToBase64String` / `Convert.FromBase64String` |
|
||||||
|
| `encoding/hex` | `System.Convert` | `Convert.ToHexString` (.NET 5+) |
|
||||||
|
| `encoding/pem` | `System.Security.Cryptography.PemEncoding` | .NET 5+ |
|
||||||
|
| `sync` | `System.Threading` | `Mutex` -> `lock` / `Monitor`; `RWMutex` -> `ReaderWriterLockSlim`; `WaitGroup` -> `CountdownEvent`; `Once` -> `Lazy<T>` |
|
||||||
|
| `sync/atomic` | `System.Threading.Interlocked` | `Interlocked.Increment`, `CompareExchange`, etc. |
|
||||||
|
| `net` | `System.Net.Sockets` | `TcpListener`, `TcpClient`, `Socket` |
|
||||||
|
| `net/http` | `System.Net.Http` / `Microsoft.AspNetCore` | Client: `HttpClient`; Server: Kestrel / minimal APIs |
|
||||||
|
| `net/url` | `System.Uri` | `Uri`, `UriBuilder` |
|
||||||
|
| `crypto/tls` | `System.Net.Security.SslStream` | Wrap `NetworkStream` with `SslStream` |
|
||||||
|
| `crypto/x509` | `System.Security.Cryptography.X509Certificates` | `X509Certificate2` |
|
||||||
|
| `crypto/sha256` | `System.Security.Cryptography.SHA256` | `SHA256.HashData()` (.NET 8+) |
|
||||||
|
| `crypto/ed25519` | `System.Security.Cryptography` | `Ed25519` support in .NET 9+ |
|
||||||
|
| `crypto/rand` | `System.Security.Cryptography.RandomNumberGenerator` | `RandomNumberGenerator.Fill()` |
|
||||||
|
| `time` | `System.TimeSpan` / `System.Threading.Timer` | `time.Duration` -> `TimeSpan`; `time.Ticker` -> `PeriodicTimer`; `time.After` -> `Task.Delay` |
|
||||||
|
| `time` (parsing) | `System.DateTime` / `System.DateTimeOffset` | `DateTime.Parse`, custom formats |
|
||||||
|
| `fmt` | String interpolation / `String.Format` | `$"..."` for most cases; `String.Format` for dynamic |
|
||||||
|
| `io` | `System.IO` | `Reader` -> `Stream`; `Writer` -> `Stream`; `io.Copy` -> `Stream.CopyTo` |
|
||||||
|
| `io/fs` | `System.IO` | `Directory`, `File`, `FileInfo` |
|
||||||
|
| `bufio` | `System.IO.BufferedStream` | Or `StreamReader` / `StreamWriter` |
|
||||||
|
| `bytes` | `System.Buffers` / `MemoryStream` | `bytes.Buffer` -> `MemoryStream` or `ArrayBufferWriter<byte>` |
|
||||||
|
| `strings` | `System.String` / `System.Text.StringBuilder` | Most methods have direct equivalents |
|
||||||
|
| `strconv` | `int.Parse`, `double.Parse`, etc. | Or `Convert` class |
|
||||||
|
| `context` | `CancellationToken` | `context.Context` -> `CancellationToken`; `context.WithCancel` -> `CancellationTokenSource` |
|
||||||
|
| `os` | `System.Environment` / `System.IO` | `os.Exit` -> `Environment.Exit`; file ops -> `File` class |
|
||||||
|
| `os/signal` | `System.Runtime.Loader.AssemblyLoadContext` | `UnloadingEvent` or `Console.CancelKeyPress` |
|
||||||
|
| `path/filepath` | `System.IO.Path` | `Path.Combine`, `Path.GetDirectoryName`, etc. |
|
||||||
|
| `sort` | `System.Linq` / `Array.Sort` | LINQ `.OrderBy()` or in-place `Array.Sort` |
|
||||||
|
| `math` | `System.Math` | Direct equivalent |
|
||||||
|
| `math/rand` | `System.Random` | `Random.Shared` for thread-safe usage (.NET 6+) |
|
||||||
|
| `regexp` | `System.Text.RegularExpressions.Regex` | Consider source generators for compiled patterns |
|
||||||
|
| `errors` | `System.Exception` | Go errors -> .NET exceptions; `errors.Is` -> pattern matching |
|
||||||
|
| `log` | `Serilog` | Project choice: Serilog via `Microsoft.Extensions.Logging` |
|
||||||
|
| `testing` | `xUnit` | `testing.T` -> xUnit `[Fact]`/`[Theory]`; `testing.B` -> BenchmarkDotNet |
|
||||||
|
| `flag` | `System.CommandLine` | Or `Microsoft.Extensions.Configuration` |
|
||||||
|
| `embed` | Embedded resources | `.csproj` `<EmbeddedResource>` items |
|
||||||
|
| `runtime` | `System.Runtime` / `System.Environment` | `runtime.GOOS` -> `RuntimeInformation.IsOSPlatform` |
|
||||||
|
|
||||||
|
### NATS Ecosystem Libraries
|
||||||
|
|
||||||
|
| Go Package | .NET Equivalent | Notes |
|
||||||
|
|---|---|---|
|
||||||
|
| `github.com/nats-io/jwt/v2` | Custom / evaluate existing | JWT claims for NATS auth; may need custom implementation matching NATS JWT spec |
|
||||||
|
| `github.com/nats-io/nkeys` | Custom implementation | Ed25519 key pairs for NATS authentication; use `System.Security.Cryptography` Ed25519 |
|
||||||
|
| `github.com/nats-io/nuid` | Custom / `System.Guid` | NATS unique IDs; simple custom implementation or adapt to `Guid` if format is flexible |
|
||||||
|
| `github.com/nats-io/nats.go` | `NATS.Net` (official) | Only used in tests; the official .NET NATS client |
|
||||||
|
|
||||||
|
### Third-Party Libraries
|
||||||
|
|
||||||
|
| Go Package | .NET Equivalent | Notes |
|
||||||
|
|---|---|---|
|
||||||
|
| `github.com/klauspost/compress` | `System.IO.Compression` | General compression: `GZipStream`, `DeflateStream`. S2/Snappy: evaluate IronSnappy NuGet or custom port |
|
||||||
|
| `github.com/minio/highwayhash` | Custom / NuGet | HighwayHash implementation; search NuGet or port the algorithm |
|
||||||
|
| `golang.org/x/crypto` | `System.Security.Cryptography` | `bcrypt` -> `Rfc2898DeriveBytes` or BCrypt.Net NuGet; `argon2` -> Konscious.Security NuGet |
|
||||||
|
| `golang.org/x/sys` | `System.Runtime.InteropServices` | Platform-specific syscalls -> P/Invoke or `RuntimeInformation` |
|
||||||
|
| `golang.org/x/time` | `System.Threading.RateLimiting` | `rate.Limiter` -> `RateLimiter` (.NET 7+); `TokenBucketRateLimiter` or `SlidingWindowRateLimiter` |
|
||||||
|
| `golang.org/x/text` | `System.Globalization` | Unicode normalization, encoding detection |
|
||||||
|
|
||||||
|
## Mapping Decision Guidelines
|
||||||
|
|
||||||
|
When choosing a .NET equivalent, follow these priorities:
|
||||||
|
|
||||||
|
1. **BCL first**: Prefer built-in .NET Base Class Library types over NuGet packages.
|
||||||
|
2. **Official packages second**: If BCL does not cover it, prefer
|
||||||
|
`Microsoft.*` or `System.*` NuGet packages.
|
||||||
|
3. **Well-maintained NuGet third**: Choose packages with active maintenance,
|
||||||
|
high download counts, and compatible licenses.
|
||||||
|
4. **Custom implementation last**: Only write custom code when no suitable
|
||||||
|
package exists. Document the rationale in the mapping notes.
|
||||||
|
|
||||||
|
For each mapping, consider:
|
||||||
|
- **API surface**: Does the .NET equivalent cover all methods used in the Go code?
|
||||||
|
- **Performance**: Are there performance-critical paths that need benchmarking?
|
||||||
|
- **Thread safety**: Go's concurrency model differs from .NET. Note any
|
||||||
|
synchronization concerns.
|
||||||
|
- **Platform support**: Does the .NET package work on all target platforms
|
||||||
|
(Linux, macOS, Windows)?
|
||||||
|
|
||||||
|
## Completion Criteria
|
||||||
|
|
||||||
|
Phase 3 is complete when ALL of the following are true:
|
||||||
|
|
||||||
|
- [ ] `library suggest` returns "All libraries have been mapped!"
|
||||||
|
- [ ] Every entry in `library list` shows status `mapped` or `verified`.
|
||||||
|
- [ ] Each mapping includes a `--package` (the NuGet package or BCL assembly),
|
||||||
|
a `--namespace` (the .NET namespace to use), and `--notes` (migration
|
||||||
|
guidance).
|
||||||
|
- [ ] `phase check 1` shows all items checked including "All libraries mapped".
|
||||||
|
- [ ] A mapping report has been exported for reference.
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
### "Library <id> not found"
|
||||||
|
|
||||||
|
The ID you passed to `library map` does not exist. Run `library suggest` to get
|
||||||
|
the current list of IDs and their import paths.
|
||||||
|
|
||||||
|
### Unsure which .NET package to use
|
||||||
|
|
||||||
|
For unfamiliar Go packages:
|
||||||
|
1. Check what the package does in the Go source (look at the import usage in the
|
||||||
|
files listed by the analyzer).
|
||||||
|
2. Search NuGet.org for equivalent functionality.
|
||||||
|
3. Check if the Go package is a thin wrapper around a well-known algorithm that
|
||||||
|
.NET implements natively.
|
||||||
|
4. When in doubt, map it as "Custom" with detailed notes and revisit during
|
||||||
|
the porting phase.
|
||||||
|
|
||||||
|
### Multiple .NET options for one Go package
|
||||||
|
|
||||||
|
When there are several valid .NET equivalents (e.g., `Newtonsoft.Json` vs
|
||||||
|
`System.Text.Json`), prefer the one that:
|
||||||
|
- Is part of the BCL or a Microsoft package.
|
||||||
|
- Has better performance characteristics.
|
||||||
|
- Has source generator support for AOT compilation.
|
||||||
|
|
||||||
|
Record the alternatives in the `--notes` field so the decision can be revisited.
|
||||||
|
|
||||||
|
### Stdlib packages showing as unmapped
|
||||||
|
|
||||||
|
The analyzer classifies imports as stdlib vs external based on whether the first
|
||||||
|
path component contains a dot. Standard library packages like `encoding/json`,
|
||||||
|
`net/http`, etc. should still be recorded in the library_mappings table so that
|
||||||
|
every import path has a documented .NET migration path. Map them using the
|
||||||
|
reference table above.
|
||||||
Reference in New Issue
Block a user