docs: add phase 1-3 instruction guides
This commit is contained in:
Joseph Doherty
291
docs/plans/phases/phase-1-decomposition.md
Normal file
291
docs/plans/phases/phase-1-decomposition.md
Normal file
@@ -0,0 +1,291 @@
|
||||
# Phase 1: Go Codebase Decomposition
|
||||
|
||||
## Objective
|
||||
|
||||
Parse the Go NATS server source code into a structured SQLite database, extracting
|
||||
modules, features (functions/methods), unit tests, external library imports, and
|
||||
inter-module dependencies. This database becomes the single source of truth that
|
||||
drives all subsequent porting phases.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
| Requirement | Version / Notes |
|
||||
|---|---|
|
||||
| Go | 1.25+ (required by `tools/go-analyzer/go.mod`) |
|
||||
| .NET SDK | 10.0+ |
|
||||
| SQLite3 CLI | 3.x (optional, for manual inspection) |
|
||||
| CGO | Must be enabled (`CGO_ENABLED=1`); the Go analyzer uses `github.com/mattn/go-sqlite3` |
|
||||
| Go source | Cloned at `golang/nats-server/` relative to the repo root |
|
||||
|
||||
Verify prerequisites before starting:
|
||||
|
||||
```bash
|
||||
go version # should print go1.25 or later
|
||||
dotnet --version # should print 10.x
|
||||
sqlite3 --version # optional, any 3.x
|
||||
echo $CGO_ENABLED # should print 1 (or be unset; we set it explicitly below)
|
||||
```
|
||||
|
||||
## Steps
|
||||
|
||||
### Step 1: Initialize the porting database
|
||||
|
||||
Create a fresh SQLite database with the porting tracker schema. This creates
|
||||
`porting.db` in the repository root with all tables, indexes, and triggers.
|
||||
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- init --db porting.db --schema porting-schema.sql
|
||||
```
|
||||
|
||||
Expected output:
|
||||
|
||||
```
|
||||
Database initialized at porting.db
|
||||
```
|
||||
|
||||
If the database already exists, this command is idempotent -- it applies
|
||||
`CREATE TABLE IF NOT EXISTS` statements and will not destroy existing data.
|
||||
|
||||
### Step 2: Build the Go analyzer
|
||||
|
||||
The Go analyzer is a standalone tool that uses `go/ast` to parse Go source files
|
||||
and writes results directly into the SQLite database.
|
||||
|
||||
```bash
|
||||
cd tools/go-analyzer && CGO_ENABLED=1 go build -o go-analyzer . && cd ../..
|
||||
```
|
||||
|
||||
This produces the binary `tools/go-analyzer/go-analyzer`. If the build fails,
|
||||
see the Troubleshooting section below.
|
||||
|
||||
### Step 3: Run the Go analyzer
|
||||
|
||||
Point the analyzer at the NATS server source and the porting database:
|
||||
|
||||
```bash
|
||||
./tools/go-analyzer/go-analyzer \
|
||||
--source golang/nats-server \
|
||||
--db porting.db \
|
||||
--schema porting-schema.sql
|
||||
```
|
||||
|
||||
Expected output (counts will vary with the NATS server version):
|
||||
|
||||
```
|
||||
Analysis complete:
|
||||
Modules: <N>
|
||||
Features: <N>
|
||||
Unit Tests: <N>
|
||||
Dependencies: <N>
|
||||
Imports: <N>
|
||||
```
|
||||
|
||||
The analyzer does the following:
|
||||
1. Walks `golang/nats-server/server/` for all `.go` files (skipping `configs/`
|
||||
and `testdata/` directories).
|
||||
2. Groups files into logical modules by directory.
|
||||
3. Parses each non-test file, extracting every `func` and method as a feature.
|
||||
4. Parses each `_test.go` file, extracting `Test*` and `Benchmark*` functions.
|
||||
5. Infers module-level dependencies from cross-package imports.
|
||||
6. Collects all import paths and classifies them as stdlib or external.
|
||||
7. Writes modules, features, unit_tests, dependencies, and library_mappings to
|
||||
the database.
|
||||
|
||||
### Step 4: Review module groupings
|
||||
|
||||
List all modules that were created to confirm the grouping makes sense:
|
||||
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- module list --db porting.db
|
||||
```
|
||||
|
||||
This shows every module with its ID, name, status, Go package, and line count.
|
||||
Verify that the major areas of the NATS server are represented (e.g., core,
|
||||
jetstream, client, auth, protocol, route, gateway, leafnode, mqtt, websocket,
|
||||
monitoring, logging, errors, subscriptions, tls, events, raft, config, accounts).
|
||||
|
||||
### Step 5: Spot-check individual modules
|
||||
|
||||
Inspect a few modules to verify that features and tests were correctly extracted:
|
||||
|
||||
```bash
|
||||
# Check the first module
|
||||
dotnet run --project tools/NatsNet.PortTracker -- module show 1 --db porting.db
|
||||
|
||||
# Check a few more
|
||||
dotnet run --project tools/NatsNet.PortTracker -- module show 2 --db porting.db
|
||||
dotnet run --project tools/NatsNet.PortTracker -- module show 3 --db porting.db
|
||||
```
|
||||
|
||||
For each module, confirm that:
|
||||
- Features list functions and methods from the corresponding Go files.
|
||||
- Each feature has a `go_file`, `go_method`, and `go_line_number`.
|
||||
- Tests are listed under the module and have `go_file` and `go_method` populated.
|
||||
- Dependencies point to other valid modules.
|
||||
|
||||
### Step 6: Verify test extraction
|
||||
|
||||
List all extracted tests to confirm test files were parsed:
|
||||
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- test list --db porting.db
|
||||
```
|
||||
|
||||
Spot-check a few individual tests for detail:
|
||||
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- test show 1 --db porting.db
|
||||
```
|
||||
|
||||
Verify that each test has a module assignment and that the `feature_id` link is
|
||||
populated where the analyzer could infer the connection from naming conventions
|
||||
(e.g., `TestConnect` links to a feature named `Connect`).
|
||||
|
||||
### Step 7: Review inter-module dependencies
|
||||
|
||||
Check the dependency graph for a representative module:
|
||||
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- dependency show module 1 --db porting.db
|
||||
```
|
||||
|
||||
Also view the full blocked-items report to see the overall dependency shape:
|
||||
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- dependency blocked --db porting.db
|
||||
```
|
||||
|
||||
And check which items have no unported dependencies and are ready to start:
|
||||
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- dependency ready --db porting.db
|
||||
```
|
||||
|
||||
### Step 8: Review extracted library mappings
|
||||
|
||||
List all external Go libraries that were detected:
|
||||
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- library list --db porting.db
|
||||
```
|
||||
|
||||
All entries should have status `not_mapped` at this point. Library mapping is
|
||||
handled in Phase 3.
|
||||
|
||||
### Step 9: Generate a baseline summary report
|
||||
|
||||
Create a summary snapshot to use as a reference for Phase 2 verification:
|
||||
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- report summary --db porting.db
|
||||
```
|
||||
|
||||
Optionally export a full markdown report for archival:
|
||||
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- report export --format md --output docs/reports/phase-1-baseline.md --db porting.db
|
||||
```
|
||||
|
||||
## Completion Criteria
|
||||
|
||||
Phase 1 is complete when ALL of the following are true:
|
||||
|
||||
- [ ] `porting.db` exists and contains data in all five tables (modules, features,
|
||||
unit_tests, dependencies, library_mappings).
|
||||
- [ ] All Go source files under `golang/nats-server/server/` are accounted for
|
||||
(no files silently skipped without a logged warning).
|
||||
- [ ] All public and private functions/methods are extracted as features.
|
||||
- [ ] All `Test*` and `Benchmark*` functions are extracted as unit_tests.
|
||||
- [ ] Test-to-feature links are populated where naming conventions allow inference.
|
||||
- [ ] Module-level dependencies are recorded in the dependencies table.
|
||||
- [ ] External import paths are recorded in the library_mappings table.
|
||||
- [ ] `phase check 1` shows all checklist items marked `[x]`:
|
||||
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- phase check 1 --db porting.db
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
```
|
||||
Phase 1: Analysis & Schema
|
||||
Run Go AST analyzer, populate DB schema, map libraries
|
||||
|
||||
Phase 1 Checklist:
|
||||
[x] Modules populated: <N>
|
||||
[x] Features populated: <N>
|
||||
[x] Unit tests populated: <N>
|
||||
[x] Dependencies mapped: <N>
|
||||
[x] Libraries identified: <N>
|
||||
[ ] All libraries mapped: 0/<N>
|
||||
```
|
||||
|
||||
Note: The "All libraries mapped" item will be unchecked -- that is expected.
|
||||
Library mapping is the concern of Phase 3.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### CGO_ENABLED not set or build fails with "gcc not found"
|
||||
|
||||
The `go-sqlite3` driver requires C compilation. Make sure you have a C compiler
|
||||
installed and CGO is enabled:
|
||||
|
||||
```bash
|
||||
# macOS -- Xcode command line tools
|
||||
xcode-select --install
|
||||
|
||||
# Then build with explicit CGO
|
||||
CGO_ENABLED=1 go build -o go-analyzer .
|
||||
```
|
||||
|
||||
### "cannot find package" errors during Go build
|
||||
|
||||
The analyzer depends on `github.com/mattn/go-sqlite3`. Run:
|
||||
|
||||
```bash
|
||||
cd tools/go-analyzer
|
||||
go mod download
|
||||
go mod verify
|
||||
```
|
||||
|
||||
### Wrong source path
|
||||
|
||||
The `--source` flag must point to the root of the cloned nats-server repository
|
||||
(the directory that contains the `server/` subdirectory). If you see
|
||||
"discovering files: no such file or directory", verify:
|
||||
|
||||
```bash
|
||||
ls golang/nats-server/server/
|
||||
```
|
||||
|
||||
### Database locked errors
|
||||
|
||||
If you run the analyzer while another process has `porting.db` open, SQLite may
|
||||
report a lock error. Close any other connections (including `sqlite3` CLI
|
||||
sessions) and retry. The schema enables WAL mode to reduce lock contention:
|
||||
|
||||
```sql
|
||||
PRAGMA journal_mode=WAL;
|
||||
```
|
||||
|
||||
### Analyzer prints warnings but continues
|
||||
|
||||
Warnings like "Warning: skipping server/foo.go: <parse error>" mean an individual
|
||||
file could not be parsed. The analyzer continues with remaining files. Investigate
|
||||
any warnings -- they may indicate a Go version mismatch or syntax not yet
|
||||
supported by the `go/ast` parser at your Go version.
|
||||
|
||||
### Empty database after analyzer runs
|
||||
|
||||
If the analyzer prints zeros for all counts, verify that:
|
||||
1. The `--source` path is correct and contains Go files.
|
||||
2. The `--schema` path points to a valid `porting-schema.sql`.
|
||||
3. The `--db` path is writable.
|
||||
|
||||
You can inspect the database directly:
|
||||
|
||||
```bash
|
||||
sqlite3 porting.db "SELECT COUNT(*) FROM modules;"
|
||||
sqlite3 porting.db "SELECT COUNT(*) FROM features;"
|
||||
sqlite3 porting.db "SELECT COUNT(*) FROM unit_tests;"
|
||||
```
|
||||
Reference in New Issue
Block a user