Add batch tracking markdown with all 42 implementation batches
This commit is contained in:
Joseph Doherty
@@ -12,203 +12,126 @@
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Generate Batch 0 Manifest and Reconcile Mapping
|
||||
## MANDATORY VERIFICATION PROTOCOL
|
||||
|
||||
**Files:**
|
||||
- Modify: `porting.db` (`batch_tests`, `implementation_batches`)
|
||||
- Create: `/tmp/batch0-implementable-tests.csv`
|
||||
> **This section is NON-NEGOTIABLE. Every task MUST follow this protocol. Skipping any step is a plan violation.**
|
||||
|
||||
**Step 1: Generate manifest**
|
||||
### What Counts as a Real Test
|
||||
|
||||
Run:
|
||||
A test is **real** (eligible for `verified` status) ONLY if ALL of these are true:
|
||||
|
||||
1. **Has Arrange/Act/Assert** — the method body contains setup, an invocation of production code, and at least one Shouldly assertion on the result
|
||||
2. **Calls production code** — the test invokes at least one method from `ZB.MOM.NatsNet.Server.*` (not just test helpers or constants)
|
||||
3. **Has meaningful assertions** — uses `ShouldBe`, `ShouldContain`, `ShouldThrow`, `ShouldNotBeNull`, etc. on values derived from the Act step
|
||||
4. **Is NOT a stub** — does not contain `throw new NotImplementedException`, `Assert.True(true)`, `Assert.Pass()`, `// TODO`, or an empty body
|
||||
|
||||
### Stub Detection Check (REQUIRED after every class)
|
||||
|
||||
After porting all tests in a class, run this stub scan **before** running the tests:
|
||||
|
||||
```bash
|
||||
sqlite3 -header -csv porting.db "
|
||||
WITH implementable AS (
|
||||
SELECT t.id, t.dotnet_class, t.dotnet_method, t.go_file, t.go_method
|
||||
FROM unit_tests t
|
||||
WHERE t.status='deferred'
|
||||
AND EXISTS (
|
||||
SELECT 1 FROM dependencies d
|
||||
JOIN features f ON f.id=d.target_id AND d.target_type='feature'
|
||||
WHERE d.source_type='unit_test' AND d.source_id=t.id
|
||||
)
|
||||
AND NOT EXISTS (
|
||||
SELECT 1 FROM dependencies d
|
||||
JOIN features f ON f.id=d.target_id AND d.target_type='feature'
|
||||
WHERE d.source_type='unit_test' AND d.source_id=t.id
|
||||
AND f.status NOT IN ('verified','complete','n_a')
|
||||
)
|
||||
)
|
||||
SELECT * FROM implementable ORDER BY dotnet_class, id;
|
||||
" > /tmp/batch0-implementable-tests.csv
|
||||
# Scan for stub patterns — ANY match means the test is NOT real
|
||||
grep -n -E "(NotImplementedException|Assert\.True\(true\)|Assert\.Pass|\.ShouldBe\(true\);$|// TODO|// PLACEHOLDER)" \
|
||||
dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/<ClassName>.Impltests.cs
|
||||
```
|
||||
|
||||
Expected: CSV created with ~553 rows.
|
||||
**If any matches are found**: fix them or reclassify the test as deferred. Do NOT proceed to run tests until all stubs are eliminated.
|
||||
|
||||
**Step 2: Reconcile Batch 0 mapping**
|
||||
|
||||
Run:
|
||||
### Assertion Count Check (REQUIRED after every class)
|
||||
|
||||
```bash
|
||||
sqlite3 porting.db "
|
||||
INSERT INTO batch_tests (batch_id, test_id)
|
||||
SELECT 0, t.id
|
||||
FROM unit_tests t
|
||||
WHERE t.status='deferred'
|
||||
AND EXISTS (
|
||||
SELECT 1 FROM dependencies d
|
||||
JOIN features f ON f.id=d.target_id AND d.target_type='feature'
|
||||
WHERE d.source_type='unit_test' AND d.source_id=t.id
|
||||
)
|
||||
AND NOT EXISTS (
|
||||
SELECT 1 FROM dependencies d
|
||||
JOIN features f ON f.id=d.target_id AND d.target_type='feature'
|
||||
WHERE d.source_type='unit_test' AND d.source_id=t.id
|
||||
AND f.status NOT IN ('verified','complete','n_a')
|
||||
)
|
||||
AND NOT EXISTS (SELECT 1 FROM batch_tests bt WHERE bt.test_id=t.id);
|
||||
|
||||
UPDATE implementation_batches
|
||||
SET test_count=(SELECT COUNT(*) FROM batch_tests WHERE batch_id=0)
|
||||
WHERE id=0;
|
||||
"
|
||||
# Count Shouldly assertions per test file — must average >= 1.5 per test method
|
||||
grep -c "\.Should" dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/<ClassName>.Impltests.cs
|
||||
grep -c "\[Fact\]\|\[Theory\]" dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/<ClassName>.Impltests.cs
|
||||
```
|
||||
|
||||
Expected: `batch show 0` now reports Batch 0 tests > 0.
|
||||
**If assertion count / test count < 1.5**: review tests for shallow coverage. Tests with only one trivial assertion (e.g., `result.ShouldNotBeNull()`) on a complex method need strengthening.
|
||||
|
||||
**Step 3: Verify mapping**
|
||||
### Per-Test Verification Loop
|
||||
|
||||
Run:
|
||||
For EVERY test method ported (not per-class, not per-wave — PER TEST):
|
||||
|
||||
```bash
|
||||
/usr/local/share/dotnet/dotnet run --project tools/NatsNet.PortTracker -- batch show 0 --db porting.db
|
||||
```
|
||||
1. **Read the Go test** — open the Go source at the line from `test show <id>` and understand what behavior it verifies
|
||||
2. **Write the C# port** — translate the behavioral intent (not line-for-line)
|
||||
3. **Run the single test** and capture output:
|
||||
```bash
|
||||
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ \
|
||||
--filter "FullyQualifiedName~ZB.MOM.NatsNet.Server.Tests.ImplBacklog.<ClassName>.<MethodName>" \
|
||||
--verbosity normal 2>&1 | tail -20
|
||||
```
|
||||
4. **Verify the output** shows `Passed: 1, Failed: 0` — if it shows `Passed: 0` the test was not discovered (name mismatch)
|
||||
5. **Only after green**: add the test ID to the verified-candidates list for this class
|
||||
|
||||
Expected: Batch 0 includes mapped tests.
|
||||
### Class-Level Gate (REQUIRED before any status updates)
|
||||
|
||||
**Step 4: Commit**
|
||||
After all tests in a class are individually verified:
|
||||
|
||||
```bash
|
||||
git add porting.db
|
||||
git commit -m "chore(porttracker): reconcile batch 0 implementable test mappings"
|
||||
```
|
||||
1. **Run the full class**:
|
||||
```bash
|
||||
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ \
|
||||
--filter "FullyQualifiedName~ZB.MOM.NatsNet.Server.Tests.ImplBacklog.<ClassName>" \
|
||||
--verbosity normal 2>&1 | tail -5
|
||||
```
|
||||
2. **Parse the summary line** — extract `Passed: N, Failed: M, Skipped: S`
|
||||
3. **Verify**: `Failed` MUST be `0`, and `Passed` MUST equal the number of tests you ported
|
||||
4. **If mismatch**: investigate before proceeding — do not mark anything verified
|
||||
|
||||
### Status Update Protocol
|
||||
|
||||
- **NEVER use `test batch-update` with more than 15 IDs at once** — process in chunks of max 15
|
||||
- **NEVER mark a test verified unless you have captured and reviewed its individual pass output**
|
||||
- **For each batch-update command**, list the specific test IDs and the evidence (class run output showing pass count)
|
||||
- **Deferred tests MUST have a reason** — when keeping a test deferred, update its notes with why (e.g., "requires running NATS server", "needs cluster topology")
|
||||
|
||||
### Checkpoint Protocol (REQUIRED between tasks)
|
||||
|
||||
After completing each task (Tasks 2-9), before starting the next:
|
||||
|
||||
1. Run the full test suite to check for regressions:
|
||||
```bash
|
||||
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --verbosity normal 2>&1 | tail -10
|
||||
```
|
||||
2. Report the total pass/fail/skip counts
|
||||
3. If any pre-existing tests broke, fix them before continuing
|
||||
4. Commit the work for this task before starting the next
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Apply Porting Template to One Seed Test Per Class
|
||||
### Task 1: Reconcile Batch 0 Mapping (**DONE** — commit ee0827d)
|
||||
|
||||
**Files:**
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/*.Impltests.cs` (one method per class)
|
||||
- Read: `golang/nats-server/server/*_test.go`
|
||||
|
||||
**Step 1: Replace one placeholder method body with behavior-faithful assertions**
|
||||
|
||||
Template:
|
||||
|
||||
```csharp
|
||||
[Fact] // T:<id>
|
||||
public void <DotnetMethod>()
|
||||
{
|
||||
// Arrange (from Go test setup)
|
||||
var subject = /* translated setup */;
|
||||
|
||||
// Act
|
||||
var result = /* invocation under test */;
|
||||
|
||||
// Assert (exact behavioral intent from Go)
|
||||
result.ShouldBe(/* expected */);
|
||||
}
|
||||
```
|
||||
|
||||
**Step 2: Run method-level filter**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
/usr/local/share/dotnet/dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ \
|
||||
--filter "FullyQualifiedName~ZB.MOM.NatsNet.Server.Tests.ImplBacklog.<ClassName>.<DotnetMethod>"
|
||||
```
|
||||
|
||||
Expected: fail first if placeholder previously masked behavior; pass after fixing setup/assertions.
|
||||
|
||||
**Step 3: Repeat for all 28 classes as seed tests**
|
||||
|
||||
Expected: each class has at least one real, non-placeholder port.
|
||||
|
||||
**Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog
|
||||
git commit -m "test(batch0): replace seed placeholders with behavior ports across classes"
|
||||
```
|
||||
Batch 0 now has 553 tests mapped. This task is complete.
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Port Account/Auth/Options Cluster (Core Low-Infra)
|
||||
### Task 2: Port Account/Auth/Options Cluster (89 tests)
|
||||
|
||||
**Max scope: 7 classes, ~89 tests. Work ONE CLASS at a time.**
|
||||
|
||||
**Files:**
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/AccountTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/AuthHandlerTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/AuthCalloutTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JwtProcessorTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ServerOptionsTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConfigReloaderTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/CertificateStoreWindowsTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/AccountTests.Impltests.cs` (18 tests)
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/AuthHandlerTests.Impltests.cs` (3 tests)
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/AuthCalloutTests.Impltests.cs` (17 tests)
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JwtProcessorTests.Impltests.cs` (28 tests)
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ServerOptionsTests.Impltests.cs` (3 tests)
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConfigReloaderTests.Impltests.cs` (19 tests)
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/CertificateStoreWindowsTests.Impltests.cs` (1 test)
|
||||
|
||||
**Step 1: Port methods for these classes from Go tests**
|
||||
**For EACH class, follow this sequence:**
|
||||
|
||||
Reference Go files:
|
||||
|
||||
- `golang/nats-server/server/accounts_test.go`
|
||||
- `golang/nats-server/server/auth_test.go`
|
||||
- `golang/nats-server/server/auth_callout_test.go`
|
||||
- `golang/nats-server/server/jwt_test.go`
|
||||
- `golang/nats-server/server/opts_test.go`
|
||||
- `golang/nats-server/server/reload_test.go`
|
||||
|
||||
**Step 2: Tag runtime-blocked cases**
|
||||
|
||||
If Go method requires live server/cluster/process orchestration, keep deferred and add explicit reason note later.
|
||||
|
||||
**Step 3: Run per-class tests**
|
||||
|
||||
Run one class at a time:
|
||||
|
||||
```bash
|
||||
/usr/local/share/dotnet/dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ \
|
||||
--filter "FullyQualifiedName~ZB.MOM.NatsNet.Server.Tests.ImplBacklog.AccountTests"
|
||||
```
|
||||
|
||||
Expected: class green before status updates.
|
||||
|
||||
**Step 4: Update statuses for proven IDs**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
IDS=$(sqlite3 porting.db "
|
||||
WITH implementable AS (
|
||||
SELECT t.id,t.dotnet_class
|
||||
FROM unit_tests t
|
||||
WHERE t.status='deferred'
|
||||
AND EXISTS (SELECT 1 FROM dependencies d JOIN features f ON f.id=d.target_id AND d.target_type='feature'
|
||||
WHERE d.source_type='unit_test' AND d.source_id=t.id)
|
||||
AND NOT EXISTS (SELECT 1 FROM dependencies d JOIN features f ON f.id=d.target_id AND d.target_type='feature'
|
||||
WHERE d.source_type='unit_test' AND d.source_id=t.id
|
||||
AND f.status NOT IN ('verified','complete','n_a'))
|
||||
)
|
||||
SELECT group_concat(id, ',') FROM implementable
|
||||
WHERE dotnet_class IN ('AccountTests','AuthHandlerTests','AuthCalloutTests','JwtProcessorTests','ServerOptionsTests','ConfigReloaderTests','CertificateStoreWindowsTests');
|
||||
")
|
||||
/usr/local/share/dotnet/dotnet run --project tools/NatsNet.PortTracker -- \
|
||||
test batch-update --ids "$IDS" --set-status verified --db porting.db --execute
|
||||
```
|
||||
|
||||
Expected: IDs verified only after passing evidence.
|
||||
|
||||
**Step 5: Commit**
|
||||
1. Read all Go test sources for the class (use `test show <id>` to get file+line for each test)
|
||||
2. Port each test method following the **Per-Test Verification Loop** above
|
||||
3. Run **Stub Detection Check** on the class file
|
||||
4. Run **Assertion Count Check** on the class file
|
||||
5. Run **Class-Level Gate** and capture output
|
||||
6. Record verified IDs (only tests with individual + class-level green evidence)
|
||||
7. Update status in chunks of max 15:
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- \
|
||||
test batch-update --ids "<max 15 comma-separated IDs>" --set-status verified --db porting.db --execute
|
||||
```
|
||||
8. For any test that requires live server/cluster infra, keep deferred and note the reason
|
||||
|
||||
**After ALL 7 classes**: run **Checkpoint Protocol**, then commit:
|
||||
```bash
|
||||
git add dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog porting.db
|
||||
git commit -m "test(batch0): port account/auth/options implementable tests"
|
||||
@@ -216,51 +139,41 @@ git commit -m "test(batch0): port account/auth/options implementable tests"
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Port Routing and Edge Transport Cluster
|
||||
### Task 3: Port Routing and Edge Transport Cluster (111 tests)
|
||||
|
||||
**Max scope: 4 classes, ~111 tests. Work ONE CLASS at a time.**
|
||||
|
||||
**Files:**
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/RouteHandlerTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/GatewayHandlerTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/LeafNodeHandlerTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/WebSocketHandlerTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/RouteHandlerTests.Impltests.cs` (25 tests)
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/GatewayHandlerTests.Impltests.cs` (21 tests)
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/LeafNodeHandlerTests.Impltests.cs` (47 tests)
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/WebSocketHandlerTests.Impltests.cs` (18 tests)
|
||||
|
||||
**Step 1: Port deterministic tests from Go route/gateway/leaf/websocket files**
|
||||
|
||||
**Step 2: Keep runtime-only topology tests deferred with notes**
|
||||
|
||||
**Step 3: Run class filters**
|
||||
**For EACH class**: follow the same sequence as Task 2 (per-test loop, stub check, assertion check, class gate, chunked status updates).
|
||||
|
||||
**After ALL 4 classes**: run **Checkpoint Protocol**, then commit:
|
||||
```bash
|
||||
/usr/local/share/dotnet/dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ \
|
||||
--filter "FullyQualifiedName~ZB.MOM.NatsNet.Server.Tests.ImplBacklog.RouteHandlerTests"
|
||||
git add dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog porting.db
|
||||
git commit -m "test(batch0): port routing/gateway/leaf/websocket implementable tests"
|
||||
```
|
||||
|
||||
Expected: pass per class.
|
||||
|
||||
**Step 4: Update statuses for passing IDs and commit**
|
||||
|
||||
Use `test batch-update --ids "$IDS" --set-status verified --execute` for this class set, then commit.
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Port Server Introspection and Events Cluster
|
||||
### Task 4: Port Server Introspection and Events Cluster (128 tests)
|
||||
|
||||
**Max scope: 4 classes, ~128 tests. Work ONE CLASS at a time.**
|
||||
|
||||
**Files:**
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/EventsHandlerTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/MonitoringHandlerTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/MessageTracerTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/NatsServerTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/EventsHandlerTests.Impltests.cs` (21 tests)
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/MonitoringHandlerTests.Impltests.cs` (76 tests)
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/MessageTracerTests.Impltests.cs` (20 tests)
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/NatsServerTests.Impltests.cs` (11 tests)
|
||||
|
||||
**Step 1: Replace placeholder assertions with behavior-based assertions from Go tests**
|
||||
**For EACH class**: follow the same sequence as Task 2.
|
||||
|
||||
**Step 2: Validate log/JSON/monitor payload contracts with strict Shouldly checks**
|
||||
|
||||
**Step 3: Run class filters and update statuses**
|
||||
|
||||
Expected: each class green before DB status updates.
|
||||
|
||||
**Step 4: Commit**
|
||||
**Special note for MonitoringHandlerTests (76 tests)**: This is the second-largest class. Process in sub-batches of ~20 tests. After every 20 tests, run the class-level gate to confirm cumulative pass count is growing and no regressions.
|
||||
|
||||
**After ALL 4 classes**: run **Checkpoint Protocol**, then commit:
|
||||
```bash
|
||||
git add dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog porting.db
|
||||
git commit -m "test(batch0): port monitoring/events/server implementable tests"
|
||||
@@ -268,117 +181,124 @@ git commit -m "test(batch0): port monitoring/events/server implementable tests"
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Port Concurrency and Process-Control Cluster
|
||||
### Task 5: Port Concurrency Cluster (20 tests)
|
||||
|
||||
**Max scope: 2 classes, ~20 tests. Work ONE CLASS at a time.**
|
||||
|
||||
**Files:**
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConcurrencyTests1.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConcurrencyTests2.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConcurrencyTests1.Impltests.cs` (18 tests)
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConcurrencyTests2.Impltests.cs` (2 tests)
|
||||
|
||||
**Step 1: Port race-safe deterministic cases only**
|
||||
|
||||
Use timeout-safe async assertions and avoid nondeterministic sleep-based checks.
|
||||
|
||||
**Step 2: Run each class repeatedly**
|
||||
|
||||
Run:
|
||||
**For EACH class**: follow the same sequence as Task 2, with this addition:
|
||||
|
||||
**Stability check** — run each class 3 times to detect flaky tests:
|
||||
```bash
|
||||
for i in 1 2 3; do
|
||||
/usr/local/share/dotnet/dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ \
|
||||
--filter "FullyQualifiedName~ZB.MOM.NatsNet.Server.Tests.ImplBacklog.ConcurrencyTests1";
|
||||
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ \
|
||||
--filter "FullyQualifiedName~ZB.MOM.NatsNet.Server.Tests.ImplBacklog.<ClassName>" \
|
||||
--verbosity normal 2>&1 | tail -5
|
||||
done
|
||||
```
|
||||
Only mark verified if all 3 runs pass with the same count.
|
||||
|
||||
Expected: stable pass across repeated runs.
|
||||
|
||||
**Step 3: Update statuses + commit**
|
||||
|
||||
Mark passing IDs verified; keep flaky/runtime-bound cases deferred with notes.
|
||||
**After BOTH classes**: run **Checkpoint Protocol**, then commit:
|
||||
```bash
|
||||
git add dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog porting.db
|
||||
git commit -m "test(batch0): port concurrency implementable tests"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 7: Port JetStream Small/Deterministic Cluster
|
||||
### Task 6: Port JetStream Small/Deterministic Cluster (18 tests)
|
||||
|
||||
**Max scope: 6 classes, ~18 tests. Work ONE CLASS at a time.**
|
||||
|
||||
**Files:**
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamVersioningTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamBatchingTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamJwtTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamTpmTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamLeafNodeTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/StorageEngineTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamVersioningTests.Impltests.cs` (2 tests)
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamBatchingTests.Impltests.cs` (1 test)
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamJwtTests.Impltests.cs` (4 tests)
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamTpmTests.Impltests.cs` (5 tests)
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamLeafNodeTests.Impltests.cs` (6 tests)
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/StorageEngineTests.Impltests.cs` (5 tests — note: 5 already verified in prior work)
|
||||
|
||||
**Step 1: Port only no-cluster/no-live-server methods**
|
||||
**For EACH class**: follow the same sequence as Task 2.
|
||||
|
||||
**Step 2: For cluster-required tests, keep deferred and annotate notes**
|
||||
|
||||
Example note: `deferred: requires running JetStream cluster/runtime`.
|
||||
|
||||
**Step 3: Run class filters; update statuses; commit**
|
||||
|
||||
Expected: deterministic classes pass; blocked cases stay deferred with reasons.
|
||||
**After ALL 6 classes**: run **Checkpoint Protocol**, then commit:
|
||||
```bash
|
||||
git add dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog porting.db
|
||||
git commit -m "test(batch0): port jetstream small/deterministic implementable tests"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 8: Port JetStream Consumer/Engine Heavy Cluster
|
||||
### Task 7: Port JetStream Consumer/Engine Heavy Cluster (125 tests)
|
||||
|
||||
**Max scope: 3 classes, ~125 tests. Work ONE CLASS at a time.**
|
||||
|
||||
**Files:**
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/NatsConsumerTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamEngineTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests2.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/NatsConsumerTests.Impltests.cs` (35 tests)
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamEngineTests.Impltests.cs` (89 tests)
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests2.Impltests.cs` (1 test)
|
||||
|
||||
**Step 1: Work test-by-test with strict triage**
|
||||
**For EACH class**: follow the same sequence as Task 2.
|
||||
|
||||
For each method:
|
||||
**Special note for JetStreamEngineTests (89 tests)**: This is the largest class. Process in sub-batches of ~15 tests. After every 15 tests:
|
||||
- Run the stub detection check on the whole file
|
||||
- Run the class-level gate and confirm cumulative pass count
|
||||
- If pass count plateaus (same count after adding 15 tests), STOP — tests are likely stubs
|
||||
|
||||
1. Port and run method-level filter.
|
||||
2. If runtime-bound, revert method to deferred state marker and keep status deferred.
|
||||
3. If unit-portable, finish port and keep in verified candidate list.
|
||||
|
||||
**Step 2: Run full class filters**
|
||||
|
||||
Expected: passing set is stable.
|
||||
|
||||
**Step 3: Update statuses and commit**
|
||||
|
||||
Use class-specific ID lists from manifest to avoid accidental status drift.
|
||||
**After ALL 3 classes**: run **Checkpoint Protocol**, then commit:
|
||||
```bash
|
||||
git add dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog porting.db
|
||||
git commit -m "test(batch0): port jetstream consumer/engine implementable tests"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 9: Port MQTT Cluster
|
||||
### Task 8: Port MQTT Cluster (57 tests)
|
||||
|
||||
**Max scope: 2 classes, ~57 tests. Work ONE CLASS at a time.**
|
||||
|
||||
**Files:**
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/MqttHandlerTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/MqttExternalTests.Impltests.cs`
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/MqttHandlerTests.Impltests.cs` (56 tests)
|
||||
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/MqttExternalTests.Impltests.cs` (1 test)
|
||||
|
||||
**Step 1: Port unit-level parser/options/utility tests**
|
||||
**For EACH class**: follow the same sequence as Task 2.
|
||||
|
||||
**Step 2: Defer broker-runtime tests with explicit notes**
|
||||
**Special note for MqttHandlerTests (56 tests)**: Process in sub-batches of ~15 tests with stub detection after each sub-batch.
|
||||
|
||||
**Step 3: Run class filters; update statuses; commit**
|
||||
|
||||
Expected: no silent skips; each deferred item has rationale.
|
||||
**After BOTH classes**: run **Checkpoint Protocol**, then commit:
|
||||
```bash
|
||||
git add dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog porting.db
|
||||
git commit -m "test(batch0): port mqtt implementable tests"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 10: Final Batch 0 Verification and Status Closure
|
||||
### Task 9: Final Batch 0 Verification and Status Closure
|
||||
|
||||
**Files:**
|
||||
- Modify: `porting.db`
|
||||
- Optional update: `reports/current.md` (via report generation script)
|
||||
- Generate: `reports/current.md` (via report generation script)
|
||||
|
||||
**Step 1: Run targeted regression sweep for all touched Batch 0 classes**
|
||||
|
||||
Run:
|
||||
**Step 1: Full regression sweep**
|
||||
|
||||
```bash
|
||||
/usr/local/share/dotnet/dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/
|
||||
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --verbosity normal 2>&1 | tail -10
|
||||
```
|
||||
|
||||
Expected: all touched classes pass.
|
||||
Capture and report total pass/fail/skip. **Failed MUST be 0.**
|
||||
|
||||
**Step 2: Validate remaining implementable deferred count**
|
||||
**Step 2: Stub audit across ALL ImplBacklog files**
|
||||
|
||||
Run:
|
||||
```bash
|
||||
grep -rn -E "(NotImplementedException|Assert\.True\(true\)|Assert\.Pass|// TODO|// PLACEHOLDER)" \
|
||||
dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/
|
||||
```
|
||||
|
||||
**If any matches remain**: fix them or reclassify the affected tests as deferred. Do NOT close the batch with stubs present.
|
||||
|
||||
**Step 3: Validate remaining implementable deferred count**
|
||||
|
||||
```bash
|
||||
sqlite3 porting.db "
|
||||
@@ -402,24 +322,21 @@ SELECT COUNT(*) FROM implementable;
|
||||
"
|
||||
```
|
||||
|
||||
Expected: `0` (or only explicitly documented runtime-blocked exceptions).
|
||||
Expected: `0` (or only explicitly documented runtime-blocked exceptions with notes).
|
||||
|
||||
**Step 3: Verify Batch 0 visibility**
|
||||
|
||||
Run:
|
||||
**Step 4: Verify Batch 0 visibility**
|
||||
|
||||
```bash
|
||||
/usr/local/share/dotnet/dotnet run --project tools/NatsNet.PortTracker -- batch show 0 --db porting.db
|
||||
/usr/local/share/dotnet/dotnet run --project tools/NatsNet.PortTracker -- report summary --db porting.db
|
||||
dotnet run --project tools/NatsNet.PortTracker -- batch show 0 --db porting.db
|
||||
dotnet run --project tools/NatsNet.PortTracker -- report summary --db porting.db
|
||||
```
|
||||
|
||||
Expected: Batch 0 test count reduced/closed and summary reflects updates.
|
||||
|
||||
**Step 4: Final commit**
|
||||
**Step 5: Generate report and final commit**
|
||||
|
||||
```bash
|
||||
git add dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog porting.db reports/current.md
|
||||
git commit -m "test(batch0): port implementable deferred tests and close batch 0"
|
||||
./reports/generate-report.sh
|
||||
git add dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog porting.db reports/
|
||||
git commit -m "test(batch0): complete batch 0 implementable tests verification"
|
||||
```
|
||||
|
||||
---
|
||||
@@ -435,10 +352,46 @@ git commit -m "test(batch0): port implementable deferred tests and close batch 0
|
||||
- `LeafNodeHandlerTests` (47)
|
||||
- `NatsConsumerTests` (35)
|
||||
|
||||
## TDD Guardrails
|
||||
## ANTI-STUB GUARDRAILS (NON-NEGOTIABLE)
|
||||
|
||||
- Never bulk-mark IDs `verified` before class-level green runs.
|
||||
- Preserve `T:<id>` comments to keep DB traceability.
|
||||
- If production behavior mismatches Go spec, stop and open feature-fix work rather than weakening assertions.
|
||||
- Keep tests deterministic: avoid timing races and process-global side effects where possible.
|
||||
These rules exist because previous attempts degraded into stub-writing after ~50 tests. **Every rule is mandatory.**
|
||||
|
||||
### Forbidden Patterns
|
||||
|
||||
The following patterns in a test body mean the test is a STUB, not a real port. **Tests containing these MUST NOT be marked verified:**
|
||||
|
||||
| Pattern | Why it's a stub |
|
||||
|---------|----------------|
|
||||
| `throw new NotImplementedException()` | Placeholder, tests nothing |
|
||||
| `Assert.True(true)` | Always passes, tests nothing |
|
||||
| `Assert.Pass()` | Always passes, tests nothing |
|
||||
| `result.ShouldBe(true);` (alone) | Likely a lazy replacement for real assertion |
|
||||
| `// TODO` or `// PLACEHOLDER` in body | Incomplete work marker |
|
||||
| Empty method body `{ }` | Tests nothing |
|
||||
| Only `ShouldNotBeNull()` on a constructor | Trivial, not behavioral |
|
||||
|
||||
### Verification Evidence Requirements
|
||||
|
||||
Before marking ANY test as `verified`, you MUST have:
|
||||
|
||||
1. **Individual test output** showing `Passed: 1` for that specific test method
|
||||
2. **Class-level output** showing total `Passed: N` matches the number of tests you ported in that class
|
||||
3. **Stub scan output** showing zero matches for the class file
|
||||
4. **Assertion count** showing >= 1.5 assertions per test on average
|
||||
|
||||
### Hard Limits
|
||||
|
||||
- **Max 15 test IDs per `batch-update` call** — forces you to think about what you're marking
|
||||
- **Max 1 class per status-update cycle** — no cross-class bulk updates
|
||||
- **Mandatory commit after each task** — prevents loss of work and creates review points
|
||||
- **If pass count plateaus after adding tests, STOP and investigate** — this means tests are stubs or not being discovered
|
||||
|
||||
### If You Get Stuck
|
||||
|
||||
If a test requires infrastructure you can't provide (running server, cluster, etc.):
|
||||
|
||||
1. Keep the test as `deferred` — do NOT write a stub that trivially passes
|
||||
2. Add a comment in the test: `// DEFERRED: requires <specific reason>`
|
||||
3. Move on to the next test
|
||||
4. This is the CORRECT behavior — marking deferred with a reason is better than a fake pass
|
||||
|
||||
|
||||
Reference in New Issue
Block a user