natsdotnet/docs/plans/2026-02-23-websocket-design.md

# WebSocket Support Design

## Overview

Port WebSocket connection support from the Go NATS server (`golang/nats-server/server/websocket.go`, ~1,550 lines) to the .NET solution. Full feature parity: client/leaf/MQTT paths, HTTP upgrade handshake, custom frame parser with masking, permessage-deflate compression, browser compatibility, origin checking, cookie-based auth, and close frame handling.

## Approach

**Raw socket with manual HTTP upgrade** and **custom frame parser** — no ASP.NET Core WebSocket middleware, no `System.Net.WebSockets`. Direct port of Go's frame-level implementation for full control over masking negotiation, compression, fragmentation, and browser quirks.

**Self-contained WebSocket module** under `src/NATS.Server/WebSocket/` with a `WsConnection` Stream wrapper that integrates transparently with the existing `NatsClient` read/write loops.

## Go Reference

- `server/websocket.go` — Main implementation (1,550 lines)
- `server/websocket_test.go` — Tests (4,982 lines)
- `server/opts.go` lines 518-610 — `WebsocketOpts` struct
- `server/client.go` — Integration points (`c.ws` field, `wsRead`, `wsCollapsePtoNB`)

## File Structure

```
src/NATS.Server/WebSocket/
  WsConstants.cs       — Opcodes, frame limits, close codes, compression magic bytes
  WsReadInfo.cs        — Per-connection frame reader state machine
  WsFrameWriter.cs     — Frame construction, masking, compression, fragmentation
  WsUpgrade.cs         — HTTP upgrade handshake validation and 101 response
  WsConnection.cs      — Stream wrapper bridging WS frames <-> NatsClient read/write
  WsOriginChecker.cs   — Same-origin and allowed-origins validation
  WsCompression.cs     — permessage-deflate via DeflateStream

tests/NATS.Server.Tests/WebSocket/
  WsUpgradeTests.cs
  WsFrameTests.cs
  WsCompressionTests.cs
  WsOriginCheckerTests.cs
  WsIntegrationTests.cs
```

Modified existing files:
- `NatsOptions.cs` — Add `WebSocketOptions` class
- `NatsServer.cs` — Second accept loop for WebSocket port
- `NatsClient.cs` — `IsWebSocket` flag, `WsInfo` metadata property

## Constants (WsConstants.cs)

Direct port of Go constants:

| Constant | Value | Purpose |
|----------|-------|---------|
| `WsTextMessage` | 1 | Text frame opcode |
| `WsBinaryMessage` | 2 | Binary frame opcode |
| `WsCloseMessage` | 8 | Close frame opcode |
| `WsPingMessage` | 9 | Ping frame opcode |
| `WsPongMessage` | 10 | Pong frame opcode |
| `WsFinalBit` | 0x80 | FIN bit in byte 0 |
| `WsRsv1Bit` | 0x40 | RSV1 (compression) in byte 0 |
| `WsMaskBit` | 0x80 | Mask bit in byte 1 |
| `WsMaxFrameHeaderSize` | 14 | Max frame header bytes |
| `WsMaxControlPayloadSize` | 125 | Max control frame payload |
| `WsFrameSizeForBrowsers` | 4096 | Browser fragmentation limit |
| `WsCompressThreshold` | 64 | Min payload size to compress |
| Close codes | 1000-1015 | RFC 6455 Section 11.7 |
| Paths | `/`, `/leafnode`, `/mqtt` | Client type routing |

## HTTP Upgrade Handshake (WsUpgrade.cs)

### Input
Raw `Stream` (TCP or TLS) after socket accept.

### Validation (RFC 6455 Section 4.2.1)
1. Parse HTTP request line — must be `GET <path> HTTP/1.1`
2. Parse headers into dictionary
3. Host header required
4. `Upgrade` header must contain `"websocket"` (case-insensitive)
5. `Connection` header must contain `"Upgrade"` (case-insensitive)
6. `Sec-WebSocket-Version` must be `"13"`
7. `Sec-WebSocket-Key` must be present
8. Path routing: `/` -> Client, `/leafnode` -> Leaf, `/mqtt` -> Mqtt
9. Origin checking via `WsOriginChecker` if configured
10. Compression: parse `Sec-WebSocket-Extensions` for `permessage-deflate`
11. No-masking: check `Nats-No-Masking` header (for leaf nodes)
12. Browser detection: `User-Agent` contains `"Mozilla/"`, Safari for `nocompfrag`
13. Cookie extraction: map configured cookie names to values
14. X-Forwarded-For: extract client IP

### Response
```
HTTP/1.1 101 Switching Protocols\r\n
Upgrade: websocket\r\n
Connection: Upgrade\r\n
Sec-WebSocket-Accept: <base64(SHA1(key + "258EAFA5-E914-47DA-95CA-C5AB0DC85B11"))>\r\n
[Sec-WebSocket-Extensions: permessage-deflate; server_no_context_takeover; client_no_context_takeover\r\n]
[Nats-No-Masking: true\r\n]
[Custom headers\r\n]
\r\n
```

### Result Type
```csharp
public readonly record struct WsUpgradeResult(
    bool Success,
    bool Compress,
    bool Browser,
    bool NoCompFrag,
    bool MaskRead,
    bool MaskWrite,
    string? CookieJwt,
    string? CookieUsername,
    string? CookiePassword,
    string? CookieToken,
    string? ClientIp,
    WsClientKind Kind);

public enum WsClientKind { Client, Leaf, Mqtt }
```

### Error Handling
Return standard HTTP error responses (400, 403, etc.) with body text for invalid requests.

## Frame Reading (WsReadInfo.cs)

### State Machine
```csharp
public struct WsReadInfo
{
    public int Remaining;        // Bytes left in current frame payload
    public bool FrameStart;      // Reading new frame header
    public bool FirstFrame;      // First frame of fragmented message
    public bool FrameCompressed; // Message is compressed (RSV1)
    public bool ExpectMask;      // Client frames should be masked
    public byte MaskKeyPos;      // Position in 4-byte mask key
    public byte[] MaskKey;       // 4-byte XOR mask
    public List<byte[]>? CompressedBuffers;
    public int CompressedOffset;
}
```

### Flow
1. Parse frame header: FIN, RSV1, opcode, mask, length, mask key
2. Handle control frames in-band (PING -> PONG, CLOSE -> close response)
3. Unmask payload bytes (XOR with cycling 4-byte key, optimized for 8-byte chunks)
4. If compressed: collect payloads across frames, decompress on final frame
5. Return unframed NATS protocol bytes

### Decompression
- Append magic trailer `[0x00, 0x00, 0xff, 0xff]` before decompressing
- Use `DeflateStream` for decompression
- Validate decompressed size against `MaxPayload`

## Frame Writing (WsFrameWriter.cs)

### Per-Connection State
```csharp
public sealed class WsFrameWriter
{
    private readonly bool _compress;
    private readonly bool _maskWrite;
    private readonly bool _browser;
    private readonly bool _noCompFrag;
    private DeflateStream? _compressor;
}
```

### Flow
1. If compression enabled and payload > 64 bytes: compress via `DeflateStream`
2. If browser client and payload > 4096 bytes: fragment into chunks
3. Build frame headers: FIN | RSV1 | opcode | MASK | length | mask key
4. If masking: generate random 4-byte key, XOR payload
5. Control frame helpers: `EnqueuePing`, `EnqueuePong`, `EnqueueClose`

### Close Frame
- 2-byte status code (big-endian) + optional UTF-8 reason
- Reason truncated to 125 bytes with `"..."` suffix
- Status code mapping from `ClientClosedReason`:
  - ClientClosed -> 1000 (Normal)
  - Auth failure -> 1008 (Policy Violation)
  - Parse error -> 1002 (Protocol Error)
  - Payload too big -> 1009 (Message Too Big)
  - Other -> 1001 (Going Away) or 1011 (Server Error)

## WsConnection Stream Wrapper (WsConnection.cs)

Extends `Stream` to transparently wrap WebSocket framing around raw I/O:

- `ReadAsync`: Calls `WsRead` to decode frames, buffers decoded payloads, returns NATS bytes to caller
- `WriteAsync`: Wraps payload in WS frames via `WsFrameWriter`, writes to inner stream
- `EnqueueControlMessage`: Called by read loop for PING responses and close frames

NatsClient's `FillPipeAsync` and `RunWriteLoopAsync` work unchanged because `WsConnection` is a `Stream`.

## NatsServer Integration

### Second Accept Loop
```
StartAsync:
  if WebSocket.Port > 0:
    create _wsListener socket
    bind to WebSocket.Host:WebSocket.Port
    start RunWebSocketAcceptLoopAsync
```

### WebSocket Accept Flow
```
Accept socket
  -> TLS negotiation (reuse TlsConnectionWrapper)
  -> WsUpgrade.TryUpgradeAsync (HTTP upgrade)
  -> Create WsConnection wrapping stream
  -> Create NatsClient with WsConnection as stream
  -> Set IsWebSocket = true, attach WsUpgradeResult
  -> RunClientAsync (same as TCP from here)
```

### NatsClient Changes
- `public bool IsWebSocket { get; set; }` flag
- `public WsUpgradeResult? WsInfo { get; set; }` metadata
- No changes to read/write loops — `WsConnection` handles framing transparently

### Shutdown
- `ShutdownAsync` also closes `_wsListener`
- WebSocket clients receive close frames before TCP disconnect
- `LameDuckShutdownAsync` includes WS clients in stagger-close

## Configuration (WebSocketOptions)

```csharp
public sealed class WebSocketOptions
{
    public string Host { get; set; } = "0.0.0.0";
    public int Port { get; set; }                     // 0 = disabled
    public string? Advertise { get; set; }
    public string? NoAuthUser { get; set; }
    public string? JwtCookie { get; set; }
    public string? UsernameCookie { get; set; }
    public string? PasswordCookie { get; set; }
    public string? TokenCookie { get; set; }
    public string? Username { get; set; }
    public string? Password { get; set; }
    public string? Token { get; set; }
    public TimeSpan AuthTimeout { get; set; } = TimeSpan.FromSeconds(2);
    public bool NoTls { get; set; }
    public string? TlsCert { get; set; }
    public string? TlsKey { get; set; }
    public bool SameOrigin { get; set; }
    public List<string>? AllowedOrigins { get; set; }
    public bool Compression { get; set; }
    public TimeSpan HandshakeTimeout { get; set; } = TimeSpan.FromSeconds(2);
    public TimeSpan? PingInterval { get; set; }
    public Dictionary<string, string>? Headers { get; set; }
}
```

### Validation Rules
- Port 0 = disabled, skip remaining validation
- TLS required unless `NoTls = true`
- AllowedOrigins must be valid URLs
- Custom headers must not use reserved names
- NoAuthUser must match existing user if specified

## Origin Checking (WsOriginChecker.cs)

1. No Origin header -> accept (per RFC for non-browser clients)
2. If `SameOrigin`: compare origin host:port with request Host header
3. If `AllowedOrigins`: check host against list, match scheme and port

Parsed from config URLs, stored as `Dictionary<string, (string scheme, int port)>`.

## Testing

### Unit Tests

**WsUpgradeTests.cs**
- Valid upgrade -> 101
- Missing/invalid headers -> error codes
- Origin checking (same-origin, allowed, blocked)
- Compression negotiation
- No-masking header
- Browser/Safari detection
- Cookie extraction
- Path routing
- Handshake timeout
- Custom/reserved headers

**WsFrameTests.cs**
- Read uncompressed frames (various sizes)
- Length encoding (7-bit, 16-bit, 64-bit)
- Masking/unmasking round-trip
- Control frames (PING, PONG, CLOSE)
- Close frame status code and reason
- Invalid frames (missing FIN on control, oversized control)
- Fragmented messages
- Compressed frame round-trip
- Browser fragmentation at 4096
- Safari no-compressed-fragmentation

**WsCompressionTests.cs**
- Compress/decompress round-trip
- Below threshold not compressed
- Large payload compression
- MaxPayload limit on decompression

**WsOriginCheckerTests.cs**
- No origin -> accepted
- Same-origin match/mismatch
- Allowed origins match/mismatch
- Scheme and port matching

### Integration Tests

**WsIntegrationTests.cs**
- Connect via raw WebSocket, CONNECT/INFO exchange
- PUB/SUB over WebSocket
- Multiple WS clients
- Mixed TCP + WS clients interoperating
- WS with compression
- Graceful close (close frame exchange)
- Server shutdown sends close frames

## Post-Implementation

- Update `differences.md` to mark WebSocket support as implemented
- Update ports file to include WebSocket port