fix(historian): #683 review — a refused tag ADD INDEX only warns; document index trade-offs
- Bootstrapper: an ALTER ... ADD INDEX refused up front (e.g. no DDL rights) logs a Warning and lets the bootstrap succeed (the index is optional); a WAL-apply failure remains unobserved, like the TTL ALTER — documented. Integration test (VARCHAR tag => refused ALTER) fails with the catch removed. - Historian.md: a dominant tag can read ~20% slower on long windows (single- threaded index scan vs parallel JIT filter); O3 writes rewrite index files; sample_modified.tag / event.source deliberately not indexed, and why. - CLAUDE.md: the concurrent-HistoryUpdate-delete caveat on the range-first order.
This commit is contained in:
+17
-2
@@ -19,7 +19,9 @@ namespace ZB.MOM.WW.OtOpcUa.Driver.Historian.QuestDb;
|
||||
/// <item><b>Tag index (#683):</b> a sample table created before the <c>tag</c> symbol index existed gains it with
|
||||
/// <see cref="QuestDbTables.AddTagIndex"/> (idempotent; a boot that finds it indexed issues no ALTER, and like the TTL
|
||||
/// the ALTER is a WAL transaction the bootstrapper does not re-read). Building it is a one-off WAL apply — ~3 s for
|
||||
/// 68M rows / 63 daily partitions measured — during which writes queue and reads keep scanning.</item>
|
||||
/// 68M rows / 63 daily partitions measured — during which writes queue and reads keep scanning. A refused ALTER is
|
||||
/// logged as a Warning, never fails the bootstrap (the index is optional); a failure at WAL-apply time would suspend
|
||||
/// the table unseen, the same blind spot as the TTL ALTER.</item>
|
||||
/// <item><b>Failure:</b> not latched. Within <see cref="BootstrapFailureCache.DefaultDuration"/> the last failure is
|
||||
/// re-thrown without touching QuestDB.</item>
|
||||
/// <item><c>AutoCreateSchema=false</c> only verifies the tables exist (no TTL changes) and throws
|
||||
@@ -145,8 +147,21 @@ internal sealed class QuestDbSchemaBootstrapper
|
||||
return;
|
||||
}
|
||||
// A WAL transaction: table_columns() keeps reporting false until it applies, so do NOT re-read to verify.
|
||||
await using (var alter = _conn.CreateCommand(conn, QuestDbTables.AddTagIndex(table)))
|
||||
// The index is optional (reads are correct without it), so a refused ALTER (e.g. no DDL rights) must not fail the
|
||||
// bootstrap: it is logged and retried on the next boot. A failure at WAL-APPLY time is not seen here (QuestDB would
|
||||
// suspend the table) — the same blind spot as the TTL ALTER.
|
||||
try
|
||||
{
|
||||
await using var alter = _conn.CreateCommand(conn, QuestDbTables.AddTagIndex(table));
|
||||
await alter.ExecuteNonQueryAsync(ct).ConfigureAwait(false);
|
||||
}
|
||||
catch (Exception ex) when (ex is not OperationCanceledException || !ct.IsCancellationRequested)
|
||||
{
|
||||
_log.LogWarning("Could not add the tag index to QuestDB table {Table} ({Exception}: {Message}); data-range lookups " +
|
||||
"of processed reads scan partitions instead of seeking (#683). Run: {Statement}",
|
||||
table, ex.GetType().Name, ex.Message, QuestDbTables.AddTagIndex(table));
|
||||
return;
|
||||
}
|
||||
_log.LogInformation("Added the tag index to QuestDB table {Table} (#683).", table);
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user