Files
AutopilotMonitor/src/Backend/AutopilotMonitor.Functions.Tests/SoftwareKeysJsonCodecTests.cs
T
Oliver Kieselbach 66fc23b609 feat(backend): SoftwareInventory contributions side-row + cascade delete helpers (PR2/7)
PR2 of the safe cascade-delete plan
(.claude/plans/session-deletion-cascade-safe.md). Adds the per-table
delete primitives the cascade worker will use in PR4 plus the §17
SessionInventoryContributions side-row that makes per-session inventory
counter increments at-most-once and exactly reversible. No production
caller invokes the cascade primitives yet; the inventory rewrite IS a
call-site change in VulnerabilityCorrelationService and ships in this PR.

== Cascade delete helpers (TableStorageService.Deletion.cs) ==
DeleteByExactKeysInBatchesAsync(tableName, keys) is the load-bearing
helper: groups by PartitionKey (Azure batch transactions are
partition-scoped), chunks at 100 actions per batch, falls back to
per-row DeleteEntityAsync with 404-ignore on whole-batch rollback.
Returns (Attempted, DeletedNow, AlreadyMissing) for accurate progress
accounting on idempotent re-runs of partial cascades. Plus 9 thin
wrappers per the §3 taxonomy: 7 keyed (EventTypeIndex, CveIndex,
SessionsByTerminal, SessionsByStage, DeadEndsByReason,
ClassifierVerdictsByIdLevel, SignalsByKind), 1 single-row
(DeviceSnapshot), and 2 enumerate-then-delete (Signals,
DecisionTransitions for the PK_BY_SESSION class). All fail-loud per
memory feedback_storage_helpers_fail_soft.

== SoftwareInventory side-row (TableStorageService.Inventory.cs) ==
Plan §17 design: each inventory-correlated session contributes ONE
SessionInventoryContributions row (PK=tenantId, RK=sessionId) with the
normalized software keys it added. Increments to SoftwareInventory
counters are now driven by deltas against this row, so:
  - WG sessions that fire software_inventory_analysis 4× no longer
    over-count by 4× per software per session
  - Cascade decrement is exact: the side-row is the source of truth
    for which counters this session contributed to
  - Pre-side-row sessions (created before this deploy) have no
    contributions row → cascade skips inventory entirely for them
    (drift accepted on the historical backlog, ages out via 90d
    retention).

Helpers (all fail-loud, all bounded retry per §12-Q10 = 10 attempts /
60s wall-clock cap, all ETag-CAS):
  - GetSessionInventoryContributionsAsync: returns prior key list or
    null on 404
  - UpsertSessionInventoryContributionsAsync: side-row CAS that
    computes (added, removed) delta against the new key set
  - DeleteSessionInventoryContributionsAsync: 404-ignore tombstone
  - IncrementSoftwareInventoryEntryAsync: ETag-CAS +1 with first-time
    AddEntity that writes full metadata
  - DecrementSoftwareInventoryEntryAsync: ETag-CAS -1, clamp >= 0,
    deletes the row when count hits zero

Codec: SoftwareKeysJsonCodec encodes the key list as raw JSON below
30 KB, gzip+Base64 above. Same shape DeletionManifestBuilder (PR1)
already reads at preflight, so the cascade decrement reads back what
the increment wrote with no schema gap. Plus deterministic ComputeDelta
(case-insensitive composite key) so concurrent correlations on the
same session collapse to a single counter update.

== VulnerabilityCorrelationService rewrite ==
The Task.Run fire-and-forget at the old line 246 now calls the new
ApplyInventoryContributionsAsync helper instead of
UpsertSoftwareInventoryAsync. Each Increment / Decrement is wrapped
in its OWN try/catch so one counter failure does NOT abort the rest
of the loop (Plan §17.3 + §17.8): without that, the side-row would
already promise the new key set while subsequent runs see an empty
delta and never retry the under-counted keys. Each individual failure
emits an `inventory_counter_write_failed` audit entry with operation
+ vendor + name + version + exception type/message via the existing
LogAuditEntryAsync. IMaintenanceRepository was added as an OPTIONAL
constructor param (positional default null) so existing 8-arg test
factories keep compiling unchanged.

ParseSoftwareInventoryItems exposed as internal static for testability;
preserves the same dedup + cpe-mapping behaviour as the old
UpsertSoftwareInventoryAsync path. The legacy method is left in place
on TableStorageService.Rules.cs for now (no remaining callers; PR7
cleanup can remove if desired).

== Tests (39 new) ==
- SoftwareKeysJsonCodecTests (12): raw vs gzip+Base64 paths,
  determinism, malformed-input rejection, delta cases (added /
  removed / identity / case-insensitive), composite-key shape.
- TableStorageDeletionBatchTests (8, SDK-mocked via internal test
  ctor): empty input, PK grouping (one batch per PK), chunking at
  100, batch 404 + all per-row 404, mixed batch 404 (2 succeed +
  3 missing) with accurate accounting, non-404 propagates, no
  over-deletion, ETag.All on every action.
- TableStorageInventoryCasTests (9): Increment first-create with
  full metadata, CAS +1 on existing row, 412 retry, 409 first-create
  race fall back to Update, empty-triple skip; Decrement 404 no-op,
  clamp-at-zero deletes the row, count>1 subtract, 0-count clamp,
  412 retry.
- VulnerabilityCorrelationServiceContributionsTests (10): Parser
  correctness, end-to-end add/remove flow with Mock.Verify counts,
  empty-input no-op, side-row exhaustion swallowed, per-counter
  failure does not abort + audits the failed key (both increment
  and decrement variants).

Backend suite: 1290/1290 green.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 19:59:28 +02:00

224 lines
8.2 KiB
C#

using System.Collections.Generic;
using System.Linq;
using AutopilotMonitor.Shared.Models.Deletion;
using Xunit;
namespace AutopilotMonitor.Functions.Tests;
/// <summary>
/// Pure-logic tests for the SoftwareKeysJson codec used by the inventory contributions side-row
/// (Plan §17). Encoding determinism + delta computation are the load-bearing properties — the
/// cascade decrement step reads back the side-row's keys and is byte-identical to what the
/// increment path wrote.
/// </summary>
public class SoftwareKeysJsonCodecTests
{
[Fact]
public void Encode_uses_raw_json_below_compression_threshold()
{
var keys = new List<DeletionDecrementKey>
{
new DeletionDecrementKey { Vendor = "Contoso", Name = "Widget", Version = "1.0" },
new DeletionDecrementKey { Vendor = "Fabrikam", Name = "Gadget", Version = "2.0" },
};
var encoded = SoftwareKeysJsonCodec.Encode(keys);
Assert.False(encoded.IsCompressed);
Assert.Equal(2, encoded.KeyCount);
Assert.Contains("Contoso", encoded.Encoded);
Assert.Contains("Fabrikam", encoded.Encoded);
}
[Fact]
public void Encode_uses_gzip_base64_above_compression_threshold()
{
// Generate >30 KB of raw JSON: 1000 distinct keys with ~30-char vendor each.
var keys = new List<DeletionDecrementKey>();
for (var i = 0; i < 1000; i++)
{
keys.Add(new DeletionDecrementKey
{
Vendor = $"VendorWithALongerNameForBulk-{i:D6}",
Name = $"Product-{i:D6}",
Version = $"1.{i}.0",
});
}
var encoded = SoftwareKeysJsonCodec.Encode(keys);
Assert.True(encoded.IsCompressed);
Assert.Equal(1000, encoded.KeyCount);
// Compressed payload should be much smaller than raw (heavy redundancy).
var rawSize = System.Text.Json.JsonSerializer.SerializeToUtf8Bytes(keys, DeletionManifestJson.SerializerOptions).Length;
Assert.True(encoded.Encoded.Length * 3 / 4 < rawSize / 3,
$"Compression should shrink payload by >3x. Got base64 {encoded.Encoded.Length} chars vs raw {rawSize} bytes.");
}
[Fact]
public void Encode_decode_round_trips_keys_in_deterministic_order()
{
var keys = new List<DeletionDecrementKey>
{
new DeletionDecrementKey { Vendor = "Tailspin", Name = "Service", Version = "3.0" },
new DeletionDecrementKey { Vendor = "Contoso", Name = "Widget", Version = "1.0" },
new DeletionDecrementKey { Vendor = "Fabrikam", Name = "Gadget", Version = "2.0" },
};
var encoded = SoftwareKeysJsonCodec.Encode(keys);
var decoded = SoftwareKeysJsonCodec.Decode(encoded.Encoded, encoded.IsCompressed);
Assert.Equal(3, decoded.Count);
// Builder sorts by (Vendor, Name, Version) so the side-row content is byte-stable
// across builds — critical for manifest-determinism (Plan §18.6).
Assert.Equal("Contoso", decoded[0].Vendor);
Assert.Equal("Fabrikam", decoded[1].Vendor);
Assert.Equal("Tailspin", decoded[2].Vendor);
}
[Fact]
public void Encode_with_compression_round_trips_through_gzip_base64()
{
// Force the compressed path with a key set that just barely exceeds the threshold.
var keys = new List<DeletionDecrementKey>();
for (var i = 0; i < 500; i++)
{
keys.Add(new DeletionDecrementKey
{
Vendor = "VendorWithALongerNameForBulk-" + i.ToString("D6"),
Name = "Product-" + i.ToString("D6"),
Version = "1." + i + ".0",
});
}
var encoded = SoftwareKeysJsonCodec.Encode(keys);
var decoded = SoftwareKeysJsonCodec.Decode(encoded.Encoded, encoded.IsCompressed);
Assert.True(encoded.IsCompressed);
Assert.Equal(500, decoded.Count);
Assert.Equal(keys[0].Vendor, decoded.First(k => k.Vendor == keys[0].Vendor).Vendor);
}
[Fact]
public void Decode_empty_string_returns_empty_list()
{
Assert.Empty(SoftwareKeysJsonCodec.Decode("", false));
Assert.Empty(SoftwareKeysJsonCodec.Decode("", true));
}
[Fact]
public void Decode_invalid_base64_throws_invalid_data()
{
var ex = Assert.Throws<System.IO.InvalidDataException>(
() => SoftwareKeysJsonCodec.Decode("!!! not base64 !!!", isCompressed: true));
Assert.Contains("Base64", ex.Message);
}
[Fact]
public void Decode_malformed_json_throws_invalid_data()
{
Assert.Throws<System.IO.InvalidDataException>(
() => SoftwareKeysJsonCodec.Decode("{not-an-array}", isCompressed: false));
}
[Fact]
public void ComputeDelta_returns_all_added_when_oldKeys_is_empty()
{
var newKeys = new[]
{
new DeletionDecrementKey { Vendor = "Contoso", Name = "Widget", Version = "1.0" },
new DeletionDecrementKey { Vendor = "Fabrikam", Name = "Gadget", Version = "2.0" },
};
var (added, removed) = SoftwareKeysJsonCodec.ComputeDelta(System.Array.Empty<DeletionDecrementKey>(), newKeys);
Assert.Equal(2, added.Count);
Assert.Empty(removed);
}
[Fact]
public void ComputeDelta_returns_one_added_when_newKeys_adds_a_phase2_app()
{
var oldKeys = new[]
{
new DeletionDecrementKey { Vendor = "Contoso", Name = "Widget", Version = "1.0" },
};
var newKeys = new[]
{
new DeletionDecrementKey { Vendor = "Contoso", Name = "Widget", Version = "1.0" },
new DeletionDecrementKey { Vendor = "Fabrikam", Name = "NewApp", Version = "2.0" },
};
var (added, removed) = SoftwareKeysJsonCodec.ComputeDelta(oldKeys, newKeys);
Assert.Single(added);
Assert.Contains(SoftwareKeysJsonCodec.CompositeKey("Fabrikam", "NewApp", "2.0"), added);
Assert.Empty(removed);
}
[Fact]
public void ComputeDelta_returns_one_removed_when_uninstall_happens_between_phases()
{
var oldKeys = new[]
{
new DeletionDecrementKey { Vendor = "Contoso", Name = "Widget", Version = "1.0" },
new DeletionDecrementKey { Vendor = "Fabrikam", Name = "OldApp", Version = "0.9" },
};
var newKeys = new[]
{
new DeletionDecrementKey { Vendor = "Contoso", Name = "Widget", Version = "1.0" },
};
var (added, removed) = SoftwareKeysJsonCodec.ComputeDelta(oldKeys, newKeys);
Assert.Empty(added);
Assert.Single(removed);
Assert.Contains(SoftwareKeysJsonCodec.CompositeKey("Fabrikam", "OldApp", "0.9"), removed);
}
[Fact]
public void ComputeDelta_is_empty_when_keys_are_identical()
{
var keys = new[]
{
new DeletionDecrementKey { Vendor = "Contoso", Name = "Widget", Version = "1.0" },
new DeletionDecrementKey { Vendor = "Fabrikam", Name = "Gadget", Version = "2.0" },
};
var (added, removed) = SoftwareKeysJsonCodec.ComputeDelta(keys, keys);
Assert.Empty(added);
Assert.Empty(removed);
}
[Fact]
public void ComputeDelta_is_case_insensitive()
{
// Upstream normalization is best-effort; "Contoso" and "contoso" must collapse so a case-
// flip between two correlation events on the same session doesn't trigger spurious deltas.
var oldKeys = new[]
{
new DeletionDecrementKey { Vendor = "Contoso", Name = "Widget", Version = "1.0" },
};
var newKeys = new[]
{
new DeletionDecrementKey { Vendor = "contoso", Name = "widget", Version = "1.0" },
};
var (added, removed) = SoftwareKeysJsonCodec.ComputeDelta(oldKeys, newKeys);
Assert.Empty(added);
Assert.Empty(removed);
}
[Fact]
public void CompositeKey_builds_three_part_separated_value()
{
var ck = SoftwareKeysJsonCodec.CompositeKey("Contoso", "Widget", "1.0");
Assert.NotEqual("ContosoWidget1.0", ck); // separator IS present
// Different orderings of the same string components produce different composites.
var ck2 = SoftwareKeysJsonCodec.CompositeKey("Widget", "Contoso", "1.0");
Assert.NotEqual(ck, ck2);
}
}