Until now, hardware-whitelist rejections were only visible to tenant admins
who actively navigated to the settings page or had a webhook configured.
Admins missed new rejected models if neither was the case.
This adds a parallel tenant-scoped notification pipeline (separate from the
existing GlobalAdmin-only feed): on the first-ever HardwareNotAllowed
distress per (tenantId, manufacturer-lower, model-lower), a persistent
bell notification is created with a deep link to the hardware-whitelist
page. Lifetime dedup via a dedicated tracker table makes it a one-shot per
model. The webhook path is unchanged and fires in parallel.
Routes are gated TenantAdminOrGA + TenantScoping.Jwt for now; an Operator
follow-up (new TenantAdminOrOperator policy) is tracked separately.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous design ran RuleEngine in Task.Run after HTTP 200 — Functions scale-in
could kill it mid-flight, leaving rule results un-persisted and forcing manual
"Analyze Now" clicks. New design enqueues onto analyze-on-enrollment-end and
runs the engine in a BackgroundService worker with visibility-timeout retry +
poison move after 5 attempts (mirrors telemetry-index-reconcile pattern).
Three trigger sites cut over: V1 ingest, V2 EventIngestProcessor primary, V2
post-vulnerability re-run. Manual "Analyze Now" remains as fallback.
Plus retry-semantics fix: RuleEngine.AnalyzeSessionAsync no longer swallows
storage exceptions in an outer try/catch (per-rule inner catch stays), and the
handler now throws when StoreRuleResultAsync returns false. Without this the
worker would have deleted messages on transient Table Storage failures and never
retried.
Tests: +4 failure-propagation, +6 envelope round-trip, 762 backend pass.
Follow-ups noted: TableStorageService helpers still swallow exceptions one
layer below; SignalRNotificationService needs interface for handler-level
unit tests; V1 ingest path missing vulnerability_report synth event.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Container App scale-to-zero clears the in-memory MSRC and KEV indexes on
every cold start, leaving a window where session correlation runs without
those data sources until the next refresh. Re-fetching ~1800 MSRC CVEs
plus the KEV catalog from upstream on every cold start is wasteful and
slow; persisting the snapshots to blob solves both problems.
New BlobBackedReferenceCache<T> generic helper handles single-blob JSON
snapshots in the autopilotmonitor / vulnerability-data container:
- msrc-index.json — full MSRC CVRF index (~350 KB for last 6 Patch Tuesdays)
- kev-catalog.json — full CISA KEV catalog (~500 KB)
Both services gained:
- EnsureHydratedAsync — idempotent, semaphore-guarded blob load
- Blob-write at the end of RefreshAsync (manual trigger + daily timer)
- Hydrate guard set after refresh so the no-op path is taken next time
VulnerabilityCacheWarmer (IHostedService) kicks off both EnsureHydratedAsync
calls fire-and-forget at app startup so re-deploys do not pay a blob-read
latency cost in their cold-start window. The correlation hot path also
calls EnsureHydratedAsync as a defensive backstop — idempotent and
semaphore-protected, so a request arriving mid-warmup awaits the same
in-flight load instead of duplicating it.
NVD's per-CPE cache stays in Table Storage — random-access lookup by CPE
key plus per-key 7d stale-refresh logic is the right shape for that
workload and would degrade if forced into a single-blob model.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
MsrcApiClient was injected into VulnerabilityCorrelationService for months
but never invoked — every report shipped with source="NVD" only. New
MsrcCorrelationService owns refresh + per-CVE lookup against an in-memory
index built from the last 6 Patch Tuesdays; daily VulnerabilityDataSync
populates it; CorrelateAsync enriches each finding with msrcDocumentId,
msrcReleaseDate, msrcSeverity, msrcBaseScore, msrcVector and flips source
to "NVD+MSRC" when the CVE was published in the indexed window.
Health check also surfaced that all worker-side traces of the daily sync
and CPE auto-resolve were sampled out of App Insights despite running.
Adds Event to host.json excludedTypes and emits TelemetryClient events
for VulnerabilityDataSyncCompleted, VulnerabilityCorrelationCompleted,
CpeAutoResolveHit/Miss, CpeMappingPersisted so customEvents queries can
prove freshness without depending on sampled traces.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The QueueTrigger binding required a Functions-host-specific app setting
(`<Connection>__queueServiceUri` with `__credential=managedidentity`) that
diverged from the project's storage-access pattern. Tables, blobs and the
index-reconcile producer all use a single `AzureStorageAccountName` (Managed
Identity) or `AzureTableStorageConnectionString` (full conn-string) and build
their clients with `DefaultAzureCredential` directly via DI — no host-side
binding glue.
In production the trigger silently failed to bind on every Function-App
worker start ("Error indexing method 'Functions.IndexReconcile'. Storage
account connection string 'AzureWebJobsAzureTableStorageConnectionString'
does not exist"), so messages enqueued by the producer would have piled up
unconsumed once `EnableIndexDualWrite` was flipped on.
Replaces:
- IndexReconcileFunction.cs (40 LoC, QueueTrigger) → DELETED
- New IndexReconcileQueueWorker.cs (BackgroundService) — same DI-based
config resolution as AzureQueueIndexReconcileProducer, full
QueueTrigger-platform parity (visibility-timeout retries, poison-queue
move after 5 failed attempts, malformed-JSON drop, idle 10s sleep).
Also pre-existing TableInitializerService is registered as IHostedService
in the same project, so the pattern is established.
Build: 0 warnings, 0 errors. Backend tests: 736/736 passed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Consumes the telemetry-index-reconcile queue (M5.d.2) and fans each
envelope out into 0–3 index-table rows via IIndexTableRepository
(M5.d.1). Closes the producer→consumer loop; once the
EnableIndexDualWrite flag is flipped, primary commits drive
eventual-consistent index upserts asynchronously.
Components:
- Microsoft.Azure.Functions.Worker.Extensions.Storage.Queues 5.5.4
package for the isolated-worker QueueTrigger binding.
- IndexReconcileHandler (plain class — not interface-abstracted per
Plan §M5.d design: DAL abstraction lives at IIndexTableRepository,
not here). Routing:
Signal → SignalsByKind (always, 1 row).
DecisionTransition:
IsTerminal=true → SessionsByTerminal
Taken=true → SessionsByStage
Taken=false + reason → DeadEndsByReason
ClassifierVerdictId!=null → ClassifierVerdictsByIdLevel
Up to 3 rows per DecisionTransition envelope (a Taken+terminal+
classifier transition fans to SessionsByTerminal + SessionsByStage
+ ClassifierVerdictsByIdLevel).
- IndexReconcileFunction thin QueueTrigger wrapper
(Connection = AzureTableStorageConnectionString). Malformed JSON
logged + dropped to avoid poison-queue retry loops; transient
repo errors propagate so the platform retries.
- DI: handler registered as Singleton in Program.cs.
Failure semantics:
- Required-field-missing envelopes (Signal without SessionSignalOrdinal
or SignalKind; DecisionTransition without StepIndex) → log warn,
drop. These won't become valid on retry — avoiding the retry loop.
- Transient Table Storage errors → rethrow. QueueTrigger platform
retries up to 5 times by default, then moves to
telemetry-index-reconcile-poison for manual investigation.
Deferred to M5.d.4:
- SessionsByStage stale-row pruning: when a session moves from
Stage-X→Stage-Y, the old Stage-X row stays. Either query-side
dedupe (per SessionId, max-Ticks RK) or reconcile-timer pruning
handles this. Deliberately not solved in the handler to avoid
a query+delete round-trip per Taken transition.
- End-to-end dual-write verification tests (Azurite harness) are
the M5.d.4 release gate before the flag is flipped.
Tests (+10 in Functions) via capture-only FakeIndexTableRepository:
- Signal happy path (1 SignalsByKind row).
- Signal missing SessionSignalOrdinal / SignalKind → 0 rows (x2).
- DecisionTransition Terminal+Taken+Classifier → 3 rows.
- DecisionTransition Taken non-terminal no-classifier → 1 row
(SessionsByStage only).
- DecisionTransition dead-end with classifier → 2 rows
(DeadEndsByReason + ClassifierVerdictsByIdLevel).
- DecisionTransition dead-end without reason → 0 rows (PK-collision
defense).
- DecisionTransition missing StepIndex → 0 rows (all paths gated).
- Unknown SourceKind → 0 rows.
- Null envelope → 0 rows.
Full Functions suite: 682 green (prev 672 + 10 new). Full solution
build green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the async fan-out producer path from Plan §2.8 / §M5.d so primary
Signals + DecisionTransitions rows can be projected onto their index
tables without blocking the hot ingest loop.
Components:
- Constants.QueueNames.TelemetryIndexReconcile (azure-queue-name-safe,
all lowercase + hyphens).
- IndexReconcileEnvelope (Shared DTO, flat shape with nullable fields
per SourceKind="Signal"/"DecisionTransition"). Avoids polymorphic
Newtonsoft deserialization — see M5.c.3 finding.
- IndexReconcileEnvelopeFactory pure static helper (FromSignal /
FromDecisionTransition / BuildBatch) — state-less projection.
- IIndexReconcileProducer (Shared interface) contract: no-op when
the feature flag is off; never rethrows underlying-queue failures.
- AzureQueueIndexReconcileProducer (Functions) — Base64-encoded JSON
messages, lazy CreateIfNotExistsAsync guarded by Interlocked flag,
Managed-Identity-or-connection-string construction mirroring
BlobStorageService / TableStorageService.
- IngestTelemetryFunction ctor gains IIndexReconcileProducer; after
primary Signals + DecisionTransitions commit it builds envelopes
and enqueues them. Event path is untouched (events don't feed the
index tables per §2.8).
- Program.cs registers the producer as a singleton.
Feature gate (AdminConfiguration.EnableIndexDualWrite, default false):
- New bool field in AdminConfiguration.
- TableConfigRepository Store + Map extension (per
feedback_table_storage_serialization).
- Producer checks config each call via AdminConfigurationService's
5-min memory cache. Flag off → primary behaviour is bit-exact.
Fault tolerance: queue-side exceptions (network, throttling, missing
queue before first CreateIfNotExists) are logged per envelope and
swallowed. The primary row is the source of truth; the 2h reconcile
timer (M5.d.4) is the safety-net recovery path.
Tests (+11 in Functions):
- 5 IndexReconcileEnvelopeFactoryTests: Signal/DecisionTransition
projection, dead-end preservation, batch ordering, null/empty.
- 3 IndexReconcileEnvelopeSerializationTests: Newtonsoft round-trip
(Signal, full DecisionTransition, minimal-envelope tolerance).
- 3 AdminConfigurationEnableIndexDualWriteTests: default-false on
new config + CreateDefault, persists when set.
Deferred to later sub-milestones:
- M5.d.3: queue-triggered consumer that maps envelopes to 0–3 index
rows using M5.d.1 TableIndexRepository.
- M5.d.4: 2h reconcile timer + release-gate dual-write verification
tests, then flip EnableIndexDualWrite=true in prod admin config.
Full Functions suite: 672 green (prev 661 + 11 new). Full solution
build green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Extract the post-parse event-processing tail of IngestEventsFunction into a
standalone EventIngestProcessor service so /api/agent/telemetry runs the same
downstream logic as /api/agent/ingest. Deliberate copy-duplication, not a
shared refactor:
- Legacy /api/agent/ingest is production-hot — it serves live traffic from
every deployed V1 agent and is NOT touched by this commit. Zero regression
risk on the existing path.
- EventIngestProcessor is a near-verbatim copy of the legacy pipeline's tail
(~1000 LoC). Duplication is accepted because legacy /api/agent/ingest is
scheduled for decommission post-v11 GA (see tasks/todo.md → Follow-Ups →
Legacy-Agent-Ausmusterung). Until then: bugs fixed in legacy must be ported
here manually.
Copied pipeline steps (all 1:1 with legacy ProcessIngestAsync lines 150-566):
StampServerFields → SanitizeEventTimestamps → StoreEventsBatchAsync →
Index updates fire-and-forget → IME agent version history → ClassifyEvents
→ App install summaries → Geo-location update → UpdateSessionStatusAsync
(all 7 status branches) → IncrementSessionEventCountAsync → Rule engine
analysis fire-and-forget + SignalR + stats → Vulnerability correlation with
WhiteGlove Part-2 merge → IncrementPlatformStatAsync TotalEvents +
SuccessfulEnrollments → RecordGatherRuleStatsAsync → Diagnostics blob
update → GetSession for updatedSession → Session age 4h warning → WhiteGlove
IsPreProvisioned error log → SendWebhookNotificationsAsync (all 3 alert
paths) → SLA breach evaluation → Admin override detection → ServerAction
fetch-and-clear with TrackEvent telemetry → BuildSignalRMessages.
Shared (not duplicated, to avoid drift in pure helpers):
IngestEventsFunction.StampServerFields, SanitizeEventTimestamps,
RecalculateAppDurations (all internal static, state-less, stable). The
internal DTOs EventClassification + AppInstallAggregationState are also
reused — they carry no behaviour, only fields.
IngestTelemetryFunction wiring:
- Return type HttpResponseData → IngestEventsOutput so the V2 endpoint can
emit SignalR messages for the real-time UI push (matches legacy binding).
- PersistItemsAsync partitions the batch: Events go through the processor
(full pipeline + AdminAction + pending ServerActions + SignalR); Signal +
Transition batches skip the processor and read control signals directly
via a lean ReadControlSignalsAsync helper on ISessionRepository — no point
firing the full pipeline for a batch that carries no events.
- EventIngestProcessor registered in Program.cs DI.
Known gap (explicitly NOT in this commit): the V2 endpoint lacks a payload
size guard (legacy reads MaxNdjsonPayloadSizeMB from tenant config in
ParseNdjsonRequest). Different payload shape (JSON array vs NDJSON), deferred
to a dedicated follow-up.
All 596 backend tests green (unchanged count — copy path has no new tests
yet; the V2 endpoint's behaviour is validated by the legacy test coverage
through the shared-helper surface + the TelemetryPayloadParser tests from
M5.b.1). Integration coverage lands with M7 test-VM smoke.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Introduces a clean abstraction layer to decouple domain services from Azure Table Storage,
enabling future migration to Cosmos DB and Event Streaming without touching business logic.
Architecture:
- 10 repository interfaces in Shared/DataAccess/ (ISessionRepository, IRuleRepository, etc.)
- IDataEventPublisher interface with NullDataEventPublisher for future event streaming
- IStorageInitializer for provider-agnostic initialization
- 5 Table Storage implementations that delegate to existing TableStorageService
- DI extension method AddTableStorageDataAccess() with AddEventStreaming<T>() hook
- Migration plan in tasks/todo.md for incremental adoption
https://claude.ai/code/session_01NHWJfPaa8TQHuPzMD7gG63
Adds a second device validation method alongside existing Autopilot Device
Validation. Corporate Identifier Validation checks devices against Intune
Corporate Device Identifiers (manufacturer + model + serial number) via the
Graph beta searchExistingIdentities endpoint.
Backend: Extract GraphTokenService from AutopilotDeviceValidator for shared
token/consent logic. Add CorporateIdentifierValidator with caching. Update
SecurityValidator with cascading validation (Autopilot first, Corporate
Identifier fallback). Wire new validator through all agent endpoints.
Frontend: Add second toggle in settings gated behind preview mode. Shared
consent flow skips redirect when consent already exists from other toggle.
Section renamed to "Enrollment Device Validation" in preview mode.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Galactic Admin Mode for Audit Logs: banner + cross-tenant endpoint (galactic/audit/logs)
- Audit page pagination (12 entries per page) with Previous/Next controls
- Platform Metrics: moved computation to backend PlatformMetricsService with 5-min cache
- New endpoint GET /api/galactic/metrics/platform replaces N+1 client-side fetches
- IME log tracker enhancements, DO telemetry pattern, and download progress improvements
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Tenant Admins can report sessions for analysis via a new button on the
session details page. Creates a ZIP with session JSON, events JSON,
analysis results, timeline TXT, events CSV, optional agent log (5 MB
limit), and optional screenshot — uploaded to central blob storage.
Admin-config page gets a new Session Reports table (Galactic Admin only)
showing all submitted reports with comment, email, and blob reference.
Export utilities (generateUiExport, generateCsvExport) extracted to
shared lib/sessionExportUtils.ts for reuse across admin export and
report modal.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
## Rules as GitHub single source of truth
- Add 75 individual JSON rule files (2 gather, 18 analyze, 55 IME patterns)
under rules/gather/, rules/analyze/, rules/ime-log-patterns/
- Add JSON Schemas for all three rule types with VS Code IntelliSense support
- Add CI combine script (rules/scripts/combine.js) + GitHub Action that
validates schemas with ajv-cli and generates rules/dist/ combined files
- Convert BuiltIn*.cs from hardcoded C# objects to embedded JSON resource loaders
- Add GitHubRuleRepository service to fetch dist/ files from raw.githubusercontent.com
- Add ReseedFromGitHubFunction: POST /api/rules/reseed-from-github?type=all|gather|analyze|ime
## IME Log Patterns
- Extend ImeLogPattern model with Description and IsBuiltIn properties
- Add ImeLogPatterns table + API constants
- Add TableStorageService CRUD methods for IME log patterns
- Add ImeLogPatternService (CRUD, merge, auto-seed from embedded JSON)
- Add ImeLogPatternsFunction: GET/PUT /api/rules/ime-log-patterns (Phase 1: read-only
for tenants, Galactic Admin can enable/disable/edit globally)
- Add IME Log Patterns management page (/ime-log-patterns)
- Wire GetAgentConfigFunction to use ImeLogPatternService instead of static class
## API route consolidation
- All rule endpoints now under /api/rules/* for single certificate exclusion path:
/api/rules/gather, /api/rules/analyze, /api/rules/ime-log-patterns,
/api/rules/reseed-from-github
## Export buttons
- Add per-rule and export-all JSON download buttons to gather-rules,
analyze-rules, and ime-log-patterns pages
## Admin configuration
- Replace old individual reseed buttons with unified "Fetch & Reseed from GitHub"
(calls new GitHub endpoint) plus compact individual-type reseed buttons
- Add CDN cache hint: GitHub raw.githubusercontent.com caches up to 5 minutes
## Preview mode
- ?preview=1 URL param seeds localStorage.previewMode (persists across navigation)
- ?preview=0 clears it; toggle in Navbar settings dropdown (indigo, eye icon)
- IME Log Patterns nav link gated behind galacticAdminMode OR previewMode
- Preview badge shown on nav link when active for normal users
## Tests
- Add xUnit test project (net8.0) with 5 tests for embedded resource loading,
known IME actions validation, and valid categories check
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Galactic Admins can block a device by serial number from the admin-configuration
page. The block is enforced at ingest time (after security validation, before
body decompression) and is configurable in duration (default 12h, max 30 days).
The agent receives a DeviceBlocked signal in the IngestEventsResponse and stops
all uploads for the session. Blocks persist across backend restarts via a new
BlockedDevices table in Azure Table Storage, with an in-memory cache for fast
ingest lookups.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sends a Telegram message to @AutopilotMonitor-PrivatePreview whenever
a new tenant signs up. The bot webhook URL is stored in the new
PreviewConfig table (PartitionKey=TelegramBot, RowKey=config,
WebhookUrl=<url>) — same lifecycle as PreviewWhitelist, easy to remove
after GA. Notification is fire-and-forget and non-fatal.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Oliver Kieselbach