mirror of
https://github.com/okieselbach/AutopilotMonitor.git
synced 2026-10-24 12:42:38 +00:00
5cb9b0b30b0ca09c01ba15d76eb0b28c46507939
129 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Oliver Kieselbach
|
5cb9b0b30b |
backend: storage SDK budgets under the agent timeout, ingest 503 on an exhausted budget, request-token fan-outs, single-flight factory budget, ProcessStall monitor (audit F09, D-203)
- StorageClientOptions: Table/Queue 8 s per attempt × 3 attempts + capped backoffs = 25.8 s < the agent's 30 s upload timeout (pinned by test); Blob 60 s × 4. Every production client takes its options from there. No Polly over the SDK. - StorageErrors.IsTransient(Exception): OperationCanceled / IO / HttpRequest / Aggregate-of-those / RequestFailed status 0; the ingest answers 503 + Retry-After instead of 500. - The ingest still takes no caller cancellation (a batch is a unit). GetAllSessionsPageAsync carries the request token down to BoundedFanOut (Global Admin list); SingleFlightCache hands its factory a 30 s budget token and evicts on timeout (tenant ids, app-summary snapshot, session stats). - ProcessStallMonitor: 1 s tick drift ≥ 1 s ⇒ ProcessStall event (sampling bypass) + metric with cause gc / threadpool / unknown from GC.GetTotalPauseDuration and thread-pool counters; GcPauseMs on request rows ≥ 100 ms. Measures the instance-local 5–10 s pauses seen in production. - Gitlinks: internal (plan, D-203, concept doc, B-9hf, lesson), .claude/commands (marker rows). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
ec0d8ad735 |
backend: blob proxy lifetime and timing, one delegation key per process, shared KEV instance, single-flight config misses (audit F11/F12)
- DiagnosticsBlobStreamer disposes the download result on every exit path (it was disposed only on the 413 branch), measures open and transfer separately and fires DiagnosticsDownloadProxied after the copy with Completed=true/false. - UserDelegationKeyCache: one user delegation key per process (24 h, re-mint under a lock with 2 h margin or when a SAS would outlive it) instead of one storage round-trip per download or upload SAS. - KevDataService is resolved as one singleton over the typed-client handler pipeline; the cache warmer used to hydrate a throwaway transient instance while the correlation service held a cold one. - Admin and tenant configuration services single-flight concurrent cache misses (Lazy<Task> per key); the MemoryCache and the in-place projection semantics stay untouched. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
fc9e9d13e6 |
backend: keep slow storage successes and count CAS conflicts as a metric (audit F10)
The dependency filter dropped every successful storage call and the expected
412/409 outcomes, so storage-side latency and ETag contention were invisible
until a call failed outright (seven days of App Insights held zero storage
dependency rows).
- StorageDependencyFilterProcessor keeps a successful storage call at or above
2 s and stamps SlowStorage=true; the operation name is on the row, the table
name is not (the Azure SDK span carries neither URL nor table).
- StorageMetrics wraps TelemetryClient.GetMetric("StorageCasConflict",
operation, table, outcome): pre-aggregated per minute, bypasses the worker's
adaptive sampling, null-tolerant for tests.
- The CAS loops report conflicts: session event count, session counter
reconcile, session store, tenant stats, MCP user/tenant usage.
- TableStorageService keeps its two-argument constructors (Moq matches by
exact arity); DI resolves the three-argument overload.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
c8ce7cdb37 |
backend: remove the V2 index tables and their dual-write pipeline (D-196)
The five cross-session index tables (SessionsByTerminal, SessionsByStage, DeadEndsByReason, ClassifierVerdictsByIdLevel, SignalsByKind) were write-only since M5.d: no endpoint, MCP tool or web page ever read them, while the dual-write cost a serial queue send per signal/transition on the ingest path, a 5-13 min full scan of Signals + DecisionTransitions every two hours and five property-filter scans per session deletion (audit 2026-09-05 F05). Removed: the tables from TableNames.All (schema hash changes, next cold start runs a full pass without them), the telemetry-index-reconcile queue name, producer, envelope, handler, queue function and 2h timer, the cross-partition Timestamp scans on the signal/transition repositories, the EnableIndexDualWrite flag (model, config mapping, admin UI, generated wire types), cascade steps 11-15 (later steps renumbered 11-18), the deletion wrappers and the offboarding wipe entries. DiscriminatorPkProp stays as a step class so manifests written before this change still resume. Order of operations: deploy first, then delete the queue pair and the five tables in the portal (B-88h). Deleting the tables before the deploy 500s every delete preview/enqueue and poisons offboarding. Submodules: .claude/commands (import_session.py), internal (D-196, docs, plan). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
86fdc5aca9 |
backend: stateless queue consumers become [QueueTrigger] functions; webhooks go through a queue; poll loop receives one message at a time
Audit 2026-09-05 F01/F02. Analyze, VulnerabilityCorrelate, TenantAutoApprove, IndexReconcile and ImeMsiArchive are now host-managed queue functions under Functions/Queue bound to the DataStorage connection (identity-based, resolved by the host): the Flex scale controller wakes an instance for a message, the host renews the lease and owns the retry/poison ladder. The four cascade workers with lease/CAS state stay self-polling BackgroundServices; their base now receives one message per poll so waiting messages can no longer expire under another worker. Terminal enrollment alerts and hardware-rejection alerts are enqueued on notification-dispatch (channel ids only, resolved at send time) instead of fire-and-forget sends the host could drop on scale-in. Requires the app setting DataStorage__queueServiceUri before deploy. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
98cfd76232 |
plans: Pro (MSP) — a permanent-Pro managing tenant confers Pro on its managed tenants, derived at read time (B-qvu)
A tenant in the owned group of a permanent-Pro tenant is effectively Pro with source "msp":
badge "Pro (MSP)" (also over its own Pro), the Pro entitlement set without the delegation
right (no transitive delegation, no slots), no trial CTA. Nothing is written for that —
TenantConfiguration.ManagedByProTenantId is a load-time projection from a process-wide
index over the Tenant Groups table (owner tier read raw, never through the config service),
so the end of the delegation, the manager losing its permanent tier, or its offboarding
revert automatically. The one write is the retention grace anchor (ProDowngradedUtc),
stamped by ProConferralService from the four paths that end a conferring relationship.
- FeatureEntitlementCatalog.Resolve → EditionResolution { Edition, Source, OwnPro };
Get(resolution) returns ProViaMsp for conferred Pro; the 3-argument ResolveEdition is
gone so every caller resolves from the config (incl. the projection)
- DelegatedAdminService home gate checks the DelegatedAdminAllowed entitlement, not the edition
- TrialExpirySweep reads through the config service: a managed tenant's expired trial emits nothing
- feature-flags + plan response carry editionSource; config/all projects payingCustomer and
managedByProTenantId; shared manifest + web/MCP codegen regenerated
- PayingCustomer: GA-only sales bookkeeping flag on the plan endpoint, no entitlement effect
- web: "Pro (MSP)" label/tooltips, plan section badge, delegated-access hints, tenant
management chips + paying checkbox
Tests: catalog matrix, ManagedTenantProIndexTests (incl. mutual management without recursion,
fail-closed on storage error), ProConferralServiceTests, projection on every config read path,
feature-flags payload, parity facts, serialization (projection never stored), plan transitions,
edition gate, sweep. Backend 5902 green, web tsc/vitest/lint clean, MCP tsc/vitest green.
Concept doc + D-188 in internal (gitlink); customer docs in the docs repo (MCP redeploy needed).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
501f994c7a |
logs proxy: three telemetry stores, a wall-clock budget and typed partial results (x29 phase 1)
POST /api/global/raw/logs (MCP query_backend_logs) now takes `source` = backend | web | mcp and `budgetSeconds` 5..180 (default 30). Parity with the az CLI was already by construction (same Kusto REST endpoint, verbatim query); the four remaining deltas were properties of the proxy and are gone (D-186): Backend - LogQuerySources (Shared vocabulary, exported as manifest section `logSources`) and LogQuerySourceCatalog: APPINSIGHTS_APP_ID → backend, APPINSIGHTS_WEB_APP_ID → web (both api.applicationinsights.io), MCP_LOG_ANALYTICS_WORKSPACE_ID → mcp (api.loganalytics.io). A missing setting makes one source answer 503 naming it, never the others. - AppInsightsQueryFunction: injectable IHttpClientFactory (named client without timeout) and TokenCredential; one CancellationTokenSource per request plus `Prefer: wait=<budget>` upstream, 504 names the budget; typed QueryBackendLogsResponse keeps the Kusto tables[].columns/rows verbatim (JsonElement cells) and adds source, budgetSeconds, elapsedMs and `partial`/`partialReason` when the store cut the result (previously swallowed); on failure the full upstream error JSON rides along as `upstream`. - TypedResponseGuard: the last SerializeBaseline entry is gone (typed request record). MCP - query_backend_logs: source (z.enum from the generated LOG_SOURCES), budgetSeconds, a client signal of budget + 15 s, rawTable result cap, description with per-source table hints. Submodules - .claude/commands: backend-logs / ux-insights / mcp-telemetry on `source`; query-appinsights.sh deleted (35 call sites rewritten); az only for the live container tail. - infra: migration scripts carry the two new app settings and role assignments (Monitoring Reader on the portal component, Log Analytics Reader on the MCP workspace). - internal: concept doc backend/log-query-proxy.md, D-186, docs and plan updated. Tests: catalog resolution, budget clamp, Kusto parsing incl. PartialError, error extraction, wire parity of the new response; backend 5832, MCP 683, web tsc, solution build green. Before live use: grant the two roles and set the two app settings (see the plan file). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
729ee56bde |
security: assume-breach alarm for the Global-Admin-only surface (x29 phase 0)
The 2026-09-04 audit found no bypass of the GA boundary, but a failed attempt left only a Warning trace without identity, and a non-GA tools/call for a GA-only MCP tool never reached the backend at all. This makes the boundary observable without adding a second alarm writer (D-185): Backend - PrivilegedRouteDenied (Security): raised from the policy middleware's deny path for every 403 by an authenticated caller on a GlobalAdminOnly route. Critical for a caller without a platform role, Warning for a Global Reader. Details carry upn, oid, tid, client source, MCP tool name, correlation id, reason and policy. Throttled one hour per caller+path per instance (PrivilegedDenialReporter, IPrivilegedDenialReporter seam). PolicyResult now surfaces the denied tier; the DENIED log line gains oid, tid, clientSource and mcpTool. - GET global/raw/access-probe: no-op GlobalAdminOnly route whose denial IS the signal (AccessProbeResponse DTO, catalog entry, parity test). - Defense in depth: AppInsightsQueryFunction and both TableQueryFunction triggers re-check IsGlobalAdmin in the body (RawGlobalAdminGate); an empty RequestContext fails closed. - Dual-register: OpsEventTypes.All, OpsAlertRulesSection.tsx, regenerated manifests. MCP - ga-tool-probe.ts: before dispatch, a caller without the GlobalAdmin role naming a strictGa tool leaves an unconditional [mcp-security] line (tool, upn, scope) and fires the access probe with X-MCP-Tool-Name; the SDK answer stays "Tool not found". - GA_STRICT_TOOL_NAMES exported and pinned equal to the registered GA-minus-Reader set. - debug-session prompt names query_backend_logs/query_table only for strictGa (Readers were told to call tools they cannot see); tool_call_rejected lines carry scope + upn; config-tool comment corrected against the actual catalog tiers. Tests: reporter (severity, details, throttle, fail-soft), middleware policy surfacing and IsPrivilegedDenial/BuildPrivilegedDenial, Program.cs middleware order source scan, raw gate 403, probe catalog + JWT pins, wire parity; MCP probe unit tests, Global Reader over HTTP on both eras, runtime refusal fires the probe, prompt gate. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
db020789e1 |
chore: private submodule docs/ becomes internal/ (knowledge + decisions + work files + private instructions)
- .gitmodules: docs -> internal, repo renamed to Autopilot-Monitor-internal; the OKF bundle now lives at internal/docs/ - .claude/CLAUDE.md rewritten: where knowledge lives, working-file rules, hard rules; imports the private half from internal/CLAUDE.md (silently absent without submodule access) - .claude/architecture-principles.md moved into the internal bundle (reconciled against the code) - tasks/ (gitignored) retired; working files live in internal/work/ - doc path references docs/<bundle> -> internal/docs/<bundle> in code comments, skills gitlink, codeql paths, build script - rule descriptions no longer cite an internal doc path (customer-visible text); rules/dist regenerated - public tree no longer names private operator tooling - .claude/commands gitlink: skill docs point at the new paths |
||
|
Oliver Kieselbach
|
12a75ef13d |
feat(delegation): MCP quota follows the managed tenant, delegated tenant slots, self-service delegation
Phase A — the budget follows the data
- A delegated (MSP) read is charged to the MANAGED tenant's organization MCP budget and checked against
its plan; the caller's own budget still follows the home tenant. 429 names the managed tenant (domain)
and the Pro upgrade path; new McpQuotaExceededResponse.targetTenantId.
- Bounded fleet aggregate (global/sessions, global/stats/sessions): exhausted managed tenants are excluded
softly — RequestContext.AllowedTenantIds is narrowed (never widened, never null; pinned) and echoed as
quotaExcludedTenants. Global Admins stay exempt and are tracked on their home tenant only.
- Tenant Group flag ChargeHomeTenantQuota (PATCH global/tenant-groups/{id}, audited per tenant): reads
through the group are charged to the assignee's home tenant (operator-managed service groups).
- McpTenantUsage rows carry UserPrincipalName / HomeTenantId / LastRequestAt; new
GET metrics/mcp-usage/organization (budget by account incl. delegated readers) rendered on MCP Usage.
- MCP: get_fleet_overview (delegated + platform) over the two subset-tier routes; error handler names
the managed tenant. Other delegated tools stay tenant-mandatory.
Phase B — delegated tenant slots
- EditionEntitlements.MaxDelegatedTenants (Community 0 / Pro 2), TenantConfiguration.MaxDelegatedTenantsOverride
(GA-only via PATCH config/{tenantId}/plan { maxDelegatedTenants }), DelegatedSlotService counting distinct
managed tenants per home tenant through identity bindings (+ pending invitations + 24 h holds), enforced
fresh on grant / group assign / add-tenant with 409 DelegatedSlotLimitReached; GA UI offers
"raise to N & retry"; GET global/delegated-slots/{tenantId}; feature-flags entitlements.maxDelegatedTenants.
Phase C — self-service delegation
- One implicit Tenant Group msp-{tenantId} per managing tenant (OwnerTenantId on the meta row);
DelegationInvitations table (invitations + slot release holds); DelegationInviteTicket (HMAC, 7 days,
locates the row only — the accepting tenant is always the JWT tenant; one-shot via ETag).
- DelegationSelfService: ordered accept chain (distinct codes), remove/revoke with 24-hour slot hold, dual
audit and live-stream disconnect, assignees bound to tenant membership, "who manages my tenant" view;
GA release-hold escape hatch. No delegations/* route carries {tenantId} (pinned).
- Web: Settings → Tenant → Delegated Access, /delegations/accept?token=, AuthGate keeps the deep link
across the activation detour, Tenant Management holds + release, owned groups read-only in GA UI.
- Offboarding wipes both sides (documented two-bucket membership); 90-day sweep of terminal rows.
- Trust pages (terms, privacy) and generated wire types updated; docs submodule bumped.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
4b8b2a5a6d |
fix(mcp): AllMembers grants tenant members only, not every authenticated token; per-user overrides stay visible
McpAccessPolicy.AllMembers returned Allowed for any valid JWT, so a Progress Portal end-user without a role could open an MCP session. The AllMembers path now requires an effective role (Admin / Operator / Viewer) in the token's tenant, resolved by the new TenantMemberRoleResolver that the policy middleware shares (table first, Entra app-role claim when the tenant opted in). Explicit grants (platform role, delegated scope, enabled McpUsers row) and the Disabled-row kill-switch keep working in a policy-independent order; auth/mcp and auth/me hand the token's app roles through. The MCP Users settings section shows the user list under AllMembers too, as the per-user override list (individual usage plan, disable = block). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
e85dd082d6 |
perf: read caches, cross-tenant fan-out and lean MCP results (2026-09 perf audit, P1)
Backend - SingleFlightCache<T>: per instance, TTL, single-flight, failures never cached, expiry sweep, optional single slot - Rule catalog caches in TableStorageService.Rules.cs: gather rules / rule states / IME patterns per partition (60 s), every Store/Delete invalidates, hand-outs are clones (GatherRule/ImeLogPattern/RuleState.Clone()); GatherRuleService and ImeLogPatternService seed behind a gate and read global/tenant concurrently; GetAgentConfig reads rules and patterns concurrently - Tenant-id snapshot (60 s) shared by every cross-tenant fan-out; the paged session list no longer rescans TenantConfiguration per page - BoundedFanOut + DrainSessionsIndexWindowAsync: one PartitionKey-anchored, RowKey-bounded, projected query per tenant; GetAllSessionStatsAsync drains with SessionStatsProjection under a 30 s single-flight cache; GetGeoWindowSessionsAsync reads the index key range; GetSessionsByDateRangeAsync fans out per tenant - App summaries: cross-tenant windowed projected reads fan out per tenant over a union projection and share a single-slot 60 s snapshot (Installs tab pair, app detail, usage refs); days clamp on global/metrics/app - CancellationToken on the uncached geo/date-range reads (IMaintenanceRepository) - EventFieldProjection accepts data.<key> slices - BackendStartupMs measured from Main (StartupClock) instead of Process.StartTime, which carried the Flex placeholder lifetime Web - Dashboard stats: 30 s SignalR debounce in the GA aggregate scope (every tenant's events drove a cross-tenant drain every 3 s) MCP - Compact JSON tool results and resources (MCP_PRETTY_JSON=true opts back in) - DEFAULT_FIRST_PAGE_SIZE 50 for every paginated tool; the auto-exhaust budget is row-based (2000 rows) so scan coverage does not shrink - Lean first-page projections: get_session_events without data, query_raw_events without DataJson; announced in-band (omittedFields/omittedNote) and restorable via fields; a continuation keeps the nextLink projection - get_session_summary fetches the timeline with the triage fields plus data.<key> slices for its guards Docs: docs/backend/read-caches-and-cross-tenant-fanout.md (submodule 11fc393). Deploy order: backend, web, then MCP (the summary's data.<key> projection needs the new backend). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
d763a885e2 |
Ops alert channels, ops-events server-side filters, and generated MCP tool vocabularies (#140)
* feat(backend,web): Telegram als Channel-Provider + ein einziger Dispatch-Pfad Notifications hatten zwei Sendewege: WebhookNotificationService.SendToChannelsAsync fuer tenant-konfigurierte Channels, und TelegramNotificationService direkt aus dem Ops-Alert-Pfad. Ein neuer Provider haette beide anfassen muessen. Neu: NotificationChannelDispatcher ist DIE Channel-Sende-API und routet pro Provider — WebhookProviderType.Telegram zum Bot, alles andere als gerenderten Webhook-POST. SendToChannelsAsync wurde aus WebhookNotificationService entfernt statt dupliziert, damit es keinen zweiten Pfad gibt; alle sieben Call-Sites ziehen um (EventIngestProcessor, RegisterSession, ReportDistress, SlaBreachEvaluation, AnalyzeOnEnrollmentEnd, TestWebhookNotification). SendOpsAlertAsync/SendNotificationAsync sind virtual (Test-Seam), TelegramNotificationService bekommt SendAlertWithResultAsync fuer die Test-Endpoints und RenderAlertText als testbaren Kern. Telegram (=40) ist der Sonderfall: kein Webhook. Der Channel-Url traegt die CHAT-ID, der Bot-Token gehoert der Plattform (PreviewConfig). Ein Tenant-Admin wuerde durch UNSEREN Bot senden — deshalb ist das Gate eine Server-Regel, keine versteckte Dropdown-Option: ValidateTelegramChannelGate sitzt in TenantConfigValidation.ValidateModel und deckt damit BEIDE Schreibpfade ab (Full-Model-PUT und den MCP-Field-Patch ueber TenantConfigPatchService) — in der Update-Function allein waere der Patch-Pfad daran vorbeigelaufen. Ein unveraenderter, GA-angelegter Channel passiert das Gate, damit ein Tenant-Admin unrelated Config weiter speichern kann. Telegram-Ziele ueberspringen SsrfGuard.ValidateWebhookUrlFormat (eine Chat-ID ist keine URL und wuerde immer failen) und werden stattdessen als numerische Id oder @username geprueft. Mitgeliefert, weil der Dispatcher und die Ops-Seite sie brauchen: NotificationChannel.RedactList / RestoreRedactedList als geteilte Helfer hochgezogen (TenantConfiguration delegiert jetzt dorthin, statt eine zweite Kopie zu pflegen), AdminConfiguration.OpsNotificationChannelsJson + GetOpsNotificationChannels() mit Legacy-Slot-Synthese, OpsAlertRule.NotifyChannelIds/IncludePayload, NotificationAlert.DataJson (nur GenericJsonRenderer serialisiert es, additiv in schemaVersion 1.0). Web: der Channel-Editor liegt jetzt in components/notifications/ChannelEditor.tsx und wird von Tenant- und Admin-Seite geteilt (showEventToggles trennt die Faelle). Telegram erscheint im Provider-Dropdown nur fuer Global Admins (Prop aus useAuth().user?.isGlobalAdmin, Hausmuster aus SectionAutopilotValidation); ein bereits konfigurierter Telegram-Channel bleibt fuer alle sichtbar, damit ein Speichern ihn nicht still verwirft. Tests: TelegramChannelProviderTests (27) — Dispatcher-Routing inkl. "Telegram erreicht NIE den SSRF-Pfad", Chat-ID-Validierung, GA-Gate auf beiden Schreibpfaden. Mutationsgeprueft: Gate invertiert => 5 rot, zurueck => 27 gruen. OpsNotificationChannelsTests + OpsAlertPayloadTests decken die mitgelieferten Modell-Additionen ab. TestNotifications ist der gemeinsame inerte Stub, der die 17 bestehenden OpsAlertDispatchService-Konstruktionen ersetzt. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(backend,web): Ops-Alerts routen auf benannte Channels statt auf drei feste Slots OpsAlertRule hatte kein Ziel. Jede matchende Regel ging an alle drei fest verdrahteten Slots in AdminConfiguration (Telegram-ChatId / Teams-URL / Slack-URL), und nur die ERSTE matchende Regel wurde ueberhaupt ausgewertet (FirstOrDefault). Ein Conversion-Signal an ein Sales-Team zu binden hiess also: das Sales-Team bekommt auch jede Security-Regel. Jetzt: AdminConfiguration.OpsNotificationChannelsJson haelt eine benannte Channel-Liste (dasselbe NotificationChannel-Modell wie tenant-seitig, nur anderer Scope), und OpsAlertRule.NotifyChannelIds bindet Regeln daran. Dispatch wertet ALLE matchenden Regeln aus (Where statt FirstOrDefault) und vereinigt ihre Ziele. Rueckwaertskompatibel ohne Migrationsjob: - Leere Channel-Liste => GetOpsNotificationChannels() synthetisiert je konfiguriertem Legacy-Slot einen Channel mit stabiler Id (legacy-telegram heisst "Autopilot Monitor IT Engineer") und uebernimmt die Legacy-Enabled-Flags — ein bewusst abgeschalteter Provider bleibt abgeschaltet. - GetAdminConfigurationFunction projiziert DIESELBE Synthese in das Read-Model, damit der Editor zeigt, was heute tatsaechlich Alerts empfaengt, und der erste Save exakt das materialisiert. Die Alternative waere eine zweite Kopie der Synthese-Regeln im Web gewesen, die vom Dispatcher wegdriftet. - Leere NotifyChannelIds = alle aktivierten Channels. Das ist, was jede heute existierende Regel traegt, also verhaelt sich am Deploy-Tag nichts anders. - Ids, die nicht mehr aufloesen, werden verworfen statt auf Broadcast zurueckzufallen: eine Regel, deren einziger Channel geloescht wurde, muss NICHTS erreichen, nicht alles. Die Legacy-Felder bleiben als Migrationsquelle stehen und werden von der Alerts-Section nicht mehr geschrieben — aufraeumen erst, wenn produktiv Channels stehen. Sicherheit: OpsNotificationChannelsJson ist secret-tragend und steht in der Deny-List von RedactedCopyForReader() (ConfigRedactionTests bewacht Drift). Der Admin-PUT hat bisher keinen Redacted-Restore — er bekommt jetzt einen fuer die Channel-Liste, damit ein zurueckgespieltes Reader-View nicht "***REDACTED***" als Chat-ID persistiert, plus die strukturelle Validierung, die auch die Tenant-Liste passiert. Neuer Endpoint POST /api/global/config/test-ops-channel (GlobalAdminOnly, im Policy-Catalog registriert). Kein neues Response-DTO: TestWebhookNotificationResponse hat identische Semantik und wird geteilt — nur der Doc-Kommentar nennt jetzt beide Routen. Web: die "Alert Providers"-Karte weicht der Channel-Liste (geteilter ChannelEditor, im Admin-Kontext mit Telegram und ohne die tenant-spezifischen Event-Toggles). Jede Regel bekommt einen Channel-Picker; leere Auswahl rendert als "all channels", nicht als Warnung — hier ist das der gueltige Default. Die Bindungslogik liegt testbar in opsChannelRouting.ts und spiegelt ResolveTargets: der erste explizite Pick startet vom LEEREN Set, sonst liesse sich eine Broadcast-Regel nie verengen. handleSaveOpsAlertConfig nimmt jetzt ein Objekt statt elf positionaler Argumente — bei einem Dutzend Feldern schreibt ein vertauschtes Paar sonst still eine Webhook-URL in den falschen Provider-Slot. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(backend): TenantTrialStarted + Event-Payload optional pro Regel Der Trial-Start war stumm. POST config/{tenantId}/trial schrieb nur einen Audit-Entry, und der GA-Pfad (PATCH plan) feuerte nur beim Downgrade — also war genau der Conversion-Moment, den Sales/Support interessiert, das einzige Plan-Ereignis ohne Ops-Event. RecordTenantTrialStartedAsync (Tenant/Info) feuert jetzt aus BEIDEN Schreibpfaden, unterschieden per selfService-Flag, und jeweils NACH dem persistierten Save (SaveConfigurationAsync ist fail-loud), damit kein Alert eine Trial ankuendigt, die nicht gespeichert wurde. Der GA-Pfad haengt an der aufgezeichneten Aenderung, nicht am Request: ein reiner planTier-Edit bleibt still, ein idempotenter Save mit demselben Datum ebenfalls, und das Beenden einer Trial (explizit null) ist ein Downgrade und bleibt bei TenantPlanDowngraded. Payload-Zustellung: WriteAsync reicht die serialisierten Details an DispatchAsync durch — aber ein Channel bekommt sie nur, wenn eine Regel OpsAlertRule.IncludePayload setzt. Default AUS, bewusst: ein Ops-Alert ist ein "da ist was, schau nach"-Signal, die meisten Payloads sind im Chat Rauschen, und manche tragen Felder, die die Baseline nie traegt (Tenant-Domain, Contact-Adresse). Das Informationsniveau aller bestehenden Regeln bleibt damit unveraendert. ResolveTargets liefert deshalb zwei Gruppen. Ein Channel, den sowohl eine Payload-Regel als auch eine schlichte Regel adressiert, landet nur in der Payload-Gruppe: eine Nachricht pro Channel, und ein unbeteiligtes Geschwister-Match kann ein bewusstes Opt-in nicht aushebeln. Fuer den angereicherten Alert flacht BuildAlert die Top-Level-Skalare in Facts ab (max 12, Werte <= 256 Zeichen, Nulls und Verschachteltes raus), damit Card-Formate und der Telegram-Klartext dieselbe Information tragen wie der GenericJson-Consumer unter "data". EventType wird immer gesetzt — derselbe Wert steckt schon im "Event"-Fact, also keine zusaetzliche Information, nur ein stabiler Schluessel zum Verzweigen. Tests: TenantTrialStartedEventTests (8) — aufgezeichnete Event-Form inkl. Payload-Feldern, und die Emissionsbedingung des GA-Pfads gegen ApplyPlanChanges gepinnt (greift bei Grant und Verlaengerung, schweigt bei Trial-Ende, planTier-only und idempotentem Save). Die HTTP-Entry-Points bleiben ungetestet — Suite-Konvention ist, die reinen Kerne zu testen. Docs: OKF-Concept backend/ops-notification-channels.md. Registrierung in docs/index.md + docs/log.md fehlt bewusst — beide Dateien tragen noch fremdes uncommittetes WIP (ops-events Query-Surface), und ein Teil-Commit darin haette es mitgenommen. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(backend,mcp): ops-events server-side field filters + generated MCP tool vocabularies The ops-events read could only narrow by category and date window, so answering "did any AgentEmergencyBreak fire yesterday?" meant pulling a whole category and filtering it client-side — and for the MCP tool that client is a language model, so the question cost a category-sized response in model context. Ops-events filter surface (mirrors the audit-log filter pattern one-for-one): - OpsEventQueryFilters (Shared) + OpsEventFilterRequest (query -> filters + pagination extras) + TableOpsEventRepository.AppendFieldFilters (filters -> OData clauses). BOTH filter builders call the appender — the single-category / unpaged one and the per-partition fan-out builder that adds the RowKey bound — so "no category named" cannot silently return unfiltered rows. - minSeverity expands to an OR-set, not a range: Table Storage compares strings lexicographically and the severity names sort Critical < Error < Info < Warning, so `Severity ge 'Warning'` would drop exactly the alert-worthy half. The ladder now lives in OpsEventSeverity.Rank, shared with OpsAlertDispatchService, so "Warning and above" in a read means what it meant when the alert rules fired. - severity is normalised onto the canonical vocabulary and an unknown value is a 400: `eq` is case-sensitive, so "?severity=warning" would otherwise return an empty page that reads exactly like "no such events happened". eventType stays verbatim — stored rows outlive the vocabulary, so a retired type must remain searchable in the historical window. - Every filter value is folded into the continuation fingerprint and echoed on nextLink; extras are appended AFTER the pre-existing category/tenantId discriminators and skipped when empty, so tokens minted before the filters existed still fingerprint identically. The tenantId drill deliberately stays in memory (OrdinalIgnoreCase; an OData eq would lose rows on casing). - MCP get_ops_events gains eventType / severity / minSeverity plus a `days` shorthand resolved client-side to a concrete dateFrom (a backend-side "last N days" would re-resolve "now" on every page and blow the token fingerprint); an explicit bound always wins over the shorthand. Fixed alongside: the paged cross-category fan-out skipped Platform (Azure Monitor alerts) because its category list had drifted from OpsEventCategory — written from day one, invisible to every paged reader, still visible on the unpaged full-table path. The list is now OpsEventCategory.All plus a reflection coverage test. That drift had a structural cause, so the same round closes it for the MCP as a whole: wire TYPES reached the MCP from day one, but vocabularies did not — types are erased at runtime and z.enum() needs values, so every tool schema hand-typed its own copy with nothing to compare against. The copies had drifted: search_sessions offered 5 of 8 session statuses (no AwaitingUser, no Incomplete — both real stored statuses), both event readers offered 4 of 6 severities (no Debug, no Trace), get_ops_events named 6 of 7 ops categories. - The codegen now emits src/generated/wire-vocabularies.generated.ts (VALUES: session statuses, event severities, ops categories/severities/types, annotation lanes/verdicts) from the manifest sections listed in MCP_VOCABULARIES; SharedManifestParityTests exports the two new ops vocabularies. Freshness is pinned twice — byte-equality against a fresh codegen run in the MCP suite, and the shared-manifests-in-sync CI job. - Tool schemas derive their z.enum() from it and descriptions interpolate the list instead of spelling it. - vocabulary-drift.guard.test.ts ratchets what is left: every inline z.enum([..]) must sit on a baseline naming why it has no backend owner (MCP-local knobs, plus rule type / connection type / CVE risk, which have no C# constants class yet). A new hand-typed list fails, and so does a stale baseline entry. - OpsEventTypes (Shared) gives the 77 ops event types a canonical home: the OpsEventService write sites use the constants (a raw literal now fails OpsEventTypeDualRegisterTests, which moved from regex-scraping to reflection), the manifest exports them, and get_resource(name="ops_event_types") serves categories + severities + types to the model from the generated copy. Suites: Backend 5427, MCP 629 + tsc, Web tsc + vitest 965, generator idempotent. Deploy order: backend BEFORE the MCP redeploy — an MCP that sends filters to an older backend gets silently unfiltered rows back and presents them as filtered. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: register the ops notification channels doc in the index and log The concept doc landed with the feature commits; its index entry and log entry were still uncommitted in the working tree. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
d055eed074 |
Typed API contract: every success response is a wire DTO, generated end-to-end into TypeScript (#136)
* feat(backend): typed API response infrastructure — IApiResponse marker, wire-settings single source, widened anonymous-body ratchet (batch 0 of typed-api-contract) IApiResponse (Shared, with SuccessMessageResponse/SuccessOnlyResponse and the ProjectedItems/WireContract attributes for the upcoming manifest export) plus generic OkAsync/CreatedAsync/JsonAsync<T> overloads constrained on it, so anonymous success bodies stop compiling once the legacy object overloads go; ApiJsonOptions is now the single source of the production wire settings (camelCase, WhenWritingNull, string enums) used by Program.cs AND the three endpoint-test harnesses (two serialized PascalCase-with-nulls, one JsonSerializerDefaults.Web — all could pass while the deployed wire differed); TypedResponseGuardTests replaces OkAsyncBaselineGuardTests with the same ratchet mechanics over two frozen baselines (44 anonymous helper sites unchanged, plus 134 anonymous success WriteAsJsonAsync sites across 76 files — error-shaped bodies stay anonymous by design) and a flatness/location reflection guard (IApiResponse implementers derive from object and live in Shared, protecting JSON key order); ApiResponseWireParityTests carries the ordinal old-vs-new serialization proof each converted site must add Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(backend): Sessions/Annotations/Reports/Raw/Progress success responses are typed wire DTOs (batch 1 of typed-api-contract) 34 anonymous success bodies across 24 functions replaced with flat IApiResponse DTOs in Shared (SessionApiModels/ReportApiModels + new Annotation/Raw/ProgressApiModels) — minimal in-place transformation, call structure/status codes untouched, declaration order == wire order, nullable exactly where a site can emit null (WhenWritingNull); fields= projections stay IReadOnlyList<object> marked [ProjectedItems(SessionSummary/ EnrollmentEvent)], raw table rows stay PascalCase-verbatim dictionaries (no DictionaryKeyPolicy — pinned by test), AnnotationWire and the distress-report aggregations now build named item types instead of anonymous ones, and shared shapes collapse to one DTO (SessionListResponse x4, SessionAnnotationListResponse x2, SessionReportListResponse/GetSessionEventsResponse paged+unpaged via absent-null nextLink); 54 new wire-parity facts serialize each OLD anonymous literal (independent fixture copied from the pre-migration site) against the new DTO ordinally, with a null case per vanishing key; ratchet baselines shrink 44->14 helper sites and 134->122 write sites; the three distress-report test files read the now-typed aggregation items directly instead of via reflection-ToDynamic Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(backend): every anonymous API success body is a typed wire DTO — ratchet baselines empty, legacy object overloads deleted (batches 2-5 of typed-api-contract) the remaining 141 anonymous success sites (Admin 36, Rules 24, Config 15, Graph 5, Metrics 16, Vulnerability 17, Infrastructure/Global/Notifications/Bootstrap/ Diagnostics 20 incl. the 429 quota body in McpQuotaEnforcementMiddleware, plus the 14 helper-call sites) now return flat IApiResponse DTOs from Shared (new Admin/ Rules/Config/Graph/Metrics/Vulnerability/Infrastructure/Notifications/Diagnostics ApiModels; GeographicApiModels extended) — same minimal in-place transformation as batch 1: declaration order == wire order, nullable exactly where a site can emit null, dual success/failure sites keep their status-code logic and reuse SuccessMessageResponse, MCP-critical key names (records/summaries/inventory) pinned; wire item shapes that were function-local (CustomsArchive Run/EntrySummary, CpeMappingDto, UnmatchedSoftwareAgg, AutoResolve items, LogPatternTestResult, CveExposureSummary/SeverityBreakdown/TopCve — records rewritten as flat classes, netstandard2.0 has no IsExternalInit) moved to Shared so the manifest export sees them; payload types that must stay in the Functions assembly (RuleDryRun, PreviewWhitelistEntity, GlobalAdminEntity, HealthCheck, GlobalNotificationDto, Backfill/Reclassification/MaintenanceResult, TenantAdminEntity, TenantConfigFieldSchema) ride as object slots — wire-identical via runtime-type serialization, proven by the parity facts; 126 new wire-parity facts total (Admin 26, Rules 12, Config 17, Graph 7, Metrics 16, Vulnerability 19, Infrastructure 21 + Dictionary-key-verbatim and null-value pins); both TypedResponseGuard baselines are now EMPTY regression guards and the untyped OkAsync/CreatedAsync(object) overloads are gone, so a new anonymous success body no longer compiles through the helper and fails the guard through raw WriteAsJsonAsync; deliberately untouched: error bodies (one anonymous shape by design), auth/me (AuthDecisionResult.Body slot shared with error bodies, tests reflect over it), CheckMcpAccess (conditional dictionary keys), raw-byte streamers Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(contract): C#→TS wire codegen — shared-manifests v2 exports the full response-type graph as generated TypeScript (batch 6 of typed-api-contract) SharedManifestParityTests now writes schemaVersion 2: the v1 catalog sections stay byte-compatible (sectionFieldMap/signalrMessages/sharedManifestChecks untouched), the consumer-less sessionSummary section is gone, and a new types section carries every wire type — roots are all IApiResponse implementers plus [WireContract]-marked types (SlaViolatorSession, ImeVersionHistoryEntry), transitively closed by the new WireTypeManifestBuilder over refs, collections, dictionaries, enums and [ProjectedItems] item types (208 types). Fields honour [JsonPropertyName]/ [JsonIgnore], get-only members export readonly, a nullable slot exports optional (WhenWritingNull omits the KEY — explicit null survives only inside collections and dictionary values, so the nullable flag lives on nested descriptors only), field order follows STJ's derived-before-base serialization for the one legacy inherited wire type (LocationSessionRow : SessionSummary), and unmappable shapes (non-string dictionary keys, foreign BCL classes) fail the build instead of degrading to unknown. C# <summary> texts travel as JSDoc: Shared builds XML docs now (doc-quality warnings CS1591/1573/1574/1584/0419 suppressed — crefs to foreign assemblies cannot resolve from Shared; CS1570 malformed-XML stays an error, which surfaced a raw 0x01 control char and an unescaped ampersand in existing comments). generate-shared-manifest-types.js emits utils/wire-types.generated.ts (interfaces + string-union enums, 2.7k lines) next to the lean literal module, npm run generate:manifests wraps it, the vitest freshness test pins BOTH outputs, and a new shared-manifests-in-sync CI job (guardrails-in-sync pattern) fails on drift Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(web): hand-written response types re-export the generated wire interfaces (batch 7+8 of typed-api-contract) types/session.ts, types/enrollment.ts and types/adminConfig.ts are re-exports over utils/wire-types.generated.ts (Session = SessionSummary alias for the 25+ barrel importers; adminConfig keeps only the excessiveEventAutoActionMode literal-union narrowing and the client-side OpsAlertRule view) — the generated types are never weakened, consumers adapt: durationSeconds is now optional as on the wire (int? — consumers guard with ?? 0, which also retires the latent NaN-minutes display on in-progress sessions), status is the SessionStatus union (fixtures on canonical values; the generic string-sort test keeps its own fixture type), required bools break hand-built fixtures once → new test/factories.ts makeSession/makeEvent (Object.assign, not spread — Partial spread re-widens and tests must force data: undefined); the string|null lifecycle branches were already truthiness-based. Duplicate local types deleted: geographic-performance now uses the real LocationSessionRow wire type (incl. DO fields), GlobalSearch the generated QuickSearchResult, sla/page.tsx the generated SlaViolatorSession (status stays a number — C# int, rendered via statusLabels); sessionExportUtils aliases SessionExportEvent = EnrollmentEvent and derives RuleResultCsvData via Pick, CSV output unchanged; the three structural ReplayInputEvent clones (InstallEvent, ScriptInputEvent, DownloadEvent) are aliases now. docs: drift-guards.md replaces the shrink-only-OkAsync section with the typed-wire-DTO contract, index.md/log.md updated. tsc 0, vitest 957, eslint 0 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(tests): SlaMetricsServiceTests anchor sessions inside the current ISO week — the 'now - 1h' fixture crossed into LAST week during the first hour of every Monday (UTC), emptying CurrentWeek and failing the three snapshot asserts (CI on Mon 2026-08-31 00:20Z; pre-existing Monday-morning time bomb, unrelated to the DTO migration); the anchor is now clamped to the week's Monday 00:00, proven green inside the same failure window Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
7a3516e570 |
feat(observability): IME pattern-drift loop — nothing the log tracker skips or stops matching stays silent: the agent keeps cumulative tracker health counters and a per-pattern match histogram (restart-safe in ImeTrackerState), reports ime_* keys in agent_metrics_snapshot (files tailed, backlog bytes, lines read, matches, oversized/timeouts/budget breaks/held tails/unanchored), raises a one-shot immediate-upload ime_tracker_degraded Warning on the first pass that skipped work, and emits ime_pattern_hits at termination (every enabled pattern, zeros included, IME version); the backend folds the histogram into the permanent per-IME-version ImePatternStats table (built-in IDs only, plausible version only, one partition read + one transactional batch) and evaluates drift without a hard-coded list — baseline = the version with the most reporting sessions (≥100), expected = ≥80 % hit rate there, drift = an expected pattern matched in none of ≥25 sessions on a newer version — raising ImePatternDriftSuspected once per version×pattern (ETag-stamped, dual-registered in the web ops-event list with a new guard test); GET metrics/ime-pattern-health (GlobalReadOrAdmin) + Global Admin → Metrics → IME Pattern Health matrix (drift/silent/low/few cells, open alerts, retired patterns) + session-page "IME Tracker" rows (latest snapshot by max sequence) + MCP get_ime_pattern_health (GA/Reader)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
fc46b4c78e |
feat(security): session-owner binding stage 1 (shadow) + dormant agent session rotation
Agent endpoints authenticated per tenant but authorized session writes on a caller-supplied session id. Sessions rows now carry the creating device identity (cert thumbprint + Intune device id from the CN, or bootstrap code + serial) in primary-only Owner* columns. SessionOwnershipPolicy evaluates register / telemetry / error-report requests against it; SessionOwnerBindingObserver records the outcome on the request row, as a warning and as a throttled SessionOwnerMismatch ops event. Nothing is rejected in this stage. Agent: errorCode parsing on 401/403, one-shot session rotation on session_owner_mismatch without feeding the auth-failure tracker, spool drop, agent_started.sessionRotated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
a306cd9e40 |
feat(security): resolve admin identity bindings automatically at grant time; per-tenant bindings panel
Granting a cross-tenant role is "type the UPN" again. AdminIdentityResolver derives the home tenant + object id from sign-in history (UserActivity: both values came from validated tokens, most recent sign-in wins), falls back to the UPN domain of an onboarded tenant (tid only, oid pinned on first sign-in), and never guesses when a UPN was seen under two tenants. Only a never-seen domain yields 422 HomeTenantUnresolved, on which the portal shows a home-tenant picker. - IMetricsRepository.GetSignInIdentitiesByUpnAsync (filtered UserActivity scan, grant-time only) - Grant endpoints: homeTenantId/objectId optional overrides; bindings dropped from the list bodies - Web: identity fields and binding pills removed from the delegated / tenant-group forms; new collapsed "Identity bindings" panel in Tenant Management -> Edit Tenant Configuration (bindings homed in that tenant; inline edit / remove via global/identity-bindings) - Tests: AdminIdentityResolverTests; grant validation now accepts an absent home tenant Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
8ec8524252 |
fix(security): bind platform and delegated roles to the immutable Entra tid+oid, not the UPN string
GlobalAdmins, DelegatedAdmins and TenantGroupAssignments resolved their role from the upn/preferred_username claim alone while the API accepts tokens from every Entra tenant, so a genuine token minted in a foreign tenant (re-registered domain) or by a recycled UPN in the home tenant inherited platform admin. - New AdminIdentityBindings table: RK = UPN -> home TenantId + ObjectId; in TableNames.All, CriticalBackupTables and the full-restore-forbidden auth set; offboarding wipes bindings homed in the offboarded tenant - AdminIdentity (upn + tid + oid) is what GlobalAdminService.GetGlobalRoleAsync and DelegatedAdminService.GetScopeAsync resolve on; order row -> binding -> Pro gate, fail-closed (no oid / no binding / tid or oid mismatch => no role), mismatches logged at Warning - Object id pinned on the first sign-in from the bound tenant (ETag-conditional, verified against the stored winner); a foreign-tenant token can never claim the pin - Grants (auth/global-admins, global/delegated-admins, tenant-group assignees) require homeTenantId (+ optional objectId) and answer 409 instead of re-homing a bound UPN; GET/PUT/DELETE global/identity-bindings is the audited rebind surface - RequestContext.ObjectId; throttle identity prefers oid so shared UPN strings across tenants no longer collide in one rate-limit bucket - Web: home tenant / object id fields on the grant and assign forms, per-UPN binding pill - Tests: binding verdict matrix + race semantics, middleware-level denials for foreign-tenant and recycled-UPN tokens; OKF doc docs/backend/admin-identity-binding.md Legacy role rows are inert until their binding is seeded - seed before deploying the backend. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
cf76c4338f |
feat(vuln): manual "Refresh NVD Cache" and "Sync EPSS" with their own last-sync lines
The admin Data Sync Status only offered KEV and MSRC; the NVD stale-row refresh and the EPSS re-score ran on the nightly timer alone. Both walks move out of the timer into VulnerabilityCacheRefreshService (singleton, one in-flight slot per walk) so the buttons and the nightly run share one code path and stamp the same timestamps (NvdCacheLastRefreshUtc — only for a walk that reached every stale row; EpssLastSyncUtc). POST vulnerability/sync-nvd starts the minutes-long, rate-limited NVD walk in the background and answers 202 (409 while running); the page polls sync-status (nvdRefreshRunning, nvdLastRun) every 10 s meanwhile. POST vulnerability/sync-epss runs synchronously and returns the counts. The timer skips a slot a manual run holds instead of queueing a duplicate. Shared manifest regenerated for the two new AdminConfiguration fields. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
a6184f165b |
feat(vuln): FIRST EPSS as fourth source, act/attend/track priority, CVSS vector; never cache a refused NVD answer
EPSS: EpssApiClient (api.first.org, 100 ids per batch, unauthenticated, fail-soft). Score, percentile and date live on the cached CVE — one batch per CPE fill, every cve_* row re-scored by the nightly sync, only changed rows rewritten and with their original CachedAt. CveIndex carries EpssScore (max, absent when unscored) and Priority; TopCve exposes both. CvePriority: act = CISA KEV, attend = EPSS >= 0.1 or CVSS >= 9.0, track = rest — riskLevel/overall_risk semantics unchanged. NVD throttle: Polly already retried 429s; the defect was that a refused answer came back as an empty response and was cached as "0 CVEs" for seven days (also the negative auto-resolve cache). NvdFetchResult separates answered from refused; nothing persists a refusal; a 429 opens a process-wide cooldown (Retry-After capped at 120 s, else 30 s) so a run fails fast instead of paying a retry chain per software item; hard-expired rows keep serving meanwhile. The NVD->cache projection and the row builder are now single functions shared by the live fill and the sync. CVSS v3 vector string persisted and shown. Portal: EPSS column + pill, priority pill, vector line; exposure panel pills. MCP tool descriptions and the CVE-audit prompt name the fields. Terms attribution + privacy page updated in the same change. OKF doc backend/vulnerability-correlation-sources.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
ea6a375c4b |
feat(security): cert-to-device binding, stage 3 GA-gated shadow
Tenant binding (stage 1) proves the client certificate was issued to this
tenant. It does not prove the certificate belongs to a device the tenant
still has: a cert lifted from a decommissioned machine keeps working.
IntuneDeviceBindingValidator resolves the Intune managedDevice id from the
certificate's Subject CN against the tenant's own managedDevices inventory.
Point-GET on managedDevices/{id} rather than an OData filter, so there is no
filter literal to inject into and a 404 is unambiguous. Cache/retry/transient
contract mirrors CloudPcDeviceValidator.
Shadow only, and visible only to Global Admins (same gate as the DevPrep
association preview) because the open question is a race, not an attack: a
device object can appear after the agent's first call. Every result carries
enrolledDateTime and the log line records enrolledAgeSeconds, so a device
created seconds before the request is distinguishable from a foreign cert.
Grep marker: CERT-DEVICE-BINDING-SHADOW.
The lookup is fire-and-forget like the DevPrep shadow so a cold Graph
round-trip never sits in the agent request path. That rules out stage 1's
request-row dimension (the row is written before the task finishes), hence
the Warning log; widening beyond preview should move to inline + dimension.
Needs DeviceManagementManagedDevices.Read.All, granted per tenant via the
customer script as feature IntuneDeviceBinding. The script's catalog,
ValidateSet and help block stay pinned to GraphFeatureCatalog by
GraphFeatureCatalogSyncTests (extended with the help-block case).
Build 0 warnings, backend 4629/4629, web 874/874, tsc clean, eslint 0.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
0d5e2572aa |
perf(telemetry): one canonical request record; drop expected storage outcomes and SignalR success deps
- RequestTelemetryMiddleware: SamplingPercentage=100 per item (worker copy was adaptively sampled, ItemCount 2-5 on weekdays, despite host.json excludedTypes) - StorageDependencyFilterProcessor: also drop 404/412/409 storage outcomes (InProc + HTTP shape) and successful *.service.signalr.net REST calls - infra/appinsights-workspace-transforms.bicep: DCR dropping the host HTTP request duplicate at ingestion (timer/queue rows kept) - docs/backend/telemetry-ingest-shaping.md; skills updated Measured baseline 7d: ~1.3 GB/week removed, requests now 100% with tenant context. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
3a89319472 |
feat(telemetry): startup duration metrics for backend and MCP
StartupTelemetryService emits BackendStartupMs (process start ->
ApplicationStarted) and BackendTableInitMs as App Insights custom
metrics with version/commit/tableInitFullPass/instance dimensions;
TableStorageService exposes LastInitialization. MCP server writes one
{"type":"boot","readyMs"} stderr line per start next to the
tool_call lines so cold-start time is queryable as a trend.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
7569061983 |
feat(admin): editable email templates — preview, test send and HTML override in Settings
- EmailTemplateService: override-or-built-in resolution per send (PreviewConfig
table, partition EmailTemplates, 5-min cache, {{domainName}} placeholder,
30k limit); send path fail-soft, admin read path strict
- GA endpoints global/email-templates/{kind} (GET/PUT/DELETE) and /test, which
mails the effective template or an unsaved draft to the caller's tenant
contact address
- portal: Settings → Email Templates with preview frame, test send, HTML editor
- mail header: brand mark drawn with CSS borders, no images left in the mails
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
8675e7ee06 |
feat(backend): provider-neutral EmailService on Mailchimp Transactional (Mandrill), replaces Resend
- EmailService + IEmailService: Mandrill messages/send via typed HttpClient with the Notification resilience policy; provider confined to one transport method and the Email:* settings (ApiKey, Endpoint, FromAddress, FromName). Resend NuGet dropped. - open/click tracking explicitly off; sent/queued/scheduled = ok, rejected/invalid = warn - callers (TenantApprovalService, PreviewWhitelistFunction) depend on IEmailService - EmailServiceTests pin wire format, result semantics and fail-soft contract - privacy page + OKF welcome-mail doc updated for the new sub-processor Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
935f0e8c71 |
feat(backend): agent-efficiency aggregate + honest scan count + filtered metrics fetch
- New GET /api/global/metrics/agent-efficiency (GlobalReadOrAdmin, ?tenantId=): per-agent-version percentiles (CPU, working set, private bytes, threads, handles, spool, API latency/requests), crash rates per version, top offender sessions per dimension, 5-min cache keyed (days, limit, tenant). First backend consumption of agent_thread_count/agent_handle_count. - PlatformAgentMetricsResponse.SessionsScanned: sessions the scan actually covered — the honest basis for the caller's truncated flag (Sessions.Count only counts sessions WITH snapshots). - Per-session fetch no longer drains the full Events partition: new GetSessionEventsByTypesAsync (multi-type OData filter + column projection + real maxResults enforcement); the 20 delivery-latency sample sessions keep the full fetch so latency deltas stay computed over all events. - Shared AgentMetricsAggregation (crash tally, payload getters, bounded fan-out, metrics event types) so the two services cannot drift. - GetAllSessionsPageAsync gains an optional select projection (default null, existing callers unchanged), pinned by an equivalence test. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
ca26cb4f7c |
backend: auto-archive IME installer on first sighting of a new version
When RecordImeVersionAsync reports a genuinely new IME version (~monthly),
EventIngestProcessor now also enqueues an ime-msi-archive job. The worker
(QueuePollingWorker, heartbeat, pause on ImeMsiArchivingEnabled=false)
downloads the installer and stores it write-once (If-None-Match:*) in the
permanent ime-archive container ({version}/IntuneWindowsAgent.msi +
provenance.json), then merges MsiArchiveStatus/BlobPath/Sha256/Bytes/SourceUrl
onto the ImeVersionHistory row - so every fleet-observed IME build stays
available for decompilation/diffing after Microsoft's versionless CDN moves on.
The download URL comes from agent telemetry (msiDownloadUrl enrichment) and is
treated as untrusted: https-only *.manage.microsoft.com allowlist with exact
filename match, canonical versionless fallback, no redirects, size cap
(MaxImeMsiDownloadSizeMB, default 250, preflight + mid-stream) enforced by a
SHA-256-hashing pass-through stream. Archiver never throws; the worker retries
only transient statuses (download/timeout/error), permanent rejections
complete with their Failed:* status persisted.
Admin settings ImeMsiArchivingEnabled (default true) + MaxImeMsiDownloadSizeMB
via the full 4-file web chain; archive fields on the version-history API are
GA-only (non-GA projection unchanged, redaction tests extended). Shared
manifests regenerated. Tests: archiver status branches via recording subclass
(allowlist, caps, 409 idempotency, cancellation propagation), config defaults.
Backend suite 4262 green, web tsc + vitest 762 green, solution 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
72136f0273 |
backend+mcp+web: evaluateOn interim triggers for analyze rules
Implements docs/rules/analyze-rule-triggers.md (spec approved today): - AnalyzeRule.EvaluateOn (absent = [enrollment_end], today's behavior bit-for-bit) with the shared AnalyzeRuleTriggers grammar/matching helpers; grammar enforced in tenant rule CRUD and the analyze-rule JSON schema. - RuleEngine: AnalyzeRunContext (Terminal/Interim/Reanalyze), interim rule filter, KO suppression on interim runs, and RuleResult update-semantics replacing the permanent dedupe freeze — interim rows refresh, the terminal pass finalizes or resolves them (ResolvedAt, kept for audit), Reanalyze preserves lifecycle markers. - Notification dedupe moves onto RuleResult.NotifiedAt (one send per session+rule, stamped after a successful send; survives the manual reanalyze rebuild). - Handler: whiteglove_sealed / interim_trigger reasons — notify + SignalR yes, IssuesDetected + rule stats terminal-only (Rule Regression Radar baselines clean). - Ingest: InterimTriggerRegistry (per-tenant 5-min TTL cache, fail-soft) gates the interim_trigger enqueue (batch eventType intersection) and the whiteglove_sealed enqueue on the first genuine WhiteGlove seal. - Storage: EvaluateOnJson on AnalyzeRules; FirstDetectedAt/LastEvaluatedAt/ IsInterim/ResolvedAt/NotifiedAt on RuleResults (nullable — legacy rows read as final). - MCP: three authoring lints (terminal-precondition trap, unknown/high-frequency on_event types, KO-on-interim), authoring guide + baked schema updated; get_session_summary marks interim and excludes resolved findings. - Web: preliminary badge on interim findings, resolved findings behind a toggle, issue counts exclude resolved (analysis section + diagnosis flow). - ANALYZE-ID-004 v1.1.0: first interim-enabled rule (on_event:hybrid_login_pending), threshold raised to 75 so only repeated overdue-login signals fire mid-run. Suites: Backend 4152, MCP 548, Web 762 + tsc + lint clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
1c5f69a9fe |
w365: CloudPc device validator — cert-CN-bound fallback for Cloud PCs (#127)
* w365: CloudPc device validator — cert-CN-bound fallback for Cloud PCs Windows 365 Cloud PCs are never Autopilot-registered, so the serial lookup (and the Corporate Identifier fallback) structurally reject them and Cloud PC first-connect enrollment could not be monitored. New stage-3 fallback in the device-validation chain: extract the Intune device id from the chain-validated MDM client certificate's Subject CN (field-verified: CN == managedDevice id == cloudPC.managedDeviceId) and require a matching cloudPC object in the claimed tenant (Graph v1.0 virtualEndpoint/cloudPCs, $filter=managedDeviceId eq). Only service-provisioned Cloud PCs have such an object — a regular enrolled non-Autopilot device still ends in the 403, and local W365 markers are never consulted for auth. - Shared: ValidatorType.CloudPc = 5, TenantConfiguration.ValidateCloudPcDevice (default false, tenant-admin writable), GraphAppPermissions.CloudPCReadAll, GraphFeatureCatalog feature W365CloudPcValidation -> CloudPC.Read.All - Backend: CloudPcDeviceValidator (AutopilotDeviceValidator sibling contract: 30/5 min cache, 2 attempts, transient->503; GUID gate before the OData filter; exact-match guard; Graph 403 = definitive "grant missing", not transient), SecurityValidator CN extraction + fallback stage + hard gate, wiring through the 5 agent endpoints, TableConfigRepository persistence - Consent script: W365CloudPcValidation in catalog/ValidateSet/help; NEW GraphFeatureCatalogSyncTests pins C# catalog <-> grant script (previously comment-only lock-step) - Bootstrap dev script v2.3-dev.3: guard 4 (uptime window) exempted while the Cloud-PC relax is active — Cloud PCs run headless for days before first connect; the fresh-profile window is the time anchor there (+ Pester tests) - Web: validation toggle + context plumbing, ValidatedBy "CloudPc" label, privacy-page validation claim generalized (Last updated bumped) - Docs: OKF concept doc backend/cloudpc-device-validation.md + index + log Verified: web tsc clean, ESLint 0 errors (99 pre-existing warnings = ratchet cap, none added). .NET build/tests and Pester run in CI — SDK download and pwsh are unavailable in this sandbox's network policy. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RqEFrcAHhWgCvgJgHGzMuv * w365: agent-side IsCloudPc flag + engine expectation "no Device-ESP phase" Companion to the CloudPc validator: the agent now classifies Windows 365 Cloud PCs locally with the bootstrap's field-verified marker AND (HKLM\SOFTWARE\Microsoft\Windows365 key + installed CloudManagedDesktopExtension service, probed via its Services registry key so net48 needs no System.ServiceProcess). SKIP-safe: any error resolves false. Never an auth input - backend admission stays exclusively the cert-CN-bound Graph lookup (ValidatedBy=CloudPc); the two signals are independent and may disagree. Two transport rails: - Session metadata: SessionRegistration.IsCloudPc -> Sessions table (sticky-true OR across re-registrations, IsSelfDeployingProfile pattern) -> SessionsIndex full mirror -> SessionSummary -> new isCloudPc search filter end to end (SearchSessions + QueryRawSessions + pagination fingerprint + all three in-memory predicates; MCP search_sessions / query_raw_sessions params + summary projection; web type, Cloud PC pill, SessionInfoCard enrollment label, CSV export appended column). - Decision engine: isCloudPc rides the EnrollmentFactsObserved payload, recorded set-once for BOTH values in EnrollmentScenarioObservations.CloudPc and annotates the scenario profile reason cloud_pc_first_connect:no_device_esp_expected - the engine-recorded expectation that Device-ESP already ran headless at provisioning and the session starts at Account Setup. Deliberately reason+observation only: no completion arm depends on DeviceSetup, Mode stays AccountSetup/IME- classified (Classic), EspConfig keeps configured-FirstSync semantics. Positive marker surfaces as cloud_pc_marker in the audit-trail census. Updater idempotency: the cloudPcAnnotates gate keeps the generic v1 reason branch from re-claiming the slot on repeat posts (would bump EvidenceOrdinal every post). Additive-nullable observation - no state schema bump (v4 convention), serializer roundtrip pinned. Tests: CloudPcDetector resolver (marker AND, throwing probe), registration body, facts-signal payload shape, updater annotation/idempotency/ self-deploying interplay, reducer observation both values + census fold, serializer roundtrip, index projection superset, sticky-true both directions. Suites: DecisionCore 500, Functions 3941, Agent V2 2154, vitest 632 - all green; tsc web+MCP clean; ESLint 0 errors (cap 99). Docs: cloudpc-device-validation.md follow-up section replaced with the shipped mechanism + log entry. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
395fdd49df |
Transactional tenant-config writes via MCP + ConfigurationBackups safety net (#120)
* feat(backend): pre-write config backup net - every tenant/admin config save snapshots first New ConfigurationBackups table (PK=tenantId or GlobalConfig, RK=reverse-ticks+guid, newest-first) holds the raw stored row as EntityJson before any overwriting save of TenantConfiguration or AdminConfiguration. Hook lives in TableConfigRepository, the single choke point all writers funnel through - fail-soft (a backup hiccup never breaks a save), skips row creations and noise-only changes (LastAuthClientId flips), prunes to the newest 2 per partition. Source/reason tagging via a new SaveTenantConfigurationAsync overload (not optional params - Moq expression trees, CS0854); portal-put/plan/auth/app-homing/offboard call sites tagged. Offboarding: backup partition wiped in the bulk phase (TenantPartitionTables) and by the marker-cleanup tombstone sweep; the GlobalConfig partition never matches a tenant GUID and survives. Table registered in TableNames.All + CriticalBackupTables.All. Groundwork for the transactional MCP config-write flow (patch/revert endpoints follow). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(backend): transactional tenant-config field patch + revert with verified writes New GA-only surface (policy catalog: GlobalAdminOnly + RouteParam): - PATCH config/{tenantId}/fields - field-level patch through TenantConfigPatchService: fresh ETag read -> field gate (deny-list incl. plan/trial, HomedAppClientId, auth provenance; unknown fields 400 with the name) -> patch onto a clone -> shared validation -> fail-CLOSED backup -> CAS replace -> fresh re-read -> verify that EXACTLY the intended fields changed -> automatic ETag-conditional rollback on drift (catches Store/Map serialization asymmetry). - POST config/{tenantId}/revert - restore from a ConfigurationBackups snapshot (latest by default); snapshots current state first (revert is revertible), preserves plan/homing/provenance fields unless includeProtectedFields. - GET config/{tenantId}/backups - metadata + masked diff only, never EntityJson. Supporting changes: IConfigRepository gains GetTenantConfigurationWithEtagAsync + TryReplaceTenantConfigurationAsync (412 -> false, no implicit backup hook); validators moved from the PUT into shared TenantConfigValidation (PUT re-pointed, forwarding shims keep test call sites); GET config/{tenantId} honors ?view=redacted so a GA's MCP session never pulls clear-text secrets into model context. Caller-tier deny-lists (TenantAdmin/DelegatedAdmin) ship now for the phase-2 flip. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(mcp): tenant-config write tools - update/revert with snapshot-backed verified writes First mutating surface on the MCP server, strictGa (real Global Admin) only: - get_tenant_config: full config read, ALWAYS ?view=redacted (secrets never reach model context; pinned by config-write-tools tests) - update_tenant_config: field-level patch via PATCH config/{tenantId}/fields; required reason; response = applied fields + masked diff + backupId - list_tenant_config_backups: snapshot metadata + masked diff only - revert_tenant_config: restore latest/named snapshot; protected fields stay current unless includeProtectedFields; revert is itself revertible New MUTATING annotation + TenantGuidSchema (path-traversal guard) in shared.ts; role-aware instructions headline (non-GA keeps the exact READ-ONLY wording); catalog snapshot test extends GA_FULL + new CONFIG_WRITE_GA_STRICT difference list (Global Reader/tenant/delegated see none of the four). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(backend): de-flake session-deletion heartbeat test - condition-gated handler instead of wall-clock window The heartbeat test held the handler for a FIXED 350ms and expected the 50ms heartbeat timer to tick >=3 times inside that window. On loaded CI runners the heartbeat Task.Delay stretches disproportionately to the handler delay, the handler completes with <3 heartbeats fired, and the awaited condition can never become true afterwards (CI failure on PR #120). The handler now holds the message in-flight UNTIL the test observed 3 heartbeats (with ct + own 15s deadline as termination guards), so the assertion is reached deterministically regardless of scheduler jitter. Verified: 12/12 local runs green, full suite green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
0e2d620ad4 |
feat(auth): self-service app-homing flip via consent flow + GA lever
Re-consent, verify and flip collapse into the existing consent flow: the
consent URL funnels legacy-homed tenants to the primary app registration,
and consent-status/access-check verify admin consent via an app-only token
probe under explicit primary credentials, then auto-flip HomedAppClientId.
- AdminConfiguration.SelfServiceAppHomingEnabled: kill switch (default off,
cache-bypassing fail-closed read); gates funnel + auto-flip + tenant-admin
manual flip. GA path deliberately survives the kill switch.
- POST config/{tenantId}/app-homing (TenantAdminOrGA): tenant admins flip to
primary only, probe-gated; GA gets force + revert. Pure decision helper
EvaluateManualFlip shared by function and tests.
- GraphTokenService.GetAccessTokenForAppAsync: probe seam with explicit app
credentials, cache-safe via app-scoped token keys.
- EntraAppRolesEnabled does not block a flip; it emits a Warning ops event
(AppHomingFlippedWithEntraRoles) reminding to re-assign enterprise-app roles.
- Web: homing badge + "App Registration Homing" panel with confirm dialog in
GA tenant management (typed homedAppClientId hardens the PUT round-trip),
funnel/success banners in tenant settings with switchAuthApp re-login,
admin-config kill-switch round-trip, ops event types dual-registered.
- Browser telemetry (App Insights): pre-init event buffer so module-init auth
events survive; strategic events across the whole flow (app_homing_flipped,
consent_flow_*, consent_verify_failed, auth_app_fallback_*, auth_app_switched,
auth_app_homing_learned, app_homing_manual_flip*).
Backend must deploy before web (additive JSON fields); everything is inert
while EntraId:LegacyClientId is unset. Enable the kill switch in the admin
portal after the dual app-reg config swap.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
f795fbcb4e |
feat(auth): dual app-reg parallel operation - legacy gktatooine app alongside new C4A8 app
True parallel operation, no forced re-consent: existing tenants keep signing in via the legacy app (1a400946), new tenants land on the C4A8 app (886ab5e2). Model "default legacy, funnel primary, learn in the background": - Web: am_auth_app flag -> NEXT_PUBLIC_ENTRA_DEFAULT_APP -> primary; get-started CTA hands over with ?authapp=primary; auth/me homedApp updates the flag so an operator tenant flip takes effect on each user's NEXT login; AADSTS90094/65001 one-shot cross-app retry, AADSTS65004 -> failed screen with previous-app link; am_login_attempt pins redirect returns to the app that started them. - Backend: TenantConfiguration.HomedAppClientId (null = legacy, GA-only flip via PUT config) + LastAuthClientId/Since observability; EntraAppRegistry resolves the acting app per tenant (no-op until the legacy config pair is set); Graph client-credentials, admin-consent URLs and the surfaced client id follow the homed app (app-scoped cache keys); JwtBearer [Authorize] pipeline now shares the middleware audience builders (was single-audience - would 401 legacy tokens after the swap); middleware trusts EntraId:LegacyClientId implicitly. - well-known association file now lists both app ids (publisher domain); deploy-web passes NEXT_PUBLIC_ENTRA_LEGACY_CLIENT_ID / _DEFAULT_APP; mcp-server.bicep default client id -> C4A8 app. Backend xUnit 3730/3730, vitest 625/625 (13 new), tsc/ESLint clean, next build ok. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
6bd5728652 |
feat(platform): public availability — tenant activation replaces Private Preview, optional auto-approve
Backend: shared TenantApprovalService (whitelist add + auto-promote shape-guard + welcome email), AdminConfiguration.AutoApproveNewTenants (default off, uncached fail-closed accessor), tenant-auto-approve queue (1-min visibility delay, worker drops on flag-off/approved/suspended/missing config, approves as "System (auto-approve)" + TenantAutoApproved ops event), 30s negative whitelist cache, gate 2 error PendingActivation with mode-neutral message, signup strings and welcome email without preview wording. Web: /preview → /activation (SWA 301, robots, AuthGate), AuthContext accepts both error codes and clears pending state on successful auth/me, activation page polls every 30s and auto-redirects into the portal, dashboard banner reframed to "Actively developed" + feedback call, terms/privacy/get-started updated, feature badges → Beta, admin-config web chain + auto-approve checkbox, OPS_EVENT_TYPES. Deliberately unchanged: PreviewWhitelist/PreviewConfig tables, internal preview/* API routes, preview_signup notification type, Microsoft DevPrep preview mentions. Verified: solution build 0 errors, xUnit 3701/3701, tsc 0 errors, vitest 612/612, ESLint ratchet holds (0/115), next build exports /activation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
48308df4ff |
feat: opt-in preservation of session diagnostics archive with session reports
Session reports persist while sessions and their diagnostics blobs are
removed by user deletion or retention. A new checkbox in the Report
Session modal (shown only when a diagnostics ZIP was uploaded, default
checked) lets the submitter include the archive: the backend streams a
server-side copy into the durable session-reports container.
- SessionReportDiagnosticsArchiveCopier: fail-soft copy (never blocks the
report), reuses the Hosted/CustomerSas read paths, re-verifies the
hosted tenant prefix via ClassifyBlobName, honours the admin size cap
and download timeout; outcome persisted as DiagnosticsCopyStatus
- Report row/API gain DiagnosticsBlobName + DiagnosticsCopyStatus
(written only when requested, omitted for legacy rows)
- report-metadata.json records includedDiagnostics {requested, blobName, status}
- Admin reports UI: second Download Diagnostics button + failure status line
- Privacy page: explicit exception for the opt-in report copy (Last updated bump)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
e0ad51b965 |
refactor(urls): one guard-enforced URL registry per component, agent downloads move to the alias
Every well-known host -- own subdomains, both blob accounts, the API origin, Graph, Entra login -- now lives in exactly one registry per component: Constants.cs (*BaseUrl) for C#, utils/config.ts for the web app, src/config.ts for the MCP server. lib/hostRouting.ts derives its hostnames from the registry and middleware.ts imports from it, retiring what turned out to be a fourth independent copy of the www/portal host pair. Three new guard tests (backend xUnit, web vitest, MCP vitest) scan non-test source and fail on any enforced host outside its registry. This is the lesson of the EU cutover: the registry existed then too, but LatestVersionsService (feeds the portal's version badge) and ValidateBootstrapCodeFunction carried their own literal copies of the legacy blob host and were missed -- kept working only by the fail-soft legacy mirror in build-agent.yml, whose silent failure would have pinned badge and self-update to stale bits with a green release. Both guards proved themselves on their first run, flagging the middleware host copy and a literal in an MCP tool description. Test files stay exempt on purpose: literals there are independent oracles, and ValidateBootstrapCodeResponseShapeTests now pins the full alias URL against the Constants-built one so registry changes must be acknowledged consciously. The two legacy readers plus the agent SelfUpdater now use Constants.AgentDownloadBaseUrl (download.autopilotmonitor.com/agent); the alias was verified live for version.json, version-v2.json and both ZIPs before the switch. The portal's bootstrap allow-list accepts the alias AND the legacy host (AGENT_DOWNLOAD_HOSTNAMES) for already-deployed customer scripts, which imposes a deploy order: web before or with backend. The legacy keepalive probes and the release mirror stay untouched -- they exist for exactly this transition. Also: the bootstrap short-link now targets www directly instead of riding the registrar's apex 301, and the welcome-mail docs button points at docs. instead of bouncing through www./docs. Suites: backend 3194, agent core 1925, DecisionCore 474, web 582, MCP 449 -- all green; next build verified (it, not tsc, proves the next.config.ts registry import). Docs: docs/url-registry.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
da958b5398 |
fix(backend): business timestamps survive storage migrations (OccurredUtc + RowKey decode)
The 2026-07-18 storage migration reset every row's system Timestamp, exposing that AuditLogs, OpsEvents and Events displayed, filtered and retention-swept on it (custom "Timestamp" writes were silent no-ops — reserved system property). All migrated rows read as 2026-07-18. - New Helpers/BusinessTimestamp: RowKey decoders (audit !revtick_guid, ops bare revtick, event yyyyMMddHHmmssfff_seq), RowKey-range OData clause builders (tick-exact, with the mandatory RowKey lt '"' guard so no lower bound can ever match legacy bare-GUID audit rows), and dual DateTimeOffset/DateTime column reads. - Writes in all three tables now persist OccurredUtc (custom column, copied verbatim by future migrations); no-op ["Timestamp"] removed. - Read mappers resolve OccurredUtc -> RowKey decode -> system Timestamp; GetEarliestSessionEventTimestampAsync uses the same resolution so StartedAt/Duration reconciliation can no longer absorb the migration date on migrated sessions. - Date-window filters and retention moved off the system Timestamp onto RowKey ranges; ops retention loses its full-table scan and the post-migration mass-delete cliff; audit retention runs a temporary second pass for legacy GUID rows (TODO: remove after 2027-03-01). - New GlobalAdmin backfill endpoint POST /api/maintenance/ backfill-occurred-utc (dry-run by default, batched, idempotent MERGE of only OccurredUtc, table=audit|ops) — Events rows are deliberately never mutated; their read path decodes the RowKey instead. - Tests: decoders, clause boundary proofs incl. GUID-safety pins, mapper precedence, backfill row decisions (Functions.Tests 3078). - Docs: docs/backend/business-timestamps.md (system Timestamp is never authoritative; migration checklist rule). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
d02fa454a7 |
fix(status): honest verdicts across all misclassification audit findings
Misclassification audit 2026-07-16 (1000 non-succeeded sessions, 22 deep
dives) surfaced six systematic misdeclaration sources. This implements
all of them:
1. DecisionCore: HelloNeverObservedPromote in the esp-exit advisory
handler — when the Hello policy read never committed and no wizard
was ever seen, hello_resolution is structurally unresolvable; promote
to AwaitingHello + hello_safety instead of hard-failing a provably
finished enrollment (sessions 2dac8298/7635ab18). Placed after the
enforcement-progress re-arm so active installs still win; advisory-
anchor variant stays exempt.
2. Ingest: both max-lifetime shapes (enrollment_failed/agent_timeout and
agent_shutting_down/max_lifetime) now route through the
EnrollmentTimeoutClassifier instead of hard Failed (tenant a53e67ec:
fully provisioned WG Part 2, user never logged in => AwaitingUser).
agent_timeout no longer counts as HasExplicitFailure; failure
webhooks fire only on actual failure verdicts.
3. Maintenance sweep: dropped the too-broad "IsPreProvisioned ne true"
exclusion (sealed devices are protected by their Pending status);
resumed WG Part-2 runs sweep normally with the window anchored to
ResumedAt. New stage-2 silence guard also fixes a latent bug where an
actively-reporting >5h enrollment could be terminalized mid-run.
4. Registration supersede: a fresh registration resolves older
non-terminal sessions of the same device (serial, placeholder-guarded)
as Incomplete("Superseded by ...") with live SignalR updates — closes
the forever-Pending orphan gap when WG Part 2 runs under a new id.
5. Admin retro-reconcile: POST /api/maintenance/reclassify-legacy
(GlobalAdminOnly, DRY-RUN BY DEFAULT) re-runs the classifier over the
pre-classifier "Session timed out" Failed backlog (~495 rows, sample
projected ~87% misdeclared) and resolves Pending orphans. Requires the
new opt-in allowTerminalReclassification override in the status
writer (admin-marked rows stay untouchable).
6. Polish: "0 of N" AccountSetup rollups keep their total (no more
fabricated "0/0" reasons, "never observed" wording when absent);
late-telemetry reconcile heals Incomplete/AwaitingUser to Succeeded
when straggler evidence proves completion; SLA page knows
AwaitingUser/Incomplete ordinals; script_failed on Success/exit-0 now
names the stderr rule in its message.
Tests: backend 2965, DecisionCore 474 (+6 new), agent 1777, web 557 —
all green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
b0105b123a |
feat(maintenance): session-deletion run budget, live watchdog counters, manual trigger
- Watchdog fix: FanoutResult is caller-allocated and mutated live per tenant/session, so 30/60min LongRunning warnings report real progress instead of the always-zero counters (today's warning showed 0/0 while 5 tenants / 54 sessions were already done). - 50min run budget: the retention fanout stops cleanly at the deadline (tenant boundary + per-session check) and reports AbortedByBudget — no more risk of the 60min Flex functionTimeout hard-aborting mid-run. New Warning OpsEvent SessionDeletionMaintenanceBudgetExceeded; Completed gains abortedByBudget. - Blob-lease serialization: dedicated sentinel _lock/session-deletion-maintenance.lock (deletion-manifests container) via SessionDeletionMaintenanceLockStore + reused MaintenanceLeaseHolder; contended runs emit SessionDeletionMaintenanceSkippedLocked and return. - Manual trigger: POST /api/global/session-deletions/maintenance/trigger (GlobalAdminOnly, policy catalog) -> fail-hard queue producer -> SessionDeletionMaintenanceQueueWorker runs the same RunCoreAsync body with triggeredBy = requesting admin; lease probe returns 409 while a run is active. New Started OpsEvent carries triggeredBy. - Web: MaintenanceStatusBanner on the Session Cleanup page derives active / budget-exceeded / failed / idle from the lifecycle ops events and offers a Run-now button (202/409 handling). New event types dual-registered in OPS_EVENT_TYPES. - Tests: 7 new (budget abort immediate/mid-tenant, live counter increments, watchdog live-progress regression, BudgetExceeded + Completed flag, SkippedLocked, Started triggeredBy); suites green (backend 2935, web 557, tsc clean). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
d8e8cd15fd |
feat(entitlements): Community/Enterprise editions with self-service trial
Fail-closed entitlement layer (FeatureEntitlementCatalog + TenantEntitlementService): edition resolves at read time (PlanTier == 'enterprise' OR active trial; legacy free/pro => Community, no migration).
- Plan/trial endpoints: PATCH config/{tid}/plan (GA: tier + grant/extend/end trial, audited, cache-safe) and POST config/{tid}/trial (tenant-admin self-service 30d, once per tenant, 409 after). Generic PUT preserves plan/trial fields for all callers. Fixes SetPlanTier writing past the config cache without audit/UpdatedBy.
- Rate limits: Enterprise floor 150/min for BOTH user (portal/JWT) and device (agent/cert) paths - floor raises the global default, an explicit per-tenant override still wins; MSP users follow their home tenant.
- Retention cap: Community 90d / Enterprise 365d, clamped in the retention fanout (warn log) + non-GA write validation 7..cap (only on changed values; 0=infinite stays GA-only).
- Delegated admin (MSP): Enterprise gate on the delegated admin's HOME tenant (JWT tid, fail-closed on unknown) - managed target tenants may be any edition. Grant-time target check removed; inert grants are suppressed at resolve time (single choke point in GetScopeAsync, applied after the UPN-keyed cache).
- MCP quota: McpQuotaService + McpQuotaEnforcementMiddleware (X-Client-Source: mcp only, GA exempt) enforce daily/monthly budgets - per-user plan override > tenant edition plan; limits from SectionUsagePlans with catalog fallbacks (community 100/3000, enterprise 1000/20000); 429 with X-MCP-Quota-* headers + Retry-After; fail-open on counter reads, fail-closed on plan resolution. /api/metrics/mcp-usage/me reports effective plan + counters.
- Trial visibility: TrialExpirySweepFunction (daily 03:30 UTC, informational) emits TenantTrialExpired (Warning, 24h look-back) + TenantTrialExpiring (Info, <=3d heads-up); dual-registered in web OPS_EVENT_TYPES.
- Web: EditionBadge + one-time trial CTA in settings header, Plan & Trial card in the GA tenant edit modal (own PATCH save), retention cap hints from entitlements, pure lib/edition.ts (vitest), feature-flags surface edition/trial/entitlements; MCP TENANT_SAFE_FIELDS + trial fields.
Verified: solution build clean; backend 2653/2653 tests; web tsc + 516/516 vitest; MCP server tsc + tests green.
|
||
|
Oliver Kieselbach
|
34582c0f0e |
feat(kill-switch): deliver kill on config channel + KillSignalDelivered ops event
Kill/Block rules were only served on the telemetry response, which cannot reach an agent whose upload loop is paused (indefinite block pins the drain until DateTime.MaxValue), whose spool is empty, or whose binary is kill-blind. Config is fetched at every process start (boot via Scheduled Task), making it the reliable control channel - no heartbeat added by design (48h emergency break + 6h max lifetime bound the remaining cases). Backend (effective on deploy): - KillSwitchEvaluator: shared device+version check for both channels; ingest wire response and check order unchanged (device first, short-circuits) - Every served Kill emits ops event KillSignalDelivered (Security/Warning), throttled 24h per tenant+serial+pattern; Blocks stay event-free - GET agent/config (incl. bootstrap wrapper) now returns DeviceBlocked/ DeviceKillSignal/UnblockAt alongside the full config; old binaries ignore the new fields - Web: KillSignalDelivered dual-registered in OPS_EVENT_TYPES Agent V2 (effective with next agent release): - CheckConfigKillSignal guard after the Phase-4 fetch: forced self-destruct (overrides SelfDestructOnComplete), enrollment-complete marker, session cleared, exit before mTLS/registration - Kill honoured ONLY from a live fetch: config disk cache strips the flags on write and read (UnrestrictedMode pattern), guard re-checks LastFetchOutcome Tests: 9 KillSwitchEvaluatorTests + 7 ProgramConfigKillSignalTests; Functions 2557/2557, Agent V2 1578/1578, web tsc clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
9f7b070e80 |
chore: remove legacy V1 agent and its backend ingest path
The V1 agent line is retired: no new V1 sessions since 2026-05-09, only orphaned straggler traffic remained on /api/agent/ingest (last seen 2026-06-29, events for long-cleaned-up sessions). Bootstrap/Intune/CI have deployed V2-only since the v2 cutover. Agent: - Delete AutopilotMonitor.Agent, .Agent.Core, .Agent.Core.Tests (projects + sln entries). V2 trio, SummaryDialog, DecisionCore.Tests and V2Isolation.targets stay. Backend: - Relocate the shared statics/DTOs V2 reused from the V1 partials (StampServerFields, SanitizeEventTimestamps, IsMaxLifetimeAgentShutdown, RecalculateAppDurations, EventClassification, AppInstallAggregationState) into EventIngestProcessor; IngestEventsOutput moves next to IngestTelemetryFunction (same namespace, zero ripple). - Delete IngestEventsFunction (route agent/ingest), BootstrapIngestEvents (route bootstrap/ingest), NdjsonParser and their DI registration. EventDataNormalizer stays (used by TelemetryPayloadParser). - Drop both routes from EndpointAccessPolicyCatalog and RequestTelemetryMiddleware.TenantHeaderTrustedPaths. - Retire the V1 update-hash oracle: LatestAgentV1*/LatestBootstrapV1* fields + GetAgentLine case 1 (default arm returns empty hashes, so stragglers gracefully skip the integrity check; ParseAgentMajor keeps its default of 1 on purpose — defaulting to 2 would feed V2 hashes to V1 binaries). TableConfigRepository Store+Map updated in lockstep. - Remove ApiEndpoints.IngestEvents/.BootstrapIngestEvents constants. Tests: NdjsonParserTests deleted; timestamp/critical-path/max-lifetime tests repointed to EventIngestProcessor; the two NDJSON pipeline tests removed (coverage retained by direct-list siblings); catalog/no-store InlineData rows updated. Docs: architecture.md (endpoint tables, ingest flow diagram, solution layout, test matrix), web SectionAgentInternals endpoint tables, build.ps1 help text. Azure: no config change needed — /api/agent is not in the client-cert exclusion paths (agent/ingest was always cert-enforced) and the /api/bootstrap prefix exclusion remains for the surviving bootstrap routes (verified via ARM on autopilotmonitor-api). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
05a591e1d4 |
feat(backend): delegated-admin (MSP mode) data model + scope service [Phase 0]
Foundation for "MSP mode" — a scoped-global authorization tier between a single-tenant member and a platform GlobalAdmin. A delegated admin may access a SUBSET of tenants (its assignment rows), resolved UPN-based and tid-agnostic, exactly like GlobalAdmins. Phase 0 is purely additive: the service is registered but no auth path consults it yet (the cross-tenant gate change lands in Phase 1), so behavior is unchanged. - Constants: DelegatedRoles/DelegatedStatus/DelegatedSource; new DelegatedAdmins table added to TableNames.All (auto-create), CriticalBackupTables.All and AuthTablesFullRestoreForbidden (identity table — full-table restore blocked). - IAdminRepository + TableAdminRepository: DelegatedAdminEntry DTO + CRUD (PK=UPN, RK=TenantId). Hot-path GetDelegatedTenantsAsync (partition scan) and reverse GetDelegatedAssigneesAsync. Queries use the typed predicate overload (no string-interpolated OData filter — avoids filter injection). - DelegatedAdminService: resolves DelegatedScope (Covers/RoleFor/CanWrite) with a 5-min cache + write-through invalidation. Fail-closed: only Active+enabled rows with a recognized role confer scope; empty role -> DelegatedReader; unknown -> dropped. - 16 unit tests (fake repo); full Functions suite 2434/2434 green. Backup catalog-count assertion bumped 15 -> 16. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
7c1f14c66b |
feat(backend+web): GA "Active Users" live presence dashboard + UserActivity retention
Global-Admin cross-tenant view of web users active "now" (made an authenticated request within a window, default 5min), backed by a new self-maintaining UserPresence table. Not App Insights (sampling) / not SignalR. Backend: - UserPresence table (one row/user, PK=tenantId, RK=SHA-256(lowercase UPN)); upserted fire-and-forget by UserPresenceMiddleware (after PolicyEnforcement, 60s per-process throttle, UPN-gated so agent cert calls are skipped). - GET /api/global/presence (GlobalReadOrAdmin) via GetActiveUsersFunction; read path propagates storage errors (5xx) instead of masking them as "0 active". - 1-day presence retention + 90-day UserActivity retention in maintenance; PlatformStats.TotalUsers clamped monotonic so the public "Users Seen" figure cannot regress after a prune. Web: - /admin/presence page (live counter + table), nav entry under GA > Metrics. - "Total Users" relabeled "Unique Users (90d)" to match the windowed metric. Backend+web only, no ConfigVersion bump, inert until redeploy. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
12d67380e9 |
feat(mcp+backend): session_debug — client-side diag-ZIP analysis via HMAC ticket
Lets tenant-admins and GAs run end-to-end session debugging over MCP: backend
telemetry plus the agent diagnostics ZIP, downloaded and analyzed locally by the
AI client. The backend never unzips/parses the archive (no parse CPU); it only
mints a short-lived signed download ticket and proxy-streams bytes.
Backend (no ConfigVersion bump, no agent change):
- DiagnosticsDownloadTicket: HMAC codec (reuses PaginationTokenSigningKey,
purpose-tagged diag-dl-v1, 10-min TTL) so a no-JWT client can download.
- DiagnosticsBlobStreamer: shared proxy-stream engine; DiagnosticsDownloadFunction
(web/JWT) refactored onto it. ClassifyBlobName/BlobDestination stay put (xUnit).
- POST diagnostics/download-ticket (MemberRead, QueryParam, GA-bypass) +
GET diagnostics/download?t= (anonymous; tenantId/blobName read only from the
signed ticket -> no retargeting). Both added to NoStoreCacheMiddleware.
MCP:
- get_session_diagnostics tool (all roles) -> absolute downloadUrl + static zipMap
+ instructions; unwraps the {success,session} envelope.
- diag_zip_layout catalog (get_resource + protocol resource).
- debug-session prompt (tenant + GA variants).
- Tool descriptions flag the client requirement (needs local file/shell tools;
pure chat clients cannot unzip).
Tests: backend 2380/2380 (ticket codec, NoStore, classify, catalog completeness);
MCP 261 passed (incl. envelope-unwrap regression). Inert until redeploy.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
bc1f11d2db |
revert(backend): undo L4 telemetry sampling — ineffective in isolated worker
Deploy-verified on |
||
|
Oliver Kieselbach
|
f4d51ea499 |
perf(backend): cut App Insights cost + ingest hot-path reads (review follow-through)
Telemetry cost (L4): worker-side adaptive sampling now excludes Request;Event so the enriched RequestTelemetryMiddleware copy is never thinned; host.json Host.Results=Error drops the duplicate unenriched host request item. One complete request row per call instead of two (one unsampled+unenriched, one sampled+enriched). Ingest hot path (H2/H3): BlockedDeviceService caches negative answers (30s revalidation, symmetric with positive entries) so healthy devices no longer pay a BlockedDevices point-read per request; SessionDeletionGuard returns its already-loaded row and IncrementSessionEventCountAsync returns the post-merge snapshot, collapsing ~4 Sessions reads/batch to ~2. Queue-worker correctness (O1/O2): new GetSessionEventsStrictAsync propagates storage failures on the analyze + vulnerability-correlate paths so a transient read fault retries/poisons instead of silently deleting the message after "analyzing" an empty stream. Fail-soft variant kept for UI/metrics surfaces. Review fixes: parse prefetched Status as string (the actual wire format, was int -> optimization never fired); session-aware block over a cached negative entry keeps its session scope instead of widening to whole-device. 2274/2274 backend tests green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
c0552b7f9d | reduced AppInsights dependency telemetry | ||
|
Oliver Kieselbach
|
467ee07810 |
refactor(queue): extract QueueClientFactory + QueuePollingWorker base (audit D2)
Eliminates the copy-paste queue scaffolding flagged as finding D2: identical Managed-Identity/connection-string QueueClient bootstrap hand-rolled across 13 sites, plus a poll-loop/poison/safe-delete skeleton duplicated in 5 workers. - QueueClientFactory: resolves the MI/connstring model once; consumed by all 5 producers, the offboarding enqueuer, the backup producer and the poison probe. - QueuePollingWorkerBase: shared bootstrap + poll loop + poison-move + safe-delete + queue-create (ExecuteAsync/ProcessOneAsync sealed to prevent drift). - QueuePollingWorker<TEnvelope>: deserialize -> validate -> handler flow with an opt-in in-flight heartbeat. - 3 vanilla workers (Index/Analyze/Vuln) collapse to ctor + HandleAsync. - SessionDeletion/Offboarding use BatchSize/UseHeartbeat/ShouldPauseAsync + Before/AfterPoisonMoveAsync hooks (kill-switch, CAS state transition, OpsEvent enrichment, Failed-state-before-poison all preserved bit-for-bit). - Backup derives from the non-generic base and overrides ProcessOneAsync (its lease + CAS state machine is intentionally bespoke). Producers get the factory only -- no shared base, their business logic is genuinely divergent. Test seams unchanged; the static SessionDeletionWorker. MaxDequeueCount reference moves to QueuePollingWorkerBase.DefaultMaxDequeueCount. 2168/2168 backend tests green, 0 warnings. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Oliver Kieselbach
|
9e689dc9d8 |
feat(backend/backup): PR2 — single-row restore + admin UI
POST /api/global/backups/{backupId}/restore-row (GA-only) with preview|commit
modes. Preview scans the ETag-pinned NDJSON blob for the requested (pk, rk),
diffs against the live row, and returns rowSha256 + currentETag for echo.
Commit re-validates everything under a short maintenance lease and writes via
UpdateEntity(IfMatch=etag, Replace) or AddEntity; 412/404/409 → CurrentRowChanged.
Backend:
- RestoreTablePreflightValidator (no I/O) + BackupRestoreInputValidator (manifest
parse, entry check, BlobName canonical, ETag-pin + SHA in one stream read)
- CriticalTableRestoreService with PreviewRowAsync (no lease) + CommitRowAsync
(MaintenanceLeaseHolder, 45 s renewal, deterministic finally release)
- BlobBackupStore.OpenNdjsonReadAsync — GetProperties + OpenRead(IfMatch=etag)
- BackupTerminalException carries a stable Code for HTTP envelope mapping
- OpsEventService.RecordBackupRowRestoredAsync (Warning; payload is
container + blobName only, never a SAS URL)
- 27 new xUnit tests; full backend suite 1979/1979 green
Web:
- /admin/backups list page with newest-first list + "Run backup now" trigger
and async job-status poller
- /admin/backups/{backupId} detail page with per-table breakdown + single-row
restore trigger
- RestoreRowDiffModal with Added/Removed/Changed/Unchanged pills and
auth-table security banner (GlobalAdmins/TenantAdmins/McpUsers)
- api.backups namespace + matching DTO types
- globalNavConfig: ga-backups under Ops
- BackupRowRestored registered in OPS_EVENT_TYPES (dual-register policy)
- 6 new vitest cases; full web suite 390/390 green
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Oliver Kieselbach
|
d4034842ee |
feat(backend/backup): PR1 — critical-table backup feature (daily timer + manual GA trigger)
Adds the full backup pipeline that snapshots 15 critical tables (Admin/Tenant config, rules, identities, audit, preview) into a logically-isolated blob container, daily at 04:00 UTC and on operator demand. Restore endpoints follow in PR2/PR3. Architecture: * Daily TimerTrigger + manual POST /api/global/backups/trigger (GA-only, fail-hard producer). HTTP returns 202 + jobId on enqueue, 500 if the queue write fails (and the BackupJobs row is rolled to Failed). * BackgroundService queue worker (not QueueTrigger — matches the Analyze/ IndexReconcile pattern that avoids the Functions-host connection-binding app-setting). Worker holds the maintenance lease for the whole run; service is intentionally lease-unaware so it stays unit-testable. * BlobBackupStore wraps the new 'critical-table-backups' container with a shared maintenance-lease sentinel at '_lock/maintenance.lock' that serialises every backup or restore op across worker/timer/single-row paths. * CriticalTableBackupService streams each table through CryptoStream(SHA256) into per-table NDJSON blobs, then writes manifest.json LAST as the durability anchor. Per-table failures degrade Outcome to Partial without aborting; per-run budget (50 min) → Skipped status. * BackupJobsRepository persists the six-state machine (Queued/Running/Completed/Failed/Skipped/BlockedTerminal) with ETag-CAS updates. Enum→string mapping rejects unknown values on read so refactors surface loudly instead of silently shifting state. * BackupJobWatchdog (30 min timer) heals stuck rows via a probe lease on the shared sentinel: Queued > 60 min → Failed, Running with stale heartbeat AND free lease → Failed (lease-busy means a live worker — leave alone). * OpsEvents: distinct CriticalTableBackupPartial (Warning) vs Failed (Error) so operators can wire softer Telegram rules for partial captures. * MaintenanceService.MonitoredPoisonQueues + OPS_EVENT_TYPES catalog registered (dual-register memory). Test coverage (15 new tests, 1952/1952 backend xUnit green): * BackupManifestJsonTests: enum-as-string (allowIntegerValues=false rejects numeric values), DictionaryKeyPolicy unset (Azure column names preserved case-faithfully), manifest field round-trip. * BackupJobsRepositoryMappingTests: full entity↔domain round-trip, unknown Kind/State/BackupOutcome rejected, legacy BackupOutcome="Failed" rejected. * CriticalTableBackupServiceTests: happy/partial/empty/budget-exceeded paths exercise the per-table loop with in-memory table + blob fakes. Setup: scripts/Backup/setup-backup-container.ps1 — idempotent az-CLI bootstrap for container + lifecycle rule (90 d delete on prefix). Soft-Delete and Versioning are NOT applied automatically (storage-account-wide blast radius — operator must validate per separate doc note). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |