gsadmin/AutopilotMonitor
1
0
Fork 0
mirror of https://github.com/okieselbach/AutopilotMonitor.git synced 2026-10-24 12:42:38 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
7323ca72b29460f73613d1da2d4cc2289e5d7ac8
AutopilotMonitor/docs/backend/ops-notification-channels.md
T
Oliver Kieselbach d763a885e2 Ops alert channels, ops-events server-side filters, and generated MCP tool vocabularies (#140) 
* feat(backend,web): Telegram als Channel-Provider + ein einziger Dispatch-Pfad

Notifications hatten zwei Sendewege: WebhookNotificationService.SendToChannelsAsync fuer
tenant-konfigurierte Channels, und TelegramNotificationService direkt aus dem Ops-Alert-Pfad.
Ein neuer Provider haette beide anfassen muessen.

Neu: NotificationChannelDispatcher ist DIE Channel-Sende-API und routet pro Provider —
WebhookProviderType.Telegram zum Bot, alles andere als gerenderten Webhook-POST.
SendToChannelsAsync wurde aus WebhookNotificationService entfernt statt dupliziert, damit es
keinen zweiten Pfad gibt; alle sieben Call-Sites ziehen um (EventIngestProcessor, RegisterSession,
ReportDistress, SlaBreachEvaluation, AnalyzeOnEnrollmentEnd, TestWebhookNotification).
SendOpsAlertAsync/SendNotificationAsync sind virtual (Test-Seam), TelegramNotificationService
bekommt SendAlertWithResultAsync fuer die Test-Endpoints und RenderAlertText als testbaren Kern.

Telegram (=40) ist der Sonderfall: kein Webhook. Der Channel-Url traegt die CHAT-ID, der Bot-Token
gehoert der Plattform (PreviewConfig). Ein Tenant-Admin wuerde durch UNSEREN Bot senden — deshalb
ist das Gate eine Server-Regel, keine versteckte Dropdown-Option:
ValidateTelegramChannelGate sitzt in TenantConfigValidation.ValidateModel und deckt damit BEIDE
Schreibpfade ab (Full-Model-PUT und den MCP-Field-Patch ueber TenantConfigPatchService) — in der
Update-Function allein waere der Patch-Pfad daran vorbeigelaufen. Ein unveraenderter, GA-angelegter
Channel passiert das Gate, damit ein Tenant-Admin unrelated Config weiter speichern kann.
Telegram-Ziele ueberspringen SsrfGuard.ValidateWebhookUrlFormat (eine Chat-ID ist keine URL und
wuerde immer failen) und werden stattdessen als numerische Id oder @username geprueft.

Mitgeliefert, weil der Dispatcher und die Ops-Seite sie brauchen: NotificationChannel.RedactList /
RestoreRedactedList als geteilte Helfer hochgezogen (TenantConfiguration delegiert jetzt dorthin,
statt eine zweite Kopie zu pflegen), AdminConfiguration.OpsNotificationChannelsJson +
GetOpsNotificationChannels() mit Legacy-Slot-Synthese, OpsAlertRule.NotifyChannelIds/IncludePayload,
NotificationAlert.DataJson (nur GenericJsonRenderer serialisiert es, additiv in schemaVersion 1.0).

Web: der Channel-Editor liegt jetzt in components/notifications/ChannelEditor.tsx und wird von
Tenant- und Admin-Seite geteilt (showEventToggles trennt die Faelle). Telegram erscheint im
Provider-Dropdown nur fuer Global Admins (Prop aus useAuth().user?.isGlobalAdmin, Hausmuster aus
SectionAutopilotValidation); ein bereits konfigurierter Telegram-Channel bleibt fuer alle sichtbar,
damit ein Speichern ihn nicht still verwirft.

Tests: TelegramChannelProviderTests (27) — Dispatcher-Routing inkl. "Telegram erreicht NIE den
SSRF-Pfad", Chat-ID-Validierung, GA-Gate auf beiden Schreibpfaden. Mutationsgeprueft: Gate
invertiert => 5 rot, zurueck => 27 gruen. OpsNotificationChannelsTests + OpsAlertPayloadTests
decken die mitgelieferten Modell-Additionen ab. TestNotifications ist der gemeinsame inerte Stub,
der die 17 bestehenden OpsAlertDispatchService-Konstruktionen ersetzt.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(backend,web): Ops-Alerts routen auf benannte Channels statt auf drei feste Slots

OpsAlertRule hatte kein Ziel. Jede matchende Regel ging an alle drei fest verdrahteten Slots in
AdminConfiguration (Telegram-ChatId / Teams-URL / Slack-URL), und nur die ERSTE matchende Regel
wurde ueberhaupt ausgewertet (FirstOrDefault). Ein Conversion-Signal an ein Sales-Team zu binden
hiess also: das Sales-Team bekommt auch jede Security-Regel.

Jetzt: AdminConfiguration.OpsNotificationChannelsJson haelt eine benannte Channel-Liste (dasselbe
NotificationChannel-Modell wie tenant-seitig, nur anderer Scope), und OpsAlertRule.NotifyChannelIds
bindet Regeln daran. Dispatch wertet ALLE matchenden Regeln aus (Where statt FirstOrDefault) und
vereinigt ihre Ziele.

Rueckwaertskompatibel ohne Migrationsjob:
- Leere Channel-Liste => GetOpsNotificationChannels() synthetisiert je konfiguriertem Legacy-Slot
  einen Channel mit stabiler Id (legacy-telegram heisst "Autopilot Monitor IT Engineer") und
  uebernimmt die Legacy-Enabled-Flags — ein bewusst abgeschalteter Provider bleibt abgeschaltet.
- GetAdminConfigurationFunction projiziert DIESELBE Synthese in das Read-Model, damit der Editor
  zeigt, was heute tatsaechlich Alerts empfaengt, und der erste Save exakt das materialisiert.
  Die Alternative waere eine zweite Kopie der Synthese-Regeln im Web gewesen, die vom Dispatcher
  wegdriftet.
- Leere NotifyChannelIds = alle aktivierten Channels. Das ist, was jede heute existierende Regel
  traegt, also verhaelt sich am Deploy-Tag nichts anders.
- Ids, die nicht mehr aufloesen, werden verworfen statt auf Broadcast zurueckzufallen: eine Regel,
  deren einziger Channel geloescht wurde, muss NICHTS erreichen, nicht alles.

Die Legacy-Felder bleiben als Migrationsquelle stehen und werden von der Alerts-Section nicht mehr
geschrieben — aufraeumen erst, wenn produktiv Channels stehen.

Sicherheit: OpsNotificationChannelsJson ist secret-tragend und steht in der Deny-List von
RedactedCopyForReader() (ConfigRedactionTests bewacht Drift). Der Admin-PUT hat bisher keinen
Redacted-Restore — er bekommt jetzt einen fuer die Channel-Liste, damit ein zurueckgespieltes
Reader-View nicht "***REDACTED***" als Chat-ID persistiert, plus die strukturelle Validierung, die
auch die Tenant-Liste passiert.

Neuer Endpoint POST /api/global/config/test-ops-channel (GlobalAdminOnly, im Policy-Catalog
registriert). Kein neues Response-DTO: TestWebhookNotificationResponse hat identische Semantik und
wird geteilt — nur der Doc-Kommentar nennt jetzt beide Routen.

Web: die "Alert Providers"-Karte weicht der Channel-Liste (geteilter ChannelEditor, im Admin-Kontext
mit Telegram und ohne die tenant-spezifischen Event-Toggles). Jede Regel bekommt einen
Channel-Picker; leere Auswahl rendert als "all channels", nicht als Warnung — hier ist das der
gueltige Default. Die Bindungslogik liegt testbar in opsChannelRouting.ts und spiegelt
ResolveTargets: der erste explizite Pick startet vom LEEREN Set, sonst liesse sich eine
Broadcast-Regel nie verengen. handleSaveOpsAlertConfig nimmt jetzt ein Objekt statt elf
positionaler Argumente — bei einem Dutzend Feldern schreibt ein vertauschtes Paar sonst still eine
Webhook-URL in den falschen Provider-Slot.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(backend): TenantTrialStarted + Event-Payload optional pro Regel

Der Trial-Start war stumm. POST config/{tenantId}/trial schrieb nur einen Audit-Entry, und der
GA-Pfad (PATCH plan) feuerte nur beim Downgrade — also war genau der Conversion-Moment, den
Sales/Support interessiert, das einzige Plan-Ereignis ohne Ops-Event.

RecordTenantTrialStartedAsync (Tenant/Info) feuert jetzt aus BEIDEN Schreibpfaden, unterschieden
per selfService-Flag, und jeweils NACH dem persistierten Save (SaveConfigurationAsync ist
fail-loud), damit kein Alert eine Trial ankuendigt, die nicht gespeichert wurde. Der GA-Pfad
haengt an der aufgezeichneten Aenderung, nicht am Request: ein reiner planTier-Edit bleibt still,
ein idempotenter Save mit demselben Datum ebenfalls, und das Beenden einer Trial (explizit null)
ist ein Downgrade und bleibt bei TenantPlanDowngraded.

Payload-Zustellung: WriteAsync reicht die serialisierten Details an DispatchAsync durch — aber ein
Channel bekommt sie nur, wenn eine Regel OpsAlertRule.IncludePayload setzt. Default AUS, bewusst:
ein Ops-Alert ist ein "da ist was, schau nach"-Signal, die meisten Payloads sind im Chat Rauschen,
und manche tragen Felder, die die Baseline nie traegt (Tenant-Domain, Contact-Adresse). Das
Informationsniveau aller bestehenden Regeln bleibt damit unveraendert.

ResolveTargets liefert deshalb zwei Gruppen. Ein Channel, den sowohl eine Payload-Regel als auch
eine schlichte Regel adressiert, landet nur in der Payload-Gruppe: eine Nachricht pro Channel, und
ein unbeteiligtes Geschwister-Match kann ein bewusstes Opt-in nicht aushebeln.

Fuer den angereicherten Alert flacht BuildAlert die Top-Level-Skalare in Facts ab (max 12, Werte
<= 256 Zeichen, Nulls und Verschachteltes raus), damit Card-Formate und der Telegram-Klartext
dieselbe Information tragen wie der GenericJson-Consumer unter "data". EventType wird immer
gesetzt — derselbe Wert steckt schon im "Event"-Fact, also keine zusaetzliche Information, nur ein
stabiler Schluessel zum Verzweigen.

Tests: TenantTrialStartedEventTests (8) — aufgezeichnete Event-Form inkl. Payload-Feldern, und die
Emissionsbedingung des GA-Pfads gegen ApplyPlanChanges gepinnt (greift bei Grant und Verlaengerung,
schweigt bei Trial-Ende, planTier-only und idempotentem Save). Die HTTP-Entry-Points bleiben
ungetestet — Suite-Konvention ist, die reinen Kerne zu testen.

Docs: OKF-Concept backend/ops-notification-channels.md. Registrierung in docs/index.md +
docs/log.md fehlt bewusst — beide Dateien tragen noch fremdes uncommittetes WIP (ops-events
Query-Surface), und ein Teil-Commit darin haette es mitgenommen.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(backend,mcp): ops-events server-side field filters + generated MCP tool vocabularies

The ops-events read could only narrow by category and date window, so answering
"did any AgentEmergencyBreak fire yesterday?" meant pulling a whole category and
filtering it client-side — and for the MCP tool that client is a language model,
so the question cost a category-sized response in model context.

Ops-events filter surface (mirrors the audit-log filter pattern one-for-one):
- OpsEventQueryFilters (Shared) + OpsEventFilterRequest (query -> filters +
  pagination extras) + TableOpsEventRepository.AppendFieldFilters (filters ->
  OData clauses). BOTH filter builders call the appender — the single-category /
  unpaged one and the per-partition fan-out builder that adds the RowKey bound —
  so "no category named" cannot silently return unfiltered rows.
- minSeverity expands to an OR-set, not a range: Table Storage compares strings
  lexicographically and the severity names sort Critical < Error < Info <
  Warning, so `Severity ge 'Warning'` would drop exactly the alert-worthy half.
  The ladder now lives in OpsEventSeverity.Rank, shared with
  OpsAlertDispatchService, so "Warning and above" in a read means what it meant
  when the alert rules fired.
- severity is normalised onto the canonical vocabulary and an unknown value is a
  400: `eq` is case-sensitive, so "?severity=warning" would otherwise return an
  empty page that reads exactly like "no such events happened". eventType stays
  verbatim — stored rows outlive the vocabulary, so a retired type must remain
  searchable in the historical window.
- Every filter value is folded into the continuation fingerprint and echoed on
  nextLink; extras are appended AFTER the pre-existing category/tenantId
  discriminators and skipped when empty, so tokens minted before the filters
  existed still fingerprint identically. The tenantId drill deliberately stays
  in memory (OrdinalIgnoreCase; an OData eq would lose rows on casing).
- MCP get_ops_events gains eventType / severity / minSeverity plus a `days`
  shorthand resolved client-side to a concrete dateFrom (a backend-side "last N
  days" would re-resolve "now" on every page and blow the token fingerprint);
  an explicit bound always wins over the shorthand.

Fixed alongside: the paged cross-category fan-out skipped Platform (Azure
Monitor alerts) because its category list had drifted from OpsEventCategory —
written from day one, invisible to every paged reader, still visible on the
unpaged full-table path. The list is now OpsEventCategory.All plus a reflection
coverage test.

That drift had a structural cause, so the same round closes it for the MCP as a
whole: wire TYPES reached the MCP from day one, but vocabularies did not — types
are erased at runtime and z.enum() needs values, so every tool schema hand-typed
its own copy with nothing to compare against. The copies had drifted:
search_sessions offered 5 of 8 session statuses (no AwaitingUser, no Incomplete
— both real stored statuses), both event readers offered 4 of 6 severities (no
Debug, no Trace), get_ops_events named 6 of 7 ops categories.

- The codegen now emits src/generated/wire-vocabularies.generated.ts (VALUES:
  session statuses, event severities, ops categories/severities/types,
  annotation lanes/verdicts) from the manifest sections listed in
  MCP_VOCABULARIES; SharedManifestParityTests exports the two new ops
  vocabularies. Freshness is pinned twice — byte-equality against a fresh
  codegen run in the MCP suite, and the shared-manifests-in-sync CI job.
- Tool schemas derive their z.enum() from it and descriptions interpolate the
  list instead of spelling it.
- vocabulary-drift.guard.test.ts ratchets what is left: every inline z.enum([..])
  must sit on a baseline naming why it has no backend owner (MCP-local knobs,
  plus rule type / connection type / CVE risk, which have no C# constants class
  yet). A new hand-typed list fails, and so does a stale baseline entry.
- OpsEventTypes (Shared) gives the 77 ops event types a canonical home: the
  OpsEventService write sites use the constants (a raw literal now fails
  OpsEventTypeDualRegisterTests, which moved from regex-scraping to reflection),
  the manifest exports them, and get_resource(name="ops_event_types") serves
  categories + severities + types to the model from the generated copy.

Suites: Backend 5427, MCP 629 + tsc, Web tsc + vitest 965, generator idempotent.

Deploy order: backend BEFORE the MCP redeploy — an MCP that sends filters to an
older backend gets silently unfiltered rows back and presents them as filtered.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs: register the ops notification channels doc in the index and log

The concept doc landed with the feature commits; its index entry and log entry
were still uncommitted in the working tree.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-01 16:57:41 +02:00

8.6 KiB
Raw Blame History

type, title, description, resource, tags, timestamp
type title description resource tags timestamp
Concept Ops Notification Channels How platform alerts are routed to named channels instead of three fixed provider slots — the shared NotificationChannel model on both the tenant and platform side, one NotificationChannelDispatcher that routes each channel to its transport, the legacy-slot synthesis that keeps pre-migration dispatch behaviour identical, the empty-means-all rule binding, why Telegram is Global-Admin-only by construction, and why an ops event's structured payload only reaches a destination when a rule opts in. /src/Backend/AutopilotMonitor.Functions/Services/OpsAlertDispatchService.cs
backend
notifications
ops-events
configuration
security
2026-09-01T00:00:00+02:00

Problem

Platform (ops) alerting had no routing. An OpsAlertRule was { EventType, MinSeverity, Enabled } with no destination, and OpsAlertDispatchService broadcast every matching rule to all three hard-wired slots in AdminConfiguration (OpsAlertTelegramChatId, OpsAlertTeamsWebhookUrl, OpsAlertSlackWebhookUrl). Three consequences followed:

  1. No per-event destinations. Binding a conversion signal to a sales channel meant every other rule reached that channel too.
  2. Only the first matching rule was evaluated (FirstOrDefault), so two rules on the same event type could not coexist.
  3. The payload never left the backend. OpsEventService.WriteAsync serialized a details object into the OpsEvents table but passed only (category, eventType, severity, message, tenantId) to the dispatch — a receiving system saw a tenant GUID and a sentence.

The tenant side already had the right model: NotificationChannel (a named list in TenantConfiguration.NotificationChannelsJson) with analyze rules binding channels by id (RuleState.NotifyChannelIdsJson). The platform side needed that model, not a second one.

Schema

One model, two scopes

NotificationChannel is unchanged and now backs both lists:

Scope Storage Bound by
Tenant TenantConfiguration.NotificationChannelsJson per-channel NotifyOn* toggles + analyze rules (NotifyChannelIdsJson)
Platform AdminConfiguration.OpsNotificationChannelsJson OpsAlertRule.NotifyChannelIds

The scopes stay separate on purpose — platform events resolve against platform channels, and a tenant's channel list can never receive an ops alert. What is shared is the model and the dispatch path, so a new provider is added once.

One dispatch path

NotificationChannelDispatcher is the channel-level send API. It routes by provider: WebhookProviderType.Telegram goes to TelegramNotificationService; everything else is a rendered webhook POST through WebhookNotificationService. Every caller that sends to configured channels — enrollment (EventIngestProcessor), session start (RegisterSessionFunction), hardware rejection (ReportDistressFunction), SLA (SlaBreachEvaluationService), analyze rules (AnalyzeOnEnrollmentEndHandler), ops alerts (OpsAlertDispatchService) and both test endpoints — goes through it.

Telegram is Global-Admin-only by construction

Telegram (WebhookProviderType.Telegram = 40) is the odd provider: it is not a webhook. The channel's Url carries the destination chat ID, and the bot token is platform-owned (PreviewConfig WebhookUrl). A tenant admin configuring one would be sending through our bot, which is why the gate is a server-side rule, not a hidden dropdown option:

  • TenantConfigValidation.ValidateTelegramChannelGate refuses a non-GA caller who adds a Telegram channel or changes an existing one's destination or enabled state. It lives in ValidateModel, so it covers both write paths — the full-model PUT and the MCP field-patch flow (TenantConfigPatchService). A check placed in UpdateTenantConfigurationFunction alone would not.
  • An unchanged GA-created channel passes, so a tenant admin can still save unrelated config.
  • Telegram destinations skip SsrfGuard.ValidateWebhookUrlFormat (a chat ID is not a URL and would always fail it) and are format-checked as a numeric id or @username instead.

Migration without a data job

AdminConfiguration.GetOpsNotificationChannels() returns the stored list, or — when it is empty — synthesizes one channel per configured legacy slot with stable ids (legacy-telegram, legacy-teams, legacy-slack) and the legacy enabled flags. GetAdminConfigurationFunction projects the same synthesis into the read model, so the editor shows exactly what receives alerts today and the first save materializes it. The legacy fields stay as the migration source and are no longer written by the Alerts section.

Rule binding: empty means all

OpsAlertRule.NotifyChannelIds null or empty = every enabled channel. That is what every rule written before routing existed carries, so behaviour on the deploy day is identical. Dispatch resolves all matching rules (Where, not FirstOrDefault) and unions their channels; ids that no longer resolve are dropped rather than falling back to broadcast — a rule whose only channel was deleted must reach nothing, not everything. The web editor mirrors the same rule in app/admin/components/opsChannelRouting.ts.

Payload delivery is opt-in per rule

WriteAsync hands the serialized details to DispatchAsync, but a channel only receives it when a rule sets OpsAlertRule.IncludePayload. Default false: an ops alert is a "something happened, go look" signal, most payloads are operational noise in a chat, and some carry data the baseline never does (tenant domain, administrator contact address). Widening what leaves the platform stays a deliberate per-rule decision.

ResolveTargets therefore returns two groups. Channels reached only by plain rules get an alert built with detailsJson: null; channels reached by an opted-in rule get the enriched one. A channel in both groups appears in the payload group only — one message per channel, and an unrelated sibling rule cannot cancel an explicit opt-in.

For the enriched alert, BuildAlert:

  • flattens top-level scalars into NotificationFacts — capped at 12 facts and 256 characters, nulls and nested values skipped — so card formats and the plain-text Telegram message carry the same information,
  • passes the raw JSON through NotificationAlert.DataJson, which only GenericJsonRenderer emits, as a real JSON object under data. Additive within schemaVersion 1.0.

NotificationAlert.EventType is set either way: it is the same value the Event fact already carries, so it adds no information — only a stable key for a generic consumer to branch on.

Examples

A trial-conversion signal reaching only a sales webhook:

  1. Admin → Alerts → add a channel Sales (Generic JSON, the internal endpoint).
  2. Enable the TenantTrialStarted rule at Info, select only Sales, and switch its Payload toggle on (off by default, so every other rule keeps the baseline alert).
  3. PlanManagementFunction fires TenantTrialStarted from both plan write paths — the self-service POST config/{tenantId}/trial and a GA grant via PATCH config/{tenantId}/plan (distinguished by selfService). Ending a trial is a downgrade and stays with TenantPlanDowngraded.
  4. The webhook receives eventType, the rendered facts, and data with domainName, contactEmail, trialStartedUtc, trialExpiresUtc, grantedBy, selfService. The operator push channel receives nothing, because it is not bound to that rule.

Gotchas

  • A new ops event type must be dual-registered in OPS_EVENT_TYPES (app/admin/components/OpsAlertRulesSection.tsx) or it cannot be selected. OpsEventTypeDualRegisterTests enforces it.
  • OpsNotificationChannelsJson is secret-bearing and is on the deny-list in AdminConfiguration.RedactedCopyForReader(). NotificationChannel.RedactList / RestoreRedactedList are shared with the tenant list — do not add a second copy.
  • Details are capped at 4096 characters in the OpsEvents table (TableOpsEventRepository) before they ever reach a channel.

Citations

  • /src/Backend/AutopilotMonitor.Functions/Services/OpsAlertDispatchService.cs
  • /src/Backend/AutopilotMonitor.Functions/Services/Notifications/NotificationChannelDispatcher.cs
  • /src/Shared/AutopilotMonitor.Shared/Models/Notifications/NotificationChannel.cs
  • /src/Shared/AutopilotMonitor.Shared/Models/Config/AdminConfiguration.cs
  • /src/Backend/AutopilotMonitor.Functions/Helpers/TenantConfigValidation.cs
  • /src/Backend/AutopilotMonitor.Functions/Functions/Config/PlanManagementFunction.cs
Reference in New Issue View Git Blame Copy Permalink