Commit Graph

6 Commits

Author SHA1 Message Date
Oliver Kieselbach 5cb9b0b30b backend: storage SDK budgets under the agent timeout, ingest 503 on an exhausted budget, request-token fan-outs, single-flight factory budget, ProcessStall monitor (audit F09, D-203)
- StorageClientOptions: Table/Queue 8 s per attempt × 3 attempts + capped backoffs = 25.8 s < the
  agent's 30 s upload timeout (pinned by test); Blob 60 s × 4. Every production client takes its
  options from there. No Polly over the SDK.
- StorageErrors.IsTransient(Exception): OperationCanceled / IO / HttpRequest / Aggregate-of-those
  / RequestFailed status 0; the ingest answers 503 + Retry-After instead of 500.
- The ingest still takes no caller cancellation (a batch is a unit). GetAllSessionsPageAsync
  carries the request token down to BoundedFanOut (Global Admin list); SingleFlightCache hands its
  factory a 30 s budget token and evicts on timeout (tenant ids, app-summary snapshot, session stats).
- ProcessStallMonitor: 1 s tick drift ≥ 1 s ⇒ ProcessStall event (sampling bypass) + metric with
  cause gc / threadpool / unknown from GC.GetTotalPauseDuration and thread-pool counters;
  GcPauseMs on request rows ≥ 100 ms. Measures the instance-local 5–10 s pauses seen in production.
- Gitlinks: internal (plan, D-203, concept doc, B-9hf, lesson), .claude/commands (marker rows).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-06 01:51:02 +02:00
Oliver Kieselbach e4203e0f15 chore: bump .claude/commands (platform-health skill, backend-logs corrections) and internal (marker-catalog rule) gitlinks; drop stale index dual-write mention in the dependency filter comment
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-05 22:53:14 +02:00
Oliver Kieselbach fc9e9d13e6 backend: keep slow storage successes and count CAS conflicts as a metric (audit F10)
The dependency filter dropped every successful storage call and the expected
412/409 outcomes, so storage-side latency and ETag contention were invisible
until a call failed outright (seven days of App Insights held zero storage
dependency rows).

- StorageDependencyFilterProcessor keeps a successful storage call at or above
  2 s and stamps SlowStorage=true; the operation name is on the row, the table
  name is not (the Azure SDK span carries neither URL nor table).
- StorageMetrics wraps TelemetryClient.GetMetric("StorageCasConflict",
  operation, table, outcome): pre-aggregated per minute, bypasses the worker's
  adaptive sampling, null-tolerant for tests.
- The CAS loops report conflicts: session event count, session counter
  reconcile, session store, tenant stats, MCP user/tenant usage.
- TableStorageService keeps its two-argument constructors (Moq matches by
  exact arity); DI resolves the three-argument overload.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-05 22:09:08 +02:00
Oliver Kieselbach 0d5e2572aa perf(telemetry): one canonical request record; drop expected storage outcomes and SignalR success deps
- RequestTelemetryMiddleware: SamplingPercentage=100 per item (worker copy was
  adaptively sampled, ItemCount 2-5 on weekdays, despite host.json excludedTypes)
- StorageDependencyFilterProcessor: also drop 404/412/409 storage outcomes
  (InProc + HTTP shape) and successful *.service.signalr.net REST calls
- infra/appinsights-workspace-transforms.bicep: DCR dropping the host HTTP
  request duplicate at ingestion (timer/queue rows kept)
- docs/backend/telemetry-ingest-shaping.md; skills updated

Measured baseline 7d: ~1.3 GB/week removed, requests now 100% with tenant context.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-23 18:31:32 +02:00
Oliver Kieselbach 5e6d3f303d adjusted AppInsights telemetry filter 2026-06-09 21:12:38 +02:00
Oliver Kieselbach c0552b7f9d reduced AppInsights dependency telemetry 2026-06-08 11:03:01 +02:00