diff --git a/docs/architecture.md b/docs/architecture.md index 0f8ce6a5..75e6ef53 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -39,9 +39,13 @@ | telemetry / events +---------------------------+ | +---------------------------+ +---------------------------+ | Notifications | -| Agent on Windows Devices | | Teams / Webhooks ... | +| Agent on Windows Devices | | Teams / Slack / Telegram | | Enrollment + app progress | +---------------------------+ +---------------------------+ + +---------------------------+ + | MCP Server | + | Azure Container App | + +---------------------------+ ``` **Core Data Flow:** @@ -60,6 +64,7 @@ Device (Agent) ──NDJSON+gzip──► Azure Functions ──► Azure Table | **Backend** | Azure Functions .NET 8 (Isolated Worker) | REST API, event processing, rule engine, storage | | **Web** | Next.js 15 + TypeScript + React 18 | Dashboard, settings, analytics, real-time UI | | **Shared** | .NET Standard 2.0 | DTOs, models, enums, constants shared between Agent & Backend | +| **MCP Server** | Azure Container App | Model Context Protocol server for AI-assisted analysis | --- @@ -77,8 +82,11 @@ Device (Agent) ──NDJSON+gzip──► Azure Functions ──► Azure Table 10. [Session Lifecycle](#session-lifecycle) 11. [Configuration Hierarchy](#configuration-hierarchy) 12. [Diagnostics & Upload](#diagnostics--upload) -13. [Infrastructure & Deployment](#infrastructure--deployment) -14. [Build & Development](#build--development) +13. [Vulnerability Management](#vulnerability-management) +14. [Notification System](#notification-system) +15. [Testing](#testing) +16. [Infrastructure & Deployment](#infrastructure--deployment) +17. [Build & Development](#build--development) --- @@ -91,6 +99,7 @@ AutopilotMonitor.sln │ ├── Agent/ │ │ ├── AutopilotMonitor.Agent/ (net48, Exe – entry point) │ │ ├── AutopilotMonitor.Agent.Core/ (net48, Lib – core logic) +│ │ ├── AutopilotMonitor.Agent.Core.Tests/ (net48, xUnit – agent unit tests) │ │ └── AutopilotMonitor.SummaryDialog/ (net48, WPF – enrollment summary UI) │ ├── Backend/ │ │ ├── AutopilotMonitor.Functions/ (net8.0, Azure Functions v4) @@ -104,12 +113,17 @@ AutopilotMonitor.sln │ ├── schema/ JSON Schema definitions │ ├── scripts/ combine.js │ └── dist/ Combined output (embedded in Functions) +├── infra/ Bicep templates (MCP server) +├── scripts/ +│ ├── Bootstrap/ Intune deployment scripts +│ └── Deployment/ CI/CD build scripts └── .github/workflows/ CI/CD pipelines ``` **Project References:** ``` Shared ◄── Agent.Core ◄── Agent + ◄── Agent.Core.Tests ◄── SummaryDialog Shared ◄── Functions ◄── Functions.Tests Web (independent – communicates via REST + SignalR) @@ -123,50 +137,93 @@ Web (independent – communicates via REST + SignalR) **File:** `src/Agent/AutopilotMonitor.Agent/Program.cs` -Three execution modes: +Four execution modes: 1. **Normal mode** (default) – Main enrollment monitoring loop 2. **`--install` mode** – Deploys agent via Scheduled Task (Intune package) 3. **`--run-gather-rules` mode** – One-shot offline data collection, then exits +4. **`--run-ime-matching` mode** – Offline IME log parsing, writes `ime_pattern_matching.log` + +Partial class files: `Program.Configuration.cs`, `Program.InstallMode.cs`, `Program.GatherRulesMode.cs`, `Program.ImeMatchingMode.cs` ### Startup Sequence (Normal Mode) 1. Single-instance check (prevent duplicate agent processes) -2. `FetchRemoteConfig()` – Backend config with 15s timeout, disk cache fallback -3. `RegisterSessionAsync()` – Register session (5 retries, exponential backoff) -4. `StartWatching()` – Enable FileSystemWatcher for event spool -5. `StartEventCollectors()` – EspAndHelloTracker (always on) -6. `StartOptionalCollectors()` – PerformanceCollector, AgentSelfMetrics, EnrollmentTracker, DesktopArrivalDetector -7. `StartGatherRuleExecutor()` – Backend-defined data collection rules -8. `InitializeAnalyzers()` + `RunStartupAnalyzers()` – Security baseline +2. Self-update: `SelfUpdater.CheckAndApplyUpdateAsync()` (downloads from Azure Blob) +3. Load configuration +4. Check enrollment-complete marker (handles post-reboot cleanup retry) +5. Check session age emergency break (zombie agent kill-switch) +6. Detect previous exit type: `clean` / `exception_crash` / `hard_kill` / `reboot_kill` / `first_run` +7. `FetchRemoteConfig()` – Backend config with 15s timeout, disk cache fallback +8. `RegisterSessionAsync()` – Register session (5 retries, exponential backoff) +9. `StartWatching()` – Enable FileSystemWatcher for event spool +10. `StartEventCollectors()` – EspAndHelloTracker (always on) +11. `StartOptionalCollectors()` – PerformanceCollector, AgentSelfMetrics, EnrollmentTracker, DesktopArrivalDetector +12. `StartGatherRuleExecutor()` – Backend-defined data collection rules +13. `InitializeAnalyzers()` + `RunStartupAnalyzers()` – Security baseline ### Key Services | Service | Location | Responsibility | |---------|----------|----------------| -| `MonitoringService` | `Core/Monitoring/Core/` | Main orchestrator: starts/stops collectors, manages upload loop | -| `BackendApiClient` | `Core/Monitoring/Network/` | HTTP client with mTLS cert + hardware headers | +| `MonitoringService` | `Core/Monitoring/Core/` | Main orchestrator: starts/stops collectors, manages upload loop (3 partial files) | +| `BackendApiClient` | `Core/Monitoring/Network/` | HTTP client with mTLS cert + hardware headers + bootstrap token | | `EventSpool` | `Core/Monitoring/Core/` | Offline event storage (JSON files), FileSystemWatcher-based | -| `EnrollmentTracker` | `Core/Monitoring/Tracking/` | Central enrollment state machine, 3 completion paths | -| `ImeLogTracker` | `Core/Monitoring/Tracking/` | Parses IME logs with backend-provided regex patterns | -| `EspAndHelloTracker` | `Core/Monitoring/Collectors/` | Windows event log monitoring (ESP exit, Hello provisioning) | -| `DesktopArrivalDetector` | `Core/Monitoring/Collectors/` | Polls for explorer.exe under real user (non-SYSTEM) | -| `PerformanceCollector` | `Core/Monitoring/Collectors/` | CPU/Memory/Disk metrics | +| `EnrollmentTracker` | `Core/Monitoring/Tracking/` | Central enrollment state machine, 3 completion paths (4 partial files) | +| `ImeLogTracker` | `Core/Monitoring/Tracking/` | Parses IME logs with backend-provided regex patterns (3 partial files) | +| `EspAndHelloTracker` | `Core/Monitoring/Collectors/` | ESP state + Hello provisioning monitoring (4 partial files) | +| `DesktopArrivalDetector` | `Core/Monitoring/Collectors/` | Polls for explorer.exe under real user (non-SYSTEM, non-DefaultUser*) | +| `PerformanceCollector` | `Core/Monitoring/Collectors/` | CPU/Memory/Disk/Network metrics | +| `AgentSelfMetricsCollector` | `Core/Monitoring/Collectors/` | Agent process self-telemetry | | `GatherRuleExecutor` | `Core/Monitoring/Collectors/` | Executes backend-defined data collection rules | +| `DeviceInfoCollector` | `Core/Monitoring/Collectors/` | Hardware spec + network/security info (3 partial files) | | `DiagnosticsPackageService` | `Core/Monitoring/Core/` | Creates ZIP + uploads via short-lived SAS URL | | `RemoteConfigService` | `Core/Configuration/` | Fetches & caches backend config with disk fallback | -| `SessionPersistence` | `Core/Monitoring/Core/` | Persists session ID, sequence counter, markers | +| `SessionPersistence` | `Core/Monitoring/Core/` | Persists session ID, sequence counter, WhiteGlove state | +| `DistressReporter` | `Core/Monitoring/Network/` | Pre-auth distress signals (no cert required) | +| `EmergencyReporter` | `Core/Monitoring/Network/` | Posts AgentErrorReport via BackendApiClient | +| `GeoLocationService` | `Core/Monitoring/Network/` | IP-based device location lookup | +| `NtpTimeCheckService` | `Core/Monitoring/Network/` | UDP NTP query to detect clock skew | +| `TimezoneService` | `Core/Monitoring/Network/` | Sets Windows timezone via `tzutil /s` | + +### Analyzers + +| Analyzer | Purpose | +|----------|---------| +| `LocalAdminAnalyzer` | Enumerates local admin accounts at start+shutdown, flags unexpected accounts | +| `SoftwareInventoryAnalyzer` | Captures installed software from registry at start+shutdown, emits `software_inventory_analysis` events | + +### Gather Rule Collectors + +Each implements `IGatherRuleCollector`: + +| Collector | Purpose | +|-----------|---------| +| `RegistryCollector` | Read registry values | +| `WmiCollector` | WMI queries | +| `EventLogCollector` | Windows Event Log queries | +| `FileCollector` | Collect file contents | +| `CommandCollector` | Run allowlisted commands | +| `LogParserCollector` | Parse log files | +| `JsonCollector` | Collect JSON from file/registry | +| `XmlCollector` | Parse XML files | + +Security enforced by `GatherRuleGuards` (allowed targets) and `DiagnosticsPathGuards` (path validation, respects `UnrestrictedMode`). ### Directory Layout (Agent.Core) ``` +Configuration/ Config loading, remote config service +Logging/ AgentLogger (file + optional console) Monitoring/ -├── Analyzers/ Security checks (LocalAdminAnalyzer) -├── Collectors/ Data collectors (ESP, Hello, Performance, GatherRules, Diagnostics) -├── Core/ Orchestration (MonitoringService, EventSpool, SessionPersistence) -├── Interop/ P/Invoke declarations (process creation, registry change notifications) -├── Network/ API client, emergency reporter, geo-location, network metrics -├── Replay/ Log replay for testing/simulation -└── Tracking/ Enrollment state machine, IME parser, script tracking +├── Analyzers/ Security checks (LocalAdminAnalyzer, SoftwareInventoryAnalyzer) +├── Collectors/ Data collectors (ESP, Hello, Performance, GatherRules, Diagnostics, Desktop) +│ └── GatherCollectors/ Per-type gather rule collectors +├── Core/ Orchestration (MonitoringService, EventSpool, SessionPersistence, CleanupService) +├── Interop/ P/Invoke declarations (process creation, registry change notifications) +├── Network/ API client, distress/emergency reporters, geo-location, NTP, network metrics +├── Replay/ Log replay for testing/simulation +└── Tracking/ Enrollment state machine, IME parser, script/app tracking, state persistence +Security/ Certificate helper, enrollment awaiter, hardware info ``` ### Event Collection & Upload @@ -210,7 +267,16 @@ Monitoring/ - **Monitoring:** Application Insights with sampling - **Entry Point:** `src/Backend/AutopilotMonitor.Functions/Program.cs` -### Endpoints (~61 Functions) +### Middleware Pipeline + +Registered in this order: +1. `RequestTelemetryMiddleware` – Wraps all requests for Application Insights telemetry +2. `CorrelationIdMiddleware` – Propagates or generates `X-Correlation-Id` +3. `AuthenticationMiddleware` – Validates JWT Bearer tokens via Azure AD OIDC metadata; caches per-tenant config managers (max 100); tracks MCP usage via `X-Client-Source: mcp` header +4. `PolicyEnforcementMiddleware` – Catalog-driven authorization (fail-closed: unregistered routes get 403); sets `RequestContext` (IsGlobalAdmin, IsTenantAdmin, UserRole) +5. `UserRateLimitMiddleware` – Per-user API rate limiting for human users + +### Endpoints (~140 Functions) **Agent-to-Cloud (device auth via cert/bootstrap):** @@ -221,13 +287,7 @@ Monitoring/ | `/agent/config` | GET | Fetch agent configuration | | `/agent/upload-url` | POST | Get short-lived SAS URL for diagnostics | | `/agent/error` | POST | Report agent errors | - -**Version Management:** - -| Route | Method | Purpose | -|-------|--------|---------| -| `/versions/block` | POST | Add version block/kill rule | -| `/versions/blocked` | GET | List active version block rules | +| `/agent/distress` | POST | Pre-auth distress signal (no cert required) | **Bootstrap (pre-MDM auth):** @@ -235,51 +295,95 @@ Monitoring/ |-------|--------|---------| | `/bootstrap/sessions` | POST/GET | Create/list bootstrap sessions | | `/bootstrap/sessions/{code}` | DELETE | Revoke bootstrap session | -| `/bootstrap/validate/{code}` | POST | Validate bootstrap code | +| `/bootstrap/validate/{code}` | GET | Validate bootstrap code (public) | | `/bootstrap/register-session` | POST | Register via bootstrap token | | `/bootstrap/ingest` | POST | Ingest via bootstrap token | -| `/bootstrap/config` | GET | Config via bootstrap token | +| `/bootstrap/config` | POST | Config via bootstrap token | +| `/bootstrap/error` | POST | Error report via bootstrap token | **Web Portal (JWT auth):** | Category | Key Routes | |----------|------------| -| **Sessions** | `GET /sessions`, `GET /sessions/{id}`, `GET /sessions/{id}/events`, `POST /sessions/{id}/analysis`, `DELETE /sessions/{id}`, `POST /sessions/{id}/mark-failed` | -| **Rules** | CRUD for `/rules/gather`, `/rules/analyze`, `/rules/ime-log-patterns`, `POST /rules/reseed-from-github` | -| **Config** | `GET/POST /global/config`, `GET/PUT /config/{tenantId}` | -| **Auth** | `GET /auth/me`, `GET /auth/is-global-admin`, CRUD `/auth/global-admins` | +| **Sessions** | `GET /sessions`, `GET /sessions/{id}`, `GET /sessions/{id}/events`, `DELETE /sessions/{id}`, `POST /sessions/{id}/mark-failed`, `POST /sessions/{id}/mark-succeeded` | +| **Search** | `GET /search/quick`, `GET /search/sessions`, `GET /search/sessions-by-event`, `GET /search/sessions-by-cve` | +| **Rules** | CRUD for `/rules/gather`, `/rules/analyze`, `/rules/ime-log-patterns`, `POST /rules/reseed-from-github`, `GET /rules/results` | +| **Config** | `GET/PUT /global/config`, `GET/PUT /config/{tenantId}`, `GET /config/all`, `GET /global/config/plan-tiers` | +| **Auth** | `GET /auth/me` | | **Tenants** | CRUD `/tenants/{id}/admins`, `POST /tenants/{id}/offboard` | -| **Devices** | `POST /devices/block`, `DELETE /devices/block/{serial}`, `GET /devices/blocked` | -| **Reports** | `POST /sessions/{id}/report`, `GET /global/session-reports` | -| **Metrics** | `GET /metrics/usage`, `GET /metrics/app`, `GET /metrics/geographic`, `GET /stats/platform` | -| **Global** | `/global/metrics/*`, `/global/audit/logs`, `/global/session-reports`, `/global/notifications` (CRUD + dismiss) | -| **Progress** | `GET /progress/sessions`, `GET /progress/sessions/{id}/events` | +| **Devices** | `POST /devices/block`, `DELETE /devices/block/{serial}`, `GET /devices/blocked`, `GET /global/devices/blocked` | +| **Versions** | `POST /versions/block`, `GET /versions/blocked`, `DELETE /versions/block/{pattern}` | +| **Reports** | `POST /sessions/{id}/report`, `GET/POST /global/session-reports`, distress reports | +| **Metrics** | `GET /metrics/usage`, `GET /metrics/app`, `GET /metrics/summary`, `GET /metrics/geographic`, `GET /metrics/geographic/sessions`, `GET /stats/platform` | +| **Global** | `/global/metrics/*`, `/global/audit/logs`, `/global/session-reports`, `/global/notifications` (CRUD + dismiss), `/global/mcp-users` | +| **Vulnerability** | `GET /sessions/{id}/vulnerability-report`, CPE mapping CRUD, `POST /vulnerability/sync`, software inventory | +| **Progress** | `GET /progress/sessions` | +| **Diagnostics** | `GET /diagnostics/download-url` | | **SignalR** | `POST /realtime/negotiate`, `POST /realtime/groups/join`, `POST /realtime/groups/leave` | | **Health** | `GET /health`, `GET /health/detailed` | +| **Feedback** | `POST /feedback` | +| **MCP** | `GET/POST/DELETE /global/mcp-users`, MCP usage metrics | +| **Raw/Debug** | AppInsights query proxy, raw events/sessions query, table query | +| **Autopilot Validation** | Consent URL + status for Graph API access | -**Timer Trigger:** +**Timer Triggers:** - `MaintenanceFunction` – Every 2 hours (`0 0 */2 * * *`): stale session detection, metrics aggregation, data cleanup +- `VulnerabilityDataSyncFunction` – Periodic CVE data sync from NVD/MSRC/CISA KEV ### Key Backend Services | Service | Responsibility | |---------|----------------| -| `TableStorageService` | Core data access for all 21 Azure Tables (split across 5 files) | +| `TableStorageService` | Core data access for all 33 Azure Tables (split across 5 partial files) | | `TenantConfigurationService` | Per-tenant config with 5-min cache | | `AdminConfigurationService` | Global config with 5-min cache, syncs rate limits to tenants | | `RateLimitService` | In-memory sliding window rate limiting (1-min window) | +| `DistressRateLimitService` | Rate limiter for the unauthenticated distress endpoint | | `SecurityValidator` | Centralized request validation (cert → rate limit → hardware → APV) | | `RuleEngine` | Server-side analyze rule evaluation with confidence scoring | -| `MaintenanceService` | Cleanup, metrics aggregation, stale session detection | +| `MaintenanceService` | Cleanup, metrics aggregation, stale session detection (+ Aggregation partial) | | `BootstrapSessionService` | Bootstrap token lifecycle (create, validate, revoke) | | `BlockedDeviceService` | Device block/kill signal management | -| `SessionReportService` | Report ZIP generation + Blob upload | -| `GraphTokenService` | MS Graph token acquisition for Autopilot device validation | | `BlockedVersionService` | Version-based block/kill rules with wildcard and ceiling patterns | -| `GlobalNotificationService` | Persistent in-app notifications for Global Admins (survives page reloads) | +| `SessionReportService` | Report ZIP generation + Blob upload | +| `BlobStorageService` | Azure Blob Storage (diagnostics upload SAS URLs) | +| `GraphTokenService` | MS Graph token acquisition for Autopilot device validation | +| `AutopilotDeviceValidator` | Validates serial against Intune Autopilot device list | +| `CorporateIdentifierValidator` | Validates against Intune Corporate Device Identifiers | +| `GlobalNotificationService` | Persistent in-app notifications for Global Admins | | `HealthCheckService` | Health checks for Storage, Processing, and Agent binary availability | | `PreviewWhitelistService` | Private Preview tenant whitelist with 5-min cache | -| `ResendEmailService` | Transactional emails via Resend.com for Preview notifications | +| `McpUserService` | MCP API user management + access control | +| `TenantAdminsService` | Tenant member/admin management (roles: Admin, Operator, Viewer) | +| `GlobalAdminService` | Global admin lookup | +| `UsageMetricsService` | Daily/rolling usage metrics | +| `PlatformMetricsService` | Platform-wide stats | +| `VulnerabilityCorrelationService` | Matches installed software against CVE data | +| `WebhookNotificationService` | Dispatches enrollment notifications via webhook (Teams/Slack) | +| `TelegramNotificationService` | Telegram bot notifications | +| `ResendEmailService` | Transactional emails via Resend API | +| `SignalRNotificationService` | Pushes real-time updates to connected web clients | +| `EventTimestampValidator` | Clamps/validates event timestamps (preserves originals) | +| `GitHubRuleRepository` | Fetches rules from GitHub for reseed | + +### Data Access Layer + +Repository pattern via interfaces in `AutopilotMonitor.Shared/DataAccess/`: + +| Repository | Purpose | +|------------|---------| +| `TableSessionRepository` | Sessions + events + indexes | +| `TableConfigRepository` | Tenant/admin configuration | +| `TableRuleRepository` | Gather/analyze rules, IME patterns, rule results | +| `TableMetricsRepository` | Usage metrics, app install summaries, platform stats | +| `TableAdminRepository` | Audit logs, global/tenant admins, MCP users | +| `TableBootstrapRepository` | Bootstrap sessions | +| `TableDeviceSecurityRepository` | Blocked devices, blocked versions | +| `TableDistressReportRepository` | Distress reports | +| `TableMaintenanceRepository` | Maintenance operations | +| `TableNotificationRepository` | Global notifications | +| `TableUserUsageRepository` | Per-user API usage tracking | +| `TableVulnerabilityRepository` | CVE data, CPE mappings, software inventory | ### Event Processing Pipeline @@ -298,6 +402,10 @@ Agent POST /api/agent/ingest (NDJSON+gzip) │ ├─ ParseNdjsonGzipRequest() → decompress + parse events │ + ├─ StampServerFields() → ReceivedAt, TenantId, SessionId + │ + ├─ EventTimestampValidator → clamp/validate timestamps + │ ├─ TableStorageService.StoreEventsBatchAsync() │ ├─ ClassifyEvents() @@ -310,6 +418,10 @@ Agent POST /api/agent/ingest (NDJSON+gzip) ├─ RuleEngine.AnalyzeSessionAsync() (on enrollment end) │ └─ StoreRuleResultAsync() → RuleResults table │ + ├─ VulnerabilityCorrelation (on software inventory events) + │ + ├─ WebhookNotificationService (Teams/Slack on enrollment complete/fail) + │ └─ SignalR broadcasts: ├─ "eventReceived" → tenant-{tenantId} ├─ "sessionStatusChanged" → tenant-{tenantId} @@ -329,6 +441,7 @@ Agent POST /api/agent/ingest (NDJSON+gzip) - **Real-time:** `@microsoft/signalr` 10.0.0 - **Maps:** Leaflet 1.9.4 + react-leaflet - **Compression:** fflate 0.8.2 +- **Telemetry:** Application Insights Web 3.3.11 ### Page Routes @@ -336,28 +449,39 @@ Agent POST /api/agent/ingest (NDJSON+gzip) |------|---------|------|------| | `/` | Landing page with platform stats | Public | — | | `/dashboard` | Session list, real-time updates | Yes | Any | -| `/sessions/[sessionId]` | Session detail, event timeline, analysis | Yes | Any | -| `/diagnosis/[sessionId]` | Rule analysis results | Yes | Any | -| `/settings` | Tenant configuration (10+ sections) | Yes | Tenant Admin | -| `/admin-configuration` | Global admin settings | Yes | Global Admin | -| `/gather-rules` | Gather rule CRUD | Yes | Tenant Admin | +| `/sessions/[sessionId]` | Session detail, event timeline, analysis, vulnerability report | Yes | Any | +| `/diagnosis/[sessionId]` | Simplified diagnosis with badges | Yes | Any | +| `/fleet-health` | App metrics, fleet analytics | Yes | Member | +| `/geographic-performance` | Geo map + session drill-down | Yes | Member | | `/analyze-rules` | Analyze rule CRUD | Yes | Tenant Admin | +| `/gather-rules` | Gather rule CRUD | Yes | Tenant Admin | | `/ime-log-patterns` | IME pattern management | Yes | Tenant Admin | -| `/fleet-health` | App metrics, fleet analytics | Yes | Tenant Admin | -| `/usage-metrics` | Tenant usage analytics | Yes | Tenant Admin | -| `/platform-metrics` | Platform-wide metrics | Yes | Global Admin | -| `/audit` | Audit log viewer | Yes | Tenant Admin | -| `/health-check` | Backend health status | Yes | Global Admin | -| `/geographic-performance` | Geo map of deployments | Yes | Global Admin | -| `/progress` | Real-time enrollment progress tracking | Yes | Tenant Admin | -| `/preview` | Private Preview waitlist/approval page | Yes | Unapproved tenants | +| `/audit` | Audit log viewer | Yes | Member | +| `/usage-metrics` | Tenant usage analytics | Yes | Member | +| **Settings** | | | | +| `/settings/tenant/[section]` | Access management, Autopilot validation, bootstrap, hardware whitelist, notifications | Yes | Tenant Admin | +| `/settings/agent/[section]` | Agent analyzers, agent settings, diagnostics, unrestricted mode | Yes | Tenant Admin | +| `/settings/management/[section]` | Data management, offboarding | Yes | Tenant Admin | +| `/settings/reporting/[section]` | MCP usage | Yes | Tenant Admin | +| **Admin (Global)** | | | | +| `/admin/metrics/[section]` | Agent metrics, MCP usage, platform usage | Yes | Global Admin | +| `/admin/reports/[section]` | Distress reports, session export, session reports, user feedback | Yes | Global Admin | +| `/admin/security/[section]` | Device block, version block, vulnerability data | Yes | Global Admin | +| `/admin/settings/[section]` | Config reseed, diagnostics log paths, global settings, MCP users, usage plans | Yes | Global Admin | +| `/admin/tenants/[section]` | Tenant config report, tenant management | Yes | Global Admin | +| `/admin/ops` | Operations page | Yes | Global Admin | +| `/admin/software` | Software inventory | Yes | Global Admin | +| **Public** | | | | +| `/progress` | Real-time enrollment progress (end users) | Public | — | +| `/docs/[section]` | Documentation (12 sections) | Public | — | | `/changelog` | Platform change log & known issues | Public | — | | `/roadmap` | Planned features & current focus areas | Public | — | | `/about` | Platform introduction & quick links | Public | — | | `/privacy` | Privacy policy & data handling | Public | — | | `/terms` | Terms of use & legal disclaimers | Public | — | -| `/go/[code]` | Bootstrap script generator (validates code → returns PS1) | Public | — | -| `/docs/[section]` | Documentation | Public | — | +| `/preview` | Private Preview waitlist/approval | Yes | Unapproved | +| `/health-check` | Backend health status | Public | — | +| `/go/[code]` | Bootstrap short-link redirector | Public | — | ### State Management @@ -365,15 +489,18 @@ React Context API (no Redux/Zustand): | Context | Purpose | |---------|---------| -| `AuthContext` | MSAL + user info + role detection (global/tenant admin) | +| `AuthContext` | MSAL + user info + role detection (global/tenant admin, operator, viewer, MCP access, bootstrap manager) | | `SignalRContext` | WebSocket connection, group subscriptions, auto-reconnect | | `TenantContext` | Current tenant ID (persisted to localStorage) | | `NotificationContext` | Toast notifications with auto-dismiss + deduplication | +| `GlobalNotificationContext` | In-app persistent notifications for Global Admins | +| `SidebarContext` | Sidebar expanded/collapsed state | | `ThemeContext` | Dark mode toggle (localStorage) | ### API Communication - `lib/authenticatedFetch.ts` – Wraps `fetch()` with Bearer token, 401 retry with token refresh +- `lib/api.ts` – Typed URL builder for all backend endpoints - `hooks/useAuthenticatedFetch.ts` – React hook with loading/error state - Tenant isolation: All endpoints append `?tenantId={id}` @@ -381,12 +508,20 @@ React Context API (no Redux/Zustand): | Component | Location | Purpose | |-----------|----------|---------| +| `GlobalSidebar` | `components/` | Main navigation sidebar | +| `Navbar` | `components/` | Top navigation bar | +| `GlobalSearch` | `components/` | Cross-session search overlay | +| `ProtectedRoute` | `components/` | Auth guard with role-based access | +| `FeedbackBubble` | `components/` | In-app feedback button | | `SessionTable` | `dashboard/components/` | Paginated, filterable, sortable session list | | `EventTimeline` | `sessions/[id]/components/` | Phase-grouped event visualization | | `PhaseTimeline` | `sessions/[id]/components/` | Visual phase progress with live activity | -| `ProtectedRoute` | `components/` | Auth guard with role-based access | +| `VulnerabilityReportSection` | `sessions/[id]/components/` | CVE findings display | +| `AnalysisResultsSection` | `sessions/[id]/components/` | Analyze rule results | | `ScriptExecutions` | `components/` | Script output viewer | | `PerformanceChart` | `components/` | Time-series metrics chart | +| `ValidationIndicator` | `components/` | Device validation status | +| `AppInsightsInit` | `components/` | Azure Application Insights initialization | ### UI Patterns @@ -406,51 +541,64 @@ React Context API (no Redux/Zustand): | Namespace | Key Classes | Purpose | |-----------|-------------|---------| -| `Models/Enrollment/` | `EnrollmentEvent`, `SessionRegistration`, `EnrollmentPhase`, `BootstrapSession` | Core enrollment data | -| `Models/Config/` | `AgentConfigResponse`, `CollectorConfiguration`, `TenantConfiguration`, `AdminConfiguration` | Configuration hierarchy | +| `Models/Enrollment/` | `EnrollmentEvent`, `SessionRegistration`, `EnrollmentPhase`, `EventSeverity`, `BootstrapSession` | Core enrollment data | +| `Models/Config/` | `AgentConfigResponse`, `CollectorConfiguration`, `AnalyzerConfiguration`, `TenantConfiguration`, `AdminConfiguration`, `McpAccessPolicy`, `PlanTierDefinition` | Configuration hierarchy | | `Models/Rules/` | `GatherRule`, `AnalyzeRule`, `ImeLogPattern`, `RuleResult` | Rules engine | -| `Models/Metrics/` | `UsageMetrics`, `PlatformStats`, `AppInstallSummary` | Analytics | -| `Models/Diagnostics/` | `DiagnosticsLogPath`, `AgentErrorReport` | Diagnostics collection | +| `Models/Metrics/` | `UsageMetrics`, `UsageMetricsSnapshot`, `PlatformStats`, `AppInstallSummary` | Analytics | +| `Models/Diagnostics/` | `DiagnosticsLogPath`, `AgentErrorReport`, `DistressReport` | Diagnostics & error reporting | +| `Models/Notifications/` | `NotificationAlert`, `NotificationSeverity`, `WebhookProviderType` | Notification system | +| `DataAccess/` | 13 repository interfaces (`ISessionRepository`, `IConfigRepository`, etc.) | Data access contracts | | `ApiModels.cs` | Request/Response pairs for all endpoints | API contracts | -| `Constants.cs` | Table names, API endpoints, event types, defaults | Shared constants | +| `Constants.cs` | Table names (33), API endpoints, event types (50+), defaults | Shared constants | ### Key Enums - **`EnrollmentPhase`**: Unknown(-1), Start(0), DevicePreparation(1), DeviceSetup(2), AppsDevice(3), AccountSetup(4), AppsUser(5), FinalizingSetup(6), Complete(7), Failed(99) -- **`Severity`**: Debug(0), Info(1), Warning(2), Error(3), Critical(4) -- **50+ Event Types**: `phase_transition`, `app_install_completed`, `enrollment_complete`, `enrollment_failed`, `esp_state_change`, `performance_snapshot`, `script_completed`, `gather_result`, etc. +- **`EventSeverity`**: Trace(-1), Debug(0), Info(1), Warning(2), Error(3), Critical(4) +- **`WebhookProviderType`**: None(0), TeamsLegacyConnector(1), TeamsWorkflowWebhook(2), Slack(10) +- **50+ Event Types**: `phase_transition`, `app_install_completed`, `enrollment_complete`, `enrollment_failed`, `esp_state_change`, `performance_snapshot`, `script_completed`, `gather_result`, `software_inventory_analysis`, etc. --- ## Data Model -### Azure Table Storage (23 Tables) +### Azure Table Storage (33 Tables) | Table | PartitionKey | RowKey | Purpose | |-------|-------------|--------|---------| | `Sessions` | TenantId | SessionId | Enrollment sessions | +| `SessionsIndex` | TenantId | IndexKey | Session search indexes | | `Events` | SessionId | Timestamp_Sequence | Individual events | +| `EventTypeIndex` | TenantId | EventType_Timestamp | Event type search index | | `AdminConfiguration` | "GlobalConfig" | "config" | Platform-wide settings | | `TenantConfiguration` | TenantId | "config" | Per-tenant settings | | `GatherRules` | TenantId | RuleId | Data collection rules | | `AnalyzeRules` | TenantId | RuleId | Issue detection rules | | `ImeLogPatterns` | TenantId | PatternId | IME log regex patterns | | `RuleResults` | TenantId | SessionId_RuleId | Analysis findings | +| `RuleStates` | TenantId | RuleId | Rule enable/disable state | | `UsageMetrics` | TenantId | MetricDate | Daily usage snapshots | | `AppInstallSummaries` | TenantId | SessionId_AppName | Per-app install data | | `PlatformStats` | "Global" | "stats" | Cumulative platform stats | | `AuditLogs` | TenantId | Timestamp_Id | Admin action audit trail | +| `UserActivity` | TenantId | UserId | User login tracking | +| `UserUsageLog` | TenantId | UserId_Timestamp | Per-user API usage tracking | | `BootstrapSessions` | TenantId / "CodeLookup" | ShortCode | OOBE bootstrap tokens | | `BlockedDevices` | TenantId | SerialNumber | Blocked devices | +| `BlockedVersions` | "BlockedVersions" | Pattern | Version block/kill rules | | `SessionReports` | TenantId | ReportId | User-submitted reports | +| `DistressReports` | TenantId | Timestamp_Id | Pre-auth distress signals | | `GlobalAdmins` | "GlobalAdmins" | UPN | Platform-level admins | -| `TenantAdmins` | TenantId | UPN | Tenant-level admins | +| `TenantAdmins` | TenantId | UPN | Tenant-level admins (Admin, Operator, Viewer) | +| `McpUsers` | "McpUsers" | UPN | MCP API users | | `PreviewWhitelist` | "Preview" | TenantId | Preview access gate | -| `UserActivity` | TenantId | UserId | User login tracking | -| `RuleStates` | TenantId | RuleId | Rule enable/disable state | | `PreviewConfig` | "Preview" | "config" | Preview feature config | -| `BlockedVersions` | "BlockedVersions" | Pattern | Version block/kill rules (wildcards, ceilings) | | `GlobalNotifications` | "GlobalNotifications" | InvertedTicks_Id | Persistent in-app notifications for Global Admins | +| `DeviceSnapshot` | TenantId | DeviceSerial | Device hardware/network snapshots | +| `VulnerabilityCache` | CacheType | CacheKey | Cached NVD/MSRC/KEV CVE data | +| `VulnerabilityReports` | TenantId | SessionId | Per-session vulnerability findings | +| `SoftwareInventory` | TenantId | SoftwareName | Aggregated software inventory | +| `CveIndex` | CveId | TenantId_SessionId | CVE → session cross-reference | ### Azure Blob Storage @@ -465,11 +613,13 @@ Session (1) ──► (N) EnrollmentEvent Session (1) ──► (N) RuleResult Session (1) ──► (N) AppInstallSummary Session (1) ──► (0..1) SessionReport +Session (1) ──► (0..1) VulnerabilityReport Session (N) ◄── (1) TenantConfiguration TenantConfiguration (N) ◄── (1) AdminConfiguration (inherits defaults) GatherRule ──► (agent executes) ──► EnrollmentEvent (gather_result) AnalyzeRule ──► (backend evaluates) ──► RuleResult ImeLogPattern ──► (agent matches) ──► EnrollmentEvent (various types) +SoftwareInventory ──► VulnerabilityCorrelation ──► VulnerabilityReport ``` --- @@ -494,11 +644,18 @@ ImeLogPattern ──► (agent matches) ──► EnrollmentEvent (various types - `X-Device-SerialNumber`, `X-Device-Manufacturer`, `X-Device-Model` - Used for whitelist validation and device identification +4. **Agent Version Header** + - `X-Agent-Version` – used for version block enforcement + **Web → Backend (user auth):** - Microsoft Entra ID multi-tenant JWT via `AuthenticationMiddleware` - Dynamic OIDC metadata per tenant (cached 24h) - Claims: `tid` (tenant), `upn` (user), `oid` (object ID) +**MCP → Backend:** +- JWT auth + `X-Client-Source: mcp` header for usage tracking +- Access controlled via `McpUserService` + ### Validation Pipeline (per agent request) ``` @@ -519,7 +676,6 @@ Agent downloads are verified using SHA-256 hashes through two independent channe - CI/CD computes SHA-256 of the agent ZIP after build - Hash is written to `version.json`: `{ "version": "1.0.x", "sha256": "..." }` - Bootstrapper and Self-Updater verify the downloaded ZIP against this hash -- Backward compatible: old agents/bootstrappers without hash checking continue to work **Channel 2: Backend Hash-Oracle (AdminConfiguration)** - CI/CD writes the SHA-256 hash to `AdminConfiguration.LatestAgentSha256` in Table Storage @@ -540,20 +696,37 @@ Bootstrapper: 3. Verify SHA-256: version.json hash > legacy Content-MD5 header > skip (backward compat) ``` +### Authorization Model + +#### Endpoint Access Policies (EndpointAccessPolicyCatalog) + +Six policy tiers (fail-closed: unregistered routes get 403): + +| Policy | Description | +|--------|-------------| +| `PublicAnonymous` | No auth required | +| `DeviceOrBootstrapAuth` | mTLS cert or bootstrap token (via `ValidateSecurityAsync`) | +| `AuthenticatedUser` | Valid JWT, any tenant | +| `MemberRead` | Admin + Operator + Viewer roles | +| `TenantAdminOrGA` | Tenant Admin or Global Admin | +| `BootstrapManagerOrGA` | Bootstrap Manager permission or Global Admin | +| `GlobalAdminOnly` | Platform-wide admin | + +#### Roles + +| Role | Scope | Capabilities | +|------|-------|-------------| +| **Global Admin** | Platform-wide | Global config, all tenants, platform metrics, health checks, MCP users | +| **Tenant Admin** | Single tenant | Tenant config, rules, admin management, device blocking, notifications | +| **Operator** | Single tenant | Write access, optionally Bootstrap Manager permission | +| **Viewer** | Single tenant | Read-only dashboard, session detail view | + ### Tenant Data Isolation - All Table Storage queries filtered by `PartitionKey = TenantId` - SignalR groups: `tenant-{tenantId}` for scoped broadcasts - JWT `tid` claim determines tenant for web requests -### Roles - -| Role | Scope | Capabilities | -|------|-------|-------------| -| **Global Admin** | Platform-wide | Global config, all tenants, platform metrics, health checks | -| **Tenant Admin** | Single tenant | Tenant config, rules, admin management, device blocking | -| **User** | Single tenant | Read-only dashboard, session detail view | - --- ## Real-Time Communication @@ -591,7 +764,7 @@ Bootstrapper: ### Gather Rules -- **Collector Types:** `registry`, `wmi`, `eventlog`, `file`, `command_allowlisted`, `logparser` +- **Collector Types:** `registry`, `wmi`, `eventlog`, `file`, `command_allowlisted`, `logparser`, `json`, `xml` - **Triggers:** `startup`, `interval`, `phase_change`, `on_event` - **Security:** Command allowlist enforced by `GatherRuleGuards` - **Output:** Emits `gather_result` event with collected data @@ -616,6 +789,7 @@ Bootstrapper: 2. GitHub Actions validates against JSON Schema + combines into `rules/dist/` 3. Combined files embedded as resources in Functions assembly 4. Served to agents via `/api/agent/config`, manageable via web UI per tenant +5. `ReseedFromGitHub` function can re-fetch from GitHub on demand --- @@ -639,6 +813,7 @@ Path 3: Desktop Arrival (No-ESP / WDP v2) - **ESP Events:** Shell-Core event log (62404=Hello wizard start, 62407=ESP exit/WhiteGlove) - **Hello Events:** User Device Registration log (300=NGC success, 301=NGC failure) - **Hello Wait:** 30s for wizard start → 300s for completion → timeout +- **HelloOutcome:** Tracked property recording Hello result - **Policy Check:** WHfB policy registry poll every 10s; skip Hello wait if not configured ### Failure Detection @@ -647,6 +822,7 @@ Path 3: Desktop Arrival (No-ESP / WDP v2) - **Recoverable failures** (Timeout) → 60s grace period before marking failed - **Auth failures** → circuit breaker (max 5 attempts or configurable timeout) - **Device-Only ESP:** 5-min timer after DeviceSetup exit; if no AccountSetup → device-only classification +- **WDP v2 gate skip:** `_enrollmentType == "v2"` → desktop arrival gate skipped (no ESP in WDP v2) ### WhiteGlove (Pre-Provisioning) @@ -663,6 +839,10 @@ Path 3: Desktop Arrival (No-ESP / WDP v2) | `session.id` / `session.seq` | Session identity + event sequence counter | | `enrollment-complete.marker` | Cleanup retry flag if previous cleanup failed | +### Signal Audit Trail + +Terminal events (`enrollment_complete`/`enrollment_failed`) include `signalsSeen` and `signalTimestamps` for full state machine transparency. Completion check events are throttled (1x/min/source) with full state machine snapshot. + ### Cleanup & Self-Destruct 1. Stop collectors, run shutdown analyzers @@ -681,6 +861,7 @@ AdminConfiguration (global, single row in Azure Table) │ CollectorIdleTimeoutMinutes (default 15) │ AgentMaxLifetimeMinutes (default 360) │ DiagnosticsGlobalLogPathsJson + │ LatestAgentSha256 │ └──► TenantConfiguration (per tenant, inherits/overrides) │ Rate limiting (override or inherit global) @@ -688,16 +869,18 @@ AdminConfiguration (global, single row in Azure Table) │ Collector intervals, Hello timeout │ Diagnostics: UploadEnabled, LogPathsJson │ Auth circuit breaker settings - │ Teams/Telegram notifications + │ Teams/Slack/Telegram notifications │ Bootstrap token enablement + │ UnrestrictedMode │ └──► AgentConfigResponse (delivered to agent via /api/agent/config) - │ ConfigVersion (currently 16) + │ ConfigVersion (currently 21) │ CollectorConfiguration (nested) │ AnalyzerConfiguration (nested) │ GatherRules[] (merged built-in + tenant) │ ImeLogPatterns[] (merged built-in + tenant) │ DiagnosticsLogPaths[] (merged global + tenant) + │ LatestAgentSha256 └ Various flags and intervals ``` @@ -736,6 +919,98 @@ AdminConfiguration (global, single row in Azure Table) --- +## Vulnerability Management + +### Architecture + +Software inventory collected by agent → correlated with CVE databases on backend → vulnerability reports per session. + +### Data Sources + +| Source | Service | Purpose | +|--------|---------|---------| +| NIST NVD | `NvdApiClient` | CVE database with CPE matching | +| Microsoft MSRC | `MsrcApiClient` | Microsoft-specific security advisories | +| CISA KEV | `KevDataService` | Known Exploited Vulnerabilities feed | + +### Flow + +1. `SoftwareInventoryAnalyzer` captures installed software at agent start+shutdown +2. Events uploaded via normal ingest pipeline +3. `VulnerabilityCorrelationService` matches software against CVE data using CPE identifiers +4. `VersionComparer` handles semantic version comparison for affected version ranges +5. Results stored in `VulnerabilityReports` table, viewable in session detail UI +6. Custom CPE mappings configurable via admin UI for unmatched software +7. `VulnerabilityDataSyncFunction` periodically refreshes CVE cache + +### Search + +Sessions can be searched by CVE ID via `GET /api/search/sessions-by-cve`, backed by the `CveIndex` table. + +--- + +## Notification System + +### Webhook Notifications + +Enrollment completion/failure events trigger webhook notifications via `WebhookNotificationService`. + +| Provider | Renderer | Format | +|----------|----------|--------| +| Teams Legacy Connector | `LegacyTeamsConnectorRenderer` | Office 365 Connector card | +| Teams Workflow Webhook | `TeamsWorkflowAdaptiveCardRenderer` | Adaptive Card | +| Slack | `SlackRenderer` | Block Kit | + +Configuration per tenant via `TenantConfiguration.WebhookUrl` + `WebhookProviderType`. + +### Telegram Notifications + +`TelegramNotificationService` sends enrollment notifications via Telegram bot API. Configured per tenant. + +### Email Notifications + +`ResendEmailService` handles transactional emails via Resend.com API (used for Preview notifications). + +### In-App Notifications + +`GlobalNotificationService` provides persistent notifications for Global Admins, stored in `GlobalNotifications` table. Survives page reloads, dismissable per user. + +--- + +## Testing + +### Backend Tests (`src/Backend/AutopilotMonitor.Functions.Tests/`) + +**Target:** .NET 8.0, xUnit + +| Test File | Coverage | +|-----------|---------| +| `IngestCriticalPathTests` | Regression guard for `StampServerFields()` — ensures `ReceivedAt`, `TenantId`, `SessionId` are stamped | +| `SecurityValidatorTests` | Certificate validation, rate limiting, hardware whitelist, device validation flows | +| `DistressValidationTests` | Distress report validation | +| `DistressRateLimitServiceTests` | Distress rate limiting | +| `EventTimestampValidationTests` | Timestamp clamping and validation | +| `NdjsonParserTests` | NDJSON parsing | +| `BuiltInRulesTests` | Built-in rule logic | +| `EndpointPolicyCatalogCompletenessTests` | Ensures every HTTP route has a catalog entry | + +### Agent Tests (`src/Agent/AutopilotMonitor.Agent.Core.Tests/`) + +**Target:** .NET Framework 4.8, xUnit + +| Test File | Coverage | +|-----------|---------| +| `DesktopArrivalDetectorTests` | Desktop arrival detection | +| `DiagnosticsPathGuardsTests` | Path guard validation | +| `GatherRuleGuardsTests` | Gather rule path/target guards | +| `AppPackageStateListTests` | App package state tracking | +| `AppPackageStateTests` | Individual package state machine | +| `CmTraceLogParserTests` | CMTrace log format parsing | +| `EnrollmentStatePersistenceTests` | State file persistence | +| `LogFilePositionTrackerTests` | Log file tail position bookmarking | + +--- + ## Infrastructure & Deployment ### Azure Resources @@ -743,13 +1018,41 @@ AdminConfiguration (global, single row in Azure Table) | Resource | Purpose | |----------|---------| | Azure Functions (Isolated Worker) | REST API, event processing, timer triggers | -| Azure Table Storage | 21 tables for sessions, events, config, rules, metrics | -| Azure Blob Storage | Diagnostics ZIPs, session reports, platform stats cache | +| Azure Table Storage | 33 tables for sessions, events, config, rules, metrics, vulnerability data | +| Azure Blob Storage | Diagnostics ZIPs, session reports, platform stats cache, agent binaries | | Azure SignalR Service | WebSocket hub for real-time updates | | Azure Static Web Apps | Next.js frontend hosting | +| Azure Container App | MCP server (provisioned via `infra/mcp-server.bicep`) | +| Azure Container Registry | MCP server container images | | Application Insights | Logging, telemetry, performance monitoring | | Microsoft Entra ID | Multi-tenant OIDC authentication | +### MCP Server + +Provisioned via Bicep (`infra/mcp-server.bicep`): +- Azure Container Registry (Basic SKU) +- Log Analytics Workspace +- Container App Environment +- Container App (`autopilotmonitor-mcp`) running from ACR image + +### Bootstrap Deployment + +`scripts/Bootstrap/Install-AutopilotMonitor.ps1` — Intune Platform Script (.ps1): +- Deployed early in Autopilot enrollment via Intune +- Downloads agent ZIP from blob storage, verifies SHA-256 integrity +- Runs `AutopilotMonitor.Agent.exe --install` +- Multi-signal guard: registry marker, OOBE state, WMI/filesystem user profile, 12h bootstrap window (prevents ghost sessions) +- `Test-ShouldBootstrapAgent.ps1` — standalone test for guard logic +- `Uninstall-AutopilotMonitor.ps1` — manual cleanup + +### Agent Build & Upload + +`scripts/Deployment/build_and_upload_release_agent_build.ps1`: +- Builds agent solution in Release mode +- Creates ZIP package +- Computes SHA-256 hash +- Uploads ZIP + `version.json` (with hash) to Azure Blob Storage + ### CI/CD (GitHub Actions) | Workflow | Trigger | Purpose | @@ -780,6 +1083,9 @@ dotnet build AutopilotMonitor.sln --nologo -v quiet # Backend tests dotnet test src/Backend/AutopilotMonitor.Functions.Tests/ +# Agent tests +dotnet test src/Agent/AutopilotMonitor.Agent.Core.Tests/ + # Web frontend cd src/Web/autopilot-monitor-web npm install @@ -796,16 +1102,15 @@ cd rules && node scripts/combine.js ``` --install Deploy via Scheduled Task (Intune package) --run-gather-rules One-shot data collection, then exit +--run-ime-matching Offline IME log parsing, write ime_pattern_matching.log --console Enable console output ---cert-thumbprint {thumb} Override cert search ---tenant-id {id} Override registry tenant +--log-level {level} Set log level (debug/info/warning/error) --api-url {url} Override API endpoint (alias: --backend-api) --bootstrap-token {token} Pre-MDM bootstrap auth --ime-log-path {path} Override IME log folder --ime-match-log {path} Write matched IME log lines to file (debug) --replay-log-dir {path} Enable log replay mode --replay-speed-factor {n} Compression factor (default 50) ---no-auth Disable cert auth --no-cleanup Disable self-destruct --reboot-on-complete Trigger reboot after enrollment --new-session Start fresh session @@ -820,7 +1125,8 @@ cd rules && node scripts/combine.js | Convention | Details | |------------|---------| | Agent endpoint security | All agent endpoints use `req.ValidateSecurityAsync()` from `SecurityValidationExtensions.cs` | -| ConfigVersion | Tracks agent capability level (currently 16 = trace events for decision auditing) | +| Route policy catalog | Every HTTP route MUST be registered in `EndpointAccessPolicyCatalog` (fail-closed → 403) | +| ConfigVersion | Tracks agent capability level (currently 21 = NTP time check + timezone auto-set) | | Phase progression | Forward-only: DeviceSetup(1) → AccountSetup(2), no backward transitions | | Phase isolation | App IDs seen in earlier phases are ignored in later phases (IME tracker) | | Completion throttling | Max 1 `completion_check` event per source per minute | @@ -828,3 +1134,5 @@ cd rules && node scripts/combine.js | Settings UI | `error` + `successMessage` state for notifications at top of `
` | | Maintenance timer | Runs every 2 hours (not daily, despite function name) | | Agent versioning | Auto-incremented: 1.0.{BuildNumber} | +| Timestamp clamping | Preserves originals + flags for troubleshooting | +| Bootstrap scripts | Must be pure ASCII (no em-dashes/Unicode) — PS 5.1 compatibility | diff --git a/scripts/Bootstrap/Install-AutopilotMonitor.ps1 b/scripts/Bootstrap/Install-AutopilotMonitor.ps1 index b369535d..26c953a7 100644 --- a/scripts/Bootstrap/Install-AutopilotMonitor.ps1 +++ b/scripts/Bootstrap/Install-AutopilotMonitor.ps1 @@ -39,8 +39,8 @@ .CHANGELOG 2026-03-31 Replaced OS age + MDM pre-flight checks with multi-signal guard: - registry deployment marker, OOBE state, WMI/filesystem user profile - detection, and 12h bootstrap window. Fixes ghost sessions on hybrid + registry deployment marker, WMI/filesystem user profile detection, + and 12h bootstrap window. OOBEInProgress dropped (unreliable). Fixes ghost sessions on hybrid join pre-provisioned devices where Intune re-targets the bootstrap script after self-destruct. 2026-03-30 Fixed non-ASCII characters (em-dashes, Unicode symbols) that broke @@ -111,10 +111,10 @@ try { # -- Pre-flight: Multi-signal guard to prevent installation on non-provisioning devices -- # Layered approach: each guard catches a different scenario. # Guard 1: Ghost re-installs (registry marker from previous deployment) - # Guard 2: Productive devices not in OOBE - # Guard 3: Productive devices with stale OOBEInProgress flag (real user profile exists) - # Guard 4: Ultra edge case (no key, OOBE stuck, no user, but device running too long) - # Guard 5: Agent binary already present from a previous run + # Guard 2: Productive devices (real user profile exists — WMI + filesystem) + # Guard 3: Bootstrap window expired (device uptime > 12h without agent) + # Guard 4: Agent binary already present from a previous run + # NOTE: OOBEInProgress is NOT used — it is unreliable (observed =0 during active enrollment). # Guard 1: Agent was already deployed on this device (registry marker survives self-destruct) $deployed = (Get-ItemProperty -Path 'HKLM:\SOFTWARE\AutopilotMonitor' -Name 'Deployed' -ErrorAction SilentlyContinue).Deployed @@ -123,15 +123,7 @@ try { exit 0 } - # Guard 2: Device must be in OOBE (= actively provisioning) - $oobeInProgress = (Get-ItemProperty -Path 'HKLM:\SYSTEM\Setup' -Name 'OOBEInProgress' -ErrorAction SilentlyContinue).OOBEInProgress - Write-Log "OOBEInProgress=$oobeInProgress" - if ($oobeInProgress -ne 1) { - Write-Log "SKIP: Device is not in OOBE (OOBEInProgress=$oobeInProgress). Not a fresh enrollment." - exit 0 - } - - # Guard 3: No real user profile should exist yet (catches stale OOBEInProgress on productive devices) + # Guard 2: No real user profile should exist yet (primary productive-device guard) # Combines WMI (Win32_UserProfile.Special flag) and filesystem for maximum reliability. $excludePattern = '^(defaultuser\d*|Public|Default( User)?|All Users)$' @@ -154,7 +146,7 @@ try { exit 0 } - # Guard 4: Bootstrap window check (ultra edge case: no key, OOBE stuck, no user profiles) + # Guard 3: Bootstrap window check (no key, no user profiles, but device running too long) # NOT "how long may enrollment take" -- agent handles that internally (6h emergency break). # This is "how old can the OOBE state be before I no longer trust it for initial install". # 12h bootstrap window vs 6h agent emergency break = consistent layered approach. @@ -167,7 +159,7 @@ try { exit 0 } - # Guard 5: Is the agent already installed? (leftover from previous run) + # Guard 4: Is the agent already installed? (leftover from previous run) if (Test-Path $AgentBinPath) { $existingAgent = Get-ChildItem -Path $AgentBinPath -Filter "AutopilotMonitor.Agent.exe" -ErrorAction SilentlyContinue if ($existingAgent) { diff --git a/scripts/Bootstrap/Test-ShouldBootstrapAgent.ps1 b/scripts/Bootstrap/Test-ShouldBootstrapAgent.ps1 index faa810cd..48a3f546 100644 --- a/scripts/Bootstrap/Test-ShouldBootstrapAgent.ps1 +++ b/scripts/Bootstrap/Test-ShouldBootstrapAgent.ps1 @@ -58,22 +58,13 @@ if ($deployed) { Write-Step -Status 'PASS' -Message 'Guard 1: No previous deployment marker found.' } -# Guard 2: Device must be in OOBE (= actively provisioning) -$oobeInProgress = Get-RegistryValueSafe -Path 'HKLM:\SYSTEM\Setup' -Name 'OOBEInProgress' -if ($oobeInProgress -ne 1) { - Write-Step -Status 'SKIP' -Message "Guard 2: Device is not in OOBE (OOBEInProgress=$oobeInProgress). Not a fresh enrollment." - $wouldInstall = $false - $reasons.Add("Not in OOBE (OOBEInProgress=$oobeInProgress)") -} else { - Write-Step -Status 'PASS' -Message 'Guard 2: Device is in OOBE (OOBEInProgress=1).' -} - -# Guard 3: No real user profile should exist yet +# Guard 2: No real user profile should exist yet (primary productive-device guard) +# NOTE: OOBEInProgress is NOT used -- it is unreliable (observed =0 during active enrollment). $excludePattern = '^(defaultuser\d*|Public|Default( User)?|All Users)$' $wmiProfileQueryFailed = $false $profileNames = @( - # WMI/CIM view + # WMI/CIM view -- Special flag reliably excludes SYSTEM/LocalService/NetworkService try { Get-CimInstance Win32_UserProfile -ErrorAction Stop | Where-Object { @@ -88,7 +79,7 @@ $profileNames = @( $wmiProfileQueryFailed = $true } - # Filesystem view + # Filesystem view -- catches profiles WMI might miss (Get-ChildItem 'C:\Users' -Directory -ErrorAction SilentlyContinue).Name ) | Where-Object { @@ -97,23 +88,23 @@ Where-Object { Select-Object -Unique if ($wmiProfileQueryFailed) { - Write-Step -Status 'WARN' -Message 'Guard 3: WMI profile query failed, filesystem check still applied.' + Write-Step -Status 'WARN' -Message 'Guard 2: WMI profile query failed, filesystem check still applied.' } if ($profileNames) { $names = ($profileNames | Select-Object -First 3) -join ', ' - Write-Step -Status 'SKIP' -Message "Guard 3: Real user profile(s) found ($names). Device appears productive." + Write-Step -Status 'SKIP' -Message "Guard 2: Real user profile(s) found ($names). Device appears productive." $wouldInstall = $false $reasons.Add("Real user profile(s) found: $names") } else { - Write-Step -Status 'PASS' -Message 'Guard 3: No real user profiles found (combined WMI/filesystem view).' + Write-Step -Status 'PASS' -Message 'Guard 2: No real user profiles found (combined WMI/filesystem view).' } -# Guard 4: Bootstrap window check -# NOT "how long may enrollment take" — agent handles that internally (6h emergency break). +# Guard 3: Bootstrap window check +# NOT "how long may enrollment take" -- agent handles that internally (6h emergency break). # This is "how old can the OOBE state be before I no longer trust it for initial install". # 12h bootstrap window vs 6h agent emergency break = consistent layered approach. -# Sleep/standby does NOT reset uptime — only real boot/restart does. +# Sleep/standby does NOT reset uptime -- only real boot/restart does. $lastBoot = $null $uptimeHours = $null @@ -122,24 +113,24 @@ try { $uptimeHours = ((Get-Date) - $lastBoot).TotalHours if ($uptimeHours -gt $MaxBootstrapWindowHours) { - Write-Step -Status 'SKIP' -Message "Guard 4: Device uptime is $([int]$uptimeHours)h. OOBE state is older than accepted bootstrap window of ${MaxBootstrapWindowHours}h." + Write-Step -Status 'SKIP' -Message "Guard 3: Device uptime is $([int]$uptimeHours)h. Older than accepted bootstrap window of ${MaxBootstrapWindowHours}h." $wouldInstall = $false $reasons.Add("Uptime exceeds bootstrap window: $([int]$uptimeHours)h > ${MaxBootstrapWindowHours}h") } else { - Write-Step -Status 'PASS' -Message "Guard 4: Device uptime is $([int]$uptimeHours)h and within bootstrap window of ${MaxBootstrapWindowHours}h." + Write-Step -Status 'PASS' -Message "Guard 3: Device uptime is $([int]$uptimeHours)h and within bootstrap window of ${MaxBootstrapWindowHours}h." } } catch { - Write-Step -Status 'WARN' -Message 'Guard 4: Could not determine LastBootUpTime / uptime.' + Write-Step -Status 'WARN' -Message 'Guard 3: Could not determine LastBootUpTime / uptime.' } -# Guard 5 (existing): Agent binary already present +# Guard 4: Agent binary already present if (Test-Path $AgentExePath) { - Write-Step -Status 'SKIP' -Message "Guard 5: Agent already installed at '$AgentExePath'." + Write-Step -Status 'SKIP' -Message "Guard 4: Agent already installed at '$AgentExePath'." $wouldInstall = $false $reasons.Add("Agent binary already present: $AgentExePath") } else { - Write-Step -Status 'PASS' -Message "Guard 5: Agent binary not present at '$AgentExePath'." + Write-Step -Status 'PASS' -Message "Guard 4: Agent binary not present at '$AgentExePath'." } Write-Host '' @@ -166,10 +157,9 @@ if ($reasons.Count -eq 0) { Write-Host '' Write-Host '=== Raw Values ===' Write-Host "Deployed marker : $deployed" -Write-Host "OOBEInProgress : $oobeInProgress" Write-Host "MaxBootstrapWindowHours : $MaxBootstrapWindowHours" Write-Host "LastBootUpTime : $lastBoot" Write-Host "UptimeHours : $([string]$uptimeHours)" Write-Host "AgentExePath : $AgentExePath" Write-Host "DetectedProfileNames : $($profileNames -join ', ')" -Write-Host '' \ No newline at end of file +Write-Host ''