diff --git a/docs/qualification/PATROL_ASSISTANT_CUSTOMER_JOURNEY.md b/docs/qualification/PATROL_ASSISTANT_CUSTOMER_JOURNEY.md index f25d7e36b..9e79b80fb 100644 --- a/docs/qualification/PATROL_ASSISTANT_CUSTOMER_JOURNEY.md +++ b/docs/qualification/PATROL_ASSISTANT_CUSTOMER_JOURNEY.md @@ -1248,4 +1248,45 @@ Raw log: `/opt/pulse-release-worker/patrol-dependency-action-oracles-live.log`, copied to `tmp/patrol-dependency-action-oracles/` at the workspace root. The live test source SHA-256 is `78b72dc44231cd3ecbd0b2ee925d53a5836af3141e695152046aa032580f1e48`. -The ordinary qualification and CLI packages pass in 4.224s and 0.008s on Go 1.26.8 with the explicit live environment unset. The test hash is unchanged. The exact staged hook remains pending for this slice. +The ordinary qualification and CLI packages pass in 4.224s and 0.008s on Go 1.26.8 with the explicit live environment unset. The test hash is unchanged. Commit `173d74a8e422` is pushed to PR #1928 after the exact four-file staged hook passed all 163 tests in 129.364s with unchanged source hashes. Remote landing remains pending. + + +### Current ordinary Assistant diagnosis remains unqualified + +One read-only ordinary Assistant request on 2026-09-06 ran from +06:49:50.441Z to 06:53:26.724Z, taking 216.283s with 15 tool calls. The +configured `claude-subscription:claude-opus-5` route returned HTTP 200. The +request explicitly used `autonomous_mode=false`. No alternate paid provider, +Patrol readiness retry or infrastructure mutation was performed. This single +request establishes neither an aggregate success rate nor a latency SLO. + +The missing-access correction works in this real run. One diagnostic read +returns failed `NO_AGENT` while preserving the monitored node identity, and +the answer correctly retains that access limit without retrying the read. +Diagnosis still fails factual review. The answer quotes the tool contract that +returned timestamps cannot establish collection uptime or missing-history cause, +then claims that a coincident alert timestamp identifies observation start. It +calls 181 seven-day returned points hourly even though their returned span is +under twelve hours. It moves the 83.3% memory minimum from the previous evening +at 20:34 to around 04:20, and uses low average CPU as evidence against pressure +without direct pressure measurements. Correct tool metadata did not prevent +these unsupported interpretations. Do not count a completed request or an +accurate missing-access message as successful diagnosis. + +Private receipts, persisted tool outputs, request/source/binary bindings and +factual evaluation are at workspace-relative +`tmp/patrol-current-assistant-check/`. The backend binary hash remains +`1e1c3a9c0c758a4a751afa16200c6fba793acc93fd56ea62e7eb432edcf5f6c8` +before and after the request. The Playwright run exercised the persisted answer +and expanded failed-tool input/output at `/patrol`, 1440x1000 and 390x1000, +including reload. Pixel inspection confirms readable answer and error evidence. +The test intercepted non-GET requests other than the one ordinary chat. Its +blocked readiness check caused the selected-route warning in the screenshots, +so those screenshots do not establish the route's unmodified readiness UI. + +The supported alternate-provider test remains awaiting its separately billed +US$20 maximum approval. The cached subscription refusal for autonomous Patrol +remains enforced. Further prompt or orchestration rules are not justified merely +because this response ignored already explicit evidence limitations. Real-model +diagnosis, approved/rejected action outcomes and wider customer readiness remain +open qualification requirements. diff --git a/docs/release-control/v6/internal/status.json b/docs/release-control/v6/internal/status.json index d8a3d2068..298d13508 100644 --- a/docs/release-control/v6/internal/status.json +++ b/docs/release-control/v6/internal/status.json @@ -10201,7 +10201,7 @@ }, { "id": "patrol-assistant-customer-outcome-qualification", - "summary": "The explicit Patrol/Assistant redesign goal, product contract and ordered source-bound proof remain in docs/qualification/PATROL_ASSISTANT_CUSTOMER_JOURNEY.md. Observations, hypotheses, proposals, executions and independently verified outcomes remain distinct. Model judgment owns diagnosis. Assistant continues the same issue and governed action records. The recorded 2026-09-05 baseline has 127 paid installs, 71 with Patrol enabled and 23 with Assistant calls. Fourteen verified resolutions came from one install, and schema 17 outcome/provider/cost fields had no adoption. Usage is not evidence of useful linked tasks. Implemented corrections preserve canonical risk, source/time/history evidence, failed-read status and diagnostic continuity while removing proposal-as-root-cause promotion, duplicate causal inference, contextless evaluations, quotas and call-count completion proxies. Named evidence and linked-issue browser matrices pass. The latest ordinary model diagnosis still makes unsupported temporal claims and remains unqualified. Claude Max explicitly refused autonomous readiness. Cached refusal and API409 enforcement remain intact. Ordinary Assistant success is not autonomy qualification. Alternate paid-provider approval remains pending, with no paid request or production mutation. The service-storage fault/oracle independently verifies bounded ENOSPC, filesystem exhaustion, unhealthy service with healthy control, recovery, refused overwrite/symlink writes and unchanged pre-existing inventory. This qualifies the oracle, not diagnosis. All twelve manifests and five full-schema regression tests pass, and the published qualification guide passes desktop/narrow browser proof. Missing-access, broader storage/backup and approved/rejected action outcomes still require full real-model qualification. Integrated CI exposed retained-query performance and disk-probe completion defects. The disk race fix b964eea767 and named-binding fix 1f41fa174d are committed and pushed to open PR #1928, each after a 163-test exact staged hook. Full affected packages and focused race proofs pass. Ten paired final-source worker benchmark rounds pass the unchanged time/bytes/allocation checker against exact base 3347, including history/chart APIs and a memory-fallback control. The 500-node read has +0.26% allocations and -1.68% bytes with no significant runtime difference. Remote CI and landing remain open. Current missing-access work reproduces topology loss with zero command connections and preserves known resource kind, parent and routing without dispatch. Reads and file mutations use one failed NO_AGENT envelope instead of installation speculation or successful write results. No-target cardinality and hostname collision boundaries have regression coverage. Full tools proof passes on Go 1.26.8 in 59.437s. The exact five-test race proof passes in 1.051s and existing monitoring-token/WebSocket execution-scope tests pass in 0.122s. The final 18-case scripted browser matrix passes at /patrol, 1440/900/390x1000, including current handler errors, investigation expansion, linked Assistant details, dismissal and reload without resubmission. This is rendering/access-boundary proof, not model diagnosis. Missing-access commit58caeda69ba3 is pushed after the exact eight-file staged hook passed 163 tests in 127.706s with unchanged hashes. On head1f41, governance, all eight Core E2E shards and CodeQL pass, but Build and Test run34015148620 is pending with no jobs. Missing PR checks are not evidence of success. Existing diagnostic permission controls and advertised hypervisor action authority remain distinct. Exact receipts and historical failed/discarded runs are retained in the plan. Independent volunteered Pro environments remain a separate wider-readiness gate. The opt-in live dependency/restart oracle now passes all four checked-in dependency and approved/rejected/autonomous service fault contracts in 65.080s on Go1.26.8. It proves injected state, unchanged fault after refused duplicate injection, explicit fixture recovery and restored baseline. All four two-pass cleanups preserve pre-existing Docker inventory. It makes no Pulse/model request and does not qualify approval, rejection or action outcomes. The new oracle test source and raw receipts are recorded in the plan. Ordinary qualification and CLI package proof passes in 4.224s and 0.008s with the live environment unset and unchanged test hash. Exact staged-hook proof for this test slice remains pending.", + "summary": "The explicit Patrol/Assistant redesign goal, product contract and ordered source-bound proof remain in docs/qualification/PATROL_ASSISTANT_CUSTOMER_JOURNEY.md. Observations, hypotheses, proposals, executions and independently verified outcomes remain distinct. Model judgment owns diagnosis. Assistant continues the same issue and governed action records. The recorded 2026-09-05 baseline has 127 paid installs, 71 with Patrol enabled and 23 with Assistant calls. Fourteen verified resolutions came from one install, and schema 17 outcome/provider/cost fields had no adoption. Usage is not evidence of useful linked tasks. Implemented corrections preserve canonical risk, source/time/history evidence, failed-read status and diagnostic continuity while removing proposal-as-root-cause promotion, duplicate causal inference, contextless evaluations, quotas and call-count completion proxies. Named evidence and linked-issue browser matrices pass. The latest ordinary model diagnosis still makes unsupported temporal claims and remains unqualified. Claude Max explicitly refused autonomous readiness. Cached refusal and API409 enforcement remain intact. Ordinary Assistant success is not autonomy qualification. Alternate paid-provider approval remains pending, with no paid request or production mutation. The service-storage fault/oracle independently verifies bounded ENOSPC, filesystem exhaustion, unhealthy service with healthy control, recovery, refused overwrite/symlink writes and unchanged pre-existing inventory. This qualifies the oracle, not diagnosis. All twelve manifests and five full-schema regression tests pass, and the published qualification guide passes desktop/narrow browser proof. Missing-access, broader storage/backup and approved/rejected action outcomes still require full real-model qualification. Integrated CI exposed retained-query performance and disk-probe completion defects. The disk race fix b964eea767 and named-binding fix 1f41fa174d are committed and pushed to open PR #1928, each after a 163-test exact staged hook. Full affected packages and focused race proofs pass. Ten paired final-source worker benchmark rounds pass the unchanged time/bytes/allocation checker against exact base 3347, including history/chart APIs and a memory-fallback control. The 500-node read has +0.26% allocations and -1.68% bytes with no significant runtime difference. Remote CI and landing remain open. Current missing-access work reproduces topology loss with zero command connections and preserves known resource kind, parent and routing without dispatch. Reads and file mutations use one failed NO_AGENT envelope instead of installation speculation or successful write results. No-target cardinality and hostname collision boundaries have regression coverage. Full tools proof passes on Go 1.26.8 in 59.437s. The exact five-test race proof passes in 1.051s and existing monitoring-token/WebSocket execution-scope tests pass in 0.122s. The final 18-case scripted browser matrix passes at /patrol, 1440/900/390x1000, including current handler errors, investigation expansion, linked Assistant details, dismissal and reload without resubmission. This is rendering/access-boundary proof, not model diagnosis. Missing-access commit58caeda69ba3 is pushed after the exact eight-file staged hook passed 163 tests in 127.706s with unchanged hashes. On head1f41, governance, all eight Core E2E shards and CodeQL pass, but Build and Test run34015148620 is pending with no jobs. Missing PR checks are not evidence of success. Existing diagnostic permission controls and advertised hypervisor action authority remain distinct. Exact receipts and historical failed/discarded runs are retained in the plan. Independent volunteered Pro environments remain a separate wider-readiness gate. The opt-in live dependency/restart oracle now passes all four checked-in dependency and approved/rejected/autonomous service fault contracts in 65.080s on Go1.26.8. It proves injected state, unchanged fault after refused duplicate injection, explicit fixture recovery and restored baseline. All four two-pass cleanups preserve pre-existing Docker inventory. It makes no Pulse/model request and does not qualify approval, rejection or action outcomes. The new oracle test source and raw receipts are recorded in the plan. Ordinary qualification and CLI package proof passes in 4.224s and 0.008s with the live environment unset and unchanged test hash. Oracle commit173d74a8e422 is pushed after its exact four-file staged hook passed 163 tests in 129.364s with unchanged hashes. The current real ordinary Assistant request takes 216.283s with 15 tool calls and one failed read. Known identity and the access limit survive correctly, but the answer contradicts explicit timestamp semantics, mislabels mixed retained points as hourly and misplaces a memory extremum. This remains failed diagnostic qualification. Source/binary-bound raw evidence and the browser-interception limitation are recorded in the plan. No autonomous retry or paid-provider request occurred. Exact-head remote CI and landing remain pending.", "owner": "project-owner", "status": "planned", "recorded_at": "2026-09-05",