From d92ce0c54bb07a82a01641a2b48e32958d45cd96 Mon Sep 17 00:00:00 2001 From: rcourtman Date: Fri, 10 Apr 2026 23:58:54 +0100 Subject: [PATCH] Anchor trial retry-after UI proof --- .../HIGH_RISK_RELEASE_VERIFICATION_MATRIX.md | 5 ++++- .../v6/internal/subsystems/cloud-paid.md | 11 +++++++++-- .../subsystems/deployment-installability.md | 14 +++++++++----- scripts/tests/test-hot-dev-bg.sh | 1 + scripts/tests/test-trial-signup-docs.sh | 9 ++++++--- tests/integration/README.md | 9 +++++++++ 6 files changed, 38 insertions(+), 11 deletions(-) diff --git a/docs/release-control/v6/internal/HIGH_RISK_RELEASE_VERIFICATION_MATRIX.md b/docs/release-control/v6/internal/HIGH_RISK_RELEASE_VERIFICATION_MATRIX.md index c3a512599..324ffa6a0 100644 --- a/docs/release-control/v6/internal/HIGH_RISK_RELEASE_VERIFICATION_MATRIX.md +++ b/docs/release-control/v6/internal/HIGH_RISK_RELEASE_VERIFICATION_MATRIX.md @@ -45,11 +45,14 @@ Companion drill: `go test ./internal/hosted/... -count=1` `cd frontend-modern && npx vitest run src/pages/__tests__/HostedSignup.test.tsx src/components/Settings/__tests__/BillingAdminPanel.test.tsx` `cd tests/integration && PULSE_E2E_USE_LOCAL_BACKEND=1 PULSE_E2E_SKIP_PLAYWRIGHT_INSTALL=1 npm test -- tests/07-trial-signup-return.spec.ts --project=chromium` + `cd tests/integration && PULSE_E2E_USE_LOCAL_BACKEND=1 PULSE_E2E_SKIP_PLAYWRIGHT_INSTALL=1 npm test -- tests/58-self-hosted-trial-rate-limit-ui.spec.ts --project=chromium` The self-hosted handoff proof must keep the canonical trial-start transport contract: `POST /api/license/trial/start` returns `409 trial_signup_required` while the hosted-signup retry burst remains open, then transitions to `429 trial_rate_limited` plus `Retry-After` backoff once - the limiter actually engages. + the limiter actually engages. The Pulse Pro browser CTA proof on + `/settings/system/billing` must surface that same canonical `Retry-After` + backoff even when `details.retry_after_seconds` disagrees. - Live rehearsal helper: `python3 scripts/release_control/hosted_signup_billing_replay_rehearsal.py --base-url --signup-email --org-name ...` - Manual scenario: diff --git a/docs/release-control/v6/internal/subsystems/cloud-paid.md b/docs/release-control/v6/internal/subsystems/cloud-paid.md index 0c58bc386..2ab2ee541 100644 --- a/docs/release-control/v6/internal/subsystems/cloud-paid.md +++ b/docs/release-control/v6/internal/subsystems/cloud-paid.md @@ -592,6 +592,11 @@ That proof must stay aligned in browser and shell coverage too: starts inside the allowed burst as the same hosted-signup redirect contract and only require `trial_rate_limited` plus backoff metadata once the limiter actually engages, rather than assuming the second attempt is already denied. +`tests/integration/tests/58-self-hosted-trial-rate-limit-ui.spec.ts` must then +prove the rendered Pulse Pro CTA on `/settings/system/billing` surfaces the +same canonical `Retry-After` backoff when the backend returns +`trial_rate_limited`, including header precedence over a conflicting +`details.retry_after_seconds` payload. That proof split is also intentional: the browser spec may attach to a reused local instance whose retry bucket is already exhausted and therefore must accept an immediate canonical `429` backoff response, while the snapshot-clean @@ -617,8 +622,10 @@ on a special-case assertion path. The `trial-signup` entry in so eval-pack metadata cannot silently drift away from the canonical hosted handoff contract while the task file stays correct. The release-control high-risk verification matrix must carry those same anchors for the -`tests/07-trial-signup-return.spec.ts` browser proof, so the RC packet cannot -describe trial-start verification through an unanchored summary path. The +`tests/07-trial-signup-return.spec.ts` transport proof and the +`tests/58-self-hosted-trial-rate-limit-ui.spec.ts` Pulse Pro CTA proof, so the +RC packet cannot describe trial-start verification through an unanchored +summary path. The canonical release-control summaries in `docs/release-control/v6/internal/ SOURCE_OF_TRUTH.md` and `docs/release-control/v6/internal/V5_TO_V6_COMMERCIAL_MIGRATION_AUDIT_2026-03-07.md` diff --git a/docs/release-control/v6/internal/subsystems/deployment-installability.md b/docs/release-control/v6/internal/subsystems/deployment-installability.md index 21ecab601..600916730 100644 --- a/docs/release-control/v6/internal/subsystems/deployment-installability.md +++ b/docs/release-control/v6/internal/subsystems/deployment-installability.md @@ -463,11 +463,15 @@ rate limited. The reused-instance browser-proof entry in `POST /api/license/trial/start`, `409 trial_signup_required`, `429 trial_rate_limited`, and `Retry-After`, so the shared trial-start docs guard can auto-discover that README alongside the rest of the governed trial-start -surface instead of relying on README-only fallback checks. The eval-pack -metadata in `tests/integration/evals/scenarios.json` must carry those same -anchors for the `trial-signup` scenario description, so deterministic and -agentic trial runs inherit the same canonical contract wording instead of -teaching a drifted summary path. +surface instead of relying on README-only fallback checks. That README must +also keep the named Pulse Pro browser proof, +`tests/58-self-hosted-trial-rate-limit-ui.spec.ts`, on the owned trial-start +surface so the user-facing retry-after proof does not drift into an orphaned +integration spec. The eval-pack metadata in +`tests/integration/evals/scenarios.json` must carry those same anchors for the +`trial-signup` scenario description, so deterministic and agentic trial runs +inherit the same canonical contract wording instead of teaching a drifted +summary path. Playwright-driven public/commercial specs that support scenario-specific endpoint overrides such as `PULSE_CLOUD_BASE_URL` or `PULSE_COMMERCIAL_BASE_URL` must layer those values through that same shared diff --git a/scripts/tests/test-hot-dev-bg.sh b/scripts/tests/test-hot-dev-bg.sh index d62f646a1..c6cfc9503 100755 --- a/scripts/tests/test-hot-dev-bg.sh +++ b/scripts/tests/test-hot-dev-bg.sh @@ -728,6 +728,7 @@ test_integration_readme_documents_trial_retry_burst_contract() { assert_contains "integration readme documents retry burst exhaustion" "${output}" "retry burst is exhausted" assert_contains "integration readme documents retry-after backoff" "${output}" "Retry-After" assert_contains "integration readme references hosted trial probe script" "${output}" "tests/integration/scripts/trial-signup-contract.sh" + assert_contains "integration readme references pulse pro retry-after ui proof" "${output}" "tests/58-self-hosted-trial-rate-limit-ui.spec.ts" } test_integration_eval_scenario_documents_trial_retry_burst_contract() { diff --git a/scripts/tests/test-trial-signup-docs.sh b/scripts/tests/test-trial-signup-docs.sh index da2eec84e..71a965d1e 100755 --- a/scripts/tests/test-trial-signup-docs.sh +++ b/scripts/tests/test-trial-signup-docs.sh @@ -147,7 +147,7 @@ main() { sed -n '112,128p' "${UPGRADE_DOC}" )" integration_output="$( - sed -n '28,40p' "${INTEGRATION_README}" + sed -n '28,48p' "${INTEGRATION_README}" )" eval_task_output="$( sed -n '1,24p' "${EVAL_TASK_DOC}" @@ -156,10 +156,10 @@ main() { sed -n '24,34p' "${EVAL_SCENARIOS_DOC}" )" high_risk_matrix_output="$( - sed -n '44,54p' "${HIGH_RISK_MATRIX_DOC}" + sed -n '44,58p' "${HIGH_RISK_MATRIX_DOC}" )" api_contracts_output="$( - sed -n '440,448p' "${API_CONTRACTS_DOC}" + sed -n '447,453p' "${API_CONTRACTS_DOC}" )" source_of_truth_output="$( sed -n '412,420p' "${SOURCE_OF_TRUTH_DOC}" @@ -194,6 +194,8 @@ main() { assert_contains "integration readme documents canonical trial-rate-limited response" "${integration_output}" "\`429 trial_rate_limited\`" assert_contains "integration readme documents reused-instance retry-after branch" "${integration_output}" "Retry-After" assert_contains "integration readme documents hosted-signup retry burst" "${integration_output}" "hosted-signup retry-burst contract" + assert_contains "integration readme names pulse pro retry-after ui proof" "${integration_output}" "tests/58-self-hosted-trial-rate-limit-ui.spec.ts" + assert_contains "integration readme documents pulse pro billing route proof" "${integration_output}" "/settings/system/billing" assert_contains "eval task documents retry-burst contract" "${eval_task_output}" "retry-burst contract" assert_contains "eval task documents canonical trial-rate-limited response" "${eval_task_output}" "\`429 trial_rate_limited\`" @@ -208,6 +210,7 @@ main() { assert_contains "high-risk matrix documents hosted-signup redirect code" "${high_risk_matrix_output}" "trial_signup_required" assert_contains "high-risk matrix documents canonical trial-rate-limited response" "${high_risk_matrix_output}" "trial_rate_limited" assert_contains "high-risk matrix documents retry-after metadata" "${high_risk_matrix_output}" "Retry-After" + assert_contains "high-risk matrix names pulse pro retry-after ui proof" "${high_risk_matrix_output}" "tests/58-self-hosted-trial-rate-limit-ui.spec.ts" assert_contains "api contracts document trial-start route" "${api_contracts_output}" "/api/license/trial/start" assert_contains "api contracts document hosted-signup redirect code" "${api_contracts_output}" "trial_signup_required" diff --git a/tests/integration/README.md b/tests/integration/README.md index 4838a45c8..1794c5f13 100644 --- a/tests/integration/README.md +++ b/tests/integration/README.md @@ -32,6 +32,10 @@ End-to-end Playwright tests that validate critical user flows against a running - Verify the response either returns `409 trial_signup_required` with hosted `/start-pro-trial` handoff or `429 trial_rate_limited` with canonical `Retry-After` backoff when the local retry bucket is already exhausted - Verify local entitlements remain unchanged until activation - Verify duplicate initiation stays on the hosted-signup retry-burst contract +- `tests/58-self-hosted-trial-rate-limit-ui.spec.ts` — Pulse Pro trial CTA retry-after UI contract: + - Open `/settings/system/billing` on the real browser shell with free-tier entitlements + - Stub `429 trial_rate_limited` on `POST /api/license/trial/start` + - Verify the Pulse Pro CTA surfaces canonical `Retry-After` guidance even if `details.retry_after_seconds` disagrees - `tests/08-cloud-hosting.spec.ts` — hosted cloud signup contract: - Public `/cloud/signup` form creates a real Stripe sandbox checkout session - Checkout completes and returns to hosted signup completion page @@ -187,6 +191,11 @@ For hosted trial initiation validation against a fresh LXC each run: open, then transition to `429 trial_rate_limited` plus `Retry-After` backoff metadata once the retry burst is exhausted and the limiter actually engages +- Pulse Pro browser proof: `tests/58-self-hosted-trial-rate-limit-ui.spec.ts` + - Exercises `/settings/system/billing` on the real browser shell and proves + the rendered CTA surfaces canonical `Retry-After` guidance when + `trial_rate_limited` is returned, including header precedence over a + conflicting `details.retry_after_seconds` payload - Full sandbox orchestration (multi-tenant + trial + cloud, with per-scenario snapshot reset): - `tests/integration/scripts/run-lxc-sandbox-evals.sh` - Includes hosted trial initiation validation and cloud subscription cancellation lifecycle verification