Anchor trial retry-after UI proof

This commit is contained in:
rcourtman
2026-04-10 23:58:54 +01:00
parent 0a4c84daaa
commit d92ce0c54b
6 changed files with 38 additions and 11 deletions
@@ -45,11 +45,14 @@ Companion drill:
`go test ./internal/hosted/... -count=1`
`cd frontend-modern && npx vitest run src/pages/__tests__/HostedSignup.test.tsx src/components/Settings/__tests__/BillingAdminPanel.test.tsx`
`cd tests/integration && PULSE_E2E_USE_LOCAL_BACKEND=1 PULSE_E2E_SKIP_PLAYWRIGHT_INSTALL=1 npm test -- tests/07-trial-signup-return.spec.ts --project=chromium`
`cd tests/integration && PULSE_E2E_USE_LOCAL_BACKEND=1 PULSE_E2E_SKIP_PLAYWRIGHT_INSTALL=1 npm test -- tests/58-self-hosted-trial-rate-limit-ui.spec.ts --project=chromium`
The self-hosted handoff proof must keep the canonical trial-start transport
contract: `POST /api/license/trial/start` returns `409
trial_signup_required` while the hosted-signup retry burst remains open,
then transitions to `429 trial_rate_limited` plus `Retry-After` backoff once
the limiter actually engages.
the limiter actually engages. The Pulse Pro browser CTA proof on
`/settings/system/billing` must surface that same canonical `Retry-After`
backoff even when `details.retry_after_seconds` disagrees.
- Live rehearsal helper:
`python3 scripts/release_control/hosted_signup_billing_replay_rehearsal.py --base-url <hosted-url> --signup-email <email> --org-name <org> ...`
- Manual scenario:
@@ -592,6 +592,11 @@ That proof must stay aligned in browser and shell coverage too:
starts inside the allowed burst as the same hosted-signup redirect contract and
only require `trial_rate_limited` plus backoff metadata once the limiter
actually engages, rather than assuming the second attempt is already denied.
`tests/integration/tests/58-self-hosted-trial-rate-limit-ui.spec.ts` must then
prove the rendered Pulse Pro CTA on `/settings/system/billing` surfaces the
same canonical `Retry-After` backoff when the backend returns
`trial_rate_limited`, including header precedence over a conflicting
`details.retry_after_seconds` payload.
That proof split is also intentional: the browser spec may attach to a reused
local instance whose retry bucket is already exhausted and therefore must
accept an immediate canonical `429` backoff response, while the snapshot-clean
@@ -617,8 +622,10 @@ on a special-case assertion path. The `trial-signup` entry in
so eval-pack metadata cannot silently drift away from the canonical hosted
handoff contract while the task file stays correct. The release-control
high-risk verification matrix must carry those same anchors for the
`tests/07-trial-signup-return.spec.ts` browser proof, so the RC packet cannot
describe trial-start verification through an unanchored summary path. The
`tests/07-trial-signup-return.spec.ts` transport proof and the
`tests/58-self-hosted-trial-rate-limit-ui.spec.ts` Pulse Pro CTA proof, so the
RC packet cannot describe trial-start verification through an unanchored
summary path. The
canonical release-control summaries in `docs/release-control/v6/internal/
SOURCE_OF_TRUTH.md` and
`docs/release-control/v6/internal/V5_TO_V6_COMMERCIAL_MIGRATION_AUDIT_2026-03-07.md`
@@ -463,11 +463,15 @@ rate limited. The reused-instance browser-proof entry in
`POST /api/license/trial/start`, `409 trial_signup_required`, `429
trial_rate_limited`, and `Retry-After`, so the shared trial-start docs guard
can auto-discover that README alongside the rest of the governed trial-start
surface instead of relying on README-only fallback checks. The eval-pack
metadata in `tests/integration/evals/scenarios.json` must carry those same
anchors for the `trial-signup` scenario description, so deterministic and
agentic trial runs inherit the same canonical contract wording instead of
teaching a drifted summary path.
surface instead of relying on README-only fallback checks. That README must
also keep the named Pulse Pro browser proof,
`tests/58-self-hosted-trial-rate-limit-ui.spec.ts`, on the owned trial-start
surface so the user-facing retry-after proof does not drift into an orphaned
integration spec. The eval-pack metadata in
`tests/integration/evals/scenarios.json` must carry those same anchors for the
`trial-signup` scenario description, so deterministic and agentic trial runs
inherit the same canonical contract wording instead of teaching a drifted
summary path.
Playwright-driven public/commercial specs that support scenario-specific
endpoint overrides such as `PULSE_CLOUD_BASE_URL` or
`PULSE_COMMERCIAL_BASE_URL` must layer those values through that same shared
+1
View File
@@ -728,6 +728,7 @@ test_integration_readme_documents_trial_retry_burst_contract() {
assert_contains "integration readme documents retry burst exhaustion" "${output}" "retry burst is exhausted"
assert_contains "integration readme documents retry-after backoff" "${output}" "Retry-After"
assert_contains "integration readme references hosted trial probe script" "${output}" "tests/integration/scripts/trial-signup-contract.sh"
assert_contains "integration readme references pulse pro retry-after ui proof" "${output}" "tests/58-self-hosted-trial-rate-limit-ui.spec.ts"
}
test_integration_eval_scenario_documents_trial_retry_burst_contract() {
+6 -3
View File
@@ -147,7 +147,7 @@ main() {
sed -n '112,128p' "${UPGRADE_DOC}"
)"
integration_output="$(
sed -n '28,40p' "${INTEGRATION_README}"
sed -n '28,48p' "${INTEGRATION_README}"
)"
eval_task_output="$(
sed -n '1,24p' "${EVAL_TASK_DOC}"
@@ -156,10 +156,10 @@ main() {
sed -n '24,34p' "${EVAL_SCENARIOS_DOC}"
)"
high_risk_matrix_output="$(
sed -n '44,54p' "${HIGH_RISK_MATRIX_DOC}"
sed -n '44,58p' "${HIGH_RISK_MATRIX_DOC}"
)"
api_contracts_output="$(
sed -n '440,448p' "${API_CONTRACTS_DOC}"
sed -n '447,453p' "${API_CONTRACTS_DOC}"
)"
source_of_truth_output="$(
sed -n '412,420p' "${SOURCE_OF_TRUTH_DOC}"
@@ -194,6 +194,8 @@ main() {
assert_contains "integration readme documents canonical trial-rate-limited response" "${integration_output}" "\`429 trial_rate_limited\`"
assert_contains "integration readme documents reused-instance retry-after branch" "${integration_output}" "Retry-After"
assert_contains "integration readme documents hosted-signup retry burst" "${integration_output}" "hosted-signup retry-burst contract"
assert_contains "integration readme names pulse pro retry-after ui proof" "${integration_output}" "tests/58-self-hosted-trial-rate-limit-ui.spec.ts"
assert_contains "integration readme documents pulse pro billing route proof" "${integration_output}" "/settings/system/billing"
assert_contains "eval task documents retry-burst contract" "${eval_task_output}" "retry-burst contract"
assert_contains "eval task documents canonical trial-rate-limited response" "${eval_task_output}" "\`429 trial_rate_limited\`"
@@ -208,6 +210,7 @@ main() {
assert_contains "high-risk matrix documents hosted-signup redirect code" "${high_risk_matrix_output}" "trial_signup_required"
assert_contains "high-risk matrix documents canonical trial-rate-limited response" "${high_risk_matrix_output}" "trial_rate_limited"
assert_contains "high-risk matrix documents retry-after metadata" "${high_risk_matrix_output}" "Retry-After"
assert_contains "high-risk matrix names pulse pro retry-after ui proof" "${high_risk_matrix_output}" "tests/58-self-hosted-trial-rate-limit-ui.spec.ts"
assert_contains "api contracts document trial-start route" "${api_contracts_output}" "/api/license/trial/start"
assert_contains "api contracts document hosted-signup redirect code" "${api_contracts_output}" "trial_signup_required"
+9
View File
@@ -32,6 +32,10 @@ End-to-end Playwright tests that validate critical user flows against a running
- Verify the response either returns `409 trial_signup_required` with hosted `/start-pro-trial` handoff or `429 trial_rate_limited` with canonical `Retry-After` backoff when the local retry bucket is already exhausted
- Verify local entitlements remain unchanged until activation
- Verify duplicate initiation stays on the hosted-signup retry-burst contract
- `tests/58-self-hosted-trial-rate-limit-ui.spec.ts` — Pulse Pro trial CTA retry-after UI contract:
- Open `/settings/system/billing` on the real browser shell with free-tier entitlements
- Stub `429 trial_rate_limited` on `POST /api/license/trial/start`
- Verify the Pulse Pro CTA surfaces canonical `Retry-After` guidance even if `details.retry_after_seconds` disagrees
- `tests/08-cloud-hosting.spec.ts` — hosted cloud signup contract:
- Public `/cloud/signup` form creates a real Stripe sandbox checkout session
- Checkout completes and returns to hosted signup completion page
@@ -187,6 +191,11 @@ For hosted trial initiation validation against a fresh LXC each run:
open, then transition to `429 trial_rate_limited` plus `Retry-After`
backoff metadata once the retry burst is exhausted and the limiter
actually engages
- Pulse Pro browser proof: `tests/58-self-hosted-trial-rate-limit-ui.spec.ts`
- Exercises `/settings/system/billing` on the real browser shell and proves
the rendered CTA surfaces canonical `Retry-After` guidance when
`trial_rate_limited` is returned, including header precedence over a
conflicting `details.retry_after_seconds` payload
- Full sandbox orchestration (multi-tenant + trial + cloud, with per-scenario snapshot reset):
- `tests/integration/scripts/run-lxc-sandbox-evals.sh`
- Includes hosted trial initiation validation and cloud subscription cancellation lifecycle verification