Commit Graph

33 Commits

Author SHA1 Message Date
rcourtman 1289728ded Point dev:verify at the Proxmox backups layout spec
The managed proof pack behind `npm run dev:verify` still named
tests/17-recovery-layout.spec.ts, which 1c38d79937 replaced with
tests/17-proxmox-backups-layout.spec.ts in July. The registered proof for
scripts/hot-dev-bg.sh pinned the stale path, so nothing flagged it.

The stale path did not fail the pack. Playwright treats each spec path as
a filter, so `test --list` with main's arguments reports 10 tests in 2
files and exits 0: the Proxmox backups layout guard had silently dropped
out of dev:verify.

The replacement spec reads the mock-mode estate, while dev:verify attaches
to the managed runtime in whatever mode it was left in. Under the managed
runtime it now skips with a `npm run mock:on` hint when mock mode is off,
instead of flipping modes in place: Monitor.SetMockMode clears the
runtime's active alerts and in-memory history on every switch. The Core
E2E harness runs mock mode and is unaffected.

test-hot-dev-bg.sh now pins the new spec and checks that every spec the
default verify command names exists under tests/integration; against
main's hot-dev-bg.sh that check fails on the retired path. The integration
README and its pin, the deployment-installability contract clause, and two
unused RECOVERY_PRODUCT_SURFACE_EXACT_FILES lists naming deleted files are
brought in line.
2026-10-08 09:56:53 +01:00
pulse-triage[bot] 57beb6010f test: reap isolated browser writers before credential cleanup
Shell-only signal traps could leave browser descendants writing sensitive state after cleanup, and simultaneous runs shared the same cookie cache. Give the Linux qualification runner an invocation-owned cookie root and a child-subreaper supervisor so catchable interruption waits for writers before removing owned artifacts. Preserve normal reports for inspection and fail closed if writers cannot be reaped. Focused fixtures cover all three signals, late detached writers, concurrent isolation and forced termination; no real-browser cleanup or release readiness is claimed.

Change-source: pulse-maintainer
2026-09-05 16:31:12 +01:00
pulse-triage[bot] 0b73d36011 test(e2e): separate quarantined multi-tenant diagnostic from tier gates
The shell multi-tenant suite previously selected a spec ignored by every project, preventing real backend diagnosis. Add an explicit desktop-only configuration that rejects tier identity, while retaining the normal quarantine. Cover discovery and tier refusal and document the evidence boundaries. A local source-built diagnostic returned five passes, one failure at organisation-switch login, and one skip; this is not release qualification.

Change-source: pulse-maintainer
2026-09-05 14:39:37 +01:00
pulse-triage[bot] 095d1cb107 Cover large-estate Proxmox section navigation
Change-source: pulse-maintainer
2026-09-01 13:29:06 +01:00
rcourtman 887b85bc83 Validate Core E2E tiering against run evidence (#1711)
Contract-Neutral: E2E test tier metadata and validation only; no deployment runtime or public contract change.

Co-authored-by: Pulse Autonomous Maintainer <rcourtman@users.noreply.github.com>
2026-08-11 23:18:14 +01:00
rcourtman f2fa92f576 Split Core E2E into gating stable and non-gating probation tiers
Mined per-spec failure data from the 32 completed Core E2E main runs
since the 2026-07-18 quarantine delist (failed-shard logs; a green
shard means every spec in it passed). 26 specs failed or retry-flaked
within the 10 most recent completed runs; they seed PROBATION_SPECS in
tests/integration/playwright.config.ts. The remaining 62 files form the
stable tier and are the only specs that can fail the e2e-verdict job.

Mechanism extends the existing quarantine list rather than adding a
parallel one: PULSE_E2E_TIER=stable ignores probation specs,
PULSE_E2E_TIER=probation runs only them, unset runs the full suite
(local behavior unchanged). CI runs both tiers per shard against the
same containers; the probation pass sits behind continue-on-error with
its own report/results dirs and artifacts, so a probation flake is
reported in the shard summary without painting main red.

Promotion rule, documented next to the list: a probation spec promotes
to stable after 10 consecutive green main runs with no failure and no
retry-flake; one incident on main demotes a stable spec back. In the
newest completed run (29729544151) every failure was in a probation
spec — under this split that run's verdict is green.

Contract-Neutral: CI-only E2E tier split: gating semantics of push-CI verdict; local npm test behavior and dev-runtime orchestration unchanged, no public contract delta
2026-07-20 10:32:26 +01:00
rcourtman f509b79caa Point integration docs at the update-flow spec
tests/integration/README.md and QUICK_START.md referenced the retired
.github/workflows/test-updates.yml; update-flow coverage now runs as
tests/79-update-flow.spec.ts inside the main suite via test-e2e.yml.
2026-07-08 01:07:20 +01:00
rcourtman 99e65d7e68 Rename retired trial acquisition proof assets 2026-04-28 18:38:10 +01:00
rcourtman 0b52968cd0 Retire self-hosted trial start route 2026-04-28 15:05:58 +01:00
rcourtman 211f84d49d Refine infrastructure onboarding flow 2026-04-23 15:49:54 +01:00
rcourtman f7d9b8252e Adopt source-manager infrastructure settings flow 2026-04-22 16:54:17 +01:00
rcourtman 0f767b6439 Harden diagnostics onboarding export proof 2026-04-22 16:16:26 +01:00
rcourtman f682ec475b Add Playwright proof for onboarding diagnostics 2026-04-22 15:58:11 +01:00
rcourtman 0dacde5066 Add Playwright proof for infrastructure onboarding 2026-04-22 15:21:45 +01:00
rcourtman d92ce0c54b Anchor trial retry-after UI proof 2026-04-10 23:58:54 +01:00
rcourtman 8610f3b808 Unify trial-start doc anchors 2026-04-10 18:32:58 +01:00
rcourtman d38dcb00da Align trial-start proofs with retry burst 2026-04-10 17:48:20 +01:00
rcourtman 659c5f5988 Prefer explicit Playwright browser base URL 2026-04-10 17:26:18 +01:00
rcourtman 2cde5c71ec Share integration browser default helper 2026-03-25 17:02:41 +00:00
rcourtman 5d08a917f6 Prefer managed browser shell in integration defaults 2026-03-25 16:53:55 +00:00
rcourtman c087db5da5 Delegate frontend dev runtime to root wrapper 2026-03-25 16:39:17 +00:00
rcourtman 88f27f6dfc Stabilize managed dev verify runtime 2026-03-25 12:14:39 +00:00
rcourtman 2880c73619 Prove managed owner-process recovery 2026-03-25 09:08:04 +00:00
rcourtman 9d74c943c7 Expand managed dev browser proof pack 2026-03-24 21:48:34 +00:00
rcourtman 25a28e3f27 Route recovery proof docs through backend wrapper 2026-03-24 21:31:12 +00:00
rcourtman c9b3ade048 Expose managed dev runtime entrypoint 2026-03-24 14:46:58 +00:00
rcourtman f4a8ac4773 Add managed dev-runtime recovery proof 2026-03-24 14:15:38 +00:00
rcourtman 778a2577b6 feat: Pulse v6 release 2026-03-18 16:06:30 +00:00
rcourtman ee63d438cc docs: standardize markdown syntax and remove deprecated sensor-proxy docs 2026-01-20 09:43:49 +00:00
rcourtman 5338ab580c Stabilize core E2E tests
- Preserve alerts activation state when saving thresholds
- Use compliant default E2E password and deterministic bootstrap token seeding
- Harden Playwright selectors, waits, and diagnostics gating
2025-12-17 19:36:48 +00:00
rcourtman 54fc259221 fix(ai): improve AI settings UX with validation and smart fallbacks
Backend:
- Add smart provider fallback when selected model's provider isn't configured
- Automatically switch to a model from a configured provider instead of failing
- Log warning when fallback occurs for visibility

Frontend (AISettings.tsx):
- Add helper functions to check if model's provider is configured
- Group model dropdown: configured providers first, unconfigured marked with ⚠️
- Add inline warning when selecting model from unconfigured provider
- Validate on save that model's provider is configured (or being added)
- Warn before clearing last configured provider (would disable AI)
- Warn before clearing provider that current model uses
- Add patrol interval validation (must be 0 or >= 10 minutes)
- Show red border + inline error for invalid patrol intervals 1-9
- Update patrol interval hint: '(0=off, 10+ to enable)'

These changes prevent confusing '500 Internal Server Error' and
'AI is not enabled or configured' errors when model/provider mismatch.
2025-12-17 18:30:19 +00:00
rcourtman 6a1a88217f Add release dry run workflow and API update integration test 2025-11-12 21:02:52 +00:00
Claude 2afdca4d30 Add comprehensive integration test suite for update flow
Implements end-to-end testing infrastructure for the Pulse update flow,
validating the entire path from UI to backend with controllable test
scenarios.

## What's Included

### Test Infrastructure
- Mock GitHub release server (Go) with controllable failure modes
- Docker Compose test environment (isolated services)
- Playwright test framework with TypeScript
- 60+ test cases across 6 test suites
- Helper library with 20+ reusable test utilities

### Test Scenarios
1. Happy Path (8 tests)
   - Valid checksums, successful update flow
   - Modal appears exactly once
   - Complete end-to-end validation

2. Bad Checksums (8 tests)
   - Server rejects invalid checksums
   - Error shown ONCE (not twice) - fixes v4.28.0 issue type
   - User-friendly error messages

3. Rate Limiting (9 tests)
   - Multiple rapid requests throttled gracefully
   - Proper rate limit headers
   - Clear error messages

4. Network Failure (10 tests)
   - Exponential backoff retry logic
   - Timeout handling
   - Graceful degradation

5. Stale Release (10 tests)
   - Backend refuses flagged releases
   - Informative error messages
   - Proper rejection logging

6. Frontend Validation (15 tests)
   - UpdateProgressModal appears exactly once
   - No duplicate modals on error
   - User-friendly error messages
   - Proper accessibility attributes

### CI/CD Integration
- GitHub Actions workflow (.github/workflows/test-updates.yml)
- Runs on PRs touching update-related code
- Separate test runs for each scenario
- Regression test to verify v4.28.0 issue prevention
- Automatic artifact uploads

### Documentation
- README.md: Architecture and overview
- QUICK_START.md: Getting started guide
- IMPLEMENTATION_SUMMARY.md: Complete implementation details
- Helper scripts for setup and test execution

## Success Criteria Met

✅ Tests run in CI on every PR touching update code
✅ All scenarios pass reliably
✅ Tests catch v4.28.0 checksum issue type automatically
✅ Frontend UX regressions are blocked

## Usage

```bash
cd tests/integration
./scripts/setup.sh    # One-time setup
npm test              # Run all tests
```

See QUICK_START.md for detailed instructions.

Addresses requirements from issue for comprehensive update flow testing
with specific focus on preventing duplicate error modals and ensuring
checksum validation works correctly.
2025-11-11 09:31:52 +00:00