Commit Graph

45 Commits

Author SHA1 Message Date
Pulse Test e1bc14bd4a Harden alert recovery and qualification 2026-08-28 11:28:38 +01:00
rcourtman 248b242034 test(alerts): automate operator qualification 2026-08-27 11:06:26 +01:00
rcourtman b92893351d Make release dry-run diagnostics fail closed (#1758)
* Make release dry-run diagnostics fail closed

Select the installed Chromium project with retries disabled, replace the unconditional pass with fail-closed API and rendered-UI readiness assertions, retain actionable runtime evidence, and guard the release workflow contract against regression.

Contract-Neutral: Release diagnostic and workflow verification hardening only; no product runtime contract changes.

* Expose stable E2E failure identities

Project Playwright JUnit failures into bounded GitHub annotations so repeated stable-tier failures can be diagnosed without rerunning or weakening the gate. Keep the full reports and runtime logs as the forensic record, and cover annotation parsing and escaping with deterministic tests.

Contract-Neutral: This changes CI failure observability only and does not alter product runtime behavior, stable-tier membership, retries, or verdict semantics.

---------

Co-authored-by: rcourtman <rcourtman@users.noreply.github.com>
2026-08-22 07:28:29 +01:00
rcourtman 887b85bc83 Validate Core E2E tiering against run evidence (#1711)
Contract-Neutral: E2E test tier metadata and validation only; no deployment runtime or public contract change.

Co-authored-by: Pulse Autonomous Maintainer <rcourtman@users.noreply.github.com>
2026-08-11 23:18:14 +01:00
rcourtman 99af2a0bf2 Accelerate mock history readiness 2026-07-20 04:21:26 +01:00
rcourtman 584aa10699 Stabilize multi-tenant org switch in release E2E 2026-07-09 14:34:46 +01:00
rcourtman 1a05c715ac Reuse authenticated state in multi-tenant release E2E 2026-07-09 13:40:35 +01:00
rcourtman c728539f07 Restore completed Core E2E verdicts: shard CI, drop release tag from test image
Every main push since the v6 branch flip was cancelled at the 45-minute
job timeout with no verdict. The flip brought the full 94-spec suite onto
main (the last green run, 2026-06-29, ran only 2 specs on the v5 main),
and it runs sequentially against a release-tagged image whose mock-fixture
gate returns 403 without a demo entitlement. Dozens of specs fail, retry
twice each, and burn the budget: of the 31 minutes of suite time in run
28907574469, 18.8 minutes were failing attempts.

- Add GO_BUILD_TAGS build arg (default release) and build the pulse:test
  e2e image with it empty, matching the dev harness the suite is green
  under. Shipped images keep the release tag; release-gate behavior keeps
  its dedicated -tags release Go tests.
- Shard Playwright 4 ways across a CI matrix (214/202/205/203 tests per
  shard) with per-shard report artifacts and an aggregate verdict job.
- Cap CI at 20 failures so an env-broken run reports red in minutes
  instead of grinding into a no-verdict cancellation.
2026-07-08 08:00:55 +01:00
rcourtman 54b7e9933d Harden integration bootstrap token seed 2026-06-14 23:32:38 +01:00
rcourtman 377fd5131d Harden release integration bootstrap gate 2026-06-14 22:26:13 +01:00
rcourtman 09ed857869 Converge UI primitives and readiness guards 2026-06-13 16:31:52 +01:00
rcourtman 391cea1b13 Fix managed hot-dev LAN browser binding 2026-05-19 09:08:32 +01:00
rcourtman 91d7102a27 Make managed hot-dev verification deterministic 2026-05-18 17:29:50 +01:00
rcourtman 99e65d7e68 Rename retired trial acquisition proof assets 2026-04-28 18:38:10 +01:00
rcourtman cc26f59f01 Retire self-hosted trial activation callback 2026-04-28 16:36:09 +01:00
rcourtman 0b52968cd0 Retire self-hosted trial start route 2026-04-28 15:05:58 +01:00
rcourtman 410590c2e1 Fix hosted mobile approval proof runtime setup 2026-04-24 17:14:47 +01:00
rcourtman badda3781c Harden hosted mobile proof setup 2026-04-24 13:13:36 +01:00
rcourtman befb0160f7 Harden hosted mobile approval proof setup 2026-04-24 11:50:59 +01:00
rcourtman a442eb6bda Normalize hot-dev auth defaults 2026-04-22 16:28:52 +01:00
rcourtman f94601e26f Add hosted staging smoke entrypoint 2026-04-15 12:17:57 +01:00
rcourtman d38dcb00da Align trial-start proofs with retry burst 2026-04-10 17:48:20 +01:00
rcourtman 259285ad40 Unify Playwright route base selection 2026-04-10 17:33:18 +01:00
rcourtman 659c5f5988 Prefer explicit Playwright browser base URL 2026-04-10 17:26:18 +01:00
rcourtman b3bfb254fe Add integrated demo commercial boundary smoke 2026-04-09 10:43:07 +01:00
rcourtman 0c1fafa800 Centralize demo commercial boundary policy 2026-04-07 10:24:53 +01:00
rcourtman ce4e418d84 Add hosted mobile token revoke helper 2026-04-02 23:26:13 +01:00
rcourtman 9bc6d6ffff Fix hosted mobile onboarding proof bootstrap 2026-03-30 10:52:06 +01:00
rcourtman 444e827392 Fix hosted approval proof tenant scoping 2026-03-30 09:33:15 +01:00
rcourtman 313defb69b Bootstrap hosted managed backend API tokens 2026-03-29 23:28:50 +01:00
rcourtman 2fdf0e16c8 Support enterprise mobile approval proof backend 2026-03-29 21:52:18 +01:00
rcourtman 6817b67fbd Scope insecure E2E TLS health checks 2026-03-29 13:50:03 +01:00
rcourtman d877763fde test(integration): restart managed runtime for verify 2026-03-28 23:36:49 +00:00
rcourtman 88f27f6dfc Stabilize managed dev verify runtime 2026-03-25 12:14:39 +00:00
rcourtman 2880c73619 Prove managed owner-process recovery 2026-03-25 09:08:04 +00:00
rcourtman d8f106de1d test(first-session): prove deterministic setup handoff 2026-03-25 00:36:47 +00:00
rcourtman f4a8ac4773 Add managed dev-runtime recovery proof 2026-03-24 14:15:38 +00:00
rcourtman 778a2577b6 feat: Pulse v6 release 2026-03-18 16:06:30 +00:00
rcourtman ef3cf946e3 chore(e2e): reduce verbose logging in pretest health checks 2025-12-19 16:23:07 +00:00
rcourtman 6ef27d31ca fix(e2e): use http module instead of fetch for health checks
Exit code 13 in Node.js indicates 'Unfinished Top-Level Await'.
Replacing fetch with native http module to see if this resolves the issue.
2025-12-19 16:11:57 +00:00
rcourtman d786e55f8f fix(e2e): add signal handlers and detailed tracing to diagnose exit code 13 2025-12-19 15:59:48 +00:00
rcourtman 98c4a08d64 fix(e2e): add debugging and container logging to diagnose CI failures
- Separate pretest (start containers) from test (run playwright) steps
- Add container log collection step that runs on failure
- Add verbose logging to pretest.mjs for better failure diagnosis
- Use PULSE_E2E_SKIP_DOCKER and PULSE_E2E_SKIP_PLAYWRIGHT_INSTALL flags
2025-12-19 15:48:35 +00:00
rcourtman 5338ab580c Stabilize core E2E tests
- Preserve alerts activation state when saving thresholds
- Use compliant default E2E password and deterministic bootstrap token seeding
- Harden Playwright selectors, waits, and diagnostics gating
2025-12-17 19:36:48 +00:00
rcourtman 54fc259221 fix(ai): improve AI settings UX with validation and smart fallbacks
Backend:
- Add smart provider fallback when selected model's provider isn't configured
- Automatically switch to a model from a configured provider instead of failing
- Log warning when fallback occurs for visibility

Frontend (AISettings.tsx):
- Add helper functions to check if model's provider is configured
- Group model dropdown: configured providers first, unconfigured marked with ⚠️
- Add inline warning when selecting model from unconfigured provider
- Validate on save that model's provider is configured (or being added)
- Warn before clearing last configured provider (would disable AI)
- Warn before clearing provider that current model uses
- Add patrol interval validation (must be 0 or >= 10 minutes)
- Show red border + inline error for invalid patrol intervals 1-9
- Update patrol interval hint: '(0=off, 10+ to enable)'

These changes prevent confusing '500 Internal Server Error' and
'AI is not enabled or configured' errors when model/provider mismatch.
2025-12-17 18:30:19 +00:00
Claude 2afdca4d30 Add comprehensive integration test suite for update flow
Implements end-to-end testing infrastructure for the Pulse update flow,
validating the entire path from UI to backend with controllable test
scenarios.

## What's Included

### Test Infrastructure
- Mock GitHub release server (Go) with controllable failure modes
- Docker Compose test environment (isolated services)
- Playwright test framework with TypeScript
- 60+ test cases across 6 test suites
- Helper library with 20+ reusable test utilities

### Test Scenarios
1. Happy Path (8 tests)
   - Valid checksums, successful update flow
   - Modal appears exactly once
   - Complete end-to-end validation

2. Bad Checksums (8 tests)
   - Server rejects invalid checksums
   - Error shown ONCE (not twice) - fixes v4.28.0 issue type
   - User-friendly error messages

3. Rate Limiting (9 tests)
   - Multiple rapid requests throttled gracefully
   - Proper rate limit headers
   - Clear error messages

4. Network Failure (10 tests)
   - Exponential backoff retry logic
   - Timeout handling
   - Graceful degradation

5. Stale Release (10 tests)
   - Backend refuses flagged releases
   - Informative error messages
   - Proper rejection logging

6. Frontend Validation (15 tests)
   - UpdateProgressModal appears exactly once
   - No duplicate modals on error
   - User-friendly error messages
   - Proper accessibility attributes

### CI/CD Integration
- GitHub Actions workflow (.github/workflows/test-updates.yml)
- Runs on PRs touching update-related code
- Separate test runs for each scenario
- Regression test to verify v4.28.0 issue prevention
- Automatic artifact uploads

### Documentation
- README.md: Architecture and overview
- QUICK_START.md: Getting started guide
- IMPLEMENTATION_SUMMARY.md: Complete implementation details
- Helper scripts for setup and test execution

## Success Criteria Met

 Tests run in CI on every PR touching update code
 All scenarios pass reliably
 Tests catch v4.28.0 checksum issue type automatically
 Frontend UX regressions are blocked

## Usage

```bash
cd tests/integration
./scripts/setup.sh    # One-time setup
npm test              # Run all tests
```

See QUICK_START.md for detailed instructions.

Addresses requirements from issue for comprehensive update flow testing
with specific focus on preventing duplicate error modals and ensuring
checksum validation works correctly.
2025-11-11 09:31:52 +00:00