mirror of
https://github.com/Studio-Saelix/sencho.git
synced 2026-08-08 18:05:10 +00:00
33b15d6cbacdc95656da696f50c29323d704a319
12 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Anso
|
27660f622b |
fix(fleet-sync): version the wire protocol and serialize per-node pushes (#967)
Hardens the scan-policy and CVE-suppression replication channel as the foundation of a multi-PR fleet sync hardening track. No new endpoints, no new tables, no schema changes; receivers still tolerate legacy payloads (absent pushedAt and controlIdentity) for rollout safety. Wire protocol: - Sender stamps every push with a strictly-increasing pushedAt and a placeholder controlIdentity. Receiver rejects strictly-older pushedAt with 409 STALE_SYNC_PUSH so the next write retries. - pushedAt comparison plus row replacement plus watermark write run in a single SQLite transaction; a partial-write window cannot leave the watermark behind the row state. Concurrency and limits: - Per-node mutex on the sender so concurrent control writes serialize per remote and never apply older state on top of newer. - Sender-side row cap at MAX_SYNC_ROWS=5000 with a 6-hour throttled truncation alert so flapping configs cannot flood the operator. - Route-level body limit raised to 5MB on POST /api/fleet/sync/:resource only; the global 100KB cap is unchanged. Oversize bodies return a structured 413 SYNC_PAYLOAD_TOO_LARGE. Determinism and hygiene: - getMatchingPolicy gains an id-ASC tiebreaker so two replicas resolve the same winner when policies tie on scope class. - Inline comment documents why getMatchingPolicy filters node_id at SQL yet still relies on JS identity matching for replicated rows. - Comment on the receive endpoint documents why no requirePaid is enforced (control's tier authorizes; replica trusts the bearer). - STALE_SYNC_PUSH 409s no longer record a node failure; they are expected protocol outcomes, not health issues. Public surface additions: - DatabaseService.transaction(fn): generic SAVEPOINT-friendly wrapper. - DatabaseService.getLocalScanPolicies / getLocalCveSuppressions: SQL filter on replicated_from_control = 0. - StaleSyncPushError: typed sentinel the route translates to 409. - fleetSyncConstants: shared MAX_SYNC_ROWS, body limit, state-key and error-code maps so the wire protocol has one source of truth. Tests: - 24 new vitest cases across fleet-sync-service, fleet-sync-routes, and database-matching-policy covering monotonic pushedAt, per-node serialization, row truncation and throttle, stale-push suppression, receiver back-compat, oversize-body 413, deterministic matching. - Full backend suite: 1757 pass / 5 skipped. |
||
|
Anso
|
ecf4dd5d52 |
feat: open security basics, manual fleet ops, and basic fleet management to Community (#930)
Realign tier guards to the user-stated philosophy: Community covers
deploy/monitor at scale plus security basics, Skipper adds automation
and advanced fleet management, Admiral keeps enterprise control.
Community now includes:
- Trivy install / uninstall / update from the Settings Hub (admin role)
- CVE suppressions CRUD (admin role; replicates fleet-wide)
- Manual image scan with vuln, secret, and misconfig results
- Stack-config scan, scan comparison
- Manual fleet snapshots: create, list, view, restore, delete
- Per-node Sencho self-update (Check Updates + per-node Update)
- Fleet Overview search, sort, filters, node-card expand, auto-refresh
Stays paid:
- Scan policies with block_on_deploy enforcement (Skipper+)
- SBOM (SPDX, CycloneDX), SARIF export (Skipper+)
- Bulk Update All across the fleet (Skipper+)
- Scheduled snapshot create (now Skipper, was Admiral)
- Trivy auto-update toggle, fleet-wide policy push (Admiral)
The Settings -> Security tab is unhidden by setting the registry tier to
null. The SecuritySection no longer early-returns a PaidGate; the policy
list, Add Policy button, and policy dialogs are wrapped in {isPaid && }.
The Fleet view drops isPaid gates on the Snapshots tab, Check Updates
button, per-node update handlers, OverviewToolbar grid controls, the
NodeCard expand affordance, and the auto-refresh notice. The
NodeUpdatesSheet receives a canBulkUpdate prop and gates the Update All
button on it. useFleetUpdateStatus and useFleetPolling drop their isPaid
guards so polling runs for Community; useFleetOverview drops the isPaid
wrap on the filter and sort path.
Backend route guards are flipped per the matrix above. The scheduler
tick and requireScheduledTaskTier add 'snapshot' to the Skipper+ branch.
Backend test assertions are inverted for the now-Community endpoints
and a positive Skipper-snapshot-task test is added.
Documentation across features/, api-reference/, and operations/ is
updated to reflect the new tier mapping.
|
||
|
Anso
|
e5b1c7b22b |
refactor(backend): collapse entitlement provider abstraction back to LicenseService (#889)
Removes backend/src/entitlements/ (registry, loadProvider, CommunityEntitlementProvider, types, headers, normalize) and the two abstraction-only tests. Relocates headers/normalize/types to services/license-*.ts. Swaps 22 consumer call sites from getEntitlementProvider() to LicenseService.getInstance(). Drops the Dockerfile install step plus PRO_PACKAGE_VERSION build-arg and github_token BuildKit secret in docker-publish.yml. Removes the now stale no-restricted-imports rule in backend/eslint.config.mjs. Net: 37 files changed, ~700 lines removed, no behavior change. Local dev no longer requires GitHub Packages auth to start the backend. Rationale and revisit conditions in docs/internal/adrs/2026-05-02-collapse-entitlement-provider.md. |
||
|
Anso
|
3324616e59 |
refactor(backend): extract EntitlementProvider abstraction (Phase 1) (#878)
* refactor(backend): extract EntitlementProvider abstraction (Phase 1)
Phase 1 of the open-core hybrid extraction described in
docs/internal/adrs/2026-05-02-open-core-hybrid-strategy.md. Introduces
the abstraction without moving any code out of the public repo; Phase
2 will actually move services/LicenseService.ts to a private
@studio-saelix/sencho-pro package.
The new backend/src/entitlements/ module contains:
- types.ts. The EntitlementProvider interface plus all tier/license
types (LicenseTier, LicenseVariant, LicenseInfo, SeatLimits,
ActivationResult, etc.). The interface mirrors the existing
LicenseService public surface so the migration was mechanical.
- registry.ts. Module-scope holder for the active provider with
setEntitlementProvider, getEntitlementProvider, and a test-only
reset helper. getEntitlementProvider throws if called before
bootstrap registers a provider; the throw is intentional fail-fast
on a bootstrap-order bug rather than a silent degradation.
- CommunityEntitlementProvider.ts. Phase 2 fallback that returns
community tier and rejects activate(). NOT instantiated in
production today; a smoke test keeps it covered against bitrot.
- loadProvider.ts. Async resolver. Phase 1 returns
LicenseService.getInstance() directly. The async signature matches
what Phase 2 needs (dynamic import of @studio-saelix/sencho-pro
with a "module not found" vs "construction threw" narrowing); the
call site does not change between phases.
- headers.ts. PROXY_TIER_HEADER and PROXY_VARIANT_HEADER constants.
These are part of the wire contract between Sencho instances and
belong in the public core regardless of which entitlement provider
is bound.
- normalize.ts. isLicenseTier, isLicenseVariant, normalizeTier,
normalizeVariant. Domain knowledge about Sencho's tier model
(legacy name maps from pre-0.38.1 versions), not LemonSqueezy
internals. Phase 2 keeps these in the public core.
services/LicenseService.ts now imports its types from
entitlements/types and adds an "implements EntitlementProvider"
clause. Re-exports the types for back-compat with ~20 type-only
consumers; a follow-up PR will sweep those imports to entitlements/
types directly before Phase 2 deletes the file.
bootstrap/startup.ts awaits loadEntitlementProvider, registers the
result, then calls initialize. shutdown.ts calls
getEntitlementProvider().destroy() instead of the LicenseService
singleton.
middleware/tierGates.ts, the chokepoint for ~154 tier-check call
sites, now reads through getEntitlementProvider. Sixteen other
production files (routes/{fleet,imageUpdates,license,permissions,
scheduledTasks,security,stacks,templates,users,webhooks},
services/{BlueprintService,CloudBackupService,SchedulerService,
SSOService}, proxy/remoteNodeProxy, websocket/{hostConsole,
remoteForwarder}, middleware/auth) had their LicenseService.getInstance
calls and utility-export imports redirected to the entitlements
module. The only remaining LicenseService.getInstance in production
code is in entitlements/loadProvider.ts itself, which is the
intentional Phase-1 binding site.
Test infrastructure: setupTestDb registers
LicenseService.getInstance() as the active provider so existing
test files using the helper need no changes. The mocking pattern
many tests use, vi.spyOn(LicenseService.getInstance(), 'getTier'),
keeps working because LicenseService.getInstance() and
getEntitlementProvider() return the same singleton in Phase 1.
scheduler-service.test.ts is the only test that does not use
setupTestDb but exercises tier-gating; it now mocks
entitlements/registry alongside its existing LicenseService mock.
Adds a smoke test for CommunityEntitlementProvider so the Phase 2
fallback class stays covered.
Adds an architecture doc at
docs/internal/architecture/entitlement-provider.md covering the
runtime registry, bootstrap order invariants, and the Phase 1 vs
Phase 2 binding table.
Test results: 89/89 backend test files pass, 1657 passing tests, 5
pre-existing skips. The pre-existing database-metrics > handles
1000+ metrics stress test continues to flake under parallel load
and pass when re-run solo, same flake observed in PRs #862, #863.
* chore(backend): drop unused entitlement type imports from LicenseService
Phase 1 of the EntitlementProvider extraction left five type imports
(ActivationResult, BillingPortalError, BillingPortalResult,
DeactivationResult, ValidationResult) unreferenced after the runtime
methods that produced them began inferring their result shapes via the
EntitlementProvider interface contract. ESLint's no-unused-vars rule
flagged them as errors and failed the lint step in CI.
|
||
|
Anso
|
4e5ba17710 |
refactor(backend): sanitize user input before logging to close CRLF injection (#807)
* refactor(backend): sanitize user input before logging to close CRLF injection
Adds a small sanitizeForLog helper that strips CR, LF, tab, and ASCII
control characters (0x00-0x1F, 0x7F) from a value before it is embedded
in a console.log/warn/error/debug call. Wraps every call site where a
user-controlled value (req.params, req.body, req.query, or a value
derived from them) flows into a log message.
Closes the bulk of the open CodeQL alerts in this family:
- 96 js/log-injection
- 28 js/tainted-format-string
The helper is in backend/src/utils/safeLog.ts. Routes still pre-validate
input at the request boundary; this is the second line of defense and
gives static analyzers a sanitizer they can trace through. JSON
responses, Docker filter labels, and other non-log call sites are
intentionally left unwrapped.
* refactor(backend): printf-style format strings for tainted-log call sites
CodeQL's js/tainted-format-string rule flags template literals in the first
arg of console.X when any interpolated value is user-controlled, regardless
of whether each value is sanitized inline. The canonical mitigation is to
use a static format string and pass values as positional args.
Converts the 28 flagged template literals to printf-style ("%s") format
strings, with sanitizeForLog applied to each positional arg. Also fills in
the log-injection wraps on 9 sites where a user-controlled value was
missed in the first sweep (agents, fleet, gitSources, imageUpdates,
GitSourceService).
No behavior change at runtime. Node's util.format substitutes %s tokens
identically to template-literal interpolation.
* fix(backend): wrap nodeId/snapshotId in fleet restore debug log
CodeQL flagged the unwrapped numeric args even though they cannot
contain control chars in practice. Apply the sanitizer for taint-flow
recognition.
|
||
|
Anso
|
add3abaece |
refactor(backend): replica guard helper for security routes (#797)
* refactor(backend): extract replica guard helper for security routes
Adds blockIfReplica(res, resource) in middleware/fleetSyncGuards.ts and
replaces six inline FleetSyncService.getRole() === 'replica' checks
across the security policies and CVE suppressions endpoints.
Error responses now use a uniform shape:
403 { error: 'Cannot modify <resource> on a replica instance.
Connect to the primary.', code: 'REPLICA_READ_ONLY' }
The new code field gives callers a stable discriminator without
matching prose.
Closes #750
* test(suppressions): match stable REPLICA_READ_ONLY code instead of prose
The replica guard helper exposes a stable code field for callers to
discriminate without grepping the human-readable error string. Switch
the replica-rejection assertion to use that code so the test no longer
breaks when the unified error template wording is tuned.
|
||
|
Anso
|
a502da54ee |
feat(sso): split SSO providers by delivery model across tiers (#754)
Custom OIDC stays on Community so self-hosters can wire any spec-compliant OIDC identity provider (Authelia, Keycloak, Authentik, Zitadel, and others). Google, GitHub, and Okta one-click presets move to Skipper. LDAP / Active Directory and scoped RBAC are Admiral-only. Backend enforces the split via a new requireTierForSsoProvider helper in middleware/tierGates.ts, applied after requireAdmin in all four ssoConfig mutation handlers. GET /sso/config (list) stays ungated so downgraded admins can still see previously-configured providers. Invalid provider ids now 400 before the tier check to avoid leaking tier information. Frontend adds a compact mode to PaidGate and AdmiralGate for inline list-item locks, and SSOSection reorders the provider cards as Custom OIDC > Google > GitHub > Okta > LDAP to reinforce the free-to-paid progression. Stale 'SSO is Admiral' copy in AdmiralGate, PaidGate, and the Admiral upgrade card on the License settings page has been replaced to reflect the new split. User-facing licensing, SSO, overview, quickstart, and security docs have been updated with the per-tier provider matrix. |
||
|
Anso
|
b329916a0c |
refactor(backend): extract auth/MFA/SSO routers from index.ts (phase 4a-2) (#735)
Second slice of Phase 4. Pulls the three auth-family route groups out of
index.ts into focused routers. All handlers move verbatim; index.ts drops
~845 lines.
New route files:
- routes/auth.ts: /api/auth core (status, setup, login, password, logout,
check, generate-node-token)
- routes/mfa.ts: /api/auth/login/mfa + full /api/auth/mfa/* surface
(status, enroll start/confirm, disable, backup-codes/regenerate,
sso-bypass)
- routes/sso.ts: /api/auth/sso/{providers,ldap,oidc/:provider/authorize,
oidc/:provider/callback} + getSSOBaseUrl helper. Module-load calls
SSOService.getInstance().seedFromEnv() so env-seeded providers are
available on the first request.
Shared lifts:
- helpers/constants.ts: MFA_REPLAY_TTL_MS + MFA_REPLAY_PURGE_INTERVAL_MS
(used by mfa.ts and the startup purge timer in index.ts) and
BCRYPT_SALT_ROUNDS (shared between setup and password-change handlers).
- middleware/auth.ts: new reissueSessionAfterTokenBump(req, res, userId)
helper collapses three copies of "bump → fetch user → re-sign cookie"
across auth.ts (password change) and mfa.ts (enrol confirm, disable).
Code review fixes:
- File-local requireEnrolledMfaUser helper in mfa.ts eliminates four
copies of "auth check + rejectApiTokenScope + load enrolled MFA" with
near-identical shape.
- Applied BCRYPT_SALT_ROUNDS to auth.ts setup + password handlers.
Mount order in index.ts: authRouter / mfaRouter / ssoRouter sit before
authGate because login / setup / SSO-callback are public; handlers that
need auth use authMiddleware directly on the route.
|
||
|
Anso
|
50e64b058b |
refactor(backend): extract 8 low-blast-radius route groups into routers (phase 4a-1) (#734)
First slice of Phase 4 (route extraction). Pulls 8 well-tested, mostly
independent route groups out of index.ts into focused Router files. No
behavior change; every handler body moves verbatim.
New route files under backend/src/routes/:
- meta.ts /api/health, /api/meta (mounted before authGate)
- license.ts /api/license/* + /api/system/update,
exports scheduleLocalUpdate for the fleet route
- permissions.ts /api/permissions/me
- convert.ts POST /api/convert
- alerts.ts /api/alerts/*
- labels.ts /api/labels/* + PUT /api/stacks/:name/labels
(exported as stackLabelsRouter)
- apiTokens.ts /api/api-tokens/*
- auditLog.ts /api/audit-log/*
Shared helper lifts:
- helpers/cacheInvalidation.ts: invalidateNodeCaches()
- middleware/tierGates.ts: requireBody (was inline in index.ts)
- utils/errors.ts: isSqliteUniqueViolation (was inline in index.ts)
- middleware/apiTokenScope.ts: rejectApiTokenScope() helper (new)
- utils/csv.ts: escapeCsvField() (new)
index.ts drops from ~7520 to ~6775 lines and now mounts the routers right
after enforceApiTokenScope. The remote proxy and fleet/auth/webhooks/users
routes remain inline in index.ts pending later Phase 4 slices.
Code review fixes: rejectApiTokenScope helper replaces duplicated
`if (req.apiTokenScope) 403 SCOPE_DENIED` blocks in apiTokens.ts and
license.ts; escapeCsvField replaces the inline CSV escape in auditLog.ts.
|
||
|
Anso
|
ca5a930c68 |
refactor(backend): extract authMiddleware and introduce createApp factory (phase 2) (#732)
Phase 2 of the index.ts refactor. Pulls the auth middleware and session cookie issuers into their own module, and introduces the app.ts factory that owns the first nine steps of the canonical middleware pipeline. New modules: - middleware/auth.ts: authMiddleware, issueSessionCookie, issueMfaPendingCookie, clearMfaPendingCookie - app.ts: createApp() factory installing trust proxy, helmet, cors, compression, cookieParser, rate limiters, conditionalJsonParser, and nodeContextMiddleware. A header comment documents all 16 canonical middleware steps and where each currently lives. Changes: - middleware/authGate.ts: createAuthGate factory removed; authGate now imports authMiddleware directly (the factory existed only to avoid a circular import while authMiddleware lived in index.ts). - services/DatabaseService.ts: added API_TOKEN_SCOPE_TO_ROLE map so the auth middleware no longer inlines a stringly-typed record. - index.ts drops ~260 lines; auth routes, authGate, auditLog, apiTokenScope, remaining routes, static serving, and the error handler continue to be registered there until their respective phases. - vitest.config.ts bumps testTimeout to 30s and hookTimeout to 45s so fork-pool workers have enough headroom to ts-node-transform the growing module graph under CPU contention (64 workers each import the full Express stack in beforeAll). Code review fixes: use getErrorMessage() util in the auth catch block instead of inline cast; promote the scope-to-role map to a typed module-level constant. |
||
|
Anso
|
856a260a11 |
refactor(backend): extract rate limiters, body parsing, and request gates (phase 1) (#731)
Phase 1 of the index.ts monolith refactor. Extracts all non-auth middleware into focused modules. Routes and authMiddleware itself stay in index.ts for now; Phase 2 introduces createApp() and extracts authMiddleware. New modules: - middleware/rateLimiters.ts: globalApiLimiter, pollingLimiter, webhookTriggerLimiter, authRateLimiter, ssoRateLimiter, trivyInstallLimiter plus the hybrid rateLimitKeyGenerator and isNodeProxyRequest helper - middleware/jsonParser.ts: conditionalJsonParser that preserves the raw stream for remote-proxy forwarding (via helpers/proxyExemptPaths) - middleware/nodeContext.ts: nodeContextMiddleware - middleware/apiTokenScope.ts: enforceApiTokenScope + DEPLOY_ALLOWED_PATTERNS - middleware/authGate.ts: createAuthGate(authMiddleware) factory + auditLog. Factory takes authMiddleware as a dependency to avoid a circular import until Phase 2 extracts the auth module. - middleware/errorHandler.ts: central error handler that preserves err.status / err.expose from body-parser and other HTTP errors - helpers/routePatterns.ts: WEBHOOK_TRIGGER_RE shared by rateLimiters and authGate index.ts drops ~290 lines. Middleware registration order is unchanged. All 1278 tests pass. Code review fixes: typed ApiTokenScope in apiTokenScope.ts; replaced 2 em dashes with colons (Directive 18); added local CachedProxyFlagReq type alias for the node_proxy memoization cast; extracted deny() helper. |
||
|
Anso
|
929e2fa6b1 |
refactor(backend): extract types, constants, and guards from index.ts (phase 0) (#730)
* refactor(backend): extract types, constants, and guards from index.ts (phase 0) Additive, behavior-preserving first step of the modular backend refactor. Moves purely static artifacts out of backend/src/index.ts so later phases can extract routes and middleware without touching shared symbols. New modules: - types/express.ts: Express Request augmentation - helpers/constants.ts: PORT, password policy, label colors, cookie names, MFA TTLs, hot-path cache TTLs - helpers/proxyExemptPaths.ts: PROXY_EXEMPT_PREFIXES + isProxyExemptPath - helpers/cookies.ts: isSecureRequest, getCookieOptions - helpers/policyGate.ts: buildPolicyGateOptions, runPolicyGate, triggerPostDeployScan - middleware/permissions.ts: ROLE_PERMISSIONS, checkPermission, requirePermission - middleware/tierGates.ts: requirePaid, requireAdmiral, requireAdmin, requireNodeProxy, requireScheduledTaskTier + effectiveTier/Variant index.ts shrinks by ~260 lines; no runtime behavior changes. All 64 vitest files and 1,278 tests pass. * refactor(backend): drop unused imports left after phase 0 extraction LicenseTier, LicenseVariant, DIGEST_CACHE_TTL_MS, and isProxyExemptPath were imported into index.ts but no longer referenced there after the phase 0 move; CI lint flagged them as errors. isProxyExemptPath will be re-imported in phase 1 when the JSON parser bypass and nodeContext middleware get extracted. Silence the no-namespace warning on the Express augmentation since the namespace syntax is required for TypeScript module augmentation. |