mirror of
https://github.com/Studio-Saelix/sencho.git
synced 2026-08-29 11:47:01 +00:00
fix(fleet-sync): version the wire protocol and serialize per-node pushes (#967)
Hardens the scan-policy and CVE-suppression replication channel as the foundation of a multi-PR fleet sync hardening track. No new endpoints, no new tables, no schema changes; receivers still tolerate legacy payloads (absent pushedAt and controlIdentity) for rollout safety. Wire protocol: - Sender stamps every push with a strictly-increasing pushedAt and a placeholder controlIdentity. Receiver rejects strictly-older pushedAt with 409 STALE_SYNC_PUSH so the next write retries. - pushedAt comparison plus row replacement plus watermark write run in a single SQLite transaction; a partial-write window cannot leave the watermark behind the row state. Concurrency and limits: - Per-node mutex on the sender so concurrent control writes serialize per remote and never apply older state on top of newer. - Sender-side row cap at MAX_SYNC_ROWS=5000 with a 6-hour throttled truncation alert so flapping configs cannot flood the operator. - Route-level body limit raised to 5MB on POST /api/fleet/sync/:resource only; the global 100KB cap is unchanged. Oversize bodies return a structured 413 SYNC_PAYLOAD_TOO_LARGE. Determinism and hygiene: - getMatchingPolicy gains an id-ASC tiebreaker so two replicas resolve the same winner when policies tie on scope class. - Inline comment documents why getMatchingPolicy filters node_id at SQL yet still relies on JS identity matching for replicated rows. - Comment on the receive endpoint documents why no requirePaid is enforced (control's tier authorizes; replica trusts the bearer). - STALE_SYNC_PUSH 409s no longer record a node failure; they are expected protocol outcomes, not health issues. Public surface additions: - DatabaseService.transaction(fn): generic SAVEPOINT-friendly wrapper. - DatabaseService.getLocalScanPolicies / getLocalCveSuppressions: SQL filter on replicated_from_control = 0. - StaleSyncPushError: typed sentinel the route translates to 409. - fleetSyncConstants: shared MAX_SYNC_ROWS, body limit, state-key and error-code maps so the wire protocol has one source of truth. Tests: - 24 new vitest cases across fleet-sync-service, fleet-sync-routes, and database-matching-policy covering monotonic pushedAt, per-node serialization, row truncation and throttle, stale-push suppression, receiver back-compat, oversize-body 413, deterministic matching. - Full backend suite: 1757 pass / 5 skipped.
This commit is contained in:
@@ -0,0 +1,70 @@
|
||||
/**
|
||||
* Shared constants for the Fleet Sync wire protocol.
|
||||
*
|
||||
* The control instance pushes scan_policies and cve_suppressions to remote
|
||||
* nodes via POST /api/fleet/sync/:resource. The payload schema is defined
|
||||
* here so both the sender (FleetSyncService) and the receiver (routes/fleet)
|
||||
* agree on limits and field names.
|
||||
*/
|
||||
|
||||
/**
|
||||
* Maximum number of rows accepted per sync push.
|
||||
*
|
||||
* Enforced on both ends:
|
||||
* - Sender: `FleetSyncService.loadResource` truncates at this cap and
|
||||
* emits a warning notification when it triggers. Operators with
|
||||
* >5000 policies on the control are exceedingly rare; truncating is
|
||||
* safer than failing every push.
|
||||
* - Receiver: `POST /api/fleet/sync/:resource` rejects payloads above
|
||||
* this cap with HTTP 413.
|
||||
*/
|
||||
export const MAX_SYNC_ROWS = 5000;
|
||||
|
||||
/**
|
||||
* Maximum body size for POST /api/fleet/sync/:resource. Sized to comfortably
|
||||
* fit MAX_SYNC_ROWS rows of either resource at the per-field length caps
|
||||
* enforced by the row validators (~1 KB per row worst-case = ~5 MB).
|
||||
*
|
||||
* The global JSON body limit stays at 100 KB; only this one route allows
|
||||
* larger bodies. See middleware/jsonParser.ts for the dispatch logic.
|
||||
*/
|
||||
export const SYNC_BODY_LIMIT = '5mb';
|
||||
|
||||
/**
|
||||
* Path prefix that the JSON body parser uses to dispatch to the larger
|
||||
* limit. Kept as a constant so any path change updates both the parser
|
||||
* and the route mount.
|
||||
*
|
||||
* CAUTION: prefix match, not exact route. Any new route under /api/fleet/sync/
|
||||
* inherits the elevated body limit. If a future route under this prefix should
|
||||
* keep the standard 100 KB cap, narrow the dispatch in middleware/jsonParser.ts
|
||||
* to method+exact-path instead of prefix.
|
||||
*/
|
||||
export const SYNC_PATH_PREFIX = '/api/fleet/sync/';
|
||||
|
||||
/** How long to suppress repeat truncation alerts after one fires. */
|
||||
export const TRUNCATION_ALERT_COOLDOWN_MS = 6 * 60 * 60 * 1000;
|
||||
|
||||
/**
|
||||
* Resource enum kept here so the state-key helpers below can type-check
|
||||
* their arguments without a cycle through FleetSyncService. The ordering
|
||||
* below mirrors `FLEET_RESOURCES` in FleetSyncService.
|
||||
*/
|
||||
export type FleetResource = 'scan_policies' | 'cve_suppressions';
|
||||
|
||||
/**
|
||||
* `system_state` keys read or written by Fleet Sync. Centralized so a typo
|
||||
* cannot silently bypass a stale-push check or a cooldown gate.
|
||||
*/
|
||||
export const SYNC_STATE_KEYS = {
|
||||
fleetRole: 'fleet_role',
|
||||
fleetSelfIdentity: 'fleet_self_identity',
|
||||
receivedPushedAt: (resource: FleetResource): string => `received_pushed_at:${resource}`,
|
||||
truncationAlertAt: (resource: FleetResource): string => `fleet_sync_truncation_alert_at:${resource}`,
|
||||
} as const;
|
||||
|
||||
/** Structured error codes returned by the receive endpoint. */
|
||||
export const SYNC_ERROR_CODES = {
|
||||
staleSyncPush: 'STALE_SYNC_PUSH',
|
||||
payloadTooLarge: 'SYNC_PAYLOAD_TOO_LARGE',
|
||||
} as const;
|
||||
Reference in New Issue
Block a user