mirror of
https://github.com/mulgadc/spinifex.git
synced 2026-10-15 17:11:30 +00:00
v1.13.0
1540 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 joshsiv-mulga
|
220141b145 | release: update go.mod for v1.13.0 v1.13.0 | ||
|
tomnewton-mulga
|
4670d09d58 | chore: RTX-6000 Supermicro demo doc draft | ||
|
Josh Siviour
|
48749576c3 |
test: rebalance E2E into in-process integration tier + shared-guest live scenarios (#556)
* test: add in-process integration test tier Roughly half of the behavioural tests under tests/e2e/ assert control-plane API shapes, authz and error codes that need no live environment, yet they can only run against a tofu-provisioned environment on a scarce self-hosted runner. This adds the tier they can move down to. The real gateway router runs in-process: SetupRoutes() is a pure function of GatewayConfig, so it serves from an httptest server over embedded NATS JetStream with real IAM/STS services and genuine SigV4 verification. Only the daemon side of the wire is faked -- the ec2.*/ebs.* NATS subjects a live node would answer are stubbed per test. Nothing is provisioned and no fixed ports are bound, so the tier runs concurrently on a shared build host and is safe as a CI front gate ahead of the live suites. It is build-tagged and kept out of preflight, which stays unit-only. * test(integration): add IAM/STS client accessors and port the zero-stub authz tests Adds IAMClient/STSClient plus ClientsWithCreds/ClientsWithSessionCreds so a test can act as a principal other than the seeded root user, and stubs the node-discovery subject in StartGateway so call paths that trigger discovery resolve immediately instead of riding out the fan-out timeout. Ports the four enforcement tests that guard on ec2:DescribeRegions, a gateway-local action, so they need no daemon-side stub at all. * test(integration): port remaining IAM/STS authz tests from e2e Ports TestIAMUserCRUD, TestIAMAccessKeyLifecycle, TestIAMUserAuthentication, TestIAMPolicyCRUD, TestIAMPolicyAttachmentEnforcement, TestIAMPolicyLifecycle, TestIAMGroupsLifecycle, TestIAMRolesAndProfiles and TestSTSAssumeRoleAndGetCallerIdentity from tests/e2e/iam/ into the in-process integration tier. Each test now self-bootstraps its own users/policies/roles instead of relying on shared suite-level state, since every integration test gets its own gateway and embedded NATS. TestIAMRolesAndProfiles stubs ec2.IamProfileAssociation.describe because DeleteInstanceProfile always counts live associations over NATS regardless of which guard actually fires. TestIAMCleanup is not ported: it only swept a shared long-lived E2E environment, and every ordering/dependency assertion it made is already covered by TestIAMPolicyLifecycle, TestIAMGroupsLifecycle and TestIAMRolesAndProfiles's own delete-conflict guards. * test(e2e): remove IAM/STS authz tests migrated to the integration tier All 13 signed-request authz tests plus TestIAMCleanup now have faithful, mutation-proven equivalents in tests/integration/ — the gateway dispatches IAM/STS actions directly with no daemon hop, so the in-process tier is a faithful and far cheaper substitute. The iam/ e2e suite now runs only TestIAMInstanceProfileAssociation and TestIMDS. Drop the now fully-dead iam_helpers_test.go alongside the six deleted test files, and remove harness.IAMGroupARN/IAMDeleteGroupBestEffort, which had no remaining caller anywhere in the repo once iam_groups_test.go was gone. * feat(integration): daemon-lite harness for key/tags/routetable/vpc Adds DaemonLite, an in-process stand-in for a live spinifex daemon that subscribes the real key, tags, route table, and VPC/subnet/SG/IGW service implementations directly to their NATS subjects, backed by memory and embedded-JetStream stores. Ports TestKeyPairs, TestTagManagement, TestRouteTableValidation, and TestReplaceRouteConvergence from tests/e2e/single so the integration tier drives genuine daemon-side business logic instead of StubSubject canned replies. vpcd (OVN) is out of scope, so its "vpc.create-sg"/"vpc.delete-sg" acknowledgements are stubbed with a fixed success reply — the SG record itself is still written by the real service impl before that event fires, so the stub only unblocks the synchronous round-trip rather than replacing tested behavior. * ci(e2e): gate live E2E jobs behind the in-process integration tier Runs the in-process integration suite first, on the ci-single runner host rather than inside a provisioned VM, so a broken control plane is caught before either live job pays the Spinifex install. Both e2e_single and e2e_multi now require it, and e2e_status treats a skipped integration job as a failure so the gating status cannot go green on a gate that never ran. Drops the shadow-mode suite selector: targeted per-diff suite selection is not the direction being pursued, and leaving a non-functional shadow job in the gating workflow only invited it to be mistaken for a live gate. * test(e2e): remove single/ tests superseded by daemon-lite ports TestKeyPairs, TestTagManagement, TestRouteTableValidation, and TestReplaceRouteConvergence now have proven-equivalent ports in tests/integration/ (mutation testing confirmed parity). Delete the e2e originals and their runX helpers, along with keypair_test.go's and tags_test.go's file-local helpers left with no caller. generateImportPubKey is shared by other single/ tests so it moves to helpers_test.go instead of being deleted. Slims the manifest-lint baseline for the removed create-call entries. * ci: run the lb e2e suite on multinode only Both e2e jobs filter SUITES by exclusion, and neither excluded lb, so the suite ran twice per e2e run: 354s on single and 654s on multi. Balancing across instances on one node is not a deployment mode we support, and ELBv2 proper requires two AZs, so the single-node run was redundant. Drops ~354s from the single-node job, which is the gate critical path. * test(integration): port 3 of 7 ECR tests to the in-process tier Wires the gateway's ECR control plane (repository/policy/lifecycle metadata over an in-process daemon-lite NATS subscriber, blob and manifest bytes via a memory object store, JWT issuer/verifier keyed off the existing IAM master key) into the integration harness, and ports TestECRControlPlane, TestECRGetLoginPassword, and TestECRIAMAuthorization_AssumedRolePullOnly from tests/e2e/ecr — each proven equivalent by mutation (evaluatePrincipalPolicy forced to allow, and the repo-describe path forced to not-found) before its E2E original was removed. The other 4 authorization tests (PullOnly, ReadOnly, DetachPolicyDeniesImmediately, DeactivatedKeyReturns401) stay on the E2E tier: AttachUserPolicy lacks the AWS-managed-ARN existence-check bypass that AttachRolePolicy/AttachGroupPolicy already have, so attaching AmazonEC2ContainerRegistryPullOnly/ReadOnly straight to a user fails NoSuchEntity in both tiers alike. Left untouched pending a fix to that shared IAM code. * fix(iam): accept AWS-managed policy ARNs on AttachUserPolicy AttachUserPolicy called getPolicyByARN unconditionally, so attaching an AWS-managed ARN to a user returned NoSuchEntity. AttachRolePolicy and AttachGroupPolicy already skip that lookup for AWS-managed ARNs and store them opaquely; the user path was missed when that guard was introduced. ListAttachedUserPolicies had the mirror of the same gap: it resolved every ARN through getPolicyByARN and silently dropped the ones it could not find, so a managed ARN would disappear from the listing rather than round-trip. Evaluation already handled these correctly via resolveAttachedPolicy, so only the write and list paths needed the guard. * test(integration): port remaining 4 ECR authorization tests Ports TestECRIAMAuthorization_PullOnly, _ReadOnly, _DetachPolicyDeniesImmediately, and _DeactivatedKeyReturns401 from tests/e2e/ecr now that AttachUserPolicy accepts AWS-managed policy ARNs. Each was proven equivalent by mutation before its E2E original was removed: PullOnly/ReadOnly/DetachPolicyDeniesImmediately fail when evaluatePrincipalPolicy is forced to always allow, and DeactivatedKeyReturns401 fails when the access-key-active check in resolveECRLongLivedPrincipal is bypassed. All mutants were reverted and a clean run confirmed before committing. tests/e2e/ecr/iam_authorization_test.go is now fully removed — every test it held has an in-process equivalent, and its helpers had no other callers in the package. * ci: select e2e suites by allow-list instead of deny-list Both jobs filtered SUITES by exclusion, so any suite named in neither deny-list ran in both jobs. That is how lb came to run twice per e2e run, and it makes adding a suite silently fan it out everywhere. Each job now names the suites it is eligible to run. Selection is still driven by SUITES, so this is behaviour-preserving for the current default: single/iam/cert on the single-node job, multinode/lb/cert on the multi-node job, exactly as before. eks, ecs and storagegrowth are listed as eligible on the single-node job but are not yet in the default SUITES. * test(e2e): delete seven tests with no coverage TestFinalClusterStats and TestInstanceLaunch assert nothing beyond what other callers of their shared helpers already imply. The multinode baseline, volume, guest-SSH, and preflight tests duplicate single-node coverage or exercise per-guest behaviour that isn't cluster-specific. TestMultinodeInstanceDistribution can't fail: three independent RunInstances calls with no placement group make a strict spread assertion flaky and a loose one meaningless. Preflight checks (/dev/kvm writability, peer SSH reachability) move into requireMultiNodeFixture so the package fixture singleton still fails fast with a clear message instead of disappearing along with the dedicated test. The remaining callers of needInstanceTrio (CrossNodeOps, CrossNodeGateway, NodeFailure, VolumeDurability) still trigger the shared trio launch on their own, so removing the distribution and volume-lifecycle tests doesn't leave it unbuilt. Also slims the manifest-lint baseline entry for the deleted multinode volume test. * fix(e2e): require full 3-node spread in TestMultinodeSpread SpreadPlacement passed at unique>=3, warned via t.Logf at unique==2, and only failed below that. A 33% spread failure (two instances colocated) passed with a log line. The 243.6s NAT-egress scenario that follows only proves what it claims to if the private trio actually landed on three distinct chassis, so that premise needs to be a hard assertion, not a warning. * refactor(e2e): drop the unused spread-assertion harness helpers AssertSpreadAcrossNodes and CountInstancesPerNode lost their only caller when the instance-distribution test was removed. Beyond being dead, the assertion helper documented minNodes=1 as an acceptable logging-only mode inherited from the bash driver, which is the pattern that let a placement test pass while every instance landed on one node. InstanceHostingNode and its ps helpers stay; five multinode tests use them. * fix(ec2): stop leaked goroutines on prepareEFIVolume error paths newViperblock starts the chunk uploader and WAL syncer unconditionally, but prepareEFIVolume only released them on the happy path via Close(). Six error returns between acquisition and Close left both goroutines running. Add a defer immediately after acquisition, matching the sibling prepareRootVolume; Close() remains the durability boundary. * test(predastore): add real predastore fixture for integration tests Extract predastore_integration_test.go's inline daemon startup into a reusable testutil/predastore package so tests/integration can point DaemonLite-style wiring at a real S3-compatible backend instead of an unreachable host. Kept as its own leaf package rather than folding into the shared testutil package: it transitively imports predastore's quicclient, whose package-level connection pool starts a permanent goroutine at init that would trip goleak checks in any of the ~80 other packages that import testutil for unrelated NATS helpers. * test(integration): migrate CreateVolume to assert real predastore persistence Add StartVolumeDaemonLite, an opt-in DaemonLite variant wired to the real predastore fixture, and migrate TestCreateVolume_PersistsToRealPredastore off memory-backed DaemonLite so CreateVolume/DescribeVolumes exercise viperblock's real S3 backend end to end instead of stopping at input validation. * test(integration): add real storage-state oracle for CreateVolume DescribeVolumes only proves the handler's own read path agrees with its own write path, both via the same deserialization; it cannot catch a bug in a field it never re-derives (e.g. a unit-conversion error in the bytes handed to viperblock, since VolumeMetadata.SizeGiB is copied from the request directly). vbscan reads a volume's persisted config.json and chunk objects straight over S3, independent of any spinifex handler code, so TestCreateVolume_PersistsToRealPredastore can assert on what viperblock actually wrote rather than only on what the handler claims. vbscan is a local reimplementation of scripts/vbrefscan's read-only scan rather than an import: spinifex cannot depend on mulga, the module that contains it. It only reimplements the config/chunk-listing half of vbrefscan — the checkpoint-based reference-integrity half needs viperblock.ParseBlockCheckpointBytes, which does not exist in the viperblock version spinifex currently depends on (v1.12.1) and is also moot for this volume: CreateVolume never calls SaveLiveCheckpoint, so a freshly created volume has no checkpoint to scan regardless of version. * refactor(test): move the predastore daemon fixture out of the coverage scope test-cover runs go test ./spinifex/..., so anything under spinifex/ is compiled and measured. The daemon fixture cannot be exercised by unit tests, so it reported a coverage gap no unit test could close. The e2e harness already avoids this by living under tests/; the fixture now does the same. vbscan stays put, being ordinary library code with its own unit tests. * test(integration): migrate RunInstances multi-count expansion off live guests RunInstances with MinCount/MaxCount=2 expands into a single per-node targeted launch (distributeInstances in gateway/ec2/instance/placement.go) before the daemon-facing NATS hop, so the real gateway router plus a stubbed daemon response is sufficient to prove it — no throwaway VM is needed to exercise the expansion logic itself. The live single-node E2E test's WaitForInstanceState/TerminateInstances sequence was teardown hygiene for the real guests it booted, not an assertion of the expansion behavior, so it is dropped rather than emulated. The migrated test additionally asserts the per-node launch request itself carries MinCount=MaxCount=2, proving the count actually propagates rather than the launch succeeding by coincidence or via the zero-daemon degenerate path in placement.go. * docs: drop a doc reference to the deleted live test * test(integration): migrate launch-template RunInstances expansion off live guests RunInstances --launch-template resolves ImageId/InstanceType from the stored template entirely gateway-side (expandLaunchTemplate, before validation/placement/the daemon-facing NATS hop), so proving it forwards the resolved fields never needed a real guest boot. Ports the assertion to tests/integration via a new opt-in StartLaunchTemplateDaemonLite (real LaunchTemplateServiceImpl over embedded JetStream) plus a direct subscription on the per-node launch subject to capture what the gateway actually dispatched, rather than only the client-visible response. The live test keeps its CRUD/versioning lifecycle (create/modify/delete version, tag-filtered describe) — none of it touches a guest — and drops only the RunInstances-from-template + wait-for-running + terminate tail, whose only job was waiting out a real boot to read back fields already proven resolved. * test(integration): migrate KV-only account-scoping checks off live tenants Ports the KeyPair/VPC-Subnet/IGW-EIGW/AccountSettings cross-account isolation assertions from tests/e2e/single/account_test.go to the in-process integration tier. Each of these resources resolves ownership from a plain account-scoped KV key (utils.AccountKey(accountID, id), or an account-keyed settings record) with no vm.Manager or real guest involved, so DaemonLite now also wires the real EgressOnlyIGWServiceImpl and AccountSettingsServiceImpl alongside the existing VPC/IGW/Key services. Two independently minted tenant accounts (via IAMService.CreateAccount/CreateUser/CreateAccessKey/CreatePolicy/ AttachUserPolicy, in-process — the same calls runAccountCreate makes without the CLI/stdout round-trip) replace the live tier's carousel so isolation is checked between real, separately owned accounts. Left live: instance scoping (Step2) and volume scoping (Step3) — checkInstanceOwnership, the actual cross-account gate for stop/start/ reboot/terminate/attach-volume, runs daemon-side only after a real vm.Manager lookup, so it cannot be exercised without a live guest. Snapshot scoping (Step5) needs a real backing volume for CreateSnapshot (viperblock, out of DaemonLite's scope), so it stays live entirely rather than split off only its two KV-only assertions. Global-resources (Step9) carries no account branching in the handlers at all, so asserting its equality across two accounts wouldn't exercise anything a fake node responder didn't already trivially satisfy — not migrated. * test(integration): migrate launch-template CRUD lifecycle off live guests TestLaunchTemplates booted zero guests: its RunInstances coverage had already moved to TestRunInstances_LaunchTemplateExpansion, leaving only template CRUD/versioning assertions occupying a full live-cluster slot. Port the remaining create/describe/version/modify/delete lifecycle, including the SourceVersion merge and a nested timestamp round-trip, into TestLaunchTemplates_CRUDLifecycle against the in-process integration gateway. * refactor(tests): move spot instance lifecycle coverage to integration tier RequestSpotInstances is a mock over on-demand RunInstances (no bidding, interruption, or reclamation), so booting real VMs in the e2e tier proved nothing spot-specific. Migrate the SIR CRUD lifecycle (request, describe, cancel, close-on-terminate, insufficient-capacity error) to a NATS-level integration test that exercises the real gateway orchestration and daemon service impl without a live guest. Real-VM-boot assertions are already covered by TestRunInstances_MultiCountExpansion and the e2e baseline tests. * fix(tests): share one embedded NATS server across the integration package ClientTokenStore in spinifex/gateway/ec2/instance is a process-wide sync.Once singleton bound to whichever *nats.Conn first initialises it. Every integration test booted its own embedded NATS server, so the first RunInstances-calling test captured the singleton, and any later test inherited a KV handle bound to that server's now-closed connection. TestMain now boots one embedded NATS+JetStream server for the whole package; every StartGateway call connects into its own dynamically provisioned NATS account (accountAuthenticator in shared_nats.go), giving each test a private JetStream namespace despite the shared server, and none of the per-test connections are closed until every test has finished - whichever one the singleton latches onto stays live for the run. Also wire the suite into the required PR gate: it was previously only invoked from e2e.yml's push-triggered (main/dev), self-hosted "Integration (front gate)" job, so a regression here never blocked a PR. The whole package runs in single-digit seconds, so it now runs inline in `make preflight` and as its own step in spinifex.yml's build_and_test job. * test(integration): migrate serial-console access checks off live guests Enable/Disable/GetSerialConsoleAccessEnabled is pure gateway+account-settings logic with no guest involved, so it moves to the in-process integration tier against the real AccountSettingsServiceImpl wired into DaemonLite. * test(integration): migrate DescribeImages round-trip off live guests runImage's AMI describe round-trip needs no real cluster: it registers an AMI against a hand-seeded snapshot via the real ImageServiceImpl, backed by a new StartImageDaemonLite helper wiring RegisterImage/DescribeImages onto an in-memory object store. * test(integration): migrate predastore object lifecycle off live guests runPredastoreObjectLifecycle only needs a real predastore daemon, not a tofu-provisioned cluster: it now runs against tests/fixtures/predastore's in-process fixture, covering the same create/put/get/delete/list contract. * test(integration): migrate ListFoundationModels catalog check off live cluster ListFoundationModels is pure gateway-side static catalog logic with no NATS/daemon hop, so it moves to the integration tier via a new BedrockClient harness accessor; this tier configures no credential resolver, so the Anthropic entry's absence is asserted explicitly rather than left optional. * test(integration): cover RunInstances client-token idempotency Proves replaying the same ClientToken through the real gateway returns the original reservation without a second daemon dispatch, and a different token launches independently. * test(integration): cover RunInstances launch-template expansion Proves version selection (omitted Version resolves to the template default, explicit Version selects that version) and field precedence (explicit RunInstances parameters override the template, unset fields inherit it) through the real gateway. * test(integration): cover RunInstances malformed-input rejection Proves a malformed ImageId and a malformed targeted-launch CapacityReservationId are rejected before any NATS dispatch, with no daemon stub registered so a regression that let either through would fail fast instead of hanging. * test(integration): cover RunInstances iam:PassRole enforcement Proves a caller granted ec2:RunInstances but not iam:PassRole on an instance profile's role is denied, and the identical session succeeds once PassRole is granted for that exact role ARN. * test(integration): cover RunInstances placement-group strategy routing Adds a placement-group daemon-lite wiring a real PlacementGroupServiceImpl over NATS, then proves spread strategy assigns exactly one instance per node while cluster strategy pins the whole launch to the single highest-capacity node. * test(integration): cover RunInstances capacity distribution across nodes Proves the default (non-placement-group) path packs a multi-instance launch across nodes proportionally to spare capacity rather than exhausting one node first. * test(e2e): merge SameSGComms and SGToSGDatapath around one client/target pair Both booted their own instance pair to check SG-to-SG datapath enforcement and same-default-SG east-west connectivity. Merging them around one shared client/target pair cuts four boots to two, and converting the inline harness.Step sequence into named t.Run stages (PortGroupMembership, AllowedTraffic, DeniedTraffic, RevokeAndVerifyBlocked, ReauthorizeAndVerifyRestored, SameSGComms) gives each its own pass/fail attribution instead of one opaque test result. SameSGComms now runs last and joins the default SG onto each ENI via ModifyNetworkInterfaceAttribute only after every SG-to-SG allow/deny assertion has already recorded its own result, since the default SG's self-reference rule admits all traffic between co-members and would otherwise defeat the denied-traffic check. * test(e2e): merge default-SG reachability, guest DNS, and egress enforcement DefaultSGReachabilityBaseline, GuestDNSResolution, and SecurityGroupEgress each booted their own instance to check one facet of "a guest is reachable only when policy says so": ingress deny/authorize, resolver + WAN egress, and egress ACL enforcement. Merging them around one shared guest cuts three boots to one, with named t.Run stages (Deny, Authorize, Resolver, WANEgress, RevokeEgress/RestoreEgress) giving each its own attribution. The egress revoke/restore round-trip now mutates this scenario's own dedicated SG instead of the shared default SG the rest of the suite depends on, isolating its egress-rule mutation from every other test. * fix(ci): single-quote e2e test_run regex across the ssh boundary * fix(e2e): anchor SG egress-enforcement probe to a WAN IP, not the gateway * test(e2e): merge single-node guest churn tests into one durability scenario Combines TestSSHProbe, TestENIHotplug, TestENIHotplugReconcile, TestVolumeDurability, TestModifyInstanceAttribute, and TestRebootInstance into TestGuestChurnDurability, a single shared-guest scenario that re-verifies a data-integrity sentinel sha256 after every perturbation stage (ENI hotplug, daemon restart, ENI detach, instance-type resize, stop/start, reboot). This proves the sentinel survives the full churn sequence rather than just one isolated stop/start. * fix(vpc): clear subnet drop gate when installing egress reroute A subnet with an attached IGW but no 0.0.0.0/0 route carries a priority-1100 drop gate on the VPC router. When a route later lands (NATGW reroute at 900, IGW reroute at 1000) the reroute is installed but the drop gate was never removed, so the higher-priority gate wins and black-holes the subnet's egress in lr_in_policy before it reaches routing or SNAT. This left NAT-gateway private-subnet egress with 100% packet loss: the SNAT stage saw zero packets and no conntrack was committed. Remove the subnet's drop gate for the same prefix once a reroute is installed. Idempotent and prefix-scoped, so a specific-prefix reroute is a no-op and leaves the default gate intact. * test(e2e): merge VPC subnet, NAT gateway, and EIP single-node tests Merges the former TestVPCSubnetE2E, TestNewVPCEgressBaseline, TestNATGateway, and TestInstanceEIP into one staged scenario, TestVPCEgressPaths, around one scenario-owned VPC and its two guests, cutting five guest boots to three. The public guest serves PublicSubnetEgress, then the NAT Gateway bastion hop, then (last, since association rewrites its public IP) the EIP flip. The private guest serves the NAT Gateway path, launched directly into the same VPC's already-existing private subnet instead of a dedicated one. RouteBeforeSubnet keeps its own second VPC and instance since it exercises the implicit main-route-table code path, a different one than the explicit route table this scenario's own VPC uses for its public subnet — collapsing the two would drop coverage of whichever path got merged away. The NAT Gateway up/down flip now repeats for an env-tunable round count (SPINIFEX_VPCEGRESS_NAT_ROUNDS, default 1), matching every other merged single-node scenario. * test: TEMPORARY mutation-check — revert |
||
|
dependabot[bot]
|
8c8dc173bd |
chore: bump the npm-dependencies group (#554)
Bumps the npm-dependencies group in /spinifex/services/spinifexui/frontend with 21 updates: | Package | From | To | | --- | --- | --- | | [@aws-sdk/client-acm](https://github.com/aws/aws-sdk-js-v3/tree/HEAD/clients/client-acm) | `3.1084.0` | `3.1089.0` | | [@aws-sdk/client-ec2](https://github.com/aws/aws-sdk-js-v3/tree/HEAD/clients/client-ec2) | `3.1084.0` | `3.1089.0` | | [@aws-sdk/client-ecr](https://github.com/aws/aws-sdk-js-v3/tree/HEAD/clients/client-ecr) | `3.1084.0` | `3.1089.0` | | [@aws-sdk/client-ecs](https://github.com/aws/aws-sdk-js-v3/tree/HEAD/clients/client-ecs) | `3.1084.0` | `3.1089.0` | | [@aws-sdk/client-eks](https://github.com/aws/aws-sdk-js-v3/tree/HEAD/clients/client-eks) | `3.1084.0` | `3.1089.0` | | [@aws-sdk/client-elastic-load-balancing-v2](https://github.com/aws/aws-sdk-js-v3/tree/HEAD/clients/client-elastic-load-balancing-v2) | `3.1084.0` | `3.1089.0` | | [@aws-sdk/client-iam](https://github.com/aws/aws-sdk-js-v3/tree/HEAD/clients/client-iam) | `3.1084.0` | `3.1089.0` | | [@aws-sdk/client-s3](https://github.com/aws/aws-sdk-js-v3/tree/HEAD/clients/client-s3) | `3.1084.0` | `3.1089.0` | | [@aws-sdk/client-sts](https://github.com/aws/aws-sdk-js-v3/tree/HEAD/clients/client-sts) | `3.1084.0` | `3.1089.0` | | [@aws-sdk/lib-storage](https://github.com/aws/aws-sdk-js-v3/tree/HEAD/lib/lib-storage) | `3.1084.0` | `3.1089.0` | | [@smithy/protocol-http](https://github.com/smithy-lang/smithy-typescript/tree/HEAD/packages/protocol-http) | `5.5.7` | `5.5.10` | | [@smithy/signature-v4](https://github.com/smithy-lang/smithy-typescript/tree/HEAD/packages/signature-v4) | `5.6.3` | `5.6.6` | | [@tanstack/react-router](https://github.com/TanStack/router/tree/HEAD/packages/react-router) | `1.170.17` | `1.170.18` | | [@tailwindcss/vite](https://github.com/tailwindlabs/tailwindcss/tree/HEAD/packages/@tailwindcss-vite) | `4.3.2` | `4.3.3` | | [@tanstack/router-plugin](https://github.com/TanStack/router/tree/HEAD/packages/router-plugin) | `1.168.19` | `1.168.20` | | [oxfmt](https://github.com/oxc-project/oxc/tree/HEAD/npm/oxfmt) | `0.58.0` | `0.59.0` | | [oxlint](https://github.com/oxc-project/oxc/tree/HEAD/npm/oxlint) | `1.73.0` | `1.74.0` | | [oxlint-tsgolint](https://github.com/oxc-project/tsgolint) | `0.24.0` | `0.25.0` | | [tailwindcss](https://github.com/tailwindlabs/tailwindcss/tree/HEAD/packages/tailwindcss) | `4.3.2` | `4.3.3` | | [ultracite](https://github.com/haydenbleasel/ultracite) | `7.9.3` | `7.9.4` | | [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) | `8.1.4` | `8.1.5` | Updates `@aws-sdk/client-acm` from 3.1084.0 to 3.1089.0 - [Release notes](https://github.com/aws/aws-sdk-js-v3/releases) - [Changelog](https://github.com/aws/aws-sdk-js-v3/blob/main/clients/client-acm/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-js-v3/commits/v3.1089.0/clients/client-acm) Updates `@aws-sdk/client-ec2` from 3.1084.0 to 3.1089.0 - [Release notes](https://github.com/aws/aws-sdk-js-v3/releases) - [Changelog](https://github.com/aws/aws-sdk-js-v3/blob/main/clients/client-ec2/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-js-v3/commits/v3.1089.0/clients/client-ec2) Updates `@aws-sdk/client-ecr` from 3.1084.0 to 3.1089.0 - [Release notes](https://github.com/aws/aws-sdk-js-v3/releases) - [Changelog](https://github.com/aws/aws-sdk-js-v3/blob/main/clients/client-ecr/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-js-v3/commits/v3.1089.0/clients/client-ecr) Updates `@aws-sdk/client-ecs` from 3.1084.0 to 3.1089.0 - [Release notes](https://github.com/aws/aws-sdk-js-v3/releases) - [Changelog](https://github.com/aws/aws-sdk-js-v3/blob/main/clients/client-ecs/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-js-v3/commits/v3.1089.0/clients/client-ecs) Updates `@aws-sdk/client-eks` from 3.1084.0 to 3.1089.0 - [Release notes](https://github.com/aws/aws-sdk-js-v3/releases) - [Changelog](https://github.com/aws/aws-sdk-js-v3/blob/main/clients/client-eks/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-js-v3/commits/v3.1089.0/clients/client-eks) Updates `@aws-sdk/client-elastic-load-balancing-v2` from 3.1084.0 to 3.1089.0 - [Release notes](https://github.com/aws/aws-sdk-js-v3/releases) - [Changelog](https://github.com/aws/aws-sdk-js-v3/blob/main/clients/client-elastic-load-balancing-v2/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-js-v3/commits/v3.1089.0/clients/client-elastic-load-balancing-v2) Updates `@aws-sdk/client-iam` from 3.1084.0 to 3.1089.0 - [Release notes](https://github.com/aws/aws-sdk-js-v3/releases) - [Changelog](https://github.com/aws/aws-sdk-js-v3/blob/main/clients/client-iam/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-js-v3/commits/v3.1089.0/clients/client-iam) Updates `@aws-sdk/client-s3` from 3.1084.0 to 3.1089.0 - [Release notes](https://github.com/aws/aws-sdk-js-v3/releases) - [Changelog](https://github.com/aws/aws-sdk-js-v3/blob/main/clients/client-s3/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-js-v3/commits/v3.1089.0/clients/client-s3) Updates `@aws-sdk/client-sts` from 3.1084.0 to 3.1089.0 - [Release notes](https://github.com/aws/aws-sdk-js-v3/releases) - [Changelog](https://github.com/aws/aws-sdk-js-v3/blob/main/clients/client-sts/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-js-v3/commits/v3.1089.0/clients/client-sts) Updates `@aws-sdk/lib-storage` from 3.1084.0 to 3.1089.0 - [Release notes](https://github.com/aws/aws-sdk-js-v3/releases) - [Changelog](https://github.com/aws/aws-sdk-js-v3/blob/main/lib/lib-storage/CHANGELOG.md) - [Commits](https://github.com/aws/aws-sdk-js-v3/commits/v3.1089.0/lib/lib-storage) Updates `@smithy/protocol-http` from 5.5.7 to 5.5.10 - [Release notes](https://github.com/smithy-lang/smithy-typescript/releases) - [Changelog](https://github.com/smithy-lang/smithy-typescript/blob/main/packages/protocol-http/CHANGELOG.md) - [Commits](https://github.com/smithy-lang/smithy-typescript/commits/@smithy/protocol-http@5.5.10/packages/protocol-http) Updates `@smithy/signature-v4` from 5.6.3 to 5.6.6 - [Release notes](https://github.com/smithy-lang/smithy-typescript/releases) - [Changelog](https://github.com/smithy-lang/smithy-typescript/blob/main/packages/signature-v4/CHANGELOG.md) - [Commits](https://github.com/smithy-lang/smithy-typescript/commits/@smithy/signature-v4@5.6.6/packages/signature-v4) Updates `@tanstack/react-router` from 1.170.17 to 1.170.18 - [Release notes](https://github.com/TanStack/router/releases) - [Changelog](https://github.com/TanStack/router/blob/main/packages/react-router/CHANGELOG.md) - [Commits](https://github.com/TanStack/router/commits/@tanstack/react-router@1.170.18/packages/react-router) Updates `@tailwindcss/vite` from 4.3.2 to 4.3.3 - [Release notes](https://github.com/tailwindlabs/tailwindcss/releases) - [Changelog](https://github.com/tailwindlabs/tailwindcss/blob/main/CHANGELOG.md) - [Commits](https://github.com/tailwindlabs/tailwindcss/commits/v4.3.3/packages/@tailwindcss-vite) Updates `@tanstack/router-plugin` from 1.168.19 to 1.168.20 - [Release notes](https://github.com/TanStack/router/releases) - [Changelog](https://github.com/TanStack/router/blob/main/packages/router-plugin/CHANGELOG.md) - [Commits](https://github.com/TanStack/router/commits/@tanstack/router-plugin@1.168.20/packages/router-plugin) Updates `oxfmt` from 0.58.0 to 0.59.0 - [Release notes](https://github.com/oxc-project/oxc/releases) - [Changelog](https://github.com/oxc-project/oxc/blob/main/npm/oxfmt/CHANGELOG.md) - [Commits](https://github.com/oxc-project/oxc/commits/oxfmt_v0.59.0/npm/oxfmt) Updates `oxlint` from 1.73.0 to 1.74.0 - [Release notes](https://github.com/oxc-project/oxc/releases) - [Changelog](https://github.com/oxc-project/oxc/blob/main/npm/oxlint/CHANGELOG.md) - [Commits](https://github.com/oxc-project/oxc/commits/oxlint_v1.74.0/npm/oxlint) Updates `oxlint-tsgolint` from 0.24.0 to 0.25.0 - [Release notes](https://github.com/oxc-project/tsgolint/releases) - [Commits](https://github.com/oxc-project/tsgolint/compare/v0.24.0...v0.25.0) Updates `tailwindcss` from 4.3.2 to 4.3.3 - [Release notes](https://github.com/tailwindlabs/tailwindcss/releases) - [Changelog](https://github.com/tailwindlabs/tailwindcss/blob/main/CHANGELOG.md) - [Commits](https://github.com/tailwindlabs/tailwindcss/commits/v4.3.3/packages/tailwindcss) Updates `ultracite` from 7.9.3 to 7.9.4 - [Release notes](https://github.com/haydenbleasel/ultracite/releases) - [Commits](https://github.com/haydenbleasel/ultracite/compare/ultracite@7.9.3...ultracite@7.9.4) Updates `vite` from 8.1.4 to 8.1.5 - [Release notes](https://github.com/vitejs/vite/releases) - [Changelog](https://github.com/vitejs/vite/blob/main/packages/vite/CHANGELOG.md) - [Commits](https://github.com/vitejs/vite/commits/v8.1.5/packages/vite) --- updated-dependencies: - dependency-name: "@aws-sdk/client-acm" dependency-version: 3.1089.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: npm-dependencies - dependency-name: "@aws-sdk/client-ec2" dependency-version: 3.1089.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: npm-dependencies - dependency-name: "@aws-sdk/client-ecr" dependency-version: 3.1089.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: npm-dependencies - dependency-name: "@aws-sdk/client-ecs" dependency-version: 3.1089.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: npm-dependencies - dependency-name: "@aws-sdk/client-eks" dependency-version: 3.1089.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: npm-dependencies - dependency-name: "@aws-sdk/client-elastic-load-balancing-v2" dependency-version: 3.1089.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: npm-dependencies - dependency-name: "@aws-sdk/client-iam" dependency-version: 3.1089.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: npm-dependencies - dependency-name: "@aws-sdk/client-s3" dependency-version: 3.1089.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: npm-dependencies - dependency-name: "@aws-sdk/client-sts" dependency-version: 3.1089.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: npm-dependencies - dependency-name: "@aws-sdk/lib-storage" dependency-version: 3.1089.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: npm-dependencies - dependency-name: "@smithy/protocol-http" dependency-version: 5.5.10 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: npm-dependencies - dependency-name: "@smithy/signature-v4" dependency-version: 5.6.6 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: npm-dependencies - dependency-name: "@tanstack/react-router" dependency-version: 1.170.18 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: npm-dependencies - dependency-name: "@tailwindcss/vite" dependency-version: 4.3.3 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: npm-dependencies - dependency-name: "@tanstack/router-plugin" dependency-version: 1.168.20 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: npm-dependencies - dependency-name: oxfmt dependency-version: 0.59.0 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: npm-dependencies - dependency-name: oxlint dependency-version: 1.74.0 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: npm-dependencies - dependency-name: oxlint-tsgolint dependency-version: 0.25.0 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: npm-dependencies - dependency-name: tailwindcss dependency-version: 4.3.3 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: npm-dependencies - dependency-name: ultracite dependency-version: 7.9.4 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: npm-dependencies - dependency-name: vite dependency-version: 8.1.5 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: npm-dependencies ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
dependabot[bot]
|
f02d9a6c02 |
chore: bump the go-dependencies group with 2 updates (#553)
Bumps the go-dependencies group with 2 updates: [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) and [github.com/aws/aws-sdk-go-v2/credentials](https://github.com/aws/aws-sdk-go-v2). Updates `github.com/aws/aws-sdk-go-v2/config` from 1.32.29 to 1.32.30 - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Commits](https://github.com/aws/aws-sdk-go-v2/compare/config/v1.32.29...config/v1.32.30) Updates `github.com/aws/aws-sdk-go-v2/credentials` from 1.19.28 to 1.19.29 - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Commits](https://github.com/aws/aws-sdk-go-v2/compare/credentials/v1.19.28...credentials/v1.19.29) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2/config dependency-version: 1.32.30 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: go-dependencies - dependency-name: github.com/aws/aws-sdk-go-v2/credentials dependency-version: 1.19.29 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: go-dependencies ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
dependabot[bot]
|
5dc945a6df |
chore: bump the github-actions group with 2 updates (#552)
Bumps the github-actions group with 2 updates: [actions/setup-go](https://github.com/actions/setup-go) and [actions/setup-node](https://github.com/actions/setup-node). Updates `actions/setup-go` from 6 to 7 - [Release notes](https://github.com/actions/setup-go/releases) - [Commits](https://github.com/actions/setup-go/compare/v6...v7) Updates `actions/setup-node` from 6 to 7 - [Release notes](https://github.com/actions/setup-node/releases) - [Commits](https://github.com/actions/setup-node/compare/v6...v7) --- updated-dependencies: - dependency-name: actions/setup-go dependency-version: '7' dependency-type: direct:production update-type: version-update:semver-major dependency-group: github-actions - dependency-name: actions/setup-node dependency-version: '7' dependency-type: direct:production update-type: version-update:semver-major dependency-group: github-actions ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
Josh Siviour
|
a01e58d90c |
feat: deliver backend ENOSPC to the guest and enable chunk GC in production (#549)
* feat: plumb GCEnabled config through to every VB construction site viperblock chunk GC is gated behind VB.GCEnabled, but spinifex never set it, so every deployed node ran with GC off by construction regardless of which viperblock binary shipped. Add a [viperblock] gc_enabled config key (default false, mirrors shardwal) and thread it through to the three places spinifex builds a VB: the nbdkit plugin it spawns per mounted volume (via a new gc_enabled plugin param), viperblockd's detached-volume opens, and the volume service's CreateVolume. * chore: repin viperblock to GC-enable lineage The gc-enable plumbing references viperblock.VB.GCEnabled, which is absent from the previously pinned v1.12.1. Repin to the feat/nbd-gc-enable commit so a standalone (GOWORK=off) build — the path the ansible deploy uses — resolves the field, and spx plus the nbdkit plugin build from the same GC-bearing viperblock lineage. * fix(storagegrowth): stop launching a nano for image-pull repro ensureImagePullInstance shared harness.DiscoverNanoInstanceType with the synthetic npass suite. A 512MB guest OOM-thrashes containerd before any disk-driven ENOSPC and conflates the storage leak with guest-memory pressure. Add a package-local imagePullInstanceType helper: an exact override via SPINIFEX_STORAGEGROWTH_INSTANCE_TYPE, otherwise the smallest advertised type at or above SPINIFEX_STORAGEGROWTH_MIN_MEM_MIB (default 16384 MiB). npass keeps using the nano selector unchanged. * feat: report guest block write errors so backend ENOSPC reaches the guest Set werror=report and rerror=report on the guest boot drive so a viperblock ENOSPC (or any other backend write failure) is surfaced to the guest as a real I/O error instead of QEMU silently pausing the VM. Without this the guest kernel never learns its disk is out of space, so it can't fail writes cleanly or let an operator react. * chore: repin predastore and viperblock to the storage-capacity-safety lineage Drop the local-path replace for predastore (a test shortcut) and pin both predastore and viperblock to their feat/storage-capacity-safety commits, so a standalone (GOWORK=off) build resolves the free-space watermark, ENOSPC classification, and backpressure plumbing that spinifex's werror/rerror change depends on. * chore: bump viperblock to 054a44a3f0f5 Realigns spx's linked viperblock with the submodule working tree, which the build role's source-drift gate compares against. * chore: repin predastore and viperblock to their dev lineage Both were pinned to pseudo-versions off feat/storage-capacity-safety. That work has now landed on dev in each repo, and both PRs were squash-merged, so the branch commits those pins named are unreachable from any branch. Repin to the dev commits carrying the same code: predastore 5699329 and viperblock 35dd9f2. |
||
|
Josh Siviour
|
08fc5e4c58 |
fix(northstar): initialise OTel SDK and source the telemetry environment (#555)
Northstar carried working northstar.dns.* instruments but shipped no telemetry, so its dashboard was empty and the only northstar data in Elasticsearch was journald scraped under the systemd unit name. Two gaps, both outside the instrumentation. northstarStartCmd never called initTelemetry, unlike every other service command, so the instruments recorded into a no-op provider. The unit also did not source /etc/spinifex/telemetry.env, which every other application unit reads for OTEL_EXPORTER_OTLP_ENDPOINT, MULGA_ENV and MULGA_SOURCE — so even an initialised SDK would have had no endpoint and no env label. Add an invariant covering the second gap: any unit launching an spx service must source the telemetry drop-in. A service missing it starts healthy and silently reports nothing, which only surfaces when someone notices an empty dashboard. Verified the test fails against the unpatched unit. |
||
|
Josh Siviour
|
ffc34b735b |
ci: emit per-run ES summary doc from e2e workflow (#551)
Add e2e_es_summary, a self-hosted job that aggregates suite-status.txt from both e2e legs and POSTs one doc per run to the Elastic sink, keyed by labels.ci_run_id, so Kibana can answer "did run X pass, and where are its artifacts" without hand-aggregating traces-apm*/logs-apm.error*. Runs on ci-single (LAN route to the sink) since e2e_status stays on GitHub-hosted ubuntu-26.04 for the PR status-report check. State is computed with the same single_ok/multi_ok logic as e2e_status so the two can never disagree. |
||
|
Josh Siviour
|
055fa26f37 |
feat(bedrock): Ochre Phase 2 — ConverseStream + InvokeModelWithResponseStream (#526)
* feat(bedrock): Ochre streaming — ConverseStream + InvokeModelWithResponseStream
Adds the two streaming bedrock-runtime operations behind a shared
eventstream.Encoder frame writer: ConverseStream reframes vLLM/Anthropic SSE
into the normalized Bedrock event taxonomy (messageStart -> contentBlock*
-> messageStop -> metadata), InvokeModelWithResponseStream wraps each
provider-native chunk in a PayloadPart frame. Fixes otelsetup's
statusRecorder to implement Flusher/Unwrap so flushes actually reach the
socket through the middleware chain, which streaming depends on.
* fix(bedrock): keep vLLM ConverseStream taxonomy ordered on edge cases
Route contentBlockStop/messageStop through an idempotent ensureStarted so a
usage-only chunk arriving before any finish_reason no longer emits metadata
ahead of messageStop, and a zero-content stream no longer emits stop events
without a preceding messageStart. Regression tests for both.
* fix(bedrock): route /model/ data-plane calls to bedrock-runtime
AWS SDKs sign bedrock-runtime with the SigV4 signing name "bedrock", the
same as the control plane; the two services are separated only by endpoint
hostname. The gateway serves one endpoint and routed purely on the signing
name, so every /model/{id}/converse*|invoke* call landed in Bedrock_Request
(control plane) and returned InvalidAction — no standard SDK/CLI could reach
any bedrock-runtime endpoint.
Disambiguate by path in GetService: a "bedrock"-scoped request to a /model/
path (data-plane-exclusive) resolves to bedrock-runtime. Control-plane paths
(/foundation-models, /custom-models, /model-invocation-job) are unaffected.
* test(bedrock): satisfy testifylint go-require/empty in streaming tests
Use assert (guarded where a value is dereferenced afterward) instead of
require inside httptest handler closures, and assert.Empty for empty-string
checks, per dev's stricter testifylint config.
|
||
|
Josh Siviour
|
8205f88b04 |
ci: build sibling repos from a co-named branch, falling back to dev (#550)
The Build/Lint/Race jobs checked out viperblock and predastore at a hardcoded ref: dev and overlaid them with a go workspace, so the go.mod pins were ignored and a spinifex change depending on an unmerged sibling change could never compile in CI. Resolve each sibling to a branch of the same name when it exists on that repo, else dev, mirroring the e2e path in scripts/tofu-cluster/build-install-artifacts.sh so unit CI and e2e CI agree. |
||
|
Josh Siviour
|
908bfbfc27 |
fix(ecr): enforce IAM authorization on OCI registry operations (#548)
Treat the ECR registry JWT as a signed identity pointer rather than a permission grant. Every /v2/* request now rehydrates the token's access key against current IAM/STS state, maps the OCI route to an ecr: action and repository ARN, and runs the identity-policy evaluator before dispatch. PullOnly/ReadOnly principals can no longer push, overwrite tags, or delete manifests, and key/session/user/role/account/policy revocation takes effect on the next request instead of at token expiry. Cross-repository blob mounts now require source-read permission and proof the digest is referenced by a manifest in the named source repo, closing an account-wide blob-fishing gap. |
||
|
Josh Siviour
|
95631bf57b |
fix(elbv2): roll back managed SG on sync LB launch failure and surface the cause (#547)
A CreateLoadBalancerSync whose data-plane launch failed returned without rolling back the NLB managed security group. The orphaned SG sat in the EKS managed control-plane VPC and blocked DeleteVpc with DependencyViolation indefinitely, wedging the cluster in DELETING. The failure was also flattened to ServerInternal, discarding the only useful diagnostic. The sync path now unwinds fully on launch failure (ENIs, managed SG, name claim, record) and maps the launch error onto its real cause: capacity shortfalls pass through as their own code, anything else stays ServerInternal. The async path deliberately keeps its failed record — the caller holds the ARN and can reclaim it via DeleteLoadBalancer. |
||
|
joshsiv-mulga
|
9ccf147fc8 |
Merge remote-tracking branch 'origin/feat/storagegrowth-rootvol' into harness-merge-dev
# Conflicts: # tests/e2e/Makefile |
||
|
Julian Sommer
|
501f787c36 | refactor(ui): extract AttachedPoliciesPanel and SystemImageRequired (#546) | ||
|
Julian Sommer
|
5b03cf72c4 | fix: prevent live volume tag clobbering (#545) | ||
|
joshsiv-mulga
|
72353e476c |
test(e2e): read the live checkpoint on-node and target the root volume
The image-pull storage-growth workload HEADed <volID>/checkpoints/blocks.live.bin with the operator box's gateway identity, which has no S3 read grant on the bucket and was refused 403 on every sample, so checkpoint_bytes read 0 across a whole run. It now HEADs the object on the node over PeerSSH using predastore's own [[auth]] credentials from predastore.toml (root-owned, so read via passwordless sudo; the CA is readable through the spinifex group). The credentials are read into shell locals, used inline for one aws s3api head-object, then unset — never echoed, never leaving the node. A probe failure that is not a legitimate 404 voids the sample (checkpoint_bytes -1 plus a probe error) rather than reporting a false zero, so a dead instrument can never look like no growth. The dead operator-side S3 client path is removed. |
||
|
Julian Sommer
|
3b44d3969e | remove git hook | ||
|
Ben Duncan
|
28a876d17d | feat(dns): resolvable ALB/NLB + EKS DNS, forwarder hardening, cross-tenant reconcile (#532) | ||
|
Julian Sommer
|
0c6a3c5b3a | Merge branch 'main' into dev | ||
|
Ben Duncan
|
8d796d74ba |
Rewrite README key sections and banner in the site voice
Align the top of the README with mulgadc.com: new headline and intro, rewritten "What is Spinifex?", a new "Three ways to deploy" section (Neocloud / on-premise / edge) and a new "AWS compatibility" service matrix (EC2, EBS, S3, VPC, IAM, ALB/NLB, EKS, ECR, ECS available; RDS Q3 2026) linking the GitHub releases. Refresh the architecture prose and Key Features, add Deploy and AWS-services nav links, and swap banner.svg for the ember-on-ink mini-stack design matching platform.svg. |
||
|
Ben Duncan
|
bc9dc36c6c |
Redesign architecture.svg in the Spinifex site theme
Rebuild the message-driven architecture diagram in the ember-on-ink theme to match platform.svg: ember gradient NATS bus, ember-tinted gateway and daemon cards, neutral clients and backend cards. Content unchanged (clients → gateway → NATS → ec2/ebs/s3/vpc/iam daemons → local backends). De-slop the copy and refresh the img alt text. |
||
|
Ben Duncan
|
7a6b8908c0 |
Redesign platform.svg in the Spinifex site theme
Replace the platform diagram with a static SVG matching the mulgadc.com front-page stack diagram: ember accent, ink panel, the portable-stack box (Neocloud / on-prem / edge) wrapping the Spinifex AWS-compatible service layer, over standard Linux on CPU/GPU hardware. Self-contained and GitHub-safe (presentation attributes only). Refresh the img alt text. |
||
|
Josh Siviour
|
3b379fea38 |
fix(ec2,eks,ecs,images): root volume sizing + revive the rotted Ubuntu catalog entries (#542)
* fix(admin): round the AMI root volume up so the image fits
`spx admin images import` sized the root volume with integer division, which
floors: a 4.94 GiB image registered a 4 GiB volume. The image is then copied
into a volume too small to hold it, is truncated, and the guest boots with no
root partition — it stalls on the root device until systemd gives up, and
nothing anywhere reports an undersized volume.
floorVolumeSizeToAMI already exists to prevent exactly this ("Under-sizing
hangs the guest in dracut"), but it raises the caller's requested size to the
AMI's recorded VolumeSizeGiB — the same floored value — so it inherited the
mistake instead of catching it.
This stayed hidden because every current system image is an exact multiple of
1 GiB, where flooring and rounding up agree. It surfaced on a 5306470400-byte
image, which would not boot.
Round up instead, via a helper so the boundaries are testable: an exact
multiple must not gain a spare GiB, and one byte over must.
* fix(eks): default a nodegroup's root volume to 20 GiB, as AWS does
CreateNodegroup stored diskSize verbatim, so omitting it persisted 0 and
the launch skipped its block device mapping entirely, dropping the worker
onto whatever volume the AMI itself declares.
That was harmless while the node AMIs baked generous virtual disks — the
GPU image declares 16 GiB for ~2 GiB of content, so the fall-through
happened to leave plenty of room. It stops being harmless as soon as an
image is sized to its contents: the worker then boots onto a volume with
barely a spare GiB and cannot pull the container images it exists to run.
Sizing the node is the nodegroup's decision either way, not a property
the image should be able to dictate by accident.
20 GiB matches the AWS EKS managed-nodegroup default. It is resolved at
create rather than at launch so DescribeNodegroup reports the size the
node actually gets.
TestCreateNodegroup_NoDiskSizeOmitsBlockDeviceMapping asserted the old
fall-through and is replaced by its inverse; the explicit-size case is
already covered by TestCreateNodegroup_DiskSizePropagatesToBlockDeviceMapping.
* fix(ecs): size a container instance's root volume to 30 GiB by default
ProvisionCapacity built its RunInstances input without a block device
mapping, so a container instance inherited whatever volume its AMI
declares. EC2 has no default of its own to fall back on — the AMI is the
only thing that decides — and a node AMI is sized to hold the image, which
leaves the agent no room for the container images it exists to pull.
30 GiB mirrors what the AWS ECS-optimized AMI declares. ProvisionCapacity
is spinifex's own API rather than an AWS one, so DiskSize is added to its
input for callers that want to size a cluster explicitly, defaulting when
omitted the same way InstanceType and Count already do.
* fix(images): pull the Ubuntu catalog images from releases, not a pinned daily
Both Ubuntu catalog entries pointed at a dated build under the codename
dailies tree, which Canonical prunes as builds age out. That path now 404s,
so "spx admin images import --name ubuntu-26.04-x86_64" fails outright for
x86_64 and arm64 alike — the entries had rotted with nothing to catch it.
Release builds are retained rather than pruned, so the pin stays resolvable
while keeping a specific, reproducible build. The filenames under releases/
carry the version instead of the codename, which matters because the sums
entry is matched on the URL's basename.
CI is unaffected and could not have caught this: it fetches resolute/current/,
a symlink that never rots, so nothing exercised the pinned path.
Debian pins a dated directory too but is fine — those builds are retained,
and its entries still resolve.
|
||
|
Julian Sommer
|
73eeb87751 | chore: enable testifylint | ||
|
joshsiv-mulga
|
4cbd2a5c88 |
test(e2e): add container image-pull storage-growth workload
Pulls a container image inside a guest onto a viperblock volume and samples guest used bytes, host backing-store bytes, nbdkit RSS and the live checkpoint size as it goes, so backing-store cost can be fitted against the bytes the guest actually wrote. Complements the N-pass overwrite workload, which rewrites one extent and so pins the block map size — that shape cannot show whether cost grows in proportion to bytes written or faster than them. Redirects both dockerd's data-root and containerd's root onto the target volume. Redirecting data-root alone leaves containerd extracting layers into /var/lib/containerd on the AMI's small root disk, which fails with ENOSPC while the target volume sits empty. |
||
|
Julian Sommer
|
b5198d55e2 | chore: enable depguard to block log | ||
|
Julian Sommer
|
bf21e92caf | refactor(oidc): use fixed-width stdlib ECDSA point encoding | ||
|
Julian Sommer
|
ac5bd613b4 | chore: enable godot | ||
|
Julian Sommer
|
03d981779b | chore: enable errorlint | ||
|
joshsiv-mulga
|
d46d257278 |
fix(e2e): don't require CA cert when SPINIFEX_AWS_INSECURE=1
The storagegrowth gate called ResolveCACert unconditionally before building the AWS client, even though newAWSClient itself skips CA resolution entirely when SPINIFEX_AWS_INSECURE=1. That blocked the documented runner-resident remote path (env12 and similar clusters with no local CA cert on disk) and the skip message falsely claimed no node was running when the real cause was a missing CA. Skip the CA check under SPINIFEX_AWS_INSECURE=1 and state the actual cause plus the escape hatch in the skip message. |
||
|
Julian Sommer
|
f94a52eb80 | chore: enable lint rules | ||
|
joshsiv-mulga
|
6e043f6453 |
test(e2e): add storage-growth N-pass overwrite suite
Rewrites one logical extent N times through full attach/format-write/detach cycles, so the guest-visible footprint stays constant while total bytes written scales with N. That shape isolates the leak where viperblock mints chunk objects under unique monotonic keys and never deletes the superseded ones: a sequential write-once supersedes nothing and so cannot exercise it. The suite drives the workload only and deliberately does not measure the backend. Measuring on-disk bytes from here would mean either a host du (which cannot separate live from dead bytes, and so cannot distinguish this leak from the unrelated one where an overwritten key's old extent is orphaned without a tombstone) or a dependency on tooling outside this repository, which would stop the suite building standalone. It instead reports what it did to a caller-named result file and asserts only what a guest can observe: a sha256 round-trip through a fresh mount per pass, and a footprint that stays constant across them. The volume is retained on request, because deleting it removes the evidence: DeleteVolume enumerates the bucket by prefix and drops every key under it, including the leaked chunks. |
||
|
Julian Sommer
|
30d4745b6a | chore: move tailwindcss to dev deps | ||
|
Julian Sommer
|
3b7b781bdf | release: update go.mod for v1.12.1 v1.12.1 | ||
|
Josh Siviour
|
92b775951b |
fix(network): poll second-VPC transit SNAT in natuplink + log AddSNAT install (#541)
Phase 8 of TestNATUplink checked the second VPC router's lr-nat-list with a single non-polling assert; the igw-default-snat row commits a beat after the gateway LRP appears, so the read raced the OVN commit and intermittently saw an empty list (flaky CI red). Wrap it in EventuallyErr, matching the sibling LRP and host-route polls in the same phase. AddSNAT logged only on the idempotent-skip path, so the first install was silent and RCA could not tell a late SNAT from a missing one. Emit an install log keyed by router + vpc_cidr + external_ip. |
||
|
Josh Siviour
|
0a8e6bca15 |
fix(admin): always pin mgmt-bridge IP in server cert SAN (#540)
Server-cert IP SANs were built solely from live interface enumeration (DiscoverLocalIPs) plus bindIP, so the canonical mgmt-bridge IP entered the SAN set only when br-mgmt happened to be up at cert-generation time. On a host where br-mgmt is down when the cert is minted, 10.15.8.1 was silently dropped and the EKS control-plane publish to https://10.15.8.1 failed cert verify, leaving the cluster stuck CREATING. Pin config.DefaultMgmtBridgeIP into the extra SANs for both the init and join cert paths so publish succeeds regardless of interface state at generation time. Add a regression test asserting the SAN is present when br-mgmt is not a live interface. |
||
|
Josh Siviour
|
d3ac8c0656 |
test(eks): in-cluster IRSA pod e2e (AssumeRoleWithWebIdentity) (#535)
Add an IRSAPod subtest that applies a real pod wired for IRSA — a projected ServiceAccount token (aud sts.amazonaws.com) plus the AWS_ROLE_ARN / AWS_WEB_IDENTITY_TOKEN_FILE / AWS_ENDPOINT_URL_STS / AWS_CA_BUNDLE env the addon-sync agent injects — and asserts the pod's in-cluster aws sts get-caller-identity resolves to the assumed IRSA role. This closes the gap left by IRSAWebIdentity, which only exchanges the token host-side. Extract registerOIDCRole (shared with IRSAWebIdentity); the pod runs on a 1-node worker nodegroup (the control-plane node is CriticalAddonsOnly-tainted) and reaches awsgw STS over the worker subnet NAT. |
||
|
joshsiv-mulga
|
1281381096 |
fix(ci): publish-system-image.sh honors SYSTEM_IMAGE_BUILD_DIR
The build script derives BUILD_DIR from SYSTEM_IMAGE_BUILD_DIR (the AMI publish workflow points it off tmpfs so the 16G Ubuntu GPU raw fits), but the publish step hardcoded /tmp and could not find the freshly built raw. GPU legs failed 'Raw image not found'; Alpine legs silently reused stale pre-existing /tmp raws on the persistent runner. Mirror the build script's derivation so publish looks where the build actually wrote. |
||
|
Josh Siviour
|
b41a193cbc | fix(eks): make provider-id.sh the single k3s node-label authority for nodegroup + GPU labels (#539) | ||
|
Julian Sommer
|
53f636692d | fix(network): scrub stale dnat_and_snat by logical IP on EIP bind (#538) | ||
|
Julian Sommer
|
b40d538c61 | refactor(admin): remove dead --no-external flag from spx admin init (#537) | ||
|
joshsiv-mulga
|
b147e2b869 |
ci: make Setup Go Workspace idempotent on ci-single runner
The publish-eks-ami matrix fans out four legs on the single persistent ci-single runner, all reusing one workspace root. go work init is create-only and checkout never scrubs the root go.work, so legs after the first fail with 'go.work already exists'. Remove any stale go.work before init so each leg re-inits cleanly. |
||
|
Julian Sommer
|
744cb1929d | fix: admin init --force preserves keys, credentials and CA (#536) v1.12.0 | ||
|
joshsiv-mulga
|
2b44de27bc |
ci: publish eks-node-gpu + ecs-agent-gpu AMIs
Expand the publish workflow to build the two Ubuntu NVIDIA GPU variants alongside the Alpine eks-node/ecs-agent AMIs. The image build+publish scripts are already manifest-driven and Ubuntu-aware; only the workflow lacked the GPU legs. Refactor the single-job body into a matrix over the four image manifests, resolved by a new 'variant' input (all|cpu|gpu) so the slow 16G GPU legs can be skipped when only the CPU AMIs need republishing. GPU images do not fit a RAM-backed /tmp, so set SYSTEM_IMAGE_BUILD_DIR onto the runner's on-disk _temp; legs run serially (max-parallel 1) on the single ci runner. |
||
|
Julian Sommer
|
4d195eb0d9 | release: update go.mod for v1.12.0 | ||
|
Bryn Mailer
|
6cdb6000ce | feat: presigned urls (#533) | ||
|
Josh Siviour
|
4d8a34b545 |
fix(eks): apply eks.amazonaws.com/nodegroup label via k3s node-label drop-in (#534)
The nodegroup label was only written to K3S_NODE_LABEL in agent.env, but k3s has no such env var, so non-GPU workers never received the label. waitWorkersReady buckets Ready nodes by that label (added in #511), so the affected nodegroups never counted toward their ACTIVE transition and stayed CREATING until timeout. Ride the label on a real k3s node-label config drop-in (15-nodegroup-label.yaml), delivered via cloud-init user-data. Drop the inert K3S_NODE_LABEL from agent.env and the initd export. |
||
|
Julian Sommer
|
1b0a14909f | chore: enable prefer-named-capture-group rule | ||
|
Julian Sommer
|
8552937c86 | chore: enable react/no-object-type-as-default-prop rule |