mirror of
https://github.com/mulgadc/spinifex.git
synced 2026-10-20 11:21:49 +00:00
dev
7 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
5865fc418e | chore(ci): lock workflow action dependencies with actions.lock (#1207) | ||
|
|
c4f92608e1 |
fix: let the OCI principal topology pick a shape its tenancy has quota for (#1185)
Service limits are per tenancy. vm-single-principal now builds in its own, so the shape the other topologies share can have zero quota there, and the apply then fails before anything about the instance principal is exercised. Measured in ours: standard-e6-core-count is 0 available, against 13 for e5 and 10 for standard3. The first principal run died on 400-LimitExceeded for Core Instance, which reads as a capacity problem and is a shape with no quota. OCI_PRINCIPAL_SHAPE overrides it for that topology alone and defaults to the shape every other topology uses, so nothing changes where quota already exists. |
||
|
|
8409799cbf |
fix: default the OCI helper to the DEFAULT profile (#1174)
* fix: default the OCI helper to the DEFAULT profile oci_env.py shipped our own internal Oracle compartment profile as the default for every reader, with a second internal name as a fallback. A fresh tenancy's config holds only `[DEFAULT]`, so the shipped default was a name nobody else can have, and the fallback chain could land a deployment in whichever tenancy a different profile happened to point at. The profile is now `DEFAULT`, overridable by `OCI_CLI_PROFILE` or `--profile`, and a named profile the config does not hold is an error rather than a fallback. `DEFAULT` needs its own existence test, because configparser treats it as the fallback for every other section and `has_section` denies it exists even when `oci setup config` wrote nothing else. `eip-move-test.sh` and the config table in the README carried the same name. CI reads its credential from the environment rather than from a profile, so nothing there referenced it. * docs: look the OCI compartment OCID up by name The guide asked the reader to paste `ocid1.compartment.oc1..xxx` three times, and the `oci` CLI it already requires can resolve the name instead. The lookup needs its own check. `oci iam compartment list --query ... | [0]` exits 0 and prints nothing when the name matches nothing, so an unset `$CID` would otherwise reach Terraform and fail with an OCI error naming neither the compartment nor the cause. Two compartments in one tenancy can share a name, which the listing command shows and the lookup cannot resolve, so the paste form stays as the answer for that case. Records why the tenancy root is the wrong value: the step 2.1 policy is scoped to whatever `$CID` holds. * Revert "docs: look the OCI compartment OCID up by name" This reverts commit |
||
|
|
7c56855526 |
fix(oci): both credential routes in the guide, and a nightly topology that proves the principal one (#1173)
* docs(oci): offer both credential routes, and default the guide to the release Step 2 was one route only, the scoped API key, with the instance principal mentioned in a TIP at the end of it. It is now "2. OCI Credentials for Spinifex" with the two routes as 2.1 and 2.2, a table to choose between them, and the instance principal first because it leaves no key on any node. The guide also assumed the dev branch throughout. `git switch dev` and `--channel dev` are now an optional note for installing a pre-release, and the documented path is the published release. * test(oci): add a vm-single-principal topology, and cover the credential branch Instance-principal authentication was broken for a release while the OCI nightly stayed green, because the run's two axes never crossed: every topology that built the address allocator authenticated with an API key, and every run that authenticated as the instance was dispatched with no external pool, so the allocator was never constructed and the credential never used. vm-single-principal closes that. It is vm-single with the credential fixed by the topology rather than by a separate switch, so the name alone decides it, and pairing it with --no-external-pool is refused rather than run: without the allocator there is nothing for it to exercise. It joins the nightly's default list, and naming it alone is the way to test this one thing in a quarter of the full run. A unit test covers the same branch without a tenancy. newClient now reaches its two constructors through package variables a test can swap, which is the only way to observe the choice off an OCI instance - both real constructors need either an instance identity or a key file on disk. * fix(oci): let the tfvars choose the credential, and stop the guide hardcoding paths The deploy driver passed instance_principal as a -var, which outranks a tfvars file, so a deployment that set it there had it silently replaced with off and the nodes went looking for a key. Same defect class as the shape and node count, and the same fix: the flag is a choice about one run and still outranks everything, and without it the deployment's own tfvars decides. The driver now reads instance_principal from terraform.tfvars and *.auto.tfvars for its pre-apply gates, taking the last assignment as Terraform does, and checks that guess against a new Terraform output once the apply has run. They have to agree, because the credential handoff is skipped on the strength of it and a wrong answer leaves a node with no credential at all. The guide now carries one export rather than two more steps. SPX_PRINCIPAL is set in step 2.1, expanded into the tfvars step 4 already writes, and read back from there in step 5, so an instance-principal deployment runs the same command as a key-based one. Getting into the node needed an address copied out of the log. Every topology now writes its hosts file, not only the multi-node one that hands it to install-node.sh, and the run prints the ssh and destroy commands when --keep leaves the infrastructure up. The guide exports TOPOLOGY once, so the paths read the same for one VM, three, or bare metal. * fix(oci): print the ssh key path as ~ rather than expanded The --keep line read ssh -i /home/you/.ssh/oci-spx, which is the invoking user's home and differs on every machine. Collapsed to ~ when the key is under $HOME, left alone when it is not. |
||
|
|
2a0805b264 | chore: bump the github-actions group across 1 directory with 2 updates (#1159) | ||
|
|
46c0c44cc2 |
feat: run every Terraform workbook on OCI, and split the guide in two (#1163)
* feat: run every Terraform workbook on OCI, and split the guide in two The OCI topology driver ran the shared driver's default five workbooks, which left ECS and all three EKS variants untested on every platform even though their assertions already existed. It now passes an explicit nine and imports the three appliance images they need — RDS, ECS and EKS each boot their own, found by a spinifex:managed-by tag and never by name. demo-app stays out because it is not a workbook: it has no .tf at all, being the container image the EKS workbooks' nested workloads modules deploy, so listing it would fail on a missing root module rather than test anything. The guide is now two documents. oci-integration/README.md is the end-user path — six steps, deploy first, and it closes the three prerequisites nothing documented before: the clone, the ref, and the SSH keypair the driver refuses to start without. architecture.md holds the sizing, the IAM policy, every variable, the three filtering layers and the troubleshooting. Its firewall advice was also backwards. node_client_cidr_allow_list is an OCI security-list rule over the whole public subnet, and a guest's public address is a secondary private IP on the node's own VNIC, so narrowing it to the operator's address cuts internet ingress to every guest rather than protecting the node. The hardening section now arms the nft policy and narrows SSH through custom.nft, which is the layer that can tell a node's listener from a guest's. The deploy step names --channel dev rather than a pinned tag. The dev tag string grows with every cut, so a pinned example goes stale within the week. * fix(oci): the guide could not work on a fresh tenancy, because the profile name was ours oci_env.py bridges ~/.oci/config into TF_VAR_* — the provider takes the tenancy, user, fingerprint, key and region as Terraform variables and reads no config file itself — but the profile it looked for was the reference tenancy's, and validate-topology.sh passed no way to change it. A fresh account's config holds [DEFAULT], so every deploy would have stopped at "OCI profile 'apacanzset03child03' was not found" before building anything. The profile is now resolved as --oci-profile, then $OCI_CLI_PROFILE, then the reference tenancy, then DEFAULT. A name passed explicitly and absent is an error listing the profiles present, never a fallback: the failure worth preventing is building a cluster in whichever tenancy happened to be first. The reference tenancy still outranks DEFAULT, so existing runs are unchanged, and CI is untouched because it takes the environment route rather than a profile. has_profile exists because configparser treats DEFAULT as the fallback for every other section rather than as a section, so has_section denies it exists even when it is the only thing in the file — which made the first version of this fallback do nothing for exactly the case it was written for. Three documentation corrections found while checking the guide against the code. The IAM policy is ten operations and needs "use vnics", not "read vnics": registering a secondary private IP is an operation on a VNIC, so OCI checks VNIC_ASSIGN, which lives in the use verb, and with read the node forms and looks healthy and every allocation fails NotAuthorizedOrNotFound. These are now the same three statements instance-principal.tf grants, which is the only version of the policy that has been exercised. The guide also promised a worked example of the credential hook that does not exist and should not: nothing here distributes a credential. It now states the contract instead — the argument shape, the two files, and that a non-zero exit stops the deploy before anything is built. And the dev channel's rate limit is documented with the tag as its remedy, rather than left in the driver's help where a reader will not see it. * fix(ci): a failing workbook's log never reached the artifact, so only the runner had it run-tofu-examples-e2e.sh writes its assertion output to the per-topology workbooks.log in the state directory. "Sweep anything left behind" deletes those directories and runs before "Upload the run", so the upload's three oci-state/.validate-* patterns matched nothing and said so to nobody. The first nine-workbook run lost the one thing it was dispatched to find: ecs-quickstart failed on vm-single and the reason existed only on the runner, which does not resolve from a workstation. A collect step now copies *.log, *.txt and *.tsv into the artifact directory before the sweep, prefixed by topology so two topologies cannot overwrite each other. The workflow already records that "Publish the result tables" sits ahead of the sweep for this reason; the logs were simply never given the same treatment. The dead patterns are removed rather than left. A path that cannot match is worse than no path, because it reads as protection that is present — which applies to the credential-hook exclusion too: it now lives in the collect step, which is the only thing that writes into the artifact tree, so the file cannot reach an artifact rather than being filtered out of one. * fix(ci): a state directory kept the last run's logs, and the artifact published them as this run's Collecting the per-topology logs surfaced the next defect immediately: a vm-single-only run produced an artifact holding bm-workbooks.log and nine vm-multi node logs, because those state directories still held the previous run's output. Anyone reading that artifact would conclude three topologies ran when one did, which is worse than the missing log it replaced. Two causes, one each side. validate-topology.sh never cleared its own logs, so a topology's directory accumulated them; it now removes *.log, nodes.txt and cloudinit-*.txt as it starts, before the first line of the new run, and never the tfstate, which is how the sweep finds hosts to destroy. The globs are named rather than a find, so what goes is readable at a glance, and :? stops the shell instead of expanding to /*.log if STATE_DIR is ever unset. That alone does not cover a topology the run skipped, whose directory is never entered. So the collect step now takes only the topologies this run drove, read back from the per-topology log the harness writes rather than from the dispatch input -- a run that names no topology gets the harness's default, and reading that here would be a second copy of the list to disagree with the first. Also proved by this: ecs-quickstart passes on a freshly built vm-single, 172s, where it failed at 293s as the seventh of nine. The failure is order or load dependent on one 32 GB node, not a property of the workbook. * fix(ci): a failed workbook run left no node journal, so the cause was unreadable The workbook log says which API call failed; only the journal says why, and the nodes are destroyed minutes later. An IAM teardown failure on the nightly came down to one swallowed warning that existed nowhere else, and by the time the artifact was read the host it was logged on had been torn down. Warnings and above first, since that is where a swallowed error surfaces, then a bounded tail for context. Unbounded, a multi-hour suite's journal would dwarf the rest of the artifact. The existing state-directory clear keeps these from going stale between runs, and the collect step already globs *.log. |
||
|
|
c6841ce1f0 |
feat(oci): separate using an instance principal from creating one (#1157)
* feat(oci): separate using an instance principal from creating one The allocator needs OCI credentials at runtime, and a node with none forms, passes every health check, then refuses every launch wanting a public address. Instance principal is the credential route that puts no key material on a node, but it needs a dynamic group and a policy, and those are tenancy-root resources the compartment-scoped user the rest of the configuration runs as cannot create. enable_instance_principal conflated the two, so the better credential route was only reachable by a tenancy admin. instance_principal now takes off, adopt or create: adopt selects the credential and creates nothing, because the matching rule covers the whole compartment and so names no instance to reference. setup-identity.sh owns the create case, with its own state. validate-topology.sh destroys its state at the end of every run, so holding tenancy resources there would let a teardown delete the tenancy's policy. Also package install-node.sh and smoke-test.sh, so a cluster can be formed from a curl | bash install with no checkout. smoke-test.sh is not optional: formation scp's it from its own directory for the post-formation verification. * feat(oci): take the credential handoff from an operator hook, and widen the workflow name An API-key deployment needs a credential on each node, and nothing in this repository can supply one: a key in user-data is readable from instance metadata for the life of the instance, and this host runs other people's guests. A manual install does not survive either, because a topology destroys itself on success. So the harness calls an operator-owned executable after formation and before the pool, the only window where a node has /etc/spinifex and has not started the allocator. Instance principal needs no handoff, so the hook is skipped there. The workflow is renamed for the vendors it will cover rather than the first one. Nothing is orphaned in the Actions UI: it has never been on the default branch and has never run, so there is no history attached to the old filename. * docs(oci): the API-key path needs the config file, not just the PEM oci_config_file names an ordinary OCI SDK config and nothing creates it, so a node holding only the key forms and then fails every allocation with no stated cause. * fix(ci): the runner context is not available in a job-level env block A workflow referencing it there is rejected before any job starts, which is why the rename produced a failed run attributed to the push. The reference had never been validated: a dispatch-only workflow that is never dispatched is never parsed. * fix(deps): bump containerd to v2.4.1 for GO-2026-6597 The image-pull DoS via crafted OCI index graph amplification is reachable from the ECS agent's containerd puller, so the vulnerability scan fails preflight on v2.4.0. A patch release with no API change. * fix(e2e): probe guest public addresses from outside the cloud's own network A VCN does not reflect its own public addresses back into itself, so a node asking for one gets nothing however well the datapath works. The workbook driver runs on the first node, so every assertion that curls or SSHes a guest's public address failed on OCI against a cluster that was serving those addresses correctly to the internet. The driver now takes E2E_PUBLIC_PROXY, a SOCKS5 proxy used for public addresses only, and validate-topology.sh supplies one with a remote dynamic forward on the connection that launches the driver. Egress is then the host running the validation, which is outside the VCN and is where a customer sits. Unset leaves every other platform on a direct probe, which is already correct there. Also adds --version TAG for a repeatable install, separates the workflow's auth choice from its external-pool choice so an API key can run with the allocator live, and captures allocator diagnostics before the teardown destroys them. * ci: read the Go version from go.mod in the release workflows Both pinned 1.27.0 by hand while every other workflow reads the file, so the toolchain they build with drifts from the one the module declares the day go.mod moves. release.yml was already inconsistent with itself: its second setup-go step reads spinifex/go.mod. * feat: publish the OCI topology results as tables on the run page A green tick on one job with 4,000 lines of log behind it is confidence for whoever wrote the harness and nothing to anybody else. Someone assessing whether Spinifex works on their cloud should be able to see that it formed and served guests on a single VM, on three VMs and on bare metal without opening a log. record now takes <gate> <status> [detail] and writes results.tsv beside the human block, so a gate's status is a field the script sets rather than a regex over prose that reads fine and matches wrong. record_workbooks turns the driver's own go-test-shaped lines into workbooks.tsv, and runs on the failing branch too: which four passed is most of what the fifth failing means. e2e-oci.sh --summary-only renders both tables to stdout and to GITHUB_STEP_SUMMARY, and the workflow calls it under always() before the sweep, because the sweep destroys the state directories the tables are read from. A skipped gate gets its own mark, and a topology that never ran says so: the one way a results table can do real harm is a reader taking an absent row for a passing one. * docs(oci): cut the deploy-by-hand appendix, and validate the config for real The appendix was 444 of 1014 lines restating an install the Terraform already does, which made the document read as though deploying by hand were a normal route. It is not: the configuration builds the VCN, the volumes and both VNICs per node and renders the cloud-init, in the same order every time. What is genuinely OCI-specific is four things, each of which already has a section, so the replacement names those and sends everyone else to the single-node and multi-node install guides. Verify end to end told the reader to ping a guest's public address from the host, which cannot work: a VCN does not route its own public addresses back into itself. That is the finding this branch is built on and the document was asserting the opposite. It now says where to probe from and why, and carries the allocator's `resolved the external VNIC` check, which existed only in the cut appendix and is the one line proving the credentials and compartment are right. `spx config validate` is not a command -- spx has admin, get, service, top and version -- so that line could only ever fail and its output and status were both discarded. It is replaced by parsing the TOML, which is what was wanted: the remap and the pool append are textual edits, and a broken result would otherwise surface as every service failing to start with nothing saying why. The remap now checks its own substitution count for the same reason, since re.sub with no match rewrites the file unchanged and exits 0. Publication of the run-page tables moves to one owner. GITHUB_STEP_SUMMARY is per step, so appending from the run as well as the publish step would have put two copies of every table on the run page. * fix(oci): check the credential hook before building anything The hook is a path from a CI variable, and whether it exists on the runner is knowable before Terraform runs. Checked where it was used, a wrong path cost the apply, the install and the formation first -- forty minutes and a bare-metal bill to be told a file is missing. Nothing about the check depends on the apply, so it moves to the top, where --dry-run reaches it too. * fix(oci): the summary must say what was built and what was not tested The first CI run headed its tables "Published release `latest channel`" over a build of a feature branch, which is the one error in a results table that would mislead a reader who trusted it. The header now states a tree build as a tree build, and names how the allocator authenticated. A run with no external pool says so in the header rather than leaving it to a SKIPPED row further down. Public-address allocation is the claim a green table is most likely to be read as making, and it is exactly the claim such a run cannot support. Both flags are passed to the publish step too, so the summary rendered after a failure describes the same run as the one that failed. Raise the dev prerelease prune from five to twenty. A pinned dev tag is how an external evaluator installs a specific build, and five is about a week of them -- short enough that the tag someone was handed stops resolving while they are still using it. * docs(oci): the workbook table is a snapshot, and the nightly is the current answer It is dated against a build five releases of work ago, and the nightly now runs the same workbooks on all three topologies and publishes its own table. Say which one to read rather than letting a stale table be taken as current. * ci: rename to e2e-cloudvendor.yml so the sidebar entry is registered from a valid file A workflow's displayed name is captured when GitHub first registers it and is not refreshed afterwards. Both OCI entries were registered from a run that could not be parsed -- a job-level env block referencing runner.temp -- so both list as their path, which reads as a missing name: key and is not one. The 2026-10-02 run was green on all three topologies and the entry still read as a path, with updated_at unchanged from the broken first run, so fixing the file is necessary and not sufficient. A new path is a new registration, and its first run reads a valid file. Dropping -nightly is also more honest: there is no schedule: here, because GitHub registers one only from the default branch, so a cron could never run dev and a feature branch as separate nightlies. * Revert "ci: rename to e2e-cloudvendor.yml so the sidebar entry is registered from a valid file" The rename makes the workflow undispatchable. `gh workflow run` resolves a filename against the default branch, and the raw dispatch endpoint returns 404 for a path GitHub has not registered -- checked twice, several minutes apart, so it is not an indexing delay. The old name works only because it is already registered, and an unrunnable workflow is a worse problem than a mislabelled one. The sidebar label needs a fresh registration, which a rename cannot provide from a feature branch. It corrects itself when the file reaches main, where GitHub re-reads it; the alternative is deleting the dead registrations' runs, which is an operator's call. * fix(ci): never publish the credential hook's output as an artifact The hook is the operator's own executable, and the one thing known about it is that it handles a private key. Its stdout and stderr are captured to credential-hook.log, which the artifact's *.log glob was collecting into a downloadable archive with a fourteen-day retention. Excluded rather than trusted to have been quiet. A hook that prints its key once while someone is debugging it should not thereby publish it. * fix(oci): the hairpin probe measured nothing on one node, and claimed too much With a single host, HOSTS[0] and HOSTS[-1] are the same address, so the node connected to itself -- which always succeeds. It reported "present" on vm-single and bm for that reason and not as a measurement. It now needs two nodes and says so otherwise. The claim it carried was also wrong. "A VCN does not reflect its own public addresses back into itself" was read off a workstation run whose node_client_cidr_allow_list was narrowed to one office address, which makes a node's own public address a non-permitted source: the refusal that follows is identical to an absent hairpin. With the committed 0.0.0.0/0 default the probe reports the hairpin present between two nodes, so the original reading was confounded and is retracted. None of this changes the remedy. A customer reaches a guest from outside the VCN, which is where the proxy's egress is, and probing from there is correct whichever of the two causes is in play -- 15 of 15 workbooks pass through it on all three topologies. What changes is what the documentation may assert: the guide now names both causes as indistinguishable rather than stating one as fact. * docs(oci): a reader could not run the first command, because nothing told them to clone The guide's first code block was `cd scripts/terraform/oci-spx`, into a directory no step had obtained. It also never mentioned the SSH keypair the driver refuses to start without, nor that the configuration is not on main or in v1.21.0 yet, so all three had to be inferred from the deploy failing. Restructured so the deploy comes first and reads as four steps and one command: get the configuration, choose one node or three, write the inputs, run it. The credential decision stays before step 4, because it changes an argument there and a node without one forms, looks healthy, and then refuses every launch that wants a public address. Everything else -- why OCI, sizing, the architecture, the IAM policy, quotas, the oci CLI, what Terraform builds, every variable -- moves under Reference, unchanged in substance. Adds what a successful run prints, naming `allocator ready` as the line to look for: every line above it appears on a node that cannot allocate, and that is the one failure the run does not announce. Also drops an empty verdict.txt committed from a local run. The directory the guide now tells customers to work in should not ship one run's artifacts. |