Ben Duncan
adb3d9876d
The AWS provider reads a bucket's tags with `ListTagsForResource` on S3 Control when it knows the account ID. S3 Control puts the account in the hostname — `https://{account}.{endpoint}` — so against an IP-addressed endpoint the request fails a DNS lookup before a packet leaves, and no endpoint configuration can help. A tagged bucket therefore fails its first read after being created. `skip_requesting_account_id` alone does not fix it: `aws-sdk-go-base` takes the account ID from the STS `GetCallerIdentity` credentials check before it consults that flag, so `skip_credentials_validation` is needed alongside it. With no account ID the provider falls back to `GetBucketTagging`, which predastore serves. `s3-webapp` is the only workbook that creates a bucket, and it uses no `aws_caller_identity` and references no other security group, so a blank account ID costs it nothing. The bucket also gets `force_destroy`, since the webapp uploads into it and a destroy of a used bucket fails `BucketNotEmpty` without it. The e2e harness now asserts the bucket carries the tags Terraform set, which is what catches this regressing, and no longer aborts the remaining workbooks on a failure — `s3-webapp` failing says nothing about whether RDS works, and four nights of `rds-quickstart` coverage were lost that way. `STOP_ON_FAIL=1` restores the early abort. Every workbook's provider versions are pinned, so a run reproduces rather than picking up whatever the registry published that morning.
title, seoTitle, description, category, tags, resources
| title | seoTitle | description | category | tags | resources | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Moving an AWS Workload to Mulga | Migrate an AWS Workload to Spinifex — Spinifex Docs | Move existing AWS workloads onto Spinifex using compatible APIs, SDKs, and Terraform across EC2, VPC, EBS, S3, IAM, STS, ELBv2, ACM, ECR, ECS, and EKS. | Migration |
|
|
Moving an AWS Workload to Mulga
Migrate existing AWS workloads to Spinifex using compatible APIs, SDKs, and Terraform.
Table of Contents
Overview
Spinifex provides drop-in compatibility with AWS APIs, making it possible to migrate existing workloads with minimal changes.
Supported Services: EC2, VPC, EBS, S3, IAM, STS, ELBv2 (ALB/NLB), ACM, ECR, ECS, EKS
Compatible Tools: AWS CLI, AWS SDKs, Terraform, kubectl (via aws eks get-token), any S3-compatible client
Instructions
Prerequisites
- A running Spinifex cluster (see Setting Up Your Cluster)
- AWS CLI configured with the
spinifexprofile:
export AWS_PROFILE=spinifex
Configure AWS CLI
aws ec2 describe-instances
aws s3 ls
Terraform
provider "aws" {
region = "ap-southeast-2"
access_key = "your-spinifex-access-key"
secret_key = "your-spinifex-secret-key"
endpoints {
ec2 = "https://localhost:9999"
iam = "https://localhost:9999"
sts = "https://localhost:9999"
elbv2 = "https://localhost:9999"
acm = "https://localhost:9999"
ecr = "https://localhost:9999"
ecs = "https://localhost:9999"
eks = "https://localhost:9999"
s3 = "https://localhost:8443"
}
skip_metadata_api_check = true
skip_region_validation = true
}
Hybrid S3 Sync
aws s3 sync s3://local-bucket/ s3://cloud-bucket/ --source-region spinifex --region us-east-1
Troubleshooting
Terraform Provider Errors
Ensure these two skip flags are set in your provider configuration:
skip_metadata_api_check = true
skip_region_validation = true
Without them, Terraform queries the EC2 instance metadata service for credentials and rejects a region name AWS does not know.
Leave skip_credentials_validation and skip_requesting_account_id unset, and point the sts endpoint at Spinifex. The provider then learns the account ID from GetCallerIdentity as it does on AWS. With the account ID blank, a security group rule that references a group in the same account reads back as <account>/sg-... and never reaches a clean plan.
Tagged S3 buckets need both flags set
There is one exception, and it applies to a provider block that manages an aws_s3_bucket with tags. AWS provider 6.x reads bucket tags through S3 Control, whose endpoint puts the account ID in the hostname — so the provider dials https://000000000001.<your-endpoint>, which cannot resolve when your endpoint is an IP address:
Error: listing tags for S3 (Simple Storage) Bucket (uploads): operation error S3 Control:
ListTagsForResource, ... dial tcp: lookup 000000000001.10.0.0.10: no such host
Set both flags in that provider block. With no account ID the provider reads tags through S3 GetBucketTagging instead, which Spinifex serves:
skip_requesting_account_id = true
skip_credentials_validation = true
Both are needed — skip_requesting_account_id on its own has no effect, because the credentials check calls sts:GetCallerIdentity and supplies the account ID before the account lookup is ever skipped. If the same configuration also needs the account ID, for aws_ecr_repository or a security group rule that references another group, put the bucket in a second, aliased provider block rather than setting the flags globally.
S3 Signature Errors
Spinifex uses AWS Signature V4. Ensure your AWS CLI is version 2.0 or higher:
aws --version
If using an older version, upgrade:
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install --update