gsadmin/spinifex
1
0
Fork 0
mirror of https://github.com/mulgadc/spinifex.git synced 2026-10-15 17:11:30 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat: Terraform S3 Webapp using IMDS (#348)

Browse Source
This commit is contained in:
Julian Sommer
2026-06-11 12:35:15 +10:00
committed by GitHub
parent 05e66bd6a0
commit 620b96f9c6
4 changed files with 138 additions and 25 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/terraform-workbooks/s3-webapp/README.md
+37 -7
View File
@@ -9,6 +9,9 @@ tags:
- flask
- webapp
- workbook
- imds
- sts
- instance-profile
resources:
- title: "Terraform AWS Provider"
url: "https://registry.terraform.io/providers/hashicorp/aws/latest"
@@ -36,18 +39,21 @@ resources:
Deploy an EC2 instance running a Flask file-sharing web application backed by S3 (Predastore). Users can upload files through a web form and browse uploaded content — demonstrating Terraform managing both compute and object storage together.
This workbook uses the idiomatic AWS credential model: the instance is launched with an **IAM instance profile** and the app pulls short-lived STS credentials from **IMDS** (`169.254.169.254`) through boto3's default credential chain. **No long-lived S3 keys are baked into the instance.** The Terraform run still uses the operator's admin credentials (to create the bucket, role, profile, and instance), but the running instance authenticates to S3 with credentials it fetches at runtime and which expire in ~1 hour.
**Architecture:**
<p align="center">
<img src="../../../.github/assets/diagrams/tf-s3-webapp.svg" alt="S3 webapp — browser to Flask EC2 instance to Predastore via S3 API" width="900">
<img src="../../../.github/assets/diagrams/tf-s3-webapp.svg" alt="S3 webapp — browser to Flask EC2 instance; the instance fetches short-lived STS credentials from IMDS (169.254.169.254) via its instance profile, then calls Predastore over the S3 API" width="900">
</p>
**What you'll learn:**
- Configuring the AWS provider with both Spinifex and Predastore endpoints
- Creating S3 buckets on Predastore via Terraform
- Deploying a Python webapp with cloud-init that talks to S3
- Passing credentials and configuration to instances via user-data
- Defining an IAM role, least-privilege managed policy, and instance profile, and passing the role to an instance (`iam:PassRole`)
- How the instance obtains short-lived credentials from IMDS via boto3's default credential chain — no static keys on the instance
- Deploying a Python webapp with cloud-init that signs S3 requests with the STS session token
**Prerequisites:**
@@ -55,6 +61,7 @@ Deploy an EC2 instance running a Flask file-sharing web application backed by S3
- Predastore running (S3 API on port 8443)
- An Ubuntu 26.04 AMI imported (see [Setting Up Your Cluster](/docs/setting-up-your-cluster))
- OpenTofu or Terraform installed
- The operator identity running the Terraform apply (`AWS_PROFILE=spinifex`) must be allowed to manage IAM and pass the role: `iam:CreateRole`, `iam:CreatePolicy`, `iam:AttachRolePolicy`, `iam:CreateInstanceProfile`, `iam:AddRoleToInstanceProfile`, and `iam:PassRole` on the new role. The bootstrap admin profile satisfies this.
- The EC2 instance must be able to reach Predastore — use the host's br-wan IP, not localhost
## Instructions
@@ -141,16 +148,39 @@ sudo systemctl status s3-webapp
sudo journalctl -u s3-webapp --no-pager -n 50
```
### Upload Fails or Files Not Appearing
### Upload Fails with `403 AccessDenied`
Check that the S3 credentials in `/opt/webapp/.env` are correct and that the bucket exists:
The instance reached S3 but was not authorized. The `.env` file deliberately holds **no** keys — credentials come from IMDS:
```bash
ssh -i s3-webapp-demo.pem ec2-user@<public_ip>
cat /opt/webapp/.env
/opt/webapp/venv/bin/python -c "import boto3; print('boto3 OK')"
cat /opt/webapp/.env # S3_ENDPOINT / S3_BUCKET / S3_REGION only — no keys
```
Check that:
- the managed policy is attached to the role (`s3:ListBucket` on the bucket, `s3:GetObject`/`s3:PutObject` on its objects), and the policy's bucket name matches `bucket_name`;
- the assumed-role session's account matches the bucket owner's account — Predastore enforces bucket ownership. Both are created by the same operator identity, so they align by construction; a mismatch surfaces as `403`.
### `NoCredentialsError` / No Credentials
boto3 could not obtain credentials, which means the IMDS credential chain did not resolve. From the instance:
```bash
ssh -i s3-webapp-demo.pem ec2-user@<public_ip>
# IMDSv2: fetch a token, then the role name behind the instance profile
TOKEN=$(curl -sX PUT http://169.254.169.254/latest/api/token \
-H 'X-aws-ec2-metadata-token-ttl-seconds: 60')
curl -s -H "X-aws-ec2-metadata-token: $TOKEN" \
http://169.254.169.254/latest/meta-data/iam/security-credentials/
# Confirm the resolved identity — expect assumed-role/s3-webapp-role/<instance-id>
aws sts get-caller-identity --endpoint-url https://<predastore_host>
```
If the role name is empty, the instance profile was not attached — confirm `iam_instance_profile` on the instance and that `iam:PassRole` is allowed for the operator.
### AMI Not Found
Ensure you have imported an Ubuntu 26.04 image:
docs/terraform-workbooks/s3-webapp/main.tf
+70 -8
View File
@@ -8,13 +8,25 @@
# Architecture:
#
# Browser ──HTTP──▶ EC2 Instance (Flask webapp, port 80)
# │
# ▼ S3 API (boto3)
# │ ▲ IMDS: short-lived STS creds (ASIA + token)
# │ └──── 169.254.169.254
# ▼ S3 API (boto3, SigV4 with session token)
# Predastore (port 8443)
#
# Credentials:
# The Terraform run uses the operator's admin keys (s3_access_key /
# s3_secret_key) to create the bucket, IAM role, instance profile, and
# instance. The INSTANCE receives no long-lived secret — boto3's default
# credential chain pulls short-lived STS credentials from IMDS
# (169.254.169.254), resolved via the attached instance profile -> role.
#
# Prerequisites:
# - Spinifex services running (gateway on port 9999)
# - Predastore running (S3 API on port 8443)
# - The operator identity (AWS_PROFILE=spinifex) must be allowed
# iam:CreateRole / iam:CreatePolicy / iam:AttachRolePolicy /
# iam:CreateInstanceProfile / iam:AddRoleToInstanceProfile and
# iam:PassRole on the new role.
# - The EC2 instance must be able to reach the Predastore endpoint.
# Set `predastore_host` to the IP reachable from inside the VPC
# (e.g. the host's br-wan IP, NOT localhost).
@@ -80,13 +92,13 @@ variable "predastore_host" {
variable "s3_access_key" {
type = string
description = "S3 access key for Predastore"
description = "Operator S3 access key for the Terraform run (creates the bucket on Predastore); the instance uses IMDS, not this key"
}
variable "s3_secret_key" {
type = string
sensitive = true
description = "S3 secret key for Predastore"
description = "Operator S3 secret key for the Terraform run; the instance uses IMDS, not this key"
}
variable "bucket_name" {
@@ -154,6 +166,55 @@ resource "aws_s3_bucket" "uploads" {
bucket = var.bucket_name
}
# ---------------------------------------------------------------------------
# IAM Instance Role — least-privilege S3 access fetched at runtime via IMDS
# ---------------------------------------------------------------------------
resource "aws_iam_role" "webapp" {
name = "s3-webapp-role"
assume_role_policy = jsonencode({
Version = "2012-10-17"
Statement = [{
Effect = "Allow"
Principal = { Service = "ec2.amazonaws.com" }
Action = "sts:AssumeRole"
}]
})
}
resource "aws_iam_policy" "webapp_s3" {
name = "s3-webapp-policy"
policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Sid = "ListBucket"
Effect = "Allow"
Action = ["s3:ListBucket"]
Resource = ["arn:aws:s3:::${var.bucket_name}"]
},
{
Sid = "ObjectRW"
Effect = "Allow"
Action = ["s3:GetObject", "s3:PutObject"]
Resource = ["arn:aws:s3:::${var.bucket_name}/*"]
}
]
})
}
resource "aws_iam_role_policy_attachment" "webapp" {
role = aws_iam_role.webapp.name
policy_arn = aws_iam_policy.webapp_s3.arn
}
resource "aws_iam_instance_profile" "webapp" {
name = "s3-webapp-profile"
role = aws_iam_role.webapp.name
}
# ---------------------------------------------------------------------------
# SSH Key Pair
# ---------------------------------------------------------------------------
@@ -273,9 +334,14 @@ resource "aws_instance" "webapp" {
subnet_id = aws_subnet.public.id
vpc_security_group_ids = [aws_security_group.webapp.id]
key_name = aws_key_pair.webapp.key_name
iam_instance_profile = aws_iam_instance_profile.webapp.name
associate_public_ip_address = true
# The role's permissions must exist before the instance boots and makes its
# first IMDS-credentialed S3 call
depends_on = [aws_iam_role_policy_attachment.webapp]
user_data_base64 = base64encode(<<-USERDATA
#!/bin/bash
set -euo pipefail
@@ -293,8 +359,6 @@ resource "aws_instance" "webapp" {
cat > /opt/webapp/.env <<'ENVFILE'
S3_ENDPOINT=https://${var.predastore_host}
S3_BUCKET=${var.bucket_name}
S3_ACCESS_KEY=${var.s3_access_key}
S3_SECRET_KEY=${var.s3_secret_key}
S3_REGION=${var.region}
ENVFILE
@@ -321,8 +385,6 @@ resource "aws_instance" "webapp" {
s3 = boto3.client(
"s3",
endpoint_url=env["S3_ENDPOINT"],
aws_access_key_id=env["S3_ACCESS_KEY"],
aws_secret_access_key=env["S3_SECRET_KEY"],
region_name=env["S3_REGION"],
verify=False,
config=Config(s3={"addressing_style": "path"}),
docs/terraform-workbooks/s3-webapp/terraform.tfvars.example
+5
View File
@@ -2,6 +2,11 @@
#
# The predastore_host must be reachable from INSIDE the VPC guest — not
# localhost. Use the host's br-wan or LAN IP, e.g. "192.168.1.10:8443".
#
# s3_access_key / s3_secret_key are the operator credentials for the Terraform
# run only — they create the bucket on Predastore and the IAM role/profile.
# They are NOT passed to the instance; the instance pulls short-lived STS
# credentials from IMDS via its instance profile.
predastore_host = "192.168.1.10:8443"
s3_access_key = "AKIAIOSFODNN7EXAMPLE"
tests/e2e/run-tofu-examples-e2e.sh
+26 -10
View File
@@ -232,15 +232,30 @@ assert_nginx_webserver() {
}
assert_s3_webapp() {
# aws s3 goes to predastore on :8443, not the awsgw on :9999 the default
# profile points at — the workbook's provider sets s3 = predastore_endpoint
# but the CLI doesn't inherit that.
local bucket sentinel endpoint
bucket=$(tofu output -raw bucket_name)
endpoint="https://${WAN_IP}:8443"
# Prove the INSTANCE wrote to S3 with IMDS-sourced STS creds: upload a file
# through the app (PutObject), then read it back from the listing (ListBucket
# + GetObject link). This exercises the full IMDS -> STS -> IAM ->
# predastore-authz path and fails closed if any link regresses — a
# harness-side CLI upload would still pass with IMDS broken.
local ip sentinel tmp
ip=$(tofu output -raw public_ip)
sentinel="spinifex-nightly-$(date +%s).txt"
echo "nightly-smoke" | aws s3 cp --endpoint-url "$endpoint" - "s3://${bucket}/${sentinel}" >/dev/null
aws s3 ls --endpoint-url "$endpoint" "s3://${bucket}/" | grep -q "$sentinel"
wait_for_http_200 "http://${ip}/" || {
log " s3-webapp: webapp not reachable on http://${ip}/"
return 1
}
tmp=$(mktemp)
echo "nightly-smoke" > "$tmp"
if ! curl -sf -F "file=@${tmp};filename=${sentinel}" "http://${ip}/upload" >/dev/null; then
rm -f "$tmp"
log " s3-webapp: upload via app failed (IMDS -> STS -> S3 PutObject?)"
return 1
fi
rm -f "$tmp"
curl -sf "http://${ip}/" | grep -q "$sentinel"
}
# Pick an instance type available on this cluster. Workbooks default to
@@ -284,8 +299,9 @@ run_workbook() {
"-var=instance_type=${INSTANCE_TYPE}"
)
# s3-webapp has three required-no-default vars; creds come from
# AWS_PROFILE=spinifex so the workbook's boto3 client authenticates.
# s3-webapp has three required-no-default vars. The s3_access/secret keys are
# the operator creds the provider uses to create the bucket + IAM role on
# predastore; the instance authenticates to S3 via IMDS, not these keys.
if [ "$example" = "s3-webapp" ]; then
local access_key secret_key
access_key=$(aws configure get aws_access_key_id --profile spinifex)