gsadmin/spinifex
1
0
Fork 0
mirror of https://github.com/mulgadc/spinifex.git synced 2026-10-16 01:22:00 +00:00
Code Issues Packages Projects Releases Wiki Activity

TF updates

Browse Source
This commit is contained in:
Ben Duncan
2026-03-31 13:19:17 +10:00
parent c9e83380df
commit b73a630f41
6 changed files with 486 additions and 58 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/terraform/bastion-private-subnet/README.md
+159
View File
@@ -0,0 +1,159 @@
---
title: "Terraform: Bastion with Private Subnet"
description: "Deploy a bastion host to access an isolated private compute instance with no internet connectivity."
category: "Terraform Workbooks"
badge: "Workbook"
tags:
- terraform
- bastion
- vpc
- security
- private subnet
- workbook
resources:
- title: "Terraform AWS Provider"
url: "https://registry.terraform.io/providers/hashicorp/aws/latest"
- title: "Spinifex Repository"
url: "https://github.com/mulgadc/spinifex"
- title: "OpenTofu"
url: "https://opentofu.org/"
---
# Terraform: Bastion with Private Subnet
> Deploy a bastion host to access an isolated private compute instance with no internet connectivity.
## Table of Contents
- [Overview](#overview)
- [Instructions](#instructions)
- [Troubleshooting](#troubleshooting)
---
## Overview
Deploy a VPC with public and private subnets where the private subnet has no internet access. A bastion host in the public subnet is the only way to reach instances in the private subnet. This pattern is used for sensitive workloads that must remain isolated from the internet — the private instance cannot make or receive any connections outside the VPC.
**Architecture:**
```
WAN ──SSH──▶ Bastion (public subnet)
│
▼ SSH (private IP)
App Server (private subnet, no internet)
```
**What you'll learn:**
- Creating public and private subnets in a VPC
- Isolating compute instances from the internet using route tables and security groups
- Using a bastion host as the sole access point to private resources
- SSH hopping through a bastion to reach private instances
**Use cases:**
- Sensitive data processing that must not have internet egress
- Internal services that should only be reachable within the VPC
- Compliance workloads requiring network isolation
**Prerequisites:**
- Spinifex installed and running (see [Installing Spinifex](/docs/installing-spinifex))
- A Debian 12 AMI imported (see [Setting Up Your Cluster](/docs/setting-up-your-cluster))
- OpenTofu or Terraform installed
- `AWS_PROFILE=spinifex` configured
## Instructions
### Step 1. Get the Template
Clone the Terraform examples from the Spinifex repository:
```bash
git clone --depth 1 --filter=blob:none --sparse https://github.com/mulgadc/spinifex.git spinifex-tf
cd spinifex-tf
git sparse-checkout set docs/terraform
cd docs/terraform/bastion-private-subnet
```
Or create a `main.tf` file and paste the full configuration below.
<!-- INCLUDE: main.tf lang:hcl -->
### Step 2. Deploy
```bash
export AWS_PROFILE=spinifex
tofu init
tofu apply
```
### Step 3. Connect
> **Note:** EC2 instances can take 30+ seconds to boot after apply. If SSH is unreachable, wait and retry.
SSH into the bastion:
```bash
ssh -i bastion-demo.pem ec2-user@<bastion_public_ip>
```
From the bastion, SSH to the private instance. The key is pre-installed at `~/.ssh/bastion-demo.pem` via cloud-init:
```bash
ssh -i ~/.ssh/bastion-demo.pem ec2-user@<private_ip>
```
### Step 4. Verify Isolation
From the private instance, confirm there is no internet connectivity:
```bash
# This should time out — the private instance has no route to the internet
curl --connect-timeout 5 https://deb.debian.org || echo "No internet access (expected)"
```
The private instance can only communicate within the VPC (`10.20.0.0/16`). Its security group restricts egress to VPC-internal traffic only.
### Clean Up
```bash
tofu destroy
```
## Troubleshooting
### Cannot SSH to Private Instance
The private instance has no public IP — it is only reachable from the bastion. SSH to the bastion first, then use the pre-installed key to hop:
```bash
ssh -i bastion-demo.pem ec2-user@<bastion_ip>
ssh -i ~/.ssh/bastion-demo.pem ec2-user@<private_ip>
```
If the key is missing on the bastion, check that cloud-init completed successfully:
```bash
sudo journalctl -u cloud-init --no-pager
ls -la ~/.ssh/bastion-demo.pem
```
### Private Instance Can Reach the Internet
If the private instance unexpectedly has internet access, check that its route table has no default route:
```bash
aws ec2 describe-route-tables --profile spinifex
```
The private route table should have no `0.0.0.0/0` route. Also verify the private security group egress is restricted to the VPC CIDR (`10.20.0.0/16`).
### AMI Not Found
Ensure you have imported a Debian 12 image:
```bash
aws ec2 describe-images --owners 000000000000 --profile spinifex
```
docs/terraform/bastion-private-subnet/main.tf
+26 -58
View File
@@ -1,33 +1,28 @@
# Example 2: Bastion Host with Private Subnet
#
# Deploys a VPC with both public and private subnets. A bastion host in the
# public subnet provides SSH access to instances in the private subnet.
# A NAT gateway gives private instances outbound internet access.
# public subnet provides SSH access to an isolated instance in the private
# subnet. The private instance has no internet connectivity — ideal for
# sensitive workloads that must remain air-gapped from the internet.
#
# Architecture:
#
# WAN ──SSH──▶ Bastion (public subnet)
# │
# ▼ SSH (private IP)
# App Server (private subnet)
# │
# ▼ outbound via NAT GW
# Internet (apt-get, etc.)
# App Server (private subnet, no internet)
#
# Usage:
# cd spinifex/scripts/iac/aws/examples/02-bastion-private-subnet
# export AWS_PROFILE=spinifex
# tofu init && tofu apply
#
# After apply:
# # SSH to bastion
# ssh -i bastion-demo.pem ec2-user@<bastion_public_ip>
# # SSH to the bastion
# ssh -i bastion-demo.pem ec2-user@<bastion_ip>
#
# # From bastion, SSH to private instance
# ssh -i /tmp/key.pem ec2-user@<private_ip>
#
# # Or use SSH proxy jump (single command from your workstation)
# ssh -i bastion-demo.pem -J ec2-user@<bastion_ip> ec2-user@<private_ip>
# # From the bastion, SSH to the private instance
# # (the key is pre-installed at ~/.ssh/bastion-demo.pem via cloud-init)
# ssh -i ~/.ssh/bastion-demo.pem ec2-user@<private_ip>
terraform {
required_version = ">= 1.6.0"
@@ -144,7 +139,7 @@ resource "aws_vpc" "main" {
}
# ---------------------------------------------------------------------------
# Internet Gateway
# Internet Gateway — only the public subnet routes through this
# ---------------------------------------------------------------------------
resource "aws_internet_gateway" "igw" {
@@ -189,7 +184,7 @@ resource "aws_route_table_association" "public" {
}
# ---------------------------------------------------------------------------
# Private Subnet — application instances live here (no public IPs)
# Private Subnet — isolated instances live here (no public IPs, no internet)
# ---------------------------------------------------------------------------
resource "aws_subnet" "private" {
@@ -203,37 +198,10 @@ resource "aws_subnet" "private" {
}
}
# ---------------------------------------------------------------------------
# NAT Gateway — gives private subnet outbound internet access
# ---------------------------------------------------------------------------
resource "aws_eip" "nat" {
domain = "vpc"
tags = {
Name = "bastion-demo-nat-eip"
}
}
resource "aws_nat_gateway" "nat" {
allocation_id = aws_eip.nat.id
subnet_id = aws_subnet.public.id
tags = {
Name = "bastion-demo-nat-gw"
}
depends_on = [aws_internet_gateway.igw]
}
# Private route table — no default route, no internet access
resource "aws_route_table" "private" {
vpc_id = aws_vpc.main.id
route {
cidr_block = "0.0.0.0/0"
nat_gateway_id = aws_nat_gateway.nat.id
}
tags = {
Name = "bastion-demo-private-rt"
}
@@ -290,11 +258,11 @@ resource "aws_security_group" "private" {
}
egress {
description = "All outbound"
description = "VPC internal only"
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
cidr_blocks = ["10.20.0.0/16"]
}
tags = {
@@ -321,10 +289,10 @@ resource "aws_instance" "bastion" {
#!/bin/bash
set -euo pipefail
mkdir -p /home/ec2-user/.ssh
cat > /home/ec2-user/.ssh/private-key.pem <<'KEY'
cat > /home/ec2-user/.ssh/bastion-demo.pem <<'KEY'
${tls_private_key.bastion.private_key_openssh}
KEY
chmod 600 /home/ec2-user/.ssh/private-key.pem
chmod 600 /home/ec2-user/.ssh/bastion-demo.pem
chown -R ec2-user:ec2-user /home/ec2-user/.ssh
USERDATA
)
@@ -335,7 +303,7 @@ resource "aws_instance" "bastion" {
}
# ---------------------------------------------------------------------------
# Private Instance (private subnet — no public IP)
# Private Instance (private subnet — no public IP, no internet)
# ---------------------------------------------------------------------------
resource "aws_instance" "private" {
@@ -355,6 +323,10 @@ resource "aws_instance" "private" {
# Outputs
# ---------------------------------------------------------------------------
output "note" {
value = "EC2 instances can take 30+ seconds to boot after apply. If SSH is unreachable, wait and retry."
}
output "bastion_public_ip" {
value = aws_instance.bastion.public_ip
}
@@ -364,15 +336,11 @@ output "private_instance_ip" {
}
output "ssh_to_bastion" {
value = "ssh -i bastion-demo.pem ec2-user@${aws_instance.bastion.public_ip}"
description = "SSH to the bastion host"
value = "ssh -i bastion-demo.pem ec2-user@${aws_instance.bastion.public_ip}"
}
output "ssh_to_private_via_bastion" {
description = "From your workstation, jump through the bastion to the private instance"
value = "ssh -i bastion-demo.pem -J ec2-user@${aws_instance.bastion.public_ip} ec2-user@${aws_instance.private.private_ip}"
}
output "ssh_from_bastion_to_private" {
description = "Once on the bastion, SSH to the private instance"
value = "ssh -i ~/.ssh/private-key.pem ec2-user@${aws_instance.private.private_ip}"
output "ssh_to_private_from_bastion" {
description = "From the bastion, SSH to the private instance (key is pre-installed via cloud-init)"
value = "ssh -i ~/.ssh/bastion-demo.pem ec2-user@${aws_instance.private.private_ip}"
}
docs/terraform/nginx-webserver/README.md
+130
View File
@@ -0,0 +1,130 @@
---
title: "Terraform: Nginx Web Server"
description: "Deploy a VPC with a public subnet and an EC2 instance running Nginx using Terraform on Spinifex."
category: "Terraform Workbooks"
badge: "Workbook"
tags:
- terraform
- nginx
- ec2
- vpc
- workbook
resources:
- title: "Terraform AWS Provider"
url: "https://registry.terraform.io/providers/hashicorp/aws/latest"
- title: "Spinifex Repository"
url: "https://github.com/mulgadc/spinifex"
- title: "OpenTofu"
url: "https://opentofu.org/"
---
# Terraform: Nginx Web Server
> Deploy a VPC with a public subnet and an EC2 instance running Nginx using Terraform on Spinifex.
## Table of Contents
- [Overview](#overview)
- [Instructions](#instructions)
- [Troubleshooting](#troubleshooting)
---
## Overview
Deploy a complete Nginx web server on Spinifex using Terraform/OpenTofu. This workbook provisions a VPC, public subnet, internet gateway, route table, security group, SSH key pair, and an EC2 instance with cloud-init user-data that installs and starts Nginx.
**What you'll learn:**
- Configuring the AWS Terraform provider to target Spinifex
- Creating a VPC with public internet access
- Provisioning an EC2 instance with cloud-init user-data
- Generating SSH key pairs with the TLS provider
**Prerequisites:**
- Spinifex installed and running (see [Installing Spinifex](/docs/installing-spinifex))
- A Debian 12 AMI imported (see [Setting Up Your Cluster](/docs/setting-up-your-cluster))
- OpenTofu or Terraform installed
- `AWS_PROFILE=spinifex` configured
## Instructions
### Step 1. Get the Template
Clone the Terraform examples from the Spinifex repository:
```bash
git clone --depth 1 --filter=blob:none --sparse https://github.com/mulgadc/spinifex.git spinifex-tf
cd spinifex-tf
git sparse-checkout set docs/terraform
cd docs/terraform/nginx-webserver
```
Or create a `main.tf` file and paste the full configuration below.
<!-- INCLUDE: main.tf lang:hcl -->
### Step 2. Deploy
```bash
export AWS_PROFILE=spinifex
tofu init
tofu apply
```
### Step 3. Verify
> **Note:** EC2 instances can take 30+ seconds to boot after apply. If SSH or HTTP is unreachable, wait and retry.
After apply completes, use the outputs to test:
```bash
curl http://<public_ip>
ssh -i nginx-demo.pem ec2-user@<public_ip>
```
Open the `web_url` output in your browser to see the Nginx welcome page.
### Clean Up
```bash
tofu destroy
```
## Troubleshooting
### AMI Not Found
Ensure you have imported a Debian 12 image. Check available AMIs:
```bash
aws ec2 describe-images --owners 000000000000 --profile spinifex
```
### Provider Connection Refused
Verify Spinifex services are running:
```bash
sudo systemctl status spinifex.target
curl -k https://localhost:9999/
```
### SSH Connection Timeout
Check that the security group allows inbound SSH (port 22) and that the instance has a public IP assigned. Verify the instance is running:
```bash
aws ec2 describe-instances --profile spinifex
```
### Nginx Not Responding
SSH into the instance and check cloud-init logs:
```bash
ssh -i nginx-demo.pem ec2-user@<public_ip>
sudo journalctl -u cloud-init --no-pager
sudo systemctl status nginx
```
docs/terraform/nginx-webserver/main.tf
+4
View File
@@ -265,6 +265,10 @@ resource "aws_instance" "nginx" {
# Outputs
# ---------------------------------------------------------------------------
output "note" {
value = "EC2 instances can take 30+ seconds to boot after apply. If SSH or HTTP is unreachable, wait and retry."
}
output "instance_id" {
value = aws_instance.nginx.id
}
docs/terraform/s3-webapp/README.md
+163
View File
@@ -0,0 +1,163 @@
---
title: "Terraform: S3-Backed Web App"
description: "Deploy a Flask file-sharing web application backed by S3 (Predastore) using Terraform on Spinifex."
category: "Terraform Workbooks"
badge: "Workbook"
tags:
- terraform
- s3
- predastore
- flask
- webapp
- workbook
resources:
- title: "Terraform AWS Provider"
url: "https://registry.terraform.io/providers/hashicorp/aws/latest"
- title: "Spinifex Repository"
url: "https://github.com/mulgadc/spinifex"
- title: "Predastore (S3)"
url: "https://github.com/mulgadc/predastore"
- title: "OpenTofu"
url: "https://opentofu.org/"
---
# Terraform: S3-Backed Web App
> Deploy a Flask file-sharing web application backed by S3 (Predastore) using Terraform on Spinifex.
## Table of Contents
- [Overview](#overview)
- [Instructions](#instructions)
- [Troubleshooting](#troubleshooting)
---
## Overview
Deploy an EC2 instance running a Flask file-sharing web application backed by S3 (Predastore). Users can upload files through a web form and browse uploaded content — demonstrating Terraform managing both compute and object storage together.
**Architecture:**
```
Browser ──HTTP──▶ EC2 Instance (Flask webapp, port 80)
│
▼ S3 API (boto3)
Predastore (port 8443)
```
**What you'll learn:**
- Configuring the AWS provider with both Spinifex and Predastore endpoints
- Creating S3 buckets on Predastore via Terraform
- Deploying a Python webapp with cloud-init that talks to S3
- Passing credentials and configuration to instances via user-data
**Prerequisites:**
- Spinifex installed and running (see [Installing Spinifex](/docs/installing-spinifex))
- Predastore running (S3 API on port 8443)
- A Debian 12 AMI imported (see [Setting Up Your Cluster](/docs/setting-up-your-cluster))
- OpenTofu or Terraform installed
- `AWS_PROFILE=spinifex` configured
- The EC2 instance must be able to reach Predastore — use the host's br-wan IP, not localhost
## Instructions
### Step 1. Get the Template
Clone the Terraform examples from the Spinifex repository:
```bash
git clone --depth 1 --filter=blob:none --sparse https://github.com/mulgadc/spinifex.git spinifex-tf
cd spinifex-tf
git sparse-checkout set docs/terraform
cd docs/terraform/s3-webapp
```
Or create the files manually and paste the full configuration below.
### Step 2. Create terraform.tfvars
Before deploying, create a `terraform.tfvars` with your Predastore credentials. The `predastore_host` must be reachable from inside the VPC — use the host's br-wan or LAN IP, not localhost.
<!-- INCLUDE: terraform.tfvars.example lang:hcl -->
### Step 3. Create main.tf
<!-- INCLUDE: main.tf lang:hcl -->
### Step 4. Deploy
```bash
export AWS_PROFILE=spinifex
tofu init
tofu apply
```
### Step 5. Test the Application
> **Note:** EC2 instances can take 30+ seconds to boot after apply. If SSH or HTTP is unreachable, wait and retry.
Open the `web_url` output in your browser. You should see the file browser UI. Upload a file and verify it appears in the list.
```bash
# Verify via CLI
curl http://<public_ip>
# Check the S3 bucket directly
aws s3 ls s3://webapp-uploads/ --profile spinifex --endpoint-url https://localhost:8443
```
### Clean Up
```bash
tofu destroy
```
## Troubleshooting
### Predastore Connection Refused from Instance
The EC2 instance cannot reach `localhost` on the host. Set `predastore_host` to the host's br-wan or LAN IP address:
```hcl
predastore_host = "192.168.1.10:8443"
```
### S3 Bucket Creation Fails
Verify Predastore is running and accessible:
```bash
curl -k https://localhost:8443/
aws s3 ls --profile spinifex --endpoint-url https://localhost:8443
```
### Flask App Not Starting
SSH into the instance and check the service:
```bash
ssh -i s3-webapp-demo.pem ec2-user@<public_ip>
sudo systemctl status s3-webapp
sudo journalctl -u s3-webapp --no-pager -n 50
```
### Upload Fails or Files Not Appearing
Check that the S3 credentials in `/opt/webapp/.env` are correct and that the bucket exists:
```bash
ssh -i s3-webapp-demo.pem ec2-user@<public_ip>
cat /opt/webapp/.env
/opt/webapp/venv/bin/python -c "import boto3; print('boto3 OK')"
```
### AMI Not Found
Ensure you have imported a Debian 12 image:
```bash
aws ec2 describe-images --owners 000000000000 --profile spinifex
```
docs/terraform/s3-webapp/main.tf
+4
View File
@@ -420,6 +420,10 @@ resource "aws_instance" "webapp" {
# Outputs
# ---------------------------------------------------------------------------
output "note" {
value = "EC2 instances can take 30+ seconds to boot after apply. If SSH or HTTP is unreachable, wait and retry."
}
output "instance_id" {
value = aws_instance.webapp.id
}