mirror of
https://github.com/mulgadc/spinifex.git
synced 2026-10-23 12:47:48 +00:00
Fresh install tweaks
This commit is contained in:
Ben Duncan
+23
-12
@@ -1,24 +1,26 @@
|
||||
# Hive installation
|
||||
|
||||
Notes for development environment installation.
|
||||
|
||||
## Dependencies
|
||||
|
||||
For running on Ubuntu 22.04
|
||||
For running on Ubuntu 22.04 and 25.04 (tested)
|
||||
|
||||
```bash
|
||||
sudo add-apt-repository universe
|
||||
sudo apt install nbdkit nbdkit-plugin-dev pkg-config qemu-system qemu-utils qemu-kvm libvirt-daemon-system libvirt-clients libvirt-dev
|
||||
sudo apt install nbdkit nbdkit-plugin-dev pkg-config qemu-system qemu-utils qemu-kvm libvirt-daemon-system libvirt-clients libvirt-dev make gcc
|
||||
```
|
||||
|
||||
Create hive user
|
||||
Ensure the Go toolkit is installed, recommended to install the latest directly from [https://go.dev/dl/](https://go.dev/dl/)
|
||||
|
||||
```bash
|
||||
sudo adduser --disabled-password hive
|
||||
Confirm go is correctly installed, and set in your $PATH
|
||||
|
||||
```
|
||||
go version
|
||||
```
|
||||
|
||||
Add hive to the libvirt group to manage VMs
|
||||
|
||||
```bash
|
||||
sudo adduser hive libvirt
|
||||
```
|
||||
go version go1.25.4 linux/amd64
|
||||
```
|
||||
|
||||
Hive provided AWS API/SDK layer functionality, which requires the AWS CLI tool to be installed to interface with the system.
|
||||
@@ -27,13 +29,22 @@ Hive provided AWS API/SDK layer functionality, which requires the AWS CLI tool t
|
||||
sudo apt install awscli
|
||||
```
|
||||
|
||||
|
||||
## Build
|
||||
|
||||
Compile `hive` binary which will be used throughout the installation tutorial.
|
||||
Create the base directory for the Hive development environment
|
||||
|
||||
```
|
||||
make
|
||||
mkdir ~/Development/mulga/
|
||||
cd ~/Development/mulga/
|
||||
git clone https://github.com/mulgadc/hive.git
|
||||
```
|
||||
|
||||
Setup the dev environment and package dependencies on [viperblock](https://github.com/mulgadc/viperblock/) and [predastore](https://github.com/mulgadc/predastore/)
|
||||
|
||||
```
|
||||
# Setup dependencies and development environment
|
||||
./scripts/clone-deps.sh # Clone viperblock + predastore repositories
|
||||
./scripts/dev-setup.sh # Setup complete development environment
|
||||
```
|
||||
|
||||
Confirm ./bin/hive exists and executable.
|
||||
|
||||
@@ -17,6 +17,79 @@ This roadmap has been integrated into the comprehensive [HIVE_DEVELOPMENT_PLAN.m
|
||||
* Add delete-volume support via EBS (s3 vol-*) for terminated instance
|
||||
* Add default LRU cache support for viperblock, depending on the instance type / volume size and system memory available.
|
||||
|
||||
## Multi-node setup
|
||||
|
||||
- Development release - v0.1.0 (single node)
|
||||
- Production release - v0.2.0 (multi node)
|
||||
|
||||
Design layout for multi-node configuration.
|
||||
|
||||
```
|
||||
# node1
|
||||
hive admin init --region ap-southeast-2 --availability-zone ap-southeast-2a --nodes 3 --bind :8443 --data-dir ~/hive/
|
||||
|
||||
# node2
|
||||
hive admin join --region ap-southeast-2 --availability-zone ap-southeast-2a --host node1.local:8443 --data-dir ~/hive/
|
||||
|
||||
# node3, optionally toggle EBS/EC2/NATs support only
|
||||
hive admin join --region ap-southeast-2 --availability-zone ap-southeast-2a --host node1.local:8443 --data-dir ~/hive/ --cap ebs,ec2,nats
|
||||
```
|
||||
|
||||
If --host is missing, the join command tries multicast broadcast to find parent node.
|
||||
|
||||
Set region with `--region ap-southeast-2` which will create a new Hive cluster for the specified region.
|
||||
|
||||
Set the availability-zone, for production it is recommended to have at least 2 availability-zones defined per region deployment of Hive.
|
||||
|
||||
Overview:
|
||||
|
||||
* init creates a cluster-id, node-id, and a short join token, starts a tiny control server on :8443, and writes DNS hint files if available.
|
||||
|
||||
* init sets target size to 3 and waits until all 3 nodes join and ack.
|
||||
|
||||
* join nodes contact node1, present the token or cluster-id, and advertise capabilities.
|
||||
|
||||
* node1 appends them to the member set and immediately pushes the current settings bundle to them.
|
||||
|
||||
* When member count reaches 3, node1 broadcasts the final cluster settings to all nodes.
|
||||
|
||||
* Every node writes the same cluster.json and starts the services that match its capabilities.
|
||||
|
||||
Node exchange payload
|
||||
|
||||
```json
|
||||
{
|
||||
"cluster_id": "c-82d5",
|
||||
"node_id": "n-rpi2",
|
||||
"addr": "rpi2.local:8443",
|
||||
"version": "0.1.0",
|
||||
"caps": ["ec2","s3"],
|
||||
"ts": 1731388800
|
||||
}
|
||||
```
|
||||
|
||||
Settings bundle (identical on all 3 once committed)
|
||||
|
||||
```json
|
||||
{
|
||||
"cluster_id": "c-82d5",
|
||||
"target_size": 3,
|
||||
"members": [
|
||||
{"node_id":"n-rpi1","addr":"rpi1.local:8443","caps":["ec2","s3","nats","ebs"]},
|
||||
{"node_id":"n-rpi2","addr":"rpi2.local:8443","caps":["ec2","s3"]},
|
||||
{"node_id":"n-rpi3","addr":"rpi3.local:8443","caps":["nats","ebs"]}
|
||||
],
|
||||
"services": {
|
||||
"ec2": {"api_bind":":9001"},
|
||||
"s3": {"api_bind":":9002","replicas":2},
|
||||
"nats":{"cluster":"c-82d5","quorum":2},
|
||||
"ebs": {"replicas":3,"block_size":4096}
|
||||
},
|
||||
"epoch": 1,
|
||||
"sig": "ed25519:..." // signed by rpi1 during init
|
||||
}
|
||||
```
|
||||
|
||||
## Original TODO Items → Development Plan Integration
|
||||
|
||||
### ✅ **Completed Integration**
|
||||
|
||||
@@ -748,7 +748,11 @@ func setupAWSCredentials(accessKey, secretKey, region, certPath string) error {
|
||||
configPath := filepath.Join(awsDir, "config")
|
||||
|
||||
// Determine profile name
|
||||
profileName := "default"
|
||||
//profileName := "default"
|
||||
|
||||
// Use hive as the default profile
|
||||
profileName := "hive"
|
||||
|
||||
if fileExists(credPath) {
|
||||
// Check if default profile already exists
|
||||
cfg, err := ini.Load(credPath)
|
||||
|
||||
+14
-10
@@ -114,17 +114,17 @@ make build
|
||||
echo "✅ Hive built successfully"
|
||||
|
||||
# Initialize Hive configuration using admin init
|
||||
echo ""
|
||||
echo "🔐 Initializing Hive configuration..."
|
||||
#echo ""
|
||||
#echo "🔐 Initializing Hive configuration..."
|
||||
|
||||
if [[ ! -f "$CONFIG_DIR/hive.toml" ]]; then
|
||||
echo "📋 Running hive admin init..."
|
||||
./bin/hive admin init --config-dir "$CONFIG_DIR"
|
||||
echo "✅ Hive configuration initialized"
|
||||
else
|
||||
echo "✅ Hive configuration already exists"
|
||||
echo " To re-initialize, run: ./bin/hive admin init --force"
|
||||
fi
|
||||
#if [[ ! -f "$CONFIG_DIR/hive.toml" ]]; then
|
||||
# echo "📋 Running hive admin init..."
|
||||
# ./bin/hive admin init --config-dir "$CONFIG_DIR"
|
||||
# echo "✅ Hive configuration initialized"
|
||||
#else
|
||||
# echo "✅ Hive configuration already exists"
|
||||
# echo " To re-initialize, run: ./bin/hive admin init --force"
|
||||
#fi
|
||||
|
||||
# Build components
|
||||
echo ""
|
||||
@@ -153,6 +153,10 @@ fi
|
||||
echo ""
|
||||
echo "🎉 Development environment setup complete!"
|
||||
echo ""
|
||||
echo "When running Hive for the first time, run the init function to create the"
|
||||
echo "default directories for data, config files and layout required."
|
||||
echo "./bin/hive admin init"
|
||||
echo ""
|
||||
echo "🚀 To start the development environment:"
|
||||
echo " ./scripts/start-dev.sh"
|
||||
echo ""
|
||||
|
||||
Reference in New Issue
Block a user