mirror of https://github.com/openziti/ziti.git synced 2026-10-27 05:57:06 +00:00

Files

T

Clint Dovholuk f6814c117a Quickstart ha only (#3724 )

* wip changes: trying to make the quickstart nothing but ha

* when on windows be nice and use a powershell friendly line continuation char

* remove --clustered usage from config create as that's the default and remove 'ha' from quickstart test

* forgot to update the tests as well

* remove other 'ha' references in the config generation and verify traffic

* remove ha refs from the quickstart ha doc

* was a bit too quick to remove these flags - putting back

* change from fatals to error and RunE. add --spiffed-id to pki creation. remove now defunct TestDatabaseFile*

* updates to docker based test

* pr feedback updates

* missed some test updates

2026-03-27 10:13:42 -04:00

9.2 KiB

Raw Permalink Blame History

HA Quickstart

You can explore Ziti HA by running three local processes on unique TCP ports. This interactive tutorial is self-contained, so you don't need to do anything else before or after this to have a working, local cluster.

Create an empty working directory. All commands are run here.
```
cd $(mktemp -d)
```

Run the first member in the background to create the cluster.

nohup ziti edge quickstart \
    --instance-id="ctrl1" \
    --ctrl-port="1281" \
    --router-port="3021" \
    --home="${PWD}" \
    --ctrl-address="127.0.0.1" \
    --router-address="127.0.0.1" \
    --trust-domain="ha-quickstart" \
&> ctrl1.log &

Confirm the first job is running and check the log for startup errors.

tail ctrl1.log; echo; jobs

Expected output:

.
.
.
... logs ...
.
.
.

[1]  + running    nohup ziti edge quickstart --instance-id="ctrl1" --ctrl-port="1281"      &

Run the second member and join the cluster.

nohup ziti edge quickstart join \
    --instance-id="ctrl2" \
    --ctrl-port="1282" \
    --router-port="3022" \
    --home="${PWD}" \
    --ctrl-address="127.0.0.1" \
    --router-address="127.0.0.1" \
    --trust-domain="ha-quickstart" \
    --cluster-member="tls:127.0.0.1:1281" \
&> ctrl2.log &

Confirm the second job is running and check the log for startup errors.

tail ctrl2.log; echo; jobs

Expected output:

.
.
.
... logs ...
.
.
.

[1]  - running    nohup ziti edge quickstart --instance-id="ctrl1" --ctrl-port="1281"      &
[2]  + running    nohup ziti edge quickstart join --instance-id="ctrl2" --ctrl-port="1282"

Run the third member and join the cluster.

nohup ziti edge quickstart join \
    --instance-id="ctrl3" \
    --ctrl-port="1283" \
    --router-port="3023" \
    --home="${PWD}" \
    --ctrl-address="127.0.0.1" \
    --router-address="127.0.0.1" \
    --trust-domain="ha-quickstart" \
    --cluster-member="tls:127.0.0.1:1281" \
&> ctrl3.log &

Confirm the third job is running and check the log for startup errors.

tail ctrl3.log; echo; jobs

Expected output:

.
.
.
... logs ...
.
.
.

[1]    running    nohup ziti edge quickstart --instance-id="ctrl1" --ctrl-port="1281"      &
[2]  - running    nohup ziti edge quickstart join --instance-id="ctrl2" --ctrl-port="1282"     
[3]  + running    nohup ziti edge quickstart join --instance-id="ctrl3" --ctrl-port="1283"

Optionally, follow interleaved logs in another window.
```
tail -F -n +1 *.log
```

List agent applications.

ziti agent list

Expected output:

╭────────┬────────────┬────────┬─────────────────────────────┬────────────┬─────────────┬───────────╮
│    PID │ EXECUTABLE │ APP ID │ UNIX SOCKET                 │ APP TYPE   │ APP VERSION │ APP ALIAS │
├────────┼────────────┼────────┼─────────────────────────────┼────────────┼─────────────┼───────────┤
│ 276912 │ ziti       │ ctrl1  │ /tmp/gops-agent.276912.sock │ controller │ v0.0.0      │           │
│ 277714 │ ziti       │ ctrl2  │ /tmp/gops-agent.277714.sock │ controller │ v0.0.0      │           │
│ 281490 │ ziti       │ ctrl3  │ /tmp/gops-agent.281490.sock │ controller │ v0.0.0      │           │
╰────────┴────────────┴────────┴─────────────────────────────┴────────────┴─────────────┴───────────╯

Identify the cluster leader.

ziti agent cluster list --app-id ctrl1

Expected output:

╭───────┬────────────────────┬───────┬────────┬─────────┬───────────╮
│ ID    │ ADDRESS            │ VOTER │ LEADER │ VERSION │ CONNECTED │
├───────┼────────────────────┼───────┼────────┼─────────┼───────────┤
│ ctrl1 │ tls:127.0.0.1:1281 │ true  │ true   │ v0.0.0  │ true      │
│ ctrl2 │ tls:127.0.0.1:1282 │ true  │ false  │ v0.0.0  │ true      │
│ ctrl3 │ tls:127.0.0.1:1283 │ true  │ false  │ v0.0.0  │ true      │
╰───────┴────────────────────┴───────┴────────┴─────────┴───────────╯

Simulate a member availability incident.

Identify the job number for the cluster leader. It may not be %1.

jobs

Job 1 belongs to ctrl1, the current leader.

[1]    running    nohup ziti edge quickstart --instance-id="ctrl1" --ctrl-port="1281"       
[2]  - running    nohup ziti edge quickstart join --instance-id="ctrl3" --ctrl-port="1283"     
[3]  + running    nohup ziti edge quickstart join --instance-id="ctrl2" --ctrl-port="1282"

kill %1

Inspect the cluster via another member ID.

ziti agent cluster list --app-id ctrl2

Expected output:

╭───────┬────────────────────┬───────┬────────┬─────────────────┬───────────╮
│ ID    │ ADDRESS            │ VOTER │ LEADER │ VERSION         │ CONNECTED │
├───────┼────────────────────┼───────┼────────┼─────────────────┼───────────┤
│ ctrl1 │ tls:127.0.0.1:1281 │ true  │ false  │ <not connected> │ false     │
│ ctrl2 │ tls:127.0.0.1:1282 │ true  │ true   │ v0.0.0          │ true      │
│ ctrl3 │ tls:127.0.0.1:1283 │ true  │ false  │ v0.0.0          │ true      │
╰───────┴────────────────────┴───────┴────────┴─────────────────┴───────────╯

Restart any disconnected member.

Any member can be restarted with the quickstart command.

nohup ziti edge quickstart \
    --instance-id="ctrl1" \
    --home="${PWD}" \
&>> ctrl1.log &

Inspect the cluster.

Once restarted, ctrl1 does not necessarily resume being the leader.

ziti agent cluster list --app-id ctrl2

Expected output:

╭───────┬────────────────────┬───────┬────────┬─────────────────┬───────────╮
│ ID    │ ADDRESS            │ VOTER │ LEADER │ VERSION         │ CONNECTED │
├───────┼────────────────────┼───────┼────────┼─────────────────┼───────────┤
│ ctrl1 │ tls:127.0.0.1:1281 │ true  │ false  │ v0.0.0          │ true      │
│ ctrl2 │ tls:127.0.0.1:1282 │ true  │ false  │ v0.0.0          │ true      │
│ ctrl3 │ tls:127.0.0.1:1283 │ true  │ true   │ v0.0.0          │ true      │
╰───────┴────────────────────┴───────┴────────┴─────────────────┴───────────╯

Stop all background jobs.

BASH

kill $(jobs -p)

ZSH

kill ${${(v)jobstates##*:*:}%=*}

Expected output:

[1]  + done       nohup ziti edge quickstart --instance-id="ctrl1" --ctrl-port="1281"
[2]    done       nohup ziti edge quickstart join --instance-id="ctrl2" --ctrl-port="1282"
[3]  + done       nohup ziti edge quickstart join --instance-id="ctrl3" --ctrl-port="1283"

9.2 KiB Raw Permalink Blame History

HA Quickstart

9.2 KiB

Raw Permalink Blame History