feat(helm): support multiple server pools for capacity expansion (#3325)

* feat(helm): support multiple server pools for capacity expansion

The server already understands multiple pools (RUSTFS_VOLUMES is split on
spaces, one pool expression each; rc admin pool/expand/rebalance/
decommission exist), but the chart could only render a single StatefulSet.

Add an opt-in pools mode:

- pools.enabled=false (default) renders byte-identical output to the
  current chart (verified against five values variants)
- pools.list renders one StatefulSet per entry; entries may set
  replicaCount (4 or 16) and a storageclass block, everything else is
  inherited from the top-level values
- pool 0 keeps the legacy StatefulSet/pod/PVC names and its (immutable)
  selector, so existing single-pool deployments can be expanded in place
  without renaming or data loss; additional pools render as
  <fullname>-poolN with a rustfs.com/pool label in their selector
- RUSTFS_VOLUMES and RUSTFS_SERVER_DOMAINS enumerate every pool
- pod anti-affinity is scoped per pool so pools may share nodes while
  each pool still spreads its own pods across distinct nodes
- template validation fails fast on unsupported replicaCount, an empty
  pools.list, or pools in standalone mode
- pools are append-only by design (list index = identity), documented in
  values.yaml and the README

* fix(helm): truncate pool names DNS-safely, document PDB pool scope

Truncate <fullname>-poolN to 63 chars by shortening the base name rather
than the suffix, so the pool index always survives and two pools of a
max-length release cannot collide. Document that the single
PodDisruptionBudget deliberately spans all pools.

* fix(helm): pool-mode anti-affinity must be preferred, not required

Found by live-testing in-place expansion on a 5-node cluster (4-replica
pool 0 + 4-replica pool 1): with requiredDuringScheduling anti-affinity
the expansion deadlocks in a cycle that cannot self-heal -

  1. the not-yet-rolled pool-0 pods still carry the unscoped required
     rule and repel every rustfs pod from their nodes, so part of pool 1
     stays Pending (no IP, no headless-DNS record);
  2. every pod that already has the new RUSTFS_VOLUMES exits fatally on
     'failed to lookup address information' for those Pending peers;
  3. the StatefulSet rolling update is gated on the crashing pod going
     Ready, so the remaining pool-0 pods never roll and keep repelling.

Because StatefulSets assign revisions per ordinal, even a subsequent
template fix cannot rescue a wedged rollout (Pending ordinals are only
recreated with the new template after the higher ordinals go Ready) -
the new pool's StatefulSet has to be recreated. Shipping the soft
affinity from the start avoids the state entirely; expansion was
re-tested end-to-end with it and converges.

Pool-mode rendering only - pools.enabled=false still renders the
existing required anti-affinity byte-identically.

---------

Co-authored-by: cxymds <Cxymds@qq.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: cxymds <cxymds@gmail.com>

helm/rustfs/values.yaml

@@ -53,6 +53,33 @@ mode:
   distributed:
     enabled: true
 
+# Server pools for horizontal capacity expansion (distributed mode only).
+# Disabled by default: the chart then behaves exactly like the classic
+# single-pool layout driven by the top-level replicaCount/storageclass.
+#
+# With pools.enabled=true, one StatefulSet is rendered per entry of
+# pools.list and RUSTFS_VOLUMES contains every pool's volume expression
+# (space separated), which is how the server discovers multiple pools.
+# Each entry may set replicaCount (4 or 16) and/or a storageclass block;
+# omitted fields inherit the top-level values.
+#
+# IMPORTANT:
+# - Pools are append-only. The list index determines the StatefulSet name
+#   (pool 0 keeps the legacy name so existing single-pool deployments can be
+#   expanded in place); never remove or reorder entries. To retire a pool,
+#   use `rc admin decommission` first.
+# - After adding a pool, run `rc admin rebalance start <alias>` to spread
+#   existing data across the new capacity.
+pools:
+  enabled: false
+  list: []
+  # Example: expand an existing 4-node deployment with a second, larger pool:
+  # list:
+  #   - {}                          # pool 0: inherits top-level values, keeps existing PVCs
+  #   - replicaCount: 4             # pool 1: new capacity
+  #     storageclass:
+  #       dataStorageSize: 10Gi
+
 secret:
   existingSecret: ""
   # SECURITY: rendering fails by default unless one of the following is true: