From 8bcffd8a04a88c7b8ef63b1dc62d35017d509b46 Mon Sep 17 00:00:00 2001 From: Rohmilchkaese <37877524+Rohmilchkaese@users.noreply.github.com> Date: Thu, 9 Jul 2026 09:58:06 +0200 Subject: [PATCH] feat(helm): support multiple server pools for capacity expansion (#3325) * feat(helm): support multiple server pools for capacity expansion The server already understands multiple pools (RUSTFS_VOLUMES is split on spaces, one pool expression each; rc admin pool/expand/rebalance/ decommission exist), but the chart could only render a single StatefulSet. Add an opt-in pools mode: - pools.enabled=false (default) renders byte-identical output to the current chart (verified against five values variants) - pools.list renders one StatefulSet per entry; entries may set replicaCount (4 or 16) and a storageclass block, everything else is inherited from the top-level values - pool 0 keeps the legacy StatefulSet/pod/PVC names and its (immutable) selector, so existing single-pool deployments can be expanded in place without renaming or data loss; additional pools render as -poolN with a rustfs.com/pool label in their selector - RUSTFS_VOLUMES and RUSTFS_SERVER_DOMAINS enumerate every pool - pod anti-affinity is scoped per pool so pools may share nodes while each pool still spreads its own pods across distinct nodes - template validation fails fast on unsupported replicaCount, an empty pools.list, or pools in standalone mode - pools are append-only by design (list index = identity), documented in values.yaml and the README * fix(helm): truncate pool names DNS-safely, document PDB pool scope Truncate -poolN to 63 chars by shortening the base name rather than the suffix, so the pool index always survives and two pools of a max-length release cannot collide. Document that the single PodDisruptionBudget deliberately spans all pools. * fix(helm): pool-mode anti-affinity must be preferred, not required Found by live-testing in-place expansion on a 5-node cluster (4-replica pool 0 + 4-replica pool 1): with requiredDuringScheduling anti-affinity the expansion deadlocks in a cycle that cannot self-heal - 1. the not-yet-rolled pool-0 pods still carry the unscoped required rule and repel every rustfs pod from their nodes, so part of pool 1 stays Pending (no IP, no headless-DNS record); 2. every pod that already has the new RUSTFS_VOLUMES exits fatally on 'failed to lookup address information' for those Pending peers; 3. the StatefulSet rolling update is gated on the crashing pod going Ready, so the remaining pool-0 pods never roll and keep repelling. Because StatefulSets assign revisions per ordinal, even a subsequent template fix cannot rescue a wedged rollout (Pending ordinals are only recreated with the new template after the higher ordinals go Ready) - the new pool's StatefulSet has to be recreated. Shipping the soft affinity from the start avoids the state entirely; expansion was re-tested end-to-end with it and converges. Pool-mode rendering only - pools.enabled=false still renders the existing required anti-affinity byte-identically. --------- Co-authored-by: cxymds Co-authored-by: houseme Co-authored-by: cxymds --- helm/README.md | 55 ++++++++ helm/rustfs/templates/_helpers.tpl | 83 ++++++++++-- helm/rustfs/templates/statefulset.yaml | 176 +++++++++++++++---------- helm/rustfs/values.yaml | 27 ++++ 4 files changed, 260 insertions(+), 81 deletions(-) diff --git a/helm/README.md b/helm/README.md index 776dd8fd2..3d1f88a31 100644 --- a/helm/README.md +++ b/helm/README.md @@ -133,6 +133,8 @@ uer. `ClusterIssuer` or `Issuer`. | | pdb.maxUnavailable | string | `1` | | | pdb.minAvailable | string | `""` | | | podAnnotations | object | `{}` | | +| pools.enabled | bool | `false` | Enable multiple server pools (capacity expansion, distributed mode only). | +| pools.list | list | `[]` | One entry per pool; entries may set `replicaCount` (4 or 16) and `storageclass`, omitted fields inherit top-level values. Append-only. | | podLabels | object | `{}` | | | podSecurityContext.fsGroup | int | `10001` | | | podSecurityContext.runAsGroup | int | `10001` | | @@ -216,6 +218,59 @@ Both approaches support pulling from private registries seamlessly and you can a - The default size for data and logs dir is **256Mi** which must satisfy the production usage,you should specify `storageclass.dataStorageSize` and `storageclass.logStorageSize` to change the size, for example, 1Ti for data and 1Gi for logs. +# Server pools (capacity expansion) + +In distributed mode the chart can run multiple **server pools** — independent +StatefulSets whose drives together form one cluster, the same expansion model +the RustFS server already supports via space-separated `RUSTFS_VOLUMES` +expressions (`rc admin pool ls` / `expand` / `rebalance` / `decommission`). + +With `pools.enabled=false` (default) the chart behaves exactly as before: +one StatefulSet driven by the top-level `replicaCount`/`storageclass`. + +To expand an existing deployment, enable pools and describe the current +layout as pool 0 plus your new capacity: + +```yaml +pools: + enabled: true + list: + - {} # pool 0: inherits top-level values and keeps the + # existing StatefulSet/pod/PVC names and data + - replicaCount: 4 # pool 1: new capacity (4 or 16) + storageclass: + dataStorageSize: 10Gi +``` + +Each entry may set `replicaCount` (4 or 16) and/or a `storageclass` block; +omitted fields inherit the top-level values. Additional pools render as +`-pool` StatefulSets; all pools share the headless service, +the main service, the configuration and the credentials. + +Notes: + +* **Pools are append-only.** The list index determines the StatefulSet name — + never remove or reorder entries. Retire a pool with + `rc admin decommission` before removing it from the list. +* During the expansion rollout, pods restart until every pod of every pool is + resolvable — the server refuses to start with unresolvable peers, so expect + a few crash/restart cycles before the cluster converges. This is harmless. +* After the cluster converges, run `rc admin rebalance start ` to + spread existing objects across the new pool. +* Pod anti-affinity in pool mode is scoped per pool and **preferred** + (soft), not required: two pools can share nodes, and each pool's own pods + spread across distinct nodes when capacity allows. Soft affinity is + load-bearing for in-place expansion — with required rules, the + not-yet-rolled pods of the existing pool block the new pool's pods from + their nodes while the rolled pods crash on the unresolvable (Pending) + peers, deadlocking the rollout on any cluster with fewer nodes than the + total pod count. Single-pool deployments (`pools.enabled=false`) keep the + chart's existing required anti-affinity unchanged. +* The PodDisruptionBudget spans all pools: with the default + `pdb.maxUnavailable: 1`, at most one pod of the whole cluster may be + evicted at a time. This is deliberately conservative — quorum safety + matters across the union of all pools. + # Installation ## Requirement diff --git a/helm/rustfs/templates/_helpers.tpl b/helm/rustfs/templates/_helpers.tpl index e5353a389..edc400f51 100644 --- a/helm/rustfs/templates/_helpers.tpl +++ b/helm/rustfs/templates/_helpers.tpl @@ -168,22 +168,80 @@ falling back to cluster.local when the value is empty or only dots. {{- .Values.clusterDomain | default "cluster.local" | trimAll "." | default "cluster.local" -}} {{- end -}} +{{/* +Return the fully qualified name of a server pool. +Pool 0 keeps the legacy single-pool name so that existing deployments can be +expanded in place without renaming their StatefulSet, pods or PVCs. +Expects a dict with keys "root" (the chart root context) and "index". +*/}} +{{- define "rustfs.poolFullname" -}} +{{- if eq (int .index) 0 -}} +{{- include "rustfs.fullname" .root -}} +{{- else -}} +{{- /* Truncate the base name, not the suffix: the pool index must survive + truncation or two long-named pools could collide. */ -}} +{{- $suffix := printf "-pool%d" (int .index) -}} +{{- printf "%s%s" (include "rustfs.fullname" .root | trunc (int (sub 63 (len $suffix))) | trimSuffix "-") $suffix -}} +{{- end -}} +{{- end }} + +{{/* +Return the normalized list of server pools as JSON. +With pools disabled this is a single pool built from the top-level +replicaCount/storageclass values, which keeps all rendered output identical +to the previous single-pool chart. With pools enabled, each entry of +pools.list becomes one pool; omitted fields inherit the top-level values. +Pools are strictly append-only: the index determines the StatefulSet name, +so entries must never be removed or reordered. +*/}} +{{- define "rustfs.pools" -}} +{{- $pools := list -}} +{{- if and .Values.pools .Values.pools.enabled -}} +{{- if not .Values.mode.distributed.enabled -}} +{{- fail "pools.enabled requires mode.distributed.enabled=true" -}} +{{- end -}} +{{- if not .Values.pools.list -}} +{{- fail "pools.enabled is true but pools.list is empty" -}} +{{- end -}} +{{- range $i, $p := .Values.pools.list -}} +{{- $p = default (dict) $p -}} +{{- $replicas := int (default $.Values.replicaCount $p.replicaCount) -}} +{{- if and (ne $replicas 4) (ne $replicas 16) -}} +{{- fail (printf "pools.list[%d].replicaCount must be 4 or 16, got %d" $i $replicas) -}} +{{- end -}} +{{- $sc := mergeOverwrite (deepCopy $.Values.storageclass) (default (dict) $p.storageclass) -}} +{{- $pools = append $pools (dict "index" $i "fullname" (include "rustfs.poolFullname" (dict "root" $ "index" $i)) "replicaCount" $replicas "storageclass" $sc) -}} +{{- end -}} +{{- else -}} +{{- $pools = append $pools (dict "index" 0 "fullname" (include "rustfs.fullname" .) "replicaCount" (int .Values.replicaCount) "storageclass" .Values.storageclass) -}} +{{- end -}} +{{- toJson $pools -}} +{{- end }} + {{/* Render RUSTFS_VOLUMES +One volume expression per server pool, joined with spaces (the server splits +RUSTFS_VOLUMES on spaces, one pool per expression). */}} {{- define "rustfs.volumes" -}} - {{- $protocol := "http" -}} {{- if .Values.mtls.enabled -}} {{- $protocol = "https" -}} {{- end -}} - -{{- if eq (int .Values.replicaCount) 4 }} -{{- printf "%s://%s-{0...%d}.%s-headless.%s.svc.%s:%d/data/rustfs{0...%d}" $protocol (include "rustfs.fullname" .) (sub (.Values.replicaCount | int) 1) (include "rustfs.fullname" . ) .Release.Namespace (include "rustfs.clusterDomain" .) (.Values.service.endpoint.port | int) (sub (.Values.replicaCount | int) 1) }} -{{- end }} -{{- if eq (int .Values.replicaCount) 16 }} -{{- printf "%s://%s-{0...%d}.%s-headless.%s.svc.%s:%d/data" $protocol (include "rustfs.fullname" .) (sub (.Values.replicaCount | int) 1) (include "rustfs.fullname" .) .Release.Namespace (include "rustfs.clusterDomain" .) (.Values.service.endpoint.port | int) }} -{{- end }} +{{- $headless := printf "%s-headless" (include "rustfs.fullname" .) -}} +{{- $ns := .Release.Namespace -}} +{{- $domain := include "rustfs.clusterDomain" . -}} +{{- $port := .Values.service.endpoint.port | int -}} +{{- $exprs := list -}} +{{- range $pool := include "rustfs.pools" . | fromJsonArray -}} +{{- $n := int $pool.replicaCount -}} +{{- if eq $n 4 -}} +{{- $exprs = append $exprs (printf "%s://%s-{0...%d}.%s.%s.svc.%s:%d/data/rustfs{0...%d}" $protocol $pool.fullname (sub $n 1) $headless $ns $domain $port (sub $n 1)) -}} +{{- else if eq $n 16 -}} +{{- $exprs = append $exprs (printf "%s://%s-{0...%d}.%s.%s.svc.%s:%d/data" $protocol $pool.fullname (sub $n 1) $headless $ns $domain $port) -}} +{{- end -}} +{{- end -}} +{{- join " " $exprs -}} {{- end }} {{/* @@ -192,13 +250,14 @@ Render RUSTFS_SERVER_DOMAINS {{- define "rustfs.serverDomains" -}} {{- $domains := list .Values.config.rustfs.domains -}} -{{- $fullname := include "rustfs.fullname" . -}} -{{- $replicaCount := int .Values.replicaCount -}} +{{- $headless := printf "%s-headless" (include "rustfs.fullname" .) -}} {{- $servicePort := .Values.service.endpoint.port | default 9000 -}} -{{- range $i := until $replicaCount -}} - {{- $podDomain := printf "%s-%d.%s-headless:%d" $fullname $i $fullname (int $servicePort) -}} +{{- range $pool := include "rustfs.pools" . | fromJsonArray -}} +{{- range $i := until (int $pool.replicaCount) -}} + {{- $podDomain := printf "%s-%d.%s:%d" $pool.fullname $i $headless (int $servicePort) -}} {{- $domains = append $domains $podDomain -}} {{- end -}} +{{- end -}} {{- join "," $domains -}} {{- end -}} diff --git a/helm/rustfs/templates/statefulset.yaml b/helm/rustfs/templates/statefulset.yaml index e58d89cb3..8128eeccd 100644 --- a/helm/rustfs/templates/statefulset.yaml +++ b/helm/rustfs/templates/statefulset.yaml @@ -1,87 +1,124 @@ {{- $logDir := .Values.config.rustfs.obs_log_directory }} {{- $logDirEnabled := ne $logDir "" }} +{{- $poolsEnabled := and .Values.pools .Values.pools.enabled }} {{- if .Values.mode.distributed.enabled }} +{{- range $pool := include "rustfs.pools" . | fromJsonArray }} --- apiVersion: apps/v1 kind: StatefulSet metadata: - name: {{ include "rustfs.fullname" . }} - namespace: {{ .Release.Namespace }} + name: {{ $pool.fullname }} + namespace: {{ $.Release.Namespace }} labels: - {{- include "rustfs.labels" . | nindent 4 }} - {{- with .Values.commonLabels }} + {{- include "rustfs.labels" $ | nindent 4 }} + {{- if $poolsEnabled }} + rustfs.com/pool: {{ $pool.index | quote }} + {{- end }} + {{- with $.Values.commonLabels }} {{- toYaml . | nindent 4 }} {{- end }} spec: - serviceName: {{ include "rustfs.fullname" . }}-headless - replicas: {{ .Values.replicaCount }} + serviceName: {{ include "rustfs.fullname" $ }}-headless + replicas: {{ $pool.replicaCount }} podManagementPolicy: Parallel selector: matchLabels: - {{- include "rustfs.selectorLabels" . | nindent 6 }} + {{- include "rustfs.selectorLabels" $ | nindent 6 }} + {{- if gt (int $pool.index) 0 }} + {{- /* Pool 0 keeps the legacy selector: it is immutable on existing + StatefulSets and must not change when pools are enabled. */}} + rustfs.com/pool: {{ $pool.index | quote }} + {{- end }} template: metadata: labels: - {{- include "rustfs.selectorLabels" . | nindent 8 }} - {{- with .Values.podLabels }} + {{- include "rustfs.selectorLabels" $ | nindent 8 }} + {{- if $poolsEnabled }} + rustfs.com/pool: {{ $pool.index | quote }} + {{- end }} + {{- with $.Values.podLabels }} {{- toYaml . | nindent 8 }} {{- end }} - {{- with .Values.podAnnotations }} + {{- with $.Values.podAnnotations }} annotations: {{- toYaml . | nindent 8 }} {{- end }} spec: - serviceAccountName: {{ include "rustfs.serviceAccountName" . }} - enableServiceLinks: {{ .Values.enableServiceLinks }} - {{- with include "chart.imagePullSecrets" . }} + serviceAccountName: {{ include "rustfs.serviceAccountName" $ }} + enableServiceLinks: {{ $.Values.enableServiceLinks }} + {{- with include "chart.imagePullSecrets" $ }} imagePullSecrets: {{- . | nindent 8 }} {{- end }} - {{- if and .Values.nodeSelector (not .Values.affinity.nodeAffinity) }} + {{- if and $.Values.nodeSelector (not $.Values.affinity.nodeAffinity) }} nodeSelector: - {{- toYaml .Values.nodeSelector | nindent 8 }} + {{- toYaml $.Values.nodeSelector | nindent 8 }} {{- end }} - {{- if .Values.affinity }} + {{- if $.Values.affinity }} affinity: nodeAffinity: - {{- if .Values.affinity.nodeAffinity }} - {{- toYaml .Values.affinity.nodeAffinity | nindent 10 }} + {{- if $.Values.affinity.nodeAffinity }} + {{- toYaml $.Values.affinity.nodeAffinity | nindent 10 }} {{- else }} {} {{- end }} - {{- if .Values.affinity.podAntiAffinity.enabled }} + {{- if $.Values.affinity.podAntiAffinity.enabled }} podAntiAffinity: + {{- if $poolsEnabled }} + {{- /* Pool-scoped and PREFERRED, not required: during in-place + expansion the not-yet-rolled pods' required rules block new + pods from their nodes, while the new pods' DNS must resolve + before the roll can proceed - required rules deadlock + expansion whenever the cluster has fewer nodes than total + pods. Soft anti-affinity converges (the scheduler still + spreads when capacity allows). */}} + preferredDuringSchedulingIgnoredDuringExecution: + - weight: 100 + podAffinityTerm: + labelSelector: + matchExpressions: + - key: app.kubernetes.io/name + operator: In + values: + - {{ include "rustfs.name" $ }} + - key: rustfs.com/pool + operator: In + values: + - {{ $pool.index | quote }} + topologyKey: {{ $.Values.affinity.podAntiAffinity.topologyKey }} + {{- else }} requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: - key: app.kubernetes.io/name operator: In values: - - {{ include "rustfs.name" . }} - topologyKey: {{ .Values.affinity.podAntiAffinity.topologyKey }} + - {{ include "rustfs.name" $ }} + topologyKey: {{ $.Values.affinity.podAntiAffinity.topologyKey }} + {{- end }} {{- end }} {{- end }} - {{- if .Values.tolerations }} + {{- if $.Values.tolerations }} tolerations: - {{- toYaml .Values.tolerations | nindent 8 }} + {{- toYaml $.Values.tolerations | nindent 8 }} {{- end }} - {{- if and .Values.topologySpreadConstraints.enabled .Values.topologySpreadConstraints.constraints }} + {{- if and $.Values.topologySpreadConstraints.enabled $.Values.topologySpreadConstraints.constraints }} topologySpreadConstraints: - {{- toYaml .Values.topologySpreadConstraints.constraints | nindent 8 }} + {{- toYaml $.Values.topologySpreadConstraints.constraints | nindent 8 }} {{- end }} - priorityClassName: {{ .Values.priorityClassName }} + priorityClassName: {{ $.Values.priorityClassName }} securityContext: - {{- toYaml .Values.podSecurityContext | nindent 8 }} + {{- toYaml $.Values.podSecurityContext | nindent 8 }} initContainers: - name: init-step - image: "{{ .Values.image.initImage.repository }}:{{ .Values.image.initImage.tag }}" - imagePullPolicy: {{ .Values.image.initImage.pullPolicy }} + image: "{{ $.Values.image.initImage.repository }}:{{ $.Values.image.initImage.tag }}" + imagePullPolicy: {{ $.Values.image.initImage.pullPolicy }} securityContext: - {{- toYaml .Values.containerSecurityContext | nindent 12 }} + {{- toYaml $.Values.containerSecurityContext | nindent 12 }} env: - name: REPLICA_COUNT - value: {{ .Values.replicaCount | quote }} + value: {{ $pool.replicaCount | quote }} command: - sh - -c @@ -98,12 +135,12 @@ spec: chmod 755 /mnt/rustfs/logs {{- end }} volumeMounts: - {{- if eq (int .Values.replicaCount) 4 }} - {{- range $i := until (int .Values.replicaCount) }} + {{- if eq (int $pool.replicaCount) 4 }} + {{- range $i := until (int $pool.replicaCount) }} - name: data-rustfs-{{ $i }} mountPath: /data/rustfs{{ $i }} {{- end }} - {{- else if eq (int .Values.replicaCount) 16 }} + {{- else if eq (int $pool.replicaCount) 16 }} - name: data mountPath: /data {{- end }} @@ -112,31 +149,31 @@ spec: mountPath: /mnt/rustfs {{- end }} containers: - - name: {{ .Chart.Name }} - image: "{{ .Values.image.rustfs.repository }}:{{ .Values.image.rustfs.tag | default .Chart.AppVersion }}" + - name: {{ $.Chart.Name }} + image: "{{ $.Values.image.rustfs.repository }}:{{ $.Values.image.rustfs.tag | default $.Chart.AppVersion }}" command: ["/usr/bin/rustfs"] - imagePullPolicy: {{ .Values.image.rustfs.pullPolicy }} + imagePullPolicy: {{ $.Values.image.rustfs.pullPolicy }} securityContext: - {{- toYaml .Values.containerSecurityContext | nindent 12 }} + {{- toYaml $.Values.containerSecurityContext | nindent 12 }} ports: - name: endpoint - containerPort: {{ .Values.service.endpoint.port }} + containerPort: {{ $.Values.service.endpoint.port }} - name: console - containerPort: {{ .Values.service.console.port }} - {{- with .Values.extraEnv }} + containerPort: {{ $.Values.service.console.port }} + {{- with $.Values.extraEnv }} env: {{- toYaml . | nindent 12 }} {{- end }} envFrom: - configMapRef: - name: {{ include "rustfs.fullname" . }}-config + name: {{ include "rustfs.fullname" $ }}-config - secretRef: - name: {{ include "rustfs.secretName" . }} + name: {{ include "rustfs.secretName" $ }} resources: - {{- toYaml .Values.resources | nindent 12 }} - {{- include "rustfs.probes" . | nindent 10 }} + {{- toYaml $.Values.resources | nindent 12 }} + {{- include "rustfs.probes" $ | nindent 10 }} volumeMounts: - {{- if .Values.mtls.enabled }} + {{- if $.Values.mtls.enabled }} - name: client-cert mountPath: /opt/tls/client_cert.pem subPath: client_cert.pem @@ -156,7 +193,7 @@ spec: mountPath: /opt/tls/ca.crt subPath: ca.crt {{- end }} - {{- with .Values.extraVolumeMounts }} + {{- with $.Values.extraVolumeMounts }} {{- toYaml . | nindent 12 }} {{- end }} {{- if $logDirEnabled }} @@ -164,21 +201,21 @@ spec: mountPath: {{ $logDir }} subPath: logs {{- end }} - {{- if eq (int .Values.replicaCount) 4 }} - {{- range $i := until (int .Values.replicaCount) }} + {{- if eq (int $pool.replicaCount) 4 }} + {{- range $i := until (int $pool.replicaCount) }} - name: data-rustfs-{{ $i }} mountPath: /data/rustfs{{ $i }} {{- end }} - {{- else if eq (int .Values.replicaCount) 16 }} + {{- else if eq (int $pool.replicaCount) 16 }} - name: data mountPath: /data {{- end }} - {{- if or .Values.mtls.enabled .Values.extraVolumes }} + {{- if or $.Values.mtls.enabled $.Values.extraVolumes }} volumes: - {{- if .Values.mtls.enabled }} + {{- if $.Values.mtls.enabled }} - name: server-cert secret: - secretName: {{ include "rustfs.fullname" . }}-server-tls + secretName: {{ include "rustfs.fullname" $ }}-server-tls items: - key: tls.crt path: rustfs_cert.pem @@ -188,7 +225,7 @@ spec: path: ca.crt - name: client-cert secret: - secretName: {{ include "rustfs.fullname" . }}-client-tls + secretName: {{ include "rustfs.fullname" $ }}-client-tls items: - key: tls.crt path: client_cert.pem @@ -197,7 +234,7 @@ spec: - key: ca.crt path: client_ca.crt {{- end }} - {{- with .Values.extraVolumes }} + {{- with $.Values.extraVolumes }} {{- toYaml . | nindent 8 }} {{- end }} {{- end }} @@ -208,18 +245,18 @@ spec: metadata: name: logs labels: - {{- toYaml .Values.commonLabels | nindent 10 }} + {{- toYaml $.Values.commonLabels | nindent 10 }} annotations: - {{- toYaml .Values.storageclass.pvcAnnotations.logs | nindent 10 }} + {{- toYaml $pool.storageclass.pvcAnnotations.logs | nindent 10 }} spec: accessModes: ["ReadWriteOnce"] - storageClassName: {{ .Values.storageclass.name }} + storageClassName: {{ $pool.storageclass.name }} resources: requests: - storage: {{ .Values.storageclass.logStorageSize }} + storage: {{ $pool.storageclass.logStorageSize }} {{- end }} - {{- if eq (int .Values.replicaCount) 4 }} - {{- range $i := until (int .Values.replicaCount) }} + {{- if eq (int $pool.replicaCount) 4 }} + {{- range $i := until (int $pool.replicaCount) }} - apiVersion: v1 kind: PersistentVolumeClaim metadata: @@ -227,28 +264,29 @@ spec: labels: {{- toYaml $.Values.commonLabels | nindent 10 }} annotations: - {{- toYaml $.Values.storageclass.pvcAnnotations.data | nindent 10 }} + {{- toYaml $pool.storageclass.pvcAnnotations.data | nindent 10 }} spec: accessModes: ["ReadWriteOnce"] - storageClassName: {{ $.Values.storageclass.name }} + storageClassName: {{ $pool.storageclass.name }} resources: requests: - storage: {{ $.Values.storageclass.dataStorageSize }} + storage: {{ $pool.storageclass.dataStorageSize }} {{- end }} - {{- else if eq (int .Values.replicaCount) 16 }} + {{- else if eq (int $pool.replicaCount) 16 }} - apiVersion: v1 kind: PersistentVolumeClaim metadata: name: data labels: - {{- toYaml .Values.commonLabels | nindent 10 }} + {{- toYaml $.Values.commonLabels | nindent 10 }} annotations: - {{- toYaml .Values.storageclass.pvcAnnotations.data | nindent 10 }} + {{- toYaml $pool.storageclass.pvcAnnotations.data | nindent 10 }} spec: accessModes: ["ReadWriteOnce"] - storageClassName: {{ .Values.storageclass.name }} + storageClassName: {{ $pool.storageclass.name }} resources: requests: - storage: {{ .Values.storageclass.dataStorageSize }} + storage: {{ $pool.storageclass.dataStorageSize }} {{- end }} {{- end }} +{{- end }} diff --git a/helm/rustfs/values.yaml b/helm/rustfs/values.yaml index 3dd6ec1ae..85d4f56a3 100644 --- a/helm/rustfs/values.yaml +++ b/helm/rustfs/values.yaml @@ -53,6 +53,33 @@ mode: distributed: enabled: true +# Server pools for horizontal capacity expansion (distributed mode only). +# Disabled by default: the chart then behaves exactly like the classic +# single-pool layout driven by the top-level replicaCount/storageclass. +# +# With pools.enabled=true, one StatefulSet is rendered per entry of +# pools.list and RUSTFS_VOLUMES contains every pool's volume expression +# (space separated), which is how the server discovers multiple pools. +# Each entry may set replicaCount (4 or 16) and/or a storageclass block; +# omitted fields inherit the top-level values. +# +# IMPORTANT: +# - Pools are append-only. The list index determines the StatefulSet name +# (pool 0 keeps the legacy name so existing single-pool deployments can be +# expanded in place); never remove or reorder entries. To retire a pool, +# use `rc admin decommission` first. +# - After adding a pool, run `rc admin rebalance start ` to spread +# existing data across the new capacity. +pools: + enabled: false + list: [] + # Example: expand an existing 4-node deployment with a second, larger pool: + # list: + # - {} # pool 0: inherits top-level values, keeps existing PVCs + # - replicaCount: 4 # pool 1: new capacity + # storageclass: + # dataStorageSize: 10Gi + secret: existingSecret: "" # SECURITY: rendering fails by default unless one of the following is true: