gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-08-20 19:42:17 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(helm): support multiple server pools for capacity expansion (#3325)

Browse Source 
* feat(helm): support multiple server pools for capacity expansion

The server already understands multiple pools (RUSTFS_VOLUMES is split on
spaces, one pool expression each; rc admin pool/expand/rebalance/
decommission exist), but the chart could only render a single StatefulSet.

Add an opt-in pools mode:

- pools.enabled=false (default) renders byte-identical output to the
  current chart (verified against five values variants)
- pools.list renders one StatefulSet per entry; entries may set
  replicaCount (4 or 16) and a storageclass block, everything else is
  inherited from the top-level values
- pool 0 keeps the legacy StatefulSet/pod/PVC names and its (immutable)
  selector, so existing single-pool deployments can be expanded in place
  without renaming or data loss; additional pools render as
  <fullname>-poolN with a rustfs.com/pool label in their selector
- RUSTFS_VOLUMES and RUSTFS_SERVER_DOMAINS enumerate every pool
- pod anti-affinity is scoped per pool so pools may share nodes while
  each pool still spreads its own pods across distinct nodes
- template validation fails fast on unsupported replicaCount, an empty
  pools.list, or pools in standalone mode
- pools are append-only by design (list index = identity), documented in
  values.yaml and the README

* fix(helm): truncate pool names DNS-safely, document PDB pool scope

Truncate <fullname>-poolN to 63 chars by shortening the base name rather
than the suffix, so the pool index always survives and two pools of a
max-length release cannot collide. Document that the single
PodDisruptionBudget deliberately spans all pools.

* fix(helm): pool-mode anti-affinity must be preferred, not required

Found by live-testing in-place expansion on a 5-node cluster (4-replica
pool 0 + 4-replica pool 1): with requiredDuringScheduling anti-affinity
the expansion deadlocks in a cycle that cannot self-heal -

  1. the not-yet-rolled pool-0 pods still carry the unscoped required
     rule and repel every rustfs pod from their nodes, so part of pool 1
     stays Pending (no IP, no headless-DNS record);
  2. every pod that already has the new RUSTFS_VOLUMES exits fatally on
     'failed to lookup address information' for those Pending peers;
  3. the StatefulSet rolling update is gated on the crashing pod going
     Ready, so the remaining pool-0 pods never roll and keep repelling.

Because StatefulSets assign revisions per ordinal, even a subsequent
template fix cannot rescue a wedged rollout (Pending ordinals are only
recreated with the new template after the higher ordinals go Ready) -
the new pool's StatefulSet has to be recreated. Shipping the soft
affinity from the start avoids the state entirely; expansion was
re-tested end-to-end with it and converges.

Pool-mode rendering only - pools.enabled=false still renders the
existing required anti-affinity byte-identically.

---------

Co-authored-by: cxymds <Cxymds@qq.com>
Co-authored-by: houseme <housemecn@gmail.com>
Co-authored-by: cxymds <cxymds@gmail.com>
This commit is contained in:
Rohmilchkaese
2026-07-09 09:58:06 +02:00
committed by GitHub
parent 10ba2273c4
commit 8bcffd8a04
4 changed files with 260 additions and 81 deletions
Download Patch File Download Diff File Expand all files Collapse all files
helm/README.md
+55
View File
@@ -133,6 +133,8 @@ uer. `ClusterIssuer` or `Issuer`. |
| pdb.maxUnavailable | string | `1` | | | pdb.maxUnavailable | string | `1` | |
| pdb.minAvailable | string | `""` | | | pdb.minAvailable | string | `""` | |
| podAnnotations | object | `{}` | | | podAnnotations | object | `{}` | |
| pools.enabled | bool | `false` | Enable multiple server pools (capacity expansion, distributed mode only). |
| pools.list | list | `[]` | One entry per pool; entries may set `replicaCount` (4 or 16) and `storageclass`, omitted fields inherit top-level values. Append-only. |
| podLabels | object | `{}` | | | podLabels | object | `{}` | |
| podSecurityContext.fsGroup | int | `10001` | | | podSecurityContext.fsGroup | int | `10001` | |
| podSecurityContext.runAsGroup | int | `10001` | | | podSecurityContext.runAsGroup | int | `10001` | |
@@ -216,6 +218,59 @@ Both approaches support pulling from private registries seamlessly and you can a
- The default size for data and logs dir is **256Mi** which must satisfy the production usage,you should specify `storageclass.dataStorageSize` and `storageclass.logStorageSize` to change the size, for example, 1Ti for data and 1Gi for logs. - The default size for data and logs dir is **256Mi** which must satisfy the production usage,you should specify `storageclass.dataStorageSize` and `storageclass.logStorageSize` to change the size, for example, 1Ti for data and 1Gi for logs.
# Server pools (capacity expansion)
In distributed mode the chart can run multiple **server pools** — independent
StatefulSets whose drives together form one cluster, the same expansion model
the RustFS server already supports via space-separated `RUSTFS_VOLUMES`
expressions (`rc admin pool ls` / `expand` / `rebalance` / `decommission`).
With `pools.enabled=false` (default) the chart behaves exactly as before:
one StatefulSet driven by the top-level `replicaCount`/`storageclass`.
To expand an existing deployment, enable pools and describe the current
layout as pool 0 plus your new capacity:
```yaml
pools:
enabled: true
list:
- {} # pool 0: inherits top-level values and keeps the
# existing StatefulSet/pod/PVC names and data
- replicaCount: 4 # pool 1: new capacity (4 or 16)
storageclass:
dataStorageSize: 10Gi
```
Each entry may set `replicaCount` (4 or 16) and/or a `storageclass` block;
omitted fields inherit the top-level values. Additional pools render as
`<fullname>-pool<N>` StatefulSets; all pools share the headless service,
the main service, the configuration and the credentials.
Notes:
* **Pools are append-only.** The list index determines the StatefulSet name —
never remove or reorder entries. Retire a pool with
`rc admin decommission` before removing it from the list.
* During the expansion rollout, pods restart until every pod of every pool is
resolvable — the server refuses to start with unresolvable peers, so expect
a few crash/restart cycles before the cluster converges. This is harmless.
* After the cluster converges, run `rc admin rebalance start <alias>` to
spread existing objects across the new pool.
* Pod anti-affinity in pool mode is scoped per pool and **preferred**
(soft), not required: two pools can share nodes, and each pool's own pods
spread across distinct nodes when capacity allows. Soft affinity is
load-bearing for in-place expansion — with required rules, the
not-yet-rolled pods of the existing pool block the new pool's pods from
their nodes while the rolled pods crash on the unresolvable (Pending)
peers, deadlocking the rollout on any cluster with fewer nodes than the
total pod count. Single-pool deployments (`pools.enabled=false`) keep the
chart's existing required anti-affinity unchanged.
* The PodDisruptionBudget spans all pools: with the default
`pdb.maxUnavailable: 1`, at most one pod of the whole cluster may be
evicted at a time. This is deliberately conservative — quorum safety
matters across the union of all pools.
# Installation # Installation
## Requirement ## Requirement
helm/rustfs/templates/_helpers.tpl
+71 -12
View File
@@ -168,22 +168,80 @@ falling back to cluster.local when the value is empty or only dots.
{{- .Values.clusterDomain | default "cluster.local" | trimAll "." | default "cluster.local" -}} {{- .Values.clusterDomain | default "cluster.local" | trimAll "." | default "cluster.local" -}}
{{- end -}} {{- end -}}
{{/*
Return the fully qualified name of a server pool.
Pool 0 keeps the legacy single-pool name so that existing deployments can be
expanded in place without renaming their StatefulSet, pods or PVCs.
Expects a dict with keys "root" (the chart root context) and "index".
*/}}
{{- define "rustfs.poolFullname" -}}
{{- if eq (int .index) 0 -}}
{{- include "rustfs.fullname" .root -}}
{{- else -}}
{{- /* Truncate the base name, not the suffix: the pool index must survive
truncation or two long-named pools could collide. */ -}}
{{- $suffix := printf "-pool%d" (int .index) -}}
{{- printf "%s%s" (include "rustfs.fullname" .root | trunc (int (sub 63 (len $suffix))) | trimSuffix "-") $suffix -}}
{{- end -}}
{{- end }}
{{/*
Return the normalized list of server pools as JSON.
With pools disabled this is a single pool built from the top-level
replicaCount/storageclass values, which keeps all rendered output identical
to the previous single-pool chart. With pools enabled, each entry of
pools.list becomes one pool; omitted fields inherit the top-level values.
Pools are strictly append-only: the index determines the StatefulSet name,
so entries must never be removed or reordered.
*/}}
{{- define "rustfs.pools" -}}
{{- $pools := list -}}
{{- if and .Values.pools .Values.pools.enabled -}}
{{- if not .Values.mode.distributed.enabled -}}
{{- fail "pools.enabled requires mode.distributed.enabled=true" -}}
{{- end -}}
{{- if not .Values.pools.list -}}
{{- fail "pools.enabled is true but pools.list is empty" -}}
{{- end -}}
{{- range $i, $p := .Values.pools.list -}}
{{- $p = default (dict) $p -}}
{{- $replicas := int (default $.Values.replicaCount $p.replicaCount) -}}
{{- if and (ne $replicas 4) (ne $replicas 16) -}}
{{- fail (printf "pools.list[%d].replicaCount must be 4 or 16, got %d" $i $replicas) -}}
{{- end -}}
{{- $sc := mergeOverwrite (deepCopy $.Values.storageclass) (default (dict) $p.storageclass) -}}
{{- $pools = append $pools (dict "index" $i "fullname" (include "rustfs.poolFullname" (dict "root" $ "index" $i)) "replicaCount" $replicas "storageclass" $sc) -}}
{{- end -}}
{{- else -}}
{{- $pools = append $pools (dict "index" 0 "fullname" (include "rustfs.fullname" .) "replicaCount" (int .Values.replicaCount) "storageclass" .Values.storageclass) -}}
{{- end -}}
{{- toJson $pools -}}
{{- end }}
{{/* {{/*
Render RUSTFS_VOLUMES Render RUSTFS_VOLUMES
One volume expression per server pool, joined with spaces (the server splits
RUSTFS_VOLUMES on spaces, one pool per expression).
*/}} */}}
{{- define "rustfs.volumes" -}} {{- define "rustfs.volumes" -}}
{{- $protocol := "http" -}} {{- $protocol := "http" -}}
{{- if .Values.mtls.enabled -}} {{- if .Values.mtls.enabled -}}
{{- $protocol = "https" -}} {{- $protocol = "https" -}}
{{- end -}} {{- end -}}
{{- $headless := printf "%s-headless" (include "rustfs.fullname" .) -}}
{{- if eq (int .Values.replicaCount) 4 }} {{- $ns := .Release.Namespace -}}
{{- printf "%s://%s-{0...%d}.%s-headless.%s.svc.%s:%d/data/rustfs{0...%d}" $protocol (include "rustfs.fullname" .) (sub (.Values.replicaCount | int) 1) (include "rustfs.fullname" . ) .Release.Namespace (include "rustfs.clusterDomain" .) (.Values.service.endpoint.port | int) (sub (.Values.replicaCount | int) 1) }} {{- $domain := include "rustfs.clusterDomain" . -}}
{{- end }} {{- $port := .Values.service.endpoint.port | int -}}
{{- if eq (int .Values.replicaCount) 16 }} {{- $exprs := list -}}
{{- printf "%s://%s-{0...%d}.%s-headless.%s.svc.%s:%d/data" $protocol (include "rustfs.fullname" .) (sub (.Values.replicaCount | int) 1) (include "rustfs.fullname" .) .Release.Namespace (include "rustfs.clusterDomain" .) (.Values.service.endpoint.port | int) }} {{- range $pool := include "rustfs.pools" . | fromJsonArray -}}
{{- end }} {{- $n := int $pool.replicaCount -}}
{{- if eq $n 4 -}}
{{- $exprs = append $exprs (printf "%s://%s-{0...%d}.%s.%s.svc.%s:%d/data/rustfs{0...%d}" $protocol $pool.fullname (sub $n 1) $headless $ns $domain $port (sub $n 1)) -}}
{{- else if eq $n 16 -}}
{{- $exprs = append $exprs (printf "%s://%s-{0...%d}.%s.%s.svc.%s:%d/data" $protocol $pool.fullname (sub $n 1) $headless $ns $domain $port) -}}
{{- end -}}
{{- end -}}
{{- join " " $exprs -}}
{{- end }} {{- end }}
{{/* {{/*
@@ -192,13 +250,14 @@ Render RUSTFS_SERVER_DOMAINS
{{- define "rustfs.serverDomains" -}} {{- define "rustfs.serverDomains" -}}
{{- $domains := list .Values.config.rustfs.domains -}} {{- $domains := list .Values.config.rustfs.domains -}}
{{- $fullname := include "rustfs.fullname" . -}} {{- $headless := printf "%s-headless" (include "rustfs.fullname" .) -}}
{{- $replicaCount := int .Values.replicaCount -}}
{{- $servicePort := .Values.service.endpoint.port | default 9000 -}} {{- $servicePort := .Values.service.endpoint.port | default 9000 -}}
{{- range $i := until $replicaCount -}} {{- range $pool := include "rustfs.pools" . | fromJsonArray -}}
{{- $podDomain := printf "%s-%d.%s-headless:%d" $fullname $i $fullname (int $servicePort) -}} {{- range $i := until (int $pool.replicaCount) -}}
{{- $podDomain := printf "%s-%d.%s:%d" $pool.fullname $i $headless (int $servicePort) -}}
{{- $domains = append $domains $podDomain -}} {{- $domains = append $domains $podDomain -}}
{{- end -}} {{- end -}}
{{- end -}}
{{- join "," $domains -}} {{- join "," $domains -}}
{{- end -}} {{- end -}}
helm/rustfs/templates/statefulset.yaml
+107 -69
View File
@@ -1,87 +1,124 @@
{{- $logDir := .Values.config.rustfs.obs_log_directory }} {{- $logDir := .Values.config.rustfs.obs_log_directory }}
{{- $logDirEnabled := ne $logDir "" }} {{- $logDirEnabled := ne $logDir "" }}
{{- $poolsEnabled := and .Values.pools .Values.pools.enabled }}
{{- if .Values.mode.distributed.enabled }} {{- if .Values.mode.distributed.enabled }}
{{- range $pool := include "rustfs.pools" . | fromJsonArray }}
--- ---
apiVersion: apps/v1 apiVersion: apps/v1
kind: StatefulSet kind: StatefulSet
metadata: metadata:
name: {{ include "rustfs.fullname" . }} name: {{ $pool.fullname }}
namespace: {{ .Release.Namespace }} namespace: {{ $.Release.Namespace }}
labels: labels:
{{- include "rustfs.labels" . | nindent 4 }} {{- include "rustfs.labels" $ | nindent 4 }}
{{- with .Values.commonLabels }} {{- if $poolsEnabled }}
rustfs.com/pool: {{ $pool.index | quote }}
{{- end }}
{{- with $.Values.commonLabels }}
{{- toYaml . | nindent 4 }} {{- toYaml . | nindent 4 }}
{{- end }} {{- end }}
spec: spec:
serviceName: {{ include "rustfs.fullname" . }}-headless serviceName: {{ include "rustfs.fullname" $ }}-headless
replicas: {{ .Values.replicaCount }} replicas: {{ $pool.replicaCount }}
podManagementPolicy: Parallel podManagementPolicy: Parallel
selector: selector:
matchLabels: matchLabels:
{{- include "rustfs.selectorLabels" . | nindent 6 }} {{- include "rustfs.selectorLabels" $ | nindent 6 }}
{{- if gt (int $pool.index) 0 }}
{{- /* Pool 0 keeps the legacy selector: it is immutable on existing
StatefulSets and must not change when pools are enabled. */}}
rustfs.com/pool: {{ $pool.index | quote }}
{{- end }}
template: template:
metadata: metadata:
labels: labels:
{{- include "rustfs.selectorLabels" . | nindent 8 }} {{- include "rustfs.selectorLabels" $ | nindent 8 }}
{{- with .Values.podLabels }} {{- if $poolsEnabled }}
rustfs.com/pool: {{ $pool.index | quote }}
{{- end }}
{{- with $.Values.podLabels }}
{{- toYaml . | nindent 8 }} {{- toYaml . | nindent 8 }}
{{- end }} {{- end }}
{{- with .Values.podAnnotations }} {{- with $.Values.podAnnotations }}
annotations: annotations:
{{- toYaml . | nindent 8 }} {{- toYaml . | nindent 8 }}
{{- end }} {{- end }}
spec: spec:
serviceAccountName: {{ include "rustfs.serviceAccountName" . }} serviceAccountName: {{ include "rustfs.serviceAccountName" $ }}
enableServiceLinks: {{ .Values.enableServiceLinks }} enableServiceLinks: {{ $.Values.enableServiceLinks }}
{{- with include "chart.imagePullSecrets" . }} {{- with include "chart.imagePullSecrets" $ }}
imagePullSecrets: imagePullSecrets:
{{- . | nindent 8 }} {{- . | nindent 8 }}
{{- end }} {{- end }}
{{- if and .Values.nodeSelector (not .Values.affinity.nodeAffinity) }} {{- if and $.Values.nodeSelector (not $.Values.affinity.nodeAffinity) }}
nodeSelector: nodeSelector:
{{- toYaml .Values.nodeSelector | nindent 8 }} {{- toYaml $.Values.nodeSelector | nindent 8 }}
{{- end }} {{- end }}
{{- if .Values.affinity }} {{- if $.Values.affinity }}
affinity: affinity:
nodeAffinity: nodeAffinity:
{{- if .Values.affinity.nodeAffinity }} {{- if $.Values.affinity.nodeAffinity }}
{{- toYaml .Values.affinity.nodeAffinity | nindent 10 }} {{- toYaml $.Values.affinity.nodeAffinity | nindent 10 }}
{{- else }} {{- else }}
{} {}
{{- end }} {{- end }}
{{- if .Values.affinity.podAntiAffinity.enabled }} {{- if $.Values.affinity.podAntiAffinity.enabled }}
podAntiAffinity: podAntiAffinity:
{{- if $poolsEnabled }}
{{- /* Pool-scoped and PREFERRED, not required: during in-place
expansion the not-yet-rolled pods' required rules block new
pods from their nodes, while the new pods' DNS must resolve
before the roll can proceed - required rules deadlock
expansion whenever the cluster has fewer nodes than total
pods. Soft anti-affinity converges (the scheduler still
spreads when capacity allows). */}}
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
labelSelector:
matchExpressions:
- key: app.kubernetes.io/name
operator: In
values:
- {{ include "rustfs.name" $ }}
- key: rustfs.com/pool
operator: In
values:
- {{ $pool.index | quote }}
topologyKey: {{ $.Values.affinity.podAntiAffinity.topologyKey }}
{{- else }}
requiredDuringSchedulingIgnoredDuringExecution: requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector: - labelSelector:
matchExpressions: matchExpressions:
- key: app.kubernetes.io/name - key: app.kubernetes.io/name
operator: In operator: In
values: values:
- {{ include "rustfs.name" . }} - {{ include "rustfs.name" $ }}
topologyKey: {{ .Values.affinity.podAntiAffinity.topologyKey }} topologyKey: {{ $.Values.affinity.podAntiAffinity.topologyKey }}
{{- end }}
{{- end }} {{- end }}
{{- end }} {{- end }}
{{- if .Values.tolerations }} {{- if $.Values.tolerations }}
tolerations: tolerations:
{{- toYaml .Values.tolerations | nindent 8 }} {{- toYaml $.Values.tolerations | nindent 8 }}
{{- end }} {{- end }}
{{- if and .Values.topologySpreadConstraints.enabled .Values.topologySpreadConstraints.constraints }} {{- if and $.Values.topologySpreadConstraints.enabled $.Values.topologySpreadConstraints.constraints }}
topologySpreadConstraints: topologySpreadConstraints:
{{- toYaml .Values.topologySpreadConstraints.constraints | nindent 8 }} {{- toYaml $.Values.topologySpreadConstraints.constraints | nindent 8 }}
{{- end }} {{- end }}
priorityClassName: {{ .Values.priorityClassName }} priorityClassName: {{ $.Values.priorityClassName }}
securityContext: securityContext:
{{- toYaml .Values.podSecurityContext | nindent 8 }} {{- toYaml $.Values.podSecurityContext | nindent 8 }}
initContainers: initContainers:
- name: init-step - name: init-step
image: "{{ .Values.image.initImage.repository }}:{{ .Values.image.initImage.tag }}" image: "{{ $.Values.image.initImage.repository }}:{{ $.Values.image.initImage.tag }}"
imagePullPolicy: {{ .Values.image.initImage.pullPolicy }} imagePullPolicy: {{ $.Values.image.initImage.pullPolicy }}
securityContext: securityContext:
{{- toYaml .Values.containerSecurityContext | nindent 12 }} {{- toYaml $.Values.containerSecurityContext | nindent 12 }}
env: env:
- name: REPLICA_COUNT - name: REPLICA_COUNT
value: {{ .Values.replicaCount | quote }} value: {{ $pool.replicaCount | quote }}
command: command:
- sh - sh
- -c - -c
@@ -98,12 +135,12 @@ spec:
chmod 755 /mnt/rustfs/logs chmod 755 /mnt/rustfs/logs
{{- end }} {{- end }}
volumeMounts: volumeMounts:
{{- if eq (int .Values.replicaCount) 4 }} {{- if eq (int $pool.replicaCount) 4 }}
{{- range $i := until (int .Values.replicaCount) }} {{- range $i := until (int $pool.replicaCount) }}
- name: data-rustfs-{{ $i }} - name: data-rustfs-{{ $i }}
mountPath: /data/rustfs{{ $i }} mountPath: /data/rustfs{{ $i }}
{{- end }} {{- end }}
{{- else if eq (int .Values.replicaCount) 16 }} {{- else if eq (int $pool.replicaCount) 16 }}
- name: data - name: data
mountPath: /data mountPath: /data
{{- end }} {{- end }}
@@ -112,31 +149,31 @@ spec:
mountPath: /mnt/rustfs mountPath: /mnt/rustfs
{{- end }} {{- end }}
containers: containers:
- name: {{ .Chart.Name }} - name: {{ $.Chart.Name }}
image: "{{ .Values.image.rustfs.repository }}:{{ .Values.image.rustfs.tag | default .Chart.AppVersion }}" image: "{{ $.Values.image.rustfs.repository }}:{{ $.Values.image.rustfs.tag | default $.Chart.AppVersion }}"
command: ["/usr/bin/rustfs"] command: ["/usr/bin/rustfs"]
imagePullPolicy: {{ .Values.image.rustfs.pullPolicy }} imagePullPolicy: {{ $.Values.image.rustfs.pullPolicy }}
securityContext: securityContext:
{{- toYaml .Values.containerSecurityContext | nindent 12 }} {{- toYaml $.Values.containerSecurityContext | nindent 12 }}
ports: ports:
- name: endpoint - name: endpoint
containerPort: {{ .Values.service.endpoint.port }} containerPort: {{ $.Values.service.endpoint.port }}
- name: console - name: console
containerPort: {{ .Values.service.console.port }} containerPort: {{ $.Values.service.console.port }}
{{- with .Values.extraEnv }} {{- with $.Values.extraEnv }}
env: env:
{{- toYaml . | nindent 12 }} {{- toYaml . | nindent 12 }}
{{- end }} {{- end }}
envFrom: envFrom:
- configMapRef: - configMapRef:
name: {{ include "rustfs.fullname" . }}-config name: {{ include "rustfs.fullname" $ }}-config
- secretRef: - secretRef:
name: {{ include "rustfs.secretName" . }} name: {{ include "rustfs.secretName" $ }}
resources: resources:
{{- toYaml .Values.resources | nindent 12 }} {{- toYaml $.Values.resources | nindent 12 }}
{{- include "rustfs.probes" . | nindent 10 }} {{- include "rustfs.probes" $ | nindent 10 }}
volumeMounts: volumeMounts:
{{- if .Values.mtls.enabled }} {{- if $.Values.mtls.enabled }}
- name: client-cert - name: client-cert
mountPath: /opt/tls/client_cert.pem mountPath: /opt/tls/client_cert.pem
subPath: client_cert.pem subPath: client_cert.pem
@@ -156,7 +193,7 @@ spec:
mountPath: /opt/tls/ca.crt mountPath: /opt/tls/ca.crt
subPath: ca.crt subPath: ca.crt
{{- end }} {{- end }}
{{- with .Values.extraVolumeMounts }} {{- with $.Values.extraVolumeMounts }}
{{- toYaml . | nindent 12 }} {{- toYaml . | nindent 12 }}
{{- end }} {{- end }}
{{- if $logDirEnabled }} {{- if $logDirEnabled }}
@@ -164,21 +201,21 @@ spec:
mountPath: {{ $logDir }} mountPath: {{ $logDir }}
subPath: logs subPath: logs
{{- end }} {{- end }}
{{- if eq (int .Values.replicaCount) 4 }} {{- if eq (int $pool.replicaCount) 4 }}
{{- range $i := until (int .Values.replicaCount) }} {{- range $i := until (int $pool.replicaCount) }}
- name: data-rustfs-{{ $i }} - name: data-rustfs-{{ $i }}
mountPath: /data/rustfs{{ $i }} mountPath: /data/rustfs{{ $i }}
{{- end }} {{- end }}
{{- else if eq (int .Values.replicaCount) 16 }} {{- else if eq (int $pool.replicaCount) 16 }}
- name: data - name: data
mountPath: /data mountPath: /data
{{- end }} {{- end }}
{{- if or .Values.mtls.enabled .Values.extraVolumes }} {{- if or $.Values.mtls.enabled $.Values.extraVolumes }}
volumes: volumes:
{{- if .Values.mtls.enabled }} {{- if $.Values.mtls.enabled }}
- name: server-cert - name: server-cert
secret: secret:
secretName: {{ include "rustfs.fullname" . }}-server-tls secretName: {{ include "rustfs.fullname" $ }}-server-tls
items: items:
- key: tls.crt - key: tls.crt
path: rustfs_cert.pem path: rustfs_cert.pem
@@ -188,7 +225,7 @@ spec:
path: ca.crt path: ca.crt
- name: client-cert - name: client-cert
secret: secret:
secretName: {{ include "rustfs.fullname" . }}-client-tls secretName: {{ include "rustfs.fullname" $ }}-client-tls
items: items:
- key: tls.crt - key: tls.crt
path: client_cert.pem path: client_cert.pem
@@ -197,7 +234,7 @@ spec:
- key: ca.crt - key: ca.crt
path: client_ca.crt path: client_ca.crt
{{- end }} {{- end }}
{{- with .Values.extraVolumes }} {{- with $.Values.extraVolumes }}
{{- toYaml . | nindent 8 }} {{- toYaml . | nindent 8 }}
{{- end }} {{- end }}
{{- end }} {{- end }}
@@ -208,18 +245,18 @@ spec:
metadata: metadata:
name: logs name: logs
labels: labels:
{{- toYaml .Values.commonLabels | nindent 10 }} {{- toYaml $.Values.commonLabels | nindent 10 }}
annotations: annotations:
{{- toYaml .Values.storageclass.pvcAnnotations.logs | nindent 10 }} {{- toYaml $pool.storageclass.pvcAnnotations.logs | nindent 10 }}
spec: spec:
accessModes: ["ReadWriteOnce"] accessModes: ["ReadWriteOnce"]
storageClassName: {{ .Values.storageclass.name }} storageClassName: {{ $pool.storageclass.name }}
resources: resources:
requests: requests:
storage: {{ .Values.storageclass.logStorageSize }} storage: {{ $pool.storageclass.logStorageSize }}
{{- end }} {{- end }}
{{- if eq (int .Values.replicaCount) 4 }} {{- if eq (int $pool.replicaCount) 4 }}
{{- range $i := until (int .Values.replicaCount) }} {{- range $i := until (int $pool.replicaCount) }}
- apiVersion: v1 - apiVersion: v1
kind: PersistentVolumeClaim kind: PersistentVolumeClaim
metadata: metadata:
@@ -227,28 +264,29 @@ spec:
labels: labels:
{{- toYaml $.Values.commonLabels | nindent 10 }} {{- toYaml $.Values.commonLabels | nindent 10 }}
annotations: annotations:
{{- toYaml $.Values.storageclass.pvcAnnotations.data | nindent 10 }} {{- toYaml $pool.storageclass.pvcAnnotations.data | nindent 10 }}
spec: spec:
accessModes: ["ReadWriteOnce"] accessModes: ["ReadWriteOnce"]
storageClassName: {{ $.Values.storageclass.name }} storageClassName: {{ $pool.storageclass.name }}
resources: resources:
requests: requests:
storage: {{ $.Values.storageclass.dataStorageSize }} storage: {{ $pool.storageclass.dataStorageSize }}
{{- end }} {{- end }}
{{- else if eq (int .Values.replicaCount) 16 }} {{- else if eq (int $pool.replicaCount) 16 }}
- apiVersion: v1 - apiVersion: v1
kind: PersistentVolumeClaim kind: PersistentVolumeClaim
metadata: metadata:
name: data name: data
labels: labels:
{{- toYaml .Values.commonLabels | nindent 10 }} {{- toYaml $.Values.commonLabels | nindent 10 }}
annotations: annotations:
{{- toYaml .Values.storageclass.pvcAnnotations.data | nindent 10 }} {{- toYaml $pool.storageclass.pvcAnnotations.data | nindent 10 }}
spec: spec:
accessModes: ["ReadWriteOnce"] accessModes: ["ReadWriteOnce"]
storageClassName: {{ .Values.storageclass.name }} storageClassName: {{ $pool.storageclass.name }}
resources: resources:
requests: requests:
storage: {{ .Values.storageclass.dataStorageSize }} storage: {{ $pool.storageclass.dataStorageSize }}
{{- end }} {{- end }}
{{- end }} {{- end }}
{{- end }}
helm/rustfs/values.yaml
+27
View File
@@ -53,6 +53,33 @@ mode:
distributed: distributed:
enabled: true enabled: true
# Server pools for horizontal capacity expansion (distributed mode only).
# Disabled by default: the chart then behaves exactly like the classic
# single-pool layout driven by the top-level replicaCount/storageclass.
#
# With pools.enabled=true, one StatefulSet is rendered per entry of
# pools.list and RUSTFS_VOLUMES contains every pool's volume expression
# (space separated), which is how the server discovers multiple pools.
# Each entry may set replicaCount (4 or 16) and/or a storageclass block;
# omitted fields inherit the top-level values.
#
# IMPORTANT:
# - Pools are append-only. The list index determines the StatefulSet name
# (pool 0 keeps the legacy name so existing single-pool deployments can be
# expanded in place); never remove or reorder entries. To retire a pool,
# use `rc admin decommission` first.
# - After adding a pool, run `rc admin rebalance start <alias>` to spread
# existing data across the new capacity.
pools:
enabled: false
list: []
# Example: expand an existing 4-node deployment with a second, larger pool:
# list:
# - {} # pool 0: inherits top-level values, keeps existing PVCs
# - replicaCount: 4 # pool 1: new capacity
# storageclass:
# dataStorageSize: 10Gi
secret: secret:
existingSecret: "" existingSecret: ""
# SECURITY: rendering fails by default unless one of the following is true: # SECURITY: rendering fails by default unless one of the following is true: