gsadmin/rustfs
1
0
Fork 0
mirror of https://github.com/rustfs/rustfs.git synced 2026-08-28 07:57:01 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(helm): flexible drivesPerNode topology with backward-compatible per-pool defaults (#4901)

Browse Source 
* helm chart: one data drive per node - fix1

* refactor(helm): prevent negative or 0 replicaCount

Co-authored-by: Copilot <copilot@github.com>

* refactor(helm): remove env REPLICA_COUNT

* feat(helm): default no logs directory to force stdout

Co-authored-by: Copilot <copilot@github.com>

* feat(helm): add drivesPerNode

* feat(helm): correct default parity with 4 nodes /  1 drive per node

* conditional render of RUSTFS_OBS_LOG_DIRECTORY

* feat(chart): add table doc for parity

* fix(chart): handle invalid annotation objects

* fix(chart): move logging and obsevability options together; default for kubernetes output to stdout

* feat(chart): better table doc for parity

* fix(chart): leave RUSTFS_OBS_LOG_DIRECTORY empty, or the defaults will attempt to write to readonly fs

* fix(chart): chart defaults as the previous version: 4 nodes with 4 drives per node

* feat(chart): render RUSTFS_STORAGE_CLASS_STANDARD in configmap

* minor fix for pvcAnnotations

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Cristian Chiru <cristi.chiru@gmail.com>

* fix(chart): better drivesPerNode handling

* fix(chart): obs_log_directory default non empty again to preserve previous deployments compatibility

* fix(chart): proper drivesPerNode impl

* fix(chart): values clarification for empty vs unset `obs_log_directory`

* feat(chart): add service externalIPs

* feat(helm): add optional service labels

* fix(helm): merge service labels

* fix(helm): storageclass pvcAnnotations

* fix(chart): move logging and obsevability options together; default for kubernetes output to stdout

* fix(helm): remove duplicate keys

* fix(helm): default drivesPerNode null, keep legacy chart behavior

* feat(helm): add template test for externalIPs

* fix(helm): per-pool drivesPerNode inference, restore regression tests

Repairs the drivesPerNode feature from #2693 so it renders and stays
upgrade-safe:

- define the missing drives inference (rustfs.poolDrives) and compute it
  per pool inside rustfs.pools, so mixed 4x4 + 16x1 pool deployments keep
  their exact legacy volumeClaimTemplates when drivesPerNode is unset
- restore the $poolsEnabled definition dropped in the rebase (chart failed
  to render at all)
- fix .Values references inside the pool range (dot is the pool there) and
  keep per-pool storageclass pvcAnnotations overrides working
- make rustfs.volumes derive the drive range from pool drives instead of
  the pod count, and relax the pools.list 4-or-16 restriction to >= 2
- reject drivesPerNode=0 explicitly instead of silently inferring
- keep default rendering identical to main: storage_class_standard stays
  unrendered by default, obs_endpoint.use_stdout stays false, clusterDomain
  value restored
- restore the clusterDomain/mTLS SAN/explicit-volumes regression tests that
  the branch deleted, keeping the new topology tests

---------

Signed-off-by: Cristian Chiru <cristi.chiru@gmail.com>
Co-authored-by: Cristian Chiru <cristi.chiru@gmail.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
This commit is contained in:
Zhengchao An
2026-07-16 17:02:04 +08:00
committed by GitHub
parent 8a126bb176
commit e2f394a897
8 changed files with 257 additions and 56 deletions
Download Patch File Download Diff File Expand all files Collapse all files
helm/rustfs/templates/_helpers.tpl
+47 -7
View File
@@ -42,6 +42,19 @@ app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end }}
{{/*
Render extra labels for the main Service resource.
Merges (in order of increasing precedence):
- commonLabels
- service.labels
*/}}
{{- define "rustfs.serviceLabels" -}}
{{- $labels := mergeOverwrite (dict) (default (dict) .Values.commonLabels) (default (dict) .Values.service.labels) }}
{{- if $labels }}
{{- toYaml $labels }}
{{- end }}
{{- end }}
{{/*
Selector labels
*/}}
@@ -185,6 +198,30 @@ Expects a dict with keys "root" (the chart root context) and "index".
{{- end -}}
{{- end }}
{{/*
Return the number of data drives (PVCs) per pod for a pool.
When .Values.drivesPerNode is set it applies to every pool. When unset the
value is inferred per pool from its replica count so that rendered output
stays identical to the pre-drivesPerNode chart: 4 replicas keep the legacy
4-drive layout, everything else (16x1 and any new topology) gets one drive.
Expects a dict with keys "root" (the chart root context) and "replicas".
*/}}
{{- define "rustfs.poolDrives" -}}
{{- if kindIs "invalid" .root.Values.drivesPerNode -}}
{{- if eq (int .replicas) 4 -}}
4
{{- else -}}
1
{{- end -}}
{{- else -}}
{{- $d := int .root.Values.drivesPerNode -}}
{{- if lt $d 1 -}}
{{- fail "drivesPerNode must be >= 1 when set" -}}
{{- end -}}
{{- $d -}}
{{- end -}}
{{- end }}
{{/*
Return the normalized list of server pools as JSON.
With pools disabled this is a single pool built from the top-level
@@ -206,14 +243,16 @@ so entries must never be removed or reordered.
{{- range $i, $p := .Values.pools.list -}}
{{- $p = default (dict) $p -}}
{{- $replicas := int (default $.Values.replicaCount $p.replicaCount) -}}
{{- if and (ne $replicas 4) (ne $replicas 16) -}}
{{- fail (printf "pools.list[%d].replicaCount must be 4 or 16, got %d" $i $replicas) -}}
{{- if lt $replicas 2 -}}
{{- fail (printf "pools.list[%d].replicaCount must be >= 2, got %d" $i $replicas) -}}
{{- end -}}
{{- $sc := mergeOverwrite (deepCopy $.Values.storageclass) (default (dict) $p.storageclass) -}}
{{- $pools = append $pools (dict "index" $i "fullname" (include "rustfs.poolFullname" (dict "root" $ "index" $i)) "replicaCount" $replicas "storageclass" $sc) -}}
{{- $drives := int (include "rustfs.poolDrives" (dict "root" $ "replicas" $replicas)) -}}
{{- $pools = append $pools (dict "index" $i "fullname" (include "rustfs.poolFullname" (dict "root" $ "index" $i)) "replicaCount" $replicas "drives" $drives "storageclass" $sc) -}}
{{- end -}}
{{- else -}}
{{- $pools = append $pools (dict "index" 0 "fullname" (include "rustfs.fullname" .) "replicaCount" (int .Values.replicaCount) "storageclass" .Values.storageclass) -}}
{{- $drives := int (include "rustfs.poolDrives" (dict "root" $ "replicas" (int .Values.replicaCount))) -}}
{{- $pools = append $pools (dict "index" 0 "fullname" (include "rustfs.fullname" .) "replicaCount" (int .Values.replicaCount) "drives" $drives "storageclass" .Values.storageclass) -}}
{{- end -}}
{{- toJson $pools -}}
{{- end }}
@@ -235,9 +274,10 @@ RUSTFS_VOLUMES on spaces, one pool per expression).
{{- $exprs := list -}}
{{- range $pool := include "rustfs.pools" . | fromJsonArray -}}
{{- $n := int $pool.replicaCount -}}
{{- if eq $n 4 -}}
{{- $exprs = append $exprs (printf "%s://%s-{0...%d}.%s.%s.svc.%s:%d/data/rustfs{0...%d}" $protocol $pool.fullname (sub $n 1) $headless $ns $domain $port (sub $n 1)) -}}
{{- else if eq $n 16 -}}
{{- $d := int $pool.drives -}}
{{- if gt $d 1 -}}
{{- $exprs = append $exprs (printf "%s://%s-{0...%d}.%s.%s.svc.%s:%d/data/rustfs{0...%d}" $protocol $pool.fullname (sub $n 1) $headless $ns $domain $port (sub $d 1)) -}}
{{- else -}}
{{- $exprs = append $exprs (printf "%s://%s-{0...%d}.%s.%s.svc.%s:%d/data" $protocol $pool.fullname (sub $n 1) $headless $ns $domain $port) -}}
{{- end -}}
{{- end -}}
helm/rustfs/templates/configmap.yaml
+3
View File
@@ -19,6 +19,9 @@ data:
{{- if .domains }}
RUSTFS_SERVER_DOMAINS: {{ include "rustfs.serverDomains" $ | quote }}
{{- end }}
{{- if .ec.storage_class_standard }}
RUSTFS_STORAGE_CLASS_STANDARD: {{ .ec.storage_class_standard | quote }}
{{- end }}
{{- with .log_rotation }}
{{- if .size }}
RUSTFS_OBS_LOG_ROTATION_SIZE_MB: {{ .size | quote }}
helm/rustfs/templates/deployment.yaml
+1 -1
View File
@@ -131,7 +131,7 @@ spec:
subPath: ca.crt
- name: client-cert
mountPath: /opt/tls/client_cert.pem
subPath: client_cert.pem
subPath: client_cert.pem
- name: client-cert
mountPath: /opt/tls/client_key.pem
subPath: client_key.pem
helm/rustfs/templates/service.yaml
+6 -2
View File
@@ -39,8 +39,8 @@ metadata:
{{- end }}
labels:
{{- include "rustfs.labels" . | nindent 4 }}
{{- with .Values.commonLabels }}
{{- toYaml . | nindent 4 }}
{{- with (include "rustfs.serviceLabels" .) }}
{{- . | nindent 4 }}
{{- end }}
spec:
{{- if eq $serviceType "ClusterIP" }}
@@ -64,6 +64,10 @@ spec:
{{- toYaml . | nindent 4 }}
{{- end }}
{{- end }}
{{- with .Values.service.externalIPs }}
externalIPs:
{{- toYaml . | nindent 4 }}
{{- end }}
ports:
- name: endpoint
port: {{ .Values.service.endpoint.port }}
helm/rustfs/templates/statefulset.yaml
+31 -18
View File
@@ -1,9 +1,13 @@
{{- $logDir := .Values.config.rustfs.obs_log_directory }}
{{- $logDirEnabled := ne $logDir "" }}
{{- $poolsEnabled := and .Values.pools .Values.pools.enabled }}
{{- if and .Values.mode.distributed.enabled (le (int .Values.replicaCount) 1) -}}
{{- fail "Distributed mode requires replicaCount >= 2" -}}
{{- end -}}
{{- if .Values.mode.distributed.enabled }}
{{- range $pool := include "rustfs.pools" . | fromJsonArray }}
{{- $drivesPerNode := int $pool.drives }}
---
apiVersion: apps/v1
kind: StatefulSet
@@ -117,17 +121,17 @@ spec:
securityContext:
{{- toYaml $.Values.containerSecurityContext | nindent 12 }}
env:
- name: REPLICA_COUNT
value: {{ $pool.replicaCount | quote }}
- name: DRIVES_PER_NODE
value: {{ $drivesPerNode | quote }}
command:
- sh
- -c
- |
if [ "$REPLICA_COUNT" -eq 4 ]; then
for i in $(seq 0 $(($REPLICA_COUNT - 1))); do
if [ "$DRIVES_PER_NODE" -gt 1 ]; then
for i in $(seq 0 $(($DRIVES_PER_NODE - 1))); do
mkdir -p /data/rustfs$i
done;
elif [ "$REPLICA_COUNT" -eq 16 ]; then
done
else
mkdir -p /data
fi
{{- if $logDirEnabled }}
@@ -135,12 +139,12 @@ spec:
chmod 755 /mnt/rustfs/logs
{{- end }}
volumeMounts:
{{- if eq (int $pool.replicaCount) 4 }}
{{- range $i := until (int $pool.replicaCount) }}
{{- if gt $drivesPerNode 1 }}
{{- range $i := until $drivesPerNode }}
- name: data-rustfs-{{ $i }}
mountPath: /data/rustfs{{ $i }}
{{- end }}
{{- else if eq (int $pool.replicaCount) 16 }}
{{- else }}
- name: data
mountPath: /data
{{- end }}
@@ -201,12 +205,12 @@ spec:
mountPath: {{ $logDir }}
subPath: logs
{{- end }}
{{- if eq (int $pool.replicaCount) 4 }}
{{- range $i := until (int $pool.replicaCount) }}
{{- if gt $drivesPerNode 1 }}
{{- range $i := until $drivesPerNode }}
- name: data-rustfs-{{ $i }}
mountPath: /data/rustfs{{ $i }}
{{- end }}
{{- else if eq (int $pool.replicaCount) 16 }}
{{- else }}
- name: data
mountPath: /data
{{- end }}
@@ -246,8 +250,11 @@ spec:
name: logs
labels:
{{- toYaml $.Values.commonLabels | nindent 10 }}
{{- $logAnn := (default (dict) $pool.storageclass.pvcAnnotations).logs | default (dict) -}}
{{- if gt (len $logAnn) 0 }}
annotations:
{{- toYaml $pool.storageclass.pvcAnnotations.logs | nindent 10 }}
{{- toYaml $logAnn | nindent 10 }}
{{- end }}
spec:
accessModes: ["ReadWriteOnce"]
storageClassName: {{ $pool.storageclass.name }}
@@ -255,16 +262,19 @@ spec:
requests:
storage: {{ $pool.storageclass.logStorageSize }}
{{- end }}
{{- if eq (int $pool.replicaCount) 4 }}
{{- range $i := until (int $pool.replicaCount) }}
{{- if gt $drivesPerNode 1 }}
{{- range $i := until $drivesPerNode }}
- apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: data-rustfs-{{ $i }}
labels:
{{- toYaml $.Values.commonLabels | nindent 10 }}
{{- $dataAnn := (default (dict) $pool.storageclass.pvcAnnotations).data | default (dict) -}}
{{- if gt (len $dataAnn) 0 }}
annotations:
{{- toYaml $pool.storageclass.pvcAnnotations.data | nindent 10 }}
{{- toYaml $dataAnn | nindent 10 }}
{{- end }}
spec:
accessModes: ["ReadWriteOnce"]
storageClassName: {{ $pool.storageclass.name }}
@@ -272,15 +282,18 @@ spec:
requests:
storage: {{ $pool.storageclass.dataStorageSize }}
{{- end }}
{{- else if eq (int $pool.replicaCount) 16 }}
{{- else }}
- apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: data
labels:
{{- toYaml $.Values.commonLabels | nindent 10 }}
{{- $dataAnn := (default (dict) $pool.storageclass.pvcAnnotations).data | default (dict) -}}
{{- if gt (len $dataAnn) 0 }}
annotations:
{{- toYaml $pool.storageclass.pvcAnnotations.data | nindent 10 }}
{{- toYaml $dataAnn | nindent 10 }}
{{- end }}
spec:
accessModes: ["ReadWriteOnce"]
storageClassName: {{ $pool.storageclass.name }}