Commit Graph

  • 0f1c734626 fix mounting ec volumes chrislu 2025-08-11 20:34:02 -07:00
  • 2a61cda2cd adding retry button chrislu 2025-08-11 18:28:02 -07:00
  • 5c93557314 detect shard sources chrislu 2025-08-11 17:36:27 -07:00
  • 7cf5ddf8ac more self contained tasks chrislu 2025-08-11 17:16:16 -07:00
  • 7622d14580 remove unused config chrislu 2025-08-11 17:08:48 -07:00
  • bddb2cd5cc keep only ec config chrislu 2025-08-11 17:02:15 -07:00
  • 96a2749355 adding EcVacuumTaskConfig chrislu 2025-08-11 16:59:55 -07:00
  • f9e7ca4308 Merge branch 'master' into add-ec-vacuum chrislu 2025-08-11 16:45:45 -07:00
  • 7889e78f4d use one http client chrislu 2025-08-11 16:44:51 -07:00
  • af3300e063 filer: server side copying (#7121) Chris Lu 2025-08-11 16:40:46 -07:00
  • 7e86045e22 chore(deps): bump golang.org/x/net from 0.42.0 to 0.43.0 (#7126) dependabot[bot] 2025-08-11 15:21:46 -07:00
  • 529c68a798 chore(deps): bump google.golang.org/api from 0.244.0 to 0.246.0 (#7127) dependabot[bot] 2025-08-11 15:06:25 -07:00
  • e3b15073a0 chore(deps): bump actions/checkout from 4 to 5 (#7125) dependabot[bot] 2025-08-11 15:06:14 -07:00
  • 5f62b11889 chore(deps): bump golang.org/x/sys from 0.34.0 to 0.35.0 (#7124) dependabot[bot] 2025-08-11 15:06:01 -07:00
  • 524c1916b6 chore(deps): bump github.com/redis/go-redis/v9 from 9.11.0 to 9.12.0 (#7123) dependabot[bot] 2025-08-11 15:05:53 -07:00
  • 412fd6ee86 chore(deps): bump github.com/ydb-platform/ydb-go-sdk/v3 from 3.113.4 to 3.113.5 (#7122) dependabot[bot] 2025-08-11 15:05:44 -07:00
  • 605b3333c1 fix sftp start with filer (#7120) Chris Lu 2025-08-11 10:56:40 -07:00
  • 9265e81fe9 S3 API: unsigned streaming (no cred) but chunks contain signatures (#7118) Chris Lu 2025-08-11 10:31:01 -07:00
  • a265a07922 fix ttl objects chrislu 2025-08-11 02:20:02 -07:00
  • f03f4383af fix missing grpc dial option chrislu 2025-08-11 02:19:22 -07:00
  • c57ccd554a worker_pb.TaskParams_VacuumParams chrislu 2025-08-11 02:11:39 -07:00
  • 9df006b49d fix min volume age chrislu 2025-08-11 01:54:10 -07:00
  • 424bb7fe11 showing criteria chrislu 2025-08-11 01:34:33 -07:00
  • 97d58e77c6 getting ec volume deletions chrislu 2025-08-11 01:03:21 -07:00
  • 553a229fd3 reduce lock scope chrislu 2025-08-11 00:21:04 -07:00
  • 57d025910d cancel context chrislu 2025-08-11 00:20:52 -07:00
  • 213dd9b470 testing chrislu 2025-08-11 00:20:37 -07:00
  • a31fc1ac01 Bypass compaction revision check to handle volumes compacted after task creation chrislu 2025-08-10 20:39:48 -07:00
  • 91d641e685 avoid dead lock chrislu 2025-08-10 20:23:18 -07:00
  • 740983d69e Delete ec_test_files.json chrislu 2025-08-10 20:15:36 -07:00
  • 0e649f710a collect deletion for ec shards chrislu 2025-08-10 20:08:45 -07:00
  • 772ee0f967 address comments chrislu 2025-08-10 19:58:52 -07:00
  • 0c6980182c ec shards with generation chrislu 2025-08-10 19:40:35 -07:00
  • 47ea1ac228 unmount chrislu 2025-08-10 19:39:24 -07:00
  • 6446893e3c metrics chrislu 2025-08-10 19:39:11 -07:00
  • 06c012ea60 Update maintenance_scanner.go chrislu 2025-08-10 19:37:54 -07:00
  • 5a6954be1b sort chrislu 2025-08-10 18:42:27 -07:00
  • 0f1ca16457 more accurate estimation chrislu 2025-08-10 18:42:22 -07:00
  • 72f0a47563 CRITICAL: Check ALL task states for volume conflicts chrislu 2025-08-10 18:04:31 -07:00
  • 751cfac7d7 Implement volume-aware task conflict checking chrislu 2025-08-10 18:02:42 -07:00
  • 5c1e6e904d CRITICAL: Restore task conflict definitions to prevent data integrity issues chrislu 2025-08-10 17:56:13 -07:00
  • 04e3a68b61 Replace deprecated ioutil with os package functions chrislu 2025-08-10 17:53:48 -07:00
  • c220ad1e69 Replace bubble sort with idiomatic sort.Slice in EC shard management chrislu 2025-08-10 17:52:30 -07:00
  • c9ca213d4c Fix linter error in master_grpc_ec_generation_test.go chrislu 2025-08-10 17:51:28 -07:00
  • 4ec743583d Address PR #7116 review comments chrislu 2025-08-10 17:50:10 -07:00
  • 5bb475c572 Fixed CodeQL Security Issue chrislu 2025-08-10 17:48:29 -07:00
  • cd75202da8 Replaced goto with Idiomatic Map Approach chrislu 2025-08-10 17:48:18 -07:00
  • cdba261913 add safety check chrislu 2025-08-10 17:41:55 -07:00
  • 179bcb777e Delete ec_vacuum_integration_test.go chrislu 2025-08-10 17:34:03 -07:00
  • 802267b281 validates all critical upgrade scenarios for the generation-aware EC vacuum system chrislu 2025-08-10 17:33:08 -07:00
  • d6e335d6e2 Integration test: end-to-end vacuum G→G+1 with live reads (zero downtime) chrislu 2025-08-10 17:23:17 -07:00
  • 884da0496c testing mixed generation chrislu 2025-08-10 17:12:20 -07:00
  • 60f975aa20 add unit tests chrislu 2025-08-10 17:07:29 -07:00
  • a3d0509c92 logs chrislu 2025-08-10 16:57:08 -07:00
  • 2e51e1dab2 ec volume UI rendering version chrislu 2025-08-10 16:54:44 -07:00
  • a187f103d1 normal volume CompactionRevision chrislu 2025-08-10 16:22:36 -07:00
  • 3087da07db metrics with generation chrislu 2025-08-10 16:17:46 -07:00
  • 3ef8a9f3b2 Mixed-version cluster compatibility chrislu 2025-08-10 15:54:30 -07:00
  • d326affc4f default generation chrislu 2025-08-10 15:44:08 -07:00
  • d341db8861 Update ec_vacuum_task.go chrislu 2025-08-10 15:39:26 -07:00
  • cf5f0b8e74 cleanupGracePeriod chrislu 2025-08-10 15:13:12 -07:00
  • 0bcb64140f Add activation step to EC vacuum: call ActivateEcGeneration when G+1 ready chrislu 2025-08-10 15:07:57 -07:00
  • 62d89fa60b EC vacuum distribution updated for generation-aware mount/copy RPCs chrislu 2025-08-10 15:04:55 -07:00
  • de9399761b EC vacuum task updated for generation-aware operation chrislu 2025-08-10 15:00:22 -07:00
  • 8797e73523 cachedLookupEcShardLocations updated for generation-specific caching chrislu 2025-08-10 14:51:55 -07:00
  • f00dc46607 VolumeEcShardRead to read from correct (vid, generation) EcVolume chrislu 2025-08-10 14:44:27 -07:00
  • 1b41544f97 detecting ec volumes chrislu 2025-08-10 14:41:06 -07:00
  • e9033136aa todo chrislu 2025-08-10 14:24:59 -07:00
  • 56d1ed77c1 copy file chrislu 2025-08-10 14:24:30 -07:00
  • 9e2e600b6d VolumeEcShardsGenerate updated for generation-specific file creation chrislu 2025-08-10 14:14:37 -07:00
  • 99f132729c MountEcShards/UnmountEcShards updated for generation support chrislu 2025-08-10 14:09:07 -07:00
  • 8c31d5e331 EcVolume creation properly refactored chrislu 2025-08-10 13:45:08 -07:00
  • ef5f9f629a Generation file layout chrislu 2025-08-10 13:32:20 -07:00
  • 50cc17e8fa ActivateEcGeneration RPC implemented chrislu 2025-08-10 13:11:47 -07:00
  • 870b2ffe07 LookupEcVolume handle generation parameter chrislu 2025-08-10 13:06:37 -07:00
  • e4f266d927 Active generation tracking implemented chrislu 2025-08-10 13:03:52 -07:00
  • 5650b6f505 Add master_pb.ActivateEcGenerationRequest/Response RPC definition chrislu 2025-08-10 12:53:10 -07:00
  • 09d740db47 Add generation field to volume_server_pb.VolumeEcShardReadRequest chrislu 2025-08-10 12:51:27 -07:00
  • 8c54b72af3 Add generation field to volume_server_pb VolumeEcShards{Copy,Generate,Rebuild,ToVolume}Request chrislu 2025-08-10 12:51:01 -07:00
  • 312fee4ddf Add generation field to volume_server_pb VolumeEcShards{Mount,Unmount}Request chrislu 2025-08-10 12:49:43 -07:00
  • 03ad8f2490 Add generation field to master_pb.VolumeEcShardInformationMessage chrislu 2025-08-10 12:47:47 -07:00
  • fc666e2e48 collect ec volume deleted bytes chrislu 2025-08-10 12:37:18 -07:00
  • 3a9ec6f30c Update worker.go chrislu 2025-08-10 02:16:23 -07:00
  • 8603a3a1e9 ec vacuum task need to register itself chrislu 2025-08-10 02:16:15 -07:00
  • bea9ca3616 generate ec data and deleted data chrislu 2025-08-10 02:15:55 -07:00
  • 32e7ba2f9d adding ec vacuum chrislu 2025-08-10 00:58:26 -07:00
  • 05a0cc156b Self-Contained Design chrislu 2025-08-10 00:15:26 -07:00
  • 96d6d27607 remove ❌ Vacuum - Completely removed ❌ Balance - Completely removed chrislu 2025-08-10 00:00:46 -07:00
  • c7f291e4e4 simplify logging chrislu 2025-08-09 23:49:45 -07:00
  • 25bbf4c3d4 Admin UI: Fetch task logs (#7114) Chris Lu 2025-08-09 21:47:29 -07:00
  • 3ac2a2e22d fix tikv listing due to expired entries (#7115) Chris Lu 2025-08-08 12:38:55 -07:00
  • fae416586b Move helm templates into folders (#7113) Devin Lauderdale 2025-08-08 12:36:01 -05:00
  • 92cebe12f0 chore: remove default replica count for all-in-one deployment (#7111) Devin Lauderdale 2025-08-07 23:18:17 -05:00
  • 535985adb6 Shell: add verbose ec encoding mode (#7105) Chris Lu 2025-08-07 00:12:05 -07:00
  • cde2d65c16 ec candidate selection needs to adjust same rack count compare (#7106) Chris Lu 2025-08-07 00:09:51 -07:00
  • b4d9618efc volume server UI: fix ec volume ui (#7104) Chris Lu 2025-08-07 00:07:03 -07:00
  • dd4880d55a fix for baidu cloud storage chrislu 2025-08-06 20:53:03 -07:00
  • 4af182f880 Context cancellation during reading range reading large files (#7093) Chris Lu 2025-08-06 10:09:26 -07:00
  • e446234e9c remove spoof-able request header (#7103) Chris Lu 2025-08-06 10:08:30 -07:00
  • 0703308270 remote address parsing should handle special cases (#7101) Chris Lu 2025-08-06 01:03:00 -07:00