Commit Graph

94 Commits

Author SHA1 Message Date
Charles GTE 334ab51125 fix: cron (#79)
* fix: cron log

---------

Co-authored-by: charles-gauthereau <charles.gauthereau@soluce-technologies.com>
2026-07-09 11:03:55 +02:00
Charles GTE ec7715ca26 fix: cron (#78)
* fix: docker-compose.yml

* fix: cron

---------

Co-authored-by: charles-gauthereau <charles.gauthereau@soluce-technologies.com>
2026-07-09 09:36:53 +02:00
Charles GTE 324a2ea3f2 fix: security (#75) 2026-07-04 12:24:37 +02:00
charles-gauthereau 608b82a254 fix: test docker_volume.rs 2026-07-02 22:08:55 +02:00
charles-gauthereau b2b2733c4d feat: docker-volume provider
refactor: move build_tar to utils::compress

refactor: move choose_restore_path to utils::common

docs: remove docker-volume README section

fix: keep archive path for multi-file non-docker-volume restores

docs: trim docker-volume README section to essentials

refactor: run docker-volume backup/restore inside spawn_blocking like other providers

docs: enable docker socket, add volume example and security notes

feat: sweep orphaned ephemeral helper containers on startup

feat: docker-volume clean-replace restore via upload_to_container

fix: serialize env-var access in docker-volume tests to avoid setenv/getenv UB

feat: docker-volume backup via download_from_container

feat: docker-volume ping via inspect_volume

fix: return extraction dir for multi-file restore archives

feat: gzip already-tar inputs directly instead of double-wrapping

feat: docker helper (client, self-image, container lifecycle, sweep)

feat: docker-volume provider
2026-07-02 21:45:11 +02:00
charles-gauthereau 34f353b68f fix: postgres keep_ownership flag 2026-06-29 19:10:49 +02:00
charles-gauthereau 694b463787 feat: conditionally omit --no-owner/--no-privileges in pg_restore based on keep_ownership option 2026-06-29 18:52:37 +02:00
charles-gauthereau aed1e86956 feat: add generic options map to DatabaseConfig
Add `options: HashMap<String, serde_json::Value>` to DatabaseConfig and
`options: Option<HashMap<...>>` to InputDatabaseConfig so callers can
pass arbitrary per-database options (e.g. keep_ownership for pg_dumpall).
Update all DatabaseConfig struct literals in test files and add three
new config-service unit tests covering present, absent, and non-bool values.
2026-06-29 18:49:01 +02:00
charles-gauthereau 6388aff1e3 fix: postgres cluster dump command 2026-06-29 17:10:11 +02:00
charles-gauthereau 4be54a614e fix: postgres cluster dump command 2026-06-29 15:18:52 +02:00
Charles GTE 16a37e033e fix: azure (#71) 2026-06-27 20:51:47 +02:00
Charles GTE 29ca5a317b feat: add-gcs-storage-provider (#70)
* fix: refactoring

* chore: add google-cloud-storage and google-cloud-auth deps

* feat: add GCS provider config model

* feat: add GCS credential, client, and stream-source helpers

StreamSource bridges build_stream's Send-only byte stream into the SDK's
StreamingSource (which send_buffered requires to be Send+Sync+'static) via a
bounded mpsc channel, avoiding any change to the shared UploadStream type.

* feat: implement GCS StorageProvider upload

* feat: register google-cloud-storage provider in factory

* test: GCS upload roundtrip against fake-gcs-server

* fix: format GCS bucket as projects/_/buckets/<name> in upload_with_client

write_object rejects the bare bucket id with "malformed bucket name"; the
production provider passed config.bucket_name unformatted, so real uploads
would always fail. Format once in upload_with_client so prod and the
fake-gcs-server test share the corrected path.

* style: cargo fmt GCS provider files

* fix: gcs
2026-06-27 20:37:22 +02:00
Charles GTE 01f5d34c96 fix: pgdump-cluster (#68)
* feat: add as_str/from_str to PostgresDumpFormat

* feat: resolve pg_dumpall/psql binary names

* feat: add include_globals field to database config

* feat: add pg_dumpall/psql globals dump and apply

* feat: add postgres backup bundle (manifest + build + resolve)

* feat: bundle globals into postgres backup when include_globals is set

* feat: replay globals before pg_restore when backup archive is a bundle

* refactor: bind FD restore tempdir guard once to clear unused warnings

* docs: demonstrate include_globals in sample databases.json

* chore: silence test-only re-export warning in non-test builds

* revert: remove include_globals feature, restore plain pg_dump/pg_restore

* feat: add pg_dumpall/psql binary names and is_superuser check

* feat: add postgresql-cluster db type and config parsing

* feat: pg_dumpall cluster backup and psql restore

* feat: route postgresql-cluster through PostgresClusterDatabase

* docs: add postgresql-cluster sample to databases.json

* refactor: split cluster mode into cluster/ module (backup, restore, database)

* test: mirror cluster tests into src/tests/domain/cluster/
2026-06-27 17:23:57 +02:00
Charles GTE 038c5523b7 fix: download-crash-large-files (#67)
* fix(restore): add ProgressTracker for download progress logging

* fix(restore): stream backup download to disk instead of buffering in RAM

* chore(restore): add download diagnostics + exact byte logging

* refactor(restore): drop ProgressTracker, keep streaming download

* feat(restore): drive download progress from RestoreInfo.size

* fix(restore): accept integer or string for RestoreInfo.size

* fix(restore): show raw bytes in download progress log

* fix(restore): collapse progress to highest milestone per chunk

* fix: downloader.rs
2026-06-25 14:55:49 +02:00
Charles GTE 66cad4e12e fix: az blog storage tests 2026-06-21 16:22:51 +02:00
Charles GTE 221ed4e7e1 feat: add azure blob storage support 2026-06-20 19:38:00 +02:00
Charles GTE 84735ac399 feat(storage): spike Azure Blob SAS auth + block round-trip against Azurite
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 17:49:54 +02:00
Charles GTE 1fece577cc fix: tests and config.rs and backup.rs args 2026-06-20 14:36:09 +02:00
Charles GTE 273475fa3b Merge branch 'dev' into feature-backup-timeout-on-large-db 2026-06-20 13:02:14 +02:00
Charles GTE f02218708a fix: windows-release.yml 2026-06-20 12:52:53 +02:00
Charles GTE 16152328b0 fix: extract the env variable to the settings.rs and some refactoring, and the trigger in windows-release.yml 2026-06-20 12:31:01 +02:00
Peter Crosthwaite 7328827435 Fix mysqldump timeout on large database backups
- Add mysql:8.4 client tools stage to Dockerfile so the real mysqldump
  binary (with --network-timeout) takes PATH precedence over MariaDB's
  mysqldump symlink, which lacks this flag
- Use --network-timeout in MySQL backup command; sets net_read_timeout and
  net_write_timeout to 86400s for the duration of the dump
- Simplify config: replace net_read_timeout/net_write_timeout/max_allowed_packet
  with a single max_packet_size field (optional, defaults to 512M)
- Log mysqldump/mariadb-dump client version as a debug message before each backup
2026-06-20 18:06:21 +10:00
Peter Crosthwaite 348eaac81b Added config options for setting timeouts on mysql/mariadb 2026-06-20 07:51:18 +10:00
tf 0f6c93ecd0 fix encoding 2026-06-19 16:31:21 +02:00
tf 2052ab0ff5 fix(postgres): resolve pg_dump/pg_restore path cross-platform 2026-06-19 15:27:45 +02:00
charles-gauthereau f94656a39b fix: postgres password slash 2026-06-19 11:29:51 +02:00
Charles GTE 1453851555 Merge pull request #61 from Portabase/fix/cpu-ram-crash
fix: cpu-ram-crash
2026-06-07 20:04:25 +02:00
charles-gauthereau 82c7122790 fix: refactoring 2026-06-07 17:32:16 +02:00
Charles GTE fba72f5c26 feat: add job logs for restoration process 2026-06-07 11:16:02 +02:00
Charles GTE 0261a07fc2 fix: test ci 2026-06-06 20:59:30 +02:00
Charles GTE c840f91680 fix: test ci 2026-06-06 20:20:52 +02:00
Charles GTE 2d52eccbb2 fix: test ci 2026-06-06 19:43:54 +02:00
Charles GTE 3c853ee7ca fix: tests and refactoring 2026-06-06 19:15:22 +02:00
Charles GTE 45d791358c fix: tracing logs 2026-06-06 11:01:55 +02:00
Charles GTE 0b915d87c9 feat: job logs 2026-06-06 10:22:27 +02:00
Charles GTE eac2853b83 feat: add-mssql-support (#56)
* fix: tests and refactoring, add justfile instead of makefile

* chore: add tiberius dependency for MSSQL support

* feat: add Mssql variant to DbType enum and config validation

* feat: add MSSQL domain adapter with tiberius ping and sqlpackage backup/restore

* feat: install sqlpackage in Docker base and prod stages

* feat: add MSSQL service to docker-compose and config examples

* fix: log backup errors instead of silently discarding them

* fix: use dotnet-install.sh instead of Microsoft apt feed (SHA1 key rejected by Debian Trixie)

* fix: add /TrustServerCertificate:true to sqlpackage commands for self-signed certs

* fix: use connection string with TrustServerCertificate for sqlpackage

* fix: use user database in MSSQL config example (system databases cannot be exported)

* feat: add seed-mssql task and seed script

* fix: docker-compose.yml
2026-05-11 22:04:36 +02:00
Charles GTE 14068846b9 fix: migration-process (#55)
* fix: s3 port type mismatch, add string_or_number_to_string deserializer

* chore: docker-compose.databases.yml

* fix: tests and refactoring, add justfile instead of makefile

* fix: restoration process for postgres/mysql and mariadb

* fix: restoration process for postgres/mysql and mariadb

* fix: postgres restore

* fix: mariadb/mysql restore, added backticks to drop command

* fix: mongodb and migration test for firebird

* fix: docker-compose.yml

---------

Co-authored-by: charlesgauthereau <charles.gauthereau@soluce-technologies.com>
2026-04-29 22:40:02 +02:00
Charles GTE 14a6c9ab0a feat: add chunk size in .env variables (#54)
Co-authored-by: charles-gauthereau <charles.gauthereau@soluce-technologies.com>
2026-04-16 11:20:14 +02:00
Charles GTE f37e4d2de9 feat: add firebird (#51)
* feat: add firebird

* feat: add firebird tests

* fix: tests and refactoring, add justfile instead of makefile
2026-03-31 15:10:05 +02:00
Charles GTE d4abcbcebc feat: separate mysql and mariadb (#49)
* feat: separate mysql and mariadb

* feat: separate mysql and mariadb
2026-03-29 09:28:30 +02:00
Charles GTE 3d5a7eeded feat: add health check ping for databases 2026-03-28 16:07:37 +01:00
Charles GTE 517f3779cd feat: sending database ping result for each databases to portabase server 2026-03-27 23:07:06 +01:00
Charles GTE 725312e1d5 fix: tests (#47)
* fix: s3 port type mismatch, add string_or_number_to_string deserializer

* fix: adding some tests

* fix: adding some tests

* fix: working on pipeline

* fix: working on pipeline

* fix: working on pipeline

* fix: working on pipeline

* fix: working on pipeline

* fix: working on pipeline

* fix: working on pipeline

* fix: working on pipeline

* fix: working on pipeline

* fix: working on pipeline

* fix: working on pipeline

* fix: working on pipeline

* fix: working on pipeline

* fix: working on pipeline

* fix: working on pipeline

* fix: working on pipeline

* fix: working on pipeline

* fix: working on pipeline

* fix: working on pipeline

* fix: working on pipeline

* fix: working on pipeline

* fix: working on pipeline

---------

Co-authored-by: charlesgauthereau <charles.gauthereau@soluce-technologies.com>
2026-03-27 22:24:59 +01:00
Charles GTE 8489a6f6e3 fix: adding retry config for s3 provider (#46) 2026-03-21 08:51:15 +01:00
charles-gauthereau e1bc77e481 fix: valkey.rs 2026-03-16 14:31:11 +01:00
charles-gauthereau fc4e4e3881 fix: log in backup.rs 2026-03-16 12:32:57 +01:00
charles-gauthereau 45c33dd031 feat: add valkey test sequences 2026-03-16 12:30:22 +01:00
charles-gauthereau cf5de3115c feat: add valkey database engine for backup 2026-03-16 12:11:01 +01:00
charles-gauthereau d6d2ef18f8 fix: internal redis port 2026-03-16 11:24:30 +01:00
Charles GTE e9147d6c60 fix: s3 port type mismatch, add string_or_number_to_string deserializer (#42)
Co-authored-by: charlesgauthereau <charles.gauthereau@soluce-technologies.com>
2026-03-16 08:57:00 +01:00