Compare commits

...

16 Commits

Author SHA1 Message Date
Florent Chehab 53a139fe58 📝(changelog) document summary privacy improvements
Record signed URL protection, secret-aware validation errors and safer
logging and Sentry capture under Unreleased.

Document the retention of redacted frame locals and application DEBUG
diagnostics.
2026-10-27 14:39:05 +01:00
Florent Chehab e1cc318d07 🔒️(summary) scrub Sentry events and breadcrumbs
Redact signed URLs and remove untrusted exception messages, request
bodies and breadcrumb payloads from Sentry events and transactions.

Preserve stack-frame locals after the existing content scrubber runs.
Cover real HTTP failures, chained exceptions and DEBUG breadcrumbs.
2026-10-27 14:39:05 +01:00
Florent Chehab fdc9f6ec9e 🔒️(summary) hide sensitive Celery task details
Publish redacted task argument representations and filter task results
and exception details from INFO-and-above Celery trace logs.

Keep task identity and DEBUG diagnostics available.
2026-10-27 14:39:05 +01:00
Florent Chehab 8b06e0d2f9 🔒️(summary) sanitize analytics client failures
Avoid chaining raw analytics client errors or logging feature-flag
exception messages. Preserve event metadata and feature-flag fallback
behavior.
2026-10-27 14:39:05 +01:00
Florent Chehab 84459eac35 🔒️(summary) sanitize LLM failures and summary progress logs
Raise safe LLM errors for upstream failures and invalid structured
responses without retaining the original exception chain.

Keep generated section titles out of INFO progress logs and preserve
the existing DEBUG diagnostics.
2026-10-27 14:39:05 +01:00
Florent Chehab 20469913ab 🔒️(summary) sanitize upstream HTTP failures
Replace raw download, transcription, webhook and Docs exceptions with
safe messages while preserving retryable exception categories.

Retain document IDs and DEBUG responses. Give webhook regression tests
their own tenant settings instead of relying on development tenants.
2026-10-27 14:29:04 +01:00
Florent Chehab 4ce6c96085 🔒️(summary) clean validation errors at application boundaries
Use cleaned validation errors for task execution, prerun signals,
transcription responses and API results. Share the same cleanup with
FastAPI validation responses while retaining ordinary input values.

Verify secret masking, preserved diagnostics and summary retries.
2026-10-27 14:29:04 +01:00
Florent Chehab 59aecb550f 🔒️(summary) protect signed URL model fields with SecretStr
Validate signed URLs as secrets and reveal them explicitly for file
access, Celery messages, webhook delivery and authorized API responses.

Cover model representations, nested broker payloads and URL consumers.
2026-10-27 14:29:04 +01:00
Florent Chehab a2c13cec46 🔒️(summary) add secret-aware validation errors
Preserve Pydantic error categories and diagnostics while masking nested
SecretStr inputs and avoiding retained validator exception objects.

Support custom error codes and safe pickling for Celery.
2026-10-27 14:29:04 +01:00
Florent Chehab 2a9c99003c 🔒️(summary) redact signed URLs in source logs and analytics
Strip URL credentials, query strings and fragments from file-service
logs and analytics source metadata using one shared helper.

Cover malformed URLs, non-string inputs and signed source URLs.
2026-10-27 13:54:32 +01:00
Florent Chehab 815baa73e8 ⬆️(summary) update ffmpeg to 8.1.3-r0
8.1.2 is not available anymore and the image build fails.
2026-10-26 13:28:56 +01:00
briquet 85670a02d5 🔖(patch) bump release to 1.36.3 2026-10-23 21:48:52 +02:00
briquet 21121e8bc1 🔒️(backend) upgrade djangorestframework to 3.18.2
Fix three vulnerabilities in djangorestframework

- CVE-2026-114136: a compressing codec in the Content-Type charset
  lets an unauthenticated body bypass Django's size limit,
  causing memory-amplification denial of service
- CVE-2026-114137: TokenProxy.delete() reports success without deleting
  the token
- CVE-2026-114138: crafted Accept headers make content negotiation burn
  seconds of CPU per request, exhausting workers
2026-10-23 21:46:44 +02:00
briquet 1717c65f2b 🔒️(frontend) upgrade nginx to 1.30.6 to fix CVE-2026-15176
Bump the nginx-unprivileged base image to 1.30.6-alpine3.24, which
fixes a heap overflow when using alias with rewrite, along with
CVE-2026-93479, CVE-2026-95101 and CVE-2026-97629.

https://nginx.org/en/security_advisories.html
2026-10-23 21:46:44 +02:00
briquet fa20d52248 🔒️(all) upgrade brotli to fix CVE-2026-111497
CVE-2026-111497 is an out-of-bounds write in the brotli decoder,
reachable by decompressing crafted brotli data.

- backend: upgrade Brotli to 1.2.1, used by urllib3 and aiohttp to
  decode "Content-Encoding: br" responses
- docker: require brotli-libs >= 1.2.1-r0 in the backend, summary and
  frontend images, the nginx base image ships 1.2.0-r1
- frontend: build the Outlook add-in on Node 22.23.4, Node 20 is EOL
  and will not get the fix

https://www.strix.ai/cve/CVE-2026-111497
2026-10-23 21:46:44 +02:00
briquet 75df71733a 🗃️(backend) add missing editable=False on client_secret_sha256
Migration 0025 declared client_secret_sha256 without editable=False,
while the model field is setting it.
2026-10-21 23:19:05 +02:00
39 changed files with 1404 additions and 137 deletions
+23 -1
View File
@@ -8,6 +8,28 @@ and this project adheres to
## [Unreleased]
### Changed
- ⬆️(summary) update ffmpeg to 8.1.3-r0
### Fixed
- 🔒️(summary) protect signed URLs in models, source logs and analytics metadata
- 🔒️(summary) mask secret validation inputs and sanitize upstream and task errors
- 🔒️(summary) scrub sensitive Sentry event and breadcrumb content while retaining
redacted frame locals and application DEBUG diagnostics
## [1.36.3] - 2026-10-23
### Fixed
- 🔒️(backend) upgrade brotli to 1.2.1 to fix CVE-2026-111497
- 🔒️(frontend) upgrade nginx to 1.30.6 to fix CVE-2026-15176
- 🔒️(docker) upgrade brotli-libs to 1.2.1 to fix CVE-2026-111497
- 🔒️(frontend) build the Outlook add-in on Node 22.23.4
- 🔒️(backend) upgrade djangorestframework to 3.18.2 to fix CVE-2026-114136,
CVE-2026-114137 and CVE-2026-114138
## [1.36.2] - 2026-10-21
### Fixed
@@ -167,7 +189,7 @@ and this project adheres to
### Fixed
- 🐛(helm) probe liveness on __lbheartbeat__ and readiness on __heartbeat__
- 🐛(helm) probe liveness on `__lbheartbeat__` and readiness on `__heartbeat__`
- 🐛(helm) render periodSeconds and failureThreshold on probes
- 🐛(backend) report the app release to Sentry instead of "NA"
- 🐛(frontend) play the waiting room notification sound on every arrival
+2 -1
View File
@@ -87,7 +87,8 @@ RUN apk --no-cache add \
libffi-dev \
pango \
libmagic \
shared-mime-info
shared-mime-info \
'brotli-libs>=1.2.1-r0'
# Copy entrypoint
+4 -3
View File
@@ -41,7 +41,7 @@ COPY ./docker/dinum-frontend/fonts/ \
./dist/assets/fonts/
# ---- Addons builder image ----
FROM node:20-alpine AS addons-builder
FROM node:22.23.4-alpine AS addons-builder
RUN apk upgrade --no-cache ca-certificates
@@ -58,12 +58,13 @@ RUN npx webpack --mode production
# ---- Front-end image ----
FROM nginxinc/nginx-unprivileged:1.30.5-alpine3.24 AS frontend-production
FROM nginxinc/nginx-unprivileged:1.30.6-alpine3.24 AS frontend-production
USER root
RUN apk upgrade --no-cache libexpat && \
apk add --no-cache --upgrade 'pcre2>=10.49-r0' 'tiff>=4.7.2-r0' \
'libx11>=1.8.14-r0' 'ca-certificates>=20261020-r0' && \
'libx11>=1.8.14-r0' 'ca-certificates>=20261020-r0' \
'brotli-libs>=1.2.1-r0' && \
apk del curl
USER nginx
+1 -1
View File
@@ -1,7 +1,7 @@
[project]
name = "agents"
version = "1.36.2"
version = "1.36.3"
requires-python = ">=3.12"
dependencies = [
"livekit-agents==1.7.0",
+1 -1
View File
@@ -9,7 +9,7 @@ resolution-markers = [
[[package]]
name = "agents"
version = "1.36.2"
version = "1.36.3"
source = { virtual = "." }
dependencies = [
{ name = "boto3" },
@@ -0,0 +1,18 @@
# Generated by Django 5.2.17 on 2026-10-21 20:49
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0028_room_deleted_at'),
]
operations = [
migrations.AlterField(
model_name='application',
name='client_secret_sha256',
field=models.CharField(blank=True, editable=False, max_length=255, null=True),
),
]
+3 -3
View File
@@ -7,7 +7,7 @@ build-backend = "uv_build"
[project]
name = "meet"
version = "1.36.2"
version = "1.36.3"
authors = [{ "name" = "DINUM", "email" = "dev@mail.numerique.gouv.fr" }]
classifiers = [
"Development Status :: 5 - Production/Stable",
@@ -25,7 +25,7 @@ license = "MIT"
requires-python = ">=3.13"
dependencies = [
"boto3==1.43.80",
"Brotli==1.2.0",
"Brotli==1.2.1",
"brevo-python==1.2.0",
"celery[redis]==5.6.3",
"dj-database-url==3.1.2",
@@ -41,7 +41,7 @@ dependencies = [
"django-timezone-field>=5.1",
"django-pydantic-field==0.5.4",
"django==5.2.17",
"djangorestframework==3.18.0",
"djangorestframework==3.18.2",
"drf_spectacular==0.30.0",
"dockerflow==2026.3.4",
"easy_thumbnails==2.10.1",
+47 -28
View File
@@ -263,30 +263,49 @@ sdist = { url = "https://files.pythonhosted.org/packages/a8/13/9c5fcc8be41af6827
[[package]]
name = "brotli"
version = "1.2.0"
version = "1.2.1"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/f7/16/c92ca344d646e71a43b8bb353f0a6490d7f6e06210f8554c8f874e454285/brotli-1.2.0.tar.gz", hash = "sha256:e310f77e41941c13340a95976fe66a8a95b01e783d430eeaf7a2f87e0a57dd0a", size = 7388632, upload-time = "2025-11-05T18:39:42.86Z" }
sdist = { url = "https://files.pythonhosted.org/packages/cf/81/1546c983678a9b4e77cb4718eaedd1b2caf24a6cdbf1eb8775938d22aff7/brotli-1.2.1.tar.gz", hash = "sha256:02b2f7b5a83485efe448a44ddf7983def05c3bb1dcf917d4f7154070260e13de", size = 7389447, upload-time = "2026-10-22T14:00:12.66Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/6c/d4/4ad5432ac98c73096159d9ce7ffeb82d151c2ac84adcc6168e476bb54674/brotli-1.2.0-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:9e5825ba2c9998375530504578fd4d5d1059d09621a02065d1b6bfc41a8e05ab", size = 861523, upload-time = "2025-11-05T18:38:34.67Z" },
{ url = "https://files.pythonhosted.org/packages/91/9f/9cc5bd03ee68a85dc4bc89114f7067c056a3c14b3d95f171918c088bf88d/brotli-1.2.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:0cf8c3b8ba93d496b2fae778039e2f5ecc7cff99df84df337ca31d8f2252896c", size = 444289, upload-time = "2025-11-05T18:38:35.6Z" },
{ url = "https://files.pythonhosted.org/packages/2e/b6/fe84227c56a865d16a6614e2c4722864b380cb14b13f3e6bef441e73a85a/brotli-1.2.0-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:c8565e3cdc1808b1a34714b553b262c5de5fbda202285782173ec137fd13709f", size = 1528076, upload-time = "2025-11-05T18:38:36.639Z" },
{ url = "https://files.pythonhosted.org/packages/55/de/de4ae0aaca06c790371cf6e7ee93a024f6b4bb0568727da8c3de112e726c/brotli-1.2.0-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:26e8d3ecb0ee458a9804f47f21b74845cc823fd1bb19f02272be70774f56e2a6", size = 1626880, upload-time = "2025-11-05T18:38:37.623Z" },
{ url = "https://files.pythonhosted.org/packages/5f/16/a1b22cbea436642e071adcaf8d4b350a2ad02f5e0ad0da879a1be16188a0/brotli-1.2.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:67a91c5187e1eec76a61625c77a6c8c785650f5b576ca732bd33ef58b0dff49c", size = 1419737, upload-time = "2025-11-05T18:38:38.729Z" },
{ url = "https://files.pythonhosted.org/packages/46/63/c968a97cbb3bdbf7f974ef5a6ab467a2879b82afbc5ffb65b8acbb744f95/brotli-1.2.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:4ecdb3b6dc36e6d6e14d3a1bdc6c1057c8cbf80db04031d566eb6080ce283a48", size = 1484440, upload-time = "2025-11-05T18:38:39.916Z" },
{ url = "https://files.pythonhosted.org/packages/06/9d/102c67ea5c9fc171f423e8399e585dabea29b5bc79b05572891e70013cdd/brotli-1.2.0-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:3e1b35d56856f3ed326b140d3c6d9db91740f22e14b06e840fe4bb1923439a18", size = 1593313, upload-time = "2025-11-05T18:38:41.24Z" },
{ url = "https://files.pythonhosted.org/packages/9e/4a/9526d14fa6b87bc827ba1755a8440e214ff90de03095cacd78a64abe2b7d/brotli-1.2.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:54a50a9dad16b32136b2241ddea9e4df159b41247b2ce6aac0b3276a66a8f1e5", size = 1487945, upload-time = "2025-11-05T18:38:42.277Z" },
{ url = "https://files.pythonhosted.org/packages/5b/e8/3fe1ffed70cbef83c5236166acaed7bb9c766509b157854c80e2f766b38c/brotli-1.2.0-cp313-cp313-win32.whl", hash = "sha256:1b1d6a4efedd53671c793be6dd760fcf2107da3a52331ad9ea429edf0902f27a", size = 334368, upload-time = "2025-11-05T18:38:43.345Z" },
{ url = "https://files.pythonhosted.org/packages/ff/91/e739587be970a113b37b821eae8097aac5a48e5f0eca438c22e4c7dd8648/brotli-1.2.0-cp313-cp313-win_amd64.whl", hash = "sha256:b63daa43d82f0cdabf98dee215b375b4058cce72871fd07934f179885aad16e8", size = 369116, upload-time = "2025-11-05T18:38:44.609Z" },
{ url = "https://files.pythonhosted.org/packages/17/e1/298c2ddf786bb7347a1cd71d63a347a79e5712a7c0cba9e3c3458ebd976f/brotli-1.2.0-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:6c12dad5cd04530323e723787ff762bac749a7b256a5bece32b2243dd5c27b21", size = 863080, upload-time = "2025-11-05T18:38:45.503Z" },
{ url = "https://files.pythonhosted.org/packages/84/0c/aac98e286ba66868b2b3b50338ffbd85a35c7122e9531a73a37a29763d38/brotli-1.2.0-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:3219bd9e69868e57183316ee19c84e03e8f8b5a1d1f2667e1aa8c2f91cb061ac", size = 445453, upload-time = "2025-11-05T18:38:46.433Z" },
{ url = "https://files.pythonhosted.org/packages/ec/f1/0ca1f3f99ae300372635ab3fe2f7a79fa335fee3d874fa7f9e68575e0e62/brotli-1.2.0-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:963a08f3bebd8b75ac57661045402da15991468a621f014be54e50f53a58d19e", size = 1528168, upload-time = "2025-11-05T18:38:47.371Z" },
{ url = "https://files.pythonhosted.org/packages/d6/a6/2ebfc8f766d46df8d3e65b880a2e220732395e6d7dc312c1e1244b0f074a/brotli-1.2.0-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:9322b9f8656782414b37e6af884146869d46ab85158201d82bab9abbcb971dc7", size = 1627098, upload-time = "2025-11-05T18:38:48.385Z" },
{ url = "https://files.pythonhosted.org/packages/f3/2f/0976d5b097ff8a22163b10617f76b2557f15f0f39d6a0fe1f02b1a53e92b/brotli-1.2.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:cf9cba6f5b78a2071ec6fb1e7bd39acf35071d90a81231d67e92d637776a6a63", size = 1419861, upload-time = "2025-11-05T18:38:49.372Z" },
{ url = "https://files.pythonhosted.org/packages/9c/97/d76df7176a2ce7616ff94c1fb72d307c9a30d2189fe877f3dd99af00ea5a/brotli-1.2.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:7547369c4392b47d30a3467fe8c3330b4f2e0f7730e45e3103d7d636678a808b", size = 1484594, upload-time = "2025-11-05T18:38:50.655Z" },
{ url = "https://files.pythonhosted.org/packages/d3/93/14cf0b1216f43df5609f5b272050b0abd219e0b54ea80b47cef9867b45e7/brotli-1.2.0-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:fc1530af5c3c275b8524f2e24841cbe2599d74462455e9bae5109e9ff42e9361", size = 1593455, upload-time = "2025-11-05T18:38:51.624Z" },
{ url = "https://files.pythonhosted.org/packages/b3/73/3183c9e41ca755713bdf2cc1d0810df742c09484e2e1ddd693bee53877c1/brotli-1.2.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:d2d085ded05278d1c7f65560aae97b3160aeb2ea2c0b3e26204856beccb60888", size = 1488164, upload-time = "2025-11-05T18:38:53.079Z" },
{ url = "https://files.pythonhosted.org/packages/64/6a/0c78d8f3a582859236482fd9fa86a65a60328a00983006bcf6d83b7b2253/brotli-1.2.0-cp314-cp314-win32.whl", hash = "sha256:832c115a020e463c2f67664560449a7bea26b0c1fdd690352addad6d0a08714d", size = 339280, upload-time = "2025-11-05T18:38:54.02Z" },
{ url = "https://files.pythonhosted.org/packages/f5/10/56978295c14794b2c12007b07f3e41ba26acda9257457d7085b0bb3bb90c/brotli-1.2.0-cp314-cp314-win_amd64.whl", hash = "sha256:e7c0af964e0b4e3412a0ebf341ea26ec767fa0b4cf81abb5e897c9338b5ad6a3", size = 375639, upload-time = "2025-11-05T18:38:55.67Z" },
{ url = "https://files.pythonhosted.org/packages/f0/85/d74a573b92ba449960eb088ce87dcb61a9dc6a0c5a7ba8d104ae180e64c3/brotli-1.2.1-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:a84e0ae14d1efe782a9f143021c4e01d5ed986a140fe5070505cc5978eaf0592", size = 861657, upload-time = "2026-10-22T13:58:10.392Z" },
{ url = "https://files.pythonhosted.org/packages/ca/bd/848b55fbd96fa1ce9f534aa39c930f864dbf224282f4a3808c0d35ea2819/brotli-1.2.1-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:c5302747f969d5ec925810176f7697712a3fa2ea0fae0c7a85aa4ad9325247fd", size = 442252, upload-time = "2026-10-22T13:58:11.647Z" },
{ url = "https://files.pythonhosted.org/packages/40/41/95d5e6774cb037851001038484bcf98b3af203664fe78d29dd03ece317e7/brotli-1.2.1-cp313-cp313-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl", hash = "sha256:7ec678c52c3ef5e37482a094d109c04cf9e4a0e2923dd1e0fbddb4bf133a976e", size = 1242773, upload-time = "2026-10-22T13:58:13.182Z" },
{ url = "https://files.pythonhosted.org/packages/8b/ad/0942abad71d8b2aabd21472efa5be31ab0be31f98271d58767b9797eec82/brotli-1.2.1-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:b5b458950312830913f837e179ce9221000ab7a20b4e42d7b48c1fcd79140de1", size = 1402440, upload-time = "2026-10-22T13:58:14.513Z" },
{ url = "https://files.pythonhosted.org/packages/a3/53/2f7e666e61a1419d527904b6018a711f10650a851e7767358abf2919625e/brotli-1.2.1-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:f37ccd159dae35d4914651d57142c8077aa67ab7e3ffef690b00bcb7a7dbd8e6", size = 1424432, upload-time = "2026-10-22T13:58:15.789Z" },
{ url = "https://files.pythonhosted.org/packages/65/85/4c57c3cd2eee42878677b4943086e6b36cefc50214a5447028268045ce63/brotli-1.2.1-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:264fd5b200f6f6170cb551a4b9fcaabf8aa846ddaefe72db4d5f49f9d3743df9", size = 1420480, upload-time = "2026-10-22T13:58:17.007Z" },
{ url = "https://files.pythonhosted.org/packages/8f/f7/65e75560e20d522db1f0c75a1b2612c6ebd542ff9df17b8920cdbe4cde68/brotli-1.2.1-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:1329c80daa60752f1d94f507479504921e358e71746c5fce9b55b3d51ac038bc", size = 1485998, upload-time = "2026-10-22T13:58:18.422Z" },
{ url = "https://files.pythonhosted.org/packages/5d/2a/6cc0cfc4170bd72f6fc0aef11984e7c583ecf0a7fec563bb4292c0d50e15/brotli-1.2.1-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:77ef5882c5318ffc3dd94c6abc6a8f1ec2af75d66522319f1a250a07d53d0546", size = 1279199, upload-time = "2026-10-22T13:58:19.709Z" },
{ url = "https://files.pythonhosted.org/packages/66/36/97f3eefc9295ce8d87b568433d38e0ca3345b82571341196817c96b2ca50/brotli-1.2.1-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:6bd29ab84bb8972370cacd310e8a91651f54476d01de1cd1a4cf0bece5eb25e1", size = 1594118, upload-time = "2026-10-22T13:58:20.857Z" },
{ url = "https://files.pythonhosted.org/packages/03/61/687b36bd6b714270ff3d7ff96df6199336abc26c1fa98da276d9776d75b7/brotli-1.2.1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:ed8e452f5cb34f59f4861d50d029c6203b5f8f11c13d4186264392a64b4def8d", size = 1489530, upload-time = "2026-10-22T13:58:22Z" },
{ url = "https://files.pythonhosted.org/packages/8c/b4/379b2e5584be70110b154021ffa61b84dc3b476d2c031317f5c3fbcfcde5/brotli-1.2.1-cp313-cp313-win32.whl", hash = "sha256:c788fdab2eeeab671f5ab375ad010ef743f6c95be67f60f878e8876a51f6c671", size = 333014, upload-time = "2026-10-22T13:58:23.112Z" },
{ url = "https://files.pythonhosted.org/packages/96/c2/0c7abb25013043d4bcf7dd768d343bf7dee6ce99ac8c172f7ee4aef1c772/brotli-1.2.1-cp313-cp313-win_amd64.whl", hash = "sha256:e6c4fa55eab6b5684b33239597876a18a35ce1280b63ab960164032afbb8783f", size = 371072, upload-time = "2026-10-22T13:58:24.161Z" },
{ url = "https://files.pythonhosted.org/packages/81/68/5f9d96203fdce3ff9ba1186ce10220e26483ffa2384cde5430dd980e9057/brotli-1.2.1-cp313-cp313-win_arm64.whl", hash = "sha256:67f4dceb22ce3a742751cee849644449b169684699aa548e77d771de304e963c", size = 380893, upload-time = "2026-10-22T13:58:25.216Z" },
{ url = "https://files.pythonhosted.org/packages/af/dd/641dd6072ea8c5e0ef7d1f8dc02149b4b0b77ef68fe12fc08b08320980d5/brotli-1.2.1-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:b8b2fc7901b31d5885f26521cf2d74feccdc126a272ba1d879b7545bb25468bc", size = 863184, upload-time = "2026-10-22T13:58:27.201Z" },
{ url = "https://files.pythonhosted.org/packages/b2/b8/3c4a50dc10f098a022519042eab84f11ef8c29abb9270fca76086ac392c3/brotli-1.2.1-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:de1bf30d819809a964060356032e6670c9923644666afaaf3d1719d05ddb844e", size = 443276, upload-time = "2026-10-22T13:58:28.47Z" },
{ url = "https://files.pythonhosted.org/packages/b1/a1/5ef931a404de2c61b2cd775e70b1652ff3f4d434f0589da22efbee590f4a/brotli-1.2.1-cp314-cp314-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl", hash = "sha256:ed62c15ce5e8b975b2710a0f7c8bb953cbffb472fccb05055be6afcba9520c19", size = 1243026, upload-time = "2026-10-22T13:58:29.934Z" },
{ url = "https://files.pythonhosted.org/packages/11/95/54e7278ee3fc589a6ac3a9bc191b5412542949411f0b38c56614d0f51282/brotli-1.2.1-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:a3942e2d365f009e5e7839114fa315e1a2bcfea12b52064a28f1a7281862a2a7", size = 1402517, upload-time = "2026-10-22T13:58:31.231Z" },
{ url = "https://files.pythonhosted.org/packages/4d/db/e065253a23a09d97dc7bd8cb47194a85479d71be35a51b5c575ae5095ebe/brotli-1.2.1-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:6532d2c58018121a9eb5882656cf862cdbbd853600cb5fbccac59bcff14f66b0", size = 1424572, upload-time = "2026-10-22T13:58:32.468Z" },
{ url = "https://files.pythonhosted.org/packages/c4/07/8948582ce173e8e43cea9fadb8c3c3ea94800fb9814119c9239c1ae6c797/brotli-1.2.1-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:a998857af4fcd19920bdf5f30f0f1a23f38472a703d33e49168cd4fd32438377", size = 1420557, upload-time = "2026-10-22T13:58:34.1Z" },
{ url = "https://files.pythonhosted.org/packages/c0/d4/65073a06d6356a8f60b4c6ce0c3ade206d1118b8b8705b548f7b72ed549a/brotli-1.2.1-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:f352f797b1477c46a20e45f4cbc3a0e258e61963c8ae8f79f84a7bfb4195b37e", size = 1486209, upload-time = "2026-10-22T13:58:35.331Z" },
{ url = "https://files.pythonhosted.org/packages/5a/19/8663beee29665f6fdf520b9c7ea333e0e930952176092be24c4c8bd8f791/brotli-1.2.1-cp314-cp314-musllinux_1_2_i686.whl", hash = "sha256:be371af72af54b6e508b68a7a87768e05b7e13023f6c82a3e3f5e4d0e829e186", size = 1279217, upload-time = "2026-10-22T13:58:36.719Z" },
{ url = "https://files.pythonhosted.org/packages/bb/a6/3851ec19707a1610412ecf4f057d7c1b5acbb8f8b211494a70e1f451ccf6/brotli-1.2.1-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:dca1bf90b3a6a7440904f128464eb95bf46ac9da7ad3c85912483dd83128fd95", size = 1594292, upload-time = "2026-10-22T13:58:38.115Z" },
{ url = "https://files.pythonhosted.org/packages/8d/13/6bc9d4cb30fb0a475cd149c2075b9f63b93257a84ff70d0783a403f4b015/brotli-1.2.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:e4a7be59d374c445766fa4ac066a04d19a757365ae00a950053d0ab7388d1039", size = 1489719, upload-time = "2026-10-22T13:58:39.365Z" },
{ url = "https://files.pythonhosted.org/packages/04/da/1ec2e9b4f500693e56e6391f128dd0d304d6d2b7f0d5090e80a802b722f1/brotli-1.2.1-cp314-cp314-win32.whl", hash = "sha256:0a30e5df5ba985b67f85e13dfd7e5a68e186d8d9b5cff7d9a8904832bb413e81", size = 338208, upload-time = "2026-10-22T13:58:40.927Z" },
{ url = "https://files.pythonhosted.org/packages/24/5e/80cd38322263da5e92ea4935423f1d3cfa2509201673dce77b772c47b997/brotli-1.2.1-cp314-cp314-win_amd64.whl", hash = "sha256:972a2e19dbaab47d0aa77c1d07f71c930bdc53538f2b33decc905e92a4be5cbd", size = 376707, upload-time = "2026-10-22T13:58:42.025Z" },
{ url = "https://files.pythonhosted.org/packages/c5/35/cb35c489fcd0a813bbb7500465af5a7bac3bee7d996f6a211bac9096ef2c/brotli-1.2.1-cp314-cp314-win_arm64.whl", hash = "sha256:a3dea7e1ac49e29306500a8627fa3eacabcc9e238d22495eea2d8e61de77065b", size = 390280, upload-time = "2026-10-22T13:58:43.256Z" },
{ url = "https://files.pythonhosted.org/packages/a2/72/85ea89c1088673a297e8b652cf0080668812144d5525db6512eadddf9d19/brotli-1.2.1-cp315-cp315-macosx_10_15_universal2.whl", hash = "sha256:eea17e3cdc6f9aecffe15fe9a9eeba8e8c049a5d347de1561eed2240cdf35a79", size = 863002, upload-time = "2026-10-22T13:58:44.325Z" },
{ url = "https://files.pythonhosted.org/packages/81/6a/bd4db4ef0c4d083d8a2e5a1d0b20239b186f261e72b14a9a0ec91c078e92/brotli-1.2.1-cp315-cp315-macosx_10_15_x86_64.whl", hash = "sha256:761db18dc32b24d00fbedca1013089eb3470053cefa8492bd9c07b11102d72bf", size = 443263, upload-time = "2026-10-22T13:58:45.474Z" },
{ url = "https://files.pythonhosted.org/packages/79/1a/08635be453d523474fdfce81f588583a4e8049c75c04df849b5a278bf027/brotli-1.2.1-cp315-cp315-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl", hash = "sha256:035f3bf861328d8b7fcb61f4a976f9913027158e5666116733d59ffc65f2745e", size = 1244379, upload-time = "2026-10-22T13:58:46.615Z" },
{ url = "https://files.pythonhosted.org/packages/3b/94/52f2facc5ae118a68c4f0e715e03c9f854507a359608804d7de2dfe8d603/brotli-1.2.1-cp315-cp315-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:aa32b51702a0b6e9fddc0d1dc66cb6ed6af112f1bba0cac2be9039546a862b5c", size = 1404133, upload-time = "2026-10-22T13:58:47.756Z" },
{ url = "https://files.pythonhosted.org/packages/1e/9b/768635c710b684574da55e12bcdbe36419fd8bf6cb97e01c72a4593f9747/brotli-1.2.1-cp315-cp315-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:810aea5958d6a2d92593fd27218df528644aeb72cc7f6d401e650752d6c98949", size = 1424875, upload-time = "2026-10-22T13:58:48.912Z" },
{ url = "https://files.pythonhosted.org/packages/46/39/7b7d53d7cea61f35f6b52ea149804523d01e532bf26017eed4351d82a7ad/brotli-1.2.1-cp315-cp315-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:f00ff4b56caf151365c77988eb85a4a71eb26fe63047d4084d5614e06d65e3cf", size = 1425345, upload-time = "2026-10-22T13:58:50.253Z" },
{ url = "https://files.pythonhosted.org/packages/cb/5e/652c788a4da5c6415c7ae31382626ba870020c9280e0f9be1a6dfec80261/brotli-1.2.1-cp315-cp315-musllinux_1_2_aarch64.whl", hash = "sha256:dafaeb38712f99e5c31ad7f02c366f1ddd3792921b94066015f8c49f197163b0", size = 1487925, upload-time = "2026-10-22T13:58:51.593Z" },
{ url = "https://files.pythonhosted.org/packages/85/c9/1b7753d02c106ff70b3d88056586ed7c2fe329d2aa5707cdaebe90372942/brotli-1.2.1-cp315-cp315-musllinux_1_2_i686.whl", hash = "sha256:972e34c5cbbfe6c90358bfae38edae1a29d3456e920da5a4664c98f221ee4df8", size = 1276526, upload-time = "2026-10-22T13:58:53.142Z" },
{ url = "https://files.pythonhosted.org/packages/a5/5f/6ae3e85bfb7a3f4711cea5a1b92d4377fddf1319a666cda213cc83b6c040/brotli-1.2.1-cp315-cp315-musllinux_1_2_ppc64le.whl", hash = "sha256:59ca34c7a83d7a3b842247183c426f891a95a8f57db41b7aac92195c51e8a83a", size = 1594681, upload-time = "2026-10-22T13:58:54.571Z" },
{ url = "https://files.pythonhosted.org/packages/8a/52/2808406d83e5462c086d69dbebb892f4ed78d2eb4b6755ea4b9e58357e3b/brotli-1.2.1-cp315-cp315-musllinux_1_2_x86_64.whl", hash = "sha256:90bd49511341dd7c40bb3acaa97cd8b9e5df4821a5714b842117e7069c7cc03e", size = 1486968, upload-time = "2026-10-22T13:58:56.008Z" },
{ url = "https://files.pythonhosted.org/packages/a9/53/35042a907c309af8defb423e58692bdc5e3e1fd3d03388892b7614270626/brotli-1.2.1-cp315-cp315-win32.whl", hash = "sha256:1880db6241276e191d3e34a6e4eea708c6629103c396f0ea21e75b0ce63a9a2c", size = 338212, upload-time = "2026-10-22T13:58:57.189Z" },
{ url = "https://files.pythonhosted.org/packages/38/6c/883a32c23603e278ae93d8df31f0573b67382de827e6e05a6e9ee0b51896/brotli-1.2.1-cp315-cp315-win_amd64.whl", hash = "sha256:09a2f112a638aca15a88f298571788795fc45d62f2bca7f3870c1152153ef399", size = 376707, upload-time = "2026-10-22T13:58:58.292Z" },
{ url = "https://files.pythonhosted.org/packages/9f/d5/2c574da9faf130a59c4c685ba1378f2f470b57fb9fa5d3b0f8b4e90579a0/brotli-1.2.1-cp315-cp315-win_arm64.whl", hash = "sha256:91097d66ebb8ca8d23372081a33405a44e2239445614888e38929bd7f5879dae", size = 390278, upload-time = "2026-10-22T13:58:59.512Z" },
]
[[package]]
@@ -910,14 +929,14 @@ wheels = [
[[package]]
name = "djangorestframework"
version = "3.18.0"
version = "3.18.2"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "django" },
]
sdist = { url = "https://files.pythonhosted.org/packages/79/bc/de04e3d4dc65e8b926700956ee70d4f084f2005603d21122d4d0683006fd/djangorestframework-3.18.0.tar.gz", hash = "sha256:2323a5111837e0b784dcb8323abc78ecc54fa2a5af7aff2677cf50cdd849477f", size = 913667, upload-time = "2026-08-07T14:53:33.571Z" }
sdist = { url = "https://files.pythonhosted.org/packages/3f/8b/2f048189610d58b29ae947f46d9c7d19c20ba427568eac906d6a2277d391/djangorestframework-3.18.2.tar.gz", hash = "sha256:8249bad7715eb37274ce5820fc5d9fd786b7579806589565f52a71c02b35f54b", size = 923551, upload-time = "2026-10-06T20:42:15.116Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/51/d7/7461738e819b853275b35fd35712a85806c09a327e1fd5f8ff766bdd317e/djangorestframework-3.18.0-py3-none-any.whl", hash = "sha256:381fc44d3249c9565c5f723850855b734e99030eb30957a49f506d3fe11d7dcb", size = 900032, upload-time = "2026-08-07T14:53:31.881Z" },
{ url = "https://files.pythonhosted.org/packages/fa/9e/81e5bfbba54f7396233df979befde49e6e13c94c2df21b94555684dddd3e/djangorestframework-3.18.2-py3-none-any.whl", hash = "sha256:a92d6a4e128a2550bd2fe2e02008a9e47fe52c180e130bb438f5ba70ee933a3f", size = 903990, upload-time = "2026-10-06T20:42:13.411Z" },
]
[[package]]
@@ -1339,7 +1358,7 @@ wheels = [
[[package]]
name = "meet"
version = "1.36.2"
version = "1.36.3"
source = { editable = "." }
dependencies = [
{ name = "aiohttp" },
@@ -1412,7 +1431,7 @@ requires-dist = [
{ name = "aiohttp", specifier = "==3.14.5" },
{ name = "boto3", specifier = "==1.43.80" },
{ name = "brevo-python", specifier = "==1.2.0" },
{ name = "brotli", specifier = "==1.2.0" },
{ name = "brotli", specifier = "==1.2.1" },
{ name = "celery", extras = ["redis"], specifier = "==5.6.3" },
{ name = "cryptography", specifier = "==50.0.1" },
{ name = "dj-database-url", specifier = "==3.1.2" },
@@ -1427,7 +1446,7 @@ requires-dist = [
{ name = "django-redis", specifier = "==7.0.0" },
{ name = "django-storages", extras = ["s3"], specifier = "==1.14.6" },
{ name = "django-timezone-field", specifier = ">=5.1" },
{ name = "djangorestframework", specifier = "==3.18.0" },
{ name = "djangorestframework", specifier = "==3.18.2" },
{ name = "dockerflow", specifier = "==2026.3.4" },
{ name = "drf-spectacular", specifier = "==0.30.0" },
{ name = "easy-thumbnails", specifier = "==2.10.1" },
+3 -2
View File
@@ -47,12 +47,13 @@ ENV VITE_MEDIA_BASE_URL=${VITE_MEDIA_BASE_URL}
RUN npm run build
# ---- Front-end image ----
FROM nginxinc/nginx-unprivileged:1.30.5-alpine3.24 AS frontend-production
FROM nginxinc/nginx-unprivileged:1.30.6-alpine3.24 AS frontend-production
USER root
RUN apk upgrade --no-cache libexpat && \
apk add --no-cache --upgrade 'pcre2>=10.49-r0' 'tiff>=4.7.2-r0' \
'libx11>=1.8.14-r0' 'ca-certificates>=20261020-r0' && \
'libx11>=1.8.14-r0' 'ca-certificates>=20261020-r0' \
'brotli-libs>=1.2.1-r0' && \
apk del curl
USER nginx
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "meet",
"version": "1.36.2",
"version": "1.36.3",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "meet",
"version": "1.36.2",
"version": "1.36.3",
"dependencies": {
"@fontsource-variable/atkinson-hyperlegible-next": "5.3.0",
"@fontsource-variable/lexend": "5.3.0",
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "meet",
"private": true,
"version": "1.36.2",
"version": "1.36.3",
"type": "module",
"scripts": {
"dev": "panda codegen && vite",
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "mail_mjml",
"version": "1.36.2",
"version": "1.36.3",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "mail_mjml",
"version": "1.36.2",
"version": "1.36.3",
"license": "MIT",
"dependencies": {
"@html-to/text-cli": "0.6.1",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "mail_mjml",
"version": "1.36.2",
"version": "1.36.3",
"description": "An util to generate html and text django's templates from mjml templates",
"type": "module",
"dependencies": {
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "sdk",
"version": "1.36.2",
"version": "1.36.3",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "sdk",
"version": "1.36.2",
"version": "1.36.3",
"license": "ISC",
"workspaces": [
"./library",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "sdk",
"version": "1.36.2",
"version": "1.36.3",
"author": "",
"license": "ISC",
"description": "",
+1 -1
View File
@@ -5,7 +5,7 @@ RUN apk add --no-cache --upgrade 'ca-certificates>=20261020-r0'
# Install ffmpeg for audio/video processing (format conversion, extraction, compression)
# See summary/core/file_service.py for usage.
RUN apk add --no-cache "ffmpeg=8.1.2-r0"
RUN apk add --no-cache "ffmpeg=8.1.3-r0" 'brotli-libs>=1.2.1-r0'
FROM base AS builder
+1 -1
View File
@@ -1,7 +1,7 @@
[project]
name = "summary"
version = "1.36.2"
version = "1.36.3"
requires-python = ">=3.13"
dependencies = [
"certifi>=2026.10.19",
+14 -3
View File
@@ -5,6 +5,7 @@ from datetime import datetime, timezone
from celery.result import AsyncResult
from fastapi import APIRouter, Depends, HTTPException, Request, status
from pydantic import ValidationError
from summary.core.analytics import get_analytics
from summary.core.celery_worker import (
@@ -13,7 +14,9 @@ from summary.core.celery_worker import (
summarize_v2_task,
)
from summary.core.config import AuthorizedTenant, get_settings
from summary.core.exceptions import CleanedValidationError
from summary.core.models import SummarizeTaskApiRequest, TranscribeTaskApiRequest
from summary.core.privacy import dump_for_transport
from summary.core.security import verify_tenant_api_key_v2
from summary.core.shared_models import (
SummarizeWebhookFailurePayload,
@@ -52,7 +55,7 @@ async def create_transcribe_task_v2(
task = process_audio_transcribe_v2_task.apply_async(
args=[
{
**request.model_dump(),
**dump_for_transport(request),
"tenant_id": request_tenant.id,
"received_at": datetime.now(timezone.utc),
}
@@ -125,7 +128,11 @@ async def get_transcribe_job_status(
if task.status == "SUCCESS":
result = task.result
return TranscribeWebhookSuccessPayload.model_validate(result).model_dump()
try:
payload = TranscribeWebhookSuccessPayload.model_validate(result)
except ValidationError as exc:
raise CleanedValidationError(exc, secret_input=True) from None
return dump_for_transport(payload)
if task.status == "FAILURE":
return TranscribeWebhookFailurePayload(
@@ -155,7 +162,11 @@ async def get_summarize_job_status(
if task.status == "SUCCESS":
result = task.result
return SummarizeWebhookSuccessPayload.model_validate(result).model_dump()
try:
payload = SummarizeWebhookSuccessPayload.model_validate(result)
except ValidationError as exc:
raise CleanedValidationError(exc, secret_input=True) from None
return dump_for_transport(payload)
if task.status == "FAILURE":
return SummarizeWebhookFailurePayload(
+6 -7
View File
@@ -5,7 +5,6 @@ import socket
import time
from collections import Counter
from functools import cached_property, lru_cache
from urllib.parse import urlsplit, urlunsplit
import redis
from celery.utils.log import get_task_logger
@@ -13,6 +12,7 @@ from posthog import Posthog
from summary.core.config import get_settings
from summary.core.models import SummarizeTaskJob, TranscribeTaskJob
from summary.core.privacy import redact_url
logger = get_task_logger(__name__)
settings = get_settings()
@@ -54,8 +54,8 @@ class Analytics:
self._client.capture(
event_name, distinct_id=distinct_id, properties=properties
)
except Exception as e:
raise AnalyticsException("Failed to capture analytics event") from e
except Exception:
raise AnalyticsException("Failed to capture analytics event") from None
def is_feature_enabled(self, feature_name: str, distinct_id: str = None) -> bool:
"""Check if a feature flag is enabled for a user."""
@@ -64,8 +64,8 @@ class Analytics:
try:
return self._client.feature_enabled(feature_name, distinct_id)
except Exception as e:
logger.error("Error checking feature flag %s: %s", feature_name, e)
except Exception:
logger.error("Error checking feature flag %s", feature_name)
return False
@@ -133,8 +133,7 @@ class MetadataManager:
}
if isinstance(task_payload, TranscribeTaskJob):
parts = urlsplit(task_payload.cloud_storage_url)
clean_url = urlunsplit((parts.scheme, parts.netloc, parts.path, "", ""))
clean_url = redact_url(task_payload.cloud_storage_url.get_secret_value())
initial_metadata["source_url"] = clean_url
initial_metadata["asr_model"] = settings.whisperx_asr_model
elif isinstance(task_payload, SummarizeTaskJob):
+105 -40
View File
@@ -3,20 +3,23 @@
# ruff: noqa: PLR0913
import json
import logging
import time
from datetime import datetime, timezone
from typing import Any
from urllib.parse import urljoin
import requests
from celery import Celery, signals
from celery import Celery, Task, signals
from celery.utils.log import get_task_logger
from openai.types.audio import Transcription
from pydantic import SecretStr, ValidationError
from requests import exceptions
from summary.core.analytics import MetadataManager, get_analytics
from summary.core.config import get_settings
from summary.core.docs_service import create_document_in_lasuite_docs
from summary.core.exceptions import CleanedValidationError
from summary.core.file_service import (
CorruptedAudioFile,
FileService,
@@ -31,6 +34,7 @@ from summary.core.models import (
SummarizeTaskJob,
TranscribeTaskJob,
)
from summary.core.privacy import dump_for_transport, redact_url
from summary.core.prompt import (
FORMAT_NEXT_STEPS,
FORMAT_PLAN,
@@ -63,8 +67,24 @@ metadata_manager = MetadataManager()
logger = get_task_logger(__name__)
def _reveal(secret: SecretStr | None) -> str | None:
"""Return the underlying value of an optional secret."""
return secret.get_secret_value() if secret is not None else None
class PrivateTask(Task):
"""Hide task arguments in Celery logs and monitoring events."""
def apply_async(self, args=None, kwargs=None, **options):
"""Publish with redacted display representations."""
options.update(argsrepr="[Filtered]", kwargsrepr="[Filtered]")
return super().apply_async(args=args, kwargs=kwargs, **options)
celery = Celery(
__name__,
task_cls=PrivateTask,
broker=settings.celery_broker_url,
backend=settings.celery_result_backend,
broker_connection_retry_on_startup=True,
@@ -76,10 +96,28 @@ celery = Celery(
celery.config_from_object("summary.core.celery_config")
class CeleryLogPrivacyFilter(logging.Filter):
"""Prevent Celery trace logs from serializing task results or exceptions."""
def filter(self, record):
"""Keep task identity and state without upstream values."""
if record.levelno < logging.INFO:
return True
data = getattr(record, "data", None)
if isinstance(data, dict):
record.msg = "Task %s[%s] finished or retried (details omitted)"
record.args = (data.get("name", "unknown"), data.get("id", "unknown"))
record.data = {}
record.exc_info = None
record.exc_text = None
return True
@signals.celeryd_init.connect
def init_celery_sentry(**_kwargs):
"""Initialize Sentry in the Celery worker."""
init_sentry()
logging.getLogger("celery.app.trace").addFilter(CeleryLogPrivacyFilter())
file_service = FileService()
@@ -162,9 +200,9 @@ def transcribe_audio(
try:
res.raise_for_status()
except requests.exceptions.HTTPError:
logger.exception("WhisperX transcription failed")
logger.error("WhisperX transcription failed")
# We reraise the error so that it can be retried by celery
raise
raise requests.exceptions.HTTPError("WhisperX request failed") from None
transcription_json: dict[str, Any] = res.json()
# We remove the "usage" key from the transcription_json dictionary
@@ -175,13 +213,16 @@ def transcribe_audio(
# We force the use of the Transcription model here
# to avoid changing too much code for now.
# Note that it should be WhisperXResponse instead.
transcription = Transcription.model_validate(
# We add a dummy "text" to make the model validate,
# Some API responses lack the "text" key.
{"text": "", **transcription_json},
extra="allow",
strict=False,
)
try:
transcription = Transcription.model_validate(
# We add a dummy "text" to make the model validate,
# Some API responses lack the "text" key.
{"text": "", **transcription_json},
extra="allow",
strict=False,
)
except ValidationError as exc:
raise CleanedValidationError(exc, secret_input=True) from None
# Logging
transcription_duration = round(time.time() - transcription_start_time, 2)
@@ -199,9 +240,9 @@ def transcribe_audio(
if raises:
raise e
redacted_cloud_storage_url = (
cloud_storage_url.split("?", 1)[0] if cloud_storage_url else None
redact_url(cloud_storage_url) if cloud_storage_url else None
)
logger.exception(
logger.error(
("Unexpected error while preparing file %s "),
redacted_cloud_storage_url,
)
@@ -230,7 +271,7 @@ def resolve_speaker_identities_and_apply_to(
logger.debug("Running resolve_speaker_identities")
try:
metadata = file_service.read_cloud_storage_json(
recording_metadata.cloud_storage_url
recording_metadata.cloud_storage_url.get_secret_value()
)
speaker_mapping = resolve_speaker_identities(
metadata,
@@ -241,21 +282,18 @@ def resolve_speaker_identities_and_apply_to(
new_transcription = speaker_mapping.apply_to(transcription.model_dump())
return WhisperXResponse.model_validate(new_transcription)
except FileServiceException as exc:
except FileServiceException:
logger.error(
"Error reading metadata for task %s; skipping speaker assignment."
" Error: %s",
"Error reading metadata for task %s; skipping speaker assignment.",
task_id,
exc,
)
return transcription
except Exception as exc:
logger.exception(
except Exception:
logger.error(
"resolve_speaker_identities failed for task %s; skipping"
" speaker assignment. Error: %s",
" speaker assignment.",
task_id,
exc,
)
return transcription
@@ -302,6 +340,17 @@ def format_actions(llm_output: dict) -> str:
return ""
def parse_llm_json(value: str) -> dict:
"""Parse an LLM response without exposing it in decoding errors."""
try:
result = json.loads(value)
if not isinstance(result, dict):
raise ValueError
return result
except (TypeError, ValueError):
raise LLMException("Invalid LLM response") from None
def summarize_transcription_internals(
*, distinct_id: str, transcript: str, session_id: str
) -> str:
@@ -342,13 +391,13 @@ def summarize_transcription_internals(
)
logger.info("Plan generated")
res = json.loads(parts)
res = parse_llm_json(parts)
parts = res.get("titles", [])
logger.info("Parts to summarize: %s", parts)
logger.info("Parts to summarize: %s", len(parts))
parts_summarized = []
for part in parts:
for i, part in enumerate(parts):
prompt_user_part = PROMPT_USER_PART.format(part=part, transcript=transcript)
logger.info("Summarizing part: %s", part)
logger.info("Summarizing part %s", i + 1)
parts_summarized.append(
llm_service.call(PROMPT_SYSTEM_PART, prompt_user_part, name="part")
)
@@ -364,7 +413,7 @@ def summarize_transcription_internals(
response_format=FORMAT_NEXT_STEPS,
)
next_steps = format_actions(json.loads(next_steps))
next_steps = format_actions(parse_llm_json(next_steps))
logger.info("Next steps generated")
@@ -437,9 +486,11 @@ def call_webhook_v2_task(
tenant_id: str,
):
"""Calls a webhook asynchrously (retry handled by celery)."""
call_webhook_v2(
payload=webhook_payload_adapter.validate_python(payload), tenant_id=tenant_id
)
try:
validated_payload = webhook_payload_adapter.validate_python(payload)
except ValidationError as exc:
raise CleanedValidationError(exc, secret_input=True) from None
call_webhook_v2(payload=validated_payload, tenant_id=tenant_id)
@celery.task(
@@ -465,7 +516,10 @@ def process_audio_transcribe_v2_task(
self: Celery task instance (passed on with bind=True)
payload: Serialized dictionary of TranscribeSummarizeTaskCreationV2
"""
payload = TranscribeTaskJob.model_validate(payload)
try:
payload = TranscribeTaskJob.model_validate(payload)
except ValidationError as exc:
raise CleanedValidationError(exc, secret_input=True) from None
logger.info(
"Transcribing for object received | Owner: %s",
payload.user_sub,
@@ -477,7 +531,7 @@ def process_audio_transcribe_v2_task(
transcription_res = WhisperXResponse(
**transcribe_audio( # type: ignore
task_id=job_id,
cloud_storage_url=payload.cloud_storage_url,
cloud_storage_url=payload.cloud_storage_url.get_secret_value(),
language=payload.language,
raises=True,
).model_dump()
@@ -514,8 +568,8 @@ def process_audio_transcribe_v2_task(
transcription_res.model_dump(),
payload.context_language,
payload.language,
payload.push_to_docs_config.download_link,
payload.push_to_docs_config.form_link,
_reveal(payload.push_to_docs_config.download_link),
_reveal(payload.push_to_docs_config.form_link),
)
create_document_in_lasuite_docs(
@@ -556,18 +610,22 @@ def process_audio_transcribe_v2_task(
transcription_data_url=file_service.get_transcript_signed_url(job_id),
)
call_webhook_v2_task.apply_async(
args=[success_payload.model_dump(), payload.tenant_id]
args=[dump_for_transport(success_payload), payload.tenant_id]
)
metadata_manager.capture(job_id, settings.posthog_transcript_success)
return success_payload.model_dump()
return dump_for_transport(success_payload)
@signals.task_prerun.connect(sender=process_audio_transcribe_v2_task)
def task_started_transcript(task_id=None, task=None, args=None, **kwargs):
"""Signal handler called before task execution begins."""
if args:
metadata_manager.create(task_id, TranscribeTaskJob.model_validate(args[0]))
try:
payload = TranscribeTaskJob.model_validate(args[0])
except ValidationError as exc:
raise CleanedValidationError(exc, secret_input=True) from None
metadata_manager.create(task_id, payload)
@signals.task_retry.connect(sender=process_audio_transcribe_v2_task)
@@ -627,7 +685,10 @@ def summarize_v2_task(
1. Run summary internals
2. Sends the final summary via webhook.
"""
payload = SummarizeTaskJob.model_validate(payload)
try:
payload = SummarizeTaskJob.model_validate(payload)
except ValidationError as exc:
raise CleanedValidationError(exc, secret_input=True) from None
summary = summarize_transcription_internals(
distinct_id=payload.user_sub,
transcript=payload.content,
@@ -652,18 +713,22 @@ def summarize_v2_task(
summary_data_url=file_service.get_summary_signed_url(job_id),
)
call_webhook_v2_task.apply_async(
args=[success_payload.model_dump(), payload.tenant_id]
args=[dump_for_transport(success_payload), payload.tenant_id]
)
metadata_manager.capture(job_id, settings.posthog_summary_success)
return success_payload.model_dump()
return dump_for_transport(success_payload)
@signals.task_prerun.connect(sender=summarize_v2_task)
def task_started_summary(task_id=None, task=None, args=None, **kwargs):
"""Signal handler called before task execution begins."""
if args:
metadata_manager.create(task_id, SummarizeTaskJob.model_validate(args[0]))
try:
payload = SummarizeTaskJob.model_validate(args[0])
except ValidationError as exc:
raise CleanedValidationError(exc, secret_input=True) from None
metadata_manager.create(task_id, payload)
@signals.task_retry.connect(sender=summarize_v2_task)
+3 -1
View File
@@ -4,7 +4,7 @@ import json
import logging
from urllib.parse import urljoin
from requests import Session
from requests import RequestException, Session
from requests.adapters import HTTPAdapter
from urllib3.util import Retry
@@ -43,6 +43,8 @@ def _post_with_retries(*, url, data, api_key: str | None = None):
response = session.post(url, json=data, timeout=(20, 3 * 60))
response.raise_for_status()
return response
except RequestException:
raise RequestException("Docs delivery failed") from None
finally:
session.close()
+62
View File
@@ -0,0 +1,62 @@
"""Validation errors suitable for logging and transport."""
from typing import get_args
from pydantic import SecretStr, ValidationError
from pydantic_core import ErrorType, PydanticCustomError
_BUILTIN_ERROR_TYPES = frozenset(get_args(ErrorType))
def clean_validation_errors(errors: list[dict]) -> list[dict]:
"""Mask SecretStr values, including those nested in validation input."""
def clean(value):
if isinstance(value, SecretStr):
return "[redacted]"
if isinstance(value, Exception):
# Do not retain validator exception objects, whose args may hold secrets.
return str(value)
if isinstance(value, dict):
return {key: clean(item) for key, item in value.items()}
if isinstance(value, list):
return [clean(item) for item in value]
if isinstance(value, tuple):
return tuple(clean(item) for item in value)
return value
return clean(errors)
class CleanedValidationError(ValidationError):
"""A new Pydantic exception with secret inputs replaced by a marker."""
def __init__(self, error, *args, secret_input=False, **kwargs):
"""State is initialized by Pydantic during __new__."""
def __new__(cls, error, *args, secret_input=False, **kwargs):
"""Accept an original error, or Pydantic's native construction arguments."""
if not isinstance(error, ValidationError):
# Pydantic uses this path when constructing or unpickling the error.
return super().__new__(cls, error, *args, **kwargs)
details = clean_validation_errors(error.errors(include_url=False))
for detail in details:
if secret_input:
# An adapter may have received the unwrapped value of a SecretStr.
detail["input"] = "[redacted]"
return cls._from_details(error.title, details)
@classmethod
def _from_details(cls, title, details):
"""Reconstruct builtin and custom Pydantic error codes."""
for detail in details:
if detail["type"] not in _BUILTIN_ERROR_TYPES:
detail["type"] = PydanticCustomError(
detail["type"], detail["msg"], detail.get("ctx")
)
return cls.from_exception_data(title, details, hide_input=True)
def __reduce__(self):
"""Preserve custom error codes when Celery pickles the exception."""
return self._from_details, (self.title, self.errors(include_url=False))
+5 -4
View File
@@ -17,6 +17,7 @@ import requests
from botocore.config import Config
from summary.core.config import get_settings
from summary.core.privacy import redact_url
from summary.core.shared_models import WhisperXResponse
settings = get_settings()
@@ -314,7 +315,7 @@ class FileService:
"""Download file from a cloud storage URL to local temporary file."""
logger.info(
"Download recording from URL | cloud_storage_url: %s",
cloud_storage_url,
redact_url(cloud_storage_url),
)
if not cloud_storage_url:
@@ -352,10 +353,10 @@ class FileService:
logger.debug("Recording local file path: %s", local_path)
return local_path
except requests.RequestException as e:
except requests.RequestException:
raise FileServiceException(
"Unexpected error while downloading object from cloud_storage_url."
) from e
) from None
def _validate_duration(self, duration_seconds: float) -> None:
"""Validate audio file duration against configured maximum."""
@@ -376,7 +377,7 @@ class FileService:
def read_cloud_storage_json(self, cloud_storage_url: str) -> dict:
"""Read and parse a JSON file from a url."""
logger.info("Reading JSON: %s", cloud_storage_url)
logger.info("Reading JSON: %s", redact_url(cloud_storage_url))
res = requests.get(cloud_storage_url, timeout=(10, 100))
res.raise_for_status()
return res.json()
+3 -3
View File
@@ -140,6 +140,6 @@ class LLMService:
response = self._client.chat.completions.create(**params)
return response.choices[0].message.content
except Exception as e:
logger.exception("LLM call failed: %s", e)
raise LLMException(f"LLM call failed: {e}") from e
except Exception:
logger.error("LLM call failed")
raise LLMException("LLM call failed") from None
+19 -6
View File
@@ -2,10 +2,17 @@
from datetime import datetime
from pydantic import AwareDatetime, BaseModel, EmailStr, Field, field_validator
from pydantic import (
AwareDatetime,
BaseModel,
ConfigDict,
EmailStr,
Field,
field_validator,
)
from summary.core.config import get_settings
from summary.core.types import Url
from summary.core.types import SecretUrl
settings = get_settings()
@@ -13,6 +20,8 @@ settings = get_settings()
class SharedV2TaskCreation(BaseModel):
"""Model that holds basic information for task creation."""
model_config = ConfigDict(hide_input_in_errors=True)
user_sub: str = Field(title="User Sub", description="The user's sub.")
user_email: EmailStr | None = Field(
default=None,
@@ -24,7 +33,9 @@ class SharedV2TaskCreation(BaseModel):
class RecordingMetadata(BaseModel):
"""Model for recording metadata."""
cloud_storage_url: Url = Field(
model_config = ConfigDict(hide_input_in_errors=True)
cloud_storage_url: SecretUrl = Field(
title="Cloud Storage URL",
description="The URL of the metadata file for speaker assignment.",
)
@@ -35,6 +46,8 @@ class RecordingMetadata(BaseModel):
class PushToDocsBaseConfig(BaseModel):
"""Model containing information for pushing transcript and summaries to docs."""
model_config = ConfigDict(hide_input_in_errors=True)
user_email: EmailStr = Field(
title="User Email", description="The user's email, future owner of the docs."
)
@@ -44,12 +57,12 @@ class PushToDocsBaseConfig(BaseModel):
class PushToDocsTranscriptConfig(PushToDocsBaseConfig):
"""Model for push to docs information for transcripts."""
download_link: Url | None = Field(
download_link: SecretUrl | None = Field(
default=None,
title="Download Link",
description="The link to download the recording.",
)
form_link: Url | None = Field(
form_link: SecretUrl | None = Field(
default=None,
title="Form Link",
description="The link to fill out a form for the recording.",
@@ -69,7 +82,7 @@ class PushToDocsSummaryConfig(PushToDocsBaseConfig):
class TranscribeTaskApiRequest(SharedV2TaskCreation):
"""Model for creating a transcribe task (used for API request)."""
cloud_storage_url: Url = Field(
cloud_storage_url: SecretUrl = Field(
title="Cloud storage URL",
description="The URL of the audio file to transcribe.",
)
+36
View File
@@ -0,0 +1,36 @@
"""Privacy helpers for observability."""
from urllib.parse import urlsplit, urlunsplit
from pydantic import BaseModel, SecretStr
def dump_for_transport(model: BaseModel) -> dict:
"""Reveal secrets explicitly for task messages, HTTP bodies and API responses."""
def reveal(value):
if isinstance(value, SecretStr):
return value.get_secret_value()
if isinstance(value, dict):
return {key: reveal(item) for key, item in value.items()}
if isinstance(value, list):
return [reveal(item) for item in value]
return value
return reveal(model.model_dump())
def redact_url(url):
"""Keep the source identifiable without logging query or user credentials."""
if not isinstance(url, str):
return "[redacted]"
# Remove query and fragment before parsing, including for malformed URLs.
base, has_query, _ = url.partition("?")
base = base.partition("#")[0]
try:
parts = urlsplit(base)
except ValueError:
return "[redacted]"
authority = parts.netloc.rsplit("@", 1)[-1]
redacted = urlunsplit(parts._replace(netloc=authority))
return redacted + ("?[redacted]" if has_query else "")
+43
View File
@@ -1,9 +1,12 @@
"""Sentry configuration."""
import re
import sentry_sdk
from sentry_sdk.scrubber import DEFAULT_DENYLIST, DEFAULT_PII_DENYLIST, EventScrubber
from summary.core.config import get_settings
from summary.core.privacy import redact_url
# Exact names (case-insensitive) of variables and dict keys to redact.
SENSITIVE_DATA_DENYLIST = [
@@ -90,5 +93,45 @@ def init_sentry() -> None:
# Task creation requests carry the content to summarize.
max_request_body_size="never",
include_local_variables=True,
before_send=before_send,
before_send_transaction=before_send,
before_breadcrumb=before_breadcrumb,
event_scrubber=build_event_scrubber(),
)
def _scrub_strings(value):
"""Remove URL credentials, queries and fragments throughout an event."""
if isinstance(value, str):
return re.sub(
r"https?://[^\s\"'<>]+",
lambda m: redact_url(m.group()),
value,
flags=re.IGNORECASE,
)
if isinstance(value, dict):
return {key: _scrub_strings(item) for key, item in value.items()}
if isinstance(value, list):
return [_scrub_strings(item) for item in value]
return value
def before_send(event, hint):
"""Keep exception types and stacks, but discard untrusted message text."""
del hint
event.pop("message", None)
event.pop("logentry", None)
request = event.get("request", {})
request.pop("data", None)
request.pop("query_string", None)
for exception in event.get("exception", {}).get("values", []):
exception["value"] = "[Filtered]"
return _scrub_strings(event)
def before_breadcrumb(crumb, hint):
"""Drop arbitrary breadcrumb text and scrub URLs in structured data."""
del hint
crumb.pop("message", None)
crumb.pop("data", None)
return _scrub_strings(crumb)
+7 -3
View File
@@ -2,7 +2,9 @@
from typing import Annotated, Literal, Union
from pydantic import BaseModel, Field, TypeAdapter
from pydantic import BaseModel, ConfigDict, Field, TypeAdapter
from summary.core.types import SecretUrl
class WordSegment(BaseModel):
@@ -65,6 +67,8 @@ class WhisperXResponse(BaseModel):
class BaseWebhook(BaseModel):
"""Base webhook payload."""
model_config = ConfigDict(hide_input_in_errors=True)
job_id: str = Field(
title="Job ID",
description="The ID of the job document in the receiver system.",
@@ -76,7 +80,7 @@ class TranscribeWebhookSuccessPayload(BaseWebhook):
type: Literal["transcript"] = Field(default="transcript")
status: Literal["success"] = Field(default="success")
transcription_data_url: str = Field(
transcription_data_url: SecretUrl = Field(
title="Transcript", description="URL to the raw transcription data."
)
@@ -121,7 +125,7 @@ class SummarizeWebhookSuccessPayload(BaseWebhook):
type: Literal["summary"] = Field(default="summary")
status: Literal["success"] = Field(default="success")
summary_data_url: str = Field(
summary_data_url: SecretUrl = Field(
title="Summary", description="URL to the raw summary data."
)
+23 -1
View File
@@ -2,9 +2,31 @@
from typing import Annotated
from pydantic import BeforeValidator, HttpUrl, TypeAdapter
from pydantic import (
AfterValidator,
BeforeValidator,
HttpUrl,
SecretStr,
TypeAdapter,
ValidationError,
)
from summary.core.exceptions import CleanedValidationError
http_url_adapter = TypeAdapter(HttpUrl)
Url = Annotated[
str, BeforeValidator(lambda value: str(http_url_adapter.validate_python(value)))
]
def _validate_secret_url(value: SecretStr) -> SecretStr:
"""Validate and normalize a URL without echoing it in validator messages."""
try:
return SecretStr(
str(http_url_adapter.validate_python(value.get_secret_value()))
)
except ValidationError as exc:
raise CleanedValidationError(exc, secret_input=True) from None
SecretUrl = Annotated[SecretStr, AfterValidator(_validate_secret_url)]
+14 -10
View File
@@ -6,6 +6,7 @@ import logging
import requests
from summary.core.config import get_settings
from summary.core.privacy import dump_for_transport
from summary.core.shared_models import (
WebhookPayloads,
)
@@ -29,16 +30,19 @@ def call_webhook_v2(
logger.debug("Submitting to %s", tenant.webhook_url)
logger.debug("Request payload: %s", payload.model_dump_json(indent=2))
response = requests.post(
tenant.webhook_url,
json=payload.model_dump(),
headers={
"Authorization": f"Bearer {tenant.webhook_api_key.get_secret_value()}",
"User-Agent": settings.app_external_user_agent,
},
timeout=(10, 20),
)
response.raise_for_status()
try:
response = requests.post(
tenant.webhook_url,
json=dump_for_transport(payload),
headers={
"Authorization": f"Bearer {tenant.webhook_api_key.get_secret_value()}",
"User-Agent": settings.app_external_user_agent,
},
timeout=(10, 20),
)
response.raise_for_status()
except requests.RequestException:
raise requests.RequestException("Webhook delivery failed") from None
try:
response_data = response.json()
+15 -1
View File
@@ -1,11 +1,15 @@
"""Application."""
from dockerflow.fastapi import router as dockerflow_router
from fastapi import FastAPI
from fastapi import FastAPI, Request
from fastapi.encoders import jsonable_encoder
from fastapi.exceptions import RequestValidationError
from fastapi.responses import JSONResponse
from summary.api.main import api_router_v2
from summary.core import checks # noqa: F401 -- registers the Dockerflow checks
from summary.core.config import get_settings
from summary.core.exceptions import clean_validation_errors
from summary.core.sentry import init_sentry
settings = get_settings()
@@ -19,3 +23,13 @@ app = FastAPI(
app.include_router(api_router_v2, prefix=settings.app_api_v2_str)
app.include_router(dockerflow_router)
@app.exception_handler(RequestValidationError)
async def validation_error_handler(request: Request, exc: RequestValidationError):
"""Keep validation details while masking SecretStr input values."""
del request
return JSONResponse(
status_code=422,
content=jsonable_encoder({"detail": clean_validation_errors(exc.errors())}),
)
+164
View File
@@ -3,6 +3,10 @@
from datetime import datetime
from unittest.mock import ANY, MagicMock, patch
import pytest
from summary.core.exceptions import CleanedValidationError
class TestTasksV2:
"""Tests for the /v2/async-jobs-jobs endpoints."""
@@ -174,3 +178,163 @@ class TestTasksV2:
assert response.status_code == 403
assert response.json() == {"detail": "Forbidden"}
def test_validation_preserves_ordinary_input(client):
"""Validation errors keep input values that are not SecretStr."""
canary = "private-meeting-content"
response = client.post(
"/api/v2/async-jobs/summarize",
headers={"Authorization": "Bearer test-api-token"},
json={"user_sub": "user", "content": {"unexpected": canary}},
)
assert response.status_code == 422
assert canary in response.text
assert response.json()["detail"] == [
{
"type": "string_type",
"loc": ["body", "content"],
"msg": "Input should be a valid string",
"input": {"unexpected": canary},
}
]
def test_validation_preserves_context(client):
"""Validator errors retain location, message and serializable context."""
canary = "private-invalid-url-signature"
response = client.post(
"/api/v2/async-jobs/transcribe",
headers={"Authorization": "Bearer test-api-token"},
json={
"user_sub": "user",
"language": "en",
"cloud_storage_url": f"invalid-url?signature={canary}",
},
)
assert response.status_code == 422
assert canary not in response.text
assert response.json()["detail"] == [
{
"type": "url_parsing",
"loc": ["body", "cloud_storage_url"],
"msg": "Input should be a valid URL, relative URL without a base",
"input": "[redacted]",
"ctx": {"error": "relative URL without a base"},
}
]
@patch(
"summary.api.route.tasks_v2.process_audio_transcribe_v2_task.apply_async",
return_value=MagicMock(id="transcribe-job"),
)
def test_signed_urls_are_unwrapped_for_task_submission(mock_apply_async, client):
"""API input secrets reach Celery intact, including nested metadata URLs."""
signed_url = "https://storage.example.com/file?signature=deterministic-signature"
response = client.post(
"/api/v2/async-jobs/transcribe",
headers={"Authorization": "Bearer test-api-token"},
json={
"user_sub": "user",
"language": "en",
"cloud_storage_url": signed_url,
"metadata": {
"cloud_storage_url": signed_url,
"started_at": "2026-01-01T10:00:00Z",
"ended_at": "2026-01-01T11:00:00Z",
},
},
)
assert response.status_code == 200
payload = mock_apply_async.call_args.kwargs["args"][0]
assert payload["cloud_storage_url"] == signed_url
assert payload["metadata"]["cloud_storage_url"] == signed_url
@pytest.mark.parametrize(
"case",
[
("transcribe", "transcript", "transcription_data_url"),
("summarize", "summary", "summary_data_url"),
],
)
@patch("summary.api.route.tasks_v2.celery")
@patch("summary.api.route.tasks_v2.AsyncResult")
def test_success_status_returns_usable_signed_url(
mock_async_result, mock_celery, client, case
):
"""Success responses explicitly reveal result URLs for authorized clients."""
job_type, payload_type, url_field = case
signed_url = "https://storage.example.com/result?signature=deterministic-signature"
mock_celery.backend.client.exists.return_value = True
mock_async_result.return_value = MagicMock(
status="SUCCESS",
args=[{"tenant_id": "test-tenant"}],
result={"job_id": "job", url_field: signed_url},
)
response = client.get(
f"/api/v2/async-jobs/{job_type}/job",
headers={"Authorization": "Bearer test-api-token"},
)
assert response.status_code == 200
assert response.json() == {
"job_id": "job",
"type": payload_type,
"status": "success",
url_field: signed_url,
}
@pytest.mark.parametrize(
"case",
[
("transcribe", "transcription_data_url"),
("summarize", "summary_data_url"),
],
)
@patch("summary.api.route.tasks_v2.celery")
@patch("summary.api.route.tasks_v2.AsyncResult")
def test_invalid_success_result_hides_revealed_url(
mock_async_result, mock_celery, client, case
):
"""Missing result fields cannot retain a raw signed URL in error details."""
job_type, url_field = case
canary = "private-result-signature"
mock_celery.backend.client.exists.return_value = True
mock_async_result.return_value = MagicMock(
status="SUCCESS",
args=[{"tenant_id": "test-tenant"}],
result={url_field: f"https://storage.example.com/result?signature={canary}"},
)
with pytest.raises(CleanedValidationError) as exc:
client.get(
f"/api/v2/async-jobs/{job_type}/job",
headers={"Authorization": "Bearer test-api-token"},
)
assert exc.value.errors()[0]["type"] == "missing"
assert exc.value.errors()[0]["loc"] == ("job_id",)
assert exc.value.errors()[0]["input"] == "[redacted]"
assert canary not in str(exc.value)
assert canary not in exc.value.json()
@pytest.mark.parametrize("nested_error", [False, True])
def test_missing_field_errors_preserve_raw_json_strings(client, nested_error):
"""Raw JSON input is retained when it has not been converted to SecretStr."""
canary = "private-validation-signature"
signed_url = f"https://storage.example.com/recording?signature={canary}"
payload = {"language": "en", "cloud_storage_url": signed_url}
if nested_error:
payload["user_sub"] = "user"
payload["metadata"] = {"cloud_storage_url": signed_url}
response = client.post(
"/api/v2/async-jobs/transcribe",
headers={"Authorization": "Bearer test-api-token"},
json=payload,
)
assert response.status_code == 422
assert canary in response.text
assert all(error["type"] == "missing" for error in response.json()["detail"])
expected_input = payload["metadata"] if nested_error else payload
assert all(error["input"] == expected_input for error in response.json()["detail"])
+6
View File
@@ -22,6 +22,12 @@ def get_settings_override():
)
@pytest.fixture()
def settings_override():
"""Provide isolated settings containing the test tenant."""
return get_settings_override()
@pytest.fixture()
def client():
"""Provide a FastAPI TestClient for tests."""
@@ -0,0 +1,171 @@
"""Secret URLs stay masked until explicitly serialized for transport."""
from unittest.mock import Mock
import pytest
from kombu.serialization import dumps, loads
from pydantic import SecretStr, ValidationError
from summary.core import celery_worker
from summary.core.models import (
PushToDocsTranscriptConfig,
RecordingMetadata,
TranscribeTaskApiRequest,
TranscribeTaskJob,
)
from summary.core.privacy import dump_for_transport
from summary.core.shared_models import (
SummarizeWebhookSuccessPayload,
TranscribeWebhookSuccessPayload,
WhisperXResponse,
)
CANARY = "private-signed-url-canary"
SIGNED_URL = f"https://storage.example.com/recording.mp4?signature={CANARY}"
METADATA = {
"cloud_storage_url": SIGNED_URL,
"started_at": "2026-01-01T10:00:00Z",
"ended_at": "2026-01-01T11:00:00Z",
}
DOCS_CONFIG = {
"user_email": "user@example.com",
"title": "Meeting",
"download_link": SIGNED_URL,
"form_link": SIGNED_URL,
}
MODEL_CASES = [
(RecordingMetadata, METADATA, "cloud_storage_url"),
(
TranscribeTaskApiRequest,
{"user_sub": "user", "language": "en", "cloud_storage_url": SIGNED_URL},
"cloud_storage_url",
),
(PushToDocsTranscriptConfig, DOCS_CONFIG, "download_link"),
(PushToDocsTranscriptConfig, DOCS_CONFIG, "form_link"),
(
TranscribeWebhookSuccessPayload,
{"job_id": "job", "transcription_data_url": SIGNED_URL},
"transcription_data_url",
),
(
SummarizeWebhookSuccessPayload,
{"job_id": "job", "summary_data_url": SIGNED_URL},
"summary_data_url",
),
]
@pytest.mark.parametrize(("model", "data", "field"), MODEL_CASES)
def test_signed_urls_are_masked_by_default(model, data, field):
"""Reprs, ordinary dumps and JSON output mask all signed URL fields."""
value = model.model_validate(data)
secret = getattr(value, field)
assert isinstance(secret, SecretStr)
assert secret.get_secret_value() == SIGNED_URL
assert CANARY not in repr(value)
assert CANARY not in str(value.model_dump())
assert CANARY not in value.model_dump_json()
assert dump_for_transport(value)[field] == SIGNED_URL
assert getattr(model.model_validate(value.model_dump()), field) == secret
@pytest.mark.parametrize(("model", "data", "field"), MODEL_CASES)
def test_missing_fields_do_not_echo_signed_urls(model, data, field):
"""Root validation errors omit raw input even before secrets are wrapped."""
required_field = (
"job_id"
if "job_id" in data
else next(
key for key in data if key not in {field, "download_link", "form_link"}
)
)
invalid_data = {key: value for key, value in data.items() if key != required_field}
with pytest.raises(ValidationError) as exc:
model.model_validate(invalid_data)
assert CANARY not in str(exc.value)
@pytest.mark.parametrize(("model", "data", "field"), MODEL_CASES[:4])
def test_invalid_signed_urls_have_sanitized_validation_messages(model, data, field):
"""URL validation still rejects invalid URLs without quoting their value."""
with pytest.raises(ValidationError) as exc:
model.model_validate({**data, field: f"not-a-url?signature={CANARY}"})
assert CANARY not in str(exc.value)
assert CANARY not in str(
exc.value.errors(include_input=False, include_context=False)
)
assert exc.value.errors()[0]["type"] == "url_parsing"
assert exc.value.errors()[0]["input"] == "[redacted]"
def test_nested_task_urls_survive_broker_serialization():
"""Nested secrets round-trip through Celery's JSON serializer as real URLs."""
payload = TranscribeTaskJob(
user_sub="user",
language="en",
cloud_storage_url=SIGNED_URL,
tenant_id="tenant",
received_at="2026-01-01T10:00:00Z",
metadata=METADATA,
push_to_docs_config=DOCS_CONFIG,
)
assert CANARY not in repr(payload)
assert CANARY not in payload.model_dump_json()
content_type, encoding, body = dumps(dump_for_transport(payload), serializer="json")
restored = TranscribeTaskJob.model_validate(loads(body, content_type, encoding))
assert restored.cloud_storage_url.get_secret_value() == SIGNED_URL
assert restored.metadata.cloud_storage_url.get_secret_value() == SIGNED_URL
assert restored.push_to_docs_config.download_link.get_secret_value() == SIGNED_URL
assert restored.push_to_docs_config.form_link.get_secret_value() == SIGNED_URL
def test_transcription_worker_uses_real_urls(monkeypatch):
"""The worker unwraps input URLs and publishes usable signed result URLs."""
transcription = WhisperXResponse(segments=())
transcribe = Mock(return_value=transcription)
monkeypatch.setattr(celery_worker, "transcribe_audio", transcribe)
monkeypatch.setattr(celery_worker, "_should_push_to_docs", Mock(return_value=True))
monkeypatch.setattr(
celery_worker, "_should_auto_create_summary", Mock(return_value=False)
)
formatter = Mock(return_value="Formatted transcript")
monkeypatch.setattr(celery_worker, "format_transcript", formatter)
monkeypatch.setattr(celery_worker, "create_document_in_lasuite_docs", Mock())
storage = Mock()
storage.get_transcript_signed_url.return_value = SIGNED_URL
monkeypatch.setattr(celery_worker, "file_service", storage)
monkeypatch.setattr(celery_worker, "metadata_manager", Mock())
publish = Mock()
monkeypatch.setattr(celery_worker.call_webhook_v2_task, "apply_async", publish)
task = celery_worker.process_audio_transcribe_v2_task
monkeypatch.setattr(task.request, "id", "job")
result = task.run(
{
"user_sub": "user",
"language": "en",
"cloud_storage_url": SIGNED_URL,
"tenant_id": "tenant",
"received_at": "2026-01-01T10:00:00Z",
"push_to_docs_config": DOCS_CONFIG,
}
)
assert transcribe.call_args.kwargs["cloud_storage_url"] == SIGNED_URL
assert formatter.call_args.args[3:] == (SIGNED_URL, SIGNED_URL)
assert result["transcription_data_url"] == SIGNED_URL
assert publish.call_args.kwargs["args"][0] == result
def test_recording_metadata_url_is_unwrapped(monkeypatch):
"""Speaker assignment fetches metadata using the real URL."""
storage = Mock()
monkeypatch.setattr(celery_worker, "file_service", storage)
resolver = Mock()
resolver.return_value.apply_to.return_value = {"segments": []}
monkeypatch.setattr(celery_worker, "resolve_speaker_identities", resolver)
celery_worker.resolve_speaker_identities_and_apply_to(
transcription=WhisperXResponse(segments=()),
recording_metadata=RecordingMetadata.model_validate(METADATA),
task_id="job",
)
storage.read_cloud_storage_json.assert_called_once_with(SIGNED_URL)
+209
View File
@@ -0,0 +1,209 @@
"""Regression tests for observability privacy boundaries."""
import logging
from unittest.mock import Mock
import pytest
import requests
from summary.core import analytics, celery_worker, webhook_service
from summary.core.llm_service import LLMException, LLMService
from summary.core.privacy import redact_url
from summary.core.sentry import before_breadcrumb, before_send
from summary.core.shared_models import SummarizeWebhookSuccessPayload
CANARY = "sensitive-canary"
URL = f"https://user:password@example.com/file?signature={CANARY}#fragment"
def test_url_redaction():
"""Credentials and signed query strings are removed."""
assert redact_url(URL) == "https://example.com/file?[redacted]"
def test_sentry_scrubs_events_and_breadcrumbs():
"""Untrusted exception messages and breadcrumb data never leave in events."""
event = {
"request": {"url": URL, "data": CANARY, "query_string": CANARY},
"exception": {
"values": [
{
"type": "ValueError",
"value": CANARY,
"stacktrace": {"frames": [{"vars": {"job_id": "job-1"}}]},
}
]
},
"extra": {"url": URL},
}
result = before_send(event, {})
assert CANARY not in str(result)
assert result["exception"]["values"][0]["type"] == "ValueError"
assert result["exception"]["values"][0]["stacktrace"]["frames"][0]["vars"] == {
"job_id": "job-1"
}
assert before_breadcrumb({"message": CANARY, "data": {"url": URL}}, {}) == {}
@pytest.mark.parametrize("scheme", ["http", "HTTP", "hTtP", "https", "HTTPS", "hTtPs"])
def test_sentry_url_scrubbing_is_case_insensitive(scheme):
"""Scheme casing cannot bypass redaction of URLs nested in event strings."""
event = {
"extra": {
"nested": [
{
"detail": (
f"prefix <{scheme}://user:password@example.com/Path"
f"?signature={CANARY}#fragment> suffix"
)
}
]
}
}
result = before_send(event, {})
assert result["extra"]["nested"][0]["detail"] == (
f"prefix <{scheme.lower()}://example.com/Path?[redacted]> suffix"
)
assert CANARY not in str(result)
def test_llm_exception_does_not_include_upstream_message(caplog):
"""LLM failures retain their category without logging upstream text."""
observability = Mock(is_enabled=False)
observability.get_openai_client.return_value.chat.completions.create.side_effect = (
ValueError(CANARY)
)
with caplog.at_level(logging.DEBUG), pytest.raises(LLMException) as exc:
LLMService(observability).call("system", CANARY, "test")
assert CANARY not in str(exc.value)
assert CANARY not in caplog.text
assert exc.value.__suppress_context__
@pytest.mark.parametrize("log_level", [logging.DEBUG, logging.INFO])
def test_webhook_content_is_only_logged_at_debug(
monkeypatch, caplog, settings_override, log_level
):
"""DEBUG retains response content; INFO and above only contain metadata."""
monkeypatch.setattr(webhook_service, "settings", settings_override)
response = Mock(status_code=200)
response.json.return_value = {"id": "document-123", "content": CANARY}
monkeypatch.setattr(webhook_service.requests, "post", Mock(return_value=response))
payload = SummarizeWebhookSuccessPayload(job_id="job", summary_data_url=URL)
with caplog.at_level(log_level):
webhook_service.call_webhook_v2(tenant_id="test-tenant", payload=payload)
assert (
webhook_service.requests.post.call_args.kwargs["json"]["summary_data_url"]
== URL
)
assert all(
CANARY not in record.getMessage()
for record in caplog.records
if record.levelno >= logging.INFO
)
if log_level == logging.DEBUG:
assert any(
record.levelno == logging.DEBUG and CANARY in record.getMessage()
for record in caplog.records
)
else:
assert CANARY not in caplog.text
assert "Document document-123 submitted (HTTP 200)" in caplog.text
def test_webhook_failure_keeps_retry_category(monkeypatch, settings_override):
"""Webhook exceptions remain retryable without upstream text."""
monkeypatch.setattr(webhook_service, "settings", settings_override)
monkeypatch.setattr(
webhook_service.requests,
"post",
Mock(side_effect=requests.RequestException(URL)),
)
payload = SummarizeWebhookSuccessPayload(job_id="job", summary_data_url=URL)
with pytest.raises(requests.RequestException) as exc:
webhook_service.call_webhook_v2(tenant_id="test-tenant", payload=payload)
assert CANARY not in str(exc.value)
assert exc.value.__suppress_context__
def test_analytics_failure_does_not_chain_payload(monkeypatch):
"""Analytics errors do not expose arbitrary client exception text."""
client = analytics.Analytics()
client._client = Mock()
client._client.capture.side_effect = ValueError(CANARY)
with pytest.raises(analytics.AnalyticsException) as exc:
client.capture("event", "user")
assert CANARY not in str(exc.value)
assert exc.value.__suppress_context__
@pytest.mark.parametrize("log_level", [logging.DEBUG, logging.INFO, logging.ERROR])
def test_celery_log_filter_respects_log_level(log_level):
"""Celery trace values remain available in DEBUG only."""
record = logging.LogRecord(
"celery.app.trace", log_level, "", 0, "%s", (CANARY,), None
)
record.data = {"name": "task", "id": "job", "return_value": URL}
assert celery_worker.CeleryLogPrivacyFilter().filter(record)
if log_level == logging.DEBUG:
assert CANARY in record.getMessage()
assert record.data["return_value"] == URL
else:
assert CANARY not in record.getMessage()
assert record.data == {}
def test_task_arguments_have_private_display_representations(monkeypatch):
"""The actual task arguments are delivered but their event repr is hidden."""
publish = Mock()
monkeypatch.setattr(celery_worker.Task, "apply_async", publish)
task = celery_worker.PrivateTask()
task.apply_async(args=[CANARY], kwargs={"url": URL})
assert publish.call_args.kwargs["args"] == [CANARY]
assert publish.call_args.kwargs["argsrepr"] == "[Filtered]"
assert publish.call_args.kwargs["kwargsrepr"] == "[Filtered]"
def test_analytics_source_url_removes_signature():
"""Task analytics keeps the source path without credentials or signature."""
from datetime import datetime, timezone # noqa: PLC0415
from summary.core.models import TranscribeTaskJob # noqa: PLC0415
manager = analytics.MetadataManager()
manager._is_disabled = False
manager.has_task_id = Mock(return_value=False)
manager._save_metadata = Mock()
payload = TranscribeTaskJob(
user_sub="user",
tenant_id="tenant",
cloud_storage_url=URL,
language="en",
received_at=datetime(2026, 1, 1, tzinfo=timezone.utc),
)
manager.create("job", payload)
metadata = manager._save_metadata.call_args.args[1]
assert metadata["source_url"] == "https://example.com/file?[redacted]"
assert CANARY not in str(metadata)
def test_invalid_llm_json_is_sanitized():
"""Invalid structured LLM output retains a retryable LLM category."""
with pytest.raises(LLMException) as exc:
celery_worker.parse_llm_json(CANARY)
assert CANARY not in str(exc.value)
assert exc.value.__suppress_context__
@pytest.mark.parametrize(
("url", "expected"),
[
(None, "[redacted]"),
("https://[invalid?secret=value", "[redacted]"),
("https://example.com/path#fragment", "https://example.com/path"),
("https://example.com/path?", "https://example.com/path?[redacted]"),
],
)
def test_redact_url_edge_cases(url, expected):
"""Malformed and non-string URLs do not expose credentials or query data."""
assert redact_url(url) == expected
+136 -6
View File
@@ -1,27 +1,32 @@
"""Tests for the Sentry configuration.
Each test raises an error from code handling meeting content and inspects the
event Sentry would send: the content must be redacted, while harmless local
variables are kept for debugging.
event Sentry would send: sensitive content is redacted and useful locals remain.
"""
import json
import logging
from collections.abc import Callable, Iterator
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from threading import Thread
from unittest.mock import Mock
import httpx
import openai
import pytest
import requests
import sentry_sdk
from botocore.exceptions import ClientError
from botocore.stub import Stubber
from sentry_sdk.integrations.logging import LoggingIntegration
from sentry_sdk.integrations.stdlib import StdlibIntegration
from sentry_sdk.transport import Transport
from summary.core import file_service
from summary.core import file_service, webhook_service
from summary.core import sentry as sentry_module
from summary.core.file_service import FileService
from summary.core.file_service import FileService, FileServiceException
from summary.core.llm_service import LLMException, LLMService
from summary.core.shared_models import WhisperXResponse
from summary.core.shared_models import SummarizeWebhookSuccessPayload, WhisperXResponse
CANARY = "CANARY-MEETING-CONTENT"
@@ -51,8 +56,14 @@ def sentry_events() -> Iterator[SentryEvents]:
transport=transport,
send_default_pii=False,
include_local_variables=True,
before_send=sentry_module.before_send,
before_breadcrumb=sentry_module.before_breadcrumb,
event_scrubber=sentry_module.build_event_scrubber(),
default_integrations=False,
integrations=[
LoggingIntegration(level=logging.DEBUG, event_level=None),
StdlibIntegration(),
],
)
def flush() -> list[str]:
@@ -63,6 +74,125 @@ def sentry_events() -> Iterator[SentryEvents]:
sentry_sdk.init() # Disable Sentry for the following tests
@pytest.fixture
def missing_recording_url() -> Iterator[str]:
"""Serve a deterministic 404 through the real instrumented HTTP client."""
class MissingRecordingHandler(BaseHTTPRequestHandler):
def do_GET(self):
self.send_response(404)
self.end_headers()
def log_message(self, format, *args):
pass
with ThreadingHTTPServer(("127.0.0.1", 0), MissingRecordingHandler) as server:
thread = Thread(target=server.serve_forever, daemon=True)
thread.start()
try:
yield (
f"http://127.0.0.1:{server.server_port}/recordings/recording-123.mp4"
f"?X-Amz-Credential={CANARY}-credential"
f"&X-Amz-Signature={CANARY}-signature"
)
finally:
server.shutdown()
thread.join()
def test_sentry_download_404_excludes_signed_url(
sentry_events: SentryEvents, missing_recording_url: str, caplog
) -> None:
"""A failed download excludes signed URLs from logs, locals and breadcrumbs."""
with caplog.at_level(logging.INFO):
try:
FileService()._download_from_cloud_storage_url(missing_recording_url)
except FileServiceException as exc:
assert exc.__cause__ is None
assert exc.__suppress_context__
sentry_sdk.capture_exception()
else:
pytest.fail("the download should have failed")
[event] = sentry_events()
assert CANARY not in event
assert CANARY not in caplog.text
assert "/recordings/recording-123.mp4?[redacted]" in caplog.text
parsed = json.loads(event)
assert [item["type"] for item in parsed["exception"]["values"]] == [
"FileServiceException"
]
download_vars = next(
frame_vars for frame_vars in _local_vars(event) if "extension" in frame_vars
)
assert download_vars["extension"] == "'.mp4'"
assert download_vars["cloud_storage_url"] == "[Filtered]"
breadcrumbs = parsed["breadcrumbs"]["values"]
assert any(crumb.get("type") == "http" for crumb in breadcrumbs)
assert any(
crumb.get("category") == "summary.core.file_service" for crumb in breadcrumbs
)
assert all("message" not in crumb and "data" not in crumb for crumb in breadcrumbs)
def test_sentry_chained_http_error_excludes_signed_url(
sentry_events: SentryEvents, missing_recording_url: str
) -> None:
"""Sentry also filters signed URLs in explicitly chained HTTP errors."""
try:
try:
response = requests.get(missing_recording_url, timeout=5)
response.raise_for_status()
except requests.HTTPError as exc:
raise FileServiceException("Download failed") from exc
except FileServiceException:
sentry_sdk.capture_exception()
[event] = sentry_events()
assert CANARY not in event
exceptions = json.loads(event)["exception"]["values"]
assert [item["type"] for item in exceptions] == [
"HTTPError",
"FileServiceException",
]
assert all(item["value"] == "[Filtered]" for item in exceptions)
def test_sentry_excludes_webhook_debug_content(
sentry_events: SentryEvents, monkeypatch, settings_override, caplog
) -> None:
"""Raw DEBUG responses stay in application logs and out of Sentry events."""
monkeypatch.setattr(webhook_service, "settings", settings_override)
signed_url = f"https://storage.example.com/result?signature={CANARY}"
response = Mock(status_code=200)
response.json.return_value = {
"id": "document-123",
"content": CANARY,
"url": signed_url,
}
monkeypatch.setattr(webhook_service.requests, "post", Mock(return_value=response))
with caplog.at_level(logging.DEBUG):
webhook_service.call_webhook_v2(
tenant_id="test-tenant",
payload=SummarizeWebhookSuccessPayload(
job_id="job", summary_data_url=signed_url
),
)
try:
raise RuntimeError("A later failure")
except RuntimeError:
sentry_sdk.capture_exception()
assert CANARY in caplog.text
[event] = sentry_events()
assert CANARY not in event
assert any(
crumb.get("category") == "summary.core.webhook_service"
and crumb.get("level") == "debug"
for crumb in json.loads(event)["breadcrumbs"]["values"]
)
def _transcript() -> WhisperXResponse:
return WhisperXResponse.model_validate(
{
@@ -121,7 +251,7 @@ def s3_stubber(monkeypatch: pytest.MonkeyPatch) -> Iterator[Stubber]:
def test_sentry_store_transcript_failure_redacts_transcript(
sentry_events: SentryEvents, s3_stubber: Stubber
) -> None:
"""A failed S3 upload does not send the transcript, but keeps the job id."""
"""A failed S3 upload redacts the transcript while retaining useful locals."""
s3_stubber.add_client_error("put_object", service_error_code="InvalidDigest")
try:
@@ -0,0 +1,26 @@
"""Formatted validation errors hide input values, including after pickling."""
import pickle
import traceback
from pydantic import ValidationError
from summary.core.exceptions import CleanedValidationError
CANARY = "private-validation-canary"
def test_formatted_errors_hide_ordinary_input_after_pickle_roundtrip():
"""Formatting hides raw input even when diagnostics intentionally retain it."""
original = ValidationError.from_exception_data(
"TestModel",
[{"type": "int_parsing", "loc": ("field",), "input": CANARY}],
)
cleaned = CleanedValidationError(original)
restored = pickle.loads(pickle.dumps(cleaned)) # noqa: S301 -- own test exception
for error in (cleaned, restored):
assert error.errors()[0]["input"] == CANARY
assert CANARY not in str(error)
assert CANARY not in repr(error)
assert CANARY not in "".join(traceback.format_exception(error))
assert error.errors()[0]["type"] == "int_parsing"
@@ -0,0 +1,223 @@
"""Validation cleanup preserves diagnostics without retaining SecretStr values."""
import asyncio
import json
import pickle
import traceback
from unittest.mock import MagicMock, Mock
import pytest
from celery.exceptions import Retry
from celery.utils.serialization import get_pickleable_exception
from fastapi import Request
from fastapi.exceptions import RequestValidationError
from pydantic import SecretStr, ValidationError
from pydantic_core import PydanticCustomError
from summary.core import celery_worker
from summary.core.exceptions import CleanedValidationError, clean_validation_errors
from summary.main import validation_error_handler
CANARY = "private-validation-canary"
@pytest.mark.parametrize("custom", [False, True])
def test_cleaned_validation_error_preserves_details_and_pickles(custom):
"""Builtin and custom validation errors retain their type, location and message."""
error_type = (
PydanticCustomError(
"custom_validation", "Invalid value: {reason}", {"reason": "test"}
)
if custom
else "int_type"
)
original = ValidationError.from_exception_data(
"TestModel",
[
{"type": error_type, "loc": ("secret",), "input": SecretStr(CANARY)},
{"type": "int_parsing", "loc": ("ordinary",), "input": "not-an-int"},
],
)
cleaned = CleanedValidationError(original)
original_details = original.errors()
details = cleaned.errors()
assert isinstance(cleaned, ValidationError)
assert cleaned.title == original.title
assert cleaned.error_count() == original.error_count()
assert details[0] == {**original_details[0], "input": "[redacted]"}
assert details[1] == original_details[1]
assert isinstance(original.errors()[0]["input"], SecretStr)
assert CANARY not in str(cleaned)
assert CANARY not in cleaned.json()
serialized = pickle.dumps(cleaned)
assert CANARY.encode() not in serialized
restored = pickle.loads(serialized) # noqa: S301 -- own in-memory test exception
assert isinstance(restored, CleanedValidationError)
assert restored.errors() == details
assert isinstance(get_pickleable_exception(cleaned), CleanedValidationError)
def test_cleanup_masks_nested_secrets_and_preserves_plain_input():
"""Missing-field inputs keep ordinary values while masking nested secrets."""
original = ValidationError.from_exception_data(
"NestedModel",
[
{
"type": "missing",
"loc": ("required",),
"input": {
"ordinary": "keep me",
"nested": [SecretStr(CANARY), {"url": SecretStr(CANARY)}],
"tuple": (SecretStr(CANARY), 42),
},
}
],
)
cleaned = CleanedValidationError(original)
assert cleaned.errors()[0]["input"] == {
"ordinary": "keep me",
"nested": ["[redacted]", {"url": "[redacted]"}],
"tuple": ("[redacted]", 42),
}
assert CANARY not in cleaned.json()
assert CANARY.encode() not in pickle.dumps(cleaned)
def test_cleanup_does_not_retain_validator_exception_objects():
"""Validator context keeps its readable message without secret-bearing args."""
original = ValidationError.from_exception_data(
"TestModel",
[
{
"type": "value_error",
"loc": (),
"input": SecretStr(CANARY),
"ctx": {"error": ValueError(SecretStr(CANARY))},
}
],
)
cleaned = CleanedValidationError(original)
assert cleaned.errors()[0]["ctx"] == {"error": "**********"}
assert CANARY.encode() not in pickle.dumps(cleaned)
def test_clean_validation_errors_keeps_all_other_fields():
"""The API cleanup preserves diagnostics and nonsecret input."""
details = [
{
"type": "test",
"loc": ("body", "field"),
"msg": "Invalid value",
"url": "https://example.com/errors/test",
"ctx": {"limit": 3},
"input": "ordinary input",
}
]
assert clean_validation_errors(details) == details
assert (
json.loads(json.dumps(clean_validation_errors(details)))[0]["input"]
== "ordinary input"
)
@pytest.mark.parametrize(
"handler_name",
[
"call_webhook_v2_task",
"process_audio_transcribe_v2_task",
"summarize_v2_task",
"task_started_transcript",
"task_started_summary",
],
)
@pytest.mark.parametrize("secret_wrapped", [False, True])
def test_task_validation_uses_cleaned_errors(handler_name, secret_wrapped):
"""Task entry points and prerun signals consistently suppress the original error."""
handler = getattr(celery_worker, handler_name)
signed_url = f"https://storage.example.com/file?signature={CANARY}"
value = SecretStr(signed_url) if secret_wrapped else signed_url
payload = {"cloud_storage_url": value, "content": value}
with pytest.raises(CleanedValidationError) as exc:
if handler_name.startswith("task_started"):
handler(task_id="job", args=[payload])
elif handler_name == "call_webhook_v2_task":
handler.run(
payload={
"type": "summary",
"status": "success",
"summary_data_url": value,
},
tenant_id="tenant",
)
elif handler_name == "summarize_v2_task":
# Exercise validation without triggering the summary task's retries.
handler._orig_run(payload=payload)
else:
handler.run(payload=payload)
assert exc.value.__suppress_context__
assert CANARY not in "".join(traceback.format_exception(exc.value))
assert CANARY not in exc.value.json()
assert CANARY.encode() not in pickle.dumps(exc.value)
def test_upstream_transcription_validation_hides_raw_input(monkeypatch):
"""Invalid ASR responses do not remain in structured validation errors."""
storage = MagicMock()
storage.prepare_audio_file.return_value.__enter__.return_value = (
Mock(),
{"duration": 1},
)
monkeypatch.setattr(celery_worker, "file_service", storage)
monkeypatch.setattr(celery_worker, "metadata_manager", Mock())
response = Mock(status_code=200)
response.json.return_value = {"text": {"content": CANARY}}
monkeypatch.setattr(celery_worker.requests, "post", Mock(return_value=response))
with pytest.raises(CleanedValidationError) as exc:
celery_worker.transcribe_audio(
task_id="job", language="en", cloud_storage_url="https://example.com/audio"
)
assert exc.value.errors()[0]["input"] == "[redacted]"
assert CANARY not in exc.value.json()
assert CANARY.encode() not in pickle.dumps(exc.value)
def test_api_error_handler_uses_secret_aware_cleanup():
"""API errors mask typed secrets while retaining other values and context."""
errors = [
{
"type": "test",
"loc": ("body", "field"),
"msg": "Invalid value",
"input": {"url": SecretStr(CANARY), "ordinary": "keep me"},
"ctx": {"limit": 3},
"url": "https://example.com/errors/test",
},
]
response = asyncio.run(
validation_error_handler(
Request({"type": "http"}), RequestValidationError(errors)
)
)
assert response.status_code == 422
assert CANARY.encode() not in response.body
assert json.loads(response.body)["detail"] == [
{
"type": "test",
"loc": ["body", "field"],
"msg": "Invalid value",
"input": {"url": "[redacted]", "ordinary": "keep me"},
"ctx": {"limit": 3},
"url": "https://example.com/errors/test",
},
]
def test_summary_task_retries_with_the_cleaned_exception(monkeypatch):
"""The summary task still retries validation failures using the safe exception."""
retry = Mock(side_effect=Retry())
monkeypatch.setattr(celery_worker.summarize_v2_task, "retry", retry)
with pytest.raises(Retry):
celery_worker.summarize_v2_task.run(payload={"received_at": SecretStr(CANARY)})
error = retry.call_args.kwargs["exc"]
assert isinstance(error, CleanedValidationError)
assert CANARY.encode() not in pickle.dumps(error)
+1 -1
View File
@@ -1484,7 +1484,7 @@ wheels = [
[[package]]
name = "summary"
version = "1.36.2"
version = "1.36.3"
source = { editable = "." }
dependencies = [
{ name = "boto3" },