(agent) support Voxtral realtime as inference engine

The Kyutai open-source model turned out not to be production-ready:
it caused disruptions in the production environment, especially on
long-running meeting sessions.

Switch to the Voxtral realtime model, which looks like a
credible competitor and behaves much better in our setup.

For now, the code handling the Voxtral realtime API lives directly
in the project. It could be extracted into an open-source package
later.

See PR #1277 for the full details of the implementation, proposed
by @cameldev.
This commit is contained in:
leo
2026-05-19 18:35:25 +02:00
committed by aleb_the_flash
parent c7e3168ba3
commit 0a0cdae896
8 changed files with 1328 additions and 539 deletions
+13 -4
View File
@@ -1,14 +1,23 @@
AWS_S3_ENDPOINT_URL=minio:9000
AWS_S3_ACCESS_KEY_ID=meet
AWS_S3_SECRET_ACCESS_KEY=password
LIVEKIT_URL=ws://livekit:7880
LIVEKIT_API_KEY=devkey
LIVEKIT_API_SECRET=secret
STT_PROVIDER=kyutai # kyutai, deepgram
STT_PROVIDER=voxtral-vllm # voxtral-vllm, kyutai, deepgram
ENABLE_SILERO_VAD=False
DEEPGRAM_API_KEY=
DEEPGRAM_API_KEY=your-deepgram-api-key
KYUTAI_STT_BASE_URL=
KYUTAI_API_KEY=
KYUTAI_STT_BASE_URL=url
KYUTAI_API_KEY=your-kyutai-api-key
VOXTRAL_VLLM_BASE_URL=wss://<host>/v1/realtime
VOXTRAL_VLLM_MODEL=voxtral-mini-4b-realtime-2602
VOXTRAL_VLLM_API_KEY=your-vllm-api-key
VOXTRAL_VLLM_TARGET_STREAMING_DELAY_MS=480
SENTRY_DSN=
SENTRY_ENVIRONMENT=