vibecoding/podx
1
0
Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

GPU fixes

Browse Source
This commit is contained in:
Tomas Kracmar
2025-10-05 14:18:54 +02:00
parent c89964694f
commit c5b92ad960
4 changed files with 34 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
README.md
View File

@@ -27,11 +27,12 @@ To run Whisper on NVIDIA GPU:
- `NVIDIA_VISIBLE_DEVICES=all` (or a specific GPU index)
- `WHISPER_DEVICE=cuda` (or `auto`)
- `WHISPER_PRECISION=float16` (recommended for GPU)
- Rebuild and start:
- `docker compose up -d --build`
- Optional: set a GPU base image for builds (amd64 typical):
- `GPU_BASE_IMAGE=nvidia/cuda:11.8.0-cudnn8-runtime-ubuntu22.04`
- Rebuild and start: `docker compose up -d --build`
- Check logs for `device='cuda'` when the transcribe worker loads the model.
This repo's app image is based on `nvidia/cuda:12.4.1-cudnn9-runtime-ubuntu22.04`, which includes the CUDA and cuDNN user-space libraries that faster-whisper requires. On non-GPU hosts it still runs on CPU.
By default we build from `python:3.11-slim`. You can override the base image at build time via `GPU_BASE_IMAGE` to a CUDA runtime tag that exists for your architecture. If you don't specify it or you're on a non-GPU host, the containers run on CPU.
## Components Overview