Windows 11 · NVIDIA GPU · Docker Desktop + WSL2
Set up Jozie AI on your Windows PC
A complete path from a fresh Windows machine to a running Jozie AI instance with GPU-accelerated fine-tuning on your NVIDIA card — every command exactly as you'll type it, in order.
Overview
Docker doesn't talk to your GPU directly on Windows — the real path is Windows → WSL2 (a real Linux kernel) → Docker Desktop → your NVIDIA driver's WSL2 support. Every step below exists to get one link of that chain working, in the order that makes failures easiest to diagnose: if step 4 fails, you know it's steps 1–3, not the app.
By the end: the full app running in Docker, your GPU visible inside it, and a real LoRA fine-tuning job trained on your dataset.
Requirements
Windows 11, or 10 22H2+
Check: Settings → System → About → "Windows specifications".
NVIDIA GPU
Any RTX card with a reasonably current driver works — this guide was written with an RTX 5070 in mind.
Administrator access
Needed for WSL2 and Docker Desktop installation.
~20 GB free disk
Docker images, base models, and the WSL2 virtual disk all add up.
Enable WSL2
Open PowerShell as Administrator (right-click Start → Terminal (Admin)) and run:
PS> wsl --installThis installs WSL2 and a default Ubuntu distribution. Restart your PC when it finishes — WSL2 doesn't fully activate until you do.
After restarting, verify it's actually version 2 (not the older WSL1):
PS> wsl -l -vYou should see your distro listed with VERSION = 2. If it says 1, run wsl --set-version <distro-name> 2.
Already had WSL2 installed before? Just run wsl --update to make sure it's current, then continue to step 2.
Check your NVIDIA driver
Modern NVIDIA drivers (Game Ready or Studio) already include WSL2 CUDA support built in — there's no separate "CUDA on WSL" driver to install anymore. You just need a reasonably current one.
PS> nvidia-smiThis should print your GPU, driver version, and CUDA version — run from regular Windows PowerShell, confirming the driver itself is installed correctly before WSL2/Docker enter the picture at all. If this fails, update your driver via NVIDIA's driver download page or GeForce Experience first.
Install Docker Desktop
Download and run the installer from docker.com. During install, keep "Use WSL 2 instead of Hyper-V" checked (it's the default).
After installing
- Open Docker Desktop, go to Settings → General, confirm "Use the WSL 2 based engine" is checked.
- Go to Settings → Resources → WSL Integration, confirm integration is enabled for your default distro.
- Click Apply & Restart if you changed anything.
Confirm it's running from PowerShell:
PS> docker --version
PS> docker compose versionVerify GPU passthrough
This is the checkpoint that matters most — it proves the entire chain (driver → WSL2 → Docker Desktop) works, before involving this app at all. Docker Desktop bundles everything needed for NVIDIA GPU passthrough on Windows; there's no separate "NVIDIA Container Toolkit" to install here (that's a bare-Linux-only extra step, not a Windows one).
PS> docker run --rm --gpus=all nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smiThe first run downloads a small image, then prints your GPU — the same table
nvidia-smi showed in step 2, just now from inside a container.
If this fails, don't move on — it's a Docker Desktop/WSL2/driver configuration issue, not something in this app, and every later step depends on it. See Troubleshooting below.
Install Ollama
Ollama runs natively on Windows (not in Docker) so it has direct access to your GPU — install it from ollama.com, then pull the two models the app expects by default:
PS> ollama pull llama3.1:8b
PS> ollama pull embeddinggemmaLeave Ollama running in the background — the containerized app reaches it automatically.
Get the app & configure
Copy the project folder onto this machine, then open PowerShell inside it.
PS> copy .env.docker.example .env
Open the new .env in a text editor and set a real JWT_SECRET
before doing anything beyond a local trial — the placeholder value is
intentionally insecure.
Start the core stack
Backend, worker, frontend, PostgreSQL+pgvector, and Redis — this is CPU-only so far, fine-tuning comes next.
PS> docker compose up -d --buildThe first run builds images and can take several minutes. Watch progress with:
PS> docker compose logs -f backend workerOnce it settles, open http://localhost — you should see the sign-in screen.
Enable GPU fine-tuning The part that needs your GPU
A Compose override file swaps just the worker
service to a CUDA-enabled image with your GPU attached — everything else from
step 7 keeps running unchanged.
PS> docker compose -f docker-compose.yml -f docker-compose.gpu.yml up -d --buildThis rebuilds and restarts just the worker container. Confirm the GPU is visible to it:
PS> docker compose exec worker nvidia-smiExpect first-run friction here. This CUDA training path
(Unsloth-based) was built and packaged without access to real NVIDIA
hardware to test against — every other part of this app was verified live
before shipping, this one genuinely wasn't. If docker compose logs
worker shows a Python error once you start a real fine-tuning job in
step 9, that's expected first-real-run debugging, not a sign of a deeper
problem — it's usually a quick library-version fix once the exact error is
visible.
First walkthrough
- Open http://localhost, click Create org, register your account.
- Knowledge Base → upload a document → wait for
cleaned. - Datasets → create a dataset → Generate from the document → approve at least 6 examples in the review queue.
- Fine-Tuning → confirm the badge in the top-right of "Start a fine-tuning job" says NVIDIA CUDA (proves the GPU worker is active) → pick the smallest base model (
unsloth/Llama-3.2-1B-Instruct-bnb-4bit) → start the job. - Watch it move through training → fusing → GGUF conversion → Ollama import. A working small run should complete in a few minutes on an RTX 5070.
- Once it's done, the model is a normal Ollama tag — try it immediately in Conversation Testing or Evaluation.
Troubleshooting
Ordered by which layer they belong to — work top to bottom if you're not sure where the problem is.
"error during connect... failed to connect to the Docker API at npipe:////"›
Docker Desktop isn't running, or hasn't finished starting yet — this is almost always the whole story, not a real configuration problem. In order:
- Check the system tray for the Docker whale icon. If it's missing, launch Docker Desktop from the Start menu. If it's animating, wait — first launch after install can take a minute or two.
- Once the tray icon's tooltip says "Engine running," retry the command.
- Still stuck? Quit Docker Desktop entirely (right-click the tray icon → Quit), run
wsl --shutdownin PowerShell, wait ~10 seconds, then relaunch Docker Desktop. - Check
docker context ls— the active context (marked*) should bedesktop-linux. Switch withdocker context use desktop-linuxif it isn't. - If Docker Desktop won't open at all, re-check step 3's settings (WSL 2 based engine, WSL Integration) — an interrupted install can leave those unset.
Step 4's nvidia-smi container test fails or hangs›
This means the driver ↔ WSL2 ↔ Docker Desktop chain isn't connected yet. Check, in order:
- Docker Desktop → Settings → General → "Use the WSL 2 based engine" is checked
- Docker Desktop → Settings → Resources → WSL Integration → your distro is enabled
wsl -l -vshows version2, not1, for your distro- Your NVIDIA driver is current — reinstall it if it was installed before you set up WSL2
- Restart Docker Desktop (right-click its tray icon → Restart), or restart Windows entirely — WSL2/driver updates sometimes need a full reboot to take effect
Docker Desktop won't start, or WSL2 errors on launch›
Usually a virtualization setting. Confirm virtualization is enabled in your PC's BIOS/UEFI (usually called "Intel VT-x" or "AMD-V"), and that Windows features "Virtual Machine Platform" and "Windows Subsystem for Linux" are both turned on (search "Turn Windows features on or off").
Port 80 or 8000 already in use›
Something else on this machine (IIS, Skype, another dev server) is holding that port. Either stop it, or edit the port mapping on the left side of the ports: lines in docker-compose.yml (e.g. change "80:80" to "8080:80") and use that port instead.
docker compose logs worker shows a Python/CUDA error during fine-tuning›
This is the expected first-run risk called out in step 8. Copy the full error text — it's almost always either a package version conflict (fixed by adjusting a pin in backend/requirements-cuda.txt) or a small API argument mismatch in backend/app/services/finetune_scripts/cuda_train.py or cuda_fuse.py. Report it back with the exact traceback for a fast fix.
Base model download seems stuck or very slow›
Base models are pulled from Hugging Face on first use (hundreds of MB to a few GB) and cached in a Docker volume so it only happens once per model. Check docker compose logs worker for actual download progress rather than assuming it's hung.
Ollama isn't reachable from the app (Dashboard shows it as unreachable)›
Confirm Ollama is actually running on Windows (check for its tray icon, or run ollama list in PowerShell). The containerized backend reaches it via host.docker.internal, which Docker Desktop wires up automatically — no manual network configuration should be needed.
Command reference
| What | Command |
|---|---|
| Start everything (CPU stack) | docker compose up -d --build |
| Start with GPU fine-tuning | docker compose -f docker-compose.yml -f docker-compose.gpu.yml up -d --build |
| Tail backend/worker logs | docker compose logs -f backend worker |
| Check GPU is visible to the worker | docker compose exec worker nvidia-smi |
| Stop everything, keep data | docker compose down |
| Stop everything, delete all data | docker compose down -v |
| Rebuild after pulling code changes | docker compose up -d --build |