LLM Proxy¶

Klangk runs an nginx reverse proxy in front of the FastAPI backend. Nginx serves the Flutter web UI, proxies API and WebSocket traffic to uvicorn, proxies hosted app URLs directly to container ports (keeping the Python backend out of that hot path), and provides the LLM proxy described below. Using nginx also enables auth_request-based JWT validation on container-to-host endpoints without adding middleware overhead to every backend request.

Pi containers access the LLM via the LLM proxy, an nginx location that proxies /llm-proxy/ requests to ${KLANGK_LLM_BASE_URL}. This is required because:

1. Pi is inside a container, LLM is on the host: Pi containers can't reach localhost:11434 (self-hosted Ollama) directly. They use host.containers.internal to reach the host, but the host's nginx serves the proxy.
2. API key security: The API key is sent in a request header by the nginx proxy rather than being baked into the container image or passed as an env var. The container's models.json contains only the proxy URL (no real API key).
3. No per-container LLM config: The backend injects KLANGK_LLM_PROXY_URL=http://host.containers.internal:<nginx_port>/llm-proxy into each container. setup_clankers.py uses this to write Pi's models.json with the proxy URL and the workspace JWT as the API key (nginx validates the JWT via auth_request before replacing it with the real API key). KLANGK_LLM_BASE_URL is only used by nginx itself.

The nginx config is generated by scripts/nginx.sh and includes:

location /llm-proxy/ {
    auth_request /auth/verify-workspace-token;
    proxy_pass $KLANGK_LLM_BASE_URL/;
    proxy_set_header Authorization "Bearer $KLANGK_LLM_API_KEY";
    proxy_ssl_server_name on;
}

In CI, devenv processes up -d starts nginx before E2E tests run.