veda/apps/litellm

LiteLLM

LiteLLM is the cluster-local LLM proxy for OpenWebUI. It routes requests to the external llama.cpp OpenAI-compatible endpoint at http://framework-llm.open-webui.svc.cluster.local:8080/v1.

The public dashboard is exposed at https://litellm.noxxos.nl through LiteLLM's native Authentik SSO. API clients should use LiteLLM bearer keys.

Secrets

Create the LiteLLM secrets before syncing. The Helm chart references these secrets directly, so the LiteLLM pod will fail with secret "litellm-sso" not found until the SSO secret exists in the litellm namespace.

Create the LiteLLM master key and salt:

kubectl create namespace litellm
kubectl -n litellm create secret generic litellm-secrets \
  --from-literal=master-key='sk-<litellm-master-key>' \
  --from-literal=salt-key='<random-high-entropy-salt>'

OpenWebUI needs a copy of the LiteLLM key in its own namespace:

kubectl -n open-webui create secret generic litellm-secrets \
  --from-literal=openwebui-api-keys='0p3n-w3bu!;sk-<litellm-master-key>'

openwebui-api-keys is semicolon-separated because OpenWebUI is configured with two OpenAI-compatible endpoints: its internal Pipelines service first, then LiteLLM.

Create an Authentik OAuth2/OIDC provider and application for LiteLLM.

External URL:

https://litellm.noxxos.nl

Allowed redirect URI:

https://litellm.noxxos.nl/sso/callback

Launch URL:

https://litellm.noxxos.nl/ui

Include the scopes openid profile email litellm_role.

Create an Authentik scope mapping named litellm_role:

if ak_is_group_member(request.user, name="admins"):
    return {"litellm_role": "proxy_admin"}
return {"litellm_role": "internal_user"}

Then create the LiteLLM SSO secret before syncing the app:

kubectl -n litellm create secret generic litellm-sso \
  --from-literal=GENERIC_CLIENT_SECRET='<authentik-client-secret>'