LiteLLM

LiteLLM is the cluster-local LLM proxy for OpenWebUI. It routes requests to the external llama.cpp OpenAI-compatible endpoint at http://framework-llm:8080/v1.

The public dashboard is exposed at https://litellm.noxxos.nl through LiteLLM's native Authentik SSO. API clients should use LiteLLM bearer keys.

Secrets

Create the LiteLLM secret before syncing:

kubectl create namespace litellm
kubectl -n litellm create secret generic litellm-secrets \
  --from-literal=master-key='sk-<litellm-master-key>' \
  --from-literal=salt-key='<random-high-entropy-salt>'

OpenWebUI needs a copy of the LiteLLM key in its own namespace:

kubectl -n open-webui create secret generic litellm-secrets \
  --from-literal=openwebui-api-keys='0p3n-w3bu!;sk-<litellm-master-key>'

openwebui-api-keys is semicolon-separated because OpenWebUI is configured with two OpenAI-compatible endpoints: its internal Pipelines service first, then LiteLLM.

Create an Authentik OAuth2/OIDC provider and application for LiteLLM.

External URL:

https://litellm.noxxos.nl

Allowed redirect URI:

https://litellm.noxxos.nl/sso/callback

Launch URL:

https://litellm.noxxos.nl/ui

Include the scopes openid profile email litellm_role.

Create an Authentik scope mapping named litellm_role:

if ak_is_group_member(request.user, name="admins"):
    return {"litellm_role": "proxy_admin"}
return {"litellm_role": "internal_user"}

Create the LiteLLM SSO secret:

kubectl -n litellm create secret generic litellm-sso \
  --from-literal=GENERIC_CLIENT_SECRET='<authentik-client-secret>'

1.6 KiB Raw Blame History

LiteLLM

Secrets

1.6 KiB

Raw Blame History