72 lines
1.9 KiB
Markdown
72 lines
1.9 KiB
Markdown
# LiteLLM
|
|
|
|
LiteLLM is the cluster-local LLM proxy for OpenWebUI. It routes requests to the
|
|
external llama.cpp OpenAI-compatible endpoint at
|
|
`http://framework-llm.open-webui.svc.cluster.local:8080/v1`.
|
|
|
|
The public dashboard is exposed at `https://litellm.noxxos.nl` through
|
|
LiteLLM's native Authentik SSO. API clients should use LiteLLM bearer keys.
|
|
|
|
## Secrets
|
|
|
|
Create the LiteLLM secrets before syncing. The Helm chart references these
|
|
secrets directly, so the LiteLLM pod will fail with `secret "litellm-sso" not
|
|
found` until the SSO secret exists in the `litellm` namespace.
|
|
|
|
Create the LiteLLM master key and salt:
|
|
|
|
```bash
|
|
kubectl create namespace litellm
|
|
kubectl -n litellm create secret generic litellm-secrets \
|
|
--from-literal=master-key='sk-<litellm-master-key>' \
|
|
--from-literal=salt-key='<random-high-entropy-salt>'
|
|
```
|
|
|
|
OpenWebUI needs a copy of the LiteLLM key in its own namespace:
|
|
|
|
```bash
|
|
kubectl -n open-webui create secret generic litellm-secrets \
|
|
--from-literal=openwebui-api-keys='0p3n-w3bu!;sk-<litellm-master-key>'
|
|
```
|
|
|
|
`openwebui-api-keys` is semicolon-separated because OpenWebUI is configured with
|
|
two OpenAI-compatible endpoints: its internal Pipelines service first, then
|
|
LiteLLM.
|
|
|
|
Create an Authentik OAuth2/OIDC provider and application for LiteLLM.
|
|
|
|
External URL:
|
|
|
|
```text
|
|
https://litellm.noxxos.nl
|
|
```
|
|
|
|
Allowed redirect URI:
|
|
|
|
```text
|
|
https://litellm.noxxos.nl/sso/callback
|
|
```
|
|
|
|
Launch URL:
|
|
|
|
```text
|
|
https://litellm.noxxos.nl/ui
|
|
```
|
|
|
|
Include the scopes `openid profile email litellm_role`.
|
|
|
|
Create an Authentik scope mapping named `litellm_role`:
|
|
|
|
```python
|
|
if ak_is_group_member(request.user, name="admins"):
|
|
return {"litellm_role": "proxy_admin"}
|
|
return {"litellm_role": "internal_user"}
|
|
```
|
|
|
|
Then create the LiteLLM SSO secret before syncing the app:
|
|
|
|
```bash
|
|
kubectl -n litellm create secret generic litellm-sso \
|
|
--from-literal=GENERIC_CLIENT_SECRET='<authentik-client-secret>'
|
|
```
|