# LiteLLM

LiteLLM is the cluster-local LLM proxy for OpenWebUI. It routes requests to the
external llama.cpp OpenAI-compatible endpoint at `http://framework-llm:8080/v1`.

The public dashboard is exposed at `https://litellm.noxxos.nl` through
OAuth2 Proxy and Authentik. API clients should use LiteLLM bearer keys.

## Secrets

Create the LiteLLM secret before syncing:

```bash
kubectl create namespace litellm
kubectl -n litellm create secret generic litellm-secrets \
  --from-literal=master-key='sk-<litellm-master-key>' \
  --from-literal=salt-key='<random-high-entropy-salt>'
```

OpenWebUI needs a copy of the LiteLLM key in its own namespace:

```bash
kubectl -n open-webui create secret generic litellm-secrets \
  --from-literal=openwebui-api-keys='0p3n-w3bu!;sk-<litellm-master-key>'
```

`openwebui-api-keys` is semicolon-separated because OpenWebUI is configured with
two OpenAI-compatible endpoints: its internal Pipelines service first, then
LiteLLM.

Create an Authentik OAuth2/OIDC provider and application for OAuth2 Proxy.

External URL:

```text
https://litellm.noxxos.nl
```

Allowed redirect URI:

```text
https://litellm.noxxos.nl/oauth2/callback
```

Create the OAuth2 Proxy secret:

```bash
kubectl -n litellm create secret generic litellm-oauth2-proxy \
  --from-literal=client-id='<authentik-client-id>' \
  --from-literal=client-secret='<authentik-client-secret>' \
  --from-literal=cookie-secret='<32-byte-base64-cookie-secret>'
```

Generate a cookie secret with:

```bash
openssl rand -base64 32 | head -c 32 | base64
```