Skip to main content

LiteLLM

LiteLLM Proxy ships an Alice guardrail. It forwards each prompt before the model call and each completion after it; Alice returns ALLOW, BLOCK, MASK or DETECT and the proxy enforces it. Which policies apply is decided per virtual key, so one proxy serves many applications.

Status: Generally available · Evaluates: Prompts, Responses, Tool calls · Vendor: BerriAI

Prompts (pre_call) and responses (post_call).

Setup​

pip install "litellm[proxy]"  # v1.101.0 or later

Configuration: ALICE_API_KEY, ALICE_API_BASE (optional), alice_app_id in key metadata

  1. Add the guardrail: alice block to config.yaml (LiteLLM v1.101.0 or later) and start the proxy.
  2. Generate one virtual key per application with alice_app_id in its metadata (the key alias is the fallback).
  3. Call the proxy with that key. Blocked requests return HTTP 400.

Example​

model_list:
    - model_name: gpt-4o
      litellm_params:
          model: openai/gpt-4o
          api_key: os.environ/OPENAI_API_KEY

guardrails:
    - guardrail_name: alice
      litellm_params:
          guardrail: alice
          mode: [pre_call, post_call]
          default_on: true
          api_key: os.environ/ALICE_API_KEY
# litellm --config config.yaml
#
# curl -sSLX POST 'http://0.0.0.0:4000/key/generate' \
# --header "Authorization: Bearer $LITELLM_API_KEY" \
# --header 'Content-Type: application/json' \
# --data '{"key_alias": "payments-bot", "metadata": {"alice_app_id": "payments-bot"}}'

Good to know​

unreachable_fallback defaults to fail_closed. Masking does not apply to streamed responses; blocking does. Sensitive request fields are stripped before anything reaches Alice.