chore: add workshop-llm LiteLLM proxy deploy + ignore vendored/generated files

Commits the workshop inference pool (LiteLLM → boards: pooled coding plans,
per-attendee virtual keys, failover) — config + docs only, no secrets. Ignores
the vendored Uno-QClaw reference repo and the generated qr-sheet.html.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
Omar Sobh
2026-07-16 11:42:05 -07:00
co-authored by Claude Opus 4.8
parent 62c435c648
commit 766f2946af
7 changed files with 283 additions and 0 deletions
+5
View File
@@ -16,6 +16,11 @@ dist-ssr
deploy/uno-q/kit-codes/ deploy/uno-q/kit-codes/
deploy/uno-q/fleet.csv deploy/uno-q/fleet.csv
deploy/lan/.env deploy/lan/.env
deploy/workshop-llm/.env
# Vendored reference repo (QClaw — its own git repo, never commit) + generated output
/Uno-QClaw/
/qr-sheet.html
# Editor directories and files # Editor directories and files
.vscode/* .vscode/*
+20
View File
@@ -0,0 +1,20 @@
# Copy to .env and fill in. NEVER commit the real .env.
# ── Backbone: paid coding plans (direct API tokens — no shim needed) ──
MOONSHOT_API_KEY=sk-xxxxxxxx # Kimi Code — platform.moonshot.ai
ZAI_API_KEY=xxxxxxxx # GLM Coding plan — z.ai
# ── Overflow tier ──
GROQ_API_KEY=gsk_xxxxxxxx # console.groq.com (paid/dev tier)
GEMINI_API_KEY=AIzaxxxxxxxx # aistudio.google.com (free tier is fine)
# ── Instructor Claude (your subscription, via claude_shim.py on the host) ──
# Start the shim on the proxy host: python claude_shim.py 8090 sonnet
# From inside the container the host is reachable as host.docker.internal.
CLAUDE_SHIM_URL=http://host.docker.internal:8090/v1
CLAUDE_SHIM_KEY=shim-ignored
# ── Proxy control plane ──
LITELLM_MASTER_KEY=sk-master-CHANGE-ME # admin key: mint/revoke attendee keys
POSTGRES_PASSWORD=CHANGE-ME
DATABASE_URL=postgresql://litellm:CHANGE-ME@db:5432/litellm
+83
View File
@@ -0,0 +1,83 @@
# Workshop inference pool (LiteLLM → ZeroClaw / Uno Q)
One OpenAI-compatible proxy the boards talk to. It pools the **paid coding
plans you already hold** and hands each attendee a scoped, temporary key — no
per-attendee signup, no GPUs to rent, automatic failover.
```
Uno Q board ──(virtual key)──▶ LiteLLM :4000 ──▶ Kimi K2 + GLM (load-balanced)
custom.cloud │ fallbacks ──▶ Groq → Gemini (overflow)
model="workshop" └─ instructor-claude ─▶ claude_shim (instructor only)
```
## What's here
| File | Purpose |
|------|---------|
| `litellm-config.yaml` | Model pool, load-balancing, fallback chain |
| `docker-compose.yml` | LiteLLM proxy + Postgres (virtual keys / budgets) |
| `.env.example` | API tokens + proxy secrets (copy to `.env`) |
| `gen-attendee-keys.sh` | Mint N temporary per-attendee keys → CSV |
| `zeroclaw-board.toml` | The block each board pastes into its config |
## Run it
```bash
cp .env.example .env # fill in MOONSHOT/ZAI/GROQ/GEMINI keys + secrets
# (instructor Claude only) start the shim on the host:
python /path/to/claude_shim.py 8090 sonnet &
docker compose up -d # proxy on :4000, admin UI on :4000/ui
# mint 30 attendee keys (workshop model only, $3 cap, 12 rpm, 24h expiry)
LITELLM_MASTER_KEY=$(grep LITELLM_MASTER_KEY .env | cut -d= -f2) \
./gen-attendee-keys.sh 30 > attendee-keys.csv
```
Then on each board, paste `zeroclaw-board.toml` into `~/.zeroclaw/config.toml`,
set `PROXY_HOST`, and drop in one key from `attendee-keys.csv`. Attendees use
`--agent cloud`; `--agent local` still runs the on-board model offline.
## Model names the boards can ask for
| `model =` | Routes to | Who |
|-----------|-----------|-----|
| `workshop` | Kimi + GLM, load-balanced → Groq → Gemini | **everyone** |
| `instructor-claude` | Claude subscription via `claude_shim` | **instructor only** |
## Terms boundary (important)
Anthropic **prohibits routing Pro/Max subscription credentials on behalf of
other users**, and enforced it in Jan 2026 (OpenClaw, OpenCode, Roo Code, Goose
were blocked for exactly this). So:
- `instructor-claude` is **only** for boards you personally drive. The attendee
keys minted by `gen-attendee-keys.sh` are scoped to `["workshop"]`, so the
proxy **refuses** attendee requests for Claude — the boundary is enforced at
the gateway, not just by convention.
- For Claude quality *for attendees*, use the metered **Anthropic API** as a
backend instead of the subscription shim.
- GLM and Kimi coding plans are **per-seat** too — buy a few cheap seats and let
LiteLLM load-balance them rather than hammering one login.
## Verify
```bash
# proxy sees the models
curl -s http://localhost:4000/v1/models -H "Authorization: Bearer $LITELLM_MASTER_KEY"
# an attendee key answers (and is capped to `workshop`)
curl -s http://localhost:4000/v1/chat/completions \
-H "Authorization: Bearer sk-attendee-01" -H "Content-Type: application/json" \
-d '{"model":"workshop","messages":[{"role":"user","content":"one word: ok?"}]}'
```
## Notes
- **Model ids drift.** Kimi/GLM rev names often — check `kimi-k2.7-code` and
`glm-4.6` (or your plan's current name) before the event.
- **`native_tools = false`** on the board is deliberate: it routes tool calls
through ZeroClaw's text parser, whose `` ```json ``-fence fix handles every
model in the pool uniformly (see `osobh/zeroclaw#4`).
- **Fallback is the resilience story.** Kill a backend and watch LiteLLM reroute
— the same L4/L5 failover the APESS platform demonstrates, one layer up.
+32
View File
@@ -0,0 +1,32 @@
# LiteLLM proxy + Postgres (virtual keys / budgets / spend tracking).
# docker compose up -d
# → proxy on :4000, admin UI on :4000/ui
services:
litellm:
image: ghcr.io/berriai/litellm:main-latest
command: ["--config", "/app/config.yaml", "--port", "4000", "--num_workers", "4"]
ports:
- "4000:4000"
volumes:
- ./litellm-config.yaml:/app/config.yaml:ro
env_file:
- .env
extra_hosts:
# so the container can reach claude_shim.py running on the host
- "host.docker.internal:host-gateway"
depends_on:
- db
restart: unless-stopped
db:
image: postgres:16-alpine
environment:
POSTGRES_DB: litellm
POSTGRES_USER: litellm
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
volumes:
- litellm-pg:/var/lib/postgresql/data
restart: unless-stopped
volumes:
litellm-pg:
+33
View File
@@ -0,0 +1,33 @@
#!/usr/bin/env bash
# Mint N temporary per-attendee virtual keys on the LiteLLM proxy.
#
# Each key gets: access to the `workshop` model ONLY (never instructor-claude),
# a spend cap, a per-key rate limit, and an expiry. Prints CSV: alias,key.
#
# LITELLM_MASTER_KEY=sk-master-… ./gen-attendee-keys.sh 30 > attendee-keys.csv
#
# Env: PROXY (default http://localhost:4000), BUDGET usd, RPM, DUR.
set -euo pipefail
PROXY="${PROXY:-http://localhost:4000}"
MASTER="${LITELLM_MASTER_KEY:?set LITELLM_MASTER_KEY}"
N="${1:-30}"
BUDGET="${BUDGET:-3.0}" # USD/attendee — covers metered Groq/Gemini overflow
RPM="${RPM:-12}" # per-attendee requests/min
DUR="${DUR:-24h}" # temporary keys expire after the event
mint() { # alias, models-json -> key
curl -sf -X POST "$PROXY/key/generate" \
-H "Authorization: Bearer $MASTER" -H "Content-Type: application/json" \
-d "{\"models\":$2,\"max_budget\":$BUDGET,\"rpm_limit\":$RPM,\"key_alias\":\"$1\",\"duration\":\"$DUR\"}" \
| python3 -c 'import sys,json;print(json.load(sys.stdin)["key"])'
}
echo "alias,key"
for i in $(seq 1 "$N"); do
alias=$(printf "attendee-%02d" "$i")
echo "$alias,$(mint "$alias" '["workshop"]')"
done
# Instructor key (adds Claude access + bigger budget) — run once, separately:
# mint instructor '["workshop","instructor-claude"]' # with BUDGET=50 DUR=720h
+77
View File
@@ -0,0 +1,77 @@
# LiteLLM proxy — workshop inference pool for ZeroClaw / Uno Q boards.
#
# One OpenAI-compatible endpoint the boards talk to. It pools your paid coding
# plans (Kimi + GLM, load-balanced), overflows to Groq → Gemini on saturation,
# and keeps your Claude subscription on a separate, restricted model name.
#
# Board-facing model names:
# workshop → Kimi K2 + GLM (load-balanced) → Groq → Gemini fallback
# instructor-claude → your Claude subscription via claude_shim
# RESTRICTED: attendee virtual keys must NOT be granted
# this model (Anthropic ToS — see README). Instructor key
# only.
#
# Model ids below are examples — verify against your plan's current model names
# (Kimi/GLM rev them often) before the event.
model_list:
# ── Backbone: pooled coding plans (same model_name ⇒ load-balanced) ──
- model_name: workshop
litellm_params:
model: openai/kimi-k2.7-code
api_base: https://api.moonshot.ai/v1
api_key: os.environ/MOONSHOT_API_KEY
rpm: 1500 # Kimi allows ~30 concurrent; leave headroom
model_info:
id: kimi-workshop
- model_name: workshop
litellm_params:
model: openai/glm-4.6
api_base: https://api.z.ai/api/paas/v4 # coding plan: /api/coding/paas/v4
api_key: os.environ/ZAI_API_KEY
model_info:
id: glm-workshop
# ── Overflow / fallback tier (also load-balanced under one name) ──
- model_name: workshop-overflow
litellm_params:
model: groq/llama-3.3-70b-versatile
api_key: os.environ/GROQ_API_KEY
model_info:
id: groq-overflow
- model_name: workshop-overflow
litellm_params:
model: gemini/gemini-2.5-flash
api_key: os.environ/GEMINI_API_KEY
model_info:
id: gemini-overflow
# ── Instructor only: Claude subscription via the claude_shim on the host ──
# Do NOT list this model on any attendee key. See README "Terms boundary".
- model_name: instructor-claude
litellm_params:
model: openai/sonnet
api_base: os.environ/CLAUDE_SHIM_URL # e.g. http://host.docker.internal:8090/v1
api_key: os.environ/CLAUDE_SHIM_KEY # any non-empty string; the shim ignores it
model_info:
id: claude-instructor
router_settings:
routing_strategy: simple-shuffle # balance the two "workshop" deployments
num_retries: 2
timeout: 120
allowed_fails: 2
cooldown_time: 30
fallbacks:
- workshop: ["workshop-overflow"] # Kimi/GLM exhausted → Groq/Gemini
litellm_settings:
drop_params: true # tolerate per-provider param differences
request_timeout: 120
set_verbose: false
general_settings:
master_key: os.environ/LITELLM_MASTER_KEY
database_url: os.environ/DATABASE_URL # Postgres → virtual keys, budgets, spend
+33
View File
@@ -0,0 +1,33 @@
# ── Workshop board wiring — paste into a board's ~/.zeroclaw/config.toml ──
#
# Points the `cloud` agent at the workshop LiteLLM proxy. The proxy pools
# Kimi + GLM and falls back to Groq → Gemini, so the board just asks for one
# model name ("workshop") and never sees the routing.
#
# Replace PROXY_HOST with the proxy's address (IP or DNS) and paste the
# attendee's virtual key from gen-attendee-keys.sh.
[providers.models.custom.cloud]
uri = "http://PROXY_HOST:4000/v1"
api_key = "sk-attendee-XX" # attendee's virtual key
model = "workshop" # Kimi+GLM load-balanced → Groq/Gemini fallback
native_tools = false # use ZeroClaw's text tool-call parser
# (the ```json-fence fix handles the pool's models)
[agents.cloud]
enabled = true
model_provider = "custom.cloud"
risk_profile = "default"
runtime_profile = "unoq"
# ── Instructor / demo board only ──────────────────────────────────────────
# Swap model + key to reach Claude via the shim. Use ONLY with the instructor
# virtual key (the one granted "instructor-claude"). Do not ship this to
# attendee boards — see README "Terms boundary".
#
# [providers.models.custom.cloud]
# uri = "http://PROXY_HOST:4000/v1"
# api_key = "sk-instructor-…"
# model = "instructor-claude"
# native_tools = false