chore: add workshop-llm LiteLLM proxy deploy + ignore vendored/generated files
Commits the workshop inference pool (LiteLLM → boards: pooled coding plans, per-attendee virtual keys, failover) — config + docs only, no secrets. Ignores the vendored Uno-QClaw reference repo and the generated qr-sheet.html. Co-Authored-By: Claude Opus 4.8 <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
62c435c648
commit
766f2946af
@@ -16,6 +16,11 @@ dist-ssr
|
|||||||
deploy/uno-q/kit-codes/
|
deploy/uno-q/kit-codes/
|
||||||
deploy/uno-q/fleet.csv
|
deploy/uno-q/fleet.csv
|
||||||
deploy/lan/.env
|
deploy/lan/.env
|
||||||
|
deploy/workshop-llm/.env
|
||||||
|
|
||||||
|
# Vendored reference repo (QClaw — its own git repo, never commit) + generated output
|
||||||
|
/Uno-QClaw/
|
||||||
|
/qr-sheet.html
|
||||||
|
|
||||||
# Editor directories and files
|
# Editor directories and files
|
||||||
.vscode/*
|
.vscode/*
|
||||||
|
|||||||
@@ -0,0 +1,20 @@
|
|||||||
|
# Copy to .env and fill in. NEVER commit the real .env.
|
||||||
|
|
||||||
|
# ── Backbone: paid coding plans (direct API tokens — no shim needed) ──
|
||||||
|
MOONSHOT_API_KEY=sk-xxxxxxxx # Kimi Code — platform.moonshot.ai
|
||||||
|
ZAI_API_KEY=xxxxxxxx # GLM Coding plan — z.ai
|
||||||
|
|
||||||
|
# ── Overflow tier ──
|
||||||
|
GROQ_API_KEY=gsk_xxxxxxxx # console.groq.com (paid/dev tier)
|
||||||
|
GEMINI_API_KEY=AIzaxxxxxxxx # aistudio.google.com (free tier is fine)
|
||||||
|
|
||||||
|
# ── Instructor Claude (your subscription, via claude_shim.py on the host) ──
|
||||||
|
# Start the shim on the proxy host: python claude_shim.py 8090 sonnet
|
||||||
|
# From inside the container the host is reachable as host.docker.internal.
|
||||||
|
CLAUDE_SHIM_URL=http://host.docker.internal:8090/v1
|
||||||
|
CLAUDE_SHIM_KEY=shim-ignored
|
||||||
|
|
||||||
|
# ── Proxy control plane ──
|
||||||
|
LITELLM_MASTER_KEY=sk-master-CHANGE-ME # admin key: mint/revoke attendee keys
|
||||||
|
POSTGRES_PASSWORD=CHANGE-ME
|
||||||
|
DATABASE_URL=postgresql://litellm:CHANGE-ME@db:5432/litellm
|
||||||
@@ -0,0 +1,83 @@
|
|||||||
|
# Workshop inference pool (LiteLLM → ZeroClaw / Uno Q)
|
||||||
|
|
||||||
|
One OpenAI-compatible proxy the boards talk to. It pools the **paid coding
|
||||||
|
plans you already hold** and hands each attendee a scoped, temporary key — no
|
||||||
|
per-attendee signup, no GPUs to rent, automatic failover.
|
||||||
|
|
||||||
|
```
|
||||||
|
Uno Q board ──(virtual key)──▶ LiteLLM :4000 ──▶ Kimi K2 + GLM (load-balanced)
|
||||||
|
custom.cloud │ fallbacks ──▶ Groq → Gemini (overflow)
|
||||||
|
model="workshop" └─ instructor-claude ─▶ claude_shim (instructor only)
|
||||||
|
```
|
||||||
|
|
||||||
|
## What's here
|
||||||
|
|
||||||
|
| File | Purpose |
|
||||||
|
|------|---------|
|
||||||
|
| `litellm-config.yaml` | Model pool, load-balancing, fallback chain |
|
||||||
|
| `docker-compose.yml` | LiteLLM proxy + Postgres (virtual keys / budgets) |
|
||||||
|
| `.env.example` | API tokens + proxy secrets (copy to `.env`) |
|
||||||
|
| `gen-attendee-keys.sh` | Mint N temporary per-attendee keys → CSV |
|
||||||
|
| `zeroclaw-board.toml` | The block each board pastes into its config |
|
||||||
|
|
||||||
|
## Run it
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cp .env.example .env # fill in MOONSHOT/ZAI/GROQ/GEMINI keys + secrets
|
||||||
|
# (instructor Claude only) start the shim on the host:
|
||||||
|
python /path/to/claude_shim.py 8090 sonnet &
|
||||||
|
|
||||||
|
docker compose up -d # proxy on :4000, admin UI on :4000/ui
|
||||||
|
|
||||||
|
# mint 30 attendee keys (workshop model only, $3 cap, 12 rpm, 24h expiry)
|
||||||
|
LITELLM_MASTER_KEY=$(grep LITELLM_MASTER_KEY .env | cut -d= -f2) \
|
||||||
|
./gen-attendee-keys.sh 30 > attendee-keys.csv
|
||||||
|
```
|
||||||
|
|
||||||
|
Then on each board, paste `zeroclaw-board.toml` into `~/.zeroclaw/config.toml`,
|
||||||
|
set `PROXY_HOST`, and drop in one key from `attendee-keys.csv`. Attendees use
|
||||||
|
`--agent cloud`; `--agent local` still runs the on-board model offline.
|
||||||
|
|
||||||
|
## Model names the boards can ask for
|
||||||
|
|
||||||
|
| `model =` | Routes to | Who |
|
||||||
|
|-----------|-----------|-----|
|
||||||
|
| `workshop` | Kimi + GLM, load-balanced → Groq → Gemini | **everyone** |
|
||||||
|
| `instructor-claude` | Claude subscription via `claude_shim` | **instructor only** |
|
||||||
|
|
||||||
|
## Terms boundary (important)
|
||||||
|
|
||||||
|
Anthropic **prohibits routing Pro/Max subscription credentials on behalf of
|
||||||
|
other users**, and enforced it in Jan 2026 (OpenClaw, OpenCode, Roo Code, Goose
|
||||||
|
were blocked for exactly this). So:
|
||||||
|
|
||||||
|
- `instructor-claude` is **only** for boards you personally drive. The attendee
|
||||||
|
keys minted by `gen-attendee-keys.sh` are scoped to `["workshop"]`, so the
|
||||||
|
proxy **refuses** attendee requests for Claude — the boundary is enforced at
|
||||||
|
the gateway, not just by convention.
|
||||||
|
- For Claude quality *for attendees*, use the metered **Anthropic API** as a
|
||||||
|
backend instead of the subscription shim.
|
||||||
|
- GLM and Kimi coding plans are **per-seat** too — buy a few cheap seats and let
|
||||||
|
LiteLLM load-balance them rather than hammering one login.
|
||||||
|
|
||||||
|
## Verify
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# proxy sees the models
|
||||||
|
curl -s http://localhost:4000/v1/models -H "Authorization: Bearer $LITELLM_MASTER_KEY"
|
||||||
|
|
||||||
|
# an attendee key answers (and is capped to `workshop`)
|
||||||
|
curl -s http://localhost:4000/v1/chat/completions \
|
||||||
|
-H "Authorization: Bearer sk-attendee-01" -H "Content-Type: application/json" \
|
||||||
|
-d '{"model":"workshop","messages":[{"role":"user","content":"one word: ok?"}]}'
|
||||||
|
```
|
||||||
|
|
||||||
|
## Notes
|
||||||
|
|
||||||
|
- **Model ids drift.** Kimi/GLM rev names often — check `kimi-k2.7-code` and
|
||||||
|
`glm-4.6` (or your plan's current name) before the event.
|
||||||
|
- **`native_tools = false`** on the board is deliberate: it routes tool calls
|
||||||
|
through ZeroClaw's text parser, whose `` ```json ``-fence fix handles every
|
||||||
|
model in the pool uniformly (see `osobh/zeroclaw#4`).
|
||||||
|
- **Fallback is the resilience story.** Kill a backend and watch LiteLLM reroute
|
||||||
|
— the same L4/L5 failover the APESS platform demonstrates, one layer up.
|
||||||
@@ -0,0 +1,32 @@
|
|||||||
|
# LiteLLM proxy + Postgres (virtual keys / budgets / spend tracking).
|
||||||
|
# docker compose up -d
|
||||||
|
# → proxy on :4000, admin UI on :4000/ui
|
||||||
|
services:
|
||||||
|
litellm:
|
||||||
|
image: ghcr.io/berriai/litellm:main-latest
|
||||||
|
command: ["--config", "/app/config.yaml", "--port", "4000", "--num_workers", "4"]
|
||||||
|
ports:
|
||||||
|
- "4000:4000"
|
||||||
|
volumes:
|
||||||
|
- ./litellm-config.yaml:/app/config.yaml:ro
|
||||||
|
env_file:
|
||||||
|
- .env
|
||||||
|
extra_hosts:
|
||||||
|
# so the container can reach claude_shim.py running on the host
|
||||||
|
- "host.docker.internal:host-gateway"
|
||||||
|
depends_on:
|
||||||
|
- db
|
||||||
|
restart: unless-stopped
|
||||||
|
|
||||||
|
db:
|
||||||
|
image: postgres:16-alpine
|
||||||
|
environment:
|
||||||
|
POSTGRES_DB: litellm
|
||||||
|
POSTGRES_USER: litellm
|
||||||
|
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
|
||||||
|
volumes:
|
||||||
|
- litellm-pg:/var/lib/postgresql/data
|
||||||
|
restart: unless-stopped
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
litellm-pg:
|
||||||
Executable
+33
@@ -0,0 +1,33 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Mint N temporary per-attendee virtual keys on the LiteLLM proxy.
|
||||||
|
#
|
||||||
|
# Each key gets: access to the `workshop` model ONLY (never instructor-claude),
|
||||||
|
# a spend cap, a per-key rate limit, and an expiry. Prints CSV: alias,key.
|
||||||
|
#
|
||||||
|
# LITELLM_MASTER_KEY=sk-master-… ./gen-attendee-keys.sh 30 > attendee-keys.csv
|
||||||
|
#
|
||||||
|
# Env: PROXY (default http://localhost:4000), BUDGET usd, RPM, DUR.
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
PROXY="${PROXY:-http://localhost:4000}"
|
||||||
|
MASTER="${LITELLM_MASTER_KEY:?set LITELLM_MASTER_KEY}"
|
||||||
|
N="${1:-30}"
|
||||||
|
BUDGET="${BUDGET:-3.0}" # USD/attendee — covers metered Groq/Gemini overflow
|
||||||
|
RPM="${RPM:-12}" # per-attendee requests/min
|
||||||
|
DUR="${DUR:-24h}" # temporary keys expire after the event
|
||||||
|
|
||||||
|
mint() { # alias, models-json -> key
|
||||||
|
curl -sf -X POST "$PROXY/key/generate" \
|
||||||
|
-H "Authorization: Bearer $MASTER" -H "Content-Type: application/json" \
|
||||||
|
-d "{\"models\":$2,\"max_budget\":$BUDGET,\"rpm_limit\":$RPM,\"key_alias\":\"$1\",\"duration\":\"$DUR\"}" \
|
||||||
|
| python3 -c 'import sys,json;print(json.load(sys.stdin)["key"])'
|
||||||
|
}
|
||||||
|
|
||||||
|
echo "alias,key"
|
||||||
|
for i in $(seq 1 "$N"); do
|
||||||
|
alias=$(printf "attendee-%02d" "$i")
|
||||||
|
echo "$alias,$(mint "$alias" '["workshop"]')"
|
||||||
|
done
|
||||||
|
|
||||||
|
# Instructor key (adds Claude access + bigger budget) — run once, separately:
|
||||||
|
# mint instructor '["workshop","instructor-claude"]' # with BUDGET=50 DUR=720h
|
||||||
@@ -0,0 +1,77 @@
|
|||||||
|
# LiteLLM proxy — workshop inference pool for ZeroClaw / Uno Q boards.
|
||||||
|
#
|
||||||
|
# One OpenAI-compatible endpoint the boards talk to. It pools your paid coding
|
||||||
|
# plans (Kimi + GLM, load-balanced), overflows to Groq → Gemini on saturation,
|
||||||
|
# and keeps your Claude subscription on a separate, restricted model name.
|
||||||
|
#
|
||||||
|
# Board-facing model names:
|
||||||
|
# workshop → Kimi K2 + GLM (load-balanced) → Groq → Gemini fallback
|
||||||
|
# instructor-claude → your Claude subscription via claude_shim
|
||||||
|
# RESTRICTED: attendee virtual keys must NOT be granted
|
||||||
|
# this model (Anthropic ToS — see README). Instructor key
|
||||||
|
# only.
|
||||||
|
#
|
||||||
|
# Model ids below are examples — verify against your plan's current model names
|
||||||
|
# (Kimi/GLM rev them often) before the event.
|
||||||
|
|
||||||
|
model_list:
|
||||||
|
# ── Backbone: pooled coding plans (same model_name ⇒ load-balanced) ──
|
||||||
|
- model_name: workshop
|
||||||
|
litellm_params:
|
||||||
|
model: openai/kimi-k2.7-code
|
||||||
|
api_base: https://api.moonshot.ai/v1
|
||||||
|
api_key: os.environ/MOONSHOT_API_KEY
|
||||||
|
rpm: 1500 # Kimi allows ~30 concurrent; leave headroom
|
||||||
|
model_info:
|
||||||
|
id: kimi-workshop
|
||||||
|
|
||||||
|
- model_name: workshop
|
||||||
|
litellm_params:
|
||||||
|
model: openai/glm-4.6
|
||||||
|
api_base: https://api.z.ai/api/paas/v4 # coding plan: /api/coding/paas/v4
|
||||||
|
api_key: os.environ/ZAI_API_KEY
|
||||||
|
model_info:
|
||||||
|
id: glm-workshop
|
||||||
|
|
||||||
|
# ── Overflow / fallback tier (also load-balanced under one name) ──
|
||||||
|
- model_name: workshop-overflow
|
||||||
|
litellm_params:
|
||||||
|
model: groq/llama-3.3-70b-versatile
|
||||||
|
api_key: os.environ/GROQ_API_KEY
|
||||||
|
model_info:
|
||||||
|
id: groq-overflow
|
||||||
|
|
||||||
|
- model_name: workshop-overflow
|
||||||
|
litellm_params:
|
||||||
|
model: gemini/gemini-2.5-flash
|
||||||
|
api_key: os.environ/GEMINI_API_KEY
|
||||||
|
model_info:
|
||||||
|
id: gemini-overflow
|
||||||
|
|
||||||
|
# ── Instructor only: Claude subscription via the claude_shim on the host ──
|
||||||
|
# Do NOT list this model on any attendee key. See README "Terms boundary".
|
||||||
|
- model_name: instructor-claude
|
||||||
|
litellm_params:
|
||||||
|
model: openai/sonnet
|
||||||
|
api_base: os.environ/CLAUDE_SHIM_URL # e.g. http://host.docker.internal:8090/v1
|
||||||
|
api_key: os.environ/CLAUDE_SHIM_KEY # any non-empty string; the shim ignores it
|
||||||
|
model_info:
|
||||||
|
id: claude-instructor
|
||||||
|
|
||||||
|
router_settings:
|
||||||
|
routing_strategy: simple-shuffle # balance the two "workshop" deployments
|
||||||
|
num_retries: 2
|
||||||
|
timeout: 120
|
||||||
|
allowed_fails: 2
|
||||||
|
cooldown_time: 30
|
||||||
|
fallbacks:
|
||||||
|
- workshop: ["workshop-overflow"] # Kimi/GLM exhausted → Groq/Gemini
|
||||||
|
|
||||||
|
litellm_settings:
|
||||||
|
drop_params: true # tolerate per-provider param differences
|
||||||
|
request_timeout: 120
|
||||||
|
set_verbose: false
|
||||||
|
|
||||||
|
general_settings:
|
||||||
|
master_key: os.environ/LITELLM_MASTER_KEY
|
||||||
|
database_url: os.environ/DATABASE_URL # Postgres → virtual keys, budgets, spend
|
||||||
@@ -0,0 +1,33 @@
|
|||||||
|
# ── Workshop board wiring — paste into a board's ~/.zeroclaw/config.toml ──
|
||||||
|
#
|
||||||
|
# Points the `cloud` agent at the workshop LiteLLM proxy. The proxy pools
|
||||||
|
# Kimi + GLM and falls back to Groq → Gemini, so the board just asks for one
|
||||||
|
# model name ("workshop") and never sees the routing.
|
||||||
|
#
|
||||||
|
# Replace PROXY_HOST with the proxy's address (IP or DNS) and paste the
|
||||||
|
# attendee's virtual key from gen-attendee-keys.sh.
|
||||||
|
|
||||||
|
[providers.models.custom.cloud]
|
||||||
|
uri = "http://PROXY_HOST:4000/v1"
|
||||||
|
api_key = "sk-attendee-XX" # attendee's virtual key
|
||||||
|
model = "workshop" # Kimi+GLM load-balanced → Groq/Gemini fallback
|
||||||
|
native_tools = false # use ZeroClaw's text tool-call parser
|
||||||
|
# (the ```json-fence fix handles the pool's models)
|
||||||
|
|
||||||
|
[agents.cloud]
|
||||||
|
enabled = true
|
||||||
|
model_provider = "custom.cloud"
|
||||||
|
risk_profile = "default"
|
||||||
|
runtime_profile = "unoq"
|
||||||
|
|
||||||
|
|
||||||
|
# ── Instructor / demo board only ──────────────────────────────────────────
|
||||||
|
# Swap model + key to reach Claude via the shim. Use ONLY with the instructor
|
||||||
|
# virtual key (the one granted "instructor-claude"). Do not ship this to
|
||||||
|
# attendee boards — see README "Terms boundary".
|
||||||
|
#
|
||||||
|
# [providers.models.custom.cloud]
|
||||||
|
# uri = "http://PROXY_HOST:4000/v1"
|
||||||
|
# api_key = "sk-instructor-…"
|
||||||
|
# model = "instructor-claude"
|
||||||
|
# native_tools = false
|
||||||
Reference in New Issue
Block a user