feat(llm): the fallback chain's last link runs on our own hardware

`local:ornith-fleet:9b` joins opus -> haiku -> glm as the final link. Every
entry above it depends on somebody else's account staying funded and
unthrottled; this one depends on a GPU in the next room. It is last because it
is the weakest model, and present because a chain whose every link is external
is not a fallback chain, it is one outage in a trench coat.

Three small changes make it work:

- `build_provider_registry` accepts a provider with an empty `api_key_env`.
  A model on our own hardware has nothing to authenticate to, and the old
  behaviour SKIPPED a keyless provider — leaving the chain quietly one link
  shorter than it reads, which is the failure mode this whole area keeps
  producing.
- `provider_family` learns `ornith`/`ollama` for BARE names. A qualified
  `local:` spec was already answered by the split, but a bare one fell through
  to "unknown", and `cross_provider_judge` would then refuse a judge that is
  genuinely a different family from the Anthropic implementer.
- A test pins that the last link survives `resolve_provider`'s split-on-FIRST-
  colon: `local:ornith-fleet:9b` is provider `local`, model `ornith-fleet:9b`.
  Splitting on the last colon would ask for a provider named
  `local:ornith-fleet`, and the symptom would be a silent fall back to the
  default provider.

Infra: Ollama on tank and architect now binds 0.0.0.0 so the gateway (which has
no GPU) can reach it. `tailscale serve` cannot — Ollama rejects a non-local Host
header as a DNS-rebinding guard and OLLAMA_ORIGINS is CORS-only, so it 403s.
0.0.0.0 still includes loopback, so the microVM vsock pipe is unaffected;
verified on both nodes. This is an explicit trade: Ollama has no auth and its
API can pull and delete models, so it is now reachable from the LAN as well as
the tailnet. The drop-in carries the ufw one-liner to close the LAN side.

Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
Omar Sobh
2026-08-09 14:13:05 -07:00
co-authored by Claude Opus 5
parent 774f17d194
commit b18e62041b
3 changed files with 49 additions and 4 deletions
+15 -2
View File
@@ -95,8 +95,21 @@ fn build_provider(config: &AppConfig) -> Result<Arc<dyn LlmProvider>, String> {
fn build_provider_registry(config: &AppConfig) -> cm_runtime::ProviderRegistry {
let mut map = std::collections::HashMap::new();
for p in &config.llm.providers {
match std::env::var(&p.api_key_env) {
Ok(key) if !key.is_empty() => {
// A provider may legitimately need no key. A model running on our own
// hardware has nothing to authenticate to, and requiring a variable
// whose value is ignored is a step that can only ever fail — silently,
// since an unset key SKIPS the provider and the first symptom is a
// fallback chain quietly one link shorter than it reads.
let key = match std::env::var(&p.api_key_env) {
Ok(k) if !k.is_empty() => Ok(k),
other if p.api_key_env.trim().is_empty() => {
let _ = other;
Ok(String::new())
}
other => other,
};
match key {
Ok(key) if !key.is_empty() || p.api_key_env.trim().is_empty() => {
let provider: Arc<dyn LlmProvider> = match p.format.as_str() {
"anthropic" => Arc::new(cm_llm::AnthropicProvider::with_base_url(
key,