feat(llm): the fallback chain's last link runs on our own hardware
`local:ornith-fleet:9b` joins opus -> haiku -> glm as the final link. Every entry above it depends on somebody else's account staying funded and unthrottled; this one depends on a GPU in the next room. It is last because it is the weakest model, and present because a chain whose every link is external is not a fallback chain, it is one outage in a trench coat. Three small changes make it work: - `build_provider_registry` accepts a provider with an empty `api_key_env`. A model on our own hardware has nothing to authenticate to, and the old behaviour SKIPPED a keyless provider — leaving the chain quietly one link shorter than it reads, which is the failure mode this whole area keeps producing. - `provider_family` learns `ornith`/`ollama` for BARE names. A qualified `local:` spec was already answered by the split, but a bare one fell through to "unknown", and `cross_provider_judge` would then refuse a judge that is genuinely a different family from the Anthropic implementer. - A test pins that the last link survives `resolve_provider`'s split-on-FIRST- colon: `local:ornith-fleet:9b` is provider `local`, model `ornith-fleet:9b`. Splitting on the last colon would ask for a provider named `local:ornith-fleet`, and the symptom would be a silent fall back to the default provider. Infra: Ollama on tank and architect now binds 0.0.0.0 so the gateway (which has no GPU) can reach it. `tailscale serve` cannot — Ollama rejects a non-local Host header as a DNS-rebinding guard and OLLAMA_ORIGINS is CORS-only, so it 403s. 0.0.0.0 still includes loopback, so the microVM vsock pipe is unaffected; verified on both nodes. This is an explicit trade: Ollama has no auth and its API can pull and delete models, so it is now reachable from the LAN as well as the tailnet. The drop-in carries the ufw one-liner to close the LAN side. Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
co-authored by
Claude Opus 5
parent
774f17d194
commit
b18e62041b
@@ -95,8 +95,21 @@ fn build_provider(config: &AppConfig) -> Result<Arc<dyn LlmProvider>, String> {
|
||||
fn build_provider_registry(config: &AppConfig) -> cm_runtime::ProviderRegistry {
|
||||
let mut map = std::collections::HashMap::new();
|
||||
for p in &config.llm.providers {
|
||||
match std::env::var(&p.api_key_env) {
|
||||
Ok(key) if !key.is_empty() => {
|
||||
// A provider may legitimately need no key. A model running on our own
|
||||
// hardware has nothing to authenticate to, and requiring a variable
|
||||
// whose value is ignored is a step that can only ever fail — silently,
|
||||
// since an unset key SKIPS the provider and the first symptom is a
|
||||
// fallback chain quietly one link shorter than it reads.
|
||||
let key = match std::env::var(&p.api_key_env) {
|
||||
Ok(k) if !k.is_empty() => Ok(k),
|
||||
other if p.api_key_env.trim().is_empty() => {
|
||||
let _ = other;
|
||||
Ok(String::new())
|
||||
}
|
||||
other => other,
|
||||
};
|
||||
match key {
|
||||
Ok(key) if !key.is_empty() || p.api_key_env.trim().is_empty() => {
|
||||
let provider: Arc<dyn LlmProvider> = match p.format.as_str() {
|
||||
"anthropic" => Arc::new(cm_llm::AnthropicProvider::with_base_url(
|
||||
key,
|
||||
|
||||
Reference in New Issue
Block a user