bench: make the CUDA embedding path discoverable when it is unavailable
CI / test (push) Failing after 2s

The GPU path worked but was effectively hidden. cudarc's build script shells out
to `nvcc`, which ships in /usr/local/cuda/bin — a directory the reference host
had installed but never exported to the login shell, so `--features
embeddings-cuda` failed with a bare "`nvcc --version` failed" panic from a
dependency's build script, and the runtime fallback then reported only
"Embedder: CPU (...)" before spending hours on work a GPU does in minutes.

Two changes, both about making the failure legible rather than changing what the
code does:

  - The CPU fallback now says why it fell back and what that costs, with the
    concrete fix. A run that silently takes two orders of magnitude longer reads
    as a hang, not as a configuration choice.
  - BENCHMARKS.md states the build-time nvcc requirement, where the toolkit
    actually installs, and that a shell file read non-interactively is the place
    to export it — `~/.zshenv` rather than `~/.zshrc`, because build scripts do
    not run in an interactive shell.

Host-side, the reference machine's CUDA exports lived in ~/.bashrc below its
non-interactive guard while the login shell is zsh, so they never applied to
anything. Moved to ~/.zshenv with duplicate-prepend guards; `nvcc --version`
and `cargo build --features embeddings-cuda` now both work over a plain
non-interactive ssh with no manual export.
This commit is contained in:
Omar Sobh
2026-08-07 08:13:59 -07:00
parent c913cd1cbf
commit 12d9d8462f
2 changed files with 18 additions and 3 deletions
@@ -43,7 +43,13 @@ impl Embedder {
d
}
Err(e) => {
eprintln!("Embedder: CPU ({e})");
// Loud, because the CPU path is correct but ~100x slower: the
// full longmemeval_s haystack is minutes on a GPU and most of a
// day on 8 cores. Silently falling back looks like a hang.
eprintln!("Embedder: CPU — CUDA unavailable ({e})");
eprintln!(
" WARNING: CPU embedding is roughly two orders of magnitude slower.\n Expect minutes for longmemeval_oracle and many hours for the full\n longmemeval_s haystack. For the GPU path, rebuild with\n `--features embeddings-cuda` and make sure `nvcc` is on PATH\n (it ships in /usr/local/cuda/bin, which is often not exported)."
);
Device::Cpu
}
};