bench: make the CUDA embedding path discoverable when it is unavailable
CI / test (push) Failing after 2s
CI / test (push) Failing after 2s
The GPU path worked but was effectively hidden. cudarc's build script shells out
to `nvcc`, which ships in /usr/local/cuda/bin — a directory the reference host
had installed but never exported to the login shell, so `--features
embeddings-cuda` failed with a bare "`nvcc --version` failed" panic from a
dependency's build script, and the runtime fallback then reported only
"Embedder: CPU (...)" before spending hours on work a GPU does in minutes.
Two changes, both about making the failure legible rather than changing what the
code does:
- The CPU fallback now says why it fell back and what that costs, with the
concrete fix. A run that silently takes two orders of magnitude longer reads
as a hang, not as a configuration choice.
- BENCHMARKS.md states the build-time nvcc requirement, where the toolkit
actually installs, and that a shell file read non-interactively is the place
to export it — `~/.zshenv` rather than `~/.zshrc`, because build scripts do
not run in an interactive shell.
Host-side, the reference machine's CUDA exports lived in ~/.bashrc below its
non-interactive guard while the login shell is zsh, so they never applied to
anything. Moved to ~/.zshenv with duplicate-prepend guards; `nvcc --version`
and `cargo build --features embeddings-cuda` now both work over a plain
non-interactive ssh with no manual export.
This commit is contained in:
@@ -43,7 +43,13 @@ impl Embedder {
|
||||
d
|
||||
}
|
||||
Err(e) => {
|
||||
eprintln!("Embedder: CPU ({e})");
|
||||
// Loud, because the CPU path is correct but ~100x slower: the
|
||||
// full longmemeval_s haystack is minutes on a GPU and most of a
|
||||
// day on 8 cores. Silently falling back looks like a hang.
|
||||
eprintln!("Embedder: CPU — CUDA unavailable ({e})");
|
||||
eprintln!(
|
||||
" WARNING: CPU embedding is roughly two orders of magnitude slower.\n Expect minutes for longmemeval_oracle and many hours for the full\n longmemeval_s haystack. For the GPU path, rebuild with\n `--features embeddings-cuda` and make sure `nvcc` is on PATH\n (it ships in /usr/local/cuda/bin, which is often not exported)."
|
||||
);
|
||||
Device::Cpu
|
||||
}
|
||||
};
|
||||
|
||||
Reference in New Issue
Block a user