bench: make the CUDA embedding path discoverable when it is unavailable
CI / test (push) Failing after 2s
CI / test (push) Failing after 2s
The GPU path worked but was effectively hidden. cudarc's build script shells out
to `nvcc`, which ships in /usr/local/cuda/bin — a directory the reference host
had installed but never exported to the login shell, so `--features
embeddings-cuda` failed with a bare "`nvcc --version` failed" panic from a
dependency's build script, and the runtime fallback then reported only
"Embedder: CPU (...)" before spending hours on work a GPU does in minutes.
Two changes, both about making the failure legible rather than changing what the
code does:
- The CPU fallback now says why it fell back and what that costs, with the
concrete fix. A run that silently takes two orders of magnitude longer reads
as a hang, not as a configuration choice.
- BENCHMARKS.md states the build-time nvcc requirement, where the toolkit
actually installs, and that a shell file read non-interactively is the place
to export it — `~/.zshenv` rather than `~/.zshrc`, because build scripts do
not run in an interactive shell.
Host-side, the reference machine's CUDA exports lived in ~/.bashrc below its
non-interactive guard while the login shell is zsh, so they never applied to
anything. Moved to ~/.zshenv with duplicate-prepend guards; `nvcc --version`
and `cargo build --features embeddings-cuda` now both work over a plain
non-interactive ssh with no manual export.
This commit is contained in:
+11
-2
@@ -300,8 +300,17 @@ matching, and MiniLM at 384 dimensions is a small embedding model.
|
|||||||
|
|
||||||
> **Run:** `cargo run --release --bin longmemeval_bench --features embeddings -- \
|
> **Run:** `cargo run --release --bin longmemeval_bench --features embeddings -- \
|
||||||
> benchmarks/longmemeval/longmemeval_s_cleaned.json --embeddings weights/all-minilm-l6-v2`
|
> benchmarks/longmemeval/longmemeval_s_cleaned.json --embeddings weights/all-minilm-l6-v2`
|
||||||
> Add `--features embeddings-cuda` (and put `nvcc` on `PATH`) for the GPU path.
|
> For the GPU path use `--features embeddings-cuda`. That requires `nvcc` on
|
||||||
> Weights: `huggingface.co/sentence-transformers/all-MiniLM-L6-v2`.
|
> `PATH` at *build* time — cudarc's build script shells out to it. The toolkit
|
||||||
|
> installs to `/usr/local/cuda/bin`, which many distributions do not export;
|
||||||
|
> check with `nvcc --version` and, if it is missing, add it somewhere every
|
||||||
|
> shell reads (for zsh that is `~/.zshenv`, not `~/.zshrc`, since build tooling
|
||||||
|
> runs non-interactively). The device is selected at runtime with a CPU
|
||||||
|
> fallback, so a machine without CUDA still produces correct numbers — just far
|
||||||
|
> more slowly, and the bench says so on startup.
|
||||||
|
>
|
||||||
|
> Weights: `huggingface.co/sentence-transformers/all-MiniLM-L6-v2` — place
|
||||||
|
> `model.safetensors` and `tokenizer.json` in the `--embeddings` directory.
|
||||||
|
|
||||||
### Oracle variant — `longmemeval_oracle`, n=500 (easier corpus, kept for continuity)
|
### Oracle variant — `longmemeval_oracle`, n=500 (easier corpus, kept for continuity)
|
||||||
|
|
||||||
|
|||||||
@@ -43,7 +43,13 @@ impl Embedder {
|
|||||||
d
|
d
|
||||||
}
|
}
|
||||||
Err(e) => {
|
Err(e) => {
|
||||||
eprintln!("Embedder: CPU ({e})");
|
// Loud, because the CPU path is correct but ~100x slower: the
|
||||||
|
// full longmemeval_s haystack is minutes on a GPU and most of a
|
||||||
|
// day on 8 cores. Silently falling back looks like a hang.
|
||||||
|
eprintln!("Embedder: CPU — CUDA unavailable ({e})");
|
||||||
|
eprintln!(
|
||||||
|
" WARNING: CPU embedding is roughly two orders of magnitude slower.\n Expect minutes for longmemeval_oracle and many hours for the full\n longmemeval_s haystack. For the GPU path, rebuild with\n `--features embeddings-cuda` and make sure `nvcc` is on PATH\n (it ships in /usr/local/cuda/bin, which is often not exported)."
|
||||||
|
);
|
||||||
Device::Cpu
|
Device::Cpu
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
|||||||
Reference in New Issue
Block a user