quantumclaw/clawhdf5
3
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

feat(accel): runtime-dispatched int8 dot product

Browse Source 
The int8-quantised HNSW index compared vectors with a scalar loop that
the compiler vectorised for the x86-64 baseline (SSE2), while the f32
path it was measured against goes through `clawhdf5-accel` and runs
AVX2. So the ~13% throughput cost recorded for `quantized_index` was a
missing kernel rather than a property of int8.

`clawhdf5_accel::dot_i8` adds a scalar fallback and an AVX2 path:
sign-extend each 16-byte half to i16, then `madd_epi16`, which
multiplies and sums adjacent pairs straight into i32 lanes. It is
dispatched through the same detected backend as the f32 kernels, and
the index now calls it.

Integer arithmetic, so the SIMD path must agree with scalar bit for
bit — tested at lengths that are and are not multiples of the block,
and at the -128 extreme for overflow.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
This commit is contained in:
osobh
2026-09-20 17:30:56 -07:00
co-authored by Claude Opus 5
parent fb58300b3f
commit 97e65f2adf
4 changed files with 140 additions and 24 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/clawhdf5-ann/src/hnsw.rs
+5 -24
View File
@@ -358,31 +358,12 @@ enum Query {
Int8(Vec<i8>, f32),
}
/// Sum of products, widened so it cannot overflow: `dim` terms of at most
/// `127 * 127`, so `i32` suffices for any realistic dimension.
/// Sum of products, widened so it cannot overflow. Runtime-dispatched to the
/// same SIMD backend as the f32 kernels, so the two storages are compared on
/// equal terms.
#[inline]
fn dot_i8(a: &[i8], b: &[i8]) -> i32 {
// Four independent accumulators over 32-lane blocks: the widening product
// has to sit in a fixed-length chunk for the vectoriser to see it, and the
// separate accumulators keep it off one dependency chain.
const LANE: usize = 8;
let (a_blocks, a_tail) = a.as_chunks::<{ LANE * 4 }>();
let (b_blocks, b_tail) = b.as_chunks::<{ LANE * 4 }>();
let mut acc = [0i32; 4];
for (x, y) in a_blocks.iter().zip(b_blocks) {
for (lane, slot) in acc.iter_mut().enumerate() {
let mut sum = 0i32;
for k in 0..LANE {
sum += i32::from(x[lane * LANE + k]) * i32::from(y[lane * LANE + k]);
}
*slot += sum;
}
}
let tail: i32 = a_tail
.iter()
.zip(b_tail)
.map(|(&x, &y)| i32::from(x) * i32::from(y))
.sum();
acc[0] + acc[1] + acc[2] + acc[3] + tail
clawhdf5_accel::dot_i8(a, b)
}
/// Magic for [`HnswIndex::graph_to_bytes`].