Omar SobhandClaude Sonnet 4.6 0e1d6a74b6 feat(batch6): windowed acceptance metrics, KV INT8 quant, col/row-parallel linear ...

Windowed acceptance rate (rtx-inference/speculative):
- WindowedAcceptanceTracker: O(1) VecDeque sliding window, p50/p95/min/max
- AcceptanceTrend enum (Rising/Falling/Stable, ±0.05 threshold)
- AcceptanceDashboard aggregator; wired into PerformanceMetrics::update()
  and dashboard(); 13 tests

KV cache INT8 quantization (rtx-inference/cache):
- KvCacheQuantMode { None, Int8 { scale_per_token }, Fp8E4M3 } enum
- KvQuantizer::encode/decode: symmetric per-block INT8 (scale=max_abs/127)
  gives 4× compression vs f32; Fp8E4M3 CPU proxy, GPU path reserved
- QuantizedKvBlock carries data+scale+mode; KvCacheConfig::quant_mode
  defaulting to None; 14 tests

ColParallel + RowParallel linear (rtx-distributed):
- ColParallelLinear: shards weight rows across TP ranks, forward_cpu()
  batch matmul + per-shard bias; no AllReduce (output shards concatenated)
- RowParallelLinear: shards weight cols across TP ranks, forward_cpu()
  partial sum + bias on rank 0 only; async forward() calls ProcessGroup
  AllReduce for real NCCL path; CPU sim is no-op
- TensorParallel::matmul() replaced zeros stub with ColParallelLinear(tp=1)
- col→row roundtrip verified within 1e-3; 9 tests

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>

2026-06-27 03:54:07 +00:00

..

basic_engine_test.rs

Initial commit

2026-03-04 00:08:42 +00:00

batch_engine_integration_test.rs

feat(perf): GPU perf batch 4 — SmoothQuant INT8 forward, varlen FA, inference graph capture

2026-06-27 01:11:08 +00:00

cache_access_pattern_test.rs

Initial commit

2026-03-04 00:08:42 +00:00

cache_page_test.rs

Initial commit

2026-03-04 00:08:42 +00:00

cache_test.rs

feat(batch6): windowed acceptance metrics, KV INT8 quant, col/row-parallel linear

2026-06-27 03:54:07 +00:00

continuous_batching.rs

Initial commit

2026-03-04 00:08:42 +00:00

device_creation_test.rs

Initial commit

2026-03-04 00:08:42 +00:00

engine_generation_test.rs

Initial commit

2026-03-04 00:08:42 +00:00

get_loaded_model_test.rs

Initial commit

2026-03-04 00:08:42 +00:00

inference_engine.rs

Initial commit

2026-03-04 00:08:42 +00:00

model_loading_test.rs

Initial commit

2026-03-04 00:08:42 +00:00

monitoring_test.rs

Initial commit

2026-03-04 00:08:42 +00:00

overflow_handling_tests.rs

Initial commit

2026-03-04 00:08:42 +00:00

paged_kv_cache.rs

Initial commit

2026-03-04 00:08:42 +00:00

quantization.rs

Initial commit

2026-03-04 00:08:42 +00:00

request_management.rs

Initial commit

2026-03-04 00:08:42 +00:00

request_metrics_complete_test.rs

Initial commit

2026-03-04 00:08:42 +00:00

request_result_test.rs

Initial commit

2026-03-04 00:08:42 +00:00

sampling_params_test.rs

Initial commit

2026-03-04 00:08:42 +00:00

shape_conversion_test.rs

Initial commit

2026-03-04 00:08:42 +00:00

simple_overflow_test.rs

Initial commit

2026-03-04 00:08:42 +00:00

speculative_decoding.rs

Initial commit

2026-03-04 00:08:42 +00:00

tensor_cat_test.rs

Initial commit

2026-03-04 00:08:42 +00:00

tensor_comparison_test.rs

Initial commit

2026-03-04 00:08:42 +00:00

tensor_methods_test.rs

Initial commit

2026-03-04 00:08:42 +00:00