osobhandClaude Fable 5
e080748d88
GPU Tests / Check GPU Availability (push) Successful in 0s
GPU Tests / Metal Tests (push) Has been skipped
CI / Format Check (push) Failing after 6s
CI / Clippy Check (push) Failing after 7s
Performance Benchmarks / Run Benchmarks (push) Failing after 7s
GPU Tests / CUDA Tests (11.8) (push) Has been skipped
GPU Tests / CUDA Tests (12.1) (push) Has been skipped
CI / Build (ubuntu-latest) (push) Failing after 7s
Documentation / Build User Guide (push) Successful in 8s
CI / Build (macos-latest) (push) Failing after 9s
CI / Test (macos-latest) (push) Has been skipped
CI / Test (ubuntu-latest) (push) Has been skipped
CI / Python Bindings (maturin) (macos-latest) (push) Has been skipped
CI / Python Bindings (maturin) (ubuntu-latest) (push) Has been skipped
CI / WASM Build + Size Check (push) Has been skipped
CI / Distributed Training Tests (push) Has been skipped
CI / Build CPU-Only (Explicit) (push) Failing after 43s
Documentation / Build API Documentation (push) Failing after 48s
CI / CI Success (push) Failing after 0s
Demos: - rtx-distllm-demo: real rtx-tensor weights per shard, real scaled-dot-product attention forward, metrics measured (Instant) instead of hardcoded constants; network topology remains a documented simulation fed by real tensor byte sizes. - rtx-model-zoo: MockInferenceEngine deleted; RealInferenceEngine loads a tiny real transformer into rtx_inference::InferenceEngine and runs genuine engine.infer per request; domain outputs are explicitly- labeled toy proxies derived from real output tokens. - rtx-inference-profiler: mock models deleted; profiles real matmul/softmax pipelines on rtx-tensor with measured latency/memory. Inference-path bugs the demos surfaced (fixed here): - ForwardPass::apply_embedding misused Tensor::gather for the embedding lookup — gather returns the indices' shape, silently dropping the hidden dim and breaking every downstream broadcast. Now uses the existing Tensor::embedding_lookup ([vocab,hidden] x [batch,seq] -> [batch,seq,hidden]). - Attention weight lookup accepts both self_attn. (HF-LLaMA) and attention. prefixes; final layer norm accepts norm.weight / model.norm.weight / ln_f.weight aliases. - Integration fixture gains the final norm weight; the previously always-failing engine tests now pass (8/8 model_loading_test). End-to-end inference through the real engine now works for the first time — verified via model_zoo_demo producing real forward-pass outputs across all categories. Co-Authored-By: Claude Fable 5 <[email protected]>
17 lines
389 B
TOML
17 lines
389 B
TOML
[package]
|
|
name = "rtx-distllm-demo"
|
|
version = "1.0.0"
|
|
edition = "2021"
|
|
authors = ["RustyStack Team"]
|
|
description = "DistributedLLM: Trillion-Parameter Inference across Thunderbolt 5 Mac Cluster"
|
|
license = "MIT OR Apache-2.0"
|
|
|
|
[dependencies]
|
|
distllm-shared = { path = "../distllm-shared" }
|
|
rtx-tensor = { workspace = true, features = ["cpu"] }
|
|
|
|
[dev-dependencies]
|
|
|
|
[lints]
|
|
workspace = true
|