osobhandClaude Fable 5
e080748d88
GPU Tests / Check GPU Availability (push) Successful in 0s
GPU Tests / Metal Tests (push) Has been skipped
CI / Format Check (push) Failing after 6s
CI / Clippy Check (push) Failing after 7s
Performance Benchmarks / Run Benchmarks (push) Failing after 7s
GPU Tests / CUDA Tests (11.8) (push) Has been skipped
GPU Tests / CUDA Tests (12.1) (push) Has been skipped
CI / Build (ubuntu-latest) (push) Failing after 7s
Documentation / Build User Guide (push) Successful in 8s
CI / Build (macos-latest) (push) Failing after 9s
CI / Test (macos-latest) (push) Has been skipped
CI / Test (ubuntu-latest) (push) Has been skipped
CI / Python Bindings (maturin) (macos-latest) (push) Has been skipped
CI / Python Bindings (maturin) (ubuntu-latest) (push) Has been skipped
CI / WASM Build + Size Check (push) Has been skipped
CI / Distributed Training Tests (push) Has been skipped
CI / Build CPU-Only (Explicit) (push) Failing after 43s
Documentation / Build API Documentation (push) Failing after 48s
CI / CI Success (push) Failing after 0s
Demos: - rtx-distllm-demo: real rtx-tensor weights per shard, real scaled-dot-product attention forward, metrics measured (Instant) instead of hardcoded constants; network topology remains a documented simulation fed by real tensor byte sizes. - rtx-model-zoo: MockInferenceEngine deleted; RealInferenceEngine loads a tiny real transformer into rtx_inference::InferenceEngine and runs genuine engine.infer per request; domain outputs are explicitly- labeled toy proxies derived from real output tokens. - rtx-inference-profiler: mock models deleted; profiles real matmul/softmax pipelines on rtx-tensor with measured latency/memory. Inference-path bugs the demos surfaced (fixed here): - ForwardPass::apply_embedding misused Tensor::gather for the embedding lookup — gather returns the indices' shape, silently dropping the hidden dim and breaking every downstream broadcast. Now uses the existing Tensor::embedding_lookup ([vocab,hidden] x [batch,seq] -> [batch,seq,hidden]). - Attention weight lookup accepts both self_attn. (HF-LLaMA) and attention. prefixes; final layer norm accepts norm.weight / model.norm.weight / ln_f.weight aliases. - Integration fixture gains the final norm weight; the previously always-failing engine tests now pass (8/8 model_loading_test). End-to-end inference through the real engine now works for the first time — verified via model_zoo_demo producing real forward-pass outputs across all categories. Co-Authored-By: Claude Fable 5 <[email protected]>
25 lines
609 B
TOML
25 lines
609 B
TOML
[package]
|
|
name = "rtx-inference-profiler"
|
|
version = "1.0.0"
|
|
edition = "2024"
|
|
rust-version = "1.92"
|
|
description = "RustyTorch++ Inference Profiler backend"
|
|
authors = ["RustyTorch Team"]
|
|
license = "MIT OR Apache-2.0"
|
|
repository = "https://github.com/rustytorch/rustytorch"
|
|
|
|
[dependencies]
|
|
inference-profiler-shared = { path = "../inference-profiler-shared" }
|
|
serde = { version = "1.0", features = ["derive"] }
|
|
serde_json = "1.0"
|
|
thiserror = "2"
|
|
rand = "0.8"
|
|
rtx-tensor = { path = "../../crates/core/rtx-tensor", features = ["cpu"] }
|
|
|
|
[dev-dependencies]
|
|
approx = "0.5"
|
|
proptest = "1.4"
|
|
|
|
[lints]
|
|
workspace = true
|