7.8 KiB
RustyTorch++ — Phase 6 Plan (Self‑Optimizing Platform & Governance)
Objective (6–12 months): Evolve RustyTorch++ from a high‑performance framework into a self‑optimizing platform with a unified data+compute graph, telemetry‑driven auto‑tuning, and end‑to‑end governance/provenance. Cement reliability, determinism, and compliance while enabling automated improvements via agents.
Reference paths
RUSTYTORCH_ROOT: /home/osobh/projects/rustytorch
RUSTG_COMPILER: /home/osobh/projects/rust/rustg (RTX 5090, sm_120)
STRATOSWARM_ROOT: /home/osobh/projects/stratoswarm
Focus is core platform evolution; app layers remain out of scope this phase.
- Scope & Non‑Goals
In scope
Unified Data + Compute Graph: integrate Arrow/Polars ETL ops into the same IR/graph as model ops; enable cross‑stage fusion where semantics allow.
Self‑Optimizing Runtime: telemetry‑guided rewrites, auto‑tuning of kernels/launch params/parallel strategies with strict guardrails.
Governance & Provenance: cryptographically signed artifacts (code→data→model→deployment), SBOMs, dataset lineage, and reproducible builds.
Zero‑Copy IO Pathways: GPUDirect Storage/RDMA hooks; page‑pinned staging; NVMe spill for KV/activations where applicable.
Heterogeneous Execution: initial policy engine to place ops across GPU/CPU (and stage stubs for future accelerators).
Agent‑in‑the‑Loop Evolution: safe sandbox loops (propose→test→select→merge) with evolution controller + conductor + security gates.
Out of scope (Phase 7+)
Multi‑tenant billing, global routing, and enterprise policy portals (product layer)
Fully automated cross‑org model exchange marketplaces
- Success Criteria (Exit / Phase‑Gate)
Unified Graph: ETL→Model→Post steps can be compiled as one graph; measurable end‑to‑end wall‑time reduction ≥15% on canonical pipelines.
Auto‑Tuning Wins: telemetry‑driven optimizations deliver ≥10–25% throughput improvement over Phase‑5 baselines without human intervention, with strict rollback.
Governance: every artifact (dataset snapshot, code hash, kernel bundle, checkpoint, container) is signed; repro build can reconstruct a result with the same metrics (± tolerance).
Zero‑Copy Paths: verified GPUDirect Storage/RDMA data path with ≥20% lower CPU load and ≤10% latency variance.
Hetero Execution: policy engine places at least 3 op families on optimal devices with no accuracy drift and ≤5% overhead when policies disable.
Agent Safety: evolution loops run continuously with no production regressions; all changes attributable and reversible.
- Architecture Work
A. Unified Data + Compute Graph
Extend IR (rtx-ir) with ETL dialect: columnar ops (filter, project, join, groupby, tokenize) + cost model.
Graph rewrite passes to pushdown ETL operations and fuse around model input/output transforms.
IO nodes for Parquet/Arrow; streamable datasets; deterministic shuffles; seed propagation.
B. Self‑Optimizing Runtime
Telemetry taps: per‑op latency, bandwidth, occupancy, cache hits, queue depths; aggregate per SKU.
Optimization agents propose: kernel param updates, fusion rewrites, graph scheduling tweaks, parallel policy changes.
Safety envelope: fixed budgets, canary benches, A/B gates, automatic rollback, signed PRs only.
C. Governance & Provenance
Provenance spec: { code_sha, data_sha, schema, env, driver, rustg_sha, kernel_cache_ver, config }.
SBOM generation; cosign signatures for artifacts; SLSA level targets for CI.
Dataset lineage store; data access policies; redaction tools for traces.
D. Zero‑Copy IO
GPUDirect Storage path (where supported): NVMe→GPU DMA with staging fallbacks.
RDMA datapath for distributed shards; bounded buffers; backpressure signals.
KV/offload paging integrated with these paths; autotuned block sizes.
E. Heterogeneous Execution
Policy engine: rules based on tensor size, op type, current load; warm/cold device migration with cost modeling.
Residency annotations in IR; copy elision where legal.
F. Agent Evolution Loop
Evolution Controller orchestrates: propose→validate→select→merge.
Conductor defines SLAs; Security enforces scans and secrets hygiene; Governance signs bundles.
- Benchmarks & Baselines
Pipelines
Text ETL (tokenization, filtering) + LLM training step
Vision ETL (decode, resize, crop, normalize) + ViT training step
Metrics
End‑to‑end wall‑time; CPU utilization; GPU util; DRAM/PCIe/NVLink GB/s; cache hit‑rates; determinism checks.
Determinism
Seeded shuffles; dataset snapshot pins; equal outputs within tolerances; graph + kernel cache hashes logged.
- CI Gates (Phase 6)
E2E Gate: ≥15% wall‑time reduction vs Phase‑5 pipeline; regression guard ≤5%/week.
Autotune Gate: optimization proposals must beat baselines by configured thresholds; otherwise auto‑revert.
Governance Gate: SBOM/sign passes; provenance manifest attached to releases; reproducer script verified.
Zero‑Copy Gate: data path tests pass; CPU load ↓ ≥20% on supported hosts.
Safety Gate: evolution controller runs only in sandbox until green; canary then promote.
- Observability & Telemetry
Unified trace spans across ETL+Model; correlation IDs per run.
Metrics: e2e_ms, etl_ms, model_ms, gds_enabled, rdma_enabled, cpu_user_pct, gpu_util, tokens_per_s, cache_hit_pct, occ_pct.
Dashboards: pipeline breakdowns; optimization suggestions & impact; provenance views.
- Deliverables
Extended rtx-ir with ETL dialect + passes; pushdown & fusion rules.
Runtime hooks for zero‑copy IO (GDS/RDMA) + fallbacks; tuning knobs.
Auto‑tuning agents integrated with safe promotion; profile DB expansion.
Governance pipeline: SBOM, signatures, provenance manifests, dataset lineage store.
Benchmarks & dashboards for ETL+Model pipelines; CI gates configured.
Docs: docs/unified_graph.md, docs/governance.md, docs/zero_copy_io.md, docs/evolution.md.
- RACI — Phase 6 Agent Ownership
Conductor (Orchestrator) — Accountable: phase delivery, dependency orchestration, gate readiness.
Data/ETL Graph Agent — Responsible: ETL dialect & pushdown; dataset sharding; deterministic transforms.
IR Rewrite & Fusion Agent — Responsible: cross‑stage fusion patterns; legality checks.
Auto‑Tuner Agent — Responsible: telemetry‑driven proposals; cache/profile DB; safe promotion.
Rust Engineer — Responsible: zero‑copy IO hooks; residency/placement plumbing; error paths.
Distributed Strategy Agent — Consulted: RDMA data paths; placement policies.
Performance Engineer — Accountable: pipeline benches; KPI dashboards; regression alerts.
Governance & Provenance Agent — Accountable: SBOM/signing; manifest/verifier; dataset lineage.
Security Engineer & Auditor — Accountable: scans; redaction; secret hygiene; policy enforcement.
Agent Organizer — Accountable: routing policies; freeze switch on gate failure.
- Risks & Mitigations
Semantic drift when fusing ETL+Model → legality proofs; golden tests; opt‑out kill switch.
Auto‑tuner regressions → canary A/B; rollback; budget caps; sandbox isolation.
GDS/RDMA environment drift → capability detection; test matrix by driver/kernel; graceful fallbacks.
Provenance gaps → mandatory manifests; CI fails on missing lineage/signatures.
Policy misplacement → conservative defaults; cost model learning; operator allow/deny lists.
- Timeline (suggested)
Months 1–2: ETL dialect & pushdown passes; unified tracing; baseline E2E benches.
Months 3–4: Zero‑copy IO hooks; telemetry taps; auto‑tuner integration (sandbox only).
Months 5–6: Governance pipeline; safe promotion of optimizations; CI gates; docs; phase‑gate review.
- Phase‑7 Handover Seeds
Enterprise policy plane (RBAC, retention, PII tooling) and compliance dashboards.
Global cache distribution and profile sharing across Stratoswarm clusters.
Heterogeneous expansion (TPU/FPGA/NPUs) with placement rules & kernels.
— End of Phase 6 Plan —