150 lines
5.8 KiB
Markdown
150 lines
5.8 KiB
Markdown
RustyTorch++ — Phase 8 Plan (Autonomous Agentic Evolution & Continuous Adaptation)
|
||
|
||
Phase 8 focus: Operationalize agent‑in‑the‑loop evolution so RustyTorch++ autonomously discovers, proposes, validates, and safely promotes improvements to compiler/runtime, kernels, distributed strategies, and serving—under strict determinism, governance, and rollback guarantees.
|
||
|
||
Reference paths
|
||
|
||
RUSTYTORCH_ROOT: /home/osobh/projects/rustytorch
|
||
|
||
RUSTG_COMPILER: /home/osobh/projects/rust/rustg (RTX 5090, sm_120)
|
||
|
||
STRATOSWARM_ROOT: /home/osobh/projects/stratoswarm
|
||
|
||
1. Scope & Non‑Goals
|
||
|
||
In scope
|
||
|
||
Evolution framework and agent swarm with closed‑loop telemetry→proposal→validation→promotion.
|
||
|
||
Multi‑objective optimizer (throughput, latency, memory, determinism, energy) with policy weights.
|
||
|
||
Safe sandboxes, A/B canaries, automatic rollback, provenance, and signed PRs.
|
||
|
||
Knowledge Graph to track change → benchmark → production impact; queryable for meta‑learning.
|
||
|
||
Out of scope (Phase 9+)
|
||
|
||
Cross‑org federated evolution sharing; economic marketplaces for profiles.
|
||
|
||
2. Success Criteria (Phase‑Gate)
|
||
|
||
Autonomous proposals land to main with human approval ≥90% and no gate bypass.
|
||
|
||
Effectiveness: ≥70% of merged proposals yield measurable wins (e.g., ≥10% step‑time, ≥15% tokens/s, or ≥20% frag% ↓) on at least one benchmark SKU.
|
||
|
||
Safety: 0 production regressions over a 3‑month observation window; all merges are signed, traceable, and reversible.
|
||
|
||
Coverage: proposals span ≥4 domains (compiler pass, kernel params, memory policy, parallel strategy, inference scheduler).
|
||
|
||
Provenance & Repro: every proposal includes repro bundle (code/data/env hashes) and rollback plan; passes determinism gates.
|
||
|
||
3. Architecture — Evolution System
|
||
|
||
A. Event & Telemetry Bus
|
||
|
||
Topics: compiler.pass, kernel.launch, occ_profile, alloc.frag, graph.capture, dist.collective, serve.scheduler, bench.result.
|
||
|
||
Emit structured JSON with hardware profile {arch, driver, rustg_sha} and seeds.
|
||
|
||
B. Proposal Engine
|
||
|
||
Optimization Agents generate candidates: IR rewrites, fusion toggles, kernel params (tile/warp/vector), mem allocator knobs, DP/TP/PP split, scheduler batch/merge windows, quant settings.
|
||
|
||
Multi‑objective scoring with constraints: determinism on, error tolerances, memory ceilings, thermal/power caps.
|
||
|
||
C. Validation Pipeline
|
||
|
||
Sandbox branch → run micro/macro benches with fixed seeds → Nsight/rocprof traces → compare vs. baselines.
|
||
|
||
A/B Canary on Stratoswarm (limited traffic, short window) → promote on pass, auto‑revert on fail.
|
||
|
||
D. Promotion & Rollback
|
||
|
||
Signed PR with Output Envelope (plan/actions/changeset/validation/rollback).
|
||
|
||
Auto‑merge when green gates + human approval.
|
||
|
||
One‑click rollback retains previous kernel cache and graph bundles.
|
||
|
||
E. Knowledge Graph
|
||
|
||
Store nodes: proposal, benchmarks, hardware, metrics deltas, deployment impact.
|
||
|
||
Queries: “what improved Flash‑Attention on sm_120?”, “which changes hurt determinism?”
|
||
|
||
4. Agents & Roles (MCP‑backed)
|
||
|
||
Conductor (Orchestrator) — ticket routing, critical path, freeze switch.
|
||
|
||
Evolution Controller — governs propose→validate→promote; policy weights; risk scoring.
|
||
|
||
Performance Engineer Agent — designs benches, parses traces, computes deltas.
|
||
|
||
IR Rewrite & Fusion Agent — emits pattern rewrites, legality proofs, pass toggles.
|
||
|
||
Kernel Synthesizer Agent — generates/compiles candidate kernels; occupancy & parity checks.
|
||
|
||
Auto‑Tuner Agent — bandit/grid search; maintains per‑SKU profile DB; cache mgmt.
|
||
|
||
Memory Planner Agent — allocator/fragments/activation remat policy tweaks.
|
||
|
||
Distributed Strategy Agent — overlap & bucket sizes; topo‑aware collectives.
|
||
|
||
Inference Scheduler Agent — batch windows, SLA lanes, KV paging, speculative thresholds.
|
||
|
||
Governance & Provenance Agent — SBOM/signatures, repro bundles, artifact policies.
|
||
|
||
Security Engineer Agent — scans, secret hygiene, sandbox isolation.
|
||
|
||
5. CI/CD Gates (Phase 8)
|
||
|
||
Determinism Gate: fixed‑seed parity (fp32 ≤1e‑6; bf16/fp16 ≤1e‑3), graph & kernel cache hash logged.
|
||
|
||
Perf Gate: domain‑specific thresholds (e.g., step‑time ≥10% faster or tokens/s ≥15% higher) else reject.
|
||
|
||
Memory Gate: frag% ↓ or ≤ baseline; no new leaks in ≥8h soak.
|
||
|
||
Safety Gate: security scan clean; SBOM present; signed artifacts; sandbox only until canary passes.
|
||
|
||
Promotion Gate: canary p95/p99 within SLA; rollback plan validated automatically.
|
||
|
||
6. Observability & Dashboards
|
||
|
||
Evolution dashboard: proposal queue, win rates, gate failures, impact by SKU/model.
|
||
|
||
Per‑proposal bundle: traces, deltas, cache hit rates, roofline view, risk score.
|
||
|
||
Longitudinal views: performance over time, auto‑tuner exploration vs. exploitation.
|
||
|
||
7. Deliverables
|
||
|
||
rtx‑evolve/ module: proposal engine, policy weights, glue to MCP agents.
|
||
|
||
Pipelines: sandbox, A/B canary on Stratoswarm, auto‑promotion, rollback automation.
|
||
|
||
Knowledge Graph: change→impact store with query API.
|
||
|
||
Bench Packs: standard micro/macro scenarios with fixtures for H100/5090/MI300 (as available).
|
||
|
||
Docs: docs/evolution.md (architecture, guardrails, SOPs), docs/bench_packs.md.
|
||
|
||
8. Risks & Mitigations
|
||
|
||
Search explosion → cap budgets, bandit pruning, learned priors from Knowledge Graph.
|
||
|
||
False wins / overfitting → holdout benches; periodic cross‑validation on unseen shapes.
|
||
|
||
Silent regressions → strict gates; production canaries; auto‑revert; human approval.
|
||
|
||
Security leakage → sandbox isolation; redaction; secrets scanning; signed artifacts.
|
||
|
||
9. Timeline (suggested, 3–4 months then ongoing)
|
||
|
||
Weeks 1–2: Event bus hardening; baseline deltas & dashboards; bench packs.
|
||
|
||
Weeks 3–4: Proposal engine MVP (IR/kernel/memory knobs); sandbox runners; CI hooks.
|
||
|
||
Weeks 5–6: Canary pipeline on Stratoswarm; auto‑tuner/bandit integration; Knowledge Graph v1.
|
||
|
||
Weeks 7–8: Policy tuning; success metrics; playbooks; phase‑gate review; ongoing continuous evolution.
|