Commit Graph
2 Commits
Author SHA1 Message Date
osobhandClaude Sonnet 5 4aaa36a57a style: cargo fmt --workspace (whitespace/wrapping only, no semantic change)
Whole-workspace rustfmt pass picked up while iterating on Mamba GPU
backward work. Verified formatting-only via diff sampling; no logic
changed.

Co-Authored-By: Claude Sonnet 5 <[email protected]>
2026-08-10 07:09:36 -07:00
Omar SobhandClaude Sonnet 4.6 25322b019d feat(batch20-23): I-JEPA + V-JEPA + Neuro-JEPA self-supervised learning
CI / Format Check (push) Failing after 13s
CI / Build (macos-latest) (push) Failing after 29s
CI / Clippy Check (push) Failing after 1m27s
Documentation / Build User Guide (push) Successful in 9s
Documentation / Build API Documentation (push) Failing after 1m23s
CI / Build CPU-Only (Explicit) (push) Failing after 1m40s
Performance Benchmarks / Run Benchmarks (push) Successful in 2m11s
CI / Build (ubuntu-latest) (push) Failing after 7m41s
CI / Test (macos-latest) (push) Has been skipped
CI / Test (ubuntu-latest) (push) Has been skipped
CI / Python Bindings (maturin) (macos-latest) (push) Has been skipped
CI / Python Bindings (maturin) (ubuntu-latest) (push) Has been skipped
CI / WASM Build + Size Check (push) Has been skipped
CI / Distributed Training Tests (push) Has been skipped
CI / CI Success (push) Failing after 0s
Implements JEPA (Joint Embedding Predictive Architecture) across 4 batches:

Batch 20 — I-JEPA core architecture (ssl/jepa.rs):
- BlockMaskStrategy: multi-block random masking (4 blocks, scale 0.15-0.20,
  aspect ratio 0.75-1.5), Fisher-Yates context subsampling; 12 tests
- JepaPredictor: narrow 6-block transformer (encoder_dim/4 predictor_dim);
  mask tokens + learned position embeddings; cross-context attention;
  in-proj/out-proj between encoder and predictor dims; 6 tests

Batch 21 — Training loop + EMA (ssl/jepa.rs):
- jepa_loss: L2 in representation space with per-block granularity; 3 tests
- EmaTargetEncoder: tau annealing tau_start→tau_end (0.996→1.0);
  shadow weight update; τ=1.0 frozen / τ=0.0 copy edge cases; 5 tests
- JepaTrainer: full I-JEPA step: mask→encode→predict→target→L2→EMA; 4 tests

Batch 22 — Evaluation protocol (ssl/jepa.rs):
- FeatureBank: L2-normalized cosine k-NN with majority vote; 3 tests
- LinearProbe: SGD-trained linear head on frozen features; CE loss;
  gradient update; 4 tests
- JepaEvaluator: linear_probe() + knn_eval() unified interface; 4 tests
- End-to-end I-JEPA training + k-NN evaluation integration test

Batch 23 — V-JEPA + Neuro-JEPA (ssl/vjepa.rs):
- PatchEmbed3D: 3D patch embeddings [T, H, W, C] → [total_patches, d]; 2 tests
- TubeMaskStrategy: space-time tube masking; spatial block selection extended
  across all temporal frames; 90% mask ratio; 7 tests
- VJepaTrainer: video analog of JepaTrainer with EMA and tube masking; 5 tests
- NeuroJepaConfig: EEG/MEG signal JEPA (64 channels × 16 time segments);
  channel-tube masking (mask entire time axis for selected channels);
  tube structure validation; 7 tests

Total: 62 tests, 0 failures

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
2026-06-27 14:15:11 +00:00