# Production Deployment

Deploy RustyTorch++ models to production environments.

## Deployment Options

| Option | Latency | Throughput | Complexity |
|--------|---------|------------|------------|
| Direct Binary | ~1ms | High | Low |
| HTTP API | ~5-10ms | Medium | Medium |
| gRPC | ~2-5ms | High | Medium |
| Kubernetes | ~10-20ms | Scalable | High |

## Direct Binary Deployment

### Building for Production

```bash
# Optimized release build
cargo build --release -p rtx-inference

# Strip symbols for smaller binary
strip target/release/rtx-inference
```

### Production Cargo.toml

```toml
[profile.release]
lto = "thin"           # Link-time optimization
codegen-units = 1      # Better optimization
panic = "abort"        # Smaller binary
strip = true           # Remove debug symbols
opt-level = 3          # Maximum optimization
```

## HTTP API Deployment

### Using rtx-serving-api

```rust
use rtx_serving_api::{Server, ServerConfig, ModelRegistry};
use rtx_inference::InferenceEngine;

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    // Load model
    let engine = InferenceEngine::load("model.bin")?;

    // Configure server
    let config = ServerConfig {
        host: "0.0.0.0".to_string(),
        port: 8080,
        max_batch_size: 32,
        timeout_ms: 5000,
        workers: num_cpus::get(),
    };

    // Start server
    let server = Server::new(config, engine)?;
    server.run().await
}
```

### API Endpoints

```
POST /v1/predict       - Single prediction
POST /v1/batch         - Batch predictions
GET  /v1/health        - Health check
GET  /v1/metrics       - Prometheus metrics
WS   /v1/stream        - Streaming predictions
```

## Model Optimization

### Quantization

```rust
use rtx_compress::{Quantizer, QuantConfig};

let config = QuantConfig {
    dtype: DType::I8,           // INT8 quantization
    calibration_samples: 1000,  // Calibration dataset size
    per_channel: true,          // Per-channel quantization
};

let quantizer = Quantizer::new(config)?;
let quantized_model = quantizer.quantize(&model)?;

// 4x smaller, ~2x faster on INT8 capable hardware
quantized_model.save("model_int8.bin")?;
```

### ONNX Export

```rust
use rtx_bindings::OnnxExporter;

let exporter = OnnxExporter::new();
exporter.export(&model, "model.onnx")?;
```

## Request Batching

```rust
use rtx_serving_api::BatchProcessor;

let processor = BatchProcessor::new(BatchConfig {
    max_batch_size: 64,
    max_wait_ms: 10,        // Wait up to 10ms to fill batch
    dynamic_batching: true,  // Adaptive batch sizes
});
```

## Load Balancing

### Multiple Instances

```yaml
# docker-compose.yml
services:
  rtx-api-1:
    image: rustytorch/serving-api
    deploy:
      replicas: 3
      resources:
        reservations:
          devices:
            - capabilities: [gpu]

  nginx:
    image: nginx
    ports:
      - "80:80"
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf
```

## Health Checks

```rust
use rtx_serving_api::HealthCheck;

// Configure health checks
let health = HealthCheck::new()
    .with_gpu_check()           // Verify GPU available
    .with_model_check(&model)   // Verify model loaded
    .with_memory_threshold(0.9); // Alert at 90% memory

// Expose at /health
server.add_health_check(health);
```

## Monitoring

### Prometheus Metrics

```rust
use rtx_monitoring::{MetricsExporter, PrometheusConfig};

let exporter = MetricsExporter::prometheus(PrometheusConfig {
    endpoint: "/metrics",
    buckets: vec![0.001, 0.005, 0.01, 0.025, 0.05, 0.1],
});

// Exposed metrics:
// - rtx_inference_latency_seconds
// - rtx_inference_throughput_requests
// - rtx_gpu_memory_used_bytes
// - rtx_batch_size_histogram
```

### Logging

```rust
use tracing_subscriber::fmt;

// Structured JSON logging for production
fmt()
    .json()
    .with_env_filter("rtx=info,tower_http=warn")
    .init();
```

## Security

### Input Validation

```rust
use rtx_serving_api::validation::InputValidator;

let validator = InputValidator::new()
    .max_sequence_length(2048)
    .max_batch_size(64)
    .allowed_dtypes(&[DType::F32, DType::F16]);

server.add_validator(validator);
```

### Rate Limiting

```rust
use tower::limit::RateLimitLayer;

let rate_limit = RateLimitLayer::new(100, Duration::from_secs(1));
// 100 requests per second
```

## Next Steps

- [Edge Deployment](./edge.md) - Deploy to edge devices
- [Docker & Kubernetes](./containers.md) - Container orchestration
- [Serving API](./serving-api.md) - API customization