User Guide

This guide explains each component of Opake's DP training pipeline: what it does, how the API works, and the practical decisions you need to make. For hands-on practice, see the Tutorials. For complete function signatures, see the API Reference.

Install with pip install opake (plus opake[...] extras).

End-to-end pipelines

Pick the track that matches your problem:

• DP-SGD end-to-end — independent Gaussian noise at every step, with per-step privacy composition. This is the standard DP training recipe. Imports from opake.dpsgd.*.
• DP-FTRL end-to-end — correlated noise across the whole training run via matrix factorization. Imports from opake.dpftrl.*.

Both pipelines share the same primitives (clipping, noise, sampling, optimizer, accounting). The topic pages below are stack-agnostic concept references; each end-to-end guide picks the right pieces and combines them.

Topics

Foundations

• Differential Privacy Concepts — What DP guarantees, how DP-SGD works, privacy budgets, composition, and amplification.
• Random Number Generation — Explicit RNG keys, splitting, fold_in, reproducibility in distributed training.

Core pipeline

• Per-Example Gradient Clipping — clipped_grad, auto_clipped_grad, adaptive_clipped_grad (DP-SGD-only), microbatching, per-group clipping.
• Noise Addition — gaussian_noise (DP-SGD) and mf_gaussian_noise with strategy factories (DP-FTRL).
• Privacy Accounting — Composable DpProcess objects, privacy metrics, calibration, the Accountant helper.
• Sampling & Microbatching — Poisson, truncated Poisson, cyclic / b-min-sep / balls-in-bins / sequential samplers.

Integration

• Optimizers — TorchOpt functional optimizers with DP-SGD bias correction and the private second-moment story for DP-FTRL.
• Serialization (API reference) — Checkpoint explicit state with opake.serialization.state_dict / from_state_dict.
• LR Scheduling — Warmup, cosine, inverse-sqrt schedules; composing with_warmup with any decay curve.
• Distributed Training — DDP with synchronized noise and gradient aggregation.
• Distributed DPTrainer — DDP setup and ddp_shard behavior for DPTrainer.
• HuggingFace Integration — DPTrainer, TrainingArguments, model patches, and LoRA.
• Memory Optimizations — Microbatching, gradient checkpointing, fused kernels, profiling.
• Privacy Auditing — Empirical privacy validation via membership inference.

Mechanism reference

• DP-SGD mechanisms — Gaussian per-step noise.
• DP-FTRL mechanisms — BandMF, BLT, BSR, BISR, λ-CGD strategies.

Reference

• Known Limitations — Flash Attention, DDP-only, in-place operations, other constraints.