GeometricMedian#

GeometricMedian aggregation rule, geometric median via smoothed Weiszfeld iterations.

Reference:: Krishna Pillutla, Sham M. Kakade, and Zaid Harchaoui. “Robust Aggregation for Federated Learning.” IEEE Transactions on Signal Processing 70 (2022): 1142-1154.

class krum.primitives.aggregators.geometric_median.GeometricMedian[source]#

Bases: Aggregator

GeometricMedian aggregation rule, geometric median of the gradients.

The geometric median is the unconstrained minimiser \(\arg\min_{y \in \mathbb{R}^d} \sum_i \|y - V_i\|\), computed here with smoothed Weiszfeld iterations (RFA oracle): starting from the mean, each step reweights the gradients by the inverse of their distance to the current estimate, floored at \(\nu\) for numerical stability. Unlike Medoid, the result is a synthetic point and generally not one of the submitted vectors.

classmethod aggregate(gradients: Sequence[Tensor] | Tensor, /, out: Tensor | None = None, *, n: int, f: int, nu: float = 0.1, tol: float = 1e-06, max_iter: int = 100, **specialized: Any) → Tensor[source]#

Aggregate gradients by approximating their geometric median.

Parameters:

gradients – Sequence of 1-D tensors containing gradients from workers.
out – Optional pre-allocated tensor to write the result into.
n – Total number of workers.
f – Number of Byzantine workers to tolerate. \(f\) is accepted for API uniformity with other aggregators but is not consulted here (the geometric median is defined for any \(n \ge 1\) and tolerates any minority of outliers).
nu – Smoothing floor on distances. Must be positive.
tol – Stop when an iteration moves the estimate by at most tol.
max_iter – Maximum number of Weiszfeld iterations. Must be at least 1.
**specialized – Additional keyword arguments.

Returns:

Approximate geometric median of shape `` (d,)

Raises:

ValueError – If \(n\), \(f\), the gradients count, nu, tol or max_iter is invalid.