Median

Median aggregation rule, coordinate-wise.

Reference:

Dong Yin, Yudong Chen, Kannan Ramchandran, and Peter Bartlett. “Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates.” In Proceedings of the 35th International Conference on Machine Learning (ICML 2018).

class aggregators.median.Median[source]

Bases: Aggregator

Median aggregation rule, coordinate-wise.

The aggregated gradient is the coordinate-wise median of the worker gradients. This provides basic Byzantine resilience: a single adversarial worker can shift at most one sample per coordinate away from the true median.

classmethod aggregate(gradients: Sequence[Tensor] | Tensor, /, out: Tensor | None = None, **specialized: Any) → Tensor[source]

Aggregate the gradients.

Parameters:

  • gradients – Sequence of 1-D tensors containing gradients from workers.

  • out – Optional pre-allocated tensor to write the result into.

  • **specialized – Additional keyword arguments.

Returns:

Coordinate-wise median of the gradients, of shape `` (d,)

See also

For a simple mean baseline, see Average. For distance-based selection, see Krum.