Krum#

Krum aggregation rule, single-gradient selection.

Reference:: Peva Blanchard, El Mahdi El Mhamdi, Rachid Guerraoui, and Julien Stainer. “Machine learning with adversaries: Byzantine tolerant gradient descent.” In Advances in Neural Information Processing Systems 30 (NIPS 2017).

class krum.primitives.aggregators.krum.Krum[source]#

Bases: MultiKrum

Krum aggregation rule, single-gradient selection.

For each worker gradient, Krum scores it by the sum of squared Euclidean distances to its \(n - f - 2\) closest peers. After torch.sort() on each row, the self-distance is 0 so column 0 is always the worker itself. Columns \(1\) through \(n - f - 2\) give \(n - f - 2\) closest other workers.

This is MultiKrum with \(m = 1\).

classmethod aggregate(gradients: Sequence[Tensor] | Tensor, /, out: Tensor | None = None, *, n: int, f: int, **specialized: Any) → Tensor[source]#

Aggregate the gradients.

Parameters:

gradients – Sequence of 1-D tensors containing gradients from workers.
out – Optional pre-allocated tensor to write the result into.
n – Total number of workers.
f – Number of Byzantine workers to tolerate.
**specialized – Additional keyword arguments.

Returns:

Aggregated gradient of shape `` (d,)

Raises:

ValueError – If \(n < 1\), \(f < 0\), \(f > n\), \(n < 2f + 3\), or len(gradients) != n.