Aggregators

Byzantine-resilient gradient aggregation rules.

In distributed learning with \(n\) workers of which up to \(f\) may be Byzantine (adversarial or faulty), the aggregator is the single point that determines whether the training converges. Each rule in this module accepts one gradient per worker and produces a single aggregated gradient that is robust to the \(f\) worst outliers.

All aggregators are stateless: each rule is a @classmethod invoked directly on the class. Specialized parameters (\(f\), \(n\), \(m\)) are keyword-only.

Available aggregators

• Average
• Median
• Trimmed Mean
• Krum
• MultiKrum
• Bulyan
• Brute
• GeoMed
• Nearest Neighbor Average
• AKSEL

Base class

class krum.primitives.aggregators.Aggregator

Bases: ABC

Abstract base class for stateless gradient aggregation rules.

Subclasses implement aggregate() as a @classmethod — no instance state is required, and the caller invokes the rule directly on the class. The first positional argument is the worker gradients; rule-specific hyperparameters (\(f\), \(n\), \(m\)) are keyword-only.

abstract classmethod aggregate(gradients: Sequence[Tensor] | Tensor, /, out: Tensor | None = None, **specialized: Any) → Tensor

Aggregate the gradients into a single tensor.

Parameters:

• gradients – Sequence of 1-D tensors containing one gradient per worker. Tensors are expected to share dtype and device.

• out – Optional pre-allocated tensor to write the result into. Must have shape \((d,)\) and the correct dtype and device. Passed through to the underlying PyTorch operation when supported.

• **specialized – Keyword-only arguments specific to each aggregation rule (e.g. \(f\), \(n\), \(m\)).

Returns:

Aggregated gradient of shape `` (d,)

Raises:

NotImplementedError – If the subclass does not implement this method.