A Little Is Enough

A Little Is Enough (ALIE) gradient attack.

Reference:

Gilad Baruch, Moriah Baruch, Yoav Goldberg, and Kfir Y. Levy. “A Little Is Enough: Circumventing Defenses For Distributed Learning.” In Advances in Neural Information Processing Systems 32 (NeurIPS 2019).

class attacks.alie.ALIEAttack

Bases: Attack

ALIE attack.

Generates Byzantine gradients from the exact coordinate-wise mean and standard deviation of the honest gradients passed to the attack. The attack perturbs the honest mean by \(z \sigma\) along the chosen Direction, where \(z\) is the attack factor.

This corresponds to a statistics-oracle variant of ALIE rather than the original paper’s more restricted information setting: the attacker is assumed to know the full honest gradient distribution, not just a subset.

classmethod generate(honest_gradients: Sequence[Tensor] | Tensor, /, out: Tensor | None = None, *, f: int, z: float | str = 'max', direction: Direction = Direction.NEGATIVE, **specialized: Any) → Tensor

Generate Byzantine gradients.

Parameters:

• honest_gradients – Sequence of \(h\) gradient vectors, one per honest worker, each of shape \((d,)\).

• out – Optional pre-allocated tensor of shape \((f, d)\) to write the result into and return.

• f – Number of Byzantine gradients to generate.

• z – Attack factor in standard-deviation units, or "max" for the largest factor that keeps the Byzantine gradients within a majority of the honest gradient distribution.

• direction – Direction of the perturbation relative to the honest mean.

• **specialized – Additional keyword arguments.

Returns:

• Byzantine gradients of shape `` (f, d)

• empty tensor of shape `` (0, d)

Raises:

• TypeError – If \(z\) is not a number or "max", direction is not a Direction, or the honest gradients are not floating-point.

• ValueError – If \(z\) is negative, \(f\) is negative, there are no honest gradients, or the worker configuration admits no non-negative ALIE factor.

static max_z(honest_gradients: Tensor, f: int) → Tensor

Compute the maximal valid ALIE attack factor for the worker configuration.

\(z_{\max}\) is the largest \(z\) such that \(\Phi(z) < (h - s) / h\), where \(h\) is the number of honest workers and \(s\) is the number of honest workers needed to form a majority among the \(n = h + f\) workers: \(s = \lfloor n / 2 \rfloor + 1 - f\).

Parameters:

• honest_gradients – Tensor of shape \((h, d)\) containing gradients from the \(h\) honest workers.

• f – Number of Byzantine gradients to generate.

Returns:

• Maximal attack factor, as a 0-D tensor on the same device and

• dtype as ``honest_gradients``.

Raises:

ValueError – If there are no honest gradients, or if the worker configuration does not admit a non-negative ALIE factor.

class attacks.alie.Direction(*values)

Bases: str, Enum

Sign of the perturbation applied by ALIE relative to the honest mean.

Chooses whether the perturbation \(z \sigma\) is added to (POSITIVE) or subtracted from (NEGATIVE) the honest coordinate-wise mean.

NEGATIVE = 'negative'

POSITIVE = 'positive'

See also

For a simple noise-based attack, see Gaussian. For a sign-reversal attack, see SignFlip.