HiddenVulnerabilitySimulation#

HiddenVulnerabilitySimulation.

Reference:: El Mahdi El Mhamdi, Rachid Guerraoui, and Sébastien Rouault. “The Hidden Vulnerability of Distributed Learning in Byzantium.” In Proceedings of the 35th International Conference on Machine Learning (ICML 2018).

One HiddenVulnerabilitySimulation instance = one (aggregator, attack, dataset, model) configuration run over multiple synchronous rounds with the Robbins-Monro learning-rate schedule \(η(t) = r_η · η_0 / (t + r_η)\) of Section 5.1.

class krum.simulations.centralised.hidden_vulnerability_icml_2018.HiddenVulnerabilitySimulation(*, r_eta: float, weight_decay: float = 0.0001, xavier_init: bool = True, lr_schedule: Literal['exponential', 'robbins_monro', 'none'] = 'robbins_monro', lr_decay: float | None = None, stop_attack_at: int | None = None, **kwargs: Any)[source]#

Bases: CentralisedSimulation

Distributed SGD simulation.

Compared to the NIPS 2017 KrumSimulation, this variant:

Uses the Robbins-Monro learning-rate schedule \(η(t) = r_η · η_0 / (t + r_η)\) (Section 5.1) with fading rate r_eta — not constant, not exponential.
Applies L2 regularization of weight 1e-4 and Xavier weight initialization on every layer (Section 5.1).
Reports three metrics per evaluation: (test_loss, test_error, test_accuracy).

evaluate() → tuple[float, float, float][source]#

Evaluate the model on the test set.

Returns:: Tuple of `` (test_loss, test_error, test_accuracy)

evaluate_train() → float[source]#

Evaluate the model on the training set.

Returns:: Training loss.

Note

Reads the full training set in a single no_grad pass (self._full_loader batches the concatenation of every worker’s dataset at once). OK for the bundled experiments (MNIST 60k × 784 ≈ 180 MB); for larger datasets, sample a fixed-size subset instead.