Models

Standard models for Byzantine-resilient distributed learning simulations.

These models implement the architectures used in the foundational papers on Byzantine-resilient distributed learning. They are provided as ready-to-use building blocks for simulations and can also serve as baselines for custom experiments.

Available models

• MLP Models
• CNN Models

Model Wrapper

class krum.primitives.models.Model(module: Module)

Model encapsulating a Module with zero-copy flat views of parameters and gradients.

All parameters and gradients are relinked to their respective, single contiguous buffer. Reading the parameters or gradients property returns a flat tensor sharing that buffer — modifying it modifies the module parameters and gradients directly. Writing to model.gradients = flat unpacks the flat vector back into each parameter’s .grad, sharing the memory of this flat vector.

The end-user is responsible for tensor copy, swap, and sharing (e.g. calling .clone() before “sending” gradients to a remote worker).

Parameters:

module – The module to encapsulate.

property gradients: Tensor

Zero-copy flat view of module gradients.

The returned tensor shares memory with each parameter’s .grad. If a parameter has no gradient yet, a zero-filled gradient is assigned.

This is a simple, fast getter once the flat gradient has been lazy-initialized on the first access. If gradients are replaced or removed externally (e.g. via module.zero_grad(set_to_none=True)), call relink_gradients() to re-synchronise the flat view.

Returns:

• Tensor of shape `` (d,)

• gradients, in the iteration order of ``module.parameters()``.

property module: Module

The encapsulated module.

Returns:

The encapsulated module.

property parameters: Tensor

Zero-copy flat view of module parameters.

The returned tensor shares memory with the model’s weights. Modifying it modifies the model directly. Clone before sending.

This is a simple, fast getter once the flat parameters have been lazy-initialized on the first access. If a parameter’s .data has been replaced externally, call relink_parameters() to re-synchronise the flat view.

Returns:

• Tensor of shape `` (d,)

• parameters, in the iteration order of ``module.parameters()``.

relink_gradients() → Tensor

Re-synchronise the flat gradient view after an external .grad replacement.

This method is necessary when a parameter’s .grad attribute has been replaced or removed since the flat gradient was built, for instance after module.zero_grad(set_to_none=True). It restores the zero-copy link between the cached flat tensor and every .grad so that the fast .gradients getter yields a consistent view again.

Returns:

• The flat gradient tensor of shape `` (d,)

• all module gradients.

relink_parameters() → Tensor

Re-synchronise the flat parameter view after an external .data replacement.

This method is necessary when a parameter’s .data has been replaced since the flat parameters were built, restoring the zero-copy link between the cached flat tensor and every parameter so that the fast .parameters getter yields a consistent view again.

Returns:

• The flat parameter tensor of shape `` (d,)

• all module parameters.