torchfoo.distributed.all_reduce_#

torchfoo.distributed.all_reduce_(x)[source]#

Perform an in-place differentiable “all reduce” operation

Parameters:: x (Tensor) – a contiguous Tensor on this device
Returns:: The input tensor, modified in-place with the all-reduced mean.
Return type:: Tensor