torchfoo.distributed.all_reduce_sum_#

torchfoo.distributed.all_reduce_sum_(x)[source]#

Perform an in-place differentiable “all reduce sum” operation

Parameters:: x (Tensor) – a contiguous Tensor on this device
Returns:: The input tensor, modified in-place with the all-reduced sum.
Return type:: Tensor