Each of N workers is responsible for: summing 1/N gradients collected from (N-1) peers Distributing the sums of the (N-1) peers ”Ring” red