Input (Replicated) Compute (No Comm) Partial Result AllReduce (Communication)
Step 0 / 5