Reductions

oneCCL specification defines the following reduction operations for Allreduce, Reduce and ReduceScatter collective operations:

enum class reduction
{
    sum    = /* unspecified */,
    prod   = /* unspecified */,
    min    = /* unspecified */,
    max    = /* unspecified */,
    custom = /* unspecified */
};
reduction::sum

elementwise summation

reduction::prod

elementwise multiplication

reduction::min

elementwise min

reduction::max

elementwise max

reduction::custom
specify user-defined reduction operation
the actual reduction function must be passed through reduction_fn operation attribute

Operation Attributes

Datatypes Collective Operations