.rst .pdf

open issue suggest edit

Contents

Elementwise

The elementwise primitive applies an operation to every element of the tensor. Variable names follow the standard Conventions.

\[\dst(\overline{x}) = Operation(\src(\overline{x})),\]

for \(\overline{x} = (x_0, \ldots, x_n)\).

Forward

The following forward operations are supported. Here \(s\) and \(d\) denote \(\src\) and \(\dst\), tensor values respectively.

Elementwise algorithm	Forward formula
eltwise_abs	\(d = \begin{cases} s & \text{if}\ s > 0 \\ -s & \text{if}\ s \leq 0 \end{cases}\)
eltwise_bounded_relu	\(d = \begin{cases} \alpha & \text{if}\ s > \alpha \geq 0 \\ s & \text{if}\ 0 < s \leq \alpha \\ 0 & \text{if}\ s \leq 0 \end{cases}\)
eltwise_clip	\(d = \begin{cases} \beta & \text{if}\ s > \beta \geq \alpha \\ s & \text{if}\ \alpha < s \leq \beta \\ \alpha & \text{if}\ s \leq \alpha \end{cases}\)
eltwise_elu, eltwise_elu_use_dst_for_bwd	\(d = \begin{cases} s & \text{if}\ s > 0 \\ \alpha (e^s - 1) & \text{if}\ s \leq 0 \end{cases}\)
eltwise_exp, eltwise_exp_use_dst_for_bwd	\(d = e^s\)
eltwise_gelu_erf	\(d = 0.5 s (1 + erf[\frac{s}{\sqrt{2}}])\)
eltwise_gelu_tanh	\(d = 0.5 s (1 + tanh[\sqrt{\frac{2}{\pi}} (s + 0.044715 s^3)])\)
eltwise_linear	\(d = \alpha s + \beta\)
eltwise_log	\(d = \log_{e}{s}\)
eltwise_logistic, eltwise_logistic_use_dst_for_bwd	\(d = \frac{1}{1+e^{-s}}\)
eltwise_pow	\(d = \alpha s^{\beta}\)
eltwise_relu, eltwise_relu_use_dst_for_bwd	\(d = \begin{cases} s & \text{if}\ s > 0 \\ \alpha s & \text{if}\ s \leq 0 \end{cases}\)
eltwise_round	\(d = round(s)\)
eltwise_soft_relu	\(d = \log_{e}(1+e^s)\)
eltwise_sqrt, eltwise_sqrt_use_dst_for_bwd	\(d = \sqrt{s}\)
eltwise_square	\(d = s^2\)
eltwise_swish	\(d = \frac{s}{1+e^{-\alpha s}}\)
eltwise_tanh, eltwise_tanh_use_dst_for_bwd	\(d = \tanh{s}\)

Backward

The backward propagation computes \(\diffsrc(\overline{s})\), based on \(\diffdst(\overline{s})\) and \(\src(\overline{s})\). However, some operations support a computation using \(\dst(\overline{s})\) memory produced during forward propagation. Refer to the table above for a list of operations supporting destination as input memory and the corresponding formulas.

The following backward operations are supported. Here \(s\), \(d\), \(ds\) and \(dd\) denote \(\src\), \(\dst\), \(\diffsrc\), and a \(\diffdst\) tensor values respectively.

Elementwise algorithm	Backward formula
eltwise_abs	\(ds = \begin{cases} dd & \text{if}\ s > 0 \\ -dd & \text{if}\ s < 0 \\ 0 & \text{if}\ s = 0 \end{cases}\)
eltwise_bounded_relu	\(ds = \begin{cases} dd & \text{if}\ 0 < s \leq \alpha, \\ 0 & \text{otherwise}\ \end{cases}\)
eltwise_clip	\(ds = \begin{cases} dd & \text{if}\ \alpha < s \leq \beta \\ 0 & \text{otherwise}\ \end{cases}\)
eltwise_elu	\(ds = \begin{cases} dd & \text{if}\ s > 0 \\ dd \cdot \alpha e^s & \text{if}\ s \leq 0 \end{cases}\)
eltwise_elu_use_dst_for_bwd	\(ds = \begin{cases} dd & \text{if}\ d > 0 \\ dd \cdot (d + \alpha) & \text{if}\ d \leq 0 \end{cases}\) only if \(\alpha \geq 0\)
eltwise_exp	\(ds = dd \cdot e^s\)
eltwise_exp_use_dst_for_bwd	\(ds = dd \cdot d\)
eltwise_gelu_erf	\(ds = dd \cdot \left(0.5 + 0.5 \, \mathrm{erf}\left({\frac{s}{\sqrt{2}}}\right) + \frac{s}{\sqrt{2\pi}}e^{-0.5s^{2}}\right)\)
eltwise_gelu_tanh	\(\begin{array}{rl} ds = & dd \\ & \cdot 0.5 (1 + \tanh[\sqrt{\frac{2}{\pi}} (s + 0.044715 s^3)]) \\ & \cdot (1 + \sqrt{\frac{2}{\pi}} (s + 0.134145 s^3) \\ & \cdot (1 - \tanh[\sqrt{\frac{2}{\pi}} (s + 0.044715 s^3)]) ) \end{array}\)
eltwise_linear	\(ds = \alpha \cdot dd\)
eltwise_log	\(ds = \frac{dd}{s}\)
eltwise_logistic	\(ds = \frac{dd}{1+e^{-s}} \cdot (1 - \frac{1}{1+e^{-s}})\)
eltwise_logistic_use_dst_for_bwd	\(ds = dd \cdot d \cdot (1 - d)\)
eltwise_pow	\(ds = dd \cdot \alpha \beta s^{\beta - 1}\)
eltwise_relu	\(ds = \begin{cases} dd & \text{if}\ s > 0 \\ \alpha \cdot dd & \text{if}\ s \leq 0 \end{cases}\)
eltwise_relu_use_dst_for_bwd	\(ds = \begin{cases} dd & \text{if}\ d > 0 \\ \alpha \cdot dd & \text{if}\ d \leq 0 \end{cases}\) only if \(alpha \geq 0\)
eltwise_soft_relu	\(ds = \frac{dd}{1 + e^{-s}}\)
eltwise_sqrt	\(ds = \frac{dd}{2\sqrt{s}}\)
eltwise_sqrt_use_dst_for_bwd	\(ds = \frac{dd}{2d}\)
eltwise_square	\(ds = dd \cdot 2 s\)
eltwise_swish	\(ds = \frac{dd}{1 + e^{-\alpha s}}(1 + \alpha s (1 - \frac{1}{1 + e^{-\alpha s}}))\)
eltwise_tanh	\(ds = dd \cdot (1 - \tanh^2{s})\)
eltwise_tanh_use_dst_for_bwd	\(ds = dd \cdot (1 - d^2)\)

Difference Between Forward Training and Forward Inference

There is no difference between the #dnnl_forward_training and #dnnl_forward_inference propagation kinds.

Execution Arguments

When executed, the inputs and outputs should be mapped to an execution argument index as specified by the following table.

Primitive input/output	Execution argument index
\(\src\)	DNNL_ARG_SRC
\(\dst\)	DNNL_ARG_DST
\(\diffsrc\)	DNNL_ARG_DIFF_SRC
\(\diffdst\)	DNNL_ARG_DIFF_DST

Operation Details

1. The dnnl::eltwise_forward::desc::desc() and dnnl::eltwise_backward::desc::desc() constructors take both parameters \(\alpha\), and \(\beta\). These parameters are ignored if they are unused by the algorithm.

2. The memory format and data type for \(\src\) and \(\dst\) are assumed to be the same, and in the API are typically denoted as data (for example dnnl::eltwise_forward::desc::desc() has a data_desc argument). The same holds for \(\diffsrc\) and \(\diffdst\). The corresponding memory descriptors are denoted as diff_data_desc.

3. Both forward and backward propagation support in-place operations, meaning that \(\src\) can be used as input and output for forward propagation, and \(\diffdst\) can be used as input and output for backward propagation. In case of an in-place operation, the original data will be overwritten. Note, however, that some algorithms for backward propagation require original \(\src\), hence the corresponding forward propagation should not be performed in-place for those algorithms. Algorithms that use \(\dst\) for backward propagation can be safely done in-place.

4. For some operations it might be beneficial to compute backward propagation based on \(\dst(\overline{s})\), rather than on \(\src(\overline{s})\), for improved performance.

Note

For operations supporting destination memory as input, \(\dst\) can be used instead of \(\src\) when backward propagation is computed. This enables several performance optimizations (see the tips below).

Data Type Support

The eltwise primitive should support the following combinations of data types.

Note

Here we abbreviate data types names for readability. For example, dnnl::memory::data_type::f32 is abbreviated to f32.

Propagation	Source / Destination	Intermediate data type
forward / backward	f32, bf16	f32
forward	f16	f16
forward	s32 / s8 / u8	f32

Here the intermediate data type means that the values coming in are first converted to the intermediate data type, then the operation is applied, and finally the result is converted to the output data type.

Data Representation

The eltwise primitive works with arbitrary data tensors. There is no special meaning associated with any logical dimensions.

Post-ops and Attributes

The eltwise primitive does not have to support any post-ops or attributes.

API

struct dnnl::eltwise_forward : public dnnl::primitive

Elementwise unary operation forward propagation primitive.

Public Functions

eltwise_forward()

Default constructor. Produces an empty object.

eltwise_forward(const primitive_desc &pd)

Constructs an eltwise forward propagation primitive.

Parameters

• pd: Primitive descriptor for an eltwise forward propagation primitive.

struct desc

Descriptor for an elementwise forward propagation primitive.

Public Functions

desc(prop_kind aprop_kind, algorithm aalgorithm, const memory::desc &data_desc, float alpha = 0, float beta = 0)

Constructs a descriptor for an elementwise forward propagation primitive.

Parameters

• aprop_kind: Propagation kind. Possible values are dnnl::prop_kind::forward_training, and dnnl::prop_kind::forward_inference.

• aalgorithm: Elementwise algorithm kind.

• data_desc: Source and destination memory descriptors.

• alpha: The alpha parameter for the elementwise operation. Specific meaning depends on the algorithm.

• beta: The beta parameter for the elementwise operation. Specific meaning depends on the algorithm.

struct primitive_desc : public dnnl::primitive_desc

Primitive descriptor for an elementwise forward propagation primitive.

Public Functions

primitive_desc()

Default constructor. Produces an empty object.

primitive_desc(const desc &adesc, const engine &aengine, bool allow_empty = false)

Constructs a primitive descriptor for an elementwise forward propagation primitive.

Parameters

• adesc: Descriptor for an elementwise forward propagation primitive.

• aengine: Engine to use.

• allow_empty: A flag signifying whether construction is allowed to fail without throwing an exception. In this case an empty object will be produced. This flag is optional and defaults to false.

primitive_desc(const desc &adesc, const primitive_attr &attr, const engine &aengine, bool allow_empty = false)

Constructs a primitive descriptor for an elementwise forward propagation primitive.

Parameters

• adesc: Descriptor for an elementwise forward propagation primitive.

• aengine: Engine to use.

• attr: Primitive attributes to use.

• allow_empty: A flag signifying whether construction is allowed to fail without throwing an exception. In this case an empty object will be produced. This flag is optional and defaults to false.

memory::desc src_desc() const

Returns a source memory descriptor.

Return

Source memory descriptor.

Return

A zero memory descriptor if the primitive does not have a source parameter.

memory::desc dst_desc() const

Returns a destination memory descriptor.

Return

Destination memory descriptor.

Return

A zero memory descriptor if the primitive does not have a destination parameter.

struct dnnl::eltwise_backward : public dnnl::primitive

Elementwise unary operation backward propagation primitive.

See

eltwise_forward

Public Functions

eltwise_backward()

Default constructor. Produces an empty object.

eltwise_backward(const primitive_desc &pd)

Constructs an eltwise backward propagation primitive.

Parameters

• pd: Primitive descriptor for an eltwise backward propagation primitive.

struct desc

Descriptor for an elementwise backward propagation primitive.

Public Functions

desc(algorithm aalgorithm, const memory::desc &diff_data_desc, const memory::desc &data_desc, float alpha = 0, float beta = 0)

Constructs a descriptor for an elementwise backward propagation primitive.

Parameters

• aalgorithm: Elementwise algorithm kind.

• diff_data_desc: Diff source and destination memory descriptors.

• data_desc: Source memory descriptor.

• alpha: The alpha parameter for the elementwise operation. Specific meaning depends on the algorithm.

• beta: The beta parameter for the elementwise operation. Specific meaning depends on the algorithm.

struct primitive_desc : public dnnl::primitive_desc

Primitive descriptor for eltwise backward propagation.

Public Functions

primitive_desc()

Default constructor. Produces an empty object.

primitive_desc(const desc &adesc, const engine &aengine, const eltwise_forward::primitive_desc &hint_fwd_pd, bool allow_empty = false)

Constructs a primitive descriptor for an elementwise backward propagation primitive.

Parameters

• adesc: Descriptor for an elementwise backward propagation primitive.

• aengine: Engine to use.

• hint_fwd_pd: Primitive descriptor for an elementwise forward propagation primitive. It is used as a hint for deciding which memory format to use.

• allow_empty: A flag signifying whether construction is allowed to fail without throwing an exception. In this case an empty object will be produced. This flag is optional and defaults to false.

primitive_desc(const desc &adesc, const primitive_attr &attr, const engine &aengine, const eltwise_forward::primitive_desc &hint_fwd_pd, bool allow_empty = false)

Constructs a primitive descriptor for an elementwise backward propagation primitive.

Parameters

• adesc: Descriptor for an elementwise backward propagation primitive.

• attr: Primitive attributes to use.

• aengine: Engine to use.

• hint_fwd_pd: Primitive descriptor for an elementwise forward propagation primitive. It is used as a hint for deciding which memory format to use.

• allow_empty: A flag signifying whether construction is allowed to fail without throwing an exception. In this case an empty object will be produced. This flag is optional and defaults to false.

memory::desc src_desc() const

Returns a source memory descriptor.

Return

Source memory descriptor.

Return

A zero memory descriptor if the primitive does not have a source parameter.

memory::desc diff_src_desc() const

Returns a diff source memory descriptor.

Return

Diff source memory descriptor.

Return

A zero memory descriptor if the primitive does not have a diff source memory with.

memory::desc diff_dst_desc() const

Returns a diff destination memory descriptor.

Return

Diff destination memory descriptor.

Return

A zero memory descriptor if the primitive does not have a diff destination parameter.

Convolution and Deconvolution Inner Product