Contents

Reorder

A primitive to copy data between two memory objects. This primitive is typically used to change the way that the data is laid out in memory.

The reorder primitive copies data between different memory formats but does not change the tensor from mathematical perspective. Variable names follow the standard Conventions.

\[\dst(\overline{x}) = \src(\overline{x})\]

for \(\overline{x} = (x_0, \ldots, x_n)\).

As described in Introduction in order to achieve the best performance some primitives (such as convolution) require special memory format which is typically referred to as an optimized memory format. The optimized memory format may match or may not match memory format that data is currently kept in. In this case a user can use reorder primitive to copy (reorder) the data between the memory formats.

Using the attributes and post-ops users can also use reorder primitive to quantize the data (and if necessary change the memory format simultaneously).

Execution Arguments

When executed, the inputs and outputs should be mapped to an execution argument index as specified by the following table.

Primitive input/output

Execution argument index

\(\src\)

DNNL_ARG_FROM

\(\dst\)

DNNL_ARG_TO

Operation Details

  1. The reorder primitive requires the source and destination tensors to have the same shape. Implicit broadcasting is not supported.

  2. While in most of the cases the reorder should be able to handle arbitrary source and destination memory formats and data types, it might happen than some combinations are not implemented. For instance:

    • Reorder implementations between weights in non-plain memory formats might be limited (but if encountered in real practice should be treated as a bug and reported to oneDNN team);

    • Weights in one Winograd format cannot be reordered to the weights of the other Winograd format;

    • Quantized weights for convolution with #dnnl_s8 source data type cannot be dequantized back to the #dnnl_f32 data type;

  3. To alleviate the problem a user may rely on fact that the reorder from original plain memory format and user’s data type to the optimized format with chosen data type should be always implemented.

Data Types Support

The reorder primitive supports arbitrary data types for the source and destination.

When converting the data from one data type to a smaller one saturation is used. For instance:

reorder(src={1024, data_type=f32}, dst={, data_type=s8})
// dst == {127}

reorder(src={-124, data_type=f32}, dst={, data_type=u8})
// dst == {0}

Data Representation

The reorder primitive works with arbitrary data tensors. There is no special meaning associated with any logical dimensions.

Post-ops and Attributes

The reorder primitive should support the following attributes and post-ops:

Type

Operation

Description

Restrictions

Attribute

Output scales

Scales the result by given scale factor(s)

Post-op

Sum

Adds the operation result to the destination tensor instead of overwriting it

For instance, the following pseudo-code

reorder(
        src = {dims={N, C, H, W}, data_type=dt_src, memory_format=fmt_src},
        dst = {dims={N, C, H, W}, data_type=dt_dst, memory_format=fmt_dst},
        attr ={
            output_scale=alpha,
            post-ops = { sum={scale=beta} },
        })

would lead to the following operation:

\[\dst(\overline{x}) = \alpha \cdot \src(\overline{x}) + \beta \cdot \dst(\overline{x})\]

Note

The intermediate operations are being done using single precision floating point data type.

API

struct dnnl::reorder : public dnnl::primitive

Reorder primitive.

Public Functions

reorder()

Default constructor. Produces an empty object.

reorder(const primitive_desc &pd)

Constructs a reorder primitive.

Parameters

  • pd: Primitive descriptor for reorder primitive.

reorder(const memory &src, const memory &dst, const primitive_attr &attr = primitive_attr())

Constructs a reorder primitive that would reorder data between memory objects having the same memory descriptors as memory objects src and dst.

Parameters

  • src: Source memory object.

  • dst: Destination memory object.

  • attr: Primitive attributes to use (optional).

void execute(const stream &astream, memory &src, memory &dst) const

Executes the reorder primitive.

Parameters

  • astream: Stream object. The stream must belong to the same engine as the primitive.

  • src: Source memory object.

  • dst: Destination memory object.

struct primitive_desc : public dnnl::primitive_desc_base

Primitive descriptor for a reorder primitive.

Public Functions

primitive_desc()

Default constructor. Produces an empty object.

primitive_desc(const engine &src_engine, const memory::desc &src_md, const engine &dst_engine, const memory::desc &dst_md, const primitive_attr &attr = primitive_attr(), bool allow_empty = false)

Constructs a primitive descriptor for reorder primitive.

Note

If allow_empty is true, the constructor does not throw if a primitive descriptor cannot be created.

Parameters

  • src_engine: Engine on which the source memory object will be located.

  • src_md: Source memory descriptor.

  • dst_engine: Engine on which the destination memory object will be located.

  • dst_md: Destination memory descriptor.

  • attr: Primitive attributes to use (optional).

  • allow_empty: A flag signifying whether construction is allowed to fail without throwing an exception. In this case an empty object will be produced. This flag is optional and defaults to false.

primitive_desc(const memory &src, const memory &dst, const primitive_attr &attr = primitive_attr(), bool allow_empty = false)

Constructs a primitive descriptor for reorder primitive.

Parameters

  • src: Source memory object. It is used to obtain the source memory descriptor and engine.

  • dst: Destination memory object. It is used to obtain the destination memory descriptor and engine.

  • attr: Primitive attributes to use (optional).

  • allow_empty: A flag signifying whether construction is allowed to fail without throwing an exception. In this case an empty object will be produced. This flag is optional and defaults to false.

engine get_src_engine() const

Returns the engine on which the source memory is allocated.

Return

The engine on which the source memory is allocated.

engine get_dst_engine() const

Returns the engine on which the destination memory is allocated.

Return

The engine on which the destination memory is allocated.

memory::desc src_desc() const

Returns a source memory descriptor.

Return

Source memory descriptor.

Return

A zero memory descriptor if the primitive does not have a source parameter.

memory::desc dst_desc() const

Returns a destination memory descriptor.

Return

Destination memory descriptor.

Return

A zero memory descriptor if the primitive does not have a destination parameter.

Pooling Resampling