Memory Descriptors and Objects#

Descriptors#

Memory descriptor is an engine-agnostic logical description of data (number of dimensions, dimension sizes, and data type), and, optionally, the information about the physical format of data in memory. If this information is not known yet, a memory descriptor can be created with format tag set to dnnl::memory::format_tag::any. This allows compute-intensive primitives to chose the most appropriate physical format for the computations. The user is then responsible for reordering their data into the proper format they do not match. See Memory Format Propagation for more details.

A memory descriptor can be initialized either by specifying dimensions, and memory format tag or strides for each of them.

User can query amount of memory required by a memory descriptor using the dnnl::memory::desc::get_size() function. The size of data in general cannot be computed as the product of dimensions multiplied by the size of the data type. So users are required to use this function for better code portability.

Two memory descriptors can be compared using the equality and inequality operators. The comparison is especially useful when checking whether it is necessary to reorder data from the user’s data format to a primitive’s format.

Along with ordinary memory descriptors with all dimensions being positive, oneDNN supports zero-volume memory descriptors with one or more dimensions set to zero. This is used to support the NumPy* convention. If a zero-volume memory is passed to a primitive, the primitive typically does not perform any computations with this memory. For example:

The concatenation primitive would ignore all memory object with zeroes in the concatenation dimension / axis.
A forward convolution with a source memory object with zero in the minibatch dimension would always produce a destination memory object with a zero in the minibatch dimension and perform no computations.
However, a forward convolution with a zero in one of the weights dimensions is ill-defined and is considered to be an error by the library because there is no clear definition on what the output values should be.

Data handle of a zero-volume memory is never accessed.

Note

Some primitives support implicit broadcast semantic when a given tensor has a dimensions set to 1 (similar to NumPY broadcast semantic). In particular, if an operation expects two tensors with same dimensions, and one of the descriptors has some dimension set to 1, that dimension will be implicitly broadcasted to match the other tensor dimension.

API

struct desc#

A memory descriptor.

Public Functions

desc()#: Constructs a zero (empty) memory descriptor. Such a memory descriptor can be used to indicate absence of an argument.

desc(const dims &adims, data_type adata_type, format_tag aformat_tag, bool allow_empty = false)#

Constructs a memory descriptor.

Note

The logical order of dimensions corresponds to the abc... format tag, and the physical meaning of the dimensions depends both on the primitive that would operate on this memory and the operation context.

Parameters:

adims – Tensor dimensions.
adata_type – Data precision/type.
aformat_tag – Memory format tag.
allow_empty – A flag signifying whether construction is allowed to fail without throwing an exception. In this case a zero memory descriptor will be constructed. This flag is optional and defaults to false.

desc(const dims &adims, data_type adata_type, const dims &strides, bool allow_empty = false)#

Constructs a memory descriptor by strides.

Note

Parameters:

adims – Tensor dimensions.
adata_type – Data precision/type.
strides – Strides for each dimension.
allow_empty – A flag signifying whether construction is allowed to fail without throwing an exception. In this case a zero memory descriptor will be constructed. This flag is optional and defaults to false.

desc submemory_desc(const dims &adims, const dims &offsets, bool allow_empty = false) const#

Constructs a memory descriptor for a region inside an area described by this memory descriptor.

Parameters:

adims – Sizes of the region.
offsets – Offsets to the region from the encompassing memory object in each dimension.
allow_empty – A flag signifying whether construction is allowed to fail without throwing an exception. In this case a zero memory descriptor will be returned. This flag is optional and defaults to false.

Returns:

A memory descriptor for the region.

desc reshape(const dims &adims, bool allow_empty = false) const#

Constructs a memory descriptor by reshaping an existing one. The new memory descriptor inherits the data type.

The operation ensures that the transformation of the physical memory format corresponds to the transformation of the logical dimensions. If such transformation is impossible, the function either throws an exception (default) or returns a zero memory descriptor depending on the allow_empty flag.

The reshape operation can be described as a combination of the following basic operations:

Add a dimension of size 1. This is always possible.
Remove a dimension of size 1.
Split a dimension into multiple ones. This is possible only if the product of all tensor dimensions stays constant.
Join multiple consecutive dimensions into a single one. This requires that the dimensions are dense in memory and have the same order as their logical counterparts.
- Here, ‘dense’ means: stride for dim[i] == (stride for dim[i + 1]) * dim[i + 1];
- And ‘same order’ means: i < j if and only if stride for dim[j] <= stride for dim[i].

Note

Reshape may fail for optimized memory formats.

Parameters:

adims – New dimensions. The product of dimensions must remain constant.
allow_empty – A flag signifying whether construction is allowed to fail without throwing an exception. In this case a zero memory descriptor will be returned. This flag is optional and defaults to false.

Returns:

A new memory descriptor with new dimensions.

desc permute_axes(const std::vector<int> &permutation, bool allow_empty = false) const#

Constructs a memory descriptor by permuting axes in an existing one.

The physical memory layout representation is adjusted accordingly to maintain the consistency between the logical and physical parts of the memory descriptor. The new memory descriptor inherits the data type.

The logical axes will be permuted in the following manner:

for (i = 0; i < ndims(); i++)
    new_desc.dims()[permutation[i]] = dims()[i];

Example:

std::vector<int> permutation = {1, 0}; // swap the first and
                                       // the second axes
dnnl::memory::desc in_md(
        {2, 3}, data_type, memory::format_tag::ab);
dnnl::memory::desc expect_out_md(
        {3, 2}, data_type, memory::format_tag::ba);

assert(in_md.permute_axes(permutation) == expect_out_md);

Parameters:

permutation – Axes permutation.
allow_empty – A flag signifying whether construction is allowed to fail without throwing an exception. In this case a zero memory descriptor will be returned. This flag is optional and defaults to false.

Returns:

A new memory descriptor with new dimensions.

memory::dims dims() const#

Returns dimensions of the memory descriptor.

Potentially expensive due to the data copy involved.

Returns:: A copy of the dimensions vector.

memory::data_type data_type() const#

Returns the data type of the memory descriptor.

Returns:: The data type.

size_t get_size() const#

Returns size of the memory descriptor in bytes.

Returns:: The number of bytes required to allocate a memory buffer for the memory object described by this memory descriptor.

bool is_zero() const#

Checks whether the memory descriptor is zero (empty).

Returns:: true if the memory descriptor describes an empty memory and false otherwise.

bool operator==(const desc &other) const#

An equality operator.

Parameters:: other – Another memory descriptor.
Returns:: Whether this and the other memory descriptors have the same format tag, dimensions, strides, etc.

bool operator!=(const desc &other) const#

An inequality operator.

Parameters:: other – Another memory descriptor.
Returns:: Whether this and the other memory descriptors describe different memory.

Objects#

Memory objects combine memory descriptors with storage for data (a data handle). With USM, the data handle is simply a pointer to void. The data handle can be queried using dnnl::memory::get_data_handle() and set using dnnl::memory::set_data_handle(). The underlying SYCL buffer, when used, can be queried using dnnl::sycl_interop::get_buffer() and set using dnnl::sycl_interop::set_buffer(). In addition, the memory descriptor and the engine underlying a memory object can be queried using dnnl::memory::get_desc() and dnnl::memory::get_engine() respectively.

API#

struct memory#

Memory object.

A memory object encapsulates a handle to a memory buffer allocated on a specific engine, tensor dimensions, data type, and memory format, which is the way tensor indices map to offsets in linear memory space. Memory objects are passed to primitives during execution.

Public Functions

memory()#

Default constructor.

Constructs an empty memory object, which can be used to indicate absence of a parameter.

memory(const desc &md, const engine &aengine, void *handle)#

Constructs a memory object.

Unless handle is equal to DNNL_MEMORY_NONE, the constructed memory object will have the underlying buffer set. In this case, the buffer will be initialized as if dnnl::memory::set_data_handle() has been called.

Memory Descriptors and Objects

Contents

Memory Descriptors and Objects#

Descriptors#

Objects#

API#