Memory Descriptors and Objects¶

Descriptors¶

Memory descriptor is an engine-agnostic logical description of data (number of dimensions, dimension sizes, and data type), and, optionally, the information about the physical format of data in memory. If this information is not known yet, a memory descriptor can be created with format tag set to dnnl::memory::format_tag::any. This allows compute-intensive primitives to chose the most appropriate format for the computations. The user is then responsible for reordering their data into the new format if the formats do not match. See Memory Format Propagation.

A memory descriptor can be initialized either by specifying dimensions, and memory format tag or strides for each of them.

User can query amount of memory required by a memory descriptor using the dnnl::memory::desc::get_size() function. The size of data in general cannot be computed as the product of dimensions multiplied by the size of the data type. So users are required to use this function for better code portability.

Two memory descriptors can be compared using the equality and inequality operators. The comparison is especially useful when checking whether it is necessary to reorder data from the user’s data format to a primitive’s format.

Along with ordinary memory descriptors with all dimensions being positive, oneDNN supports zero-volume memory descriptors with one or more dimensions set to zero. This is used to support the NumPy* convention. If a zero-volume memory is passed to a primitive, the primitive typically does not perform any computations with this memory. For example:

The concatenation primitive would ignore all memory object with zeroes in the concatenation dimension / axis.
A forward convolution with a source memory object with zero in the minibatch dimension would always produce a destination memory object with a zero in the minibatch dimension and perform no computations.
However, a forward convolution with a zero in one of the weights dimensions is ill-defined and is considered to be an error by the library because there is no clear definition on what the output values should be.

Data handle of a zero-volume memory is never accessed.

API

struct dnnl::memory::desc¶

A memory descriptor.

Public Functions

desc()¶: Constructs a zero (empty) memory descriptor. Such a memory descriptor can be used to indicate absence of an argument.

desc(const dims &adims, data_type adata_type, format_tag aformat_tag, bool allow_empty = false)¶

Constructs a memory descriptor.

Note

The logical order of dimensions corresponds to the abc... format tag, and the physical meaning of the dimensions depends both on the primitive that would operate on this memory and the operation context.

Parameters

adims: Tensor dimensions.
adata_type: Data precision/type.
aformat_tag: Memory format tag.
allow_empty: A flag signifying whether construction is allowed to fail without throwing an exception. In this case a zero memory descriptor will be constructed. This flag is optional and defaults to false.

desc(const dims &adims, data_type adata_type, const dims &strides, bool allow_empty = false)¶

Constructs a memory descriptor by strides.

Note

Parameters

adims: Tensor dimensions.
adata_type: Data precision/type.
strides: Strides for each dimension.
allow_empty: A flag signifying whether construction is allowed to fail without throwing an exception. In this case a zero memory descriptor will be constructed. This flag is optional and defaults to false.

desc submemory_desc(const dims &adims, const dims &offsets, bool allow_empty = false) const¶

Constructs a memory descriptor for a region inside an area described by this memory descriptor.

Return

A memory descriptor for the region.

Parameters

adims: Sizes of the region.
offsets: Offsets to the region from the encompassing memory object in each dimension.
allow_empty: A flag signifying whether construction is allowed to fail without throwing an exception. In this case a zero memory descriptor will be returned. This flag is optional and defaults to false.

desc reshape(const dims &adims, bool allow_empty = false) const¶

Constructs a memory descriptor by reshaping an existing one. The new memory descriptor inherits the data type.

The operation ensures that the transformation of the physical memory format corresponds to the transformation of the logical dimensions. If such transformation is impossible, the function either throws an exception (default) or returns a zero memory descriptor depending on the allow_empty flag.

The reshape operation can be described as a combination of the following basic operations:

Add a dimension of size 1. This is always possible.
Remove a dimension of size 1.
Split a dimension into multiple ones. This is possible only if the product of all tensor dimensions stays constant.
Join multiple consecutive dimensions into a single one. This requires that the dimensions are dense in memory and have the same order as their logical counterparts.
- Here, ‘dense’ means: stride for dim[i] == (stride for dim[i + 1]) * dim[i + 1];
- And ‘same order’ means: i < j if and only if stride for dim[j] <= stride for dim[i].

Note

Reshape may fail for optimized memory formats.

Return

A new memory descriptor with new dimensions.

Parameters

adims: New dimensions. The product of dimensions must remain constant.
allow_empty: A flag signifying whether construction is allowed to fail without throwing an exception. In this case a zero memory descriptor will be returned. This flag is optional and defaults to false.

desc permute_axes(const std::vector<int> &permutation, bool allow_empty = false) const¶

Constructs a memory descriptor by permuting axes in an existing one.

The physical memory layout representation is adjusted accordingly to maintain the consistency between the logical and physical parts of the memory descriptor. The new memory descriptor inherits the data type.

The logical axes will be permuted in the following manner:

for (i = 0; i < ndims(); i++)
    new_desc.dims()[permutation[i]] = dims()[i];

Example:

std::vector<int> permutation = {1, 0}; // swap the first and
                                       // the second axes
dnnl::memory::desc in_md(
        {2, 3}, data_type, memory::format_tag::ab);
dnnl::memory::desc expect_out_md(
        {3, 2}, data_type, memory::format_tag::ba);

assert(in_md.permute_axes(permutation) == expect_out_md);

Return

A new memory descriptor with new dimensions.

Parameters

permutation: Axes permutation.
allow_empty: A flag signifying whether construction is allowed to fail without throwing an exception. In this case a zero memory descriptor will be returned. This flag is optional and defaults to false.

memory::dims dims() const¶

Returns dimensions of the memory descriptor.

Potentially expensive due to the data copy involved.

Return: A copy of the dimensions vector.

memory::data_type data_type() const¶

Returns the data type of the memory descriptor.

Return: The data type.

size_t get_size() const¶

Returns size of the memory descriptor in bytes.

Return: The number of bytes required to allocate a memory buffer for the memory object described by this memory descriptor.

bool is_zero() const¶

Checks whether the memory descriptor is zero (empty).

Return: true if the memory descriptor describes an empty memory and false otherwise.

bool operator==(const desc &other) const¶

An equality operator.

Return

Whether this and the other memory descriptors have the same format tag, dimensions, strides, etc.

Parameters

other: Another memory descriptor.

bool operator!=(const desc &other) const¶

An inequality operator.

Return

Whether this and the other memory descriptors describe different memory.

Parameters

other: Another memory descriptor.

Objects¶

Memory objects combine memory descriptors with storage for data (a data handle). With USM, the data handle is simply a pointer to void. The data handle can be queried using dnnl::memory::get_data_handle() and set using dnnl::memory::set_data_handle(). The underlying SYCL buffer, when used, can be queried using dnnl::sycl_interop::get_buffer() and set using dnnl::sycl_interop::set_buffer(). In addition, the memory descriptor and the engine underlying a memory object can be queried using dnnl::memory::get_desc() and dnnl::memory::get_engine() respectively.

API¶

struct dnnl::memory¶

Memory object.

A memory object encapsulates a handle to a memory buffer allocated on a specific engine, tensor dimensions, data type, and memory format, which is the way tensor indices map to offsets in linear memory space. Memory objects are passed to primitives during execution.

Public Functions

memory()¶

Default constructor.

Constructs an empty memory object, which can be used to indicate absence of a parameter.

memory(const desc &md, const engine &aengine, void *handle)¶

Constructs a memory object.

Unless handle is equal to DNNL_MEMORY_NONE, the constructed memory object will have the underlying buffer set. In this case, the buffer will be initialized as if dnnl::memory::set_data_handle() has been called.

See

memory::set_data_handle()

Parameters

md: Memory descriptor.
aengine: Engine to store the data on.
handle: Handle of the memory buffer to use.
- A pointer to the user-allocated buffer. In this case the library doesn’t own the buffer.
- The DNNL_MEMORY_ALLOCATE special value. Instructs the library to allocate the buffer for the memory object. In this case the library owns the buffer and the memory allocation kind of the underlying buffer is dnnl::sycl_interop::memory_kind::usm.
- DNNL_MEMORY_NONE to create dnnl::memory without an underlying buffer.

memory(const desc &md, const engine &aengine)¶

Constructs a memory object.

The underlying buffer for the memory will be allocated by the library. The memory allocation kind of the underlying buffer is dnnl::sycl_interop::memory_kind::usm.

Parameters

md: Memory descriptor.
aengine: Engine to store the data on.

desc get_desc() const¶: Returns the associated memory descriptor.

engine get_engine() const¶: Returns the associated engine.

void *get_data_handle() const¶

Returns the underlying memory buffer.

On the CPU engine, or when using USM, this is a pointer to the allocated memory.

void set_data_handle(void *handle, const stream &astream) const¶

Sets the underlying memory buffer.

This function may write zero values to the memory specified by the handle if the memory object has a zero padding area. This may be time consuming and happens each time this function is called. The operation is always blocking and the stream parameter is a hint.

Note

Even when the memory object is used to hold values that stay constant during the execution of the program (pre-packed weights during inference, for example), the function will still write zeroes to the padding area if it exists. Hence, the handle parameter cannot and does not have a const qualifier.

Parameters

handle: Memory buffer to use. On the CPU engine or when USM is used, the memory buffer is a pointer to the actual data. It must have at least dnnl::memory::desc::get_size() bytes allocated.
astream: Stream to use to execute padding in.

void set_data_handle(void *handle) const¶

Sets the underlying memory buffer.

See documentation for dnnl::memory::set_data_handle(void *, const stream &) const for more information.

Parameters

handle: Memory buffer to use. For the CPU engine, the memory buffer is a pointer to the actual data. It must have at least dnnl::memory::desc::get_size() bytes allocated.

template<typename T = void> T *map_data() const¶

Maps a memory object and returns a host-side pointer to a memory buffer with a copy of its contents.

Mapping enables read/write directly from/to the memory contents for engines that do not support direct memory access.

Mapping is an exclusive operation - a memory object cannot be used in other operations until it is unmapped via dnnl::memory::unmap_data() call.

Note

Any primitives working with the memory should be completed before the memory is mapped. Use dnnl::stream::wait() to synchronize the corresponding execution stream.

Note

The map_data and unmap_data functions are provided mainly for debug and testing purposes and their performance may be suboptimal.

Return

Pointer to the mapped memory.

Template Parameters

T: Data type to return a pointer to.

void unmap_data(void *mapped_ptr) const¶

Unmaps a memory object and writes back any changes made to the previously mapped memory buffer.

Note

The map_data and unmap_data functions are provided mainly for debug and testing purposes and their performance may be suboptimal.

Parameters

mapped_ptr: A pointer previously returned by dnnl::memory::map_data().

enum dnnl::sycl_interop::memory_kind¶

Memory allocation kinds.

Values:

enumerator usm¶: USM memory allocation kind.

enumerator buffer¶: Buffer memory allocation kind.

memory dnnl::sycl_interop::make_memory(const memory::desc &adesc, const engine &aengine, memory_kind akind, void *ahandle = DNNL_MEMORY_ALLOCATE)¶

Creates a memory object of a specified description and of a specified memory allocation kind, for a specified engine.

Note

If akind is dnnl::sycl_interop::memory_kind::buffer, and ahandle is not DNNL_MEMORY_ALLOCATE or DNNL_MEMORY_NONE, an exception is thrown.

Return

Memory object described by adesc memory descriptor, which has akind memory allocation kind, and is attached to the aengine engine.

Parameters

adesc: Memory descriptor that describes the data.
aengine: Engine to store the data on.
akind: Memory allocation kind.
ahandle: Handle of the memory data to use. This parameter is optional. By default, the underlying memory buffer is allocated internally, its memory allocation kind is dnnl::sycl_interop::memory_kind::usm, and the library owns the buffer. If handle is provided, the library does not own the buffer.

memory dnnl::sycl_interop::make_memory(const memory::desc &adesc, const stream &astream, memory_kind akind, void *ahandle = DNNL_MEMORY_ALLOCATE)¶

Creates a memory object of a specified description and of a specified memory allocation kind, for a specified stream.

Note

If akind is dnnl::sycl_interop::memory_kind::buffer, and ahandle is not DNNL_MEMORY_ALLOCATE or DNNL_MEMORY_NONE, an exception is thrown.

Return

Memory object described by adesc memory descriptor, which has akind memory allocation kind, and used withing the astream stream.

Parameters

adesc: Memory descriptor that describes the data.
astream: Stream object where the data is used.
akind: Memory allocation kind.
ahandle: Handle of the memory data to use. This parameter is optional. By default, the underlying memory buffer is allocated internally, its memory allocation kind is dnnl::sycl_interop::memory_kind::usm, and the library owns the buffer. If handle is provided, the library does not own the buffer.

template<typename T, int ndims> memory dnnl::sycl_interop::make_memory(const memory::desc &adesc, const engine &aengine, cl::sycl::buffer<T, ndims> abuffer)¶

Creates a memory object using a specified SYCL buffer.

Note

When such memory object is created, it is implied that its memory allocation kind is dnnl::sycl_interop::memory_kind::buffer.

Return

Memory object which holds a abuffer SYCL buffer described by the adesc memory descriptor and attached to the aengine engine.

Template Parameters

T: Data type of the specified SYCL buffer.
ndims: Number of dimensions of the specified SYCL buffer.

Parameters

adesc: Memory descriptor that describes the data within the specified buffer.
aengine: Engine to store the data on.
abuffer: SYCL buffer.

template<typename T, int ndims> memory dnnl::sycl_interop::make_memory(const memory::desc &adesc, const stream &astream, cl::sycl::buffer<T, ndims> abuffer)¶

Creates a memory object using a specified SYCL buffer.

Note

When such memory object is created, it is implied that its memory allocation kind is dnnl::sycl_interop::memory_kind::buffer.

Return

Memory object which holds a abuffer SYCL buffer described by the adesc memory descriptor and used within the astream stream.

Template Parameters

T: Data type of the specified SYCL buffer.
ndims: Number of dimensions of the specified SYCL buffer.

Parameters

adesc: Memory descriptor that describes the data within the specified buffer.
astream: Stream object where the data is used.
abuffer: SYCL buffer.

memory_kind dnnl::sycl_interop::get_memory_kind(const memory &amemory)¶

Returns the memory allocation kind of a specified memory object.

Note

The memory allocation kind of a memory object could be changed during its lifetime, by setting the USM handle or SYCL buffer of said memory object.

Return

Memory allocation kind of the amemory memory object.

Parameters

amemory: Memory object.

template<typename T, int ndims> void dnnl::sycl_interop::set_buffer(memory &amemory, cl::sycl::buffer<T, ndims> abuffer)¶

Sets the SYCL buffer underlying a specified memory object.

Note

By setting the SYCL buffer of a memory object its memory allocation kind will be changed to dnnl::sycl_interop::memory_kind::buffer.

Template Parameters

T: Data type of the specified SYCL buffer.
ndims: Number of dimensions of the specified SYCL buffer.

Parameters

amemory: Memory object that will store the abuffer SYCL buffer.
abuffer: SYCL buffer to be stored in the amemory memory object.

template<typename T, int ndims> void dnnl::sycl_interop::set_buffer(memory &amemory, cl::sycl::buffer<T, ndims> abuffer, stream &astream)¶

Sets the SYCL buffer underlying a specified memory object in a specified stream.

Template Parameters

T: Data type of the specified SYCL buffer.
ndims: Number of dimensions of the specified SYCL buffer.

Parameters

amemory: Memory object that will store the abuffer SYCL buffer.
abuffer: SYCL buffer to be stored in the amemory memory object and used in the astream stream.
astream: Stream object within which the amemory memory object is used.

template<typename T, int ndims = 1> cl::sycl::buffer<T, ndims> dnnl::sycl_interop::get_buffer(const memory &amemory)¶

Returns the SYCL buffer underlying a specified memory object.

Return

SYCL buffer of type T with ndims dimensions, underlying the amemory memory object.

Template Parameters

T: Data type of the specified SYCL buffer.
ndims: Number of dimensions of the specified buffer.

Parameters

amemory: Memory object.

DNNL_MEMORY_NONE¶: Special pointer value that indicates that a memory object should not have an underlying buffer.

DNNL_MEMORY_ALLOCATE¶: Special pointer value that indicates that the library needs to allocate an underlying buffer for a memory object.

Memory Formats Primitives

oneAPI Specification 1.1-provisional-rev-1 documentation

Memory Descriptors and Objects¶

Descriptors¶

Objects¶

API¶