Execution Model

To execute a primitive, a user needs to pass memory arguments and a stream to the dnnl::primitive::execute() member function.

The primitive’s computations are executed on the computational device corresponding to the engine on which the primitive (and memory arguments) were created and happens within the context of the stream.

Engine

Engine is abstraction of a computational device: a CPU, a specific GPU card in the system, etc. Most primitives are created to execute computations on one specific engine. The only exceptions are reorder primitives that transfer data between two different engines.

Engines correspond to and can be constructed from pairs of the DPC++ sycl::device and sycl::context objects. Alternatively, oneDNN itself can create and own the corresponding objects.

struct dnnl::engine

An execution engine.

Public Types

enum class kind

Kinds of engines.

Values:

enumerator any

An unspecified engine.

enumerator cpu

CPU engine.

enumerator gpu

GPU engine.

Public Functions

engine() = default

Constructs an empty engine. An empty engine cannot be used in any operations.

engine(kind akind, size_t index)

Constructs an engine.

Parameters

• akind – The kind of engine to construct.

• index – The index of the engine. Must be less than the value returned by get_count() for this particular kind of engine.

kind get_kind() const

Returns the kind of the engine.

Returns

The kind of the engine.

Public Static Functions

static size_t get_count(kind akind)

Returns the number of engines of a certain kind.

Parameters

akind – The kind of engines to count.

Returns

The number of engines of the specified kind.

engine dnnl::sycl_interop::make_engine(const cl::sycl::device &adevice, const cl::sycl::context &acontext)

Creates an engine object using a specified SYCL device and SYCL context objects.

Parameters

• adevice – SYCL device.

• acontext – SYCL context.

Returns

Engine object for the adevice SYCL device, within the specified acontext SYCL context.

cl::sycl::device dnnl::sycl_interop::get_device(const engine &aengine)

Returns the SYCL device underlying a specified engine object.

Parameters

aengine – Engine object.

Returns

SYCL device object underlying the aengine engine object.

cl::sycl::context dnnl::sycl_interop::get_context(const engine &aengine)

Returns the SYCL context underlying a specified engine object.

Parameters

aengine – Engine object.

Returns

SYCL context object underlying the aengine engine object.

Stream

A stream is an encapsulation of execution context tied to a particular engine. They are passed to dnnl::primitive::execute() when executing a primitive.

Streams correspond to and can be constructed from DPC++ sycl::queue objects. Alternatively, oneDNN itself can create and own the corresponding objects. Streams are considered to be ephemeral and can be created / destroyed as long these operation do not violate DPC++ synchronization requirements.

Similar to DPC++ queues, streams can be in-order and out-of-order (see the relevant portion of the DPC++ specification for the explanation). The desired behavior can be specified using dnnl::stream::flags value. A stream created from a DPC++ queue inherits its behavior.

struct dnnl::stream

An execution stream.

Public Types

enum class flags : unsigned

Stream flags. Can be combined using the bitwise OR operator.

Values:

enumerator in_order

In-order execution.

enumerator out_of_order

Out-of-order execution.

enumerator default_flags

Default stream configuration.

Public Functions

stream()

Constructs an empty stream. An empty stream cannot be used in any operations.

stream(const engine &aengine, flags aflags = flags::default_flags)

Constructs a stream for the specified engine and with behavior controlled by the specified flags.

Parameters

• aengine – Engine to create the stream on.

• aflags – Flags controlling stream behavior.

engine get_engine() const

Returns

The associated engine.

stream &wait()

Waits for all primitives executing in the stream to finish.

Returns

The stream itself.

stream dnnl::sycl_interop::make_stream(const engine &aengine, cl::sycl::queue &aqueue)

Creates a stream for a specified engine and SYCL queue objects.

Parameters

• aengine – Engine object to use for the stream.

• aqueue – SYCL queue to use for the stream.

Returns

Stream object for the aengine engine object, which holds the aqueue SYCL queue object.

cl::sycl::queue dnnl::sycl_interop::get_queue(const stream &astream)

Returns the SYCL queue underlying a specified stream object.

Parameters

astream – Stream object.

Returns

SYCL queue underlying the astream stream object.