Common Definitions#

This section lists common types and definitions used by all or multiple graph operations.

Logical tensor enums and type#

struct logical_tensor#

Logical tensor object.

Public Types

enum class data_type#

Data Type.

Values:

enumerator undef#

enumerator f16#: 16-bit/half-precision floating point.

enumerator bf16#: non-standard 16-bit (bfloat16 w/ 7 bit mantissa) floating point.

enumerator f32#: 32-bit/single-precision floating point.

enumerator s32#: 32-bit signed integer.

enumerator s8#: 8-bit signed integer.

enumerator u8#: 8-bit unsigned integer.

enum class layout_type#

Layout type.

Values:

enumerator undef#: Undefined layout type.

enumerator any#: Any means to let the library to decide the layout for a tensor during partition compilation.

enumerator strided#: Strided means that the layout of a tensor is determined by the strides field in the logical tensor.

enumerator opaque#: Opaque means that the layout of a tensor is the library specific. Usually, an opaque layout is generated by a partition which is compiled with layout type any.

enum class property_type#

Tensor property.

Values:

enumerator undef#: Undefined tensor property.

enumerator variable#: Variable means the tensor may be changed during computation or between different iterations.

enumerator constant#: Constant means the tensor will keep unchanged during computation and between different iterations. It’s useful for the library to apply optimizations for constant tensors or cache constant tensors inside the library. For example, constant weight tensors in inference scenarios.

using dim = dnnl_dim_t#: Integer type for representing dimension sizes and indices.

using dims = std::vector<dim>#: Vector of dimensions. Implementations are free to force a limit on the vector’s length.

Public Functions

logical_tensor() = default#: Default constructor Construct an empty object

logical_tensor(const logical_tensor &other) = default#: Copy constructor.

logical_tensor &operator=(const logical_tensor &other) = default#: Assignment operator.

logical_tensor(size_t tid, data_type dtype, int32_t ndims, layout_type ltype, property_type ptype = property_type::undef)#

Constructs a logical tensor object with ID, data type, ndims, layout type, and property type.

Parameters:

tid – Logical tensor ID.
dtype – Elements data type.
ndims – Number of dimensions. DNNL_GRAPH_UNKNOWN_NDIMS for an unknown number of dimensions, 0 for a scalar tensor.
ltype – Layout type.
ptype – Property type.

inline logical_tensor(size_t tid, data_type dtype, layout_type ltype = layout_type::undef)#

Delegated constructor.

Parameters:

tid – Logical tensor ID.
dtype – Elements data type.
ltype – Layout type.

logical_tensor(size_t tid, data_type dtype, const dims &adims, layout_type ltype, property_type ptype = property_type::undef)#

Constructs a logical tensor object with basic information and detailed dims.

Parameters:

tid – Logical tensor ID.
dtype – Elements data type.
adims – Logical tensor dimensions. DNNL_GRAPH_UNKNOWN_DIM for dimensions of unknown size, 0 for zero-dimension tensor.
ltype – Layout type. If strided, the strides field in the output logical tensor will be deduced accordingly.
ptype – Property type.

logical_tensor(size_t tid, data_type dtype, const dims &adims, const dims &strides, property_type ptype = property_type::undef)#

Constructs a logical tensor object with detailed dims and strides. The layout_type of the output logical tensor object will always be strided.

Parameters:

tid – Logical tensor ID.
dtype – Elements data type.
adims – Logical tensor dimensions. DNNL_GRAPH_UNKNOWN_DIM for dimensions of unknown size, 0 for zero-dimension tensor.
strides – Logical tensor strides. DNNL_GRAPH_UNKNOWN_DIM for unknown stride. No negative stride is supported.
ptype – Property type.

logical_tensor(size_t tid, data_type dtype, const dims &adims, size_t lid, property_type ptype = property_type::undef)#

Constructs a logical tensor object with detailed dims and an opaque layout ID. layout_type of the output logical tensor object will always be opaque.

Parameters:

tid – Logical tensor ID.
dtype – Elements data type.
adims – Logical tensor dimensions. DNNL_GRAPH_UNKNOWN_DIM for dimensions of unknown size, 0 for zero-dimension tensor.
lid – Opaque layout id.
ptype – Property type

dims get_dims() const#

Returns the dimensions of a logical tensor.

Returns:: A vector describing the size of each dimension.

size_t get_id() const#

Returns the unique id of a logical tensor.

Returns:: An integer value describing the ID.

data_type get_data_type()#

Returns the data type of a logical tensor.

Returns:: The data type.

property_type get_property_type() const#

Returns the property type of a logical tensor.

Returns:: The property type.

layout_type get_layout_type() const#

Returns the layout type of a logical tensor.

Returns:: The layout type.

size_t get_layout_id() const#

Returns the layout ID of a logical tensor. The API should be called on a logical tensor with opaque layout type. Otherwise, an exception will be raised.

Returns:: Layout ID.

dims get_strides() const#

Returns the strides of a logical tensor. The API should be called on a logical tensor with strided layout type. Otherwise, an exception will be raised.

Returns:: A vector describing the stride size of each dimension.

size_t get_mem_size() const#

Returns memory size in bytes required by this logical tensor.

Returns:: The memory size in bytes.

bool is_equal(const logical_tensor &lt)#

Compares if two logical tenors are equal. Users can decide accordingly if layout reordering is needed for two logical tensors. The method will return true for below two circumstances:

the two logical tensors are equal regarding each field in the struct, eg. id, ndims, dims, layout type, property, etc.
If all other fields are equal but the layout types in two logical tensors are different, the method will return true when the underlying memory layout is the same. For example, one logical tensor has strided layout type while the other one has opaque layout type, but underneath, both layouts are NHWC, the method will still return true for this case.

Parameters:: lt – The input logical tensor to be compared.
Returns:: true if the two logical tensors are equal. false otherwise

Operation attributes and kinds#

struct op#

An op object.

Public Types

enum class kind#

Kinds of operations.

Values:

enumerator Abs#

enumerator AbsBackward#

enumerator Add#

enumerator AvgPool#

enumerator AvgPoolBackward#

enumerator BatchNormForwardTraining#

enumerator BatchNormInference#

enumerator BatchNormTrainingBackward#

enumerator BiasAdd#

enumerator BiasAddBackward#

enumerator Clamp#

enumerator ClampBackward#

enumerator Concat#

enumerator Convolution#

enumerator ConvolutionBackwardData#

enumerator ConvolutionBackwardWeights#

enumerator ConvTranspose#

enumerator ConvTransposeBackwardData#

enumerator ConvTransposeBackwardWeights#

enumerator Dequantize#

enumerator Divide#

enumerator DynamicDequantize#

enumerator DynamicQuantize#

enumerator Elu#

enumerator EluBackward#

enumerator End#

enumerator Exp#

enumerator GELU#

enumerator GELUBackward#

enumerator HardSigmoid#

enumerator HardSigmoidBackward#

enumerator HardSwish#

enumerator HardSwishBackward#

enumerator Interpolate#

enumerator InterpolateBackward#

enumerator LayerNorm#

enumerator LayerNormBackward#

enumerator LeakyReLU#

enumerator Log#

enumerator LogSoftmax#

enumerator LogSoftmaxBackward#

enumerator MatMul#

enumerator Maximum#

enumerator MaxPool#

enumerator MaxPoolBackward#

enumerator Minimum#

enumerator Mish#

enumerator MishBackward#

enumerator Multiply#

enumerator PReLU#

enumerator PReLUBackward#

enumerator Quantize#

enumerator Reciprocal#

enumerator ReduceL1#

enumerator ReduceL2#

enumerator ReduceMax#

enumerator ReduceMean#

enumerator ReduceMin#

enumerator ReduceProd#

enumerator ReduceSum#

enumerator ReLU#

enumerator ReLUBackward#

enumerator Reorder#

enumerator Round#

enumerator Sigmoid#

enumerator SigmoidBackward#

enumerator SoftMax#

enumerator SoftMaxBackward#

enumerator SoftPlus#

enumerator SoftPlusBackward#

enumerator Sqrt#

enumerator SqrtBackward#

enumerator Square#

enumerator SquaredDifference#

enumerator StaticReshape#

enumerator StaticTranspose#

enumerator Subtract#

enumerator Tanh#

enumerator TanhBackward#

enumerator TypeCast#

enumerator Wildcard#

enumerator LastSymbol#

enum class attr#

Attributes of operations. Different operations support different attributes. Check the document of each operation for what attributes are supported and what are the potential values for them. Missing required attribute or illegal attribute value may lead to failure when adding the operation to a graph.

Values:

enumerator undef#: Undefined op attribute.

enumerator alpha#: Specifies an alpha attribute to an op.

enumerator beta#: Specifies an beta attribute to an op.

enumerator epsilon#: Specifies an epsilon attribute to an op.

enumerator max#: Specifies a max attribute to an op.

enumerator min#: Specifies a min attribute to an op.

enumerator momentum#: Specifies a momentum attribute to an op.

enumerator scales#: Specifies a scales attribute to an op.

enumerator axis#: Specifies an axis attribute to an op.

enumerator begin_norm_axis#: Specifies a begin_norm_axis attribute to an op.

enumerator groups#: Specifies a groups attribute to an op.

enumerator axes#: Specifies an axes attribute to an op.

enumerator dilations#: Specifies a dilations attribute to an op.

enumerator dst_shape#: Specifies an dst_shape attribute to an op.

enumerator kernel#: Specifies a kernel attribute to an op.

enumerator order#: Specifies an order attribute to an op.

enumerator output_padding#: Specifies an output_padding attribute to an op.

enumerator pads_begin#: Specifies a pads_begin attribute to an op.

enumerator pads_end#: Specifies a pads_end attribute to an op.

enumerator shape#: Specifies a shape attribute to an op.

enumerator sizes#: Specifies a sizes attribute to an op.

enumerator src_shape#: Specifies an src_shape attribute to an op.

enumerator strides#: Specifies a strides attribute to an op.

enumerator weights_shape#: Specifies a weight_shape attribute to an op.

enumerator zps#: Specifies a zps attribute to an op.

enumerator exclude_pad#: Specifies an exclude_pad attribute to an op.

enumerator keep_dims#: Specifies a keep_dims attribute to an op.

enumerator keep_stats#: Specifies a keep_stats attribute to an op.

enumerator per_channel_broadcast#: Specifies a per_channel_broadcast attribute to an op.

enumerator special_zero#: Specifies a special_zero attribute to an op.

enumerator transpose_a#: Specifies a transpose_a attribute to an op.

enumerator transpose_b#: Specifies a transpose_b attribute to an op.

enumerator use_affine#: Specifies an use_affine attribute to an op.

enumerator use_dst#: Specifies an use_dst attribute to an op.

enumerator auto_broadcast#: Specifies an auto_broadcast attribute to an op. The value can be “none” or “numpy”.

enumerator auto_pad#: Specifies an auto_pad attribute to an op. The value can be “none”, “same_upper”, “same_lower”, or “valid”.

enumerator coordinate_transformation_mode#: Specifies an coordinate_transformation_mode attribute to an op. The value can be “half_pixel” or “align_corners”. The attribute is defined for Interpolate operations.

enumerator data_format#: Specifies a data_format of an op. The value can be “NCX” or “NXC”.

enumerator mode#: Specifies a mode attribute of an op. The value can be “nearest”, “linear”, “bilinear”, or “trilinear”. The attribute is defined for Interpolate operations.

enumerator qtype#: Specifies a qtype attribute to an op. The value can be “per_channel” or “per_tensor”. The attribute is defined for quantization operations.

enumerator rounding_type#: Specifies a rounding_type attribute to an op. The value can be “ceil” or “floor”.

enumerator weights_format#: Specifies a weights_format of an op. The value can be “OIX”, “XIO”, “IOX”, or “XOI”. Different operations may support different values.

Public Functions

op(size_t id, kind akind, const std::string &name = "")#

Constructs an op object with an unique ID, an operation kind, and a name string.

Parameters:

id – The unique ID of the op.
akind – The op kind specifies which computation is represented by the op, such as Convolution or ReLU.
name – The string added as the op name.

op(size_t id, kind akind, const std::vector<logical_tensor> &inputs, const std::vector<logical_tensor> &outputs, const std::string &name = "")#

Constructs an op object with an unique ID, an operation kind, and input/output logical tensors.

Parameters:

id – The unique ID of this op.
akind – The op kind specifies which computation is represented by this op, such as Convolution or ReLU.
inputs – Input logical tensor to be bound to this op.
outputs – Output logical tensor to be bound to this op.
name – The string added as the op name.

void add_input(const logical_tensor &t)#

Adds an input logical tensor to the op.

Parameters:: t – Input logical tensor.

void add_inputs(const std::vector<logical_tensor> &ts)#

Adds a vector of input logical tensors to the op.

Parameters:: ts – The list of input logical tensors.

void add_output(const logical_tensor &t)#

Adds an output logical tensor to the op.

Parameters:: t – Output logical tensor.

void add_outputs(const std::vector<logical_tensor> &ts)#

Adds a vector of output logical tensors to the op.

Parameters:: ts – The list of output logical tensors.

template<typename Type> op &set_attr(attr name, const Type &value)#

Sets the attribute according to the name and type.

Template Parameters:

Type – Attribute’s type.

Parameters:

name – Attribute’s name.
value – The attribute’s value.

Returns:

The Op self. Raises an exception if Type is incompatible with name.

Graph objects member functions#

struct graph#

A graph object.

Public Functions

graph(dnnl::engine::kind engine_kind)#

Constructs a graph with an engine kind.

Parameters:: engine_kind – Engine kind.

graph(dnnl::engine::kind engine_kind, dnnl::fpmath_mode mode)#

Creates a new empty graph with an engine kind and a floating-point math mode. All partitions returned from the graph will inherit the engine kind and floating-point math mode.

Parameters:

engine_kind – Engine kind.
mode – Floating-point math mode.

status add_op(const op &op, bool allow_exception = true)#

Adds an op into the graph to construct a computational DAG. The API will return failure if the operator has already been added to the graph or the operation cannot pass the schema check in the library (eg. input and output numbers and data types, the attributes of the operation, etc.).

Parameters:

op – An operation to be added.
allow_exception – A flag indicating whether the method is allowed to throw an exception if it fails to add the op to the graph.

Returns:

status::success or a status describing the error otherwise.

void finalize()#: Finalizes a graph. It means users have finished adding operations into the graph and the graph is ready for partitioning. Adding a new operation into a finalized graph will return failures. Similarly, partitioning on a un-finalized graph will also return failures.

bool is_finalized() const#

Checks if a graph is finalized.

Returns:: True if the graph is finalized or false if the graph is not finalized.

std::vector<partition> get_partitions(partition::policy policy = partition::policy::fusion)#

Gets filtered partitions from a graph. Partitions will be claimed internally according to the capability of the library, the engine kind of the graph, and the policy.

Parameters:: policy – Partition policy, defaults to policy dnnl::graph::partition::policy::fusion.
Returns:: A vector storing the partitions.

Macros to specify unknown shapes#

DNNL_GRAPH_UNKNOWN_NDIMS#: A wildcard value for number of dimensions which is unknown at a tensor or operation creation time.

DNNL_GRAPH_UNKNOWN_DIM#: A wildcard value for dimensions that are unknown at a tensor or operation creation time.

Common Definitions

Contents

Common Definitions#

Logical tensor enums and type#

Operation attributes and kinds#

Graph objects member functions#

Macros to specify unknown shapes#