oneMKL Architecture

The oneMKL element of oneAPI has several general assumptions, requirements and recommendations for all domains contained therein. These will be addressed in this architecture section. In particular, DPC++ allows for a great control over the execution of kernels on the various devices. We discuss the supported execution models of oneMKL APIs in Execution Model. A discussion of how data is stored and passed in and out of the APIs is addressed in Memory Model. The general structure and design of oneMKL APIs including namespaces and common data types are expressed in API Design. The exceptions and error handling are described in Exceptions and Error Handling. Finally all the other necessary aspects related to oneMKL architecture can be found in Other Features including versioning and discussion of pre and post conditions. Other nonessential, but useful aspects of the oneMKL architecture and design may also be found in the oneMKL Appendix.

Execution Model

This section describes the execution environment common to all oneMKL functionality. The execution environment includes how data is provided to computational routines in Use of Queues, support for several devices in Device Usage, synchronous and asynchronous execution models in Asynchronous Execution and Host Thread Safety.

Use of Queues

The sycl::queue defined in the oneAPI DPC++ specification is used to specify the device and features enabled on that device on which a task will be enqueued. There are two forms of computational routines in oneMKL: class based Member Functions and standalone Non-Member Functions. As these may interact with the sycl::queue in different ways, we provide a section for each one to describe assumptions.

Non-Member Functions

Each oneMKL non-member computational routine takes a sycl::queue reference as its first parameter:

mkl::domain::routine(sycl::queue &q, ...);

All computation performed by the routine shall be done on the hardware device(s) associated with this queue, with possible aid from the host, unless otherwise specified. In the case of an ordered queue, all computation shall also be ordered with respect to other kernels as if enqueued on that queue.

A particular oneMKL implementation may not support the execution of a given oneMKL routine on the specified device(s). In this case, the implementation may either perform the computation on the host or throw an exception. See Exceptions and Error Handling for the possible exceptions.

Member Functions

oneMKL class-based APIs, such as those in the RNG and DFT domains, require a sycl::queue as an argument to the constructor or another setup routine. The execution requirements for computational routines from the previous section also apply to computational class methods.

Device Usage

oneMKL itself does not currently provide any interfaces for controlling device usage: for instance, controlling the number of cores used on the CPU, or the number of execution units on a GPU. However, such functionality may be available by partitioning a sycl::device instance into subdevices, when supported by the device.

When given a queue associated with such a subdevice, a oneMKL implementation shall only perform computation on that subdevice.

Asynchronous Execution

The oneMKL API is designed to allow asynchronous execution of computational routines, to facilitate concurrent usage of multiple devices in the system. Each computational routine enqueues work to be performed on the selected device, and may (but is not required to) return before execution completes.

Hence, it is the calling application’s responsibility to ensure that any inputs are valid until computation is complete, and likewise to wait for computation completion before reading any outputs. This can be done automatically when using DPC++ buffers, or manually when using Unified Shared Memory (USM) pointers, as described in the sections below.

Unless otherwise specified, asynchronous execution is allowed, but not guaranteed, by any oneMKL computational routine, and may vary between implementations and/or versions. oneMKL implementations must clearly document whether execution is guaranteed to be asynchronous for each supported routine. Regardless, calling applications shall not launch any oneMKL computational routine with a dependency on a future oneMKL API call, even if this computational routine executes asynchronously (i.e. a oneMKL implementation may assume no antidependencies are present). This guarantee allows oneMKL implementations to reserve resources for execution without risking deadlock.

Synchronization When Using Buffers

sycl::buffer objects automatically manage synchronization between kernel launches linked by a data dependency (either read-after-write, write-after-write, or write-after-read).

oneMKL routines are not required to perform any additional synchronization of sycl::buffer arguments.

Synchronization When Using USM APIs

When USM pointers are used as input to, or output from, a oneMKL routine, it becomes the calling application’s responsibility to manage possible asynchronicity.

To help the calling application, all oneMKL routines with at least one USM pointer argument also take an optional reference to a list of input events, of type std::vector<sycl::event>, and have a return value of type sycl::event representing computation completion:

sycl::event mkl::domain::routine(..., std::vector<sycl::event> &in_events = {});

The routine shall ensure that all input events (if the list is present and non-empty) have occurred before any USM pointers are accessed. Likewise, the routine’s output event shall not be complete until the routine has finished accessing all USM pointer arguments.

For class methods, “argument” includes any USM pointers previously provided to the object via the class constructor or other class methods.

Host Thread Safety

All oneMKL member and non-member functions shall be host thread safe. That is, they may be safely called simultaneously from concurrent host threads. However, oneMKL objects in class-based APIs may not be shared between concurrent host threads unless otherwise specified.

Memory Model

The oneMKL memory model shall follow directly from the oneAPI memory model. Mainly, oneMKL shall support two modes of encapsulating data for consumption on the device: the buffer memory abstraction model and the pointer-based memory model using Unified Shared Memory (USM). These two paradigms shall also support both synchronous and asynchronous execution models as described in Asynchronous Execution.

The Buffer Memory Model

The SYCL 1.2.1 specification defines the buffer container templated on the provided data type which encapsulates the data in a SYCL application across both host and devices. It provides the concept of accessors as the mechanism to access the buffer data with different modes to read and or write into that data. These accessors allow SYCL to create and manage the data dependencies in the SYCL graph that order the kernel executions. With the buffer model, all data movement is handled by the SYCL runtime supporting both synchronous and asynchronous execution.

oneMKL provides APIs where buffers (in particular 1D buffers, sycl::buffer<T,1>) contain the memory for all non scalar input and output data arguments. See Synchronization When Using Buffers for details on how oneMKL routines manage any data dependencies with buffer arguments. Any higher dimensional buffer must be converted to a 1D buffer prior to use in oneMKL APIs, e.g., via buffer::reinterpret.

Unified Shared Memory Model

While the buffer model is powerful and elegantly expresses data dependencies, it can be a burden for programmers to replace all pointers and arrays by buffers in their C++ applications. DPC++ also provides pointer-based addressing for device-accessible data, using the Unified Shared Memory (USM) model. Correspondingly, oneMKL provides USM APIs in which non-scalar input and output data arguments are passed by USM pointer.

USM devices and system configurations vary in their ability to share data between devices and between a device and the host. oneMKL implementations may only assume that user-provided USM pointers are accessible by the device associated with the user-provided queue. In particular, an implementation must not assume that USM pointers can be accessed by any other device, or by the host, without querying the DPC++ runtime. An implementation must accept any device-accessible USM pointer regardless of how it was created (sycl::malloc_device, sycl::malloc_shared, etc.).

Unlike buffers, USM pointers cannot automatically manage data dependencies between kernels. Users may use in-order queues to ensure ordered execution, or explicitly manage dependencies with sycl::event objects. To support the second use case, oneMKL USM APIs accept input events (prerequisites before computation can begin) and return an output event (indicating computation is complete). See Synchronization When Using USM APIs for details.

API Design

This section discusses the general features of oneMKL API design. In particular, it covers the use of namespaces and data types from C++, from DPC++ and new ones introduced for oneMKL APIs.

oneMKL namespaces

The oneMKL library uses C++ namespaces to organize routines by mathematical domain. All oneMKL objects and routines shall be contained within the oneapi::mkl base namespace. The individual oneMKL domains use a secondary namespace layer as follows:

namespace	oneMKL domain or content
oneapi::mkl	oneMKL base namespace, contains general oneMKL data types, objects, exceptions and routines
oneapi::mkl::blas	Dense linear algebra routines from BLAS and BLAS like extensions. The oneapi::mkl::blas namespace should contain two namespaces column_major and row_major to support both matrix layouts. See BLAS Routines
oneapi::mkl::lapack	Dense linear algebra routines from LAPACK and LAPACK like extensions. See LAPACK Routines
oneapi::mkl::sparse	Sparse linear algebra routines from Sparse BLAS and Sparse Solvers. See Sparse Linear Algebra
oneapi::mkl::dft	Discrete Fourier Transforms. See Discrete Fourier Transform Functions
oneapi::mkl::rng	Random number generator routines. See Random Number Generators
oneapi::mkl::vm	Vector mathematics routines, e.g. trigonometric, exponential functions acting on elements of a vector. See Vector Math

Note

Inside each oneMKL domain, there are many routines, classes, enums and objects defined which constitute the breadth and scope of that oneMKL domain. It is permitted for a library implementation of the oneMKL specification to implement either all, one or more than one of the domains in oneMKL. However, within an implementation of a specific domain, all relevant routines, classes, enums and objects (including those relevant enums and objects which live outside a particular domain in the general oneapi::mkl namespace must be both declared and defined in the library so that an application that uses that domain could build and link against that library implementation successfully.

It is however acceptable to throw the runtime exception oneapi::mkl::unimplemented inside of the routines or class member functions in that domain that have not been fully implemented. For instance, a library may choose to implement the oneMKL BLAS functionality and in particular may choose to implement only the gemm api for their library, in which case they must also include all the other blas namespaced routines and throw the oneapi::mkl::unimplemented exception inside all the others.

In such a case, the implemented routines in such a library should be communicated clearly and easily understood by users of that library.

Standard C++ datatype usage

oneMKL uses C++ STL data types for scalars where applicable:

• Integer scalars are C++ fixed-size integer types (std::intN_t, std::uintN_t).

• Complex numbers are represented by C++ std::complex types.

In general, scalar integer arguments to oneMKL routines are 64-bit integers (std::int64_t or std::uint64_t). Integer vectors and matrices may have varying bit widths, defined on a per-routine basis.

DPC++ datatype usage

oneMKL uses the following DPC++ data types:

• SYCL queue sycl::queue for scheduling kernels on a SYCL device. See Use of Queues for more details.

• SYCL buffer sycl::buffer for buffer-based memory access. See The Buffer Memory Model for more details.

• Unified Shared Memory (USM) for pointer-based memory access. See Unified Shared Memory Model for more details.

• SYCL event sycl::event for output event synchronization in oneMKL routines with USM pointers. See Synchronization When Using USM APIs for more details.

• Vector of SYCL events std::vector<sycl::event> for input events synchronization in oneMKL routines with USM pointers. See Synchronization When Using USM APIs for more details.

Note

The class sycl::vector_class has been removed from SYCL 2020 and the standard class std::vector should be used instead for vector of SYCL events in oneMKL routines with USM pointers

oneMKL defined datatypes

oneMKL dense and sparse linear algebra routines use scoped enum types as type-safe replacements for the traditional character arguments used in C/Fortran implementations of BLAS and LAPACK. These types all belong to the oneapi::mkl namespace.

Each enumeration value comes with two names: A single-character name (the traditional BLAS/LAPACK character) and a longer, more descriptive name. The two names are exactly equivalent and may be used interchangeably.

transpose

The transpose type specifies whether an input matrix should be transposed and/or conjugated. It can take the following values:

Short Name	Long Name	Description
transpose::N	transpose::nontrans	Do not transpose or conjugate the matrix.
transpose::T	transpose::trans	Transpose the matrix (without complex conjugation).
transpose::C	transpose::conjtrans	Perform Hermitian transpose (transpose and conjugate). Is the same as transpose::trans for real matrices.

uplo

The uplo type specifies whether the lower or upper triangle of a triangular, symmetric, or Hermitian matrix should be accessed. It can take the following values:

Short Name	Long Name	Description
uplo::U	uplo::upper	Access the upper triangle of the matrix.
uplo::L	uplo::lower	Access the lower triangle of the matrix.

In both cases, elements that are not in the selected triangle are not accessed or updated.

diag

The diag type specifies the values on the diagonal of a triangular matrix. It can take the following values:

Short Name	Long Name	Description
diag::N	diag::nonunit	The matrix is not unit triangular. The diagonal entries are stored with the matrix data.
diag::U	diag::unit	The matrix is unit triangular (the diagonal entries are all 1’s). The diagonal entries in the matrix data are not accessed.

side

The side type specifies the order of matrix multiplication when one matrix has a special form (triangular, symmetric, or Hermitian):

Short Name	Long Name	Description
side::L	side::left	The special form matrix is on the left in the multiplication.
side::R	side::right	The special form matrix is on the right in the multiplication.

offset

The offset type specifies whether the offset to apply to an output matrix is a fix offset, column offset or row offset. It can take the following values

Short Name	Long Name	Description
offset::F	offset::fix	The offset to apply to the output matrix is fix, all the inputs in the C_offset matrix has the same value given by the first element in the co array.
offset::C	offset::column	The offset to apply to the output matrix is a column offset, that is to say all the columns in the C_offset matrix are the same and given by the elements in the co array.
offset::R	offset::row	The offset to apply to the output matrix is a row offset, that is to say all the rows in the C_offset matrix are the same and given by the elements in the co array.

index_base

The index_base type specifies how values in index arrays are interpreted. For instance, a sparse matrix stores nonzero values and the indices that they correspond to. The indices are traditionally provided in one of two forms: C/C++-style using zero-based indices, or Fortran-style using one-based indices. The index_base type can take the following values:

Name	Description
index_base::zero	Index arrays for an input matrix are provided using zero-based (C/C++ style) index values. That is, indices start at 0.
index_base::one	Index arrays for an input matrix are provided using one-based (Fortran style) index values. That is, indices start at 1.

layout

The layout type specifies how a dense matrix A with leading dimension lda is stored as one dimensional array in memory. The layouts are traditionally provided in one of two forms: C/C++-style using row_major layout, or Fortran-style using column_major layout. The layout type can take the following values:

Short Name	Long Name	Description
layout::R	layout::row_major	For row major layout, the elements of each row of a dense matrix A are contiguous in memory while the elements of each column are at distance lda from the element in the same column and the previous row.
layout::C	layout::col_major	For column major layout, the elements of each column a dense matrix A are contiguous in memory while the elements of each row are at distance lda from the element in the same row and the previous column.

Note

oneMKL Appendix may contain other API design decisions or recommendations that may be of use to the general developer of oneMKL, but which may not necessarily be part of the oneMKL specification.

Exceptions and Error Handling

oneMKL error handling relies on the mechanism of C++ exceptions. Should error occur, it will be propagated at the point of a function call where it is caught using standard C++ error handling mechanism.

Exception classification

Exception classification in oneMKL is aligned with C++ Standard Library classification. oneMKL introduces class that defines the base class in the hierarchy of oneMKL exception classes. All oneMKL routines throw exceptions inherited from this base class. In the hierarchy of oneMKL exceptions, oneapi::mkl::exception is the base class inherited from std::exception class. All other oneMKL exception classes are derived from this base class.

This specification does not require implementations to perform error-checking. However, if an implementation does provide error-checking, it shall use the following exception classes. Additional implementation-specific exception classes can be used for exceptional conditions not fitting any of these classes.

Common exceptions

Exception class	Description
oneapi::mkl::exception	Reports general unspecified problem
oneapi::mkl::unsupported_device	Reports a problem when the routine is not supported on a specific device
oneapi::mkl::host_bad_alloc	Reports a problem that occurred during memory allocation on the host
oneapi::mkl::device_bad_alloc	Reports a problem that occurred during memory allocation on a specific device
oneapi::mkl::unimplemented	Reports a problem when a specific routine has not been implemented for the specified parameters
oneapi::mkl::invalid_argument	Reports problem when arguments to the routine were rejected
oneapi::mkl::uninitialized	Reports problem when a handle (descriptor) has not been initialized
oneapi::mkl::computation_error	Reports any computation errors that have occurred inside a oneMKL routine
oneapi::mkl::batch_error	Reports errors that have occurred inside a batch oneMKL routine

LAPACK specific exceptions

Exception class	Description
oneapi::mkl::lapack::exception	Base class for all LAPACK exceptions providing access to info code familiar to users of conventional LAPACK API. All LAPACK related exceptions can be handled with catch block for this class.
oneapi::mkl::lapack::invalid_argument	Reports errors when arguments provided to the LAPACK subroutine are inconsistent or do not match expected values. Class extends base oneapi::mkl::invalid_argument with ability to access conventional status info code.
oneapi::mkl::lapack::computation_error	Reports computation errors that have occurred during call to LAPACK subroutine. Class extends base oneapi::mkl::computation_error with ability to access conventional status info code familiar to LAPACK users.
oneapi::mkl::lapack::batch_error	Reports errors that have occurred during batch LAPACK computations. Class extends base oneapi::mkl::batch_error with ability to access individual exception objects for each of the issues observed in a batch and an info code. The info code contains the number of errors that occurred in a batch. Positions of problems in a supplied batch that experienced issues during computations can be retrieved with ids() method, and list of particular exceptions can be obtained with exceptions() method of the exception object. Possible exceptions for a batch are documented for corresponding non-batch API.

Other Features

This section covers all other features in the design of oneMKL architecture.

Current Version of this oneMKL Specification

This is the oneMKL specification which is part of the oneAPI specification version 1.0.0.

Pre/Post Condition Checking

The individual oneMKL computational routines will define any preconditions and postconditions and will define in this specification any specific checks or verifications that should be enabled for all implementations.