oneMath Architecture

The oneMath element of oneAPI has several general assumptions, requirements and recommendations for all domains contained therein. These will be addressed in this architecture section. In particular, DPC++ allows for a great control over the execution of kernels on the various devices. We discuss the supported execution models of oneMath APIs in Execution Model. A discussion of how data is stored and passed in and out of the APIs is addressed in Memory Model. The general structure and design of oneMath APIs including namespaces and common data types are expressed in API Design. The exceptions and error handling are described in Exceptions and Error Handling. Finally all the other necessary aspects related to oneMath architecture can be found in Other Features including versioning and discussion of pre and post conditions. Other nonessential, but useful aspects of the oneMath architecture and design may also be found in the oneMath Appendix.

Execution Model

This section describes the execution environment common to all oneMath functionality. The execution environment includes how data is provided to computational routines in Use of Queues, support for several devices in Device Usage, synchronous and asynchronous execution models in Asynchronous Execution and Host Thread Safety.

Use of Queues

The sycl::queue defined in the oneAPI DPC++ specification is used to specify the device and features enabled on that device on which a task will be enqueued. There are two forms of computational routines in oneMath: class based Member Functions and standalone Non-Member Functions. As these may interact with the sycl::queue in different ways, we provide a section for each one to describe assumptions.

Non-Member Functions

Each oneMath non-member computational routine takes a sycl::queue reference as its first parameter:

math::domain::routine(sycl::queue &q, ...);

All computation performed by the routine shall be done on the hardware device(s) associated with this queue, with possible aid from the host, unless otherwise specified. In the case of an ordered queue, all computation shall also be ordered with respect to other kernels as if enqueued on that queue.

A particular oneMath implementation may not support the execution of a given oneMath routine on the specified device(s). In this case, the implementation may either perform the computation on the host or throw an exception. See Exceptions and Error Handling for the possible exceptions.

Member Functions

oneMath class-based APIs, such as those in the RNG and DFT domains, require a sycl::queue as an argument to the constructor or another setup routine. The execution requirements for computational routines from the previous section also apply to computational class methods.

Device Usage

oneMath itself does not currently provide any interfaces for controlling device usage: for instance, controlling the number of cores used on the CPU, or the number of execution units on a GPU. However, such functionality may be available by partitioning a sycl::device instance into subdevices, when supported by the device.

When given a queue associated with such a subdevice, a oneMath implementation shall only perform computation on that subdevice.

Asynchronous Execution

The oneMath API is designed to allow asynchronous execution of computational routines, to facilitate concurrent usage of multiple devices in the system. Each computational routine enqueues work to be performed on the selected device, and may (but is not required to) return before execution completes.

Hence, it is the calling application’s responsibility to ensure that any inputs are valid until computation is complete, and likewise to wait for computation completion before reading any outputs. This can be done automatically when using DPC++ buffers, or manually when using Unified Shared Memory (USM) pointers, as described in the sections below.

Unless otherwise specified, asynchronous execution is allowed, but not guaranteed, by any oneMath computational routine, and may vary between implementations and/or versions. oneMath implementations must clearly document whether execution is guaranteed to be asynchronous for each supported routine. Regardless, calling applications shall not launch any oneMath computational routine with a dependency on a future oneMath API call, even if this computational routine executes asynchronously (i.e. a oneMath implementation may assume no antidependencies are present). This guarantee allows oneMath implementations to reserve resources for execution without risking deadlock.

Synchronization When Using Buffers

sycl::buffer objects automatically manage synchronization between kernel launches linked by a data dependency (either read-after-write, write-after-write, or write-after-read).

oneMath routines are not required to perform any additional synchronization of sycl::buffer arguments.

Synchronization When Using USM APIs

When USM pointers are used as input to, or output from, a oneMath routine, it becomes the calling application’s responsibility to manage possible asynchronicity.

To help the calling application, all oneMath routines with at least one USM pointer argument also take an optional reference to a list of input events, of type std::vector<sycl::event>, and have a return value of type sycl::event representing computation completion:

sycl::event math::domain::routine(..., const std::vector<sycl::event> &in_events = {});

The routine shall ensure that all input events (if the list is present and non-empty) have occurred before any USM pointers are accessed. Likewise, the routine’s output event shall not be complete until the routine has finished accessing all USM pointer arguments.

For class methods, “argument” includes any USM pointers previously provided to the object via the class constructor or other class methods.

Host Thread Safety

All oneMath member and non-member functions shall be host thread safe. That is, they may be safely called simultaneously from concurrent host threads. However, oneMath objects in class-based APIs may not be shared between concurrent host threads unless otherwise specified.

Memory Model

The oneMath memory model shall follow directly from the oneAPI memory model. Mainly, oneMath shall support two modes of encapsulating data for consumption on the device: the buffer memory abstraction model and the pointer-based memory model using Unified Shared Memory (USM). These two paradigms shall also support both synchronous and asynchronous execution models as described in Asynchronous Execution.

The Buffer Memory Model

The SYCL 1.2.1 specification defines the buffer container templated on the provided data type which encapsulates the data in a SYCL application across both host and devices. It provides the concept of accessors as the mechanism to access the buffer data with different modes to read and or write into that data. These accessors allow SYCL to create and manage the data dependencies in the SYCL graph that order the kernel executions. With the buffer model, all data movement is handled by the SYCL runtime supporting both synchronous and asynchronous execution.

oneMath provides APIs where buffers (in particular 1D buffers, sycl::buffer<T,1>) contain the memory for all non scalar input and output data arguments. See Synchronization When Using Buffers for details on how oneMath routines manage any data dependencies with buffer arguments. Any higher dimensional buffer must be converted to a 1D buffer prior to use in oneMath APIs, e.g., via buffer::reinterpret.

Unified Shared Memory Model

While the buffer model is powerful and elegantly expresses data dependencies, it can be a burden for programmers to replace all pointers and arrays by buffers in their C++ applications. DPC++ also provides pointer-based addressing for device-accessible data, using the Unified Shared Memory (USM) model. Correspondingly, oneMath provides USM APIs in which non-scalar input and output data arguments are passed by USM pointer.

USM devices and system configurations vary in their ability to share data between devices and between a device and the host. oneMath implementations may only assume that user-provided USM pointers are accessible by the device associated with the user-provided queue. In particular, an implementation must not assume that USM pointers can be accessed by any other device, or by the host, without querying the DPC++ runtime. An implementation must accept any device-accessible USM pointer regardless of how it was created (sycl::malloc_device, sycl::malloc_shared, etc.).

Unlike buffers, USM pointers cannot automatically manage data dependencies between kernels. Users may use in-order queues to ensure ordered execution, or explicitly manage dependencies with sycl::event objects. To support the second use case, oneMath USM APIs accept input events (prerequisites before computation can begin) and return an output event (indicating computation is complete). See Synchronization When Using USM APIs for details.

API Design

This section discusses the general features of oneMath API design. In particular, it covers the use of namespaces and data types from C++, from DPC++ and new ones introduced for oneMath APIs.

oneMath namespaces

The oneMath library uses C++ namespaces to organize routines by mathematical domain. All oneMath objects and routines shall be contained within the oneapi::math base namespace. The individual oneMath domains use a secondary namespace layer as follows:

namespace	oneMath domain or content
oneapi::math	oneMath base namespace, contains general oneMath data types, objects, exceptions and routines
oneapi::math::blas	Dense linear algebra routines from BLAS and BLAS like extensions. The oneapi::math::blas namespace should contain two namespaces column_major and row_major to support both matrix layouts. See BLAS Routines
oneapi::math::lapack	Dense linear algebra routines from LAPACK and LAPACK like extensions. See LAPACK Routines
oneapi::math::sparse	Sparse linear algebra routines from Sparse BLAS and Sparse Solvers. See Sparse Linear Algebra
oneapi::math::dft	Discrete Fourier Transforms. See Discrete Fourier Transform Functions
oneapi::math::rng	Random number generator routines. See Random Number Generators
oneapi::math::vm	Vector mathematics routines, e.g. trigonometric, exponential functions acting on elements of a vector. See Vector Math
oneapi::math::stats	Routines that compute basic statistical estimates for single and double precision multi-dimensional datasets. See Summary Statistics

Note

Inside each oneMath domain, there are many routines, classes, enums and objects defined which constitute the breadth and scope of that oneMath domain. It is permitted for a library implementation of the oneMath specification to implement either all, one or more than one of the domains in oneMath. However, within an implementation of a specific domain, all relevant routines, classes, enums and objects (including those relevant enums and objects which live outside a particular domain in the general oneapi::math namespace must be both declared and defined in the library so that an application that uses that domain could build and link against that library implementation successfully.

It is however acceptable to throw the runtime exception oneapi::math::unimplemented inside of the routines or class member functions in that domain that have not been fully implemented. For instance, a library may choose to implement the oneMath BLAS functionality and in particular may choose to implement only the gemm api for their library, in which case they must also include all the other blas namespaced routines and throw the oneapi::math::unimplemented exception inside all the others.

In such a case, the implemented routines in such a library should be communicated clearly and easily understood by users of that library.

Standard C++ datatype usage

oneMath uses C++ STL data types for scalars where applicable:

• Integer scalars are C++ fixed-size integer types (std::intN_t, std::uintN_t).

• Complex numbers are represented by C++ std::complex types.

In general, scalar integer arguments to oneMath routines are 64-bit integers (std::int64_t or std::uint64_t). Integer vectors and matrices may have varying bit widths, defined on a per-routine basis.

DPC++ datatype usage

oneMath uses the following DPC++ data types:

• SYCL queue sycl::queue for scheduling kernels on a SYCL device. See Use of Queues for more details.

• SYCL buffer sycl::buffer for buffer-based memory access. See The Buffer Memory Model for more details.

• Unified Shared Memory (USM) for pointer-based memory access. See Unified Shared Memory Model for more details.

• SYCL event sycl::event for output event synchronization in oneMath routines with USM pointers. See Synchronization When Using USM APIs for more details.

• Vector of SYCL events std::vector<sycl::event> for input events synchronization in oneMath routines with USM pointers. See Synchronization When Using USM APIs for more details.

Note

The class sycl::vector_class has been removed from SYCL 2020 and the standard class std::vector should be used instead for vector of SYCL events in oneMath routines with USM pointers

oneMath defined datatypes

oneMath dense and sparse linear algebra routines use scoped enum types as type-safe replacements for the traditional character arguments used in C/Fortran implementations of BLAS and LAPACK. These types all belong to the oneapi::math namespace.

Each enumeration value comes with two names: A single-character name (the traditional BLAS/LAPACK character) and a longer, more descriptive name. The two names are exactly equivalent and may be used interchangeably.

transpose

The transpose type specifies whether an input matrix should be transposed and/or conjugated. It can take the following values:

Short Name	Long Name	Description
transpose::N	transpose::nontrans	Do not transpose or conjugate the matrix.
transpose::T	transpose::trans	Transpose the matrix (without complex conjugation).
transpose::C	transpose::conjtrans	Perform Hermitian transpose (transpose and conjugate). Is the same as transpose::trans for real matrices.

uplo

The uplo type specifies whether the lower or upper triangle of a triangular, symmetric, or Hermitian matrix should be accessed. It can take the following values:

Short Name	Long Name	Description
uplo::U	uplo::upper	Access the upper triangle of the matrix.
uplo::L	uplo::lower	Access the lower triangle of the matrix.

In both cases, elements that are not in the selected triangle are not accessed or updated.

diag

The diag type specifies the values on the diagonal of a triangular matrix. It can take the following values:

Short Name	Long Name	Description
diag::N	diag::nonunit	The matrix is not unit triangular. The diagonal entries are stored with the matrix data.
diag::U	diag::unit	The matrix is unit triangular (the diagonal entries are all 1’s). The diagonal entries in the matrix data are not accessed.

side

The side type specifies the order of matrix multiplication when one matrix has a special form (triangular, symmetric, or Hermitian):

Short Name	Long Name	Description
side::L	side::left	The special form matrix is on the left in the multiplication.
side::R	side::right	The special form matrix is on the right in the multiplication.

offset

The offset type specifies whether the offset to apply to an output matrix is a fix offset, column offset or row offset. It can take the following values

Short Name	Long Name	Description
offset::F	offset::fix	The offset to apply to the output matrix is fix, all the inputs in the C_offset matrix has the same value given by the first element in the co array.
offset::C	offset::column	The offset to apply to the output matrix is a column offset, that is to say all the columns in the C_offset matrix are the same and given by the elements in the co array.
offset::R	offset::row	The offset to apply to the output matrix is a row offset, that is to say all the rows in the C_offset matrix are the same and given by the elements in the co array.

index_base

The index_base type specifies how values in index arrays are interpreted. For instance, a sparse matrix stores nonzero values and the indices that they correspond to. The indices are traditionally provided in one of two forms: C/C++-style using zero-based indices, or Fortran-style using one-based indices. The index_base type can take the following values:

Name	Description
index_base::zero	Index arrays for an input matrix are provided using zero-based (C/C++ style) index values. That is, indices start at 0.
index_base::one	Index arrays for an input matrix are provided using one-based (Fortran style) index values. That is, indices start at 1.

layout

The layout type specifies how a dense matrix A with leading dimension lda is stored as one dimensional array in memory. The layouts are traditionally provided in one of two forms: C/C++-style using row_major layout, or Fortran-style using column_major layout. The layout type can take the following values:

Short Name	Long Name	Description
layout::R	layout::row_major	For row major layout, the elements of each row of a dense matrix A are contiguous in memory while the elements of each column are at distance lda from the element in the same column and the previous row.
layout::C	layout::col_major	For column major layout, the elements of each column a dense matrix A are contiguous in memory while the elements of each row are at distance lda from the element in the same row and the previous column.

Note

oneMath Appendix may contain other API design decisions or recommendations that may be of use to the general developer of oneMath, but which may not necessarily be part of the oneMath specification.

Exceptions and Error Handling

oneMath error handling relies on the mechanism of C++ exceptions. Should error occur, it will be propagated at the point of a function call where it is caught using standard C++ error handling mechanism.

Exception classification

Exception classification in oneMath is aligned with C++ Standard Library classification. oneMath introduces class that defines the base class in the hierarchy of oneMath exception classes. All oneMath routines throw exceptions inherited from this base class. In the hierarchy of oneMath exceptions, oneapi::math::exception is the base class inherited from std::exception class. All other oneMath exception classes are derived from this base class.

This specification does not require implementations to perform error-checking. However, if an implementation does provide error-checking, it shall use the following exception classes. Additional implementation-specific exception classes can be used for exceptional conditions not fitting any of these classes.

Common exceptions

Exception class	Description
oneapi::math::exception	Reports general unspecified problem
oneapi::math::unsupported_device	Reports a problem when the routine is not supported on a specific device
oneapi::math::host_bad_alloc	Reports a problem that occurred during memory allocation on the host
oneapi::math::device_bad_alloc	Reports a problem that occurred during memory allocation on a specific device
oneapi::math::unimplemented	Reports a problem when a specific routine has not been implemented for the specified parameters
oneapi::math::invalid_argument	Reports problem when arguments to the routine were rejected
oneapi::math::uninitialized	Reports problem when a handle (descriptor) has not been initialized
oneapi::math::computation_error	Reports any computation errors that have occurred inside a oneMath routine
oneapi::math::batch_error	Reports errors that have occurred inside a batch oneMath routine

LAPACK specific exceptions

Exception class	Description
oneapi::math::lapack::exception	Base class for all LAPACK exceptions providing access to info code familiar to users of conventional LAPACK API. All LAPACK related exceptions can be handled with catch block for this class.
oneapi::math::lapack::invalid_argument	Reports errors when arguments provided to the LAPACK subroutine are inconsistent or do not match expected values. Class extends base oneapi::math::invalid_argument with ability to access conventional status info code.
oneapi::math::lapack::computation_error	Reports computation errors that have occurred during call to LAPACK subroutine. Class extends base oneapi::math::computation_error with ability to access conventional status info code familiar to LAPACK users.
oneapi::math::lapack::batch_error	Reports errors that have occurred during batch LAPACK computations. Class extends base oneapi::math::batch_error with ability to access individual exception objects for each of the issues observed in a batch and an info code. The info code contains the number of errors that occurred in a batch. Positions of problems in a supplied batch that experienced issues during computations can be retrieved with ids() method, and list of particular exceptions can be obtained with exceptions() method of the exception object. Possible exceptions for a batch are documented for corresponding non-batch API.

Other Features

This section covers all other features in the design of oneMath architecture.

Specification Version and Compliance

Each oneMath domain must define a preprocessor macro to represent the version of the specification that the implementation is compliant with.

The macros for each domain are listed as follows:

ONEMATH_BLAS_SPEC_VERSION

ONEMATH_LAPACK_SPEC_VERSION

ONEMATH_SPBLAS_SPEC_VERSION

ONEMATH_DFT_SPEC_VERSION

ONEMATH_RNG_SPEC_VERSION

ONEMATH_STATS_SPEC_VERSION

ONEMATH_VM_SPEC_VERSION

The specification version can be created by appending all digits of the specification version in the format of <MAJOR><MINOR>. MINOR version always uses two digits. This version can be used to check the compatibility of the implementation with the specification version. Note that the revision is not included here because it reflects changes only for the specification document without affecting the implementation. If the implementation is not compliant with any release of the specification, then the macro must have a numerical value of 001. Versions between 001 and 100 are reserved for alpha/beta support of the specification in oneMKL and should only be used for special cases. A preprocessor may evaluate an undefined macro to 0, so 000 should not be used.

Version Example

oneAPI 1.1 rev 1 will be represented as a numerical value of 101

oneAPI 1.2 rev 1 will be represented as a numerical value of 102

oneAPI 1.2 rev 2 will be represented as a numerical value of 102

Macro Example

// For oneAPI 1.2 rev 1
#define ONEMATH_BLAS_SPEC_VERSION 102

// For oneAPI 1.2 rev 2
#define ONEMATH_DFT_SPEC_VERSION 102

// For oneAPI 1.3 rev 1
#define ONEMATH_VM_SPEC_VERSION 103

Versioning details are defined here: uxlfoundation/oneAPI-spec

Pre/Post Condition Checking

The individual oneMath computational routines will define any preconditions and postconditions and will define in this specification any specific checks or verifications that should be enabled for all implementations.