Memory Allocation and External Allocators#

There are two models of memory management in oneVPL: internal and external.

External Memory Management#

In the external memory model, the application must allocate sufficient memory for input and output parameters and buffers and deallocate it when oneVPL functions complete their operations. During execution, the oneVPL functions use callback functions to the application to manage memory for video frames through the external allocator interface mfxFrameAllocator.

If an application needs to control the allocation of video frames, it can use callback functions through the mfxFrameAllocator interface. If an application does not specify an allocator, an internal allocator is used. However, if an application uses video memory surfaces for input and output, it must specify the hardware acceleration device and an external frame allocator using mfxFrameAllocator.

The external frame allocator can allocate different frame types:

The external frame allocator responds only to frame allocation requests for the requested memory type and returns mfxStatus::MFX_ERR_UNSUPPORTED for all other types. The allocation request uses flags (part of the memory type field) to indicate which oneVPL class initiated the request so that the external frame allocator can respond accordingly.

The following example shows a simple external frame allocator:

 1#define ALIGN32(X) (((mfxU32)((X)+31)) & (~ (mfxU32)31))
 2
 3typedef struct {
 4   mfxU16 width, height;
 5   mfxU8 *base;
 6} mid_struct;
 7
 8mfxStatus fa_alloc(mfxHDL pthis, mfxFrameAllocRequest *request, mfxFrameAllocResponse *response) {
 9   UNUSED_PARAM(pthis);
10   if (!(request->Type&MFX_MEMTYPE_SYSTEM_MEMORY))
11      return MFX_ERR_UNSUPPORTED;
12   if (request->Info.FourCC!=MFX_FOURCC_NV12)
13      return MFX_ERR_UNSUPPORTED;
14   response->NumFrameActual=request->NumFrameMin;
15   for (int i=0;i<request->NumFrameMin;i++) {
16      mid_struct *mmid=(mid_struct *)malloc(sizeof(mid_struct));
17      mmid->width=ALIGN32(request->Info.Width);
18      mmid->height=ALIGN32(request->Info.Height);
19      mmid->base=(mfxU8*)malloc(mmid->width*mmid->height*3/2);
20      response->mids[i]=mmid;
21   }
22   return MFX_ERR_NONE;
23}
24
25mfxStatus fa_lock(mfxHDL pthis, mfxMemId mid, mfxFrameData *ptr) {
26   UNUSED_PARAM(pthis);
27   mid_struct *mmid=(mid_struct *)mid;
28   ptr->Pitch=mmid->width;
29   ptr->Y=mmid->base;
30   ptr->U=ptr->Y+mmid->width*mmid->height;
31   ptr->V=ptr->U+1;
32   return MFX_ERR_NONE;
33}
34
35mfxStatus fa_unlock(mfxHDL pthis, mfxMemId mid, mfxFrameData *ptr) {
36   UNUSED_PARAM(pthis);
37   UNUSED_PARAM(mid);
38   if (ptr) ptr->Y=ptr->U=ptr->V=ptr->A=0;
39   return MFX_ERR_NONE;
40}
41
42mfxStatus fa_gethdl(mfxHDL pthis, mfxMemId mid, mfxHDL *handle) {
43   UNUSED_PARAM(pthis);
44   UNUSED_PARAM(mid);
45   UNUSED_PARAM(handle);
46   return MFX_ERR_UNSUPPORTED;
47}
48
49mfxStatus fa_free(mfxHDL pthis, mfxFrameAllocResponse *response) {
50   UNUSED_PARAM(pthis);
51   for (int i=0;i<response->NumFrameActual;i++) {
52      mid_struct *mmid=(mid_struct *)response->mids[i];
53      free(mmid->base); free(mmid);
54   }
55   return MFX_ERR_NONE;
56}

For system memory, it is highly recommended to allocate memory for all planes of the same frame as a single buffer (using one single malloc call).

Internal Memory Management#

In the internal memory management model, oneVPL provides interface functions for frames allocation:

These functions are used together with mfxFrameSurfaceInterface for surface management. The surface returned by these functions is a reference counted object and the application must call mfxFrameSurfaceInterface::Release after finishing all operations with the surface. In this model the application does not need to create and set the external allocator to oneVPL.

Another method to obtain an internally allocated surface is to call MFXVideoDECODE_DecodeFrameAsync() with a working surface equal to NULL (see Simplified decoding procedure). In this scenario, the decoder will allocate a new refcountable mfxFrameSurface1 and return it to the user. All assumed contracts with the user are similar to the MFXMemory_GetSurfaceForXXX functions.

mfxFrameSurfaceInterface#

oneVPL API version 2.0 introduces mfxFrameSurfaceInterface. This interface is a set of callback functions to manage the lifetime of allocated surfaces, get access to pixel data, and obtain native handles and device abstractions (if suitable). Instead of directly accessing mfxFrameSurface1 structure members, it’s recommended to use the mfxFrameSurfaceInterface if present or call external allocator callback functions if set.

The following pseudo code shows the usage of mfxFrameSurfaceInterface for memory sharing:

 1// lets decode frame and try to access output in an optimal way.
 2sts = MFXVideoDECODE_DecodeFrameAsync(session, NULL, NULL, &outsurface, &syncp);
 3if (MFX_ERR_NONE == sts)
 4{
 5    mfxStatus s = outsurface->FrameInterface->GetDeviceHandle(outsurface,
 6                                                  &device_handle, &device_type);
 7    // if application or component is familar with mfxHandleType and it's
 8    // possible to share memory created by device_handle.
 9    if (MFX_ERR_NONE == s && isDeviceTypeCompatible(device_type)
10                          && isPossibleForMemorySharing(device_handle)) {
11        // get native handle and type
12        outsurface->FrameInterface->GetNativeHandle(outsurface,
13                                                      &resource, &resource_type);
14        if (isResourceTypeCompatible(resource_type)) {
15            //use memory directly
16            ProcessNativeMemory(resource);
17            outsurface->FrameInterface->Release(outsurface);
18        }
19    } else {
20      // Application or component is not aware about such DeviceHandle or
21      // Resource type need to map to system memory.
22      outsurface->FrameInterface->Map(outsurface, MFX_MAP_READ);
23      ProcessSystemMemory(outsurface);
24      outsurface->FrameInterface->Unmap(outsurface);
25      outsurface->FrameInterface->Release(outsurface);
26    }
27}