dma-buf

环境

• Linux kernel Document
• https://www.kernel.org/doc/html/latest/driver-api/dma-buf.html#c.dma_buf

1 Overview

• Two definitions of dma-buf subsystem as the following comments described.

• (1) The dma-buf subsystem provides the framework for sharing buffers for hardware (DMA) access across multiple device drivers and subsystems, and for synchronizing asynchronous hardware access. This is used, for example, by drm “prime” multi-GPU support.
• (2) The dma-buf framework provides a generic method for sharing buffers between multiple devices. Device drivers that support dma-buf can export a DMA buffer to userspace as a file descriptor (known as the exporter role), import a DMA buffer from userspace using a file descriptor previously exported for a different or the same device (known as the importer role), or both.

• The three main components of dma-buf subsystem are:
  • (1) dma-buf, representing a sg_table and exposed to userspace as a file descriptor to allow passing between devices.
  • (2) fence, which provides a mechanism to signal when one device as finished access.
  • (3) reservation, which manages the shared or exclusive fence(s) associated with the buffer.

2 Use the dma-buf

2.1 exporter and importer

• Any device driver which wishes to be a part of DMA buffer sharing, can do so as either the ‘exporter’ of buffers, or the ‘importer’ of buffers. Say a driver A wants to use buffers created by driver B, then we call B as the exporter, and A as buffer-user/importer.

• The exporter

  • implements and manages operations in struct dma_buf_ops for the buffer,
  • allows other users to share the buffer by using dma_buf sharing APIs,
  • manages the details of buffer allocation, wrapped in a struct dma_buf,
  • decides about the actual backing storage where this allocation happens, and takes care of any migration of scatterlist - for all (shared) users of this buffer.

• The buffer-user/importer

  • is one of (many) sharing users of the buffer.
  • doesn’t need to worry about how the buffer is allocated, or where.
  • and needs a mechanism to get access to the scatterlist that makes up this buffer in memory, mapped into its own address space, so it can access the same area of memory. This interface is provided by struct dma_buf_attachment.

• Any exporters or users of the dma-buf buffer sharing framework must have a ‘select DMA_SHARED_BUFFER’ in their respective Kconfigs.

2.2 DMA buffer file descriptor

• Mostly a DMA buffer file descriptor is simply an opaque object for userspace, and hence the generic interface exposed is very minimal.
• userspace must have a way to request O_CLOEXEC flag be set when the dma-buf fd is created.

2.3 Basic Operation and Device DMA Access

2.3.1 exporter operation

• The exporter defines his exporter instance using DEFINE_DMA_BUF_EXPORT_INFO()and calls dma_buf_export() to wrap a private buffer object into a dma_buf.
• It then exports that dma_buf to userspace as a file descriptor by calling dma_buf_fd().
• This following codes come from intel virtual kvmgt.c

/* #define DEFINE_DMA_BUF_EXPORT_INFO(name)	\
struct dma_buf_export_info name = { .exp_name = KBUILD_MODNAME, \
			 .owner = THIS_MODULE }
*/
struct dma_buf *i915_gem_prime_export(struct drm_device *dev,
				      struct drm_gem_object *gem_obj, int flags)
{
	struct drm_i915_gem_object *obj = to_intel_bo(gem_obj);
	DEFINE_DMA_BUF_EXPORT_INFO(exp_info);				//

	exp_info.ops = &i915_dmabuf_ops;
	exp_info.size = gem_obj->size;
	exp_info.flags = flags;
	exp_info.priv = gem_obj;			
	exp_info.resv = obj->resv;			 
	return drm_gem_dmabuf_export(dev, &exp_info);		//->dma_buf_export
}

int intel_vgpu_get_dmabuf(struct intel_vgpu *vgpu, unsigned int dmabuf_id)
{
	struct dma_buf *dmabuf;
	int dmabuf_fd;
	
	dmabuf = i915_gem_prime_export(dev, &obj->base, DRM_CLOEXEC | DRM_RDWR);
	ret = dma_buf_fd(dmabuf, DRM_CLOEXEC | DRM_RDWR);	//return dma_buf fd
	dmabuf_fd = ret;
	
	return dmabuf_fd;
}

2.3.2 importer operation

• scene 1
  • Only let the file-descriptor convert to a dma_buf by using dma_buf_get(). because memory has allocated from the importer or no more other operations.
• like the above exporter dmabuf_fd will be used in QEMU(GTK output) in the function

void egl_dmabuf_import_texture(QemuDmaBuf *dmabuf)
{
    EGLImageKHR image = EGL_NO_IMAGE_KHR;
    EGLint attrs[64];
    int i = 0;

    attrs[i++] = EGL_DMA_BUF_PLANE0_FD_EXT;
    attrs[i++] = dmabuf->fd;

    image = eglCreateImageKHR(qemu_egl_display, EGL_NO_CONTEXT, 
    							EGL_LINUX_DMA_BUF_EXT, NULL, attrs);                          
    glGenTextures(1, &dmabuf->texture);
    glBindTexture(GL_TEXTURE_2D, dmabuf->texture);
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
    
    glEGLImageTargetTexture2DOES(GL_TEXTURE_2D, (GLeglImageOES)image);
    eglDestroyImageKHR(qemu_egl_display, image);
}

• scene 2
  • Userspace passes this file-descriptors to all drivers it wants this buffer to share with:
    • First the file-descriptor is converted to a dma_buf using dma_buf_get().
    • Then the buffer is attached to the device using **dma_buf_attach()**.
    • Once the buffer is attached to all devices userspace , then it can initiate DMA access to the shared buffer. In the kernel this is done by calling dma_buf_map_attachment() and dma_buf_unmap_attachment().

2.4 CPU Access to DMA Buffer Objects

2.4.1 fallback opeartions

• Fallback operations in the kernel, for example when a device is connected over USB and the kernel needs to shuffle the data around first before sending it away. Cache coherency is handled by braketing any transactions with calls to dma_buf_begin_cpu_access() and dma_buf_end_cpu_access() access.

 dma_buf_begin_cpu_access()
 braketing any transactions
 dma_buf_end_cpu_access() 

2.4.2 vmap

• Since for most kernel internal dma-buf accesses need the entire buffer, a vmap interface is introduced.

// dma_buf_vmap - Create virtual mapping for the buffer object into kernel address space. 
// This call may fail due to lack of virtual mapping address space
void *dma_buf_vmap(struct dma_buf *dmabuf)
void dma_buf_vunmap(struct dma_buf *dmabuf, void *vaddr)