DRM vulkan同步(dma fence)机制 Sync Objects

DRM Sync Objects
DRM synchronisation objects (syncobj, see struct drm_syncobj) provide a container for a synchronization primitive which can be used by userspace to explicitly synchronize GPU commands, can be shared between userspace processes, and can be shared between different DRM drivers. Their primary use-case is to implement Vulkan fences and semaphores. The syncobj userspace API provides ioctls for several operations:

Creation and destruction of syncobjs
Import and export of syncobjs to/from a syncobj file descriptor
Import and export a syncobj’s underlying fence to/from a sync file
Reset a syncobj (set its fence to NULL)
Signal a syncobj (set a trivially signaled fence)
Wait for a syncobj’s fence to appear and be signaled
At it’s core, a syncobj is simply a wrapper around a pointer to a struct dma_fence which may be NULL. When a syncobj is first created, its pointer is either NULL or a pointer to an already signaled fence depending on whether the DRM_SYNCOBJ_CREATE_SIGNALED flag is passed to DRM_IOCTL_SYNCOBJ_CREATE. When GPU work which signals a syncobj is enqueued in a DRM driver, the syncobj fence is replaced with a fence which will be signaled by the completion of that work. When GPU work which waits on a syncobj is enqueued in a DRM driver, the driver retrieves syncobj’s current fence at the time the work is enqueued waits on that fence before submitting the work to hardware. If the syncobj’s fence is NULL, the enqueue operation is expected to fail. All manipulation of the syncobjs’s fence happens in terms of the current fence at the time the ioctl is called by userspace regardless of whether that operation is an immediate host-side operation (signal or reset) or or an operation which is enqueued in some driver queue. DRM_IOCTL_SYNCOBJ_RESET and DRM_IOCTL_SYNCOBJ_SIGNAL can be used to manipulate a syncobj from the host by resetting its pointer to NULL or setting its pointer to a fence which is already signaled.

Host-side wait on syncobjs
DRM_IOCTL_SYNCOBJ_WAIT takes an array of syncobj handles and does a host-side wait on all of the syncobj fences simultaneously. If DRM_SYNCOBJ_WAIT_FLAGS_WAIT_ALL is set, the wait ioctl will wait on all of the syncobj fences to be signaled before it returns. Otherwise, it returns once at least one syncobj fence has been signaled and the index of a signaled fence is written back to the client.

Unlike the enqueued GPU work dependencies which fail if they see a NULL fence in a syncobj, if DRM_SYNCOBJ_WAIT_FLAGS_WAIT_FOR_SUBMIT is set, the host-side wait will first wait for the syncobj to receive a non-NULL fence and then wait on that fence. If DRM_SYNCOBJ_WAIT_FLAGS_WAIT_FOR_SUBMIT is not set and any one of the syncobjs in the array has a NULL fence, -EINVAL will be returned. Assuming the syncobj starts off with a NULL fence, this allows a client to do a host wait in one thread (or process) which waits on GPU work submitted in another thread (or process) without having to manually synchronize between the two. This requirement is inherited from the Vulkan fence API.

Import/export of syncobjs
DRM_IOCTL_SYNCOBJ_FD_TO_HANDLE and DRM_IOCTL_SYNCOBJ_HANDLE_TO_FD provide two mechanisms for import/export of syncobjs.

The first lets the client import or export an entire syncobj to a file descriptor. These fd’s are opaque and have no other use case, except passing the syncobj between processes. All exported file descriptors and any syncobj handles created as a result of importing those file descriptors own a reference to the same underlying struct drm_syncobj and the syncobj can be used persistently across all the processes with which it is shared. The syncobj is freed only once the last reference is dropped. Unlike dma-buf, importing a syncobj creates a new handle (with its own reference) for every import instead of de-duplicating. The primary use-case of this persistent import/export is for shared Vulkan fences and semaphores.

The second import/export mechanism, which is indicated by DRM_SYNCOBJ_FD_TO_HANDLE_FLAGS_IMPORT_SYNC_FILE or DRM_SYNCOBJ_HANDLE_TO_FD_FLAGS_EXPORT_SYNC_FILE lets the client import/export the syncobj’s current fence from/to a sync_file. When a syncobj is exported to a sync file, that sync file wraps the sycnobj’s fence at the time of export and any later signal or reset operations on the syncobj will not affect the exported sync file. When a sync file is imported into a syncobj, the syncobj’s fence is set to the fence wrapped by that sync file. Because sync files are immutable, resetting or signaling the syncobj will not affect any sync files whose fences have been imported into the syncobj.

struct drm_syncobj
sync object.

Definition

struct drm_syncobj {
struct kref refcount;
struct dma_fence __rcu *fence;
struct list_head cb_list;
spinlock_t lock;
struct file *file;
};
Members

refcount
Reference count of this object.
fence
NULL or a pointer to the fence bound to this object.

This field should not be used directly. Use drm_syncobj_fence_get() and drm_syncobj_replace_fence() instead.

cb_list
List of callbacks to call when the fence gets replaced.
lock
Protects cb_list and write-locks fence.
file
A file backing for this syncobj.
Description

This structure defines a generic sync object which wraps a dma_fence.

void drm_syncobj_get(struct drm_syncobj * obj)
acquire a syncobj reference

Parameters

struct drm_syncobj * obj
sync object
Description

This acquires an additional reference to obj. It is illegal to call this without already holding a reference. No locks required.

void drm_syncobj_put(struct drm_syncobj * obj)
release a reference to a sync object.

Parameters

struct drm_syncobj * obj
sync object.
struct dma_fence * drm_syncobj_fence_get(struct drm_syncobj * syncobj)
get a reference to a fence in a sync object

Parameters

struct drm_syncobj * syncobj
sync object.
Description

This acquires additional reference to drm_syncobj.fence contained in obj, if not NULL. It is illegal to call this without already holding a reference. No locks required.

Return

Either the fence of obj or NULL if there’s none.

struct drm_syncobj * drm_syncobj_find(struct drm_file * file_private, u32 handle)
lookup and reference a sync object.

Parameters

struct drm_file * file_private
drm file private pointer
u32 handle
sync object handle to lookup.
Description

Returns a reference to the syncobj pointed to by handle or NULL. The reference must be released by calling drm_syncobj_put().

void drm_syncobj_add_point(struct drm_syncobj * syncobj, struct dma_fence_chain * chain, struct dma_fence * fence, uint64_t point)
add new timeline point to the syncobj
Parameters

struct drm_syncobj * syncobj
sync object to add timeline point do
struct dma_fence_chain * chain
chain node to use to add the point
struct dma_fence * fence
fence to encapsulate in the chain node
uint64_t point
sequence number to use for the point
Description

Add the chain node as new timeline point to the syncobj.

void drm_syncobj_replace_fence(struct drm_syncobj * syncobj, struct dma_fence * fence)
replace fence in a sync object.

Parameters

struct drm_syncobj * syncobj
Sync object to replace fence in
struct dma_fence * fence
fence to install in sync file.
Description

This replaces the fence on a sync object.

int drm_syncobj_find_fence(struct drm_file * file_private, u32 handle, u64 point, u64 flags, struct dma_fence ** fence)
lookup and reference the fence in a sync object

Parameters

struct drm_file * file_private
drm file private pointer
u32 handle
sync object handle to lookup.
u64 point
timeline point
u64 flags
DRM_SYNCOBJ_WAIT_FLAGS_WAIT_FOR_SUBMIT or not
struct dma_fence ** fence
out parameter for the fence
Description

This is just a convenience function that combines drm_syncobj_find() and drm_syncobj_fence_get().

Returns 0 on success or a negative error value on failure. On success fence contains a reference to the fence, which must be released by calling dma_fence_put().

void drm_syncobj_free(struct kref * kref)
free a sync object.

Parameters

struct kref * kref
kref to free.
Description

Only to be called from kref_put in drm_syncobj_put.

int drm_syncobj_create(struct drm_syncobj ** out_syncobj, uint32_t flags, struct dma_fence * fence)
create a new syncobj
Parameters

struct drm_syncobj ** out_syncobj
returned syncobj
uint32_t flags
DRM_SYNCOBJ_* flags
struct dma_fence * fence
if non-NULL, the syncobj will represent this fence
Description

This is the first function to create a sync object. After creating, drivers probably want to make it available to userspace, either through drm_syncobj_get_handle() or drm_syncobj_get_fd().

Returns 0 on success or a negative error value on failure.

int drm_syncobj_get_handle(struct drm_file * file_private, struct drm_syncobj * syncobj, u32 * handle)
get a handle from a syncobj
Parameters

struct drm_file * file_private
drm file private pointer
struct drm_syncobj * syncobj
Sync object to export
u32 * handle
out parameter with the new handle
Description

Exports a sync object created with drm_syncobj_create() as a handle on file_private to userspace.

Returns 0 on success or a negative error value on failure.

int drm_syncobj_get_fd(struct drm_syncobj * syncobj, int * p_fd)
get a file descriptor from a syncobj
Parameters

struct drm_syncobj * syncobj
Sync object to export
int * p_fd
out parameter with the new file descriptor
Description

Exports a sync object created with drm_syncobj_create() as a file descriptor.

Returns 0 on success or a negative error value on failure.

signed long drm_timeout_abs_to_jiffies(int64_t timeout_nsec)
calculate jiffies timeout from absolute value

Parameters

int64_t timeout_nsec
timeout nsec component in ns, 0 for poll
Description

Calculate the timeout in jiffies from an absolute time in sec/nsec.