Kernel space DMA and User space DMA

Author: Joseph(Honggang Yang) <ganggexiongqi@gmail.com>
Contents: Kernel space DMA and User space DMA
Date: 11-02-2011
Last modified: 11-02-2011
=======================================================
1. The general steps of using DMA in kernel space:

 1.1 coherent DMA mapping
    
    REF: driver/net/8139too.c

    - create coherent DMA mapping
    - write the bus address to the hardware's DMA address registers
    - For data sending, copy data to the DMA buffer.
    - Enable DMA sending(the data in DMA buffer will be send to the deivce),
              and data receive DMA(set the corresponding hardware register,
              then, the data received operation may start at any time)
    - in the interrupt handler:
      If this is a receive interupt, disable the receive DMA interrupt.
         copy the data received which is placed in DMA buffer.
      If this is a sending interupt which means our send operation is compete(or Tx error)
          prepare to send other data.
 1.2 stream DMA mapping

    REF: drivers/net/via-rhine.c

-<a> Allocate the phisical contiguous buffer used for Tx (tx_buf) and Rx (rx_buf) //e.g. kmalloc()
-<b> Copy data to Tx buf
-<c> Create DMA mapping for data sending:
      tx_buf_dma = dma_map_single(&dev, tx_buf, tx_buf_size, DMA_TO_DEVICE);
-<d> Create DMA mapping for data Rx:
      rx_buf_dma = dma_map_single(&dev, rx_buf, rx_buf_size, DMA_FROM_DEVICE);
-<e> Write tx_buf_dma and rx_buf_dma to the hardware registers used to find the buf to place the
   received data or the data you want to send out.
-<f> Enable the Tx and Rx of the hardware device

-- In the interrupt handler:
   if this is Tx complete interrupt:
      unmap the DMA mapping, then, you can reuse the tx_buf ...
      dma_unmap_single(&dev, tx_buf, tx_buf_size, DMA_TO_DEVICE);
   if this is a RxOK interrupt:
      Mask the Rx interrupt.
      Get the buffer(then, driver can access the rx_buf)//or just unmap (dma_unmap_single(&dev, rx_buf_dma, rx_buf_size, DMA_FROM_DEVICE);)  
       dma_sync_single_for_cpu(&dev, rx_buf_dma, rx_buf_size, DMA_FROM_DEVICE);    
      Copy the data received to the consumer.
      Return the buffer to Device(then, the device can put new data in the rx_buf)//or (dma_map_single(&dev, rx_buf, rx_buf_size, DMA_FROM_DEVICE);)
       dma_sync_single_for_device(&dev, rx_buf_dma, rx_buf_size, DMA_FROM_DEVICE);
     Unmask the Rx interrupt.

2. User space DMA (stream DMA mapping)

 In order to surport userspace DMA, we have to do someting in the kernel space.
 Here is a analysis of uio-dma.c, which is writen by Max Krasnyansky(<maxk@qualcomm.com>).
 The following will show you how it complete the task of surporting the userspace DMA demands.
 
 If the uspace program want to do the DMA operation, the operations mentioned in part 1.2 must be
 done. The following problems must be solved:
      a> how to find a physical contiguous memory to do DMA operation
      b> how to create a stream DMA mapping
      c> how to delete DMA mapping
      d> how to get the received data
      e> how to put data in the Tx buffer  

3. uio DMA userspace APIs
 3.1
/**
 * Allocate new DMA area.
 * @param fd UIO DMA file descriptor
 * @param size of the area
 * @param cache caching mode
 * @param dma_mask mask of the DMA address range
 * @param memnode memory node to allocate the memory area on
 */
struct uio_dma_area *uio_dma_alloc(int fd, unsigned int size,
        unsigned int cache, uint64_t dma_mask, unsigned int memnode);

         struct uio_dma_area *da;
         struct uio_dma_alloc_req areq;
          ...
         /* <1> allocate the uio dma area */
         ioctl(fd, UIO_DMA_ALLOC, (unsigned long) &areq);  
          ...
         /* <2> map the uio dma area to userspace, da->addr is the uio dma area's virtual
                address can be used by the userspace driver */
         da->addr = mmap(NULL, da->size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, da->mmap_offset);
         ...
         return da;
         -----
   NOTE: This process solves the problem a> and e> in part 2.

 3.2
/**
 * Free DMA area.
 * @param fd UIO DMA file descriptor
 * @param pointer to @see uio_dma_area
 */
int uio_dma_free(int fd, struct uio_dma_area *a);

        ...
        ioctl(fd, UIO_DMA_FREE, (unsigned long) &freq)
        ...

 3.3
/**
 * Map DMA area to a device.
 * @param fd UIO DMA file descriptor
 * @param pointer to @see uio_dma_area
 * @param devid uio dma device id
 * @param dir direction
 * @return pointer to new @see uio_dma_mapping
 */
struct uio_dma_mapping * uio_dma_map(int fd, struct uio_dma_area *area, uint32_t devid, unsigned int dir);
 
        struct {
             struct uio_dma_map_req req;
             uint64_t dmaddr[area->chunk_count];
        } mreq;
        ...
        /* Here we allocate "sizeof(uint64_t) * area->chunk_count" in order to save "m->dmaddr[i]"*/
        m = malloc(sizeof(*m) + sizeof(uint64_t) * area->chunk_count);
        ...
        /* In kernel part will do stream DMA mapping:
           uio_dma_mapping_add() will be called.
             ...
             // m->dmaddr[i] contains every chunk's bus address which can be used by DMA devices.
             m->dmaddr[i] = dma_map_single(ud->device, area->addr[i], area->chunk_size, dir);
             ...
        */
        err = ioctl(fd, UIO_DMA_MAP, (unsigned long) &mreq);
        ...

   NOTE:  m->dmaddr[i] contains every chunk's bus addresses can be used by DMA devices.
          
          This process solves the problem c> in part 2.
 
   ***** As to the @devid, we must do the following steps:
   *****  Call uio_dma_device_open(struct device *dev, uint32_t *id) to get uio dma device id,
   *****  and you can create a file under /sys/class/uio/uio0/device/uio_dma_id. So as to
   *****  save the value of @devid to the userspace.
 
 3.4
/**
 * Unmap DMA area from a device.
 * @param fd UIO DMA file descriptor
 * @param pointer to @see uio_dma_mapping
 * @return 0 on success, <0 otherwise (sets errno)
 */
int uio_dma_unmap(int fd, struct uio_dma_mapping *m);

        ...
        /*
        * in kernel delete the DMA mapping:
        * dma_unmap_single(ud->device, m->dmaddr[i], m->area->chunk_size, m->direction)
        */
        ioctl(fd, UIO_DMA_UNMAP, (unsigned long) &ureq)
        ...

============================================================================================================
REF:
  1. DMA in user space (uio dma) //code analysis  http://blog.csdn.net/ganggexiongqi/article/details/6929659
  2.