usbip协议很简单,总共有4对命令:
OP_REQ_DEVLIST
OP_REP_DEVLIST
OP_REQ_IMPORT
OP_REP_IMPORT
USBIP_CMD_SUBMIT
USBIP_RET_SUBMIT
USBIP_CMD_UNLINK
USBIP_RET_UNLINK
分为控制命令和数据传输命令两大类。控制命令主要是用来list设备或者在attach前获取导出设备的信息。
下面是列举(list)已导出的usb设备(export)信息的时序图(usbip list -r的流程)
(图一)
下面是传输usb数据时的命令时序图(usbip attach的流程):
(图二)
在C/S模式中基本都是client发出请求,譬如当client在“usbip attach -r 192.168.100.191 -b 2-1.1”前,进行了“usbip list -r 192.168.100.191”,用于询问host端能导出多少个设备以及设备信息,usbip-host支持最多导出8个usb设备(譬如我可以共享6个U盘和1个鼠标+1个键盘等)。
一) OP_REQ_DEVLIST命令;向server获取设备信息:
(方向:vhci-hcd->usbip-host)
字段1:描述usbip版本,WORD
字段2:命令id,WORD,用于标识OP_REQ_DEVLIST这条命令
字段3:保留字,DWORD
二) OP_REP_DEVLIST命令;为上一条命令的回应。
(方向:usbip-host->vhci-hcd):
看该回应的协议字段可以明显知道,其实就是usb的“标准设备描述符”和“标准接口描述符”的一部分(没有了端点数等),具体意义不用多说,搞usb的都能理解,详情的话请参考《USB2.0官方协议规范》的第九章(CH9)。无论OP_REQ_DEVLIST还是OP_REP_DEVLIST的前8个字节(绿色框)意义类似,结构体在tool/usb/usbip/libsrc/usbip_common.h处定义,说明这对命令是应用程序传输的,不是在驱动传输,而且通过阅读usbip工具代码可以知道,tcp的创建和建立连接均在应用程序usbip工具上创建,至于驱动需要使用socket时,是采取将socket的描述符传入内核,内核利用这个句柄,就能直接利用这个已打开的socket进行通信,无需再在内核建立连接之类的,linux内核会使用kernel_sendmsg和kernel_recvmsg发送和接收tcp数据。
//公共的
/* Common header for all the kinds of PDUs. */
struct op_common {
uint16_t version;
#define OP_REQUEST (0x80 << 8)
#define OP_REPLY (0x00 << 8)
uint16_t code;
/* status codes defined in usbip_common.h */
uint32_t status; /* op_code status (for reply) */
} __attribute__((packed));
//回应特有的
#define SYSFS_PATH_MAX 256
#define SYSFS_BUS_ID_SIZE 32
struct op_devlist_reply {
uint32_t ndev;
/* followed by reply_extra[] */
} __attribute__((packed));
struct usbip_usb_device {
char path[SYSFS_PATH_MAX];
char busid[SYSFS_BUS_ID_SIZE];
uint32_t busnum;
uint32_t devnum;
uint32_t speed;
uint16_t idVendor;
uint16_t idProduct;
uint16_t bcdDevice;
uint8_t bDeviceClass;
uint8_t bDeviceSubClass;
uint8_t bDeviceProtocol;
uint8_t bConfigurationValue;
uint8_t bNumConfigurations;
uint8_t bNumInterfaces;
} __attribute__((packed));
struct usbip_usb_interface {
uint8_t bInterfaceClass;
uint8_t bInterfaceSubClass;
uint8_t bInterfaceProtocol;
uint8_t padding; /* alignment */
} __attribute__((packed));
struct op_devinfo_reply {
struct usbip_usb_device udev;
struct usbip_usb_interface uinf[];
} __attribute__((packed));
其中 0x08偏移处代表server端导出多少个usb设备,如果Number of exported devices为0,就结束了,client不会再发后面的内容,如果导出多个设备,每个设备的信息都是下面的信息,即有多组struct op_devinfo_reply数据,同理,后面0x143偏移处也有一个bNumInterfaces,如果为0,就没有struct usbip_usb_interface的内容,否则每一组是struct usbip_usb_interface,多个就有多组struct usbip_usb_interface数据。
static int get_exported_devices(char *host, int sockfd)函数就利用socket获取设备描述符信息和配置描述符信息的
该命令在运行“usbip list -r 192.168.100.191”时交互。
三) OP_REQ_IMPORT命令;请求attach一个远程usb设备
(方向:vhci-hcd->usbip-host):
前8个字节不用多说,是公共头,偏移 8处是一个描述busid总线号的字符串,有32个字节空间。
四) OP_REP_IMPORT,回应请求attach一个远程usb设备
(方向:usbip-host->vhci-hcd):
跟OP_REP_DEVLIST内容差不多,只是没有了“导出usb设备数量”和“接口描述符”信息。只有“设备描述符”信息,看代码一目了然:
struct op_import_reply {
struct usbip_usb_device udev;
// struct usbip_usb_interface uinf[];
} __attribute__((packed));
static int query_import_device(int sockfd, char *busid)函数就是通过socket获取并解析OP_REP_IMPORT命令内容。
该命令在运行“usbip attach -r 192.168.100.191 -b 2-1.1”时交互。交互完后,即可进行usb URB消息的传输(图二)。而后面两组命令:
USBIP_CMD_SUBMIT
USBIP_RET_SUBMIT
和
USBIP_CMD_UNLINK
USBIP_RET_UNLINK,专门用于usb通信用途,其中USBIP_CMD_UNLINK和USBIP_RET_UNLINK是在usb接口驱动(如U盘驱动、者hid键鼠驱动、usb-skeleton.c等)出现异常时,或者需要终止usb通信等时(接口驱动调用usb_kill_urb()),就会生成unlink命令,用于异常处理,以及内核回收urb对象。而USBIP_CMD_SUBMIT和USBIP_RET_SUBMIT则是当usb接口驱动调用usb_submit_urb()时生成的命令,用于正常usb设备通信。
五) USBIP_CMD_SUBMIT命令,提交一个URB。
(方向:vhci-hcd->usbip-host)
transfer_flags取值参考:
六) USBIP_RET_SUBMIT(方向:usbip-host->vhci-hcd),对提交的URB进行回复,对应于linux URB对象注册的“complete完成回调函数”返回。这得益于usb_submit_urb()时异步的,直接返回,不等待底层主机控制器完成usb数据的传输,直到“完成回调函数”的回调,即代表数据已经传输给usb设备了,得到了U盘等设备的状态返回status,如果异常,会有相应的处理,等下一篇文章会详细分析usbip驱动代码。
urb的状态:
usb_submit_urb的完成函数中通过URB结构体的status成员可以获知其原因,如0表示传输成功,
-ENOENT表示被usb_kill_urb()杀死,
-ECONNRESET表示被usb_unlink_urb()杀死,
-EPROTO表示传输中发生了bitstuff错误或者硬件未能及时收到响应数据包,
-ENODEV表示USB设备已被移除,
-EXDEV表示等时传输仅完成了一部分等。
下文有内核结构struct urb的描述,可关注该结构体的typedef void (*usb_complete_t)(struct urb *);完成回调函数指针和int status; 状态值的英文注释,帮助我们理解。
上面两条命令的字段意义理解需要阅读linux内核usb子系统的重量级对象URB:
/**
* struct urb - USB Request Block
* @urb_list: For use by current owner of the URB.
* @anchor_list: membership in the list of an anchor
* @anchor: to anchor URBs to a common mooring
* @ep: Points to the endpoint's data structure. Will eventually
* replace @pipe.
* @pipe: Holds endpoint number, direction, type, and more.
* Create these values with the eight macros available;
* usb_{snd,rcv}TYPEpipe(dev,endpoint), where the TYPE is "ctrl"
* (control), "bulk", "int" (interrupt), or "iso" (isochronous).
* For example usb_sndbulkpipe() or usb_rcvintpipe(). Endpoint
* numbers range from zero to fifteen. Note that "in" endpoint two
* is a different endpoint (and pipe) from "out" endpoint two.
* The current configuration controls the existence, type, and
* maximum packet size of any given endpoint.
* @stream_id: the endpoint's stream ID for bulk streams
* @dev: Identifies the USB device to perform the request.
* @status: This is read in non-iso completion functions to get the
* status of the particular request. ISO requests only use it
* to tell whether the URB was unlinked; detailed status for
* each frame is in the fields of the iso_frame-desc.
* @transfer_flags: A variety of flags may be used to affect how URB
* submission, unlinking, or operation are handled. Different
* kinds of URB can use different flags.
* @transfer_buffer: This identifies the buffer to (or from) which the I/O
* request will be performed unless URB_NO_TRANSFER_DMA_MAP is set
* (however, do not leave garbage in transfer_buffer even then).
* This buffer must be suitable for DMA; allocate it with
* kmalloc() or equivalent. For transfers to "in" endpoints, contents
* of this buffer will be modified. This buffer is used for the data
* stage of control transfers.
* @transfer_dma: When transfer_flags includes URB_NO_TRANSFER_DMA_MAP,
* the device driver is saying that it provided this DMA address,
* which the host controller driver should use in preference to the
* transfer_buffer.
* @sg: scatter gather buffer list, the buffer size of each element in
* the list (except the last) must be divisible by the endpoint's
* max packet size if no_sg_constraint isn't set in 'struct usb_bus'
* @num_mapped_sgs: (internal) number of mapped sg entries
* @num_sgs: number of entries in the sg list
* @transfer_buffer_length: How big is transfer_buffer. The transfer may
* be broken up into chunks according to the current maximum packet
* size for the endpoint, which is a function of the configuration
* and is encoded in the pipe. When the length is zero, neither
* transfer_buffer nor transfer_dma is used.
* @actual_length: This is read in non-iso completion functions, and
* it tells how many bytes (out of transfer_buffer_length) were
* transferred. It will normally be the same as requested, unless
* either an error was reported or a short read was performed.
* The URB_SHORT_NOT_OK transfer flag may be used to make such
* short reads be reported as errors.
* @setup_packet: Only used for control transfers, this points to eight bytes
* of setup data. Control transfers always start by sending this data
* to the device. Then transfer_buffer is read or written, if needed.
* @setup_dma: DMA pointer for the setup packet. The caller must not use
* this field; setup_packet must point to a valid buffer.
* @start_frame: Returns the initial frame for isochronous transfers.
* @number_of_packets: Lists the number of ISO transfer buffers.
* @interval: Specifies the polling interval for interrupt or isochronous
* transfers. The units are frames (milliseconds) for full and low
* speed devices, and microframes (1/8 millisecond) for highspeed
* and SuperSpeed devices.
* @error_count: Returns the number of ISO transfers that reported errors.
* @context: For use in completion functions. This normally points to
* request-specific driver context.
* @complete: Completion handler. This URB is passed as the parameter to the
* completion function. The completion function may then do what
* it likes with the URB, including resubmitting or freeing it.
* @iso_frame_desc: Used to provide arrays of ISO transfer buffers and to
* collect the transfer status for each buffer.
*
* This structure identifies USB transfer requests. URBs must be allocated by
* calling usb_alloc_urb() and freed with a call to usb_free_urb().
* Initialization may be done using various usb_fill_*_urb() functions. URBs
* are submitted using usb_submit_urb(), and pending requests may be canceled
* using usb_unlink_urb() or usb_kill_urb().
*
* Data Transfer Buffers:
*
* Normally drivers provide I/O buffers allocated with kmalloc() or otherwise
* taken from the general page pool. That is provided by transfer_buffer
* (control requests also use setup_packet), and host controller drivers
* perform a dma mapping (and unmapping) for each buffer transferred. Those
* mapping operations can be expensive on some platforms (perhaps using a dma
* bounce buffer or talking to an IOMMU),
* although they're cheap on commodity x86 and ppc hardware.
*
* Alternatively, drivers may pass the URB_NO_TRANSFER_DMA_MAP transfer flag,
* which tells the host controller driver that no such mapping is needed for
* the transfer_buffer since
* the device driver is DMA-aware. For example, a device driver might
* allocate a DMA buffer with usb_alloc_coherent() or call usb_buffer_map().
* When this transfer flag is provided, host controller drivers will
* attempt to use the dma address found in the transfer_dma
* field rather than determining a dma address themselves.
*
* Note that transfer_buffer must still be set if the controller
* does not support DMA (as indicated by bus.uses_dma) and when talking
* to root hub. If you have to trasfer between highmem zone and the device
* on such controller, create a bounce buffer or bail out with an error.
* If transfer_buffer cannot be set (is in highmem) and the controller is DMA
* capable, assign NULL to it, so that usbmon knows not to use the value.
* The setup_packet must always be set, so it cannot be located in highmem.
*
* Initialization:
*
* All URBs submitted must initialize the dev, pipe, transfer_flags (may be
* zero), and complete fields. All URBs must also initialize
* transfer_buffer and transfer_buffer_length. They may provide the
* URB_SHORT_NOT_OK transfer flag, indicating that short reads are
* to be treated as errors; that flag is invalid for write requests.
*
* Bulk URBs may
* use the URB_ZERO_PACKET transfer flag, indicating that bulk OUT transfers
* should always terminate with a short packet, even if it means adding an
* extra zero length packet.
*
* Control URBs must provide a valid pointer in the setup_packet field.
* Unlike the transfer_buffer, the setup_packet may not be mapped for DMA
* beforehand.
*
* Interrupt URBs must provide an interval, saying how often (in milliseconds
* or, for highspeed devices, 125 microsecond units)
* to poll for transfers. After the URB has been submitted, the interval
* field reflects how the transfer was actually scheduled.
* The polling interval may be more frequent than requested.
* For example, some controllers have a maximum interval of 32 milliseconds,
* while others support intervals of up to 1024 milliseconds.
* Isochronous URBs also have transfer intervals. (Note that for isochronous
* endpoints, as well as high speed interrupt endpoints, the encoding of
* the transfer interval in the endpoint descriptor is logarithmic.
* Device drivers must convert that value to linear units themselves.)
*
* If an isochronous endpoint queue isn't already running, the host
* controller will schedule a new URB to start as soon as bandwidth
* utilization allows. If the queue is running then a new URB will be
* scheduled to start in the first transfer slot following the end of the
* preceding URB, if that slot has not already expired. If the slot has
* expired (which can happen when IRQ delivery is delayed for a long time),
* the scheduling behavior depends on the URB_ISO_ASAP flag. If the flag
* is clear then the URB will be scheduled to start in the expired slot,
* implying that some of its packets will not be transferred; if the flag
* is set then the URB will be scheduled in the first unexpired slot,
* breaking the queue's synchronization. Upon URB completion, the
* start_frame field will be set to the (micro)frame number in which the
* transfer was scheduled. Ranges for frame counter values are HC-specific
* and can go from as low as 256 to as high as 65536 frames.
*
* Isochronous URBs have a different data transfer model, in part because
* the quality of service is only "best effort". Callers provide specially
* allocated URBs, with number_of_packets worth of iso_frame_desc structures
* at the end. Each such packet is an individual ISO transfer. Isochronous
* URBs are normally queued, submitted by drivers to arrange that
* transfers are at least double buffered, and then explicitly resubmitted
* in completion handlers, so
* that data (such as audio or video) streams at as constant a rate as the
* host controller scheduler can support.
*
* Completion Callbacks:
*
* The completion callback is made in_interrupt(), and one of the first
* things that a completion handler should do is check the status field.
* The status field is provided for all URBs. It is used to report
* unlinked URBs, and status for all non-ISO transfers. It should not
* be examined before the URB is returned to the completion handler.
*
* The context field is normally used to link URBs back to the relevant
* driver or request state.
*
* When the completion callback is invoked for non-isochronous URBs, the
* actual_length field tells how many bytes were transferred. This field
* is updated even when the URB terminated with an error or was unlinked.
*
* ISO transfer status is reported in the status and actual_length fields
* of the iso_frame_desc array, and the number of errors is reported in
* error_count. Completion callbacks for ISO transfers will normally
* (re)submit URBs to ensure a constant transfer rate.
*
* Note that even fields marked "public" should not be touched by the driver
* when the urb is owned by the hcd, that is, since the call to
* usb_submit_urb() till the entry into the completion routine.
*/
struct urb {
/* private: usb core and host controller only fields in the urb */
struct kref kref; /* reference count of the URB */
void *hcpriv; /* private data for host controller */
atomic_t use_count; /* concurrent submissions counter */
atomic_t reject; /* submissions will fail */
int unlinked; /* unlink error code */
/* public: documented fields in the urb that can be used by drivers */
struct list_head urb_list; /* list head for use by the urb's
* current owner */
struct list_head anchor_list; /* the URB may be anchored */
struct usb_anchor *anchor;
struct usb_device *dev; /* (in) pointer to associated device */
struct usb_host_endpoint *ep; /* (internal) pointer to endpoint */
unsigned int pipe; /* (in) pipe information */
unsigned int stream_id; /* (in) stream ID */
int status; /* (return) non-ISO status */
unsigned int transfer_flags; /* (in) URB_SHORT_NOT_OK | ...*/
void *transfer_buffer; /* (in) associated data buffer */
dma_addr_t transfer_dma; /* (in) dma addr for transfer_buffer */
struct scatterlist *sg; /* (in) scatter gather buffer list */
int num_mapped_sgs; /* (internal) mapped sg entries */
int num_sgs; /* (in) number of entries in the sg list */
u32 transfer_buffer_length; /* (in) data buffer length */
u32 actual_length; /* (return) actual transfer length */
unsigned char *setup_packet; /* (in) setup packet (control only) */
dma_addr_t setup_dma; /* (in) dma addr for setup_packet */
int start_frame; /* (modify) start frame (ISO) */
int number_of_packets; /* (in) number of ISO packets */
int interval; /* (modify) transfer interval
* (INT/ISO) */
int error_count; /* (return) number of ISO errors */
void *context; /* (in) context for completion */
usb_complete_t complete; /* (in) completion routine */
struct usb_iso_packet_descriptor iso_frame_desc[0];
/* (in) ISO ONLY */
};
linux内核的大牛们对内核重要的结构体都会有一段详细的注释,urb也不例外,注释比代码长。
对于USBIP_CMD_SUBMIT命令重点关注这几个(in)变量:
pipe
transfer_flags
transfer_buffer_length
setup_packet
transfer_buffer
以及complete完成函数的理解。
能从pipe变量里得到协议里的方向direction和端点号ep,从transfer_flags得到命令里的transfer_flags字段,从transfer_buffer_length得到发送数据长度,setup_packet是当pipe指定为0号端口时(控制传输)使用,长度为8字节,不是控制传输阶段要填写全0,从transfer_buffer得到URB data数据。命令的其他字段是关于ISO等时传输相关的。
而对于USBIP_RET_SUBMIT命令重点关注以下几个变量:
pipe(in)
status(return)
actual_length(return)
setup_packet(in)
transfer_buffer(in)
能从pipe变量里得到协议里的方向direction和端点号ep,status得到U盘设备处理后的状态反馈,譬如error码或success等,actual_length代表U盘返回的数据长度,transfer_buffer则是U盘返回的数据。setup_packet为控制传输数据。
七八)USBIP_CMD_UNLINK和USBIP_RET_UNLINK命令