论文A CNN-based prototype method of unstructured surgical state perception and navigation

A CNN-based prototype method of unstructured surgical state perception and navigation for an endovascular surgery robot

1.摘要

Performance of robot-assisted endovascular surgery (ES) remains highly dependent on an individual surgeon’s skills, due to common adoption of master-slave robotic structure. Surgeons’ skill modeling and unstructured surgical state perception pose prohibitive challenges for an autonomous ES robot. In this paper, a novel convolutional neural network (CNN)-based framework is proposed to address these challenges for navigation of an ES robot based on surgeons’ skill learning. An operating action probability estimator is proposed by integrating a two-dimensional CNN, with which the features of a surgical state image are extracted and then directly mapped to the action probability. A one-dimensional CNN with multi-input is developed to recognize the guide wire operating force condition. An eye-hand collaborative servoing algorithm is proposed to combine the outputs of these two networks and to control the robot under a closed-loop architecture. A real-world ES robot is employed for data collection and task performance evaluation in laboratory condition. Compared with the state of the art, the CNN-based method shows its capability of adapting to different situations and achieves similar success rate and average operating time. Robotic operation performs similar operating trajectory and maintains similar level of operating force with manual operation. The CNN-based method can be easily extended to many other surgical robots.

由于通常采用主从机器人结构，机器人辅助血管内手术（ES）的性能仍然高度依赖于外科医生的技能。自主ES机器人面临的挑战包括对外科医生的技能建模和非结构化的手术状态感知。在本文中，基于卷积神经网络（CNN）的框架学习外科医生技能来解决ES机器人导航面临的挑战。通过集成二维CNN，提出了操作动作概率估计器，该二维CNN通过提取手术状态图像的特征，然后直接映射到动作概率。开发具有多输入的一维CNN以识别导线操作力。提出了一种眼手协同伺服算法，以结合这两个网络并在闭环下控制机器人。真实的ES机器人被用于实验室条件下，用于收集数据和任务性能评估。与现有技术相比，基于CNN的方法展示了其适应不同情况的能力，并获得了相似的成功率和平均运行时间。可实现机械手手动操作相似的操作轨迹并保持相似的操作力水平。 基于CNN方法的方法可以轻松地扩展到许多其它外科手术机器人。

2.方案

The input of the network is the preprocessed current surgical state image, and the output is the estimated action probability distribution(pushing forward, pushing forward synchronously with rotat-ing, rotating, pulling back synchronously with rotating, andpulling back; the displacements of the pushing and pulling action are set as 0.6 mm, and the angular displacement of the rotating action is set as 6°).
网络的输入是经过预处理的当前手术状态图像，输出是估计的动作概率分布（向前推动，与旋转同步前进，旋转，与旋转同步向后拉，以及向后拉；推拉动作位移设置为0.6毫米，并且旋转动作角位移为设为6°）。