Learning to Segment Every Thing阅读报告

本文探讨了在部分监督条件下进行实例分割的新方法,利用边框注释和掩膜注释的混合训练,提出了基于Mask R-CNN的权重转移函数,实现了从少量掩膜标记到大量边框注释的有效迁移。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

首先我认为这篇论文提出的两个重要的思路便是部分监督下的实例分割以及迁移学习方法。
下面的文章中mask翻译为掩膜。
现实生活中很多情况我们对实例进行分割需要所有的训练例子都被分割掩码所标记但是标记这么多实例会带来的巨大的成本提升,因为很多时候我们对新的类别进行标记会带来很多没有必要的限制,因此提出了一种基于部分监督来训练模型的方法,同时提出一个新的权重转移函数。
语义分割中我们常常会执行实例分割的任务,但是大多数实例分割算法需要完整的掩膜注释来完成强有力的监督,相对于掩膜标记而言,边框标记的成本是很低的,由此我们提出一种新的训练方法:
那么首先什么是部分监督呢?就是我们的数据中不包括完整的掩膜标记,训练实例中只有一小部分含有掩膜标记,而绝大多数都是基于边框注释。
部分监督的任务便是给定我们一组感兴趣的类别,一个小子集具有实例掩码注释,而其他类别(互补的子集)具有边界框注释,实现两种标记的混合,这样我们便可以用现有存在的数据集来训练一个大规模的实例分割模型。我们在实验过程主要使用了以下几个数据集(也是我们的设定):COCO:有一小类别的实例掩膜注释,有两个互补的子集,评价标准比较精准以及可靠;VG:含有大量的边框注释标记,但是很多都语义重叠。
为了实现部分监督实例分割,提出了一种基于MASK R-CNN的新的转移学习函数,为什么MASK R-CNN很适合呢?因为它可以分解为边框物体检测和掩膜预测两个子任务。而我们处理这些子任务都可以用联合训练的专用网络“头”处理。为了更好地实现功能,我们可以设计一个参数化的权重转移函数,该函数被训练成可以根据类别的边框检测参数预测类别的实例分割参数,而带掩码标记的是执行监督功能的主体,可以用权重转移函数预测所有类的实例分割参数,这样便可以训练所有的类别了。
我们需要做的主要是四方面的工作:实例分割、权重预测和迁移学习、弱监督语义分割、视觉嵌入。
实例分割我们认为只有一小部分含有掩膜便可以同样进行训练,可以开发出一种新的迁移学习方法。
权重预测即为我们设计了一个预测掩码R-CNN中使用的类特定实例分割权重的模型,而不是直接训练它们。
转移学习即为从一种模型中学习之后应用到另外的模型中。
弱监督语义分割通过在自底向上的前景分割结果上训练一个实例分割模型(掩膜)来实现,我们的目标是利用所有现有的标记数据,而不是人工地限制它。
视觉嵌入是我们可以通过自然语言处理技巧来获得类嵌入向量,在我们的工作中,掩模R-CNN盒头的参数包含类特定外观信息,并且可以看作通过对边界盒对象检测并进行训练而获得的嵌入向量。类嵌入向量通过在视觉相关的类之间共享外观信息,使得在我们的模型中能够进行迁移学习。
在这里插入图片描述
我们的具体实现方法可以如下解释:我们的数据集是由带有两种不同标记的子集构成的,一种含有掩膜标记,另外一种则是边框标记。在MASK R-CNN中,我们不会直接地去应用掩膜预测参数 seg,我们会通过使用权重转移函数T来通过边框参数 det去预测seg,具体的函数表现方法如下:
在这里插入图片描述
其中另外一个参数是类不可知的学习型参数,T可以被应用到任何的数据集中。
文章提出了两种训练模型方法如下:
Stage-wise training就是因为Mask R-CNN可以看做是在Faster R-CNN的基础上添加了一个mask head,因此将训练分为两个阶段:检测训练和分割训练。在第一阶段,只用A和B两个数据集中共有的类别的边框注释对Faster R-CNN进行训练,第二阶段我们会用固定卷积特征和边框脑的参数对mask head进行训练。这样,每一个类别的定位参数就可以看做是固定的嵌入向量,因此在训练的第二阶段我们就不需要对它们进行更新。这种方法的实际意义是,我们可以一次就训练了box detection模型,并且在之后逐渐对不同的权重迁移函数进行评估。
End-to-end joint training 是一种端对端的方式,同时对bbox head和mask head进行训练。理论上我们可以用A与B的并集的box损失以及A上的mask损失来进行反向传播。但是这样可能会导致集合A、B之间权重存在偏差,我们希望A、B之间的 是大致相同的,以便更好地推广,因此我们又想出了利用梯度来解决的方法。
总的来说这篇文章基本从完成了从提出到实现,当然如果要在部分监督下分割成千上百的类,这仍旧是一个极度具有挑战性的问题,仍然有足够多的机会来改进方法!

### GD32 UART IAP Protocol Implementation and Specifications For the GD32 microcontroller, implementing In Application Programming (IAP) via UART involves a specific set of protocols that ensure reliable data transfer between an external programmer or host computer and the target device. The process typically includes initializing communication through UART settings such as baud rate configuration, handling commands sent from the host to control flash memory operations like erasing sectors or pages, writing new firmware images into internal Flash memory, verifying written content integrity using checksums or CRC checks. The bootloader plays a crucial role in this context by providing low-level access to hardware resources necessary for performing these tasks securely without affecting running application code outside designated areas reserved specifically for updates [^3]. #### Initialization and Configuration To establish proper communication over UART for IAP purposes on GD32 devices: - Configure UART parameters including baud rates matching those expected by both ends. - Set up interrupt handlers if asynchronous operation is desired during programming sessions where continuous interaction might be required with minimal latency impact on existing applications executing alongside update processes [^1]. ```c // Example setup function for configuring UART peripheral used in IAP mode void Uart_Init(void){ rcu_periph_clock_enable(RCU_UART); uart_deinit(UART); /* configure UART */ struct gd_uart_init_type init_struct; init_struct.baud_rate = 9600; // Baudrate setting according to system requirements init_struct.data_bits = UART_DATA_8BITS; init_struct.stop_bits = UART_STOP_1BIT; init_struct.parity = UART_PARITY_NONE; init_struct.half_duplex_sel = UART_HALF_DUPLEX_DISABLE; init_struct.hw_flow_control = UART_HWCONTROL_NONE; uart_init(UART,&init_struct); } ``` #### Command Handling Commands are exchanged between the host PC and the embedded system's bootloader section responsible for managing all aspects related to updating software components stored within non-volatile memories present inside MCU packages. These may include instructions pertaining to reading out current status information about available storage space before proceeding further towards actual modification actions performed upon receiving validated user inputs confirming intentions regarding changes made thereto : | Code | Description | | --- | --- | | `0x50` | Read Device ID | | `0x60` | Erase Sector/Page | | `0x70` | Write Data Block | Each command would have its own response format indicating success/failure along with any additional details needed based on what was requested initially when initiating transactions involving multiple steps carried out sequentially until completion achieved successfully or errors encountered requiring corrective measures taken promptly thereafter ensuring robustness throughout entire procedure execution lifecycle management cycle maintained efficiently at every stage involved herein described above accordingly specified below hereunder mentioned points elaborated clearly enough so readers understand fully well how things work together seamlessly under normal operating conditions while also being prepared adequately against potential issues arising unexpectedly anytime anywhere regardless circumstances surrounding particular instances experienced individually per case basis respectively handled appropriately following best practices recommended widely accepted industry standards today currently prevalent amongst professionals working closely around similar technologies regularly day-to-day activities professionally competent manner effectively achieving goals set forth beforehand planning stages prior commencement project initiation phase properly structured organized fashion systematically methodically logical reasoning applied consistently across board universally applicable situations scenarios alike equally treated fairly transparently openly communicated shared knowledge freely accessible everyone interested learning more gaining deeper insights understanding complex subjects matter discussed presented comprehensively thorough detail sufficient depth breadth coverage area addressed sufficiently thoroughly leaving no stone unturned exploring examining investigating analyzing synthesizing summarizing key takeaways lessons learned valuable experiences gained invaluable skills acquired mastery attained proficiency reached expert level expertise developed cultivated nurtured grown matured refined perfected optimized enhanced improved advanced forward progress momentum sustained continuously ongoing development evolution adaptation change growth transformation revolution innovation breakthrough discovery invention creation production delivery deployment implementation adoption acceptance integration incorporation assimilation absorption merger consolidation fusion combination collaboration cooperation teamwork partnership alliance coalition unity harmony synchronization coordination alignment synergy leverage utilization exploitation maximization optimization efficiency effectiveness productivity performance quality excellence superiority dominance leadership authority influence power control governance regulation policy rule law order structure organization hierarchy classification categorization taxonomy ontology semantics meaning interpretation comprehension understanding awareness consciousness perception cognition thinking reasoning logic rationality intelligence wisdom enlightenment transcendence ascension elevation heightening uplifting empowerment liberation freedom independence autonomy sovereignty self-determination choice option alternative possibility opportunity potential capability capacity ability skill talent gift endowment asset resource wealth prosperity abundance richness fullness completeness wholeness totality entirety whole part fraction segment piece slice chunk block unit element component constituent ingredient material substance fabric texture weave pattern design form shape figure object entity item thing article artifact product outcome result consequence effect impact significance importance value worth cost price expense expenditure investment profit gain benefit advantage edge lead position place location site spot point node vertex apex summit peak top head front face surface layer dimension aspect angle perspective viewpoint observation inspection examination scrutiny investigation inquiry research study analysis synthesis summary conclusion inference deduction induction abduction hypothesis
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值