基于segment anything model（SAM）相关性研究的各个方向论文/项目汇总

已于 2023-05-19 21:46:17 修改
阅读量1k 收藏 9
点赞数 1
分类专栏： 目标检测论文精读 文章标签： 人工智能 计算机视觉 深度学习
于 2023-05-19 21:32:21 首次发布
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/weixin_42237113/article/details/130773871
版权

目录

简介

有关anything相关的主流任务: 2d检测相关（AnyObject）, 3d检测相关（Any3D）,AI生成相关（AnyGeneration）, AI模型优化相关(), AI任务相关, etc.

  • AnyObject - 分割、检测、分类、医学图像、OCR、姿态等。
  • AnyGeneration - 文本到图像的生成、编辑、修复、样式转换等。
  • Any3D - 3D 生成、分割等。
  • AnyModel - 任何修剪、任何量化、模型重使用。
  • AnyTask -LLM 控制器 + ModelZoo，通用解码，多任务学习。
  • AnyX - 其他主题：字幕等

anything项目整理

AnyObject

Title & AuthorsIntroUseful Links

Segment Anything
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alex Berg, Wan-Yen Lo, Piotr Dollar, Ross Girshick
> Meta Research
> Preprint’23

[Segment Anything (Project)]		在这里插入图片描述
[Github]
[Page]
[Demo]

OVSeg: Open-Vocabulary Semantic Segmentation with Mask-adapted CLIP
Feng Liang, Bichen Wu, Xiaoliang Dai, Kunpeng Li, Yinan Zhao, Hang Zhang, Peizhao Zhang, Peter Vajda, Diana Marculescu
> Meta Research
> Preprint’23

[OVSeg (Project)]		image[Github]
[Page]

Learning to Segment Every Thing
Ronghang Hu, Piotr Dollar, Kaiming He, Trevor Darrell, Ross Girshick
> UC Berkeley, FAIR
> CVPR’18

[seg_every_thing (Project)]		image[Github]
[Page]

Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection
Shilong Liu and Zhaoyang Zeng and Tianhe Ren and Feng Li and Hao Zhang and Jie Yang and Chunyuan Li and Jianwei Yang and Hang Su and Jun Zhu and Lei Zhang
> IDEA-Research
> Preprint’23

[Grounded-SAM, GroundingDINO (Project)]		在这里插入图片描述
[Github]
[Demo]

SegGPT: Segmenting Everything In Context
Xinlong Wang, Xiaosong Zhang, Yue Cao, Wen Wang, Chunhua Shen, Tiejun Huang
> BAAI-Vision
> Preprint’23

[SegGPT (Project)]		image[Github]
V3Det: Vast Vocabulary Visual Detection Dataset
Jiaqi Wang, Pan Zhang, Tao Chu, Yuhang Cao, Yujie Zhou, Tong Wu, Bin Wang, Conghui He, Dahua Lin
> Shanghai AI Laboratory, CUHK
> Preprint’23		image–

segment-anything-video (Project)
Kadir Nar		在这里插入图片描述

[Github]

Towards Segmenting Anything That Moves
Achal Dave, Pavel Tokmakov, Deva Ramanan
> ICCV’19 Workshop

[segment-any-moving (Project)]		[Github]

Semantic Segment Anything
Jiaqi Chen, Zeyu Yang, Li Zhang

[Semantic-Segment-Anything (Project)]		image[Github]

Grounded Segment Anything: From Objects to Parts (Project)
Peize Sun and Shoufa Chen		[Github]

GroundedSAM-zero-shot-anomaly-detection (Project)
Yunkang Cao		image[Github]

Segment Anything Labelling Tool (SALT) (Project)
Anurag Ghosh		[Github]

Prompt-Segment-Anything (Project)
Rockey		[Github]

SAM-RBox (Project)
Qingyun Li		intro[Github]

VISAM (Project)
Feng Yan, Weixin Luo, Yujie Zhong, Yiyang Gan, Lin Ma		[Github]

Segment Anything EO tools: Earth observation tools for Meta AI Segment Anything (Project)
Aliaksandr Hancharenka, Alexander Chichigin		[Github]

napari-segment-anything: Segment Anything Model (SAM) native Qt UI (Project)
Jordão Bragantini, Kyle I S Harrington, Ajinkya Kulkarni		image[Github]

SAM-Medical-Imaging: Segment Anything Model (SAM) native Qt UI (Project)
Jordão Bragantini, Kyle I S Harrington, Ajinkya Kulkarni		image[Github]

OCR-SAM: Combining MMOCR with Segment Anything & Stable Diffusion. (Project)
Zhenhua Yang, Qing Jiang		[Github]

segment-anything-u-specify: using sam+clip to segment any objs u specify with text prompts. (Project)
MaybeShewill-CV		[Github]

Segment Everything Everywhere All at Once
Xueyan Zou, Jianwei Yang, Hao Zhang, Feng Li, Linjie Li, Jianfeng Gao, Yong Jae Lee

[SEEM (Project)]		[Github]

SegDrawer: Simple static web-based mask drawer (Project)
Harry		[Github]

Magic Copy: a Chrome extension (Project)
Harry		image[Github]

Track Anything: Segment Anything Meets Videos
Jinyu Yang, Mingqi Gao, Zhe Li, Shang Gao, Fangjing Wang, Feng Zheng

[Track-Anything (Project)]		[Github]
[Demo]

Count Anything (Project)
Liqi Yan		image[Github]

Segment-and-Track-Anything (Project)
Zongxin Yang		image[Github]

Pose for Everything: Towards Category-Agnostic Pose Estimation
Lumin Xu*, Sheng Jin*, Wang Zeng, Wentao Liu, Chen Qian, Wanli Ouyang, Ping Luo, Xiaogang Wang
> CUHK, SenseTime
> ECCV’22 Oral

[Pose-for-Everything (Project)]		[Github]

Relate Anything Model (Project)
Zujin Guo*, Bo Li*, Jingkang Yang*, Zijian Zhou*, Ziwei Liu
> MMLab@NTU
> VisCom Lab, KCL/TongJi		Github

SegmentAnyRGBD (Project)
Jun Cen, Yizheng Wu, Xingyi Li, Jingkang Yang, Yixuan Pei, Lingdong Kong
> Visual Intelligence Lab@HKUST,
> HUST,
> MMLab@NTU,
> Smiles Lab@XJTU,
> NUS		Github



AnyGeneration

Title & AuthorsIntroUseful Links

High-Resolution Image Synthesis with Latent Diffusion Models
Robin Rombach and Andreas Blattmann and Dominik Lorenz and Patrick Esser and Björn Ommer
> LMU München, Runway ML
> CVPR’22

[Stable-Diffusion (Project)]		intro[Github]
[Page]
[Demo]

Adding Conditional Control to Text-to-Image Diffusion Models
Lvmin Zhang, Maneesh Agrawala
> Stanford University
> Preprint’23

[ControlNet (Project)]		intro[Github]
[Demo]
GigaGAN: Large-scale GAN for Text-to-Image Synthesis
Minguk Kang, Jun-Yan Zhu, Richard Zhang, Jaesik Park, Eli Shechtman, Sylvain Paris, Taesung Park
> POSTECH, Carnegie Mellon University, Adobe Research
> CVPR’23		image[Page]

Inpaint-Anything: Segment Anything Meets Image Inpainting (Project)
Tao Yu		[Github]

IEA: Image Editing Anything (Project)
Zhengcong Fei		intro[Github]

EditAnything (Project)
Shanghua Gao, Pan Zhou		[Github]

Segment Anything for Stable Diffusion Webui (Project)
Chengsong Zhang		image[Github]

Segment Anything with Clip (Project)
Jinwoo Park		intro[Github]

ShowAnything: Edit and Generate Anything In Image and Video (Project)
Showlab, NUS		Github

Transfer-Any-Style: About An interactive demo based on Segment-Anything for style transfer (Project)
LV-Lab, NUS		Github



Any3D

Title & AuthorsIntroUseful Links

Anything-3D: Segment-Anything + 3D, Let’s lift the anything to 3D (Project)
LV-Lab, NUS		Github

SAM 3D Selector: Utilizing segment-anything to help the region selection of 3D point cloud or mesh. (Project)
Nexuslrf		Github

3D-Box via Segment Anything. (Project)
dvlab-research		[Github]

Segment Anything 3D (Project)
Yunhan Yang, Xiaoyang Wu		[Github]



AnyModel

Title & AuthorsIntroUseful Links
[
DepGraph: Towards Any Structural Pruning
Gongfan Fang, Xinyin Ma, Mingli Song, Michael Bi Mi, Xinchao Wang
> Learning and Vision Lab @ NUS
> CVPR’23

[Torch-Pruning (Project)]		[Github]
[Demo]

MQBench: Towards Reproducible and Deployable Model Quantization Benchmark
Yuhang Li and Mingzhu Shen and Jian Ma and Yan Ren and Mingxin Zhao and Qi Zhang and Ruihao Gong and Fengwei Yu and Junjie Yan
> SenseTime Research
> NeurIPS’21

[MQBench (Project)]		intro[Github]
[Page]

OTOv2: Automatic, Generic, User-Friendly
Tianyi Chen, Luming Liang, Tianyu Ding, Ilya Zharkov
> Microsoft
> ICLR’23

[Only Train Once (Project)]		intro[Github]

Deep Model Reassembly
Xingyi Yang, Daquan Zhou, Songhua Liu, Jingwen Ye, Xinchao Wang
LV Lab, NUS
> NeurIPS’22

[Deep Model Reassembly (Project)]
[Github]
[Page]



AnyTask

Title & AuthorsIntroUseful Links

HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in HuggingFace
Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, Yueting Zhuang
> Zhejiang University, MSRA
Preprint’23

[Jarvis (Project)]		[Github]
[Demo]
TaskMatrix.AI: Completing Tasks by Connecting Foundation Models with Millions of APIs
Yaobo Liang, Chenfei Wu, Ting Song, Wenshan Wu, Yan Xia, Yu Liu, Yang Ou, Shuai Lu, Lei Ji, Shaoguang Mao, Yun Wang, Linjun Shou, Ming Gong, Nan Duan
> Microsoft
> > Preprint’23		[Github]

Generalized Decoding for Pixel, Image and Language
Xueyan Zou, Zi-Yi Dou, Jianwei Yang, Zhe Gan, Linjie Li, Chunyuan Li, Xiyang Dai, Harkirat Behl, Jianfeng Wang, Lu Yuan, Nanyun Peng, Lijuan Wang, Yong Jae Lee, Jianfeng Gao
> Microsoft
> CVPR’23

[X-Decoder (Project)]		intro[Github]
[Page]
[Demo]

Pre-Trained Image Processing Transformer
Chen, Hanting and Wang, Yunhe and Guo, Tianyu and Xu, Chang and Deng, Yiping and Liu, Zhenhua and Ma, Siwei and Xu, Chunjing and Xu, Chao and Gao, Wen
> Huawei-Noah
> CVPR’21

[Pretrained-IPT (Project)]		[Github]

OpenAGI: When LLM Meets Domain Experts
Yingqiang Ge, Wenyue Hua, Jianchao Ji, Juntao Tan, Shuyuan Xu, Yongfeng Zhang
> Rutgers University
> Preprint’23

[OpenAGI (Project)]		Github



AnyX

Title & AuthorsIntroUseful Links

Caption Anything: Interactive Image Description with Diverse Multimodal Controls
Teng Wang, Jinrui Zhang, Junjie Fei, Hao Zheng, Yunlong Tang, Zhe Li, Mingqi Gao, Shanshan Zhao
> SUSTech VIP Lab
> Preprint’23

Caption Anything (Project)		[Github]
[Demo]

Image2Paragraph:Transform Image into Unique Paragraph (Project)
Jinpeng Wang		Github
…



论文汇总

AnyObejct

PaperFirst AuthorVenueTopic
Segment AnythingAlexander KirillovPreprint’23Segmentation
Learning to Segment Every ThingRonghang HuCVPR’18
Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object DetectionShilong LiuPreprint’23Grouding+Detection
SegGPT: Segmenting Everything In ContextXinlong WangPreprint’23Segmentation
V3Det: Vast Vocabulary Visual Detection DatasetJiaqi WangPreprint’23Dataset
Pose for Everything: Towards Category-Agnostic Pose EstimationLumin XuECCV’22 OralPose

AnyGeneration

PaperFirst AuthorVenueTopic
High-Resolution Image Synthesis with Latent Diffusion ModelsRobin RombachCVPR’22Text-to-Image Generation
Adding Conditional Control to Text-to-Image Diffusion ModelsLvmin ZhangPreprint’23Controlllable Generation
GigaGAN: Large-scale GAN for Text-to-Image SynthesisMinguk KangCVPR’23Large-scale GAN
Inpaint Anything: Segment Anything Meets Image InpaintingTao YuPreprint’23Inpainting

AnyModel

PaperFirst AuthorVenueTopic
DepGraph: Towards Any Structural PruningGongfan FangCVPR’23Network Pruning
MQBench: Towards Reproducible and Deployable Model Quantization BenchmarkYuhang LiNeurIPS’21Network Quantization
OTOv2: Automatic, Generic, User-FriendlyTianyi ChenICLR’23Network Pruning
Deep Model ReassemblyXingyi YangNeurIPS’22Model Reuse

AnyTask

PaperFirst AuthorVenueTopic
HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in HuggingFaceYongliang ShenPreprint’23Modelzoo + LLM
TaskMatrix.AI: Completing Tasks by Connecting Foundation Models with Millions of APIsYaobo LiangPreprint’23Modelzoo + LLM
Generalized Decoding for Pixel, Image and LanguageXueyan ZouCVPR’23Multi Tasking
Pre-Trained Image Processing TransformerChen, HantingCVPR’21Low-level Vision
AI扩展坞
关注
  • 1
    点赞
  • 踩
  • 9
    收藏
    觉得还不错? 一键收藏
  • 打赏
    打赏
  • 0
    评论
专栏目录
OCR-SAM：智能OCR解决方案，助力高效文本提取
gitblog_00083的博客
04-27 423
OCR-SAM：智能OCR解决方案，助力高效文本提取 项目地址:https://gitcode.com/yeungchenwa/OCR-SAM 项目简介 OCR-SAM 是一个开源的光学字符识别（Optical Character Recognition, OCR）系统，由开发者Yeung Chenwa创建。这个项目基于深度学习模型，旨在提供快速、准确的文字检测和识别功能，尤其适用于图像中的文字...
【Relate-Anything】来了！帮你搞定一切关系！
我爱计算机视觉
04-25 338
关注公众号，发现CV技术之美本月初，Meta推出的一款可以【分割一切】的模型已经引起了广泛的关注。今天，我们很荣幸向大家介绍一款名为“Relate-Anything-Model（RAM）”的机器学习模型。它赋予了Segment Anything Model（SAM）识别不同视觉概念之间的各种视觉关系的能力。该模型由南洋理工大学MMLab团队和伦敦国王学院和同济大学的VisCom实验室的同学利用闲暇...
SAM-DETR学习笔记Accelerating DETR Convergence via Semantic-Aligned Matching
BestHammer的博客
02-10 1059
SAM-DETR论文
RelateAnything：分割之后再识别对象之间关系，赋予 SAM 新技能
idol24的博客
05-04 317
本文来源 机器之心编辑部本月初，Meta 推出的「分割一切（Segment Anything Model，SAM）」模型引起了广泛的关注。最近，来自南洋理工大学 MMLab团队、伦敦国王学院和同济大学VisCom 实验室的研究者们联合推出了一款名为 「Relate-Anything-Model（RAM）」的新模型。RAM 模型赋予了 Segment Anything Model（SAM）识别...
深度学习论文: Segment Any Anomaly without Training via Hybrid Prompt Regularization
mingo_敏
08-28 2569
提出了一种新的框架，即Segment Any Anomaly + (SAA+)，用于零样本异常分割，并通过混合提示正则化方法改善现代基础模型的适应性。现有的异常分割模型通常依赖于特定领域的微调，限制了它们在无数异常模式之间的泛化能力。将不同的基础模型进行协同组装，以利用多模态先验知识来进行异常定位。为了适应非参数基础模型在异常分割中的应用，进一步引入了从领域专家知识和目标图像上下文中派生的混合提示作为正则化手段。
使用Dino+SAM+Stable diffusion 自动进行图片的修改
deephub
04-30 5609
它可以分割任何的图片，但是如果需要分割特定的物体，则需要需要点、框的特定提示才能准确分割图像。这些技术为探索令人兴奋的图像处理世界提供了坚实的基础 并为艺术家和开发者提供巨大的创造潜力。除了分割以外，我们还可以通过将 SAM 与 Grounding Dino 和 Stable Diffusion 相结合，获得高度准确图像分割结果，并且对分割后的图像进行细微的更改。然后我们创建一个 GroundingDino 模型的实例。绘时我们还需要一个背景的掩码，这个就是上面掩码的反操作。可以看到效果还是很好的。
Segment Anything Model自动全局语义分割
11-08
Segment Anything Model自动全局语义分割，包含彩色掩膜导出代码，包含二值化掩膜代码。 Segment Anything Model是一种端到端的深度学习模型，它以全局的方式对图像进行语义分割。这意味着它不仅可以识别和分割预...
AIGC论文-SAM-Segment Anything
02-29
AIGC论文-SAM-Segment Anything
segment-anything
04-18
"Segment Anything"是Facebook AI团队开源的一个先进图像处理工具，主要专注于图像分割任务。这个工具的目的是为了帮助研究人员和开发者更高效地实现对图像中特定对象的精确识别和分离，从而进行深度学习模型的训练...
TensorRT-使用TensorRT部署加速Segment-Anything分割一切算法-SGA-优质大模型算法部署项目实战
02-29
在本项目实战中，我们将深入探讨如何利用TensorRT这一高效的深度学习推理库，来加速Segment-Anything分割一切算法（SGA）的部署。Segment-Anything是一个高质量的大模型算法，能够处理各种复杂的图像分割任务，而...
ISAM操作说明，基于segment anything的自动化图片标注工具
最新发布
04-19
在本场景中，ISAM被应用于一个基于"Segment Anything"的自动化图片标注工具，这为图像处理和计算机视觉任务提供了高效且精确的标注解决方案。 "Segment Anything"自动化图片标注工具可能利用了深度学习技术，尤其是...
【Segment Any RGBD】帮你分割一切深度图！
我爱计算机视觉
04-27 384
关注公众号，发现CV技术之美本月初，Meta推出的一款可以【分割一切】的模型Segment Anything Model (SAM) 已经引起了广泛的关注。今天，我们很荣幸向大家介绍一款名为“Segment Any RGBD（SAD）”的机器学习模型。与以往所有使用SAM的工具的不同之处在于，SAD读入的图片可以是经过渲染之后的深度图，让SAM直接根据几何信息来分割图像。该项目是由Visual I...
【论文笔记】SAM3D: Zero-Shot 3D Object Detection via Segment Anything Model
weixin_45657478的博客
06-07 2924
【论文笔记】SAM3D: Zero-Shot 3D Object Detection via Segment Anything Model
【街景影像】街景影像分析入门（三） 基于OV-SEG的街景影像语义分割一键式部署教程
123456789
12-18 1704
街景语义分割是获取街景图像中建成环境要素的重要手段。目前互联网上可以接触到的街景语义分割方法（如Deeplab,PSPNet）大多用五六年前的技术，，很多时候不能满足分析需要。这篇博文介绍使用2023年最新的基于CLIP的语义分割OV-SEG进行分割。OV-SEG很强大，基本上想分割什么类型都可以，不限于CityScape和ADK20的类型。
Segment Anything Model (SAM) 部分报告汇总
xiongxyowo的博客
04-25 2745
Segment Anything Model (SAM) 部分报告汇总
【多模态】14、Segment Anything | Meta 推出超强悍可分割一切的模型 SAM
呆呆的猫的博客
04-08 6878
本文主要介绍 segment anything
Segment Anything专题论文和代码汇总
如果想成为中心，那么就到中心去吧。
05-24 2808
分段任意事物模型（SAM）的成功证明了以数据为中心的机器学习的重要性。然而，由于标注遥感（RS）图像的困难和高成本，大量有价值的RS数据仍未被标记，特别是在像素级。在本研究中，我们利用SAM和现有的RS目标检测数据集，开发了一个有效的管道来生成一个大规模的RS分割数据集，称为SAMRS。SAMRS在大小上比现有的高分辨率RS分割数据集多出几个数量级，并提供了对象类别、位置和实例信息，可以用于语义分割、实例分割和对象检测，可以单独或组合。我们还从各个方面对SAMRS进行了全面的分析。
初探PE-Prompt Engineering
康康的博客
05-03 951
prompt engineering 学习
Segment anything model（SAM）有哪些局限性
04-25
Segment anything model (SAM) 是一种针对图像语义分割的模型，它主要的局限性包括以下几点： 1. 训练时间较长：SAM 模型需要大量的数据和计算资源来进行训练，并且训练时间较长，这使得 SAM 模型不太适合一些时间...

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

AI扩展坞

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付：¥1
获取中
扫码支付

您的余额不足，请更换扫码支付或充值

打赏作者

实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值