打破CloseAI的垄断，开源一个多模态版本的OpenAI o1！-CSDN博客

关注公众号，发现CV技术之美

本文为粉丝投稿。

项目简介

OpenAI 的 O1 通过引入推理链条来增强逻辑推理能力，受到了广泛关注。然而，CloseAI 并没有对 OpenAI O1 进行开源，甚至没有公布足够的技术细节。最近，社区内尝试了一些低成本实现 OpenAI O1 的方案，比如 G1 和 O1。

为了应对图像理解这一关键能力的需求，我们开发了 Multimodal Open-O1（MO1）项目，利用开源多模态模型（如 Qwen2VL 7B）作为测试模型，来实现多模态版本的 OpenAI O1。值得一提的是，MO1 项目中的模型可以灵活替换为其他开源或闭源的多模态大模型（MLLM），以满足不同的应用需求。

目前，该项目还处于开发初期阶段，我们真诚地欢迎更多开发者和研究者加入我们，共同参与开发和优化。请给项目加星（star），并贡献你的智慧和力量！

项目链接

https://github.com/xmu-xiaoma666/Multimodal-Open-O1

实验设置

我们使用 Qwen2VL 7B 作为测试模型，该模型可以灵活替换为其他开源或闭源的多模态大模型（MLLM）。实验结果显示，在某些案例中，利用推理链策略，模型性能确实优于原有方案。

实验记录

多模态场景

Case1: 物体计数

image

问题：How many fruits are there in the picture?

思考过程：

Thinking Step 1:	The task is to count the number of fruits in the picture. The image contains eight different fruits.
Thinking Step 2:	To count the fruits, I will examine each one individually. There are 8 fruits in total.
Final Answer	There are 8 fruits in the picture.

最后的回答：

There are 8 fruits in the picture.

Case2: 方向感知

image 1

问题：右下角的水果是什么？

思考过程：

Step	Content
Thinking Step 1:	The image shows a collection of fruits. The bottom right corner contains a cluster of dark purple grapes. This is the fruit we need to identify.
Final Answer	The fruit in the bottom right corner of the image is a cluster of dark purple grapes.

最后的回答：

The fruit in the bottom right corner of the image is a cluster of dark purple grapes.

Noting：在相同的问题下，QWen2-VL的答案是：右下角的水果是石榴。

image 2

Case3: 行列感知

image 3

问题：What is the second fruit in the first line?

思考过程：

Step	Content
Thinking Step 1:	To find the second fruit in the first line, I need to analyze the image and determine the sequence of fruits. The first fruit in the first line is an apple. The second fruit should be the next in the sequence. Let's examine the options.
Thinking Step 2:	Upon examining the sequence of fruits in the image, I can see that the first fruit is an apple. The second fruit in the first line should logically follow the sequence. Let's consider the options available in the image.
Thinking Step 3:	After analyzing the sequence of fruits in the image, I can conclude that the second fruit in the first line is a strawberry. This is based on the logical progression of the sequence and the presence of the strawberry in the image.
Final Answer	The second fruit in the first line is a strawberry.

最后的回答：

The second fruit in the first line is a strawberry.

纯文本场景

Case1: 字母计数（英文）

image 4

问题：How many 'R's are in the word strawberry?

思考过程：

Step	Content
Thinking Step 1:	The problem is to count the number of 'R's in the word 'strawberry'. To solve this, I need to examine each letter in the word and determine if it is an 'R'.
Thinking Step 2:	Now, I will go through each letter in the word 'strawberry' and check if it is an 'R'. I will start with the first letter and continue until the end of the word.
Thinking Step 3:	I have now examined each letter in the word 'strawberry'. The letters are: s, t, r, a, w, b, r, r, b, r, r, y. Among these, the letters 'R' appear three times.
Final Answer	The word 'strawberry' contains three 'R's.

最后的回答：

The word 'strawberry' contains three 'R's.

Noting：在相同的问题下，QWen2-VL的答案是：The word "strawberry" contains one 'R'.

image 5

Case2: 六年级数学题

image 6

问题：A tank of water, using 50% and 5 barrels, leaving 30%. How many barrels of water are in this tank?

思考过程：

Step	Content
Thinking Step 1:	The problem involves a tank of water with a certain amount of water initially. The tank is filled with 50% of its capacity and then 5 barrels of water are added. After this addition, the tank is filled to 30% of its capacity. The task is to determine the total number of barrels of water in the tank.
Thinking Step 2:	Failed to generate step after 3 attempts. Error: JSON decode error : {"title": "Analyzing the Problem", "content": "Let's denote the total capacity of the tank as C barrels. Initially, the tank is filled with 50% of its capacity, which means it contains 0.5C barrels of water. Then, 5 barrels of water are added to the tank, increasing the amount of water to 0.5C + 5 barrels. After this addition, the tank is filled to 30% of its capacity, which means it contains 0.3C barrels of water. We can set up the equation 0.5C + 5 =
Final Answer	To find the total number of barrels of water in the tank, we can set up the equation 0.5C + 5 = 0.3C, where C represents the total capacity of the tank in barrels. Solving for C, we get C = 25 barrels. Therefore, the tank contains 25 barrels of water.

最后的回答：

To find the total number of barrels of water in the tank, we can set up the equation 0.5C + 5 = 0.3C, where C represents the total capacity of the tank in barrels. Solving for C, we get C = 25 barrels. Therefore, the tank contains 25 barrels of water

Noting：在相同的问题下，QWen2-VL的答案是错误的：It is not possible to determine the total number of barrels of water in the tank based on the information provided. The statement "using 50% and 5 barrels, leaving 30%" does not provide enough information to determine the total number of barrels in the tank.

image 7

Case3: 一年级数学题（中文）

image 8

问题：小明折了9只纸飞机，比小军少折3只，小军折了几只纸飞机？

思考过程：

Step	Content
Thinking Step 1:	小明折了9只纸飞机，比小军少折3只。这意味着小军折的纸飞机数量比小明多3只。为了找出小军折了多少只纸飞机，我们需要从小明折的纸飞机数量中加上这个差值。
Thinking Step 2:	根据题目，小明折了9只纸飞机，比小军少折3只。这意味着小军折的纸飞机数量比小明多3只。因此，我们可以用小明折的纸飞机数量加上3只来计算小军折的纸飞机数量。所以，小军折了9 + 3 = 12只纸飞机。
Final Answer	小军折了12只纸飞机。

最后的回答：

The word 'strawberry' contains three 'R's.

Case4: 一年级数学题（英文）

image 9

问题：There are 10 apples in the basket, one of which was eaten by Little Ding Ding and two by Dad. How many more are there now?

思考过程：

Step	Content
Thinking Step 1:	There are 10 apples in the basket. One apple was eaten by Little Ding Ding and two apples were eaten by Dad. To find out how many apples are left, I need to subtract the apples eaten from the total number of apples. This involves basic arithmetic operations.
Thinking Step 2:	Little Ding Ding ate 1 apple and Dad ate 2 apples. To find out how many apples are left, I need to subtract the total number of apples eaten from the initial number of apples. This involves basic arithmetic operations.
Thinking Step 3:	To find out how many apples are left, I need to subtract the total number of apples eaten from the initial number of apples. This involves basic arithmetic operations. After subtracting, I will provide the final answer.
Final Answer	There are 7 apples left in the basket.

最后的回答：

There are 7 apples left in the basket.

Case5: 三年级数学题（中文）

image 10

问题：商店有 4筐苹果,每筐 55 千克,已经卖出 135 千克,还剩多少千克苹果?

思考过程：

Step	Content
Thinking Step 1:	首先，我们需要确定商店总共有多少千克苹果。由于有4筐苹果，每筐55千克，我们可以计算出总重量。然后，我们需要从总重量中减去已经卖出的135千克，以找出剩余的苹果重量。
Thinking Step 2:	To find out how many kilograms of apples are left, we need to subtract the amount sold from the total amount available. The total amount available is 4 baskets * 55 kilograms/basket = 220 kilograms. After selling 135 kilograms, the remaining amount is 220 kilograms - 135 kilograms = 85 kilograms.
Final Answer	There are 85 kilograms of apples left.

最后的回答：

There are 85 kilograms of apples left.