为什么 Llama 3.1 405B 比 GPT-4o 和 Claude 3.5 Sonnet 好得多

最新推荐文章于 2024-09-13 21:31:56 发布

子然在打码

最新推荐文章于 2024-09-13 21:31:56 发布

阅读量99

点赞数

文章标签： llama 人工智能机器学习

原文链接：https://www.aisolink.com/why-llama-3-1-405b-is-so-much-better-than-gpt-4o-and-claude-3-5-sonnet

版权

原文：为什么 Llama 3.1 405B 比 GPT-4o 和 Claude 3.5 Sonnet 好得多 | 爱搜AI工具资源导航站

Llama 3.1 405B 是第一个开源模型，在常识、可操纵性、数学、工具使用和多语言翻译等功能方面与领先的专有人工智能模型相当。Meta宣布推出Llama 3.1，这是迄今为止最大的开源AI模型，并在多项基准测试中超越了OpenAI的GPT-4o和Anthropic的Claude 3.5 Sonnet！

在本分步指南中，我们将介绍 Llama 3.1 405B 是什么、如何在本地使用 Llama 3.1 405B 以及为什么 Llama 3.1 405B 比 GPT-4o 和 Claude 3.5 Sonnet 好得多。

什么是Llama 3.1 40 5B ？

Llama 3.1 405B 是 Meta 最大的模型，使用超过 15 万亿个代币进行训练。为此，Meta 优化了整个训练堆栈，并在超过 16,000 个 H100 GPU 上进行训练，使其成为第一个以这种规模训练的 Llama 模型。根据 Meta 的说法，这个版本的原始模型（Llama 1 和 Llama 2）具有 128K 上下文长度，改进了推理和编码能力。 Meta还升级了多语言8B和70B型号。

Llama 3.1 40 5B 具有许多吸引用户的特性和功能，例如：

RAG 和工具使用 – Meta 指出您可以使用 Llama 系统组件通过“零次工具使用”来扩展模型，并使用 RAG 构建代理行为。
多语言——Llama 3 天然支持多语言处理。预训练数据包含约50%的多语言token，可以处理和理解多种语言。
编程和推理——Llama 3 具有强大的编程能力，可以生成高质量的代码，并对语法和逻辑有很强的理解。它可以创建复杂的代码结构并在各种编程任务中表现良好。 Llama 3 擅长逻辑推理、解决问题、分析和推理。它可以有效地处理复杂的逻辑任务并解决复杂的问题。
多模态模型——多模态模型已经开发出来，支持图像识别、视频识别和语音理解功能，但这些模型仍在开发中，尚未广泛发布。

基准测试结果

Meta 将 Llama 3.1 405B 模型与 GPT-4、GPT-4o 和 Claude 3.5 sonnet 等模型进行了比较。结果显示，Llama 3.1 在数学推理、复杂推理、多语言支持等测试数据集上表现优于 GPT-4o 和 Claude 3.5 十四行诗，长文本处理能力也很出色，零滚动质量获得 95.2 分。

与Claude 3.5 sonnet 相比，其工具利用能力（BFCL、Nexus）有所不足。虽然在多任务语言理解、Human Eval、MATH等测试数据集上的表现略逊于闭源模型，但分数差异并不大。

另外，手动评测结果显示，Llama 3.1 405B模型的输出性能与GPT-4和Claude 3.5 Sonnet相当，略逊于GPT-4o。仅看这个基准分数，似乎很有希望。

基准测试结果表明，llama 3.1 405B是一个优秀的语言模型，具有强大的语言建模能力、数学推理能力、复杂推理能力和长文本处理能力。但在工具利用能力和多语言支持方面仍有提升空间。

现在我已经介绍了基准分数，我将尝试使用它们并看看它们的表现如何。

如何在本地使用Llama 3.1 40 5B？

Ollama 是启动和运行本地语言模型的最快方式。我们建议尝试 Llama 3.1 8b，它的大小令人印象深刻，并且在大多数硬件上都表现良好。

在此处下载 Ollama（它应该引导您完成其余步骤）打开终端并运行 ollama run llama3.1-8b。Groq 现在托管 Llama 3.1 型号，包括 70B 和 8B 型号。早些时候，他们提供了最大的405B型号，但由于高流量和服务器问题已暂时删除。与此同时，Llama 3.1 70B 和 8B 模型仍然可用，并且可以以每秒 250 个令牌的惊人速度生成响应。
前往 groq.com 并使用免费帐户登录。接下来，从下拉菜单中选择 llama 3.1 模型。

Llama 3.1 405B 对比 Claude 3.5 Sonnet 对比

我们来对比一下 Claude 3.5 Sonnet 和 GPT-4o，看看 Llama 3.1 405B 的性能如何。此次，我们将核实以下事项。

Coding

选择了一个与排序算法相关的中级问题。该问题名为“前 K 个频繁元素”，涉及在给定数据集中查找 K 个最频繁的元素进行测试。

Given an integer array nums and an integer k, return the k most
frequent elements. You may return the answer in any order.

Example 1
Input: nums = [1,1,1,2,2,3], k = 2 Output: [1,2] or [2,1]

Example 2
Input: nums = [1], k = 1 Output: [1]

Constr aints

1 <= nums.length <= 105
–104 <= nums[i] <= 104

Llama 3.1 405B

“`python
import heapq
from collections import Counter

def topKFrequent(nums, k):
“””
Returns the k most frequent elements in the given array.

Args:
nums (list): The input array of integers.
k (int): The number of most frequent elements to return.

Returns:
list: The k most frequent elements in any order.
“””
# Count the frequency of each element
count = Counter(nums)

# Use a heap to find the k most frequent elements
return heapq.nlargest(k, count.keys(), key=count.get)
”’
Here’s an explanation of the code:

1. We use the `Counter` class from the `collections` module to count the frequency of each element in the input array `nums`. This gives us a dictionary `count` where the keys are the unique elements in `nums` and the values are their frequencies.
2. We use the `heapq.nlargest` function to find the k most frequent elements. We pass `k` as the first argument, `count.keys()` as the second argument (which gives us an iterable of the unique elements in `nums`), and `key=count.get` as the third argument (which tells `heapq.nlargest` to use the frequency of each element as the key for comparison).
3. The `heapq.nlargest` function returns a list of the k most frequent elements in any order.

Example use cases:

“`python
print(topKFrequent([1,1,1,2,2,3], 2)) # [1, 2] or [2, 1]
print(topKFrequent([1], 1)) # [1]
“`

Note that this solution has a time complexity of O(n log k) where n is the length of the input array, because we use a heap to find the k most frequent elements. This is efficient for large inputs because the heap operations are logarithmic in the size of the heap.

GPT-4o

To solve the problem of finding the k most frequent elements in an integer array, you can use a combination of a dictionary to count the occurrences and a heap to keep track of the top k elements. Here’s a step-by-step approach:

Count the Frequencies: Use a dictionary to count the frequency of each element in the array.
Use a Heap for Top k Elements: Use a min-heap to keep track of the top k elements based on their frequencies. Python’s heapq library is useful for this purpose.
Extract the Elements from the Heap: After processing the array, the heap will cont ain the k most frequent elements.
Here’s the implementation of the above logic:

import heapq
from collections import Counter

def topKFrequent(nums, k):
# Step 1: Count the frequency of each element
count = Counter(nums)

# Step 2: Use a heap to keep track of the top k elements
heap = []

for num, freq in count.items():
heapq.heappush(heap, (freq, num))
if len(heap) > k:
heapq.heappop(heap)

# Step 3: Extract the elements from the heap
return [num for freq, num in heap]

# Example 1
nums1 = [1, 1, 1, 2, 2, 3]
k1 = 2
print(topKFrequent(nums1, k1)) # Output: [1, 2] or [2, 1]

# Example 2
nums2 = [1]
k2 = 1
print(topKFrequent(nums2, k2)) # Output: [1]

在 Visual Studio code 上运行代码时，所有三个解决方案都是正确的。然而，Llama3 的解决方案是最简洁的，因为它直接使用了 heapq.nlargest 。相比之下，GPT-4 和 Claude 3.5 Sonnet 手动管理堆。由于直接使用 heapq.nlargest ，Llama3 的解决方案可能稍微更高效、更简单。总的来说，每个解决方案都正确地解决了问题，但 Llama3 的方法因其简单性和直接使用 Python 堆函数而脱颖而出

简单而强大创意助手ChatGPT Prompts使用指南

Mathematics

然后我比较并验证数学能力，我们将让参与者回答概率问题，这是数据科学中的一个关键概念。

Alice has 2 kids and one of them is a girl. What is the probability that the
other child is also a girl?

You can assume that there are an equal number of males and females in the
world.

A) 0.5
B) 0.25
C) 0.333
D) 0.75

Llama 3.1 405B

This is a classic problem!

At first glance, it might seem like the probability is 50% (1/2), since the other child could be either a boy or a girl. However, this is actually a subtle mistake.

When we say “one of the children is a girl”, we‘re not specifying which child is the girl. This means that the girl could be either the first child or the second child.

Let‘s analyze the possible scenarios:

1. The first child is a girl, and the second child is a boy (GB).
2. The first child is a girl, and the second child is a girl (GG).
3. The first child is a boy, and the second child is a girl (BG).

Notice that we‘ve already accounted for the fact that one of the children is a girl. Now, we want to find the probability that the other child is also a girl.

Out of the three possible scenarios, only one (GG) has both children as girls. However, we need to consider that the girl could be either the first or the second child, so we have two possible ways to get the GG scenario: the first child is a girl and the second child is a girl, or the first child is a girl (but we don‘t know that yet) and the second child is a girl (which we do know).

So, when we say “one of the children is a girl”, we‘re essentially combining the GB and BG scenarios, as well as the GG scenario. This means we have a total of 3 possible scenarios, and 1 out of those 3 has both children as girls (GG).

However, since we have already established that one of the children is a girl, the sample space is reduced to the 3 scenarios mentioned earlier (GB, GG, and BG), with each scenario being equally likely. And out of those 3 scenarios, 1/3 of them have the other child also being a girl.

Therefore, the probability that the other child is also a girl is 1/3, or approximately 33.33%.

It‘s worth noting that this problem is often counterintuitive, and many people initially think the probability is 50%. However, by carefully analyzing the possible scenarios and considering the reduced sample space, we can see that the correct probability is indeed 1/3.

GPT-4o

to solve this problem, we need to analyze the possible scenarios and apply basic probability principles.

Given that Alice has two children and we know one of them is a girl, we want to determine the probability that the other child is also a girl.

List all possible combinations of two children:

Girl, Girl (GG)
Girl, Boy (GB)
Boy, Girl (BG)
Boy, Boy (BB)
Exclude the combinations that do not meet the given condition (one of the children is a girl):

The combination Boy, Boy (BB) does not meet the condition because neither child is a girl.
So, we are left with:

Girl, Girl (GG)
Girl, Boy (GB)
Boy, Girl (BG)
Determine the probability:

We now have three equally likely combinations: GG, GB, and BG.
In two of these combinations (GB and BG), the other child is a boy.
In one of these combinations (GG), the other child is a girl.

Thus, the probability that the other child is also a girl is:

Probability= Number of favorable outcomes /Total possible outcomes = 1/3

Therefore, the probability that the other child is also a girl is 1/3 or approximately 33.33%.

Claude 3.5 Sonnet

Let‘s approach this step-by-step:

First, let‘s consider all possible combinations of two children:

Boy-Boy (BB)
Boy-Girl (BG)
Girl-Boy (GB)
Girl-Girl (GG)

We‘re told that Alice has at least one girl. This eliminates the BB possibility.
So we‘re left with three equally likely possibilities:

Boy-Girl (BG)
Girl-Boy (GB)
Girl-Girl (GG)

Out of these three possibilities, only one (GG) satisfies the condition that the other child is also a girl.
Therefore, the probability is:
1 (favorable outcome) / 3 (total possible outcomes) = 1/3

Thus, the probability that Alice‘s other child is also a girl is 1/3 or about 33.33%.