一、前言
去年8月份Meta开源Code Llama 7B/13B/34B系列时，做过一个关于Code Llama和ChatGPT的Java编码能力测评。当时Code Llama使用的是34B Instruct版本，ChatGPT使用的是3.5版本，结果显示Code Llama 34B Instruct相比ChatGPT3.5，还是有一些差距，特别是中文理解方面不够理想。
到今年2月，几个月时间里，大模型如雨后春笋、层出不穷，随着版本迭代，智能水平快速发展。1月16日智谱发布GLM-4，1月29日Meta开源其代码大模型的最大参数版本Code Llama 70B，2月4日阿里发布QWen1.5。
在这些最新大模型中，本次选取了五个排名和评分靠前的大模型：EvalPlus排行榜HumanEval排名仅次于GPT4的代码大模型DeepSeek-Coder-33B-Instruct、最新发布的主流代码大模型CodeLlama-70B-Instruct、最近风头正盛的智谱GLM-4、大家使用最多的ChatGpt3.5、公认最强的ChatGPT4，来看下在Java编码能力方面表现如何，希望能对代码大模型的选型起到一定参考作用。
二、本次测评大模型简介
1、DeepSeek-Coder-33B-Instruct
DeepSeek是知名私募巨头幻方量化旗下的人工智能公司深度求索。据国盛证券研报，在云算力端，目前国内只有商汤、百度、腾讯、字节、阿里和幻方有超过1万张A100芯片储备，可见幻方的硬实力不容小觑。23年11月开源了代码大模型DeepSeek Coder，包括7B和33B，支持免费商用。在EvalPlus排行榜(https://evalplus.github.io/leaderboard.html)上，DeepSeek-Coder-33B-instruct在HumanEval paas@1评分75.0分，仅次于GPT-4-Turbo(Nov 2023)的81.7分和GPT-4(May 2023)的79.3分，超过GPT-3.5-Turbo (Nov 2023)的70.7分和GPT-3.5 (May 2023)的66.5分，位列第三名。
在线试用地址：https://chat.deepseek.com/coder
2、CodeLlama-70B-Instruct
1月29日Meta新开源的代码大模型CodeLlama-70B-Instruct，可以说从去年8月到现在，半年磨一剑。在EvalPlus排行榜(https://evalplus.github.io/leaderboard.html)上，最新的CodeLlama-70B-Instruct的HumanEval paas@1评分58.5分，低于GPT-3.5，相比CodeLlama-34B-Instruct进步不少。
在线试用地址：https://labs.perplexity.ai/
3、GLM-4
GLM-4是智谱1月16日新发布的第四个版本，前三个版本是开源可免费商用的，GLM-4目前没有。根据GLM-4发布会介绍，与ChatGLM3相比，GLM-4在综合能力上实现了全面跃升，性能提升了60%，号称已经逼近GPT-4，估计是要开始变现了。
在线试用地址：https://www.chatglm.cn/main/alltoolsdetail
4、ChatGPT
ChatGPT目前有4和3.5两个版本。GPT4是业界公认最强者，基本上是一直被模仿，从未被超越，但是一分钱一分货。本次测评使用的是GPT4是GPT-4-0613版本。GPT3.5是目前使用人数最多的，性价比高，智能方面表现一直也是靠前的，传说已经被很多竞争对手超越了。
在线试用地址：https://chat.openai.com/
三、Java编码能力测评
Linus说过：Talk is cheap. Show me your code。多说无益，我们直接上代码来测评下Java编码能力，按运行结果的正确性、生成代码的正确性、代码风格是否清晰简洁、代码的复用性、代码的注释可读性几个维度进行分析，使用100分制打分。
1、测评一：日期工具类代码生成
测评一来源于实际工作中遇到的一个开发场景，需要生成根据当前日期返回对应不同月度、年度范围的四个维度的一个工具类。
对应的提示词如下：
使用JDK8生成下面的四个方法，并提供main函数进行自测：
1）根据给定日期，返回当月的第一天和最后一天的日期，出参使用日期数组。
2）根据给定日期，返回去年当月的第一天和最后一天的日期，出参使用日期数组。
3）根据给定日期，返回当年的第一天和最后一天的日期，出参使用日期数组。
4）根据给定日期，返回去年的第一天和最后一天的日期，出参使用日期数组。
1）DeepSeek-Coder-33B-Instruct
生成的代码：
package ai.deepseek;
import java.time.LocalDate;
import java.time.YearMonth;
import java.time.temporal.TemporalAdjusters;
public class DateUtils {
public static LocalDate[] getCurrentMonthStartAndEnd(LocalDate date) {
YearMonth yearMonth = YearMonth.from(date);
LocalDate firstDayOfMonth = yearMonth.atDay(1);
LocalDate lastDayOfMonth = yearMonth.atEndOfMonth();
return new LocalDate[]{firstDayOfMonth, lastDayOfMonth};
}
public static LocalDate[] getLastYearSameMonthStartAndEnd(LocalDate date) {
LocalDate lastYearDate = date.minusYears(1);
return getCurrentMonthStartAndEnd(lastYearDate);
}
public static LocalDate[] getCurrentYearStartAndEnd(LocalDate date) {
LocalDate firstDayOfYear = date.with(TemporalAdjusters.firstDayOfYear());
LocalDate lastDayOfYear = date.with(TemporalAdjusters.lastDayOfYear());
return new LocalDate[]{firstDayOfYear, lastDayOfYear};
}
public static LocalDate[] getLastYearStartAndEnd(LocalDate date) {
LocalDate lastYearDate = date.minusYears(1);
return getCurrentYearStartAndEnd(lastYearDate);
}
public static void main(String[] args) {
LocalDate currentDate = LocalDate.now();
LocalDate[] currentMonthDates = getCurrentMonthStartAndEnd(currentDate);
System.out.println("Current month start: " + currentMonthDates[0]);
System.out.println("Current month end: " + currentMonthDates[1]);
LocalDate[] lastYearSameMonthDates = getLastYearSameMonthStartAndEnd(currentDate);
System.out.println("Last year same month start: " + lastYearSameMonthDates[0]);
System.out.println("Last year same month end: " + lastYearSameMonthDates[1]);
LocalDate[] currentYearDates = getCurrentYearStartAndEnd(currentDate);
System.out.println("Current year start: " + currentYearDates[0]);
System.out.println("Current year end: " + currentYearDates[1]);
LocalDate[] lastYearDates = getLastYearStartAndEnd(currentDate);
System.out.println("Last year start: " + lastYearDates[0]);
System.out.println("Last year end: " + lastYearDates[1]);
}
}
运行的结果正确。
Current month start: 2024-02-01
Current month end: 2024-02-29
Last year same month start: 2023-02-01
Last year same month end: 2023-02-28
Current year start: 2024-01-01
Current year end: 2024-12-31
Last year start: 2023-01-01
Last year end: 2023-12-31
测评分析：
加分项：生成的代码能正确编译；运行结果正确；在原方法基础上提炼出了可复用的方法；
减分项：方法命名有点中国式英文的风格，不够优雅；代码没有注释；
评分：80分。
2）CodeLlama-70B-Instruct
生成的代码：
package ai.codellama70b;
import java.time.LocalDate;
import java.time.Month;
import java.time.Year;
import java.time.YearMonth;
import java.time.temporal.ChronoUnit;
import java.time.temporal.TemporalAdjusters;
import java.util.Arrays;
public class DateUtils {
public static LocalDate[] getFirstAndLastDayOfMonth(LocalDate date) {
LocalDate firstDay = date.withDayOfMonth(1);
LocalDate lastDay = date.with(TemporalAdjusters.lastDayOfMonth());
return new LocalDate[]{firstDay, lastDay};
}
public static LocalDate[] getFirstAndLastDayOfLastYearMonth(LocalDate date) {
YearMonth yearMonth = YearMonth.from(date).minusYears(1);
LocalDate firstDay = yearMonth.atDay(1);
LocalDate lastDay = yearMonth.atEndOfMonth();
return new LocalDate[]{firstDay, lastDay};
}
public static LocalDate[] getFirstAndLastDayOfYear(LocalDate date) {
Year year = Year.from(date);
LocalDate firstDay = year.atDay(1);
LocalDate lastDay = year.atDay(year.length());
return new LocalDate[]{firstDay, lastDay};
}
public static LocalDate[] getFirstAndLastDayOfLastYear(LocalDate date) {
Year year = Year.from(date).minusYears(1);
LocalDate firstDay = year.atDay(1);
LocalDate lastDay = year.atDay(year.length());
return new LocalDate[]{firstDay, lastDay};
}
public static void main(String[] args) {
LocalDate date = LocalDate.now();
System.out.println("First and last day of month: " + Arrays.toString(getFirstAndLastDayOfMonth(date)));
System.out.println("First and last day of last year month: " + Arrays.toString(getFirstAndLastDayOfLastYearMonth(date)));
System.out.println("First and last day of year: " + Arrays.toString(getFirstAndLastDayOfYear(date)));
System.out.println("First and last day of last year: " + Arrays.toString(getFirstAndLastDayOfLastYear(date)));
}
}
运行结果正确。
First and last day of month: [2024-02-01, 2024-02-29]
First and last day of last year month: [2023-02-01, 2023-02-28]
First and last day of year: [2024-01-01, 2024-12-31]
First and last day of last year: [2023-01-01, 2023-12-31]
测评分析：
加分项：运行正确；方法命名较好；
减分项：生成的代码需要增加"import java.time.temporal.TemporalAdjusters;"才能编译通过；代码没有做到复用；代码没有注释；
评分：60分
3）GLM-4
生成的代码：
package ai.glm4;
import java.time.LocalDate;
public class DateUtils {
public static void main(String[] args) {
LocalDate currentDate = LocalDate.now(); // 当前日期
System.out.println("当前日期: " + currentDate);
// 测试第一个方法
LocalDate[] currentMonthDates = getFirstAndLastDayOfCurrentMonth(currentDate);
System.out.println("当月第一天: " + currentMonthDates[0] + ", 当月最后一天: " + currentMonthDates[1]);
// 测试第二个方法
LocalDate[] lastYearCurrentMonthDates = getFirstAndLastDayOfLastYearMonth(currentDate);
System.out.println("去年当月第一天: " + lastYearCurrentMonthDates[0] + ", 去年当月最后一天: " + lastYearCurrentMonthDates[1]);
// 测试第三个方法
LocalDate[] currentYearDates = getFirstAndLastDayOfCurrentYear(currentDate);
System.out.println("当年第一天: " + currentYearDates[0] + ", 当年最后一天: " + currentYearDates[1]);
// 测试第四个方法
LocalDate[] lastYearDates = getFirstAndLastDayOfLastYear(currentDate);
System.out.println("去年第一天: " + lastYearDates[0] + ", 去年最后一天: " + lastYearDates[1]);
}
// 方法1：根据给定日期，返回当月的第一天和最后一天的日期
public static LocalDate[] getFirstAndLastDayOfCurrentMonth(LocalDate date) {
LocalDate firstDayOfMonth = date.withDayOfMonth(1);
LocalDate lastDayOfMonth = date.withDayOfMonth(date.lengthOfMonth());
return new LocalDate[]{firstDayOfMonth, lastDayOfMonth};
}
// 方法2：根据给定日期，返回去年当月的第一天和最后一天的日期
public static LocalDate[] getFirstAndLastDayOfLastYearMonth(LocalDate date) {
LocalDate firstDayOfLastYearMonth = date.minusYears(1).withDayOfMonth(1);
LocalDate lastDayOfLastYearMonth = firstDayOfLastYearMonth.withDayOfMonth(firstDayOfLastYearMonth.lengthOfMonth());
return new LocalDate[]{firstDayOfLastYearMonth, lastDayOfLastYearMonth};
}
// 方法3：根据给定日期，返回当年的第一天和最后一天的日期
public static LocalDate[] getFirstAndLastDayOfCurrentYear(LocalDate date) {
LocalDate firstDayOfYear = date.withDayOfYear(1);
LocalDate lastDayOfYear = date.withDayOfYear(date.lengthOfYear());
return new LocalDate[]{firstDayOfYear, lastDayOfYear};
}
// 方法4：根据给定日期，返回去年的第一天和最后一天的日期
public static LocalDate[] getFirstAndLastDayOfLastYear(LocalDate date) {
LocalDate firstDayOfLastYear = date.minusYears(1).withDayOfYear(1);
LocalDate lastDayOfLastYear = firstDayOfLastYear.withDayOfYear(firstDayOfLastYear.lengthOfYear());
return new LocalDate[]{firstDayOfLastYear, lastDayOfLastYear};
}
}
运行结果正确。
当前日期: 2024-02-03
当月第一天: 2024-02-01, 当月最后一天: 2024-02-29
去年当月第一天: 2023-02-01, 去年当月最后一天: 2023-02-28
当年第一天: 2024-01-01, 当年最后一天: 2024-12-31
去年第一天: 2023-01-01, 去年最后一天: 2023-12-31
测评分析：
加分项：代码编译正确；运行结果正确；方法命名较好；提供了清晰的中文注释；
减分项：没有实现复用；
评分：90分。
4）GPT-3.5
生成的代码：
package ai.gpt35;
import java.time.LocalDate;
public class DateRangeCalculator {
public static void main(String[] args) {
LocalDate currentDate = LocalDate.now();
// Test case 1
LocalDate[] currentMonthRange = getMonthRange(currentDate);
System.out.println("Current Month Range: " + currentMonthRange[0] + " to " + currentMonthRange[1]);
// Test case 2
LocalDate[] lastYearMonthRange = getLastYearMonthRange(currentDate);
System.out.println("Last Year's Current Month Range: " + lastYearMonthRange[0] + " to " + lastYearMonthRange[1]);
// Test case 3
LocalDate[] currentYearRange = getYearRange(currentDate);
System.out.println("Current Year Range: " + currentYearRange[0] + " to " + currentYearRange[1]);
// Test case 4
LocalDate[] lastYearRange = getLastYearRange(currentDate);
System.out.println("Last Year Range: " + lastYearRange[0] + " to " + lastYearRange[1]);
}
// Method 1: Get current month's first and last day
public static LocalDate[] getMonthRange(LocalDate date) {
LocalDate firstDay = date.withDayOfMonth(1);
LocalDate lastDay = date.withDayOfMonth(date.lengthOfMonth());
return new LocalDate[]{firstDay, lastDay};
}
// Method 2: Get last year's current month's first and last day
public static LocalDate[] getLastYearMonthRange(LocalDate date) {
LocalDate lastYearDate = date.minusYears(1);
return getMonthRange(lastYearDate);
}
// Method 3: Get current year's first and last day
public static LocalDate[] getYearRange(LocalDate date) {
LocalDate firstDay = date.withDayOfYear(1);
LocalDate lastDay = date.withDayOfYear(date.lengthOfYear());
return new LocalDate[]{firstDay, lastDay};
}
// Method 4: Get last year's first and last day
public static LocalDate[] getLastYearRange(LocalDate date) {
LocalDate lastYearDate = date.minusYears(1);
return getYearRange(lastYearDate);
}
}
运行结果正确。
Current Month Range: 2024-02-01 to 2024-02-29
Last Year's Current Month Range: 2023-02-01 to 2023-02-28
Current Year Range: 2024-01-01 to 2024-12-31
Last Year Range: 2023-01-01 to 2023-12-31
测评分析结果：
加分项：编译正确；运行结果正确；类命名比较好；抽象了公共方法，实现代码复用；提供了注释；
减分项：方法命名有点中式英文；只提供了英文注释；
评分：90分。
5）GPT-4-0613
生成的代码：
package ai.gpt4;
import java.text.SimpleDateFormat;
import java.util.Calendar;
import java.util.Date;
public class DateUtils {
// 方法1：返回当月的第一天和最后一天的日期
public static Date[] getFirstAndLastDayOfMonth(Date date) {
Calendar calendar = Calendar.getInstance();
calendar.setTime(date);
calendar.set(Calendar.DAY_OF_MONTH, 1);
Date firstDayOfMonth = calendar.getTime();
calendar.add(Calendar.MONTH, 1);
calendar.add(Calendar.DAY_OF_MONTH, -1);
Date lastDayOfMonth = calendar.getTime();
return new Date[]{firstDayOfMonth, lastDayOfMonth};
}
// 方法2：返回去年当月的第一天和最后一天的日期
public static Date[] getFirstAndLastDayOfLastYearMonth(Date date) {
Calendar calendar = Calendar.getInstance();
calendar.setTime(date);
calendar.add(Calendar.YEAR, -1);
calendar.set(Calendar.DAY_OF_MONTH, 1);
Date firstDayOfLastYearMonth = calendar.getTime();
calendar.add(Calendar.MONTH, 1);
calendar.add(Calendar.DAY_OF_MONTH, -1);
Date lastDayOfLastYearMonth = calendar.getTime();
return new Date[]{firstDayOfLastYearMonth, lastDayOfLastYearMonth};
}
// 方法3：返回当年的第一天和最后一天的日期
public static Date[] getFirstAndLastDayOfYear(Date date) {
Calendar calendar = Calendar.getInstance();
calendar.setTime(date);
calendar.set(Calendar.DAY_OF_YEAR, 1);
Date firstDayOfYear = calendar.getTime();
calendar.set(Calendar.DAY_OF_YEAR, calendar.getActualMaximum(Calendar.DAY_OF_YEAR));
Date lastDayOfYear = calendar.getTime();
return new Date[]{firstDayOfYear, lastDayOfYear};
}
// 方法4：返回去年的第一天和最后一天的日期
public static Date[] getFirstAndLastDayOfLastYear(Date date) {
Calendar calendar = Calendar.getInstance();
calendar.setTime(date);
calendar.add(Calendar.YEAR, -1);
calendar.set(Calendar.DAY_OF_YEAR, 1);
Date firstDayOfLastYear = calendar.getTime();
calendar.set(Calendar.DAY_OF_YEAR, calendar.getActualMaximum(Calendar.DAY_OF_YEAR));
Date lastDayOfLastYear = calendar.getTime();
return new Date[]{firstDayOfLastYear, lastDayOfLastYear};
}
// 主函数用于自测
public static void main(String[] args) {
Date today = new Date();
Date[] monthDates = getFirstAndLastDayOfMonth(today);
System.out.println("当月的第一天: " + monthDates[0]);
System.out.println("当月的最后一天: " + monthDates[1]);
Date[] lastYearMonthDates = getFirstAndLastDayOfLastYearMonth(today);
System.out.println("去年当月的第一天: " + lastYearMonthDates[0]);
System.out.println("去年当月的最后一天: " + lastYearMonthDates[1]);
Date[] yearDates = getFirstAndLastDayOfYear(today);
System.out.println("当年的第一天: " + yearDates[0]);
System.out.println("当年的最后一天: " + yearDates[1]);
Date[] lastYearDates = getFirstAndLastDayOfLastYear(today);
System.out.println("去年的第一天: " + lastYearDates[0]);
System.out.println("去年的最后一天: " + lastYearDates[1]);
}
}
运行结果日期正确，但加上了运行时的时分秒，有点画蛇添足。
当月的第一天: Thu Feb 01 20:58:37 CST 2024
当月的最后一天: Thu Feb 29 20:58:37 CST 2024
去年当月的第一天: Wed Feb 01 20:58:37 CST 2023
去年当月的最后一天: Tue Feb 28 20:58:37 CST 2023
当年的第一天: Mon Jan 01 20:58:37 CST 2024
当年的最后一天: Tue Dec 31 20:58:37 CST 2024
去年的第一天: Sun Jan 01 20:58:37 CST 2023
去年的最后一天: Sun Dec 31 20:58:37 CST 2023
测评分析结果：
加分项：编译正确；运行结果正确；方法命名比较好；提供了中文注释；
减分项：没有代码复用；结果加上了运行时的时分秒，有点画蛇添足；
评分：85分。
2、测评二：报表数据填充代码生成
测评二也是来源于实际工作中的一个需求场景。报表数据的填充补全，入参是只有部分日期的当月报表数据，需要补充当月其它日期的数据记录，输出完整的当月数据记录。
对应的提示词如下：
有一个对象列表，List<TendencyChartVO>，TendencyChartVO对象有两个字段:dt,name, 其中有一个dt字段，为日期类型。name为String类型。
对象列表里可能只有当月其中几天的数据。帮我写一个函数，填充当月剩余的日期数据，只填充dt字段值就可以，填充好以后对List<TendencyChartVO>按日期排序。同时提供main函数进行自测。
1）DeepSeek-Coder-33B-Instruct
生成的代码：
package ai.deepseek;
import java.time.LocalDate;
import java.time.YearMonth;
import java.time.ZoneId;
import java.util.ArrayList;
import java.util.Comparator;
import java.util.Date;
import java.util.List;
public class TendencyChartVO {
private Date dt;
private String name;
public TendencyChartVO(Date dt, String name) {
this.dt = dt;
this.name = name;
}
public Date getDt() {
return dt;
}
public String getName() {
return name;
}
@Override
public String toString() {
return "TendencyChartVO{" +
"dt=" + dt +
", name='" + name + '\'' +
'}';
}
public static List<TendencyChartVO> fillAndSort(List<TendencyChartVO> list) {
// 获取当前日期
LocalDate currentDate = LocalDate.now();
// 获取当前月份的第一天和最后一天
LocalDate startOfMonth = currentDate.withDayOfMonth(1);
LocalDate endOfMonth = YearMonth.from(currentDate).atEndOfMonth();
// 创建一个新的列表来存储填充后的数据
List<TendencyChartVO> filledList = new ArrayList<>(list);
// 遍历当月的每一天
for (LocalDate date = startOfMonth; !date.isAfter(endOfMonth); date = date.plusDays(1)) {
// 将LocalDate转换为Date
Date dt = Date.from(date.atStartOfDay(ZoneId.systemDefault()).toInstant());
// 检查是否在原始列表中存在
LocalDate finalDate = date;
if (list.stream().noneMatch(vo -> convertToLocalDate(vo.getDt()).equals(finalDate))) {
// 如果不存在，创建一个新的TendencyChartVO对象并添加到列表中
filledList.add(new TendencyChartVO(dt, ""));
}
}
// 对列表进行排序
filledList.sort(Comparator.comparing(vo -> convertToLocalDate(vo.getDt())));
return filledList;
}
private static LocalDate convertToLocalDate(Date date) {
return date.toInstant().atZone(ZoneId.systemDefault()).toLocalDate();
}
public static void main(String[] args) {
// 创建一个示例列表
List<TendencyChartVO> list = new ArrayList<>();
list.add(new TendencyChartVO(Date.from(LocalDate.now().minusDays(5).atStartOfDay(ZoneId.systemDefault()).toInstant()), "Test1"));
list.add(new TendencyChartVO(Date.from(LocalDate.now().minusDays(2).atStartOfDay(ZoneId.systemDefault()).toInstant()), "Test2"));
// 填充并排序列表
List<TendencyChartVO> filledList = fillAndSort(list);
// 打印结果
filledList.forEach(System.out::println);
}
}
运行结果不正确，需要把minusDays改成plusDays后正确。
TendencyChartVO{dt=Mon Jan 29 00:00:00 CST 2024, name='Test1'}
TendencyChartVO{dt=Thu Feb 01 00:00:00 CST 2024, name='Test2'}
TendencyChartVO{dt=Fri Feb 02 00:00:00 CST 2024, name=''}
TendencyChartVO{dt=Sat Feb 03 00:00:00 CST 2024, name=''}
TendencyChartVO{dt=Sun Feb 04 00:00:00 CST 2024, name=''}
TendencyChartVO{dt=Mon Feb 05 00:00:00 CST 2024, name=''}
TendencyChartVO{dt=Tue Feb 06 00:00:00 CST 2024, name=''}
TendencyChartVO{dt=Wed Feb 07 00:00:00 CST 2024, name=''}
TendencyChartVO{dt=Thu Feb 08 00:00:00 CST 2024, name=''}
TendencyChartVO{dt=Fri Feb 09 00:00:00 CST 2024, name=''}
TendencyChartVO{dt=Sat Feb 10 00:00:00 CST 2024, name=''}
TendencyChartVO{dt=Sun Feb 11 00:00:00 CST 2024, name=''}
TendencyChartVO{dt=Mon Feb 12 00:00:00 CST 2024, name=''}
TendencyChartVO{dt=Tue Feb 13 00:00:00 CST 2024, name=''}
TendencyChartVO{dt=Wed Feb 14 00:00:00 CST 2024, name=''}
TendencyChartVO{dt=Thu Feb 15 00:00:00 CST 2024, name=''}
TendencyChartVO{dt=Fri Feb 16 00:00:00 CST 2024, name=''}
TendencyChartVO{dt=Sat Feb 17 00:00:00 CST 2024, name=''}
TendencyChartVO{dt=Sun Feb 18 00:00:00 CST 2024, name=''}
TendencyChartVO{dt=Mon Feb 19 00:00:00 CST 2024, name=''}
TendencyChartVO{dt=Tue Feb 20 00:00:00 CST 2024, name=''}
TendencyChartVO{dt=Wed Feb 21 00:00:00 CST 2024, name=''}
TendencyChartVO{dt=Thu Feb 22 00:00:00 CST 2024, name=''}
TendencyChartVO{dt=Fri Feb 23 00:00:00 CST 2024, name=''}
TendencyChartVO{dt=Sat Feb 24 00:00:00 CST 2024, name=''}
TendencyChartVO{dt=Sun Feb 25 00:00:00 CST 2024, name=''}
TendencyChartVO{dt=Mon Feb 26 00:00:00 CST 2024, name=''}
TendencyChartVO{dt=Tue Feb 27 00:00:00 CST 2024, name=''}
TendencyChartVO{dt=Wed Feb 28 00:00:00 CST 2024, name=''}
TendencyChartVO{dt=Thu Feb 29 00:00:00 CST 2024, name=''}
测评分析结果：
加分项：提供了比较清晰的中文注释；
减分项：编译有一处错误，51行的date需要改成final类型；运行结果不正确，需要把70-71行的minusDays方法改成plusDays才正确；方法内直接用了当前日期，可扩展性不好；
评分：50分。
2）CodeLlama-70B-Instruct
生成的代码：
package ai.codellama70b;
import java.time.LocalDate;
import java.time.format.DateTimeFormatter;
import java.util.ArrayList;
import java.util.Collections;
import java.util.Comparator;
import java.util.List;
public class TendencyChartVO {
private String dt;
private String name;
public TendencyChartVO(String dt, String name) {
this.dt = dt;
this.name = name;
}
public String getDt() {
return dt;
}
public void setDt(String dt) {
this.dt = dt;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public static void main(String[] args) {
List<TendencyChartVO> list = new ArrayList<>();
list.add(new TendencyChartVO("2021-01-01", "name1"));
list.add(new TendencyChartVO("2021-01-03", "name2"));
list.add(new TendencyChartVO("2021-01-05", "name3"));
list.add(new TendencyChartVO("2021-01-07", "name4"));
list.add(new TendencyChartVO("2021-01-09", "name5"));
list.add(new TendencyChartVO("2021-01-11", "name6"));
list.add(new TendencyChartVO("2021-01-13", "name7"));
list.add(new TendencyChartVO("2021-01-15", "name8"));
list.add(new TendencyChartVO("2021-01-17", "name9"));
list.add(new TendencyChartVO("2021-01-19", "name10"));
list.add(new TendencyChartVO("2021-01-21", "name11"));
list.add(new TendencyChartVO("2021-01-23", "name12"));
list.add(new TendencyChartVO("2021-01-25", "name13"));
list.add(new TendencyChartVO("2021-01-27", "name14"));
list.add(new TendencyChartVO("2021-01-29", "name15"));
list.add(new TendencyChartVO("2021-01-31", "name16"));
fillMissingDates(list);
System.out.println("List after filling missing dates:");
for (TendencyChartVO vo : list) {
System.out.println(vo.getDt() + " " + vo.getName());
}
}
private static void fillMissingDates(List<TendencyChartVO> list) {
LocalDate startDate = LocalDate.parse(list.get(0).getDt(), DateTimeFormatter.ofPattern("yyyy-MM-dd"));
LocalDate endDate = LocalDate.parse(list.get(list.size() - 1).getDt(), DateTimeFormatter.ofPattern("yyyy-MM-dd"));
for (LocalDate date = startDate; date.isBefore(endDate.plusDays(1)); date = date.plusDays(1)) {
boolean found = false;
for (TendencyChartVO vo : list) {
if (date.equals(LocalDate.parse(vo.getDt(), DateTimeFormatter.ofPattern("yyyy-MM-dd")))) {
found = true;
break;
}
}
if (!found) {
list.add(new TendencyChartVO(date.format(DateTimeFormatter.ofPattern("yyyy-MM-dd")), null));
}
}
Collections.sort(list, Comparator.comparing(TendencyChartVO::getDt));
}
}
运行结果正确。
List after filling missing dates:
2021-01-01 name1
2021-01-02 null
2021-01-03 name2
2021-01-04 null
2021-01-05 name3
2021-01-06 null
2021-01-07 name4
2021-01-08 null
2021-01-09 name5
2021-01-10 null
2021-01-11 name6
2021-01-12 null
2021-01-13 name7
2021-01-14 null
2021-01-15 name8
2021-01-16 null
2021-01-17 name9
2021-01-18 null
2021-01-19 name10
2021-01-20 null
2021-01-21 name11
2021-01-22 null
2021-01-23 name12
2021-01-24 null
2021-01-25 name13
2021-01-26 null
2021-01-27 name14
2021-01-28 null
2021-01-29 name15
2021-01-30 null
2021-01-31 name16
测评分析结果：
加分项：编译正确；运行结果正确；方法命名比较好；
减分项：代码不够简洁；没有注释；
评分：80分。
3）GLM-4
生成的代码：
package ai.glm4;
import java.text.SimpleDateFormat;
import java.util.ArrayList;
import java.util.Calendar;
import java.util.Date;
import java.util.List;
public class TendencyChartFiller {
static class TendencyChartVO {
Date dt;
String name;
public TendencyChartVO(Date dt, String name) {
this.dt = dt;
this.name = name;
}
@Override
public String toString() {
SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd");
return "TendencyChartVO{" +
"dt=" + sdf.format(dt) +
", name='" + name + '\'' +
'}';
}
}
public static void fillMissingDates(List<TendencyChartVO> list) {
SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd");
Calendar calendar = Calendar.getInstance();
calendar.setTime(list.get(0).dt);
int daysInMonth = calendar.getActualMaximum(Calendar.DAY_OF_MONTH);
for (int i = 1; i <= daysInMonth; i++) {
Date currentDate = strToDate(sdf.format(calendar.getTime()));
boolean found = list.stream().anyMatch(vo -> sdf.format(vo.dt).equals(sdf.format(currentDate)));
if (!found) {
list.add(new TendencyChartVO(currentDate, ""));
}
calendar.add(Calendar.DAY_OF_MONTH, 1);
}
list.sort((o1, o2) -> o1.dt.compareTo(o2.dt));
}
public static Date strToDate(String str) {
SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd");
try {
return sdf.parse(str);
} catch (Exception e) {
e.printStackTrace();
return null;
}
}
public static void main(String[] args) {
List<TendencyChartVO> list = new ArrayList<>();
list.add(new TendencyChartVO(strToDate("2024-02-01"), "name1"));
list.add(new TendencyChartVO(strToDate("2024-02-03"), "name2"));
list.add(new TendencyChartVO(strToDate("2024-02-05"), "name3"));
fillMissingDates(list);
for (TendencyChartVO vo : list) {
System.out.println(vo);
}
}
}
运行结果正确。
TendencyChartVO{dt=2024-02-01, name='name1'}
TendencyChartVO{dt=2024-02-02, name=''}
TendencyChartVO{dt=2024-02-03, name='name2'}
TendencyChartVO{dt=2024-02-04, name=''}
TendencyChartVO{dt=2024-02-05, name='name3'}
TendencyChartVO{dt=2024-02-06, name=''}
TendencyChartVO{dt=2024-02-07, name=''}
TendencyChartVO{dt=2024-02-08, name=''}
TendencyChartVO{dt=2024-02-09, name=''}
TendencyChartVO{dt=2024-02-10, name=''}
TendencyChartVO{dt=2024-02-11, name=''}
TendencyChartVO{dt=2024-02-12, name=''}
TendencyChartVO{dt=2024-02-13, name=''}
TendencyChartVO{dt=2024-02-14, name=''}
TendencyChartVO{dt=2024-02-15, name=''}
TendencyChartVO{dt=2024-02-16, name=''}
TendencyChartVO{dt=2024-02-17, name=''}
TendencyChartVO{dt=2024-02-18, name=''}
TendencyChartVO{dt=2024-02-19, name=''}
TendencyChartVO{dt=2024-02-20, name=''}
TendencyChartVO{dt=2024-02-21, name=''}
TendencyChartVO{dt=2024-02-22, name=''}
TendencyChartVO{dt=2024-02-23, name=''}
TendencyChartVO{dt=2024-02-24, name=''}
TendencyChartVO{dt=2024-02-25, name=''}
TendencyChartVO{dt=2024-02-26, name=''}
TendencyChartVO{dt=2024-02-27, name=''}
TendencyChartVO{dt=2024-02-28, name=''}
TendencyChartVO{dt=2024-02-29, name=''}
测评分析结果：
加分项：编译正确；运行结果正确；类和方法命名比较好；
减分项：没有注释；
评分：85分。
4）GPT-3.5
生成的代码：
package ai.gpt35;
import java.text.SimpleDateFormat;
import java.util.ArrayList;
import java.util.Calendar;
import java.util.Collections;
import java.util.Comparator;
import java.util.Date;
import java.util.List;
public class TendencyChartFiller {
public static void main(String[] args) {
// Sample data with incomplete dates
List<TendencyChartVO> tendencyChartList = new ArrayList<>();
tendencyChartList.add(new TendencyChartVO(parseDate("2024-02-01"), "Data1"));
tendencyChartList.add(new TendencyChartVO(parseDate("2024-02-03"), "Data2"));
// Fill and sort
fillAndSortTendencyChartList(tendencyChartList);
// Print result
for (TendencyChartVO vo : tendencyChartList) {
System.out.println("Date: " + formatDate(vo.getDt()) + ", Name: " + vo.getName());
}
}
public static void fillAndSortTendencyChartList(List<TendencyChartVO> list) {
// Get the first and last day of the current month
Calendar calendar = Calendar.getInstance();
calendar.set(Calendar.DAY_OF_MONTH, 1);
Date firstDay = calendar.getTime();
calendar.add(Calendar.MONTH, 1);
calendar.add(Calendar.DAY_OF_MONTH, -1);
Date lastDay = calendar.getTime();
// Fill missing dates in the list
while (firstDay.before(lastDay)) {
boolean containsDate = false;
for (TendencyChartVO vo : list) {
if (vo.getDt().equals(firstDay)) {
containsDate = true;
break;
}
}
if (!containsDate) {
list.add(new TendencyChartVO(new Date(firstDay.getTime()), null));
}
calendar.setTime(firstDay);
calendar.add(Calendar.DAY_OF_MONTH, 1);
firstDay = calendar.getTime();
}
// Sort the list by date
Collections.sort(list, Comparator.comparing(TendencyChartVO::getDt));
}
public static Date parseDate(String dateString) {
try {
SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd");
return sdf.parse(dateString);
} catch (Exception e) {
e.printStackTrace();
return null;
}
}
public static String formatDate(Date date) {
SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd");
return sdf.format(date);
}
}
class TendencyChartVO {
private Date dt;
private String name;
public TendencyChartVO(Date dt, String name) {
this.dt = dt;
this.name = name;
}
public Date getDt() {
return dt;
}
public String getName() {
return name;
}
}
运行结果运行结果不正确，2024-02-01多了一条，2024-2-29少了一条。
Date: 2024-02-01, Name: Data1
Date: 2024-02-01, Name: null
Date: 2024-02-02, Name: null
Date: 2024-02-03, Name: Data2
Date: 2024-02-03, Name: null
Date: 2024-02-04, Name: null
Date: 2024-02-05, Name: null
Date: 2024-02-06, Name: null
Date: 2024-02-07, Name: null
Date: 2024-02-08, Name: null
Date: 2024-02-09, Name: null
Date: 2024-02-10, Name: null
Date: 2024-02-11, Name: null
Date: 2024-02-12, Name: null
Date: 2024-02-13, Name: null
Date: 2024-02-14, Name: null
Date: 2024-02-15, Name: null
Date: 2024-02-16, Name: null
Date: 2024-02-17, Name: null
Date: 2024-02-18, Name: null
Date: 2024-02-19, Name: null
Date: 2024-02-20, Name: null
Date: 2024-02-21, Name: null
Date: 2024-02-22, Name: null
Date: 2024-02-23, Name: null
Date: 2024-02-24, Name: null
Date: 2024-02-25, Name: null
Date: 2024-02-26, Name: null
Date: 2024-02-27, Name: null
Date: 2024-02-28, Name: null
测评分析结果：
加分项：编译正确；有英文注释；
减分项：运行结果不正确，2024-02-01多了一条，2024-2-29少了一条；代码不够清晰简洁；
评分：50分。
5）GPT-4-0613
生成的代码：
package ai.gpt4;
import java.time.LocalDate;
import java.time.temporal.TemporalAdjusters;
import java.util.ArrayList;
import java.util.Collections;
import java.util.Comparator;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
class TendencyChartVO {
LocalDate dt;
String name;
public TendencyChartVO(LocalDate dt, String name) {
this.dt = dt;
this.name = name;
}
// Getter 和 Setter 方法省略...
@Override
public String toString() {
return "TendencyChartVO{" + "dt=" + dt + ", name='" + name + '\'' + '}';
}
public static void main(String[] args) {
List<TendencyChartVO> list = new ArrayList<>();
list.add(new TendencyChartVO(LocalDate.of(2024, 2, 5), "数据1"));
list.add(new TendencyChartVO(LocalDate.of(2024, 2, 15), "数据2"));
List<TendencyChartVO> filledList = fillDatesAndSort(list, 2024, 2);
for (TendencyChartVO item : filledList) {
System.out.println(item);
}
}
private static List<TendencyChartVO> fillDatesAndSort(List<TendencyChartVO> list, int year, int month) {
Set<LocalDate> existingDates = new HashSet<>();
for (TendencyChartVO item : list) {
existingDates.add(item.dt);
}
LocalDate firstDayOfMonth = LocalDate.of(year, month, 1);
LocalDate lastDayOfMonth = firstDayOfMonth.with(TemporalAdjusters.lastDayOfMonth());
for (LocalDate date = firstDayOfMonth; !date.isAfter(lastDayOfMonth); date = date.plusDays(1)) {
if (!existingDates.contains(date)) {
list.add(new TendencyChartVO(date, null));
}
}
Collections.sort(list, new Comparator<TendencyChartVO>() {
@Override
public int compare(TendencyChartVO o1, TendencyChartVO o2) {
return o1.dt.compareTo(o2.dt);
}
});
return list;
}
}
运行结果正确。
TendencyChartVO{dt=2024-02-01, name='null'}
TendencyChartVO{dt=2024-02-02, name='null'}
TendencyChartVO{dt=2024-02-03, name='null'}
TendencyChartVO{dt=2024-02-04, name='null'}
TendencyChartVO{dt=2024-02-05, name='数据1'}
TendencyChartVO{dt=2024-02-06, name='null'}
TendencyChartVO{dt=2024-02-07, name='null'}
TendencyChartVO{dt=2024-02-08, name='null'}
TendencyChartVO{dt=2024-02-09, name='null'}
TendencyChartVO{dt=2024-02-10, name='null'}
TendencyChartVO{dt=2024-02-11, name='null'}
TendencyChartVO{dt=2024-02-12, name='null'}
TendencyChartVO{dt=2024-02-13, name='null'}
TendencyChartVO{dt=2024-02-14, name='null'}
TendencyChartVO{dt=2024-02-15, name='数据2'}
TendencyChartVO{dt=2024-02-16, name='null'}
TendencyChartVO{dt=2024-02-17, name='null'}
TendencyChartVO{dt=2024-02-18, name='null'}
TendencyChartVO{dt=2024-02-19, name='null'}
TendencyChartVO{dt=2024-02-20, name='null'}
TendencyChartVO{dt=2024-02-21, name='null'}
TendencyChartVO{dt=2024-02-22, name='null'}
TendencyChartVO{dt=2024-02-23, name='null'}
TendencyChartVO{dt=2024-02-24, name='null'}
TendencyChartVO{dt=2024-02-25, name='null'}
TendencyChartVO{dt=2024-02-26, name='null'}
TendencyChartVO{dt=2024-02-27, name='null'}
TendencyChartVO{dt=2024-02-28, name='null'}
TendencyChartVO{dt=2024-02-29, name='null'}
测评分析结果：
加分项：编译正确；运行结果正确；代码清晰；方法命名比较好；
减分项：没有注释；方法的第二、三入参有点多余；
评分：85分。
3、测评三：代码补全
测评三是需要实现一个字符串数据处理方法，根据注释里的实现步骤，进行代码补全。
提示词如下：
你是一个优秀的java开发工程师，致力于写出高质量的代码。现在给你一个代码类，根据代码骨架和注释补全生成完整的代码。
要求：
1.保证代码的准确性，可读性，可维护性；
2.使用子函数封装，每个方法尽量不超过50行；
3.在驼峰格式转成下划线格式的方法中，打印出转成下划线格式后的那个值；
4.包括main函数，可以进行自测；
public class Transformer {
public String transform(String source) {
//1.对source实现对驼峰格式转成下划线格式的功能
//2.再进行MD5摘要
//3.最后加上当前时间戳，以$符号连接。
}
}
1）DeepSeek-Coder-33B-Instruct
生成的代码：
package ai.deepseek;
import org.apache.commons.codec.digest.DigestUtils;
import org.apache.commons.lang3.StringUtils;
import java.time.Instant;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
public class Transformer {
public String transform(String source) {
// 1. 对source实现对驼峰格式转成下划线格式的功能
String underlineFormat = camelToUnderline(source);
System.out.println("Underline format: " + underlineFormat);
// 2. 再进行MD5摘要
String md5Digest = DigestUtils.md5Hex(underlineFormat);
// 3. 最后加上当前时间戳，以$符号连接
long timestamp = Instant.now().toEpochMilli();
return md5Digest + "$" + timestamp;
}
private String camelToUnderline(String source) {
// 使用正则表达式匹配驼峰格式，并替换为下划线格式
Matcher matcher = Pattern.compile("[A-Z]").matcher(source);
StringBuffer sb = new StringBuffer(source);
if (matcher.find()) {
sb = new StringBuffer();
matcher.appendReplacement(sb, "_" + matcher.group(0).toLowerCase());
matcher.appendTail(sb);
} else {
return source;
}
return camelToUnderline(sb.toString());
}
public static void main(String[] args) {
Transformer transformer = new Transformer();
String source = "camelCaseExample";
String transformed = transformer.transform(source);
System.out.println("Transformed: " + transformed);
}
}
运行结果正确。
Underline format: camel_case_example
Transformed: fd7cbc4baf7cc4d892d5475c85308d4d$1707009362279
测评分析结果：
加分项：编译正确；运行结果正确；代码清晰，会使用第三方工具类简化代码；有中文注释；
减分项：没有考虑首字母大写时的特殊处理情况；
评分：85分。
2）CodeLlama-70B-Instruct
生成的代码：
package ai.codellama70b;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.text.SimpleDateFormat;
import java.util.Date;
public class Transformer {
public String transform(String source) {
String camelCase = camelToUnderline(source);
String md5 = md5(camelCase);
String timestamp = timestamp();
return md5 + "$" + timestamp;
}
private String camelToUnderline(String source) {
// 将驼峰格式转换为下划线格式
System.out.println(source.replaceAll("([A-Z])", "_$1").toLowerCase());
return source.replaceAll("([A-Z])", "_$1").toLowerCase();
}
private String md5(String source) {
// 对字符串进行MD5摘要
try {
MessageDigest md = MessageDigest.getInstance("MD5");
byte[] bytes = md.digest(source.getBytes());
StringBuilder result = new StringBuilder();
for (byte b : bytes) {
String hex = Integer.toHexString(b & 0xFF);
if (hex.length() == 1) {
result.append("0");
}
result.append(hex);
}
return result.toString();
} catch (NoSuchAlgorithmException e) {
e.printStackTrace();
}
return null;
}
private String timestamp() {
// 获取当前时间戳
SimpleDateFormat sdf = new SimpleDateFormat("yyyyMMddHHmmss");
return sdf.format(new Date());
}
public static void main(String[] args) {
Transformer transformer = new Transformer();
String source = "HelloWorld";
String result = transformer.transform(source);
System.out.println(result);
}
}
运行结果不正确。
_hello_world
f36965787cd64350c4960ebdc24c4ad8$20240204092431
测评分析结果：
加分项：编译正确；有中文注释；
减分项：没有考虑首字母大写时的特殊处理情况，导致运行结果不正确；驼峰转下划线格式重复调用了两次；
评分：60分。
3）GLM-4
生成的代码：
package ai.glm4;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.text.SimpleDateFormat;
import java.util.Date;
public class Transformer {
public String transform(String source) {
String underscored = camelToUnderscore(source);
String md5 = getMD5(underscored);
String timestamp = getCurrentTimestamp();
return md5 + "$" + timestamp;
}
private String camelToUnderscore(String source) {
StringBuilder result = new StringBuilder();
for (char c : source.toCharArray()) {
if (Character.isUpperCase(c)) {
result.append("_").append(Character.toLowerCase(c));
} else {
result.append(c);
}
}
System.out.println("Underscored value: " + result.toString());
return result.toString();
}
private String getMD5(String input) {
try {
MessageDigest md = MessageDigest.getInstance("MD5");
md.update(input.getBytes());
byte[] digest = md.digest();
StringBuilder sb = new StringBuilder();
for (byte b : digest) {
sb.append(String.format("%02x", b));
}
return sb.toString();
} catch (NoSuchAlgorithmException e) {
throw new RuntimeException("MD5 algorithm not found", e);
}
}
private String getCurrentTimestamp() {
return new SimpleDateFormat("yyyy-MM-dd HH:mm:ss").format(new Date());
}
public static void main(String[] args) {
Transformer transformer = new Transformer();
String result = transformer.transform("HelloWorld");
System.out.println("Transformed result: " + result);
}
}
运行结果不正确。
Underscored value: _hello_world
Transformed result: f36965787cd64350c4960ebdc24c4ad8$2024-02-04 19:47:36
测评分析结果：
加分项：编译正确；
减分项：没有考虑首字母大写时的特殊处理情况，导致运行结果不正确；没有注释；
评分：60分。
4）GPT-3.5
生成的代码：
package ai.gpt35;
import java.security.MessageDigest;
import java.text.SimpleDateFormat;
import java.util.Date;
public class Transformer {
public static void main(String[] args) {
Transformer transformer = new Transformer();
// Test case: transform a sample source string
String source = "sampleSourceString";
String result = transformer.transform(source);
System.out.println("Transformed Result: " + result);
}
public String transform(String source) {
// 1. Convert camel case to underscore format
String underscoredString = convertCamelToUnderscore(source);
System.out.println("Underscored Format: " + underscoredString);
// 2. Calculate MD5 digest
String md5Digest = calculateMD5(underscoredString);
// 3. Append current timestamp with "$" symbol
String result = md5Digest + "$" + getCurrentTimestamp();
return result;
}
private String convertCamelToUnderscore(String camelCase) {
StringBuilder result = new StringBuilder();
for (int i = 0; i < camelCase.length(); i++) {
char currentChar = camelCase.charAt(i);
if (Character.isUpperCase(currentChar) && i > 0) {
result.append("_").append(Character.toLowerCase(currentChar));
} else {
result.append(currentChar);
}
}
return result.toString();
}
private String calculateMD5(String input) {
try {
MessageDigest md = MessageDigest.getInstance("MD5");
byte[] bytes = md.digest(input.getBytes());
StringBuilder result = new StringBuilder();
for (byte b : bytes) {
result.append(String.format("%02x", b));
}
return result.toString();
} catch (Exception e) {
e.printStackTrace();
return "";
}
}
private String getCurrentTimestamp() {
SimpleDateFormat sdf = new SimpleDateFormat("yyyyMMddHHmmss");
return sdf.format(new Date());
}
}
运行结果正确。
Underscored Format: sample_source_string
Transformed Result: 091513cdffb0896114910a2a3b4e1c22$20240204195225
测评分析结果：
加分项：编译正确；运行结果正确；代码清晰；有英文注释；
减分项：没有考虑首字母大写时的特殊处理情况；
评分：80分。
5）GPT-4-0613
生成的代码：
package ai.gpt4;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
public class Transformer {
public static void main(String[] args) {
Transformer transformer = new Transformer();
String result = transformer.transform("SampleInputStringForTesting");
System.out.println(result);
}
public String transform(String source) {
String underscored = camelToUnderscore(source);
String md5Digest = md5(underscored);
return md5Digest + "$" + System.currentTimeMillis();
}
private String camelToUnderscore(String source) {
if (source == null) {
return null;
}
StringBuilder builder = new StringBuilder();
for (char c : source.toCharArray()) {
if (Character.isUpperCase(c)) {
if (builder.length() > 0) {
builder.append('_');
}
builder.append(Character.toLowerCase(c));
} else {
builder.append(c);
}
}
String underscored = builder.toString();
System.out.println(underscored); // 打印转换后的字符串
return underscored;
}
private String md5(String source) {
try {
MessageDigest md = MessageDigest.getInstance("MD5");
byte[] digest = md.digest(source.getBytes());
return bytesToHex(digest);
} catch (NoSuchAlgorithmException e) {
throw new RuntimeException("MD5 algorithm not found", e);
}
}
private String bytesToHex(byte[] bytes) {
StringBuilder hexString = new StringBuilder();
for (byte b : bytes) {
String hex = Integer.toHexString(0xff & b);
if (hex.length() == 1) {
hexString.append('0');
}
hexString.append(hex);
}
return hexString.toString();
}
}
运行结果正确。
sample_input_string_for_testing
4fbf32fbec3b2adbed3d52e15090a2d7$1707047846034
测评分析结果：
加分项：编译正确；运行结果正确；代码清晰；考虑首字母大写时的特殊处理情况；
减分项：没有注释；
评分：90分。
4、测评四：代码优化
测评四是一个数组内数据处理的算法优化。在数字数组内查找是否有两个数的和等于给定的值，原方法的算法复杂度是O(n^2)，需要通过优化来降低算法复杂度。
提示词如下：
你是一个优秀的java开发工程师，致力于写出高质量的代码。现在给你一个类，对这个类进行优化，给出优化后的完整代码。
要求：
1.保证代码的准确性，可读性，可维护性；
2.代码尽量简洁，能使用lambda表达式最好；
3.保证语法正确性；
4.包括main函数，可以进行自测。
public class DoubleSumToValue {
public static boolean doubleSumToValue(int[] listOfInts, int value) {
if (listOfInts.length < 2) {
return false;
} else {
for (int i = 0; i < listOfInts.length - 1; i++) {
for (int j = i + 1; j < listOfInts.length; j++) {
if (listOfInts[i] + listOfInts[j] == value) {
return true;
}
}
}
return false;
}
}
public static void main(String[] args) {
int[] listOfInts = {1, 2, 3, 4, 5};
int value = 8;
boolean result = doubleSumToValue(listOfInts, value);
System.out.println("Result: " + result);
}
}
1）DeepSeek-Coder-33B-Instruct
生成的代码：
package ai.deepseek;
import java.util.HashSet;
import java.util.Set;
public class DoubleSumToValue {
public static boolean doubleSumToValue(int[] listOfInts, int value) {
if (listOfInts.length < 2) {
return false;
}
Set<Integer> complements = new HashSet<>();
for (int num : listOfInts) {
if (complements.contains(num)) {
return true;
}
complements.add(value - num);
}
return false;
}
public static void main(String[] args) {
int[] listOfInts = {1, 2, 3, 4, 5};
int value = 8;
boolean result = doubleSumToValue(listOfInts, value);
System.out.println("Result: " + result);
}
}
运行结果正确。
Result: true
测评分析结果：
加分项：编译正确；运行结果正确；算法复杂度优化为O(n)；集合命名为complements，能表达相减的余数的含义；
减分项：没有注释；
评分：85分。
2）CodeLlama-70B-Instruct
生成的代码：
package ai.codellama70b;
import java.util.HashSet;
import java.util.Set;
public class DoubleSumToValue {
public static boolean doubleSumToValue(int[] listOfInts, int value) {
if (listOfInts.length < 2) {
return false;
}
Set<Integer> set = new HashSet<>();
for (int num : listOfInts) {
if (set.contains(value - num)) {
return true;
}
set.add(num);
}
return false;
}
public static void main(String[] args) {
int[] listOfInts = {1, 2, 3, 4, 5};
int value = 8;
boolean result = doubleSumToValue(listOfInts, value);
System.out.println("Result: " + result);
}
}
运行结果正确。
Result: true
测评分析结果：
加分项：编译正确；运行结果正确；算法复杂度优化为O(n)；
减分项：集合直接命名为set，没有表达出语义；没有注释；
评分：80分。
3）GLM-4
生成的代码：
package ai.glm4;
import java.util.HashSet;
import java.util.Set;
public class DoubleSumToValue {
public static boolean doubleSumToValue(int[] listOfInts, int value) {
if (listOfInts.length < 2) {
return false;
}
Set<Integer> seen = new HashSet<>();
for (int num : listOfInts) {
if (seen.contains(value - num)) {
return true;
}
seen.add(num);
}
return false;
}
public static void main(String[] args) {
int[] listOfInts = {1, 2, 3, 4, 5};
int value = 8;
boolean result = doubleSumToValue(listOfInts, value);
System.out.println("Result: " + result);
}
}
运行结果正确。
Result: true
测评分析结果：
加分项：编译正确；运行结果正确；算法复杂度优化为O(n)；
减分项：集合命名为seen，看见过的，没有把复数表示出来；没有注释；
评分：80分。
4）GPT-3.5
生成的代码：
package ai.gpt35;
import java.util.HashSet;
import java.util.Set;
public class DoubleSumToValue {
public static boolean doubleSumToValue(int[] listOfInts, int value) {
if (listOfInts.length < 2) {
return false;
}
Set<Integer> set = new HashSet<>();
for (int num : listOfInts) {
int complement = value - num;
if (set.contains(complement)) {
return true;
}
set.add(num);
}
return false;
}
public static void main(String[] args) {
int[] listOfInts = {1, 2, 3, 4, 5};
int value = 8;
boolean result = doubleSumToValue(listOfInts, value);
System.out.println("Result: " + result);
}
}
运行结果正确。
Result: true
测评分析结果：
加分项：编译正确；运行结果正确；算法复杂度优化为O(n)；集合命名为set，里面用到了complement，能表达相减的余数的含义。；
减分项：没有注释；
评分：85分。
5）GPT-4-0613
生成的代码：
package ai.gpt4;
import java.util.HashSet;
/**
* Utility class to check if any two distinct numbers in an array sum up to a specific value.
*/
public class DoubleSumToValue {
/**
* Checks if any two distinct numbers in the array sum up to the given value.
* @param numbers Array of integers to check.
* @param targetValue The target sum value.
* @return true if two distinct numbers sum up to targetValue, false otherwise.
*/
public static boolean hasPairWithSum(int[] numbers, int targetValue) {
HashSet<Integer> seenNumbers = new HashSet<>();
for (int number : numbers) {
if (seenNumbers.contains(targetValue - number)) {
return true;
}
seenNumbers.add(number);
}
return false;
}
public static void main(String[] args) {
int[] numbers = {1, 2, 3, 4, 5};
int targetValue = 8;
boolean result = hasPairWithSum(numbers, targetValue);
System.out.println("Result: " + result);
}
}
运行结果正确。
Result: true
测评分析结果：
加分项：编译正确；运行结果正确；算法复杂度优化为O(n)；集合命名为seenNumbers，表示看见过的数字，另外对方法进行了重命名；增加了英文注释；
减分项：没有中文注释；
评分：90分。
5、测评五：代码纠错
日常开发中经常会碰到，需要寻找或者检查一段代码中的是否有bug的情况。测评五是比对两个同类型对象是否相同的代码，其中"StringUtils.isEmpty(that.storeId)?null:this.storeId)"这句存在bug。
提示词如下：
帮我查找这段代码中的bug，给出完整的类并包括main函数，可以进行自测对象为空的情况:
@Override
public boolean equals(Object o) {
if (this == o) {
return true;
}
if (!(o instanceof SkuKey)) {
return false;
}
SkuKey that = (SkuKey) o;
return new EqualsBuilder()
.append(skuId, that.skuId)
.append(StringUtils.isEmpty(storeId)?null:storeId, StringUtils.isEmpty(that.storeId)?null:this.storeId)
.isEquals();
}
1）DeepSeek-Coder-33B-Instruct
生成的代码：
package ai.deepseek;
import org.apache.commons.lang3.builder.EqualsBuilder;
import org.apache.commons.lang3.StringUtils;
public class SkuKey {
private String skuId;
private String storeId;
public SkuKey(String skuId, String storeId) {
this.skuId = skuId;
this.storeId = storeId;
}
@Override
public boolean equals(Object o) {
if (this == o) {
return true;
}
if (!(o instanceof SkuKey)) {
return false;
}
SkuKey that = (SkuKey) o;
return new EqualsBuilder()
.append(skuId, that.skuId)
.append(normalizeStoreId(storeId), normalizeStoreId(that.storeId))
.isEquals();
}
private String normalizeStoreId(String storeId) {
return StringUtils.isEmpty(storeId) ? null : storeId;
}
public static void main(String[] args) {
// 测试对象为空的情况
SkuKey key1 = new SkuKey("123", "store1");
SkuKey key2 = new SkuKey("123", null);
SkuKey key3 = new SkuKey(null, "store1");
SkuKey key4 = new SkuKey(null, null);
System.out.println("key1 equals key2: " + key1.equals(key2)); // Expected: true
System.out.println("key1 equals key3: " + key1.equals(key3)); // Expected: false
System.out.println("key2 equals key3: " + key2.equals(key3)); // Expected: false
System.out.println("key1 equals key4: " + key1.equals(key4)); // Expected: false
System.out.println("key2 equals key4: " + key2.equals(key4)); // Expected: false
System.out.println("key3 equals key4: " + key3.equals(key4)); // Expected: true
}
}
运行结果按照main方法自测用例，注释的预期值，有两个不符合预期。
key1 equals key2: false
key1 equals key3: false
key2 equals key3: false
key1 equals key4: false
key2 equals key4: false
key3 equals key4: false
测评分析结果：
加分项：编译正确；能找到并修复代码中的bug；能提炼出方法进行复用；
减分项：按照main方法自测用例，注释的预期值，有两个不符合预期；
评分：80分。
2）CodeLlama-70B-Instruct
生成的代码：
package ai.codellama70b;
import org.apache.commons.lang3.builder.EqualsBuilder;
import org.apache.commons.lang3.StringUtils;
public class SkuKey {
private String skuId;
private String storeId;
public SkuKey(String skuId, String storeId) {
this.skuId = skuId;
this.storeId = storeId;
}
@Override
public boolean equals(Object o) {
if (this == o) {
return true;
}
if (!(o instanceof SkuKey)) {
return false;
}
SkuKey that = (SkuKey) o;
return new EqualsBuilder()
.append(skuId, that.skuId)
.append(StringUtils.isEmpty(storeId) ? null : storeId, StringUtils.isEmpty(that.storeId) ? null : that.storeId)
.isEquals();
}
public static void main(String[] args) {
SkuKey skuKey1 = new SkuKey("123", "");
SkuKey skuKey2 = new SkuKey("123", null);
System.out.println(skuKey1.equals(skuKey2));
}
}
测评分析结果：
加分项：编译正确；
减分项：生成的代码不稳定，试了几次，有时候能找到bug，有时找不到bug；
评分：50分。
3）GLM-4
生成的代码：
package ai.glm4;
import org.apache.commons.lang3.StringUtils;
import org.apache.commons.lang3.builder.EqualsBuilder;
import org.apache.commons.lang3.builder.HashCodeBuilder;
public class StarSkuKey {
private String skuId;
private String storeId;
public StarSkuKey(String skuId, String storeId) {
this.skuId = skuId;
this.storeId = storeId;
}
@Override
public boolean equals(Object o) {
if (this == o) {
return true;
}
if (!(o instanceof StarSkuKey)) {
return false;
}
StarSkuKey that = (StarSkuKey) o;
return new EqualsBuilder()
.append(skuId, that.skuId)
.append(storeId, that.storeId)
.isEquals();
}
@Override
public int hashCode() {
return new HashCodeBuilder(17, 37)
.append(skuId)
.append(storeId)
.toHashCode();
}
public static void main(String[] args) {
StarSkuKey key1 = new StarSkuKey("123", "456");
StarSkuKey key2 = new StarSkuKey("123", "456");
StarSkuKey key3 = new StarSkuKey("123", null);
StarSkuKey key4 = new StarSkuKey(null, "456");
StarSkuKey key5 = new StarSkuKey(null, null);
System.out.println("key1 equals key2: " + key1.equals(key2)); // true
System.out.println("key1 equals key3: " + key1.equals(key3)); // false
System.out.println("key1 equals key4: " + key1.equals(key4)); // false
System.out.println("key1 equals key5: " + key1.equals(key5)); // false
System.out.println("key3 equals key5: " + key3.equals(key5)); // true
System.out.println("key4 equals key5: " + key4.equals(key5)); // false
// Test null object
System.out.println("key1 equals null: " + key1.equals(null)); // false
}
}
测评分析结果：
加分项：编译正确；能找到并修复代码中的bug；
减分项：修改了代码，缺失了原来比对""和null含义相同的逻辑；自测预期结果和实际结果有不相符的情况；
评分：60分。
4）GPT-3.5
生成的代码：
package ai.gpt35;
import org.apache.commons.lang3.builder.EqualsBuilder;
import org.apache.commons.lang3.StringUtils;
public class SkuKey {
private String skuId;
private String storeId;
public SkuKey(String skuId, String storeId) {
this.skuId = skuId;
this.storeId = storeId;
}
// getters and setters
@Override
public boolean equals(Object o) {
if (this == o) {
return true;
}
if (!(o instanceof SkuKey)) {
return false;
}
SkuKey that = (SkuKey) o;
return new EqualsBuilder()
.append(skuId, that.skuId)
.append(StringUtils.isEmpty(storeId) ? null : storeId, StringUtils.isEmpty(that.storeId) ? null : that.storeId)
.isEquals();
}
public static void main(String[] args) {
// Test case: objects with empty storeId
SkuKey key1 = new SkuKey("sku1", "");
SkuKey key2 = new SkuKey("sku1", "");
System.out.println("Objects are equal: " + key1.equals(key2));
}
}
测评分析结果：
加分项：编译正确；能找到并修复代码中的bug；
减分项：自测用例有点少；
评分：80分。
5）GPT-4-0613
生成的代码：
package ai.gpt4;
import org.apache.commons.lang3.StringUtils;
import org.apache.commons.lang3.builder.EqualsBuilder;
public class SkuKey {
private String skuId;
private String storeId;
// 构造函数
public SkuKey(String skuId, String storeId) {
this.skuId = skuId;
this.storeId = storeId;
}
// equals 方法
@Override
public boolean equals(Object o) {
if (this == o) {
return true;
}
if (!(o instanceof SkuKey)) {
return false;
}
SkuKey that = (SkuKey) o;
return new EqualsBuilder()
.append(skuId, that.skuId)
.append(StringUtils.isEmpty(storeId) ? null : storeId, StringUtils.isEmpty(that.storeId) ? null : that.storeId)
.isEquals();
}
// hashCode 方法（如果重写 equals，通常也需要重写 hashCode）
@Override
public int hashCode() {
// hashCode 实现，可以使用 Objects.hashCode() 或者 HashCodeBuilder
// 这里仅为示例，实际情况可能需要更复杂的逻辑
return skuId.hashCode() + (storeId == null ? 0 : storeId.hashCode());
}
// main 方法，用于自测
public static void main(String[] args) {
SkuKey skuKey1 = new SkuKey("sku123", "store456");
SkuKey skuKey2 = new SkuKey("sku123", "store456");
SkuKey skuKey3 = new SkuKey("sku123", "");
SkuKey skuKey4 = new SkuKey("sku123", null);
System.out.println("skuKey1 equals skuKey2: " + skuKey1.equals(skuKey2)); // 应该输出 true
System.out.println("skuKey1 equals skuKey3: " + skuKey1.equals(skuKey3)); // 应该输出 false
System.out.println("skuKey1 equals skuKey4: " + skuKey1.equals(skuKey4)); // 应该输出 false
System.out.println("skuKey3 equals skuKey4: " + skuKey3.equals(skuKey4)); // 应该输出 true
}
}
测评分析结果：
加分项：编译正确；能找到并修复代码中的bug；自测用例比较全，能get到值为null和值为""要识别成相等的点；有中文注释；
减分项：无；
评分：100分。
6、测评六：单元测试
测评六是日常开发中必备的单元测试代码生成。给定一个问题数据查询引擎类，需要实现对应的单元测试代码。
提示词如下：
你是一个优秀的java开发工程师，致力于写出高质量的单元测试代码。现在给你一个类，帮我生成单元测试代码，要求使用junit4,mockito3,尽量保证100%的代码行覆盖率，同时保证代码的准确性，可读性。
import com.demo.dao.mapper.ViewResultMapper;
import com.demo.model.ViewColumn;
import com.demo.model.ViewConfig;
import org.apache.commons.collections4.CollectionUtils;
import org.apache.commons.lang3.StringUtils;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
@Service("problemDataViewQueryEngine")
public class ProblemDataViewQueryEngine implements DataViewQueryEngine{
@Autowired
private ViewResultMapper viewResultMapper;
@Override
public LinkedHashMap<String, List<Map<String, Object>>> queryViewData(List<ViewConfig> viewConfigModelList, Integer year, Integer quarter) {
int currentYear = 2023;
int currentQuarter = 1;
if (CollectionUtils.isEmpty(viewConfigModelList)) {
throw new IllegalArgumentException("Parameter viewConfigModelList is empty");
}
if (year == null) {
throw new IllegalArgumentException("Parameter year is empty");
}
if (quarter == null) {
throw new IllegalArgumentException("Parameter quarter is empty");
}
//存放返回值
LinkedHashMap<String, List<Map<String, Object>>> viewData = new LinkedHashMap<>();
for (ViewConfig viewConfig : viewConfigModelList) {
//sheet页名称
String sheetName = viewConfig.getSheet();
for (ViewColumn columnConfig : viewConfig.getColumns()) {
if (StringUtils.isEmpty(columnConfig.getSql())) {
continue;
}
if (year.equals(currentYear) && quarter.equals(currentQuarter)) {
columnConfig.setSql(columnConfig.getSql().replace("#{currentyear}", year.toString()));
columnConfig.setSql(columnConfig.getSql().replace("#{currentquarter}", quarter.toString()));
} else {
columnConfig.setSql(columnConfig.getSql().replace("#{year}", year.toString()));
columnConfig.setSql(columnConfig.getSql().replace("#{quarter}", quarter.toString()));
}
//查询数据
List<Map<String, Object>> viewResultList = viewResultMapper.selectDataView(columnConfig.getSql());
viewData.put(sheetName, viewResultList);
}
}
return viewData;
}
}
1）DeepSeek-Coder-33B-Instruct
生成的单元测试如下。代码编译正确，生成了6个测试用例，行覆盖率达到100%。
package ai.deepseek;
import ai.demo.dao.mapper.ViewResultMapper;
import ai.demo.model.ViewColumn;
import ai.demo.model.ViewConfig;
import org.junit.Before;
import org.junit.Test;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.MockitoAnnotations;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import static org.junit.Assert.assertEquals;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.mockito.Mockito.*;
public class ProblemDataViewQueryEngineTestDeepseek {
@Mock
private ViewResultMapper viewResultMapper;
@InjectMocks
private ProblemDataViewQueryEngine problemDataViewQueryEngine;
@Before
public void setUp() {
MockitoAnnotations.initMocks(this);
}
@Test
public void testQueryViewData_withValidInput() {
// Arrange
List<ViewConfig> viewConfigModelList = new ArrayList<>();
ViewConfig viewConfig = new ViewConfig();
viewConfig.setSheet("Sheet1");
List<ViewColumn> columns = new ArrayList<>();
ViewColumn column = new ViewColumn();
column.setSql("SELECT * FROM table WHERE year = #{year} AND quarter = #{quarter}");
columns.add(column);
viewConfig.setColumns(columns);
viewConfigModelList.add(viewConfig);
List<Map<String, Object>> viewResultList = new ArrayList<>();
Map<String, Object> viewResult = new LinkedHashMap<>();
viewResult.put("column1", "value1");
viewResultList.add(viewResult);
when(viewResultMapper.selectDataView(anyString())).thenReturn(viewResultList);
// Act
LinkedHashMap<String, List<Map<String, Object>>> result = problemDataViewQueryEngine.queryViewData(viewConfigModelList, 2022, 1);
// Assert
assertEquals(1, result.size());
assertEquals(viewResultList, result.get("Sheet1"));
verify(viewResultMapper, times(1)).selectDataView(anyString());
}
@Test
public void testQueryViewData_withEmptyViewConfigModelList() {
// Arrange
List<ViewConfig> viewConfigModelList = new ArrayList<>();
// Act and Assert
assertThrows(IllegalArgumentException.class, () -> {
problemDataViewQueryEngine.queryViewData(viewConfigModelList, 2022, 1);
});
}
@Test
public void testQueryViewData_withNullYear() {
// Arrange
List<ViewConfig> viewConfigModelList = new ArrayList<>();
ViewConfig viewConfig = new ViewConfig();
viewConfig.setSheet("Sheet1");
viewConfigModelList.add(viewConfig);
// Act and Assert
assertThrows(IllegalArgumentException.class, () -> {
problemDataViewQueryEngine.queryViewData(viewConfigModelList, null, 1);
});
}
@Test
public void testQueryViewData_withNullQuarter() {
// Arrange
List<ViewConfig> viewConfigModelList = new ArrayList<>();
ViewConfig viewConfig = new ViewConfig();
viewConfig.setSheet("Sheet1");
viewConfigModelList.add(viewConfig);
// Act and Assert
assertThrows(IllegalArgumentException.class, () -> {
problemDataViewQueryEngine.queryViewData(viewConfigModelList, 2022, null);
});
}
@Test
public void testQueryViewData_withEmptySql() {
// Arrange
List<ViewConfig> viewConfigModelList = new ArrayList<>();
ViewConfig viewConfig = new ViewConfig();
viewConfig.setSheet("Sheet1");
List<ViewColumn> columns = new ArrayList<>();
ViewColumn column = new ViewColumn();
column.setSql("");
columns.add(column);
viewConfig.setColumns(columns);
viewConfigModelList.add(viewConfig);
// Act
LinkedHashMap<String, List<Map<String, Object>>> result = problemDataViewQueryEngine.queryViewData(viewConfigModelList, 2022, 1);
// Assert
assertEquals(0, result.size());
verify(viewResultMapper, never()).selectDataView(anyString());
}
@Test
public void testQueryViewData_withCurrentYearAndQuarter() {
// Arrange
List<ViewConfig> viewConfigModelList = new ArrayList<>();
ViewConfig viewConfig = new ViewConfig();
viewConfig.setSheet("Sheet1");
List<ViewColumn> columns = new ArrayList<>();
ViewColumn column = new ViewColumn();
column.setSql("SELECT * FROM table WHERE year = #{currentyear} AND quarter = #{currentquarter}");
columns.add(column);
viewConfig.setColumns(columns);
viewConfigModelList.add(viewConfig);
List<Map<String, Object>> viewResultList = new ArrayList<>();
Map<String, Object> viewResult = new LinkedHashMap<>();
viewResult.put("column1", "value1");
viewResultList.add(viewResult);
when(viewResultMapper.selectDataView(anyString())).thenReturn(viewResultList);
// Act
LinkedHashMap<String, List<Map<String, Object>>> result = problemDataViewQueryEngine.queryViewData(viewConfigModelList, 2023, 1);
// Assert
assertEquals(1, result.size());
assertEquals(viewResultList, result.get("Sheet1"));
verify(viewResultMapper, times(1)).selectDataView(anyString());
}
}
测评分析结果：
加分项：编译正确；生成了6个测试用例，运行通过6个测试用例，行覆盖率达到100%；
减分项：试第二次，行覆盖率降到80%，不够稳定；
评分：90分。
2）CodeLlama-70B-Instruct
生成的单元测试如下。代码编译正确，生成了1个测试用例，行覆盖率76%。
package ai.codellama70b;
import static org.junit.Assert.assertEquals;
import static org.mockito.Mockito.when;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.HashMap;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.junit.MockitoJUnitRunner;
import ai.demo.dao.mapper.ViewResultMapper;
import ai.demo.model.ViewColumn;
import ai.demo.model.ViewConfig;
@RunWith(MockitoJUnitRunner.class)
public class ProblemDataViewQueryEngineTestCodeLlama70b {
@Mock
private ViewResultMapper viewResultMapper;
@InjectMocks
private ProblemDataViewQueryEngine problemDataViewQueryEngine;
@Test
public void testQueryViewData() {
// Arrange
List<ViewConfig> viewConfigModelList = new ArrayList<>();
ViewConfig viewConfig = new ViewConfig();
viewConfig.setSheet("Sheet1");
List<ViewColumn> columns = new ArrayList<>();
ViewColumn column = new ViewColumn();
column.setSql("select * from table");
columns.add(column);
viewConfig.setColumns(columns);
viewConfigModelList.add(viewConfig);
Integer year = 2022;
Integer quarter = 1;
Map<String, Object> result1 = new HashMap<>();
result1.put("id", 1);
result1.put("name", "John");
Map<String, Object> result2 = new HashMap<>();
result2.put("id", 2);
result2.put("name", "Jane");
List<Map<String, Object>> viewResultList = Arrays.asList(result1, result2);
when(viewResultMapper.selectDataView(column.getSql())).thenReturn(viewResultList);
// Act
LinkedHashMap<String, List<Map<String, Object>>> viewData = problemDataViewQueryEngine.queryViewData(viewConfigModelList, year, quarter);
// Assert
assertEquals(viewResultList, viewData.get("Sheet1"));
}
}
测评分析结果：
加分项：编译正确；生成了1个测试用例，运行通过1个测试用例，行覆盖率76%；
减分项：只生成一个测试用例，行覆盖率不够高；
评分：76分。
3）GLM-4
生成的单元测试如下。代码编译正确，生成了5个测试用例，行覆盖率80%。
package ai.glm4;
import ai.demo.dao.mapper.ViewResultMapper;
import ai.demo.model.ViewColumn;
import ai.demo.model.ViewConfig;
import org.junit.Before;
import org.junit.Test;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.MockitoAnnotations;
import java.util.*;
import static org.junit.Assert.*;
import static org.mockito.Mockito.*;
public class ProblemDataViewQueryEngineTestglm {
@InjectMocks
private ProblemDataViewQueryEngine engine;
@Mock
private ViewResultMapper viewResultMapper;
@Before
public void setUp() {
MockitoAnnotations.openMocks(this);
}
@Test(expected = IllegalArgumentException.class)
public void testQueryViewDataWithEmptyViewConfigModelList() {
engine.queryViewData(null, 2023, 1);
}
@Test(expected = IllegalArgumentException.class)
public void testQueryViewDataWithNullYear() {
List<ViewConfig> viewConfigModelList = new ArrayList<>();
engine.queryViewData(viewConfigModelList, null, 1);
}
@Test(expected = IllegalArgumentException.class)
public void testQueryViewDataWithNullQuarter() {
List<ViewConfig> viewConfigModelList = new ArrayList<>();
engine.queryViewData(viewConfigModelList, 2023, null);
}
@Test
public void testQueryViewData() {
List<ViewConfig> viewConfigModelList = Arrays.asList(new ViewConfig());
ViewColumn columnConfig = new ViewColumn();
columnConfig.setSql("SELECT * FROM table WHERE year = #{year} AND quarter = #{quarter}");
viewConfigModelList.get(0).setColumns(Arrays.asList(columnConfig));
viewConfigModelList.get(0).setSheet("Sheet1");
Map<String, Object> resultMap = new HashMap<>();
resultMap.put("column1", "value1");
List<Map<String, Object>> resultList = Arrays.asList(resultMap);
when(viewResultMapper.selectDataView(anyString())).thenReturn(resultList);
LinkedHashMap<String, List<Map<String, Object>>> viewData = engine.queryViewData(viewConfigModelList, 2023, 1);
assertNotNull(viewData);
assertEquals(1, viewData.size());
assertEquals(resultList, viewData.get("Sheet1"));
verify(viewResultMapper, times(1)).selectDataView(anyString());
}
@Test
public void testQueryViewDataWithCurrentYearAndQuarter() {
List<ViewConfig> viewConfigModelList = Arrays.asList(new ViewConfig());
ViewColumn columnConfig = new ViewColumn();
columnConfig.setSql("SELECT * FROM table WHERE year = #{currentyear} AND quarter = #{currentquarter}");
viewConfigModelList.get(0).setColumns(Arrays.asList(columnConfig));
viewConfigModelList.get(0).setSheet("Sheet1");
Map<String, Object> resultMap = new HashMap<>();
resultMap.put("column1", "value1");
List<Map<String, Object>> resultList = Arrays.asList(resultMap);
when(viewResultMapper.selectDataView(anyString())).thenReturn(resultList);
LinkedHashMap<String, List<Map<String, Object>>> viewData = engine.queryViewData(viewConfigModelList, 2023, 1);
assertNotNull(viewData);
assertEquals(1, viewData.size());
assertEquals(resultList, viewData.get("Sheet1"));
verify(viewResultMapper, times(1)).selectDataView(anyString());
}
}
测评分析结果：
加分项：编译正确；生成了5个测试用例，运行通过5个测试用例，行覆盖率80%；
减分项：行覆盖率不够高；
评分：80分。
4）GPT-3.5
生成的单元测试如下。代码编译正确，生成了1个测试用例，行覆盖率76%。
package ai.gpt35;
import ai.demo.dao.mapper.ViewResultMapper;
import ai.demo.model.ViewColumn;
import ai.demo.model.ViewConfig;
import org.junit.Before;
import org.junit.Test;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.MockitoAnnotations;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import static org.junit.Assert.assertEquals;
import static org.mockito.Mockito.*;
public class ProblemDataViewQueryEngineTestgpt35 {
@Mock
private ViewResultMapper viewResultMapper;
@InjectMocks
private ProblemDataViewQueryEngine queryEngine;
@Before
public void setUp() {
MockitoAnnotations.initMocks(this);
}
@Test
public void testQueryViewData() {
// Mock data
List<ViewConfig> viewConfigModelList = new ArrayList<>();
ViewConfig viewConfig = new ViewConfig();
viewConfig.setSheet("TestSheet");
ViewColumn columnConfig = new ViewColumn();
columnConfig.setSql("SELECT * FROM TestTable");
List<ViewColumn> columns = new ArrayList<>();
columns.add(columnConfig);
viewConfig.setColumns(columns);
viewConfigModelList.add(viewConfig);
// Mock parameters
Integer year = 2023;
Integer quarter = 1;
// Mock result
List<Map<String, Object>> mockResult = new ArrayList<>();
// Add test data to mockResult
// Mock behavior
when(viewResultMapper.selectDataView(anyString())).thenReturn(mockResult);
// Perform the test
LinkedHashMap<String, List<Map<String, Object>>> result = queryEngine.queryViewData(viewConfigModelList, year, quarter);
// Verify behavior
verify(viewResultMapper, times(1)).selectDataView(anyString());
// Assertions
assertEquals(1, result.size()); // Check the size of the result map
assertEquals(mockResult, result.get("TestSheet")); // Check the result map content
}
// Add more test cases as needed
}
测评分析结果：
加分项：编译正确；生成了1个测试用例，运行通过1个测试用例，行覆盖率76%；
减分项：只生成1个测试用例，行覆盖率不够高；
评分：76分。
5）GPT-4-0613
生成的单元测试如下。代码编译正确，生成了4个测试用例，执行成功4个测试用例，行覆盖率88%。
package ai.gpt4;
import ai.demo.dao.mapper.ViewResultMapper;
import ai.demo.model.ViewColumn;
import ai.demo.model.ViewConfig;
import org.junit.Before;
import org.junit.Test;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.MockitoAnnotations;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import static org.mockito.Mockito.*;
import static org.junit.Assert.*;
public class ProblemDataViewQueryEngineTestGpt4 {
@Mock
private ViewResultMapper viewResultMapper;
@InjectMocks
private ProblemDataViewQueryEngine problemDataViewQueryEngine;
@Before
public void setUp() {
MockitoAnnotations.initMocks(this);
}
@Test(expected = IllegalArgumentException.class)
public void testQueryViewDataWithEmptyViewConfigModelList() {
problemDataViewQueryEngine.queryViewData(new ArrayList<>(), 2023, 1);
}
@Test(expected = IllegalArgumentException.class)
public void testQueryViewDataWithNullYear() {
List<ViewConfig> viewConfigModelList = new ArrayList<>();
viewConfigModelList.add(new ViewConfig());
problemDataViewQueryEngine.queryViewData(viewConfigModelList, null, 1);
}
@Test(expected = IllegalArgumentException.class)
public void testQueryViewDataWithNullQuarter() {
List<ViewConfig> viewConfigModelList = new ArrayList<>();
viewConfigModelList.add(new ViewConfig());
problemDataViewQueryEngine.queryViewData(viewConfigModelList, 2023, null);
}
@Test
public void testQueryViewData() {
// 构造测试数据
String sheetName = "TestSheet";
String sql = "SELECT * FROM table WHERE year=#{year} AND quarter=#{quarter}";
List<Map<String, Object>> mockResultList = new ArrayList<>();
Map<String, Object> mockResult = new LinkedHashMap<>();
mockResult.put("column1", "value1");
mockResultList.add(mockResult);
ViewColumn viewColumn = new ViewColumn();
viewColumn.setSql(sql);
List<ViewColumn> viewColumns = new ArrayList<>();
viewColumns.add(viewColumn);
ViewConfig viewConfig = new ViewConfig();
viewConfig.setSheet(sheetName);
viewConfig.setColumns(viewColumns);
List<ViewConfig> viewConfigModelList = new ArrayList<>();
viewConfigModelList.add(viewConfig);
// 配置Mockito行为
when(viewResultMapper.selectDataView(anyString())).thenReturn(mockResultList);
// 执行测试方法
LinkedHashMap<String, List<Map<String, Object>>> result = problemDataViewQueryEngine.queryViewData(viewConfigModelList, 2023, 1);
// 验证结果
assertNotNull(result);
assertEquals(1, result.size());
assertTrue(result.containsKey(sheetName));
assertEquals(mockResultList, result.get(sheetName));
// 验证Mockito交互
verify(viewResultMapper).selectDataView(anyString());
}
}
测评分析结果：
加分项：编译正确；生成了4个测试用例，运行通过4个测试用例，行覆盖率88%；
减分项：行覆盖率不够高；
评分：88分。
四、总结
以上通过代码生成、代码补全、代码优化、代码纠错、单元测试这几个方面，共6个测评案例，对大模型在Java编码能力进行了测评分析。汇总出测评的得分表如下。
| 大模型 | 日期工具类 | 报表数据填充 | 代码补全 | 代码优化 | 代码纠错 | 单元测试 | 平均分 |
| DeepSeek-Coder-33B | 80分 | 50分 | 85分 | 85分 | 80分 | 90分 | 78.3分 |
| CodeLlama-70B | 60分 | 80分 | 60分 | 80分 | 50分 | 76分 | 67.7分 |
| GLM-4 | 90分 | 85分 | 60分 | 80分 | 60分 | 80分 | 75.8分 |
| GPT-3.5 | 90分 | 50分 | 80分 | 85分 | 80分 | 76分 | 76.8分 |
| GPT-4 | 85分 | 85分 | 90分 | 90分 | 100分 | 88分 | 89.7分 |
可以看到，在Java编码能力方面，GPT-4-0613 > DeepSeek-Coder-33B-Instruct > GPT-3.5 > GLM-4 > CodeLlama-70B-Instruct。整体来看，GPT-4在六项测评中都表现得比较稳定，其它四个大模型都存在在某项测评中表现不太稳定的情况。在编码大模型的选型方面，如果追求性能，首选还是GPT-4，但是也比较贵。如果从安全性和性价比考虑的话，私有化部署开源编码大模型DeepSeek-Coder-33B-Instruct应该是一个不错的选择。DeepSeek-Coder-33B-Instruct、GPT-3.5、GLM-4这三个大模型编码能力比较接近，相差不大。至于CodeLlama-70B-Instruct，还是有点差距（小扎加油！）。
从本次测评生成的具体代码可以看出，在给出某个功能比较明确的需求设计说明，类似我们写某个功能模块的详细设计这类的文档之后，代码大模型大部分情况可以快速实现这个功能需求。虽然实现的不一定好，在代码的可复用性、可读性、可扩展性方面还有所欠缺，但大部分也能达到初级程序员的水平，在算法优化方面甚至能达到中高级程序员水平。在实际使用过程中，可以通过代码大模型来加快程序的开发。之前需要程序员完全自己实现代码，现在可以先让大模型输出第一版，完成80%的代码，在这个基础上再去完善，提升研发效率。
874
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
