ChatGPT制定Meta分析的文献检索策略，准确性如何?

本文链接：https://blog.csdn.net/statmax/article/details/135984893

今年初，有研究者撰写了一篇“Letter to the Editor”投稿到《Journal of Translational Medicine》（IF=7.4），信件中提到了关于使用ChatGPT制定Meta分析的文献检索策略是否可行的问题。

在这封信中，作者对ChatGPT在解码核心问题并搜索世界范围内三个用于指导研究人员和方法学家的数据库中的整个文献的能力进行了关键评估。

为了执行比较性搜索策略的分析，作者使用了PROSPERO平台提供的记录(#CRD42023391396)，旨在回答中心问题：“肥胖个体在进行了减重手术后何时出现体重再增？”

该问题的PICOT组成部分为：1.人群—18岁以上的肥胖个体；2.干预—减重手术；3.对照—饮食、药物或安慰剂；4.结果—体重再增的时间；5.研究类型—临床试验。

在构建了中心问题之后，作者要求ChatGPT为MEDLINE数据库创建搜索策略，具体输入内容如下：

ChatGPT回复如下：

下一步，研究者要求ChatGPT重新制定检索策略，用于在另外两个广泛用于系统综述和Meta分析的数据库中进行检索：LILACS和Embase。具体输入内容如下：

ChatGPT回复如下：

作者又展示了由方法学家执行并经图书管理员验证的手动搜索策略：

MEDLINE
#1 “"Body-Weight Trajectory"[Mesh] OR "Body Weight Trajectory" OR "Body-Weight Trajectories" OR "Weight Trajectory" OR "Trajectories, Weight" OR "Trajectory, Weight" OR "Weight Trajectories" OR "Weight Change Trajectory" OR "Weight Change Trajectories" OR "Weight Gain Trajectory" OR "Trajectories, Weight Gain" OR "Trajectory, Weight Gain" OR "Weight Gain"[Mesh] OR "Gain, Weight" OR "Gains, Weight" OR "Weight Gains" OR "Weight Cycling" OR "Cycling, Weight" OR "weight regain" OR "weight recidivism" OR "relapse" OR "recrudescence" OR "weight failure" #2 “Bariatric Surgery” [Mesh] OR “Surgeries, Bariatric” OR “Surgery, Bariatric” OR “Metabolic Surgery” OR “Metabolic Surgeries” OR “Surgeries, Metabolic” OR “Surgery, Metabolic” OR “Bariatric Surgical Procedures” OR “Bariatric Surgical Procedure” OR “Procedure, Bariatric Surgical” OR “Procedures, Bariatric Surgical” OR “Surgical Procedure, Bariatric” OR “Surgical Procedures, Bariatric” OR “Bariatric Surgeries” OR “Stomach Stapling” OR “Stapling, Stomach” #3 Obesity [Mesh] OR “obesity” OR “obese” #4 ((clinical[Title/Abstract] AND trial[Title/Abstract]) OR clinical trials as topic[MeSH Terms] OR clinical trial[Publication Type] OR random*[Title/Abstract] OR random allocation[MeSH Terms] OR therapeutic use[MeSH Subheading])

MEDLINE

#1 “"Body-Weight Trajectory"[Mesh] OR "Body Weight Trajectory" OR "Body-Weight Trajectories" OR "Weight Trajectory" OR "Trajectories, Weight" OR "Trajectory, Weight" OR "Weight Trajectories" OR "Weight Change Trajectory" OR "Weight Change Trajectories" OR "Weight Gain Trajectory" OR "Trajectories, Weight Gain" OR "Trajectory, Weight Gain" OR "Weight Gain"[Mesh] OR "Gain, Weight" OR "Gains, Weight" OR "Weight Gains" OR "Weight Cycling" OR "Cycling, Weight" OR "weight regain" OR "weight recidivism" OR "relapse*" OR "recrudescence*" OR "weight failure"
#2 “Bariatric Surgery” [Mesh] OR “Surgeries, Bariatric” OR “Surgery, Bariatric” OR “Metabolic Surgery” OR “Metabolic Surgeries” OR “Surgeries, Metabolic” OR “Surgery, Metabolic” OR “Bariatric Surgical Procedures” OR “Bariatric Surgical Procedure” OR “Procedure, Bariatric Surgical” OR “Procedures, Bariatric Surgical” OR “Surgical Procedure, Bariatric” OR “Surgical Procedures, Bariatric” OR “Bariatric Surgeries” OR “Stomach Stapling” OR “Stapling, Stomach”
#3 Obesity [Mesh] OR “obesity” OR “obese”
#4 ((clinical[Title/Abstract] AND trial[Title/Abstract]) OR clinical trials as topic[MeSH Terms] OR clinical trial[Publication Type] OR random*[Title/Abstract] OR random allocation[MeSH Terms] OR therapeutic use[MeSH Subheading])

http://EMBASE.COM
('body weight change'/syn OR 'weight regain'/syn OR 'weight management'/syn OR ‘weight relapse’/syn) AND [embase]/lim AND ('Bariatric Surgery'/syn) AND [embase]/lim AND ('obesity'/syn OR 'obese patient'/syn) AND [embase]/lim AND 'clinical trial'/de OR 'randomized controlled trial'/de OR 'randomization'/de OR 'single blind procedure'/de OR 'double blind procedure'/de OR 'crossover procedure'/de OR 'placebo'/de OR 'prospective study'/de OR ('randomi?ed controlled' NEXT/1 trial) OR rct OR 'randomly allocated' OR 'allocated randomly' OR 'random allocation' OR (allocated NEAR/2 random) OR (single NEXT/1 blind) OR (double NEXT/1 blind) OR ((treble OR triple) NEAR/1 blind) OR placebo*

http://EMBASE.COM

('body weight change'/syn OR 'weight regain'/syn OR 'weight management'/syn OR ‘weight relapse’/syn) AND [embase]/lim
AND
('Bariatric Surgery'/syn) AND [embase]/lim
AND
('obesity'/syn OR 'obese patient'/syn) AND [embase]/lim
AND
'clinical trial'/de OR 'randomized controlled trial'/de OR 'randomization'/de OR 'single blind procedure'/de OR 'double blind procedure'/de OR 'crossover procedure'/de OR 'placebo'/de OR 'prospective study'/de OR ('randomi?ed controlled' NEXT/1 trial*) OR rct OR 'randomly allocated' OR 'allocated randomly' OR 'random allocation' OR (allocated NEAR/2 random) OR (single NEXT/1 blind*) OR (double NEXT/1 blind*) OR ((treble OR triple) NEAR/1 blind*) OR placebo*

经过比对后，作者得出，ChatGPT尽管回复速度较快，但它创建的检索策略在构成上存在一些问题。比如：

1. ChatGPT没有在检索策略中插入同义词（Entry Terms）和研究者临床实践中常使用的词。

2. 结构上，ChatGPT创建的检索策略没有正确地使用连接词（布尔逻辑）。例如，obese people不能作为weight regain的替代术语，也就是obese people和weight regain不应该使用OR来连接。

3. ChatGPT在检索时插入了检索截止日期。

4. ChatGPT未在检索式中插入研究类型的限定，比如随机临床试验。

小编有话说：

对于作者提到的问题，除了第4点作者在给ChatGPT命令时并没有提及外，另外三点的的确确存在这样的问题。因此，建议研究者谨慎使用ChatGPT进行制定信息搜索策略。尽管ChatGPT很方便并且响应速度很快，但仍会存在内容和结构的问题。