基于autodl与llama-factory微调llama3（四）

一、prompt更迭

在后续的微调过程中，我们着重对模型容易出现的问题进行针对prompt的改进，具体迭代了将近十个版本，最后选定的prompt如下：

其具体解释为：

首先介绍我们的任务需要与输入：这篇文章是医学文献摘要。从给定的医学摘要中提取以下数据，并以指定的JSON格式输出。提取的固定字段包括（不再赘述，见下方英文）

我们对abs的数值进行了较为详尽的描述，翻译为中文表示的是：

-IV bin abs:与结果相对应的干预组参与者的绝对人数或属性值

然后在说明完具体的固定字段内容后，详细表示了我们的具体需求，具体到非常多的实际情况：

所有固定数据字段将被封装在fixed_data中，所有可变数据字段将被封装在variable_data中。固定数据字段可以出现多次，当它们出现时，多个值应该以逗号分隔的格式表示。变量数据字段采用数组形式，数组中的每一项都是从文章中找到的，其中包含结果(结果)的描述和该结果的值，使得数组包含多个结果条目及其相关值。重要的是要注意，在variable_data数组中的项中，结果作为主键，下面必须是与结果相对应的参数。同时，如果variable_data条目中的结果不对应任何iv或cv类型参数，则认为该结果无效，由于缺乏数据，不应出现在variable_data中。任何以iv和cv开头的字段应该只包含数字和单位数据，不包含描述性文本。百分比只能出现在“iv-bin-percent”和“cv-bin-percent”中，不能出现在其他结果属性值中。模型必须只输出下面描述的JSON格式，格式输出完成后立即停止输出，不输出任何与JSON格式无关的描述性文本。

This passage is a medical literature abstract.         Extract the following data from the given medical abstract and output in the specified JSON format.         The extracted fixed fields include: -Total participants: The total number of participants in the study.         - Intervention participants: The number of participants in the intervention group.         - Control participants: The number of participants in the control group.         - Age: The age range or average age of participants.         - Intervention age: The age range or average age of participants in the intervention group.         - Control age: The age range or average age of participants in the control group.         - Eligibility: The eligibility criteria for participants.         - Condition: The medical condition or conditions being studied.         - Location: The location(s) where the study was conducted.         - Ethnicity: The ethnicity of participants.         - Intervention: The type of intervention used.         - Control: The type of control used.         - Outcome measure: The primary outcome measure(s) of the study.         - Conclusion: The conclusion of the study.         The extracted variable fields include (for each outcome event): - Outcome: The outcome event being described.         - IV Bin Abs: The absolute number or attribute value of intervention group participants corresponding to the outcome.         - CV Bin Abs: The absolute number or attribute value of control group participants corresponding to the outcome.         - IV Bin Percent: Percentage of the number of participants in the intervention group or percentage of attribute values corresponding to the outcome.         - CV Bin Percent: Percentage of the number of participants in the control group or percentage of attribute values corresponding to the outcome.         - IV Cont Mean: The mean value of the outcome measure for the intervention group.         - CV Cont Mean: The mean value of the outcome measure for the control group.         - IV Cont Median: The median value of the outcome measure for the intervention group.         - CV Cont Median: The median value of the outcome measure for the control group.         - IV Cont SD: The standard deviation of the outcome measure for the intervention group.         - CV Cont SD: The standard deviation of the outcome measure for the control group.         All fixed data fields will be wrapped in fixed_data, and all variable data fields will be wrapped in variable_data.         Fixed data fields can appear multiple times, and when they do, multiple values should be expressed in a comma-separated format.         The variable data fields are in array form, and each item in the array is found from the article, containing the description of the result (outcome) and the values of that result, making the array contain multiple outcome entries and their related values.         It is important to note that within an item in the variable_data array, the outcome serves as the primary key, and the following must be the parameters corresponding to the outcome.         Meanwhile, if an outcome in a variable_data entry does not correspond to any iv or cv type parameters, it is considered as an invalid outcome and should not appear in variable_data due to the lack of data.         Any fields starting with iv and cv should only contain numerical and unit data, without descriptive text.         The percentage can only appear in "iv-bin-percent" and "cv-bin-percent" and cannot appear in other outcome property values.         The model must only output the JSON format described below, and immediately stop output when the format output is complete, and do not output any descriptive text unrelated to the JSON format.The specific format is as follows.   {\"fixed_data\": {\"total-participants\": \"\", \"intervention-participants\": \"\", \"control-participants\": \"\", \"age\": \"\", \"intervention-age\": \"\", \"control-age\": \"\", \"eligibility\": \"\", \"condition\": \"\", \"location\": \"\", \"ethnicity\":  \"\", \"intervention\": \"\", \"control\": \"\", \"outcome-measure\": \"\", \"conclusion\": \"\"}, \"variable_data\":  [{\"outcome\": \"\", \"iv-bin-abs\": \"\", \"cv-bin-abs\":             \"\", \"iv-bin-percent\":    \"\", \"cv-bin-percent\": \"\", \"iv-cont-mean\": \"\", \"cv-cont-mean\": \"\", \"iv-cont-median\": \"\", \"cv-cont-median\": \"\", \"iv-cont-sd\": \"\", \"cv-cont-sd\": \"\"}, {\"outcome\": \"\", \"iv-bin-abs\": \"\", \"cv-bin-abs\": \"\", \"iv-bin-percent\":  \"\", \"cv-bin-percent\": \"\", \"iv-cont-mean\": \"\", \"cv-cont-mean\": \"\", \"iv-cont-median\": \"\", \"cv-cont-median\":  \"\", \"iv-cont-sd\": \"\", \"cv-cont-sd\":             \"\"}]}

二、数据集的完善与精炼

鉴于我们模型对百分比的输出不是很完美，因此我们基于之前已经选过的数据集再次进行精挑，从中额外标注以及确定筛选出了总计30篇标注。每一篇的质量可以参考下图：

三、微调的loss曲线

四、微调结果的展现

原文献：

Ribociclib plus letrozole versus chemotherapy for postmenopausal women with hormone receptor-positive, HER2-negative, luminal B breast cancer (CORALLEEN): an open-label, multicentre, randomised, phase 2 trial. In hormone receptor-positive, HER2-negative early stage breast cancer, cyclin-dependent kinases 4 and 6 (CDK4/6) inhibition in combination with endocrine therapy could represent an alternative to multiagent chemotherapy. We aimed to evaluate the biological and clinical activity of neoadjuvant ribociclib plus letrozole in the luminal B subtype of early stage breast cancer. CORALLEEN is a parallel-arm, multicentre, randomised, open-label, phase 2 trial completed across 21 hospitals in Spain. We recruited postmenopausal women (≥18 years) with stage I-IIIA hormone receptor-positive, Eastern Cooperative Oncology Group Performance Status 0-1, HER2-negative breast cancer and luminal B by PAM50 with histologically confirmed, operable primary tumour size of at least 2 cm in diameter as measured by MRI. Patients were randomly assigned (1:1) using a web-based system and permuted blocks of 25 to receive either six 28-days cycles of ribociclib (oral 600 mg once daily for 3 weeks on, 1 week off) plus daily letrozole (oral 2·5 mg/day) or four cycles of doxorubicin (intravenous 60 mg/m2) and cyclophosphamide (intravenous 600 mg/m2) every 21 days followed by weekly paclitaxel (intravenous 80 mg/m2) for 12 weeks. The total duration of the neoadjuvant therapy was 24 weeks. Randomisation was stratified by tumour size and nodal involvement. Samples were prospectively collected at baseline (day 0), day 15, and surgery. The primary endpoint was to evaluate the proportion of patients with PAM50 low-risk-of-relapse (ROR) disease at surgery in the modified intention-to-treat population including all randomly assigned patients who received study drug and had a baseline and at least one post-baseline measurement of ROR score. The PAM50 ROR risk class integrated gene expression data, tumour size, and nodal status to define prognosis. This trial was registered at ClinicalTrials. gov, NCT03248427. Between July 27, 2017 to Dec 7, 2018, 106 patients were enrolled. At baseline, of the 106 patients, 92 (87%) patients had high ROR disease (44 [85%] of 52 in the ribociclib and letrozole group and 48 [89%] of 54 in the chemotherapy group) and 14 (13%) patients had intermediate-ROR disease (eight [15%] and six [11%]). Median follow-up was 200·0 days (IQR 191·2-206·0). At surgery, 23 (46·9%; 95% CI 32·5-61·7) of 49 patients in the ribociclib plus letrozole group and 24 (46·1%; 32·9-61·5) of 52 patients in the chemotherapy group were low-ROR. The most common grade 3-4 adverse events in the ribociclib plus letrozole group were neutropenia (22 [43%] of 51 patients) and elevated alanine aminotransferase concentrations (ten [20%]). The most common grade 3-4 adverse events in the chemotherapy group were neutropenia (31 [60%] of 52 patients) and febrile neutropenia (seven [13%]). No deaths were observed during the study in either group. Our results suggest that some patients with high-risk, early stage, hormone receptor-positive, HER2-negative breast cancer could achieve molecular downstaging of their disease with CDK4/6 inhibitor and endocrine therapy. Novartis, Nanostring, Breast Cancer Research Foundation-AACR Career Development Award.

对应的模型输出结果：

通过整体观察，结果的对应关系较好，且能够准确识别数据值等内容，因此该模型的整体效果将作为我们预期的合并权重的模型。