应用场景
- 运行 cesm2.1.3 或 cesm2.2.0,设置 restart 频率时,case.build, case.submit 都没有问题,但在模型 execution finish 之后,归档环节,中间的 restart 文件并无法全部copy下来,而是直接删除了。预留下来的restart 文件只剩下模拟结束的时刻
- xml设置如下:
./xmlchange NTASKS=80
./xmlchange DATM_CLMNCEP_YR_ALIGN=1980
./xmlchange DATM_CLMNCEP_YR_START=1980
./xmlchange DATM_CLMNCEP_YR_END=1980
./xmlchange RUN_STARTDATE=1980-01-01
./xmlchange STOP_N=12
./xmlchange STOP_OPTION=nmonths
./xmlchange CLM_FORCE_COLDSTART='on'
./xmlchange REST_OPTION=nmonths
./xmlchange REST_N=3
- case.submit 之后模型开始运行之后,显示如下:
2020-12-19 18:02:28 MODEL EXECUTION BEGINS HERE
run command is mpirun -np 80 /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/bld/cesm.exe >> cesm.log.$LID 2>&1
2020-12-20 03:30:13 MODEL EXECUTION HAS FINISHED
check for resubmit
dout_s True
mach ubuntu
resubmit_num 0
Submit job case.st_archive
Starting job script case.st_archive
st_archive starting
moving /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/rof.log.201219-180227.gz to /home/dayon/cesm/archive/CLM50SpGs_hcru_ecmwf/logs/rof.log.201219-180227.gz
moving /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/cesm.log.201219-180227.gz to /home/dayon/cesm/archive/CLM50SpGs_hcru_ecmwf/logs/cesm.log.201219-180227.gz
moving /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/atm.log.201219-180227.gz to /home/dayon/cesm/archive/CLM50SpGs_hcru_ecmwf/logs/atm.log.201219-180227.gz
moving /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/cpl.log.201219-180227.gz to /home/dayon/cesm/archive/CLM50SpGs_hcru_ecmwf/logs/cpl.log.201219-180227.gz
moving /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/lnd.log.201219-180227.gz to /home/dayon/cesm/archive/CLM50SpGs_hcru_ecmwf/logs/lnd.log.201219-180227.gz
-------------------------------------------
Archiving restarts for date date(1980, 4, 1, 0, 0, 0)
-------------------------------------------
Archiving restarts for datm (atm)
Archiving restarts for clm (lnd)
removing interim restart file /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.clm2.r.1980-04-01-00000.nc
WARNING: ncdump -v locfnh /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.clm2.rh0.1980-04-01-00000.nc failed rc=1
out=
err=ncdump: locfnh: No such variable
removing interim restart file /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.clm2.rh0.1980-04-01-00000.nc
Archiving restarts for mosart (rof)
removing interim restart file /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.mosart.r.1980-04-01-00000.nc
WARNING: ncdump -v locfnh /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.mosart.rh0.1980-04-01-00000.nc failed rc=1
out=
err=ncdump: locfnh: No such variable
removing interim restart file /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.mosart.rh0.1980-04-01-00000.nc
Archiving restarts for drv (cpl)
removing interim restart file /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.cpl.r.1980-04-01-00000.nc
Archiving restarts for dart (esp)
-------------------------------------------
Archiving restarts for date date(1980, 7, 1, 0, 0, 0)
-------------------------------------------
Archiving restarts for datm (atm)
Archiving restarts for clm (lnd)
removing interim restart file /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.clm2.r.1980-07-01-00000.nc
WARNING: ncdump -v locfnh /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.clm2.rh0.1980-07-01-00000.nc failed rc=1
out=
err=ncdump: locfnh: No such variable
removing interim restart file /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.clm2.rh0.1980-07-01-00000.nc
Archiving restarts for mosart (rof)
removing interim restart file /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.mosart.r.1980-07-01-00000.nc
WARNING: ncdump -v locfnh /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.mosart.rh0.1980-07-01-00000.nc failed rc=1
out=
err=ncdump: locfnh: No such variable
removing interim restart file /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.mosart.rh0.1980-07-01-00000.nc
Archiving restarts for drv (cpl)
removing interim restart file /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.cpl.r.1980-07-01-00000.nc
Archiving restarts for dart (esp)
-------------------------------------------
Archiving restarts for date date(1980, 10, 1, 0, 0, 0)
-------------------------------------------
Archiving restarts for datm (atm)
Archiving restarts for clm (lnd)
removing interim restart file /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.clm2.r.1980-10-01-00000.nc
WARNING: ncdump -v locfnh /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.clm2.rh0.1980-10-01-00000.nc failed rc=1
out=
err=ncdump: locfnh: No such variable
removing interim restart file /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.clm2.rh0.1980-10-01-00000.nc
Archiving restarts for mosart (rof)
removing interim restart file /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.mosart.r.1980-10-01-00000.nc
WARNING: ncdump -v locfnh /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.mosart.rh0.1980-10-01-00000.nc failed rc=1
out=
err=ncdump: locfnh: No such variable
removing interim restart file /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.mosart.rh0.1980-10-01-00000.nc
Archiving restarts for drv (cpl)
removing interim restart file /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.cpl.r.1980-10-01-00000.nc
Archiving restarts for dart (esp)
-------------------------------------------
Archiving restarts for date date(1981, 1, 1, 0, 0, 0)
-------------------------------------------
Archiving restarts for datm (atm)
Archiving restarts for clm (lnd)
copying file /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.clm2.r.1981-01-01-00000.nc to /home/dayon/cesm/archive/CLM50SpGs_hcru_ecmwf/rest/1981-01-01-00000/CLM50SpGs_hcru_ecmwf.clm2.r.1981-01-01-00000.nc
Copying /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.clm2.h0.1980-12.nc to /home/dayon/cesm/archive/CLM50SpGs_hcru_ecmwf/rest/1981-01-01-00000/CLM50SpGs_hcru_ecmwf.clm2.h0.1980-12.nc
WARNING: ncdump -v locfnh /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.clm2.rh0.1981-01-01-00000.nc failed rc=1
out=
err=ncdump: locfnh: No such variable
copying file /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.clm2.rh0.1981-01-01-00000.nc to /home/dayon/cesm/archive/CLM50SpGs_hcru_ecmwf/rest/1981-01-01-00000/CLM50SpGs_hcru_ecmwf.clm2.rh0.1981-01-01-00000.nc
Archiving restarts for mosart (rof)
copying file /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.mosart.r.1981-01-01-00000.nc to /home/dayon/cesm/archive/CLM50SpGs_hcru_ecmwf/rest/1981-01-01-00000/CLM50SpGs_hcru_ecmwf.mosart.r.1981-01-01-00000.nc
Copying /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.mosart.h0.1980-12.nc to /home/dayon/cesm/archive/CLM50SpGs_hcru_ecmwf/rest/1981-01-01-00000/CLM50SpGs_hcru_ecmwf.mosart.h0.1980-12.nc
WARNING: ncdump -v locfnh /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.mosart.rh0.1981-01-01-00000.nc failed rc=1
out=
err=ncdump: locfnh: No such variable
copying file /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.mosart.rh0.1981-01-01-00000.nc to /home/dayon/cesm/archive/CLM50SpGs_hcru_ecmwf/rest/1981-01-01-00000/CLM50SpGs_hcru_ecmwf.mosart.rh0.1981-01-01-00000.nc
Archiving restarts for drv (cpl)
copying file /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.cpl.r.1981-01-01-00000.nc to /home/dayon/cesm/archive/CLM50SpGs_hcru_ecmwf/rest/1981-01-01-00000/CLM50SpGs_hcru_ecmwf.cpl.r.1981-01-01-00000.nc
Archiving restarts for dart (esp)
Archiving history files for datm (atm)
Archiving history files for clm (lnd)
moving /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.clm2.h0.1980-02.nc to /home/dayon/cesm/archive/CLM50SpGs_hcru_ecmwf/lnd/hist/CLM50SpGs_hcru_ecmwf.clm2.h0.1980-02.nc
moving /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.clm2.h0.1980-08.nc to /home/dayon/cesm/archive/CLM50SpGs_hcru_ecmwf/lnd/hist/CLM50SpGs_hcru_ecmwf.clm2.h0.1980-08.nc
moving /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.clm2.h0.1980-01.nc to /home/dayon/cesm/archive/CLM50SpGs_hcru_ecmwf/lnd/hist/CLM50SpGs_hcru_ecmwf.clm2.h0.1980-01.nc
moving /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.clm2.h0.1980-03.nc to /home/dayon/cesm/archive/CLM50SpGs_hcru_ecmwf/lnd/hist/CLM50SpGs_hcru_ecmwf.clm2.h0.1980-03.nc
moving /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.clm2.h0.1980-07.nc to /home/dayon/cesm/archive/CLM50SpGs_hcru_ecmwf/lnd/hist/CLM50SpGs_hcru_ecmwf.clm2.h0.1980-07.nc
moving /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.clm2.h0.1980-05.nc to /home/dayon/cesm/archive/CLM50SpGs_hcru_ecmwf/lnd/hist/CLM50SpGs_hcru_ecmwf.clm2.h0.1980-05.nc
moving /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.clm2.h0.1980-10.nc to /home/dayon/cesm/archive/CLM50SpGs_hcru_ecmwf/lnd/hist/CLM50SpGs_hcru_ecmwf.clm2.h0.1980-10.nc
moving /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.clm2.h0.1980-04.nc to /home/dayon/cesm/archive/CLM50SpGs_hcru_ecmwf/lnd/hist/CLM50SpGs_hcru_ecmwf.clm2.h0.1980-04.nc
moving /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.clm2.h0.1980-06.nc to /home/dayon/cesm/archive/CLM50SpGs_hcru_ecmwf/lnd/hist/CLM50SpGs_hcru_ecmwf.clm2.h0.1980-06.nc
moving /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.clm2.h0.1980-11.nc to /home/dayon/cesm/archive/CLM50SpGs_hcru_ecmwf/lnd/hist/CLM50SpGs_hcru_ecmwf.clm2.h0.1980-11.nc
copying /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.clm2.h0.1980-12.nc to /home/dayon/cesm/archive/CLM50SpGs_hcru_ecmwf/lnd/hist/CLM50SpGs_hcru_ecmwf.clm2.h0.1980-12.nc
moving /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.clm2.h0.1980-09.nc to /home/dayon/cesm/archive/CLM50SpGs_hcru_ecmwf/lnd/hist/CLM50SpGs_hcru_ecmwf.clm2.h0.1980-09.nc
Archiving history files for mosart (rof)
moving /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.mosart.h0.1980-04.nc to /home/dayon/cesm/archive/CLM50SpGs_hcru_ecmwf/rof/hist/CLM50SpGs_hcru_ecmwf.mosart.h0.1980-04.nc
moving /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.mosart.h0.1980-02.nc to /home/dayon/cesm/archive/CLM50SpGs_hcru_ecmwf/rof/hist/CLM50SpGs_hcru_ecmwf.mosart.h0.1980-02.nc
moving /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.mosart.h0.1980-05.nc to /home/dayon/cesm/archive/CLM50SpGs_hcru_ecmwf/rof/hist/CLM50SpGs_hcru_ecmwf.mosart.h0.1980-05.nc
moving /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.mosart.h0.1980-09.nc to /home/dayon/cesm/archive/CLM50SpGs_hcru_ecmwf/rof/hist/CLM50SpGs_hcru_ecmwf.mosart.h0.1980-09.nc
moving /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.mosart.h0.1980-08.nc to /home/dayon/cesm/archive/CLM50SpGs_hcru_ecmwf/rof/hist/CLM50SpGs_hcru_ecmwf.mosart.h0.1980-08.nc
moving /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.mosart.h0.1980-06.nc to /home/dayon/cesm/archive/CLM50SpGs_hcru_ecmwf/rof/hist/CLM50SpGs_hcru_ecmwf.mosart.h0.1980-06.nc
copying /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.mosart.h0.1980-12.nc to /home/dayon/cesm/archive/CLM50SpGs_hcru_ecmwf/rof/hist/CLM50SpGs_hcru_ecmwf.mosart.h0.1980-12.nc
moving /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.mosart.h0.1980-03.nc to /home/dayon/cesm/archive/CLM50SpGs_hcru_ecmwf/rof/hist/CLM50SpGs_hcru_ecmwf.mosart.h0.1980-03.nc
moving /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.mosart.h0.1980-11.nc to /home/dayon/cesm/archive/CLM50SpGs_hcru_ecmwf/rof/hist/CLM50SpGs_hcru_ecmwf.mosart.h0.1980-11.nc
moving /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.mosart.h0.1980-01.nc to /home/dayon/cesm/archive/CLM50SpGs_hcru_ecmwf/rof/hist/CLM50SpGs_hcru_ecmwf.mosart.h0.1980-01.nc
moving /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.mosart.h0.1980-10.nc to /home/dayon/cesm/archive/CLM50SpGs_hcru_ecmwf/rof/hist/CLM50SpGs_hcru_ecmwf.mosart.h0.1980-10.nc
moving /home/dayon/cesm/scratch/CLM50SpGs_hcru_ecmwf/run/CLM50SpGs_hcru_ecmwf.mosart.h0.1980-07.nc to /home/dayon/cesm/archive/CLM50SpGs_hcru_ecmwf/rof/hist/CLM50SpGs_hcru_ecmwf.mosart.h0.1980-07.nc
Archiving history files for drv (cpl)
Archiving history files for dart (esp)
st_archive completed
ERROR: No result from jobs [('case.run', None), ('case.st_archive', 'case.run or case.test')]
解析:
根据模型设置,3个月出一次 restart 文件,因此1980-01-01开始,restart 的频率应该是1980-04-01, 1980-07-01, 1980-10-01;其实这些文件再模型运行过程中是产生了,在 run 文件夹下面。但从最后的 archiving (归档) 环节被removing(移除) 了,因此中间产生的 restart 文件也没能留存下来。只有模拟最后时刻 1981-01-01 的文件是被copy。
当然,有两种情况:
情况1:如果模型正常运行成功,可以不用保存中间产生的 restart 文件,只能保存最后一个时刻的 restart 文件即可 情况2:如果模型运行中断,但中间产生的 restart 文件 就仍然留存在 run 文件夹下面,可以在 debug 之后接着使用
此外,还有第3种例外,如果模型运行成功,你又想保存中间产生的 restart 文件,又当如何?
其实只需修改一个参数即可,如下。之后中间产生的 restart 文件 也都将记录在 archive/下面的 rest/ 文件夹下面。
cd $CASEROOT
./xmlchange DOUT_S_SAVE_INTERIM_RESTART_FILES="TRUE"