7行Python代码批量下载CMIP6数据

数据下载后是.sh文件,如果是Unix系统,那应该很方便,直接运行就可以下载,但是文件比较大,难以保证有问题后可以继续下载。建议用迅雷下载,要下载前得先把所有链接给拿出来。

我们先看看sh文件主要包含下载链接的内容:


#These are the embedded files to be downloaded
download_files="$(cat <<EOF--dataset.file.url.chksum_type.chksum
'cLeaf_Lmon_BCC-CSM2-MR_historical_r1i1p1f1_gn_185001-201412.nc' 'http://cmip.bcc.cma.cn/thredds/fileServer/cmip6_data/CMIP/BCC/BCC-CSM2-MR/historical/r1i1p1f1/Lmon/cLeaf/gn/v20181114/cLeaf_Lmon_BCC-CSM2-MR_historical_r1i1p1f1_gn_185001-201412.nc' 'SHA256' 'de1e094c0097fd247f4ba6dfe27a0e1c899a81fcfa62782bc1917e683d4c274d'
'cLitter_Lmon_BCC-CSM2-MR_historical_r1i1p1f1_gn_185001-201412.nc' 'http://cmip.bcc.cma.cn/thredds/fileServer/cmip6_data/CMIP/BCC/BCC-CSM2-MR/historical/r1i1p1f1/Lmon/cLitter/gn/v20181114/cLitter_Lmon_BCC-CSM2-MR_historical_r1i1p1f1_gn_185001-201412.nc' 'SHA256' '282f8db93910f11d2600f82198e9e08f7c7adb1dc7265a64a12c5c2293129f10'
'gpp_Lmon_BCC-CSM2-MR_historical_r1i1p1f1_gn_185001-201412.nc' 'http://cmip.bcc.cma.cn/thredds/fileServer/cmip6_data/CMIP/BCC/BCC-CSM2-MR/historical/r1i1p1f1/Lmon/gpp/gn/v20181114/gpp_Lmon_BCC-CSM2-MR_historical_r1i1p1f1_gn_185001-201412.nc' 'SHA256' 'a56380441459046c88d96a4f2f028ce060138848f90d7e88940186290dcb4c15'
'lai_Lmon_BCC-CSM2-MR_historical_r1i1p1f1_gn_185001-201412.nc' 'http://cmip.bcc.cma.cn/thredds/fileServer/cmip6_data/CMIP/BCC/BCC-CSM2-MR/historical/r1i1p1f1/Lmon/lai/gn/v20181114/lai_Lmon_BCC-CSM2-MR_historical_r1i1p1f1_gn_185001-201412.nc' 'SHA256' 'ec70585662a9f66ddf3868224d34544ba84bcb598c539860384cf3b14fad4701'
'npp_Lmon_BCC-CSM2-MR_historical_r1i1p1f1_gn_185001-201412.nc' 'http://cmip.bcc.cma.cn/thredds/fileServer/cmip6_data/CMIP/BCC/BCC-CSM2-MR/historical/r1i1p1f1/Lmon/npp/gn/v20181114/npp_Lmon_BCC-CSM2-MR_historical_r1i1p1f1_gn_185001-201412.nc' 'SHA256' 'ea9af7d6dceef5b9c319926970b23d5ddf78eac7a07e13376a7cccbd342eece7'
'cLeaf_Lmon_BCC-CSM2-MR_ssp126_r1i1p1f1_gn_201501-210012.nc' 'http://cmip.bcc.cma.cn/thredds/fileServer/cmip6_data/ScenarioMIP/BCC/BCC-CSM2-MR/ssp126/r1i1p1f1/Lmon/cLeaf/gn/v20190314/cLeaf_Lmon_BCC-CSM2-MR_ssp126_r1i1p1f1_gn_201501-210012.nc' 'SHA256' '4603bcebaf345d2afbab65c6cb09ffa80f16e9b6b9bf629c80b50ba5e1adc3e2'
'cLitter_Lmon_BCC-CSM2-MR_ssp126_r1i1p1f1_gn_201501-210012.nc' 'http://cmip.bcc.cma.cn/thredds/fileServer/cmip6_data/ScenarioMIP/BCC/BCC-CSM2-MR/ssp126/r1i1p1f1/Lmon/cLitter/gn/v20190314/cLitter_Lmon_BCC-CSM2-MR_ssp126_r1i1p1f1_gn_201501-210012.nc' 'SHA256' '4ed0e4a336b9a4a5c565cd6dc0e0fb2d6cdc308df7ca2e3129612c4b8b61b332'
'gpp_Lmon_BCC-CSM2-MR_ssp126_r1i1p1f1_gn_201501-210012.nc' 'http://cmip.bcc.cma.cn/thredds/fileServer/cmip6_data/ScenarioMIP/BCC/BCC-CSM2-MR/ssp126/r1i1p1f1/Lmon/gpp/gn/v20190314/gpp_Lmon_BCC-CSM2-MR_ssp126_r1i1p1f1_gn_201501-210012.nc' 'SHA256' '91a9afd8f44dab26795870b177806ea381e1d9dd58bca0b2bb3bbe53e568a456'
'lai_Lmon_BCC-CSM2-MR_ssp126_r1i1p1f1_gn_201501-210012.nc' 'http://cmip.bcc.cma.cn/thredds/fileServer/cmip6_data/ScenarioMIP/BCC/BCC-CSM2-MR/ssp126/r1i1p1f1/Lmon/lai/gn/v20190314/lai_Lmon_BCC-CSM2-MR_ssp126_r1i1p1f1_gn_201501-210012.nc' 'SHA256' 'c20f9d2f4b6200e273d8882472cba888bb40238b913ea1e54f0753c0999d14c6'
'npp_Lmon_BCC-CSM2-MR_ssp126_r1i1p1f1_gn_201501-210012.nc' 'http://cmip.bcc.cma.cn/thredds/fileServer/cmip6_data/ScenarioMIP/BCC/BCC-CSM2-MR/ssp126/r1i1p1f1/Lmon/npp/gn/v20190314/npp_Lmon_BCC-CSM2-MR_ssp126_r1i1p1f1_gn_201501-210012.nc' 'SHA256' '87d8b2e8a03e5772da3f2088fd46d1c3f6919e13e3a8bdecbfe42e57e7681cbb'
'cLeaf_Lmon_BCC-CSM2-MR_ssp245_r1i1p1f1_gn_201501-210012.nc' 'http://cmip.bcc.cma.cn/thredds/fileServer/cmip6_data/ScenarioMIP/BCC/BCC-CSM2-MR/ssp245/r1i1p1f1/Lmon/cLeaf/gn/v20190308/cLeaf_Lmon_BCC-CSM2-MR_ssp245_r1i1p1f1_gn_201501-210012.nc' 'SHA256' '8534630e3d56ac07353bf37442743dc307dac83766f609bd56c2f9aa603fc042'
'cLitter_Lmon_BCC-CSM2-MR_ssp245_r1i1p1f1_gn_201501-210012.nc' 'http://cmip.bcc.cma.cn/thredds/fileServer/cmip6_data/ScenarioMIP/BCC/BCC-CSM2-MR/ssp245/r1i1p1f1/Lmon/cLitter/gn/v20190308/cLitter_Lmon_BCC-CSM2-MR_ssp245_r1i1p1f1_gn_201501-210012.nc' 'SHA256' 'f605f5c9c3264e86671e90de4b9eb2bca39998738f4353282f43b36e14897bee'
'gpp_Lmon_BCC-CSM2-MR_ssp245_r1i1p1f1_gn_201501-210012.nc' 'http://cmip.bcc.cma.cn/thredds/fileServer/cmip6_data/ScenarioMIP/BCC/BCC-CSM2-MR/ssp245/r1i1p1f1/Lmon/gpp/gn/v20190308/gpp_Lmon_BCC-CSM2-MR_ssp245_r1i1p1f1_gn_201501-210012.nc' 'SHA256' '4a1906d949cfe7bb32aa1e98be1f9379106c468a19a6af59388f818c28dcaaae'
'lai_Lmon_BCC-CSM2-MR_ssp245_r1i1p1f1_gn_201501-210012.nc' 'http://cmip.bcc.cma.cn/thredds/fileServer/cmip6_data/ScenarioMIP/BCC/BCC-CSM2-MR/ssp245/r1i1p1f1/Lmon/lai/gn/v20190308/lai_Lmon_BCC-CSM2-MR_ssp245_r1i1p1f1_gn_201501-210012.nc' 'SHA256' '5c9dd0afdc821a4cf04e475427cfe0eb97f446541da2a56b709e13f979b378f9'
'npp_Lmon_BCC-CSM2-MR_ssp245_r1i1p1f1_gn_201501-210012.nc' 'http://cmip.bcc.cma.cn/thredds/fileServer/cmip6_data/ScenarioMIP/BCC/BCC-CSM2-MR/ssp245/r1i1p1f1/Lmon/npp/gn/v20190308/npp_Lmon_BCC-CSM2-MR_ssp245_r1i1p1f1_gn_201501-210012.nc' 'SHA256' 'f75d3efa206129a4a3ae83498b96181365ebf13c403d34e50ad138211e3fa35a'
'cLeaf_Lmon_BCC-CSM2-MR_ssp585_r1i1p1f1_gn_201501-210012.nc' 'http://cmip.bcc.cma.cn/thredds/fileServer/cmip6_data/ScenarioMIP/BCC/BCC-CSM2-MR/ssp585/r1i1p1f1/Lmon/cLeaf/gn/v20190308/cLeaf_Lmon_BCC-CSM2-MR_ssp585_r1i1p1f1_gn_201501-210012.nc' 'SHA256' 'd9ac55eaea69cc49a82825a9a8a33b0db478ed44716d9164be6b1dc5d82690a5'
'cLitter_Lmon_BCC-CSM2-MR_ssp585_r1i1p1f1_gn_201501-210012.nc' 'http://cmip.bcc.cma.cn/thredds/fileServer/cmip6_data/ScenarioMIP/BCC/BCC-CSM2-MR/ssp585/r1i1p1f1/Lmon/cLitter/gn/v20190308/cLitter_Lmon_BCC-CSM2-MR_ssp585_r1i1p1f1_gn_201501-210012.nc' 'SHA256' '88aff041ca1d3308c3091abc5b153942dbb302f1ef899667a98d2b566c4720fd'
'gpp_Lmon_BCC-CSM2-MR_ssp585_r1i1p1f1_gn_201501-210012.nc' 'http://cmip.bcc.cma.cn/thredds/fileServer/cmip6_data/ScenarioMIP/BCC/BCC-CSM2-MR/ssp585/r1i1p1f1/Lmon/gpp/gn/v20190308/gpp_Lmon_BCC-CSM2-MR_ssp585_r1i1p1f1_gn_201501-210012.nc' 'SHA256' 'c78f47640ce38a0eab737cf3bf83c95156de442b69a47ca75237bcf1091586c4'
'lai_Lmon_BCC-CSM2-MR_ssp585_r1i1p1f1_gn_201501-210012.nc' 'http://cmip.bcc.cma.cn/thredds/fileServer/cmip6_data/ScenarioMIP/BCC/BCC-CSM2-MR/ssp585/r1i1p1f1/Lmon/lai/gn/v20190308/lai_Lmon_BCC-CSM2-MR_ssp585_r1i1p1f1_gn_201501-210012.nc' 'SHA256' '7a32961e29157f9809971e0261f9c90085f504483682d9e4ec9bbe3db1593bc1'
'npp_Lmon_BCC-CSM2-MR_ssp585_r1i1p1f1_gn_201501-210012.nc' 'http://cmip.bcc.cma.cn/thredds/fileServer/cmip6_data/ScenarioMIP/BCC/BCC-CSM2-MR/ssp585/r1i1p1f1/Lmon/npp/gn/v20190308/npp_Lmon_BCC-CSM2-MR_ssp585_r1i1p1f1_gn_201501-210012.nc' 'SHA256' '889f38580eaebf1c1a4c85ca6b0205b4c8fc4040fcd613c08ccc341e34bcb7bb'
EOF--dataset.file.url.chksum_type.chksum
)"

然后把所有.nc的链接拷贝复制出来,如:

http://cmip.bcc.cma.cn/thredds/fileServer/cmip6_data/CMIP/BCC/BCC-CSM2-MR/historical/r1i1p1f1/Lmon/cLeaf/gn/v20181114/cLeaf_Lmon_BCC-CSM2-MR_historical_r1i1p1f1_gn_185001-201412.nc

如果链接少还好,一多就麻烦了,下面使用七行代码正则匹配出所有下载链接:

import re
f = open("wget-20220930092535.sh",'r')
content = f.read()
patten = "http://.*.nc"
result = re.findall(patten, content)
wf = open("result.txt",'w')
wf.write('\n'.join(result))

结果:

http://cmip.bcc.cma.cn/thredds/fileServer/cmip6_data/CMIP/BCC/BCC-CSM2-MR/historical/r1i1p1f1/Lmon/cLeaf/gn/v20181114/cLeaf_Lmon_BCC-CSM2-MR_historical_r1i1p1f1_gn_185001-201412.nc
http://cmip.bcc.cma.cn/thredds/fileServer/cmip6_data/CMIP/BCC/BCC-CSM2-MR/historical/r1i1p1f1/Lmon/cLitter/gn/v20181114/cLitter_Lmon_BCC-CSM2-MR_historical_r1i1p1f1_gn_185001-201412.nc
http://cmip.bcc.cma.cn/thredds/fileServer/cmip6_data/CMIP/BCC/BCC-CSM2-MR/historical/r1i1p1f1/Lmon/gpp/gn/v20181114/gpp_Lmon_BCC-CSM2-MR_historical_r1i1p1f1_gn_185001-201412.nc
http://cmip.bcc.cma.cn/thredds/fileServer/cmip6_data/CMIP/BCC/BCC-CSM2-MR/historical/r1i1p1f1/Lmon/lai/gn/v20181114/lai_Lmon_BCC-CSM2-MR_historical_r1i1p1f1_gn_185001-201412.nc
http://cmip.bcc.cma.cn/thredds/fileServer/cmip6_data/CMIP/BCC/BCC-CSM2-MR/historical/r1i1p1f1/Lmon/npp/gn/v20181114/npp_Lmon_BCC-CSM2-MR_historical_r1i1p1f1_gn_185001-201412.nc
http://cmip.bcc.cma.cn/thredds/fileServer/cmip6_data/ScenarioMIP/BCC/BCC-CSM2-MR/ssp126/r1i1p1f1/Lmon/cLeaf/gn/v20190314/cLeaf_Lmon_BCC-CSM2-MR_ssp126_r1i1p1f1_gn_201501-210012.nc
http://cmip.bcc.cma.cn/thredds/fileServer/cmip6_data/ScenarioMIP/BCC/BCC-CSM2-MR/ssp126/r1i1p1f1/Lmon/cLitter/gn/v20190314/cLitter_Lmon_BCC-CSM2-MR_ssp126_r1i1p1f1_gn_201501-210012.nc
http://cmip.bcc.cma.cn/thredds/fileServer/cmip6_data/ScenarioMIP/BCC/BCC-CSM2-MR/ssp126/r1i1p1f1/Lmon/gpp/gn/v20190314/gpp_Lmon_BCC-CSM2-MR_ssp126_r1i1p1f1_gn_201501-210012.nc
http://cmip.bcc.cma.cn/thredds/fileServer/cmip6_data/ScenarioMIP/BCC/BCC-CSM2-MR/ssp126/r1i1p1f1/Lmon/lai/gn/v20190314/lai_Lmon_BCC-CSM2-MR_ssp126_r1i1p1f1_gn_201501-210012.nc
http://cmip.bcc.cma.cn/thredds/fileServer/cmip6_data/ScenarioMIP/BCC/BCC-CSM2-MR/ssp126/r1i1p1f1/Lmon/npp/gn/v20190314/npp_Lmon_BCC-CSM2-MR_ssp126_r1i1p1f1_gn_201501-210012.nc
http://cmip.bcc.cma.cn/thredds/fileServer/cmip6_data/ScenarioMIP/BCC/BCC-CSM2-MR/ssp245/r1i1p1f1/Lmon/cLeaf/gn/v20190308/cLeaf_Lmon_BCC-CSM2-MR_ssp245_r1i1p1f1_gn_201501-210012.nc
http://cmip.bcc.cma.cn/thredds/fileServer/cmip6_data/ScenarioMIP/BCC/BCC-CSM2-MR/ssp245/r1i1p1f1/Lmon/cLitter/gn/v20190308/cLitter_Lmon_BCC-CSM2-MR_ssp245_r1i1p1f1_gn_201501-210012.nc
http://cmip.bcc.cma.cn/thredds/fileServer/cmip6_data/ScenarioMIP/BCC/BCC-CSM2-MR/ssp245/r1i1p1f1/Lmon/gpp/gn/v20190308/gpp_Lmon_BCC-CSM2-MR_ssp245_r1i1p1f1_gn_201501-210012.nc
http://cmip.bcc.cma.cn/thredds/fileServer/cmip6_data/ScenarioMIP/BCC/BCC-CSM2-MR/ssp245/r1i1p1f1/Lmon/lai/gn/v20190308/lai_Lmon_BCC-CSM2-MR_ssp245_r1i1p1f1_gn_201501-210012.nc
http://cmip.bcc.cma.cn/thredds/fileServer/cmip6_data/ScenarioMIP/BCC/BCC-CSM2-MR/ssp245/r1i1p1f1/Lmon/npp/gn/v20190308/npp_Lmon_BCC-CSM2-MR_ssp245_r1i1p1f1_gn_201501-210012.nc
http://cmip.bcc.cma.cn/thredds/fileServer/cmip6_data/ScenarioMIP/BCC/BCC-CSM2-MR/ssp585/r1i1p1f1/Lmon/cLeaf/gn/v20190308/cLeaf_Lmon_BCC-CSM2-MR_ssp585_r1i1p1f1_gn_201501-210012.nc
http://cmip.bcc.cma.cn/thredds/fileServer/cmip6_data/ScenarioMIP/BCC/BCC-CSM2-MR/ssp585/r1i1p1f1/Lmon/cLitter/gn/v20190308/cLitter_Lmon_BCC-CSM2-MR_ssp585_r1i1p1f1_gn_201501-210012.nc
http://cmip.bcc.cma.cn/thredds/fileServer/cmip6_data/ScenarioMIP/BCC/BCC-CSM2-MR/ssp585/r1i1p1f1/Lmon/gpp/gn/v20190308/gpp_Lmon_BCC-CSM2-MR_ssp585_r1i1p1f1_gn_201501-210012.nc
http://cmip.bcc.cma.cn/thredds/fileServer/cmip6_data/ScenarioMIP/BCC/BCC-CSM2-MR/ssp585/r1i1p1f1/Lmon/lai/gn/v20190308/lai_Lmon_BCC-CSM2-MR_ssp585_r1i1p1f1_gn_201501-210012.nc
http://cmip.bcc.cma.cn/thredds/fileServer/cmip6_data/ScenarioMIP/BCC/BCC-CSM2-MR/ssp585/r1i1p1f1/Lmon/npp/gn/v20190308/npp_Lmon_BCC-CSM2-MR_ssp585_r1i1p1f1_gn_201501-210012.nc

直接全部拷贝到迅雷中,就可以下载了

 

  • 2
    点赞
  • 7
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
使用Python处理CMIP 6数据可以通过以下步骤实现: 1. 获取数据:首先需要获取CMIP 6数据集。可以通过访问相应的数据仓库或使用Google引擎网站(如CERA-20C数据集)下载数据。 2. 导入数据:将数据文件导入Python环境中。可以使用Python内置的netCDF库或者第三方库(如xarray)来导入数据。这些库可以轻松处理netCDF格式的数据,提供了方便的数据访问和操作方法。 3. 数据预处理:根据需要,对数据预处理。例如,可以选择特定时间范围、降低数据的空间分辨率、进数据插补或填充缺失值等。 4. 数据分析和可视化:使用Python中的数据分析库(如pandas、numpy和scipy)进数据分析。可以计算统计量(如均值、标准差、相关性等)或进时空分析。 5. 数据存储:根据需要,将分析结果存储为新的数据文件。可以选择将数据保存为新的netCDF文件或其他格式(如CSV、Excel等)。 6. 数据可视化:使用Python中的可视化库(如matplotlib和seaborn)对数据可视化。可以绘制时间序列图、空间分布图、散点图等,以便更好地理解和呈现数据。 7. 数据分享:将处理和分析过的数据和可视化结果分享给其他人。可以将数据和结果发布在网站上、分享到学术论坛上或利用Jupyter Notebook创建交互式报告。 总之,使用Python处理CMIP 6数据可以帮助我们更好地理解和分析气候模型输出,为气候研究和决策提供有力的支持。

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值