python 自动下载ERA5 netCDF4格式数据 INFO Request is queued

2024-05-21 11:18:46,271 INFO Welcome to the CDS
2024-05-21 11:18:46,289 INFO Sending request to https://cds.climate.copernicus.eu/api/v2/resources/reanalysis-era5-pressure-levels
2024-05-21 11:18:46,512 INFO Request is completed
2024-05-21 11:18:46,543 INFO Downloading https://download-0013-clone.copernicus-climate.eu/cache-compute-0013/cache/data2/adaptor.mars.internal-1716235630.3098733-25379-2-ba28b4a9-58c9-426e-b468-9288b408f697.nc to D:\ERA5\vertical_velovity\925\2024040821.nc (5K)
  0%|          | 0.00/5.04k [00:00<?, ?B/s]                                             2024-05-21 11:18:47,365 INFO Download rate 6.3K/s
2024-05-21 11:18:47,772 INFO Welcome to the CDS
2024-05-21 11:18:47,780 INFO Sending request to https://cds.climate.copernicus.eu/api/v2/resources/reanalysis-era5-pressure-levels
2024-05-21 11:18:47,978 INFO Request is queued

从日志提示信息显示看，请求pending，接口服务器存在问题，相当于去窗口处理事情，窗口等待。

存在的问题：代码不报错，一直pending，对于自动下载数据时间成本较大，

解决思路：自动跳过排队等待时间长的时刻，并记录到缺测文件夹

解决方案： 

在D:\python\Lib\site-packages\cdsapi，目录下找到安装包的源代码，修改request请求响应时间找到timeout，retry_max 参数，源码中设置的为60，500.经过实验将其改成0.4，5在响应时间与排队等待时间会相对较优。（注意，响应时间变小以后会造成访问网站频次增加，对网站造成压力，可能会有封号的风险）

代码：

import cdsapi
import os

c = cdsapi.Client()

#参数设置，均为字符串列表
year = '2024'
month = '04'
day = ['08']                                 #日期列表
time = ['21:00', '22:00', '23:00']           #hourly列表
variable = ['vertical_velovity']             #变量列表
pressure_level =['925']                      #高度层列表 

#数据下载
for v in variable:
    for l in pressure_level:
        fo_path = 'D:\\ERA5\\'+v+'\\'+l      #文件存储路径设置
        folder = os.path.exists(fo_path)
        if not folder:
            os.makedirs(fo_path)
        for d in day:
            for t in time:
                try:
                    fi_path = fo_path+'\\'+year+month+d+t[0:2]+'.nc'    #文件命名
                    c.retrieve(
                    'reanalysis-era5-pressure-levels',
                    {
                        'product_type': 'reanalysis',
                        'format': 'netcdf',
                        'variable': v,
                        'pressure_level':l ,
                        'year': year,
                        'month': month,
                        'day':d, 
                        'time':t,
                        'area': [
                            35, 90, 0,
                            140,
                        ],
                        'grid': [1,1],
                    },
                    fi_path)
                except Exception as e: #响应时间过长，缺失文件记录
                    print(e)
                    fo_path = 'D:\\ERA5\\缺失'
                    folder = os.path.exists(fo_path)
                    if not folder:
                        os.makedirs(fo_path)
                    file_name = v+l+year+month+d+t[0:2]+'.nc'
                    fi_path = os.path.join(fo_path,file_name)
                    with open(fi_path,'w') as f:
                        print('文件缺失：',fi_path)
                    f.close()
                    

效果：