cdo | 常用命令

简朴-ocean

于 2024-05-31 23:03:30 发布

阅读量845

点赞数 22

分类专栏： python-海洋数据处理文章标签： java 人工智能算法

本文链接：https://blog.csdn.net/weixin_44237337/article/details/139363021

版权

python-海洋数据处理专栏收录该内容

72 篇文章 225 订阅

订阅专栏

整理一下平时经常会使用的cdo命令

如何来更改netcdf数据中的变量名呢？

假设我现在有一个sst月平均数据,希望将里面的变量名称sst修改为sst_new

netcdf oisst_monthly {
dimensions:
        lat = 180 ;
        lon = 360 ;
        time = UNLIMITED ; // (476 currently)
        nbnds = 2 ;
variables:
        float lat(lat) ;
                lat:units = "degrees_north" ;
                lat:long_name = "Latitude" ;
                lat:actual_range = 89.5f, -89.5f ;
                lat:standard_name = "latitude" ;
                lat:axis = "Y" ;
                lat:coordinate_defines = "center" ;
        float lon(lon) ;
                lon:units = "degrees_east" ;
                lon:long_name = "Longitude" ;
                lon:actual_range = 0.5f, 359.5f ;
                lon:standard_name = "longitude" ;
                lon:axis = "X" ;
                lon:coordinate_defines = "center" ;
        short sst(time, lat, lon) ;
                sst:long_name = "Monthly Mean of Sea Surface Temperature" ;
                sst:unpacked_valid_range = -5.f, 40.f ;
                sst:actual_range = -1.8f, 35.56862f ;
                sst:units = "degC" ;
                sst:add_offset = 0.f ;
                sst:scale_factor = 0.01f ;
                sst:missing_value = 32767s ;
                sst:precision = 2s ;
                sst:least_significant_digit = 2s ;
                sst:var_desc = "Sea Surface Temperature" ;
                sst:dataset = "NOAA Optimum Interpolation (OI) SST V2" ;
                sst:level_desc = "Surface" ;
                sst:statistic = "Mean" ;
                sst:parent_stat = "Weekly Mean" ;
                sst:standard_name = "sea_surface_temperature" ;
                sst:cell_methods = "time: mean (monthly from weekly values interpolated to daily)" ;
                sst:valid_range = -500s, 4000s ;

使用nco

ncrename -v sst,sst_new oisst_monthly.nc

修改结果,修改成功了

ncdump -h oisst_monthly.nc                                                                                                                                    
netcdf oisst_monthly {
dimensions:
        lat = 180 ;
        lon = 360 ;
        time = UNLIMITED ; // (476 currently)
        nbnds = 2 ;
variables:
        float lat(lat) ;
                lat:units = "degrees_north" ;
                lat:long_name = "Latitude" ;
                lat:actual_range = 89.5f, -89.5f ;
                lat:standard_name = "latitude" ;
                lat:axis = "Y" ;
                lat:coordinate_defines = "center" ;
        float lon(lon) ;
                lon:units = "degrees_east" ;
                lon:long_name = "Longitude" ;
                lon:actual_range = 0.5f, 359.5f ;
                lon:standard_name = "longitude" ;
                lon:axis = "X" ;
                lon:coordinate_defines = "center" ;
        short sst_new(time, lat, lon) ;
                sst_new:long_name = "Monthly Mean of Sea Surface Temperature" ;
                sst_new:unpacked_valid_range = -5.f, 40.f ;
                sst_new:actual_range = -1.8f, 35.56862f ;
                sst_new:units = "degC" ;
                sst_new:add_offset = 0.f ;
                sst_new:scale_factor = 0.01f ;
                sst_new:missing_value = 32767s ;
                sst_new:precision = 2s ;
                sst_new:least_significant_digit = 2s ;
                sst_new:var_desc = "Sea Surface Temperature" ;
                sst_new:dataset = "NOAA Optimum Interpolation (OI) SST V2" ;
                sst_new:level_desc = "Surface" ;
                sst_new:statistic = "Mean" ;
                sst_new:parent_stat = "Weekly Mean" ;
                sst_new:standard_name = "sea_surface_temperature" ;
                sst_new:cell_methods = "time: mean (monthly from weekly values interpolated to daily)" ;
                sst_new:valid_range = -500s, 4000s ;

当然，使用nco还可以修改属性的名称；下面是一个将 netcdf 文件中的 miss_value和 _FillValue 更改为零的示例:

ncatted -O -a missing_value, MITVAR,o,f,0 dasilva94_monthly_sst.nc
ncatted -O -a _FillValue,        MITVAR,o,f,0 dasilva94_monthly_sst.nc

其中，MITVAR是变量名，o 表示覆盖; f 表示浮点类型; 0是新值。

使用cdo

在刚刚修改为sst_new的基础上，使用cdo将其修改为sst_renew

cdo chname,sst_new,sst_renew oisst_monthly.nc oisst_monthly_renew.nc

修改结果

ncdump -h oisst_monthly_renew.nc                                                                                                                                  ✔  anaconda3  95% hdd   16:49:39
netcdf oisst_monthly_renew {
dimensions:
       time = UNLIMITED ; // (476 currently)
       bnds = 2 ;
       lon = 360 ;
       lat = 180 ;
variables:
       double time(time) ;
               time:standard_name = "time" ;
               time:long_name = "Time" ;
               time:bounds = "time_bnds" ;
               time:units = "days since 1800-1-1 00:00:00" ;
               time:calendar = "standard" ;
               time:axis = "T" ;
       double time_bnds(time, bnds) ;
       float lon(lon) ;
               lon:standard_name = "longitude" ;
               lon:long_name = "Longitude" ;
               lon:units = "degrees_east" ;
               lon:axis = "X" ;
       float lat(lat) ;
               lat:standard_name = "latitude" ;
               lat:long_name = "Latitude" ;
               lat:units = "degrees_north" ;
               lat:axis = "Y" ;
       short sst_renew(time, lat, lon) ;
               sst_renew:standard_name = "sea_surface_temperature" ;
               sst_renew:long_name = "Monthly Mean of Sea Surface Temperature" ;
               sst_renew:units = "degC" ;
               sst_renew:add_offset = 0.f ;
               sst_renew:scale_factor = 0.01f ;
               sst_renew:_FillValue = 32767s ;
               sst_renew:missing_value = 32767s ;
               sst_renew:unpacked_valid_range = -5.f, 40.f ;
               sst_renew:actual_range = -1.8f, 35.56862f ;
               sst_renew:precision = 2s ;
               sst_renew:least_significant_digit = 2s ;
               sst_renew:var_desc = "Sea Surface Temperature" ;
               sst_renew:dataset = "NOAA Optimum Interpolation (OI) SST V2" ;
               sst_renew:level_desc = "Surface" ;
               sst_renew:statistic = "Mean" ;
               sst_renew:parent_stat = "Weekly Mean" ;
               sst_renew:cell_methods = "time: mean (monthly from weekly values interpolated to daily)" ;
               sst_renew:institution = "NCEP" ;

经过测试，对于变量名称的修改，cdo的速度要明显快于nco(在我这个测试个例里面);当然，nco的修改结果还是在原本的数据里面，而cdo的修改结果保留在一个新的nc文件里面

如何使用wget从ftp或者网页直接下载数据呢？

通过写一个shell脚本来实现

Index of /data/sea-surface-temperature-optimum-interpolation/v2.1/access/avhrr

https://www.ncei.noaa.gov/data/sea-surface-temperature-optimum-interpolation/v2.1/access/avhrr/

一个完整的数据为：https://www.ncei.noaa.gov/data/sea-surface-temperature-optimum-interpolation/v2.1/access/avhrr/202404/oisst-avhrr-v02r01.20240401.nc

可以发现其链接的组成很有规则，下载一个文件时，我们可以直接wget https://www.ncei.noaa.gov/data/sea-surface-temperature-optimum-interpolation/v2.1/access/avhrr/202404/oisst-avhrr-v02r01.20240401.nc

但是如果下载多个文件呢，就可以根据它有规则命名的属性来进行循环迭代下载

#!/bin/bash
# 循环遍历每个年份
for year in {2021..2021}; do
   # 循环遍历每个月份
   for month in {01..12}; do
       # 循环遍历每一天
       for day in {01..31}; do
           # 构建文件名
           filename="oisst-avhrr-v02r01.${year}${month}${day}.nc"
           # 构建下载链接
           url="https://www.ncei.noaa.gov/data/sea-surface-temperature-optimum-interpolation/v2.1/access/avhrr/${year}${month}/${filename}"
           # 使用 wget 下载文件
           wget -o ./get -c -nH  -c  -r -A nc "$url"
       done
   done
done

如何对于多个netcdf文件进行求和计算呢？

cdo enssum file1 file2 out.nc

或者

cdo enssum *_sst.nc out.nc

如何对于netcdf数据进行插值呢？

守恒插值 (适用于降水数据)


cdo remapcon,r180x90  pr.nc  pr_regrid_2x2.nc

双线性插值

cdo remapbic,r144x73 in.nc out_interp.nc

如何修改数据calendar的类型呢？

cdo setcalendar,standard ua_day_EC-Earth3_data.nc ua_day_EC-Earth3_data_stand.nc

如何合并多个netcdf文件呢?

将多个nc数据按照时间维度进行合并，屡试不爽的方法。当然最好是保证要合并的数据都在同一个文件夹内。

  cdo mergetime GPM*.nc GPM-2004-05-07.nc

也可以通过python基于xr.concat()进行合并，当时还是cdo一行命令好使呀。虽然合并后的数据可能会偏大。

如何对于netcdf进行时间滤波呢？

对于数据是月平均来说，如果要进行10个月的低通滤波，可以使用以下命令：

cdo lowpass,0.1 sst.anomaly.nc out.nc

对于数据是日平均来说，如果要进行30-60天的带通滤波

cdo bandpass,365/60,365/30 sst.day.ano.nc out.nc

注意数据中不能存在nan值或者缺测值

如何处理数据中存在的nan值或者缺测值？

将正常的数据转换为Nan值

cdo setmissval,nan input.nc output.nc

将Nan值转化为其他值

cdo setmissval,0 input.nc output.nc

https://dwang.wordpress.com/

https://code.mpimet.mpg.de/boards/2/topics/4057

https://code.mpimet.mpg.de/boards/2/topics/11084

python & cdo ：https://code.mpimet.mpg.de/projects/cdo/wiki/Cdo%7Brbpy%7D

cdo使用手册：http://www.idris.fr/media/ada/cdo.pdf