GCRM-IO
该软件对实现全局云解析模型 (GCRM) 模拟代码的 I/O 内核的PnetCDF方法的性能进行基准测试 。 GCRM 由科罗拉多州立大学开发。 GCRM 的 I/O 模块称为 GIO 库，由太平洋西北国家实验室开发。 GCRM和GIO是用Fortran90编写的。该软件包从GCRM中提取I/O内核并将其转换为C语言，以实现更灵活的参数设置（例如MPI进程数、最大级别）和缓冲区管理的动态内存分配。
编译
make
这个错误信息表明在链接阶段遇到了“多重定义”错误。具体来说，是变量 debug 在多个源文件中重复定义了。解决这个问题的方法之一是使用 extern 关键字声明全局变量，然后在一个源文件中定义它。
vim src/gio.h
:28
vim src/gcrm.h
:26
vim gcrm_param_init.c
:31
make
使用GCRM-IO
找到输入参数文件的文件输出路径修改为自己的输出路径。
grep "wkliao" . -nr
mpirun -n 4 ../run/gcrm_io zgrd.in
(base) fakerth@fakerth-IdeaCentre-GeekPro-17IRB:/opt/software/GCRM-IO-master/inputs$ mpirun -n 4 ../run/gcrm_io zgrd.in
while loop num_dumps=1 time_ZGrd=0.000000 gio_default_frequency=60
while loop num_dumps=2 time_ZGrd=60.000000 gio_default_frequency=60
GIO: ----------------------------------------------------------
GIO statistics: (cumulative across all MPI processes)
GIO: total_time_in_API 14.7271
GIO: time_in_nf_put_var 0.0000
GIO: time_in_nf_put_var_grid 0.0071
GIO: time_in_nf_put_att 0.0022
GIO: time_in_nf_def_dim 0.0170
GIO: time_in_nf_def_var 0.0008
GIO: time_in_update_time 0.0006
GIO: time_in_nf_create 0.0749
GIO: time_in_nf_open 0.1836
GIO: time_in_nf_close 0.0371
GIO: time_in_nf_enddef 0.0159
GIO: time_in_nf_inq_varid 0.0061
GIO: time_in_nf_inq_dimlen 0.0000
GIO: time_in_nf_iput 0.1884
GIO: time_in_nf_iget 0.0000
GIO: time_in_nf_wait 10.3141
GIO: time_in_API_copy 0.9216
GIO: time_in_avgs 0.1083
GIO: ----------------------------------------------------------
GIO: bytes_API_write (Bytes): 3654442380
GIO: bytes_API_write (MiB): 3485.1478
GIO: bytes_API_write (GiB): 3.4035
GIO: bandwidth for writes (MiB/sec): 946.5977
GIO: bandwidth for writes (GiB/sec): 0.9244
GIO: ----------------------------------------------------------
---- MPI file info used ----
MPI File Info: [ 0] key = cb_buffer_size, value = 16777216
MPI File Info: [ 1] key = romio_cb_read, value = enable
MPI File Info: [ 2] key = romio_cb_write, value = enable
MPI File Info: [ 3] key = cb_nodes, value = 1
MPI File Info: [ 4] key = romio_no_indep_rw, value = true
MPI File Info: [ 5] key = romio_cb_pfr, value = disable
MPI File Info: [ 6] key = romio_cb_fr_types, value = aar
MPI File Info: [ 7] key = romio_cb_fr_alignment, value = 1
MPI File Info: [ 8] key = romio_cb_ds_threshold, value = 0
MPI File Info: [ 9] key = romio_cb_alltoall, value = automatic
MPI File Info: [10] key = ind_rd_buffer_size, value = 4194304
MPI File Info: [11] key = ind_wr_buffer_size, value = 524288
MPI File Info: [12] key = romio_ds_read, value = automatic
MPI File Info: [13] key = romio_ds_write, value = disable
MPI File Info: [14] key = romio_synchronized_flush, value = disabled
MPI File Info: [15] key = romio_visibility_immediate, value = true
MPI File Info: [16] key = cb_config_list, value = *:1
MPI File Info: [17] key = romio_filesystem_type, value = UFS: Generic ROMIO driver for all UNIX-like file systems
MPI File Info: [18] key = mpi_memory_alloc_kinds, value = mpi,system
MPI File Info: [19] key = romio_aggregator_list, value = 0
MPI File Info: [20] key = striping_unit, value = 0
MPI File Info: [21] key = striping_factor, value = 0
MPI File Info: [22] key = start_iodevice, value = 0
MPI File Info: [23] key = nc_header_align_size, value = 0
MPI File Info: [24] key = nc_var_align_size, value = 0
MPI File Info: [25] key = nc_record_align_size, value = 0
MPI File Info: [26] key = nc_header_read_chunk_size, value = 262144
MPI File Info: [27] key = nc_in_place_swap, value = auto
MPI File Info: [28] key = nc_ibuf_size, value = 16777216
MPI File Info: [29] key = pnetcdf_subfiling, value = disable
MPI File Info: [30] key = nc_num_subfiles, value = 0
MPI File Info: [31] key = nc_hash_size_dim, value = 256
MPI File Info: [32] key = nc_hash_size_var, value = 256
MPI File Info: [33] key = nc_hash_size_gattr, value = 64
MPI File Info: [34] key = nc_hash_size_vattr, value = 8
==== gcrm-io-pnetcdf 1.0.0 released on RELEASE_DATE ====
---- Run-time parameters ---------------------------------
---- Number of processes = 4
---- level_max (global horizontal grid resolution) = 5
---- sbdmn_iota (see grid_params.h) = 1
---- level_glbl (see grid_params.h) = 2
---- km (number of vertical layers) = 256
---- cell_max (global number of cells) = 10242
---- im (local number of cells along i) = 18
---- jm (local number of cells along j) = 18
---- nsdm_glbl (global number of blocks) = 40
---- nsdm (local number of blocks) = 10
---- using physics variables = enabled
---- number of files written = 40
---- number of grid variables written = 16
---- number of field variables written = 74
---- number of snapshot dumps = 2
---- I/O method = nonblocking_collective
---- cdf_output_path = /home/fakerth/111/
---------------------------------------------------------------
Timing results (max among all processes)
init time= 0.09 sec
Max comp time= 3.70 sec
Max I/O time= 3.76 sec
finalize time= 0.05 sec
---------------------------------------------------------------
Write amount= 3654.51 MB = 3485.22 MiB
Read amount= 9.08 MB = 8.66 MiB
I/O amount= 3663.60 MB = 3493.88 MiB
= 3.66 GB = 3.41 GiB
I/O bandwidth= 974.18 MB/sec = 929.05 MiB/sec
= 0.97 GB/sec = 0.91 GiB/sec
---------------------------------------------------------------
memory MAX usage (among 4 procs) = 1579.44 MiB
memory MIN usage (among 4 procs) = 1579.42 MiB
memory AVG usage (among 4 procs) = 1579.43 MiB
mpi-tilt-io
此应用程序的目的是测试非连续环境下的底层MPI-IO和文件系统实现访问工作负载。应用程序在逻辑上将数据文件划分为密集的二维一组瓷砖，如下所示：
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
控制文件解释的参数如下：
- nr_tiles_x：x维度中的瓦片数（行）
- nr_tiles_y：y维度中的瓦片数（列）
- sz_tile_x：瓦片的x维中的元素数
- sz_tile_y：瓦片的y维度中的元素数
- sz_element：元素的大小（以字节为单位）
- overlap_x：x维度上相邻瓦片之间共享的元素数量
- overlap_y：y维度上相邻瓦片之间共享的元素数量
- filename—要操作的文件的名称
下载地址：
https://www.mcs.anl.gov/research/projects/pio-benchmark/code/mpi-tile-io-01022003.tgz
编译
修改Makefile中的mpi环境变量。
vim Makefile
make
使用mpi-tile-io
由于没有说明文档，使用参数列表可以从源码中看到。
mpirun -np 4 ./mpi-tile-io --nr_tiles_x 2 --nr_tiles_y 2 --sz_tile_x 256 --sz_tile_y 256 --sz_element 4096 --filename /home/fakerth/111/README --collective 1 --write_file 1
(base) fakerth@fakerth-IdeaCentre-GeekPro-17IRB:/opt/software/mpi-tile-io$ mpirun -np 4 ./mpi-tile-io --nr_tiles_x 2 --nr_tiles_y 2 --sz_tile_x 256 --sz_tile_y 256 --sz_element 4096 --filename /home/fakerth/111/README --collective 1 --write_file 1
# mpi-tile-io run on fakerth-IdeaCentre-GeekPro-17IRB
# 4 process(es) available, 4 used
# filename: /home/fakerth/111/README
# collective I/O on
# 0 byte header
# 512 x 512 element dataset, 4096 bytes per element
# 2 x 2 tiles, each tile is 256 x 256 elements
# tiles overlap by 0 elements in X, 0 elements in Y
# total file size is ~1024.00 Mbytes, 1 file(s) total.
# Times are total for all operations of the given type
# Open: min_t = 0.000861, max_t = 0.000863, mean_t = 0.000862, var_t = 0.000000
# Write: min_t = 1.114985, max_t = 1.114991, mean_t = 1.114989, var_t = 0.000000
# Close: min_t = 0.000070, max_t = 0.000426, mean_t = 0.000165, var_t = 0.000000
# Note: bandwidth values based on max_t (worst case)
Write Bandwidth = 918.393 Mbytes/sec
扫一扫
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
