不具普遍性
1.
size of the communicator(5) is smalller than the size of Cartesian topology(10)
原因:parallel.ini忘记改成跟切分的时候一致了。
2.
Fatal error in MPI_Sendrecv: Message truncated, error stack:
MPI_Sendrecv(230).................: MPI_Sendrecv(sbuf=0x39837f0, scount=5776, MPI_BYTE, dest=4, stag=4, rbuf=0x3984e90, rcount=5776, MPI_BYTE, src=2, rtag=4, MPI_COMM_WORLD, status=0x715430) failed
MPIDI_CH3U_Receive_data_found(129): Message from rank 2 and tag 4 truncated; 55696 bytes received but buffer size is 5776
原因:boundbox.ini输出错了,应该要保持和颗粒网格大小一致,可以手动运行到这一步,手动改,之后再想想咋解决一步完成。最主要就是还是传递的信息确实大小不对。
3.
/rebuild.sh line 11: patch: command not found
原因:由于有些计算节点没有patch工具包,解决方法在openfoam wmake编译时节点没有patch_imastrid的博客-CSDN博客
4.
boundary文件中有些边界没读到,是因为distributor是shared为关键词,make.sh文件中句子可以修改:
###--- Make boundary types. ---###
checkMesh | grep "singly connected\|shared edge" > tempfile1
5.
Time = 0
Courant Number mean: 3.01246e-05 max: 0.0365493
DILUPBiCG: Solving for Ux, Initial residual = nan, Final residual = nan, No Iterations 1001
DILUPBiCG: Solving for Uy, Initial residual = nan, Final residual = nan, No Iterations 1001
DILUPBiCG: Solving for Uz, Initial residual = nan, Final residual = nan, No Iterations 1001
GAMGPCG: Solving for p, Initial residual = nan, Final residual = nan, No Iterations 11
GAMGPCG: Solving for p, Initial residual = nan, Final residual = nan, No Iterations 11
[0]
[0]
[0] --> FOAM FATAL IO ERROR:
[0] wrong token type - expected Scalar, found on line 0 the word 'nan'
[0]
[0] [1] [4]
[4]
[4] --> FOAM FATAL IO ERROR: file: /ncsfs02/yyxie/parallel120/parallel120_test/processor0/system/data.solverPerformance.p at line 0.
[0]
[0] From function operator>>(Istream&, Scalar&)
[0] in file lnInclude/Scalar.C at line
[4] wrong token type - expected Scalar, found on line 0 the word 'nan'
[4]
[4] file: /ncsfs02/yyxie/parallel120/parallel120_test/processor4/system/data.solverPerformance.p at line 0.
[4]
[4] From function operator>>(Istream&, Scalar&)93.
[0]
FOAM parallel run exiting
[0]
原因:第一步就出问题,应该是前面设置就出错了。居然是节点的问题...k208的求解器不能求解PISO...具体不清楚是为啥,但是我换成别的节点就行了,比如k228等等
6.
Time = 0.303
Courant Number mean: 9.6689e+25 max: 2.61778e+32
DILUPBiCG: Solving for Ux, Initial residual = 1, Final residual = 0.000786645, No Iterations 67
DILUPBiCG: Solving for Uy, Initial residual = 1, Final residual = 0.000918154, No Iterations 66
DILUPBiCG: Solving for Uz, Initial residual = 1, Final residual = 0.000712793, No Iterations 67
GAMGPCG: Solving for p, Initial residual = 1, Final residual = nan, No Iterations 11
GAMGPCG: Solving for p, Initial residual = nan, Final residual = nan, No Iterations 11
[2]
[2]
[8]
[8]
[8] --> FOAM FATAL IO ERROR:
[8] wrong token type - expected Scalar, found on line 0 the word 'nan'
[6] [1] [0] [2] --> FOAM FATAL IO ERROR:
[2] wrong token type - expected Scalar, found on line 0 the word 'nan'
[2]
[2] file: /ncsfs02/yyxie/parallel120/parallel120/processor2/system/data.solverPerformance.p at line 0.
[2]
[2] From function operator>>(Istream&, Scalar&)
[2] [9]
[9]
[9] --> FOAM FATAL IO ERROR:
[9] wrong token type - expected Scalar, found on line 0 the word 'nan'
[9]
[9] file: /ncsfs02/yyxie/parallel120/parallel120/processor9/system/data.solverPerformance.p at line 0.
[9]
[9] From function operator>>(Istream&, Scalar&)
[9] in file lnInclude/Scalar.C at line 93.
[9]
FOAM parallel run exiting
[9]
原因:这个是算到某一步就是nan了。。。应该是不收敛。正在思考怎么解决。
发现换成别的节点,0时刻的计算都会不一样。而且没有开头的报错句子:
sh: numactl: command not found
sh: numactl: command not found
sh: numactl: command not found
由于5个进程,3个在k208跑的,2个在183跑。应该是k208没这个command,因此有可能不收敛和它有关系。好像不是这个问题,救命。而且这个command not found现在在任何节点也没有报了。。。可能又是节点暂时抽风吧。
一些影响收敛的总结,最后有不收敛的一些可能因素:
openfoam串行收敛,并行不收敛_imastrid的博客-CSDN博客
发现是solid stress的参数没设置对,pstar改成10能收敛。还需要搜更多文献来判断一般用什么值好。也不是这个原因。。。。。继续排查
最后总结:基本都是因为颗粒堆积(即颗粒太多或者速度不够)或者节点抽风问题。
最终问题已经排查出来了:这些颗粒堆积主要是因为,并行边界的流体网格和颗粒网格不匹配,使得边界颗粒分配错区域,然后又没有被删掉。
7.
想要分多几块区域并行计算,想提高计算速度,但是运行总出现以下问题:
Starting time loop
Time = 0
Courant Number mean: 8.2246e-05 max: 0.0997866
rank: 4, Number of outside parcels is: 0
rank: 5, Number of outside parcels is: 0
rank: 0, Number of outside parcels is: 0
rank: 1, Number of outside parcels is: 0
rank: 3, Number of outside parcels is: 0
rank: 2, Number of outside parcels is: 0
DILUPBiCG: Solving for Ux, Initial residual = 1, Final residual = 1.40667e-11, No Iterations 1
DILUPBiCG: Solving for Uy, Initial residual = 1, Final residual = 1.13372e-11, No Iterations 1
DILUPBiCG: Solving for Uz, Initial residual = 1, Final residual = 1.16978e-11, No Iterations 1
GAMGPCG: Solving for p, Initial residual = 1, Final residual = 5.76869e-05, No Iterations 7
GAMGPCG: Solving for p, Initial residual = 0.111191, Final residual = 6.28914e-05, No Iterations 2
time step continuity errors : sum local = 5.5144e-06, global = 1.90473e-09, cumulative = 1.90473e-09
GAMGPCG: Solving for p, Initial residual = 0.0188875, Final residual = 2.26998e-05, No Iterations 3
GAMGPCG: Solving for p, Initial residual = 0.00571711, Final residual = 4.39705e-07, No Iterations 3
time step continuity errors : sum local = 4.29642e-08, global = -4.2329e-11, cumulative = 1.8624e-09
[1] #0 Foam::error::printStack(Foam::Ostream&)sh: addr2line: command not found
addr2line failed
[1] #1 Foam::sigSegv::sigHandler(int)sh: addr2line: command not found
addr2line failed
[1] #2 ?sh: addr2line: command not found
addr2line failed
[1] #3 sh: which: command not found
?
[1] #4 __libc_start_mainsh: addr2line: command not found
addr2line failed
[1] #5 sh: which: command not found
?
APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11)
或者
Starting time loop
Time = 0
Courant Number mean: 0.188764 max: 3.95571
rank: 4, Number of outside parcels is: 0
rank: 5, Number of outside parcels is: 0
rank: 0, Number of outside parcels is: 0
rank: 1, Number of outside parcels is: 0
rank: 3, Number of outside parcels is: 0
rank: 2, Number of outside parcels is: 0
DILUPBiCG: Solving for Ux, Initial residual = 1, Final residual = 8.13386e-05, No Iterations 3
DILUPBiCG: Solving for Uy, Initial residual = 1, Final residual = 0.000307587, No Iterations 3
DILUPBiCG: Solving for Uz, Initial residual = 1, Final residual = 8.44655e-05, No Iterations 3
GAMGPCG: Solving for p, Initial residual = 1, Final residual = 2.99442e-05, No Iterations 5
GAMGPCG: Solving for p, Initial residual = 0.263998, Final residual = 9.31053e-06, No Iterations 3
time step continuity errors : sum local = 8.98934e-07, global = -2.99344e-09, cumulative = -2.99344e-09
GAMGPCG: Solving for p, Initial residual = 0.0625925, Final residual = 1.21207e-05, No Iterations 4
GAMGPCG: Solving for p, Initial residual = 0.0745601, Final residual = 2.57919e-07, No Iterations 4
time step continuity errors : sum local = 2.60687e-08, global = 4.84023e-11, cumulative = -2.94504e-09
APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11)
都是:
APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11)
这句话,我不知道为啥。
目前排查,感觉主要是因为前处理画网格的时候就没把网格画好,在一些地方有不合格的网格,用这句检查:
checkMesh -allGeometry -allTopology
会出现:
有一句:Cells with small determinant(< 0.001) found, number of cells: 400
再运行这一句,然后用paraview查看这些cells在哪:
foamToVTK -cellSet underdeterminedCells
发现是在分布板那的网格,不知道这些网格被挑出来会怎么处理。感觉有可能和它们有关。
但也不一定。头大。因为,不在分布板这切割并行区域,它也不行。
还是同样的问题,出现在切分圆柱案例时,想不通为啥:
Parameters for reinserting.
XMIN = -0.127
YMIN = -0.127
ZMIN = 0
XMAX = 0.127
YMAX = 0.127
ZMAX = 0.1
U = 0
V = 0
W = 0
[0] #0 Foam::error::printStack(Foam::Ostream&) at ??:?
[0] #1 Foam::sigSegv::sigHandler(int) at ??:?
[0] #2 ? in "/lib64/libpthread.so.0"
[0] #3 ? at ??:?
[0] #4 ? at ??:?
[0] #5 ? at mppic/inter-phase/transParameters.H:15
[0] #6 __libc_start_main in "/lib64/libc.so.6"
[0] #7 ? at ??:?
APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11)
Parameters for reinserting.
XMIN = -0.127
YMIN = -0.127
ZMIN = 0
XMAX = 0.127
YMAX = 0.127
ZMAX = 0.1
U = 0
V = 0
W = 0
[2] #0 Foam::error::printStack(Foam::Ostream&)[1] #0 Foam::error::printStack(Foam::Ostream&) in "/opt/OpenFOAM/OpenFOAM-2.3.x/platforms/linux64IccDPOpt/lib/libOpenFOAM.so"
[2] #1 Foam::sigSegv::sigHandler(int) in "/opt/OpenFOAM/OpenFOAM-2.3.x/platforms/linux64IccDPOpt/lib/libOpenFOAM.so"
[1] #1 Foam::sigSegv::sigHandler(int) in "/opt/OpenFOAM/OpenFOAM-2.3.x/platforms/linux64IccDPOpt/lib/libOpenFOAM.so"
[2] #2 ? in "/lib64/libpthread.so.0"
[2] #3 in "/opt/OpenFOAM/OpenFOAM-2.3.x/platforms/linux64IccDPOpt/lib/libOpenFOAM.so"
[1] #2 ?? in "/lib64/libpthread.so.0"
[1] #3 ? in "/ncsfs02/yyxie/testReactionMPI/Li-MPI/main"
[2] #4 in "/ncsfs02/yyxie/testReactionMPI/Li-MPI/main"
[1] #4 ?? in "/ncsfs02/yyxie/testReactionMPI/Li-MPI/main"
[2] #5 ? in "/ncsfs02/yyxie/testReactionMPI/Li-MPI/main"
[1] #5 ?[0] #0 Foam::error::printStack(Foam::Ostream&) in "/opt/OpenFOAM/OpenFOAM-2.3.x/platforms/linux64IccDPOpt/lib/libOpenFOAM.so"
[0] #1 Foam::sigSegv::sigHandler(int) in "/opt/OpenFOAM/OpenFOAM-2.3.x/platforms/linux64IccDPOpt/lib/libOpenFOAM.so"
[0] #2 ? in "/lib64/libpthread.so.0"
[0] #3 ? at mppic/inter-phase/transParameters.H:15
[2] #6 __libc_start_main at mppic/inter-phase/transParameters.H:15
[1] #6 __libc_start_main in "/lib64/libc.so.6"
[2] #7 ? in "/lib64/libc.so.6"
[1] #7 [3] #0 Foam::error::printStack(Foam::Ostream&)? in "/opt/OpenFOAM/OpenFOAM-2.3.x/platforms/linux64IccDPOpt/lib/libOpenFOAM.so"
[3] #1 Foam::sigSegv::sigHandler(int)[4] #0 Foam::error::printStack(Foam::Ostream&) in "/opt/OpenFOAM/OpenFOAM-2.3.x/platforms/linux64IccDPOpt/lib/libOpenFOAM.so"
[3] #2 ? in "/opt/OpenFOAM/OpenFOAM-2.3.x/platforms/linux64IccDPOpt/lib/libOpenFOAM.so"
[4] #1 Foam::sigSegv::sigHandler(int) in "/lib64/libpthread.so.0"
[3] #3 in "/ncsfs02/yyxie/testReactionMPI/Li-MPI/main"
[0] #4 ?? in "/opt/OpenFOAM/OpenFOAM-2.3.x/platforms/linux64IccDPOpt/lib/libOpenFOAM.so"
[4] #2 ? in "/lib64/libpthread.so.0"
[4] #3 in "/ncsfs02/yyxie/testReactionMPI/Li-MPI/main"
? in "/ncsfs02/yyxie/testReactionMPI/Li-MPI/main"
APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11)
过了大半年了,好像有点知道为啥了。在mapping的时候,有些区域的最大映射数已经超过我所设置的最大映射数了。
还是不行。。。。好有毒。大爷的