最近在做mapreduce系统的资源监控模块(项目全部基于erlang).
概要:监控各个map或者reduce的任务进程,每个进程都有各自的内存上限.
该监控系统提供进程监控注册,注册监控的进程在其内存超出其内存上限,该
监控系统则将告知mapreduce系统杀死进程.
在测试时,需要模拟进程不断消耗内存,于是根据如何搞垮erlang的文章(
见前一篇转载文章)的原理,利用消息共泛,不断给自己发送2阶乘的二进制数据,
使其内存不断增大,达到自身内存上限,使监控系统能够监控到.
具体代码片段如下:
consume_memory() ->
consume_memory(100000).
consume_memory(TimeCount) ->
Pid = spawn(fun() -> loop_receive() end),
send_msg(Pid, TimeCount),
{ok, Pid}.
send_msg(_Pid, 0) ->
ok;
send_msg(Pid, N) ->
Pid ! <<5:(N)>>,
timer:sleep(300),
send_msg(Pid, N-1).
loop_receive() ->
Result = erlang:process_info(self(), [memory]),
?INFO_F("comsumed memory : ~w ~n", [Result]),
receive
_ ->
loop_receive()
end.
测试了一下不行,内存增加到5600Byte就不会继续增加了,发现loop_receive()
这边是递归调用,所以为了能够内存继续增加,于是做点小修改,使其不递归,
不调用同一个地址的loop_receive,改成去调用try语句.
代码如下(测试通过):
consume_memory() ->
consume_memory(100000).
consume_memory(TimeCount) ->
Pid = spawn(fun() -> loop_receive() end),
send_msg(Pid, TimeCount),
{ok, Pid}.
send_msg(_Pid, 0) ->
ok;
send_msg(Pid, N) ->
Pid ! <<5:(N)>>,
timer:sleep(300),
send_msg(Pid, N-1).
loop_receive_old() ->
Result = erlang:process_info(self(), [memory]),
?INFO_F("comsumed memory : ~w ~n", [Result]),
receive
_ ->
loop_receive_old()
end.
loop_receive() ->
Result = erlang:process_info(self(), [memory]),
?INFO_F("comsumed memory : ~w ~n", [Result]),
try
receive
_ ->
loop_receive()
end
catch
_:_ ->
loop_receive()
end.
ps: error
send_msg(Pid, TimeCount),
改成
spawn(fun() -> send_msg(Pid, TimeCount) end),
查看哪些进程占用内存最高
spawn(fun() -> etop:start([{output, text}, {interval, 5}, {lines, 20}, {sort, memory}]) end)
新开辟一个进程,不影响erlang shell输入,以输出text方式启动etop,刷新间隔5s,输出行数20,按照内存降序排序.