python manager ulimit_python测试脚本运行到3万多个子线程时出现`thread.error: start new thread`...

在公司服务器上运行了一个mosquitto server端,并在服务器上运行如下python脚本创建10万个moquitto_sub客户端(公司以前人写的)连接到服务器:

#!/usr/bin/env python

#coding=utf-8

import os

import threading

from time import ctime,sleep

import subprocess

process = []

def mqtt_sub_test(func):

# print "mosquitto thread %d" %(func)

# os.system("mosquitto_sub -h 120.237.96.34 -t rock_test_mosquitto -k 10")

user = "rock_mosquitto_id-3w-%d" %(func)

print user

p = subprocess.Popen(["./mosquitto_sub", "-h", "127.0.0.1", "-t", "llltest", "-k", "10"])

process.append(p)

print "process.append %d" %(func)

threads = []

for i in range(1,100000):

t = threading.Thread(target=mqtt_sub_test,args=(i,))

threads.append(t)

if __name__ == '__main__':

for t in threads:

t.setDaemon(True)

t.start()

sleep(0.01)

raw_input('q')

for sub in process:

print sub.pid

a =sub.kill()

if (a!= None):

print a

print "all over %s" %ctime()

但是当创建子线程数大概达到32300个的时候,程序报错:错误信息如下

Traceback (most recent call last):

File "mqtt_python_3w.py", line 45, in

t.start()

File "/usr/lib/python2.6/threading.py", line 474, in start

_start_new_thread(self.__bootstrap, ())

thread.error: can't start new thread

Python subprocess.Popen “OSError: [Errno 12] Cannot allocate memory”

此时运行系统命令诸如ls等,都会出现Cannot allocate memory的错误。

查看下系统内存,16G的系统内存使用了4G,内存是足够的,运行ulimit -a输出信息如下

$ ulimit -a

core file size (blocks, -c) 0

data seg size (kbytes, -d) unlimited

scheduling priority (-e) 20

file size (blocks, -f) unlimited

pending signals (-i) 16382

max locked memory (kbytes, -l) 64

max memory size (kbytes, -m) unlimited

open files (-n) 1024

pipe size (512 bytes, -p) 8

POSIX message queues (bytes, -q) 819200

real-time priority (-r) 0

stack size (kbytes, -s) 8192

cpu time (seconds, -t) unlimited

max user processes (-u) unlimited

virtual memory (kbytes, -v) unlimited

file locks (-x) unlimited

现在想解决的问题如下:

可以修改系统那些参数来扩展最大线程数?

或者说如何写测试程序

诸位前辈的任何回答将不甚感激,谢谢了

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值