python requests fails with 500 401 while curl is ok

版权声明:本文为博主原创文章,未经博主允许不得转载。 https://blog.csdn.net/albertjone/article/details/88659958

python requests fails with 500 401 while curl is ok

总结for the impatients

背景:

想通过创建大量的bucket来测试bucket加载的速度,但又不想动后台,所以灵机一动想了个方法,用
python加chrome自带的curl生成方法(谢谢超哥教我这个方法)
在这里插入图片描述
这里就不截图了,网上借一张。
可是在将curl翻译到python后,在执行的时候报了500,看了后台直接报了Django的报错,对于问题
的排错一点帮助一点都不方便。
下面是排错:

[Tue Mar 19 11:12:09.892156 2019] [:error] [pid 20]   File "/var/lib/kolla/venv/lib/python2.7/site-packages/django/core/handlers/exception.py", line 41, in inner
[Tue Mar 19 11:12:09.892161 2019] [:error] [pid 20]     response = get_response(request)
[Tue Mar 19 11:12:09.892165 2019] [:error] [pid 20]   File "/var/lib/kolla/venv/lib/python2.7/site-packages/django/core/handlers/base.py", line 249, in _legacy_get_response
[Tue Mar 19 11:12:09.892170 2019] [:error] [pid 20]     response = self._get_response(request)
[Tue Mar 19 11:12:09.892174 2019] [:error] [pid 20]   File "/var/lib/kolla/venv/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, in _get_response
[Tue Mar 19 11:12:09.892179 2019] [:error] [pid 20]     response = self.process_exception_by_middleware(e, request)
[Tue Mar 19 11:12:09.892183 2019] [:error] [pid 20]   File "/var/lib/kolla/venv/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, in _get_response
[Tue Mar 19 11:12:09.892260 2019] [:error] [pid 20]     response = wrapped_callback(request, *callback_args, **callback_kwargs)
[Tue Mar 19 11:12:09.892267 2019] [:error] [pid 20]   File "/var/lib/kolla/venv/lib/python2.7/site-packages/django/views/generic/base.py", line 68, in view
[Tue Mar 19 11:12:09.892272 2019] [:error] [pid 20]     return self.dispatch(request, *args, **kwargs)
[Tue Mar 19 11:12:09.892276 2019] [:error] [pid 20]   File "/var/lib/kolla/venv/lib/python2.7/site-packages/django/views/generic/base.py", line 88, in dispatch
[Tue Mar 19 11:12:09.892281 2019] [:error] [pid 20]     return handler(request, *args, **kwargs)
[Tue Mar 19 11:12:09.892285 2019] [:error] [pid 20]   File "/var/lib/kolla/venv/lib/python2.7/site-packages/openstack_dashboard/api/rest/utils.py", line 141, in _wrapped
[Tue Mar 19 11:12:09.892297 2019] [:error] [pid 20]     if request.body:
[Tue Mar 19 11:12:09.892347 2019] [:error] [pid 20]   File "/var/lib/kolla/venv/lib/python2.7/site-packages/django/http/request.py", line 274, in body
[Tue Mar 19 11:12:09.892372 2019] [:error] [pid 20]     six.reraise(UnreadablePostError, UnreadablePostError(*e.args), sys.exc_info()[2])
[Tue Mar 19 11:12:09.892396 2019] [:error] [pid 20]   File "/var/lib/kolla/venv/lib/python2.7/site-packages/django/http/request.py", line 272, in body
[Tue Mar 19 11:12:09.892421 2019] [:error] [pid 20]     self._body = self.read()
[Tue Mar 19 11:12:09.892454 2019] [:error] [pid 20]   File "/var/lib/kolla/venv/lib/python2.7/site-packages/django/http/request.py", line 333, in read
[Tue Mar 19 11:12:09.892488 2019] [:error] [pid 20]     six.reraise(UnreadablePostError, UnreadablePostError(*e.args), sys.exc_info()[2])
[Tue Mar 19 11:12:09.892527 2019] [:error] [pid 20]   File "/var/lib/kolla/venv/lib/python2.7/site-packages/django/http/request.py", line 331, in read
[Tue Mar 19 11:12:09.892558 2019] [:error] [pid 20]     return self._stream.read(*args, **kwargs)
[Tue Mar 19 11:12:09.892592 2019] [:error] [pid 20]   File "/var/lib/kolla/venv/lib/python2.7/site-packages/django/core/handlers/wsgi.py", line 47, in read
[Tue Mar 19 11:12:09.892624 2019] [:error] [pid 20]     result = self.buffer + self._read_limited()
[Tue Mar 19 11:12:09.892668 2019] [:error] [pid 20]   File "/var/lib/kolla/venv/lib/python2.7/site-packages/django/core/handlers/wsgi.py", line 41, in _read_limited
[Tue Mar 19 11:12:09.892701 2019] [:error] [pid 20]     result = self.stream.read(size)
[Tue Mar 19 11:12:09.892732 2019] [:error] [pid 20] UnreadablePostError: request data read error

解决过程:

查了下资料,可能是因为上面中header的方法中少加了几个header,再核对一遍后确实没有问题。
在接下来的搜索中发现了一个非常方便的网站 Convert cURL command syntax to Python requests, Node.js, R, PHP, Strest, Go, JSON, and Rust code可以将curl 转化成 requests python, 对比后发现是少加了一个data。

work的代码

代码:

headers = {
    'Cookie': 'login_region="http://172.16.50.248:5000/v3"; login_domain=; OUTFOX_SEARCH_USER_ID_NCOO=503886389.7780545; recent_project=15181b5daee243b1a9e1ad98db93bab7; horizon_language=zh-cn; django_timezone=Asia%2FShanghai; horizon_pagesize=20; instance_log_length=35; csrftoken=0akLbYsVvAw6X4Z9TJh87wJ4zJa7VcHnkykFQq8zmtnh6F9jWbeHiW0BV4LfaVW0; sessionid=ni3t78vk6qsxk9rnlfhv2dhe3gqc270e',
    'Origin': 'https://172.16.50.251',
    'Accept-Encoding': 'gzip, deflate, br',
    'Accept-Language': 'en-US,en;q=0.9,zh-CN;q=0.8,zh;q=0.7',
    'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36',
    'Content-Type': 'application/json;charset=UTF-8',
    'Accept': 'application/json, text/plain, */*',
    'Referer': 'https://172.16.50.251/storage/containers/',
    'X-Requested-With': 'XMLHttpRequest',
    'Connection': 'keep-alive',
    'X-CSRFToken': '0akLbYsVvAw6X4Z9TJh87wJ4zJa7VcHnkykFQq8zmtnh6F9jWbeHiW0BV4LfaVW0'
}
# 之前没有加这个data
data = '{"is_public":false}'

for i in range(10, 111111):
    url = 'https://172.16.50.251/api/s3/buckets/obj_createtest_%s/metadata/' % str(i)
    rep = requests.post(url, headers=headers, verify=False, data=data)
    try:
        rep.raise_for_status()
    except Exception as e:
        print(e)

总结:

以后出现requests的各个方法出现报错可以先排查下是否是 header和data,param中少了什么东西。

展开阅读全文

没有更多推荐了,返回首页