request下载gitlab工程文件

无奈公司gitlabAPI权限授权问题迟迟解决不了，于是就来模拟登陆了，上一篇就用的是selenium来登陆下载gitlab上的文件，但是比较臃肿，所以还是再次来用requests这个东西来解决吧。

首先参考这个文章:

https://yq.aliyun.com/articles/252128?spm=a2c4e.11155435.0.0.23ad75b5w4duc6

后来发现在gitlab上不行，始终都是报错:

Traceback (most recent call last):
  File "D:/Users/EX-FEIXIAOHUI001/PycharmProjects/untitled/requests_gitlab.py", line 47, in <module>
    response = session.post(session_url, headers=session_header, data=login_data)
  File "C:\Python27\lib\site-packages\requests\sessions.py", line 549, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "C:\Python27\lib\site-packages\requests\sessions.py", line 502, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Python27\lib\site-packages\requests\sessions.py", line 634, in send
    history = [resp for resp in gen] if allow_redirects else []
  File "C:\Python27\lib\site-packages\requests\sessions.py", line 134, in resolve_redirects
    raise TooManyRedirects('Exceeded %s redirects.' % self.max_redirects, response=resp)
requests.exceptions.TooManyRedirects: Exceeded 30 redirects.

给出的原因说是重定向次数太多了。

后来就去尝试加入这个：

'Referer':'http://pagit.paic.com.cn/'

居然成功了。referer本来是referrer这个单词，但是现在将错就错。这个东西是指当前请求要来自该地址。看了下网上的资料，觉得这个东西没啥大作用，就没有深入研究了。

最后附上源码:

import requests
import re

login_url = 'http://pagit.paic.com.cn/users/sign_in?redirect_to_referer=yes'
username = 'EX-FEIXIAOHUI001'
password = '*****'

user_headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.110 Safari/537.36',
    'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8',
    'Accept-Encoding': 'gzip, deflate',
    'Accept-Language': 'zh-CN,zh;q=0.8',
	'Content-Type': 'application/x-www-form-urlencoded'
}

session_header = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.110 Safari/537.36',
    'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8',
    'Accept-Encoding': 'gzip, deflate',
    'Accept-Language': 'zh-CN,zh;q=0.8',
	# 'Upgrade-Insecure-Requests':'1',
	'Referer':'http://pagit.paic.com.cn/'
}

session = requests.Session()
response = session.get(login_url,headers=user_headers)
pattern = re.compile(r'<input name="authenticity_token" type="hidden" value="(.*)" />')
# print response.content
authenticity_token = pattern.findall(response.content)[0]
login_data = {
    'utf8': '%E2%9C%93',
    'authenticity_token': authenticity_token,
	'username': username,
    'password': password
}
session_url = "http://pagit.paic.com.cn/users/auth/ldapmain/callback"
response = session.post(session_url, headers=session_header, data=login_data)

zip_url = 'http://pagit.paic.com.cn/EX-FEIXIAOHUI001/project_config/repository/archive.zip'
r = session.get(zip_url)
with open("demo.zip",'wb') as code:
	code.write(r.content)