好吧,实际上涉及的更多,我们需要解析页面中要传递的值,即__EVENTVALIDATION,__VIEWSTATEGENERATOR等。我使用bs4在执行初始get后解析所需的值:import requests
from bs4 import BeautifulSoup
d = {"ctl00$ctl13": "ctl00$ctl13|ctl00$MainContent$physicianSearchView$btnSearch",
"ctl00$MainContent$physicianSearchView$txtLastName": "Train",
'ctl00$MainContent$physicianSearchView$txtFirstName': "Anthony",
'ctl00$MainContent$physicianSearchView$txtCity': "Calgary",
"__VIEWSTATEENCRYPTED":"",
'ctl00$MainContent$physicianSearchView$txtPostalCode': "",
'ctl00$MainContent$physicianSearchView$rblPractice': "",
'ctl00$MainContent$physicianSearchView$ddDiscipline': "",
'ctl00$MainContent$physicianSearchView$rblGender': "Male",
'ctl00$MainContent$physicianSearchView$txtPracticeInterests': "",
'ctl00$MainContent$physicianSearchView$ddApprovals': "",
'ctl00$MainContent$physicianSearchView$ddLanguage': "",
"__EVENTTARGET": "ctl00$MainContent$physicianSearchView$btnSearch",
"__EVENTARGUMENT": "",
'ctl00$MainContent$physicianSearchView$hfPrefetchUrl': "http://service.cpsa.ca/OnlineService/OnlineService.svc/Services/GetAlbertaCities?name=",
'ctl00$MainContent$physicianSearchView$hfRemoveUrl': "http://service.cpsa.ca/OnlineService/OnlineService.svc/Services/GetAlbertaCities?name=%QUERY",
'__ASYNCPOST': 'true'}
h ={ "X-MicrosoftAjax":"Delta = true",
"X-Requested-With":"XMLHttpRequest",
"User-Agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.87 Safari/537.36"
}
with requests.session() as s:
r = s.get("http://search.cpsa.ca/PhysicianSearch",headers=h)
soup = BeautifulSoup(r.content, "lxml")
ev = soup.select("#__EVENTVALIDATION" )[0]["value"]
vs = soup.select("#__VIEWSTATE")[0]["value"]
vsg = soup.select("#__VIEWSTATEGENERATOR")[0]["value"]
d["__EVENTVALIDATION"] = ev
d["__VIEWSTATEGENERATOR"] = vsg
d["__VIEWSTATE"] = vs
r = requests.post('http://search.cpsa.ca/PhysicianSearch', data=d,headers=h)
print(r.content)
现在,我们在内容末尾获取数据:
^{pr2}$
获得上述结果所需的最小值为:h ={ "X-MicrosoftAjax":"Delta = true",
"X-Requested-With":"XMLHttpRequest",
"User-Agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.87 Safari/537.36"
}
d = {"ctl00$ctl13": "ctl00$ctl13|ctl00$MainContent$physicianSearchView$btnSearch",
"ctl00$MainContent$physicianSearchView$txtLastName": "Train",
'ctl00$MainContent$physicianSearchView$txtFirstName': "Anthony",
'ctl00$MainContent$physicianSearchView$txtCity': "Calgary",
'ctl00$MainContent$physicianSearchView$rblGender': "Male",
"__EVENTTARGET": "ctl00$MainContent$physicianSearchView$btnSearch",
'ctl00$MainContent$physicianSearchView$hfPrefetchUrl': "http://service.cpsa.ca/OnlineService/OnlineService.svc/Services/GetAlbertaCities?name=",
'ctl00$MainContent$physicianSearchView$hfRemoveUrl': "http://service.cpsa.ca/OnlineService/OnlineService.svc/Services/GetAlbertaCities?name=%QUERY",
'__ASYNCPOST': 'true'}
with requests.session() as s:
r = s.get("http://search.cpsa.ca/PhysicianSearch",headers=h)
soup = BeautifulSoup(r.content,"lxml")
ev = soup.select("#__EVENTVALIDATION" )[0]["value"]
vs = soup.select("#__VIEWSTATE")[0]["value"]
d["__EVENTVALIDATION"] = ev
d["__VIEWSTATE"] = vs
r = requests.post('http://search.cpsa.ca/PhysicianSearch', data=d,headers=h)
print(r.content)
您的帖子缺少这些值,如果您删除我在中添加的任何值,您将看到与您自己的代码相同的输出:1|#||4|50|pageRedirect||%2fError.aspx%3faspxerrorpath%3d%2fPhysicianSearch|