python网页点击下载按钮_python：如何从下载按钮隐藏链接的网页下载数据？

When click the button shown below, I got a .csv file:

I want to do this automatically using python where I can specify the date etc.

I find here that one can use pandas pd.read_csv to read data from webpage, but first one need to get the right url. However in my case I don't know what the url is.

Besides, I also want to specify the date and the contract etc. myself.

Before asking, I actually tried to the dev tool, I still can't see the url, and I don't know how to make it programatic.

解决方案

The javascript exportData('excel') results in a form that is submitted. By using Chrome devtools and the Network panel, you can figure out the headers and the post data used, and then write a python script to submit an identical http request.

import requests

url = 'http://www.dce.com.cn/publicweb/quotesdata/exportMemberDealPosiQuotesData.html'

formdata = {

'memberDealPosiQuotes.variety':'a',

'memberDealPosiQuotes.trade_type':0,

'contract.contract_id':'all',

'contract.variety_id':'a',

'exportFlag':'excel',

}

response = requests.post(url, data=formdata)

filename = response.headers.get('Content-Disposition').split('=')[-1]

with open(filename, 'wb') as fp:

fp.write(response.content)

It's probably possible to find ways to modify the post data to fetch different data. Either by reverse engineering, by trial and error or by finding some documentation.

For example, you can include fields for year and date:

'year':2017,

'month':3,

'day':20