本来做前端的梦想结果阴差阳错要做一个爬虫工程师,世界上很多都是这样子,没有选择的权利或者说自己也不知道该做什么,如果有机会就试一下吧。
爬虫工程师必须要会的前端、后端py和java都要熟悉,然后逆向js等,于是好好努力吧。
首先学习了基本的请求接口和返回数据
"""
记录学习py爬虫经过 应用:百度翻译
"""
import json
from urllib.request import Request,urlopen
from urllib.parse import urlencode
import ssl
import requests
if __name__ == '__main__':
post_url = 'https://fanyi.baidu.com/sug'
headers = {
'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.100 Safari/537.36',
'Cookie':'BIDUPSID=B1E171E5F11854D63354F717A17E6393; PSTM=1581786928; BAIDUID=B1E171E5F11854D6A3DDB423905EBD8D:FG=1; REALTIME_TRANS_SWITCH=1; FANYI_WORD_SWITCH=1; HISTORY_SWITCH=1; SOUND_SPD_SWITCH=1; SOUND_PREFER_SWITCH=1; BDUSS_BFESS=3NxUlF-cXdhOER5cURNVlllfjZkbEJDSVBaUHprUHJ2cmVvNmJxblVoUmxNMXBmRVFBQUFBJCQAAAAAAAAAAAEAAAAuqz6Pc2s0ZHc4AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAGWmMl9lpjJfV; BDSFRCVID=dLCOJeC62uAmohcrOHwnhHcZaEubZUnTH6aohhipnmV4D24cW-0VEG0Pox8g0KubnX5kogKKKgOTHICF_2uxOjjg8UtVJeC6EG0Ptf8g0M5; H_BDCLCKID_SF=JR-t_KKhfCvjDbjRKITs5tIt-fTMetJyaR3Ua4bvWJ5TMCojQ6jC36LX-J5bXPv7KK5b2bo92J6KShPC-tns2lRyWtFHWqjG3m0fWJ3q3l02Vb5ae-t2yUoD0f5D0PRMW23i0h7mWpTTsxA45J7cM4IseboJLfT-0bc4KKJxbnLWeIJEjjCaD6J0eHLjqbQXHD3yWbRHK4o_KROvhjRih58gyxomtjjH-TQqoJc-KljtfRnS-PrfbfPuLtTqLUkqKCOE3--2BJI2eR7zqt7Y5fFfQttjQPTOfIkja-KEQJb0fR7TyURvbU47y-rm0q4Hb6b9BJcjfU5MSlcNLTjpQT8r5MDOK5OuJRLDVCt2fI-KMIv4btOEK4LS52Tea4raK4oJ3R7tKRA_Hn7zePChyf4pbt-qJfJgbT7u3-otLUnHjhnnW4QqD6Liqt6nBT5Ka2cWolRhLKJBOloN3jJzK4IkQN3T0UuO5bRiLRodbqoMDn3oyURJXp0nytbly5jtMgOBBJ0yQ4b4OR5JjxonDh83bG7MJUutfJufoC--tC02bP365IIhDJkt-q6KaC62aJ3B-hOvWJ5TDqnJQj3x06KXMMvt54v7KaQK0lvctn3cShn1hbjSQ4KObHQK-T3pBNcZ0l8K3l02V-bIe-t2ynLVQROTQtRMW20j0h7mWIQvsxA45J7cM4IseboJLfT-0bc4KKJxbnLWeIJEjjCKe5c0DHAfq6nfb5kXQb5H-bnEDnrmbtbsq4tehHRbWUJ9WDTm_Doa0R6ojbO4XU7bh4kn-n6K0RQayHTX-pPKKR7sjbTxh4c4jM3XjMbUe6Op3mkjbn7yfn02OP5PhlAhMP4syPRiKMRnWg5mKfA-b4ncjRcTehoM3xI8LNj405OTbIFO0KJzJCcjqR8Zj58WjTvP; BDORZ=B490B5EBF6F3CD402E515D22BCDA1598; delPer=0; PSINO=3; H_PS_PSSID=32657_32606_1458_32535_31660_32045_32675_32116_32618_32501; Hm_lvt_64ecd82404c51e03dc91cb9e8c025574=1598771174; Hm_lpvt_64ecd82404c51e03dc91cb9e8c025574=1598771174; __yjsv5_shitong=1.0_7_cb06f4e05cdef7fb4ceb8e323edc41300c9f_300_1598771173318_112.10.84.122_533d08af; yjs_js_security_passport=603125aa335f3e246156f371c688da91f64324be_1598771174_jsorigin: https://fanyi.baidu.comreferer: https://fanyi.baidu.com/translate?aldtype=16047&query=a&keyfrom=baidu&smartresult=dict&lang=auto2zh',
'x-requested-with':'XMLHttpRequest'
}
word = input('enter a word:')
data = {
'kw':word
}
response = requests.post(url=post_url,data=data,headers=headers)
dic_obj = response.json()
fileName = word +'.json'
print(dic_obj)
fp = open(fileName,'w',encoding='utf-8')
json.dump(dic_obj,fp=fp,ensure_ascii=False)
print('over')
写了第一个简单的百度翻译爬虫,至于java先放一放,毕竟py的学习资料比较多。