爬虫杂记

1.用faks_useragent生成随机UserAgent,
这样就不用写一个useragent列表了

先下载fake-useragent
pip install fake-useragent

然后导入UserAgent
from fake_useragent import UserAgent
ua = UserAgent()
usere_agent = ua.random

如果出现 fake_useragent.errors.FakeUserAgentError: Maximum amount of retries reached 错误
就禁用缓存
 ua = UserAgent(use_cache_server=False)

2. 写爬虫要用到pycropto
   pip install pycrypto报错:
   error: Microsoft Visual C++ 14.0 is required. Get it with “Microsoft Visual C++ Build Tools”: http://landinghub.visualstudio.com/visual-cpp-build-tools
   网上百度说要安装Microsoft Visual C++
   在github上看到pycryptodemo可以替代pycrypto
   试了下果真可以
   只要 pip install pycryptodemo 就可以了…

3.在字符串中插入其他字符串

方法一
def insert_str():
    # 有一个字符串
    str_1 = '月圆星疏夜,/n'
    # 把字符串转为 list
    str_list = list(str