[Handbook]一键某察者新闻

Python Code

"""
@author: MR.N
@created: 2022/9/24 Sat. 23:45
@file: se_guancha_1
@version: 1.0
"""

from httpkit import *
from scrapy import Selector
# import re
from filtertags import *
import time
import os


def test(url='https://www.guancha.cn'):
    remote_task = RemoteTask(url=url)
    ret = []
    code = get_res_objects2(remote_task=remote_task, ret=ret, dtype=0)
    if code != 'success':
        return None
    content = ret[0]
    sel = Selector(text=content)
    # 新闻标题包含于具有特定特征码的h4标签
    links = sel.xpath('//ul/li/h4').getall()
    sources = []
    for link in links:
        # 包含新闻标题的A标签具有title属性
        if len(filter_html_tags(link)) > 4:
            sources.append(filter_html_tags(link).replace('  ', '').replace('\n', ''))
    if len(sources) < 1:
        print('err', content)
    return sources


if __name__ == '__main__':
    items = test()
    # print('\n\n'.join(items))
    for item in items:
        print('\n\n\t' + item)
        time.sleep(.6)

Example

Error

URLError / SSLEOFError

Description

“urllib.error.URLError: <urlopen error EOF occurred in violation of protocol (_ssl.c:1131)>” or “ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:1131)”

Cause(s)

1. 不稳定的网络状态
   2）HTTPS(SSL)握手失败拒绝服务
   3）urllib3不同版本的SSL配置错误

user@pc:~$ pip install urllib3==1.25.11