Python HTTPBasicAuthHandler 的使用及手写服务端测试

最新推荐文章于 2024-09-11 11:23:14 发布

追风筝的程序员

最新推荐文章于 2024-09-11 11:23:14 发布

阅读量2k

点赞数

分类专栏： python 文章标签： python 爬虫 java servlet http

本文链接：https://blog.csdn.net/hai1124948893/article/details/89483149

版权

python 专栏收录该内容

3 篇文章 0 订阅

订阅专栏

今天看了崔庆才著的 <<Python3网络爬虫开发实战的>> 的3.1使用urllib

第三点高级用法

这里的网站弹出输入账号密码的提示框,需要使用

HTTPBasicAuthHandler 来处理这个页面才能继续爬虫,然而平时没看到那个网站是这样的,为了测试代码只好自己用javaweb写一个

package com.hai;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.util.Base64;

public class LoginServlet extends HttpServlet {
    @Override
    protected void service(HttpServletRequest request, HttpServletResponse response)
            throws ServletException, IOException {
        //如果未验证
        //authorization解码后格式 Basic username:password
        String authorization = request.getHeader("authorization");
        if (authorization != null) {
            String[] auth = authorization.split(" ");
            //此处只验证密码, 获取username:password 中的password
            String password = new String(Base64.getDecoder().decode(auth[1]), "utf-8").split(":")[1];
            System.out.println(password);
            if("abc".equals(password)){
                //验证成功
                response.setContentType("text/html;charset=utf-8");
                response.getWriter().println("验证成功");
                return;
            }
        }
        //返回401状态码并设置header authorization
        response.setStatus(401);
        response.setHeader("WWW-authenticate", "Basic Realm=\"test\"");
    }
}

将此类映射成路径 /login

python 爬虫部分

from urllib.request import HTTPPasswordMgrWithDefaultRealm, HTTPBasicAuthHandler, build_opener
from urllib.error import URLError

username = '123'
password = 'abc'
url = "http://127.0.0.1/login"

p = HTTPPasswordMgrWithDefaultRealm()
p.add_password(None, url, username, password)
auth_handler = HTTPBasicAuthHandler(p)
opener = build_opener(auth_handler)
try:
    result = opener.open(url)
    html = result.read().decode("utf-8")
    print(html)
except URLError as e:
    print(e.reason)

运行服务器后再运行py