爬虫编码获取不同编码接口解决

最新推荐文章于 2021-01-17 09:30:00 发布

花村大地主

最新推荐文章于 2021-01-17 09:30:00 发布

阅读量648

点赞数 1

分类专栏： javascript

javascript 专栏收录该内容

27 篇文章 0 订阅

订阅专栏

解决办法

直接用 iconv-lite 模块进行转码。

iconv-lite 是一个进行编码转换的模块（node 默认编码 utf-8）。需要 decode 的编码必须是 Buffer 类型。

###使用axios

const iconv = require("iconv-lite");
const axios = require("axios");
axios.get(`url`, { responseType: "arraybuffer" }).then(function(response){
    let html = iconv.decode(response.data, "gb2312");
    console.log(html);
})

node http模块

const http = require("http");
const querystring = require("querystring");
const iconv = require("iconv-lite");
let postData = {
    username:"username"
};
let req = http.request({
    hostname: "xxxx.com",
    port: 80,
    path: "/pathname",
    method: "POST",
    headers: {
        "Content-Type": "application/x-www-form-urlencoded"
    }
}, (res) => {
    let chunks = [];
    res.on("data", (chunk) => {
        chunks.push(chunk);
    });
    res.on("end", () => {
        let html = iconv.decode(Buffer.concat(chunks), "gb2312");
    });
});
req.on("error", (e) => {
    console.error(`problem with request: ${e.message}`);
});

// write data to request body
req.write(querystring.stringify(postData));
req.end();

文章转自： https://daihaoxin.github.io/post/94238009.html