webmagic百度指数查询

需求：本地一批关键词，需要查询百度指数，5118一次只能查100条，5W不得手软，看接口0.013元1条，网上找了百度指数接口，都不怎么便宜，那就自己动手了，用webmagic也是很方便快捷的。
1、配置
webmagic两个包、打包成jar可以挂在服务器慢慢运行。

    <dependencies>
        <!-- webmagic核心包 -->
        <dependency>
            <groupId>us.codecraft</groupId>
            <artifactId>webmagic-core</artifactId>
            <version>0.7.4</version>
        </dependency>
        <!-- webmagic扩展包 -->
        <dependency>
            <groupId>us.codecraft</groupId>
            <artifactId>webmagic-extension</artifactId>
            <version>0.7.4</version>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.12</version>
        </dependency>
    </dependencies>
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-assembly-plugin</artifactId>
                <version>2.5.5</version>
                <configuration>
                    <archive>
                        <manifest>
                            <mainClass>com.booy.task.OrderProcessor</mainClass>
                        </manifest>
                    </archive>
                    <descriptorRefs>
                        <descriptorRef>jar-with-dependencies</descriptorRef>
                    </descriptorRefs>
                </configuration>
            </plugin>
        </plugins>
    </build>

log4j日志文件：

log4j.rootLogger=INFO,A1 
log4j.appender.A1=org.apache.log4j.ConsoleAppender
log4j.appender.A1.layout=org.apache.log4j.PatternLayout
log4j.appender.A1.layout.ConversionPattern=%-d{yyyy-MM-dd HH:mm:ss,SSS} [%t] [%c]-[%p] %m%n

2、主类

public class BaiduZhishu implements PageProcessor {
   

    @Override
    public void process(Page page) {
   
        Document doc = page.getHtml().getDocument();
        String docStr = doc.toString();
        
        Pattern statusP = Pattern.compile("status\\\":\\d*?(?=,)");
        Matcher statusM = statusP.matcher(docStr);
        statusM.find();
        String statusGroup = statusM.group();
        int i = Integer.parseInt(statusGroup.substring(8, statusGroup.length()));

        String allZhishu = null