2021年北京积分落户名单公布了，爬了两个多小时得到了所有数据，竟然有惊人的发现（附源码）

static class Person {

private int id;

private String number;

private String name;

private int year;

private int month;

private String company;

private double totalScore;

private double[] detailScore;

}

获取落户名单

获取落户名单的Ajax请求返回的居然是HTML，想法比较惊奇。直接写个正则表达式，提取想要的数据。

private final static Pattern LIST_PATTERN = Pattern.compile(

“[^<]*?<td[>]?>(\S?)[^<]*?<td[>]?>(\S?)[^<]*?<td[>]?>(\d+)\-(\d+)[^<]*?<td[>]?>(\S*?)[^<]*?<td[>]?>(\S?)[^<]*?<td[>]?>[^{<]*?<a[\s\S]*?οnclick=“showDetails\(‘(\d+)’\)”>查看</a>[}<]?[^<]*?”);

private static List findPersonList() throws InterruptedException {

String url = “http://fuwu.rsj.beijing.gov.cn/jf2021integralpublic/settlePerson/tablePage”;

List personList = new ArrayList<>();

for (int page = 0; page <= 6040; page += 10) {

Map<String, String> params = new HashMap<>();

params.put(“name”, “”);

params.put(“rows”, “10”);

params.put(“page”, Integer.toString(page));

String result = HttpUtils.doPost(url, params);

Matcher matcher = LIST_PATTERN.matcher(result);

while (matcher.find()) {

Person person = new Person();

person.setNumber(matcher.group(1));

person.setName(matcher.group(2));

person.setYear(Integer.parseInt(matcher.group(3)));

person.setMonth(Integer.parseInt(matcher.group(4)));

person.setCompany(matcher.group(5));

person.setTotalScore(Double.parseDouble(matcher.group(6)));

person.setId(Integer.parseInt(matcher.group(7)));

personList.add(person);

}

log.info("page: {} ", page);

Thread.sleep(1000);

}

return personList;

}

获取积分详情

积分详情的Ajax请求返回也是HTML，直接写10个正则表达式，提取想要的数据。

private final static Pattern[] DETAIL_PATTERN_ARRAY = {

Pattern.compile(“合法稳定就业[^<]*?<td[>]*?>([\d\.\-]+)”),

Pattern.compile(“合法稳定住所[^<]*?<td[>]*?>([\d\.\-]+)”),

Pattern.compile(“教育背景[^<]*?<td[>]*?>([\d\.\-]+)”),

Pattern.compile(“扣除取得学历（学位）期间累计的居住及就业分值[^<]*?<td[>]*?>([\d\.\-]+)”),

Pattern.compile(“创新创业[^<]*?<td[>]*?>([\d\.\-]+)”),

Pattern.compile(“职住区域[^<]*?<td[>]*?>([\d\.\-]+)”),

Pattern.compile(“纳税[^<]*?<td[>]*?>([\d\.\-]+)”),

Pattern.compile(“年龄[^<]*?<td[>]*?>([\d\.\-]+)”),

Pattern.compile(“荣誉表彰[^<]*?<td[>]*?>([\d\.\-]+)”),

最后

由于篇幅原因，就不多做展示了
[外链图片转存中…(img-8SrhMrgM-1719150815991)]

[外链图片转存中…(img-peBgZujX-1719150815991)]

[外链图片转存中…(img-0UAmVum6-1719150815991)]

由于篇幅原因，就不多做展示了