2024年北京积分落户名单公布了，爬了两个多小时得到了所有数据，竟然有惊人的发现（附源码）(1)

获取落户名单

获取落户名单的Ajax请求返回的居然是HTML，想法比较惊奇。直接写个正则表达式，提取想要的数据。

private final static Pattern LIST_PATTERN = Pattern.compile(

“[^<]*?<td[>]?>(\S?)[^<]*?<td[>]?>(\S?)[^<]*?<td[>]?>(\d+)\-(\d+)[^<]*?<td[>]?>(\S*?)[^<]*?<td[>]?>(\S?)[^<]*?<td[>]?>[^{<]*?<a[\s\S]*?οnclick=“showDetails\(‘(\d+)’\)”>查看</a>[}<]?[^<]*?”);

private static List findPersonList() throws InterruptedException {

String url = “http://fuwu.rsj.beijing.gov.cn/jf2021integralpublic/settlePerson/tablePage”;

List personList = new ArrayList<>();

for (int page = 0; page <= 6040; page += 10) {

Map<String, String> params = new HashMap<>();

params.put(“name”, “”);

params.put(“rows”, “10”);

params.put(“page”, Integer.toString(page));

String result = HttpUtils.doPost(url, params);

Matcher matcher = LIST_PATTERN.matcher(result);

while (matcher.find()) {

Person person = new Person();

person.setNumber(matcher.group(1));

person.setName(matcher.group(2));

person.setYear(Integer.parseInt(matcher.group(3)));

person.setMonth(Integer.parseInt(matcher.group(4)));

person.setCompany(matcher.group(5));

person.setTotalScore(Double.parseDouble(matcher.group(6)));

person.setId(Integer.parseInt(matcher.group(7)));

personList.add(person);

}

log.info("page: {} ", page);

Thread.sleep(1000);

}

return personList;

}

获取积分详情

积分详情的Ajax请求返回也是HTML，直接写10个正则表达式，提取想要的数据。

private final static Pattern[] DETAIL_PATTERN_ARRAY = {

Pattern.compile(“合法稳定就业[^<]*?<td[>]*?>([\d\.\-]+)”),

Pattern.compile(“合法稳定住所[^<]*?<td[>]*?>([\d\.\-]+)”),

Pattern.compile(“教育背景[^<]*?<td[>]*?>([\d\.\-]+)”),

Pattern.compile(“扣除取得学历（学位）期间累计的居住及就业分值[^<]*?<td[>]*?>([\d\.\-]+)”),

Pattern.compile(“创新创业[^<]*?<td[>]*?>([\d\.\-]+)”),

Pattern.compile(“职住区域[^<]*?<td[>]*?>([\d\.\-]+)”),

Pattern.compile(“纳税[^<]*?<td[>]*?>([\d\.\-]+)”),

Pattern.compile(“年龄[^<]*?<td[>]*?>([\d\.\-]+)”),

Pattern.compile(“荣誉表彰[^<]*?<td[>]*?>([\d\.\-]+)”),

Pattern.compile(“守法记录[^<]*?<td[>]*?>([\d\.\-]+)”),

};

private static void enrichPersonList(List personList) throws InterruptedException {

String url = “http://fuwu.rsj.beijing.gov.cn/jf2021integralpublic/settlePerson/settlePersonDetails”;

for (int i = 0; i < personList.size(); i++) {

Person person = personList.get(i);

Map<String, String> params = new HashMap<>();

params.put(“id”, Integer.toString(person.getId()));

String result = HttpUtils.doPost(url, params);

double[] detailScore = new double[DETAIL_PATTERN_ARRAY.length];

for (int j = 0; j < DETAIL_PATTERN_ARRAY.length; j++) {

最后

由于篇幅限制，小编在此截出几张知识讲解的图解

239527151)]

[外链图片转存中…(img-RfD0Vqza-1714239527151)]

[外链图片转存中…(img-pfgozecR-1714239527151)]

[外链图片转存中…(img-bpobZ4bg-1714239527152)]

本文已被CODING开源项目：【一线大厂Java面试题解析+核心总结学习笔记+最新讲解视频+实战项目源码】收录