列车信息获取(一)——数据总览
第一次写博客，不知道怎么写。主要写的就是通过script的跨域访问去访问12306对外暴露的接口。然后取得车次信息并通过SSM框架保存在数据库中
开发环境:
JDK1.8+idea+SSM+httpclient4.5.3+Mysql5.7.16
获得的数据如下：
获得的数据大约有9000条。
1. 获得数据源的URL
这是获得12306上所有的车次信息。进入12306网站的车次查询页面.页面会自动加载信息,通过开发者模式获得URL.点开响应时可能会因为浏览器变慢.
2. 搭建开发环境
数据获取后要保存到数据库中,所以采用idea搭建SSM框架,实际是初学只会SSM.具体搭建就不展示了,博客上很多搭建SSM环境的教程.附上现在需要的pom文件.
<properties>
<!-- httpclient版本 -->
<httpclient.version>4.5.3</httpclient.version>
<!-- 阿里巴巴连接池版本 -->
<druid.version>1.0.18</druid.version>
<!-- apache组件版本 -->
<commons-lang3.version>3.4</commons-lang3.version>
<commons-net.version>3.3</commons-net.version>
<!-- spring版本 -->
<spring.version>4.3.4.RELEASE</spring.version>
<!-- MyBatis版本 -->
<mybatis.version>3.4.1</mybatis.version>
<!-- junit版本 -->
<junit.version>4.11</junit.version>
<!-- 日志包 -->
<slf4j.version>1.7.21</slf4j.version>
<!-- mysql版本 -->
<mysql.version>5.1.40</mysql.version>
<!-- jackson版本 -->
<jackson.version>2.7.3</jackson.version>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>2.7</version>
</dependency>
<!-- 阿里巴巴连接池 -->
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>druid</artifactId>
<version>${druid.version}</version>
</dependency>
<!-- MYSQL START -->
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>${mysql.version}</version>
</dependency>
<!-- MYSQL END -->
<!-- dbcp连接池的jar包. -->
<dependency>
<groupId>commons-dbcp</groupId>
<artifactId>commons-dbcp</artifactId>
<version>1.4</version>
</dependency>
<!-- MyBatis START -->
<dependency>
<groupId>org.mybatis</groupId>
<artifactId>mybatis</artifactId>
<version>${mybatis.version}</version>
</dependency>
<dependency>
<groupId>org.mybatis</groupId>
<artifactId>mybatis-spring</artifactId>
<version>1.3.0</version>
</dependency>
<dependency>
<groupId>org.mybatis.generator</groupId>
<artifactId>mybatis-generator-core</artifactId>
<version>1.3.2</version>
</dependency>
<dependency>
<groupId>org.mybatis.generator</groupId>
<artifactId>mybatis-generator-maven-plugin</artifactId>
<version>1.3.5</version>
</dependency>
<!-- MyBatis END -->
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>${httpclient.version}</version>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpcore</artifactId>
<version>4.4.4</version>
</dependency>
<!-- pagehelper分页类 -->
<dependency>
<groupId>com.github.pagehelper</groupId>
<artifactId>pagehelper</artifactId>
<version>${pagehelper.version}</version>
</dependency>
<!-- pagehelper配套需要的jar包 -->
<dependency>
<groupId>com.github.jsqlparser</groupId>
<artifactId>jsqlparser</artifactId>
<version>0.9.5</version>
</dependency>
<!-- junit 包 hamcrest-core自动加进来 -->
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>${junit.version}</version>
<scope>test</scope>
</dependency>
<!-- SPRING START -->
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-aop</artifactId>
<version>${spring.version}</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-beans</artifactId>
<version>${spring.version}</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-core</artifactId>
<version>${spring.version}</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-context</artifactId>
<version>${spring.version}</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-expression</artifactId>
<version>${spring.version}</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-aspects</artifactId>
<version>${spring.version}</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-web</artifactId>
<version>4.3.4.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-instrument</artifactId>
<version>${spring.version}</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-instrument-tomcat</artifactId>
<version>${spring.version}</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-jdbc</artifactId>
<version>${spring.version}</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-jms</artifactId>
<version>${spring.version}</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-orm</artifactId>
<version>${spring.version}</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-oxm</artifactId>
<version>${spring.version}</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-test</artifactId>
<version>${spring.version}</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-tx</artifactId>
<version>${spring.version}</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-webmvc</artifactId>
<version>${spring.version}</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-webmvc-portlet</artifactId>
<version>${spring.version}</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-websocket</artifactId>
<version>${spring.version}</version>
</dependency>
<!-- SPRING END -->
<!-- 日志 START -->
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>${slf4j.version}</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.7.21</version>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
</dependency>
<!-- 日志 END -->
<!-- JACKSON START -->
<!-- 格式化对象，方便输出日志 -->
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>fastjson</artifactId>
<version>1.2.7</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-core</artifactId>
<version>2.7.3</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-annotations</artifactId>
<version>2.7.3</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.7.3</version>
</dependency>
<!-- JACKSON END -->
<!-- Apache工具组件 -->
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>${commons-lang3.version}</version>
</dependency>
<dependency>
<groupId>commons-net</groupId>
<artifactId>commons-net</artifactId>
<version>${commons-net.version}</version>
</dependency>
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>2.4</version>
</dependency>
<dependency>
<groupId>commons-codec</groupId>
<artifactId>commons-codec</artifactId>
<version>1.10</version>
</dependency>
</dependencies>3. 获取数据
上面说到了从12306获得数据URL的格式为var train_list ={“2017-07-31”:{“D”:[{},{}]……,”G”:[{}{}]}},正因为如此我们可以使用script跨域访问.想了解的可以看: 通过script标签跨域、ajax跨域
<script src="https://kyfw.12306.cn/otn/resources/js/query/train_list.js?scriptVersion=1.0"></script>然后我们就可以在代码中使用上面JS中的train_list.train_list的数据是json格式.可以直接用jquery遍历.
下面附上整个HTML的代码:
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
<meta http-equiv="Access-Control-Allow-Origin" content="*">
<title>车次获取</title>
<link rel="stylesheet" href="/css/bootstrap.min.css"/>
<link rel="stylesheet" href="/bootstrap-table/bootstrap-table.css"/>
<script src="/js/jquery-2.2.3.min.js"></script>
<script src="/js/bootstrap.min.js"></script>
<script src="/js/vue.min.js"></script>
<script src="https://kyfw.12306.cn/otn/resources/js/query/train_list.js?scriptVersion=1.0"></script>
<script>
$(function () {
var trainlist = train_list;
var vm = new Vue({
el: "#app",
data: {
traval: '',
namelist: [],
},
watch: {},
methods: {
getalltrainline: function () {
$.each(trainlist, function (i, v) {
if (i == "2017-07-31") {
$.each(v, function (tou, che) {
//CDGKOTZ
if (tou == vm.traval) {
$.each(che, function (k, va) {
$.ajax({
url: "/train/addtrainline",
type: "POST",
data: {
trainno: va.train_no,
traintype: va.station_train_code.substring(0, 1),
traincode: va.station_train_code.substring(0, va.station_train_code.indexOf('(')),
trainstart: va.station_train_code.substring(va.station_train_code.indexOf('(') + 1, va.station_train_code.indexOf('-')),
trainend: va.station_train_code.substring(va.station_train_code.indexOf('-') + 1, va.station_train_code.indexOf(')'))
}
});
})
}
})
}
})
},
}
})
})
</script>
</head>
<body>
<div class="container" id="app">
<div class="row clearfix">
<div class="col-md-12 column">
<h4>这个页面都是跟火车有关系的东西呢</h4>
</div>
<div class="col-md-3 column">
<label>名称CDGKOTZ</label>
<input type="text" class="form-control" placeholder="请输入开头,如C,G,D" v-model="traval">
<button type="button" class="btn btn-primary" v-on:click="getalltrainline()">开始添加车次数据
</button>
</div>
</div>
</div>
</div>
</body>
</html>Controller代码:
@Controller
@RequestMapping("/train")
public class TrainController {
@Autowired
private TrainServiceImlp trainService;
@RequestMapping("/addtrainline")
@ResponseBody
public int allUnit(HttpServletRequest request) {
Trainline trainline = new Trainline();
trainline.setTrainno(request.getParameter("trainno"));
trainline.setTraintype(request.getParameter("traintype"));
trainline.setTraincode(request.getParameter("traincode"));
trainline.setTrainstart(request.getParameter("trainstart"));
trainline.setTrainend(request.getParameter("trainend"));
int i = trainService.addTrainline(trainline);
return i;
}
}service代码:
public interface ITrainService {
int addTrainline(Trainline trainline);
List<Trainline> getlines();
int addlinestations(List<Linestation> list);
}
@Service
public class TrainServiceImlp implements ITrainService {
@Resource
private TrainlineMapper trainlineMapper;
@Resource
private LinestationMapper linestationMapper;
public int addTrainline(Trainline trainline) {
return trainlineMapper.insert(trainline);
}
}下面的mapper,mapper.xml和实体类都是MyBatis Generator自动生成的,为了篇幅就简要复制..不会的可以百度也可以参考作者的博客:MyBatis Generator 详解
TrainlineMapper:
public interface TrainlineMapper {
int insert(Trainline record);
}TrainlineMapper.xml:
<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd" >
<mapper namespace="com.jeasen.hrms.dto.TrainlineMapper" >
<resultMap id="BaseResultMap" type="com.jeasen.hrms.pojo.Trainline" >
<id column="trainid" property="trainid" jdbcType="INTEGER" />
<result column="trainno" property="trainno" jdbcType="VARCHAR" />
<result column="traintype" property="traintype" jdbcType="VARCHAR" />
<result column="traincode" property="traincode" jdbcType="VARCHAR" />
<result column="trainstart" property="trainstart" jdbcType="VARCHAR" />
<result column="trainend" property="trainend" jdbcType="VARCHAR" />
</resultMap>
<insert id="insert" parameterType="com.jeasen.hrms.pojo.Trainline" >
insert into trainline (trainid, trainno, traintype,
traincode, trainstart, trainend
)
values (#{trainid,jdbcType=INTEGER}, #{trainno,jdbcType=VARCHAR}, #{traintype,jdbcType=VARCHAR},
#{traincode,jdbcType=VARCHAR}, #{trainstart,jdbcType=VARCHAR}, #{trainend,jdbcType=VARCHAR}
)
</insert>
</mapper>Pojo实体类:
package com.jeasen.hrms.pojo;
public class Trainline {
private Integer trainid;
private String trainno;
private String traintype;
private String traincode;
private String trainstart;
private String trainend;
public Integer getTrainid() {
return trainid;
}
public void setTrainid(Integer trainid) {
this.trainid = trainid;
}
public String getTrainno() {
return trainno;
}
public void setTrainno(String trainno) {
this.trainno = trainno == null ? null : trainno.trim();
}
public String getTraintype() {
return traintype;
}
public void setTraintype(String traintype) {
this.traintype = traintype == null ? null : traintype.trim();
}
public String getTraincode() {
return traincode;
}
public void setTraincode(String traincode) {
this.traincode = traincode == null ? null : traincode.trim();
}
public String getTrainstart() {
return trainstart;
}
public void setTrainstart(String trainstart) {
this.trainstart = trainstart == null ? null : trainstart.trim();
}
public String getTrainend() {
return trainend;
}
public void setTrainend(String trainend) {
this.trainend = trainend == null ? null : trainend.trim();
}
}最后附上sql语句:
SET FOREIGN_KEY_CHECKS=0;
DROP TABLE IF EXISTS `trainline`;
CREATE TABLE `trainline` (
`trainid` int(11) NOT NULL AUTO_INCREMENT,
`trainno` varchar(66) DEFAULT NULL,
`traintype` varchar(22) DEFAULT NULL,
`traincode` varchar(22) DEFAULT NULL,
`trainstart` varchar(22) DEFAULT NULL,
`trainend` varchar(22) DEFAULT NULL,
PRIMARY KEY (`trainid`)
) ENGINE=InnoDB AUTO_INCREMENT=9058 DEFAULT CHARSET=utf8;
第一次写博客,水平很菜,还请各位多多包涵.
3万+
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
