Spring Cloud Data Flow--大数据操作工具,作为Spring XD的替代产品,它是一个混合计算模型,结合了流数据与批量数据的处理方式。为数据微服务提供了业务流程,包括长期的 Spring Cloud Stream 应用程序和短期的 Spring Cloud Task 应用程序。
之前很少写博客,主要是国内相关资料少之又少,踩了很多坑,谷歌也没多大帮助,深知那种无助的感觉,所以记录一下,与喜欢研究技术的朋友分享一下,希望此文会有小小的帮助,微服务的明天会更好
1.通过vagrant安装DCOS,我们会得到一个Marathon端点url:http://m1.dcos/service/marathon,留作配置用,如果是其它比如CLI、GUI的安装方式,也可配置其服务的ip地址。service安装这里不做过多介绍,可以通过dcos的universe安装,也可以通过json文件安装
2.安装mysql可以通过DCOS进行安装,如果你有一个mysql数据库亦可不必安装
curl -X POST http://m1.dcos/service/marathon/v2/apps -d @mysql.json -H "Content-type: application/json"
3.安装rabbitmq,如果你有一个rabbitmq服务器亦可不必安装,此处注意,对于springboot应用,应当在应用配置文件中配置mq的用户名密码,如:
curl -X POST http://m1.dcos/service/marathon/v2/apps -d @rabbitmq.json -H "Content-type: application/json"
4.安装redis,如果你有一个redis服务器亦可不必安装
curl -X POST http://m1.dcos/service/marathon/v2/apps -d @redis.json -H "Content-type: application/json"
5.安装chronos,如果你有一个chronos服务器亦可不必安装,速度慢说明拉取镜像慢,此处共享下百度云链接:http://pan.baidu.com/s/1dFaQvSX 密码:3j9c,下载到agent节点直接docker load < chronos.tar,docker images查看即可,重新执行,速度杠杠滴。
dcos package install chronos
6.获取springclouddataflow的json配置文件
$ wget https://raw.githubusercontent.com/spring-cloud/spring-cloud-dataflow-server-mesos/v1.0.0.RELEASE/src/etc/marathon/scdf-server.json
7.直接贴上我的配置文件
{
"id": "/spring-cloud-data-flow",
"instances": 1,
"cpus": 2,
"mem": 4000,
"disk": 3000,
"gpus": 0,
"backoffSeconds": 1,
"backoffFactor": 1.15,
"maxLaunchDelaySeconds": 3600,
"container": {
"type": "DOCKER",
"docker": {
"image": "springcloud/spring-cloud-dataflow-server-mesos:latest",
"network": "BRIDGE",
"portMappings": [
{
"containerPort": 9393,
"hostPort": 0,
"servicePort": 10000,
"protocol": "tcp",
"name": "default"
}
],
"privileged": false,
"forcePullImage": false
}
},
"healthChecks": [
{
"gracePeriodSeconds": 120,
"intervalSeconds": 60,
"timeoutSeconds": 20,
"maxConsecutiveFailures": 0,
"portIndex": 0,
"path": "/management/health",
"protocol": "HTTP",
"ignoreHttp1xx": false
}
],
"upgradeStrategy": {
"minimumHealthCapacity": 1,
"maximumOverCapacity": 1
},
"unreachableStrategy": {
"inactiveAfterSeconds": 300,
"expungeAfterSeconds": 600
},
"killSelection": "YOUNGEST_FIRST",
"requirePorts": true,
"env": {
"JDBC_DRIVER": "org.mariadb.jdbc.Driver",
"MESOS_CHRONOS_URI": "http://172.16.1.77:10105",
"REDIS_HOST": "172.16.1.61",
"RABBITMQ_PORT": "6392",
"MESOS_MARATHON_URI": "http://m1.dcos/service/marathon",
"REDIS_PORT": "6379",
"JDBC_PASSWORD": "1234321",
"JDBC_URL": "jdbc:mysql://172.16.1.145:3306/test",
"SPRING_APPLICATION_JSON": "{\"spring.cloud.deployer.mesos.marathon.apiEndpoint\":\"${MESOS_MARATHON_URI}\",\"spring.cloud.deployer.mesos.c