一、准备工作
- Centos6.5的系统
- Cloudera Manager 版本5.6
- Spark2.3依赖的jdk1.8
- 网上的资料提供的下载地址下载不了,只能基于原有的
SPARK2-2.3.0.cloudera2-1.cdh5.13.3.p0.316101-el7.parcel
修改 - 从头制作可以参考如下
二、开始适配
1.parcel包制作
包名规则:
以SPARK2-2.3.0.cloudera3-1.cdh5.6.0.p0.1-el6.parcel
为例,第一个-
前是包名,最后一个-
后是运行平台,el6是代表centos6系统,el7表示centos7系统,中间为版本号,修改的重点也是版本号好对应后续文件
修改meta/parcel.json为
{
"schema_version": 1,
"name": "SPARK2",
"version": "2.3.0.cloudera3-1.cdh5.6.0.p0.1",
"components": [
{
"name": "spark2\n",
"pkg_release": "na\n",
"pkg_version": "na\n",
"version": "2.3.0.cloudera3\n"
}
],
"depends": "CDH (>= 5.5), CDH (<< 6)",
"extraVersionInfo": {
"baseVersion": "cdh5.6",
"patchCount": "0"
},
"groups": [
"spark"
],
"packages": [],
"provides": [
"spark2"
],
"replaces": "SPARK",
"scripts": {
"defines": "spark2_env.sh"
},
"setActiveSymlink": true,
"users": {
"spark": {
"extra_groups": [],
"home": "/var/lib/spark",
"longname": "Spark",
"shell": "/sbin/nologin"
}
}
}
修改meta/spark2_env.sh为
#!/bin/bash
CDH_DIRNAME=${PARCEL_DIRNAME:-"SPARK2-2.3.0.cloudera3-1.cdh5.6.0.p0.1-el6"}
export CDH_SPARK2_HOME=$PARCELS_ROOT/$CDH_DIRNAME/lib/spark2
修改\lib\spark2\cloudera\spark2_version.properties
# Autogenerated build properties
version=2.3.0.cloudera3
git.hash=9f5baab06f127486a030024877fc13a3992f100f
cloudera.hash=9f5baab06f127486a030024877fc13a3992f100f
cloudera.cdh.hash=na
cloudera.cdh-packaging.hash=na
clo