Ambari 2.7.5编译前戏
其实真不想编译,被公司里的一个老哥忽悠着去搞。直接用官网的2.7.4的不香嘛,编译报的错也是不好受。这里不贴怎么编译网上教程一堆~。再说都是去年编译的时候,花了大半个月来折腾。
核心点就几个:
(1)要把那三四个大jar包下载到,然后就报啥错就解决啥错。
(2)接着编译成功后怎么知道自己需要哪几个rpm包,所以这就需要跟2.7.4官方自带的rpm进行比较。然后找出对应的包。
(3)有些rpm包是编译后没有的,然后可以用2.7.4的那个包的组件替换。比如收费的那个组件,O(∩_∩)O。
(4) 接着比对两版本的rpm字节大小(虽然编译出了但大小差距太大也不敢用有点慌)。
31M Oct 28 19:07 ambari-agent-2.7.5.0-0.x86_64.rpm 37M Aug 26 2019 ambari-agent-2.7.4.0-118.x86_64.rpm
87M Oct 28 19:07 ambari-infra-manager-2.7.5.0-0.noarch.rpm 87M Aug 26 2019 ambari-infra-manager-2.7.4.0-118.noarch.rpm
167M Oct 28 19:07 ambari-infra-solr-2.7.5.0-0.noarch.rpm 167M Aug 26 2019 ambari-infra-solr-2.7.4.0-118.noarch.rpm
22M Oct 28 19:07 ambari-infra-solr-client-2.7.5.0-0.noarch.rpm 22M Aug 26 2019 ambari-infra-solr-client-2.7.4.0-118.noarch.rpm
51M Oct 28 19:07 ambari-logsearch-logfeeder-2.7.5.0-0.noarch.rpm 51M Aug 26 2019 ambari-logsearch-logfeeder-2.7.4.0-118.noarch.rpm
77M Oct 28 19:07 ambari-logsearch-portal-2.7.5.0-0.noarch.rpm 79M Aug 26 2019 ambari-logsearch-portal-2.7.4.0-118.noarch.rpm
466M Oct 28 19:07 ambari-metrics-collector-2.7.5.0-0.x86_64.rpm 466M Aug 26 2019 ambari-metrics-collector-2.7.4.0-118.x86_64.rpm
1.5K Oct 28 19:07 ambari-metrics-common-2.7.5.0-0.noarch.rpm 2.6K Aug 26 2019 ambari-metrics-common-2.7.4.0-118.noarch.rpm
46M Oct 28 19:07 ambari-metrics-grafana-2.7.5.0-0.x86_64.rpm 18M Aug 26 2019 ambari-metrics-grafana-2.7.4.0-118.x86_64.rpm
30M Oct 28 19:07 ambari-metrics-hadoop-sink-2.7.5.0-0.x86_64.rpm 30M Aug 26 2019 ambari-metrics-hadoop-sink-2.7.4.0-118.x86_64.rpm
32M Oct 28 19:07 ambari-metrics-monitor-2.7.5.0-0.x86_64.rpm 32M Aug 26 2019 ambari-metrics-monitor-2.7.4.0-118.x86_64.rpm
115M Oct 28 19:07 ambari-server-2.7.5.0-0.x86_64.rpm 371M Aug 26 2019 ambari-server-2.7.4.0-118.x86_64.rpm
我也不清楚为啥ambari-server这个包就只有115M左右,官方的2.7.4版本的ambari-server rpm包就有370多M。有哪个老哥知道的说一下呀,或者说从官网下载了2.7.5版本的ambari版本,透漏一下大小。先安装起来用一下看有啥问题。
Ambari 2.7.5 安装
当安装上去后,你会发现没有选择的版本为空,去年是解决过了的,这几天又重新安装起来,忘了过程了又瞎折腾了几天。虽然是临时解决的方案但总得记下来,以后再需要安装玩也可以翻出来看,有根解的方案的老哥就给一个。
反正出问题就得知道问题的原因是啥才能对症下药吧,然后出发点当然是从看log入手,反复安装对比了2.7.4跟2.7.5的启动日志。发下了不同点。
2.7.4的日志的却别点在下面
2021-03-21 12:09:11,698 INFO [main] StackContext:222 - Loaded all VDF in 8795ms
2021-03-21 12:09:11,720 INFO [main] StackManager:208 - Adding stack HDP-2.0 to the database
2021-03-21 12:09:11,746 INFO [main] StackManager:208 - Adding stack HDP-2.0.6 to the database
2021-03-21 12:09:11,749 INFO [main] StackManager:208 - Adding stack HDP-2.0.6.GlusterFS to the database
2021-03-21 12:09:11,773 INFO [main] StackManager:208 - Adding stack HDP-2.1 to the database
2021-03-21 12:09:11,776 INFO [main] StackManager:208 - Adding stack HDP-2.1.GlusterFS to the database
2021-03-21 12:09:11,789 INFO [main] StackManager:208 - Adding stack HDP-2.2 to the database
2021-03-21 12:09:11,791 INFO [main] StackManager:208 - Adding stack HDP-2.3 to the database
2021-03-21 12:09:11,793 INFO [main] StackManager:208 - Adding stack HDP-2.3.ECS to the database
2021-03-21 12:09:11,806 INFO [main] StackManager:208 - Adding stack HDP-2.3.GlusterFS to the database
2021-03-21 12:09:11,819 INFO [main] StackManager:208 - Adding stack HDP-2.4 to the database
2021-03-21 12:09:11,821 INFO [main] StackManager:208 - Adding stack HDP-2.5 to the database
2021-03-21 12:09:11,835 INFO [main] StackManager:208 - Adding stack HDP-2.6 to the database
2021-03-21 12:09:11,848 INFO [main] StackManager:208 - Adding stack HDP-3.0 to the database
2021-03-21 12:09:11,854 INFO [main] StackManager:208 - Adding stack HDP-3.1 to the database
2021-03-21 12:09:11,855 INFO [main] StackManager:250 - Creating links
2.7.5的日志在下面
2021-03-21 16:22:30,889 INFO [main] StackContext:222 - Loaded all VDF in 7147ms
2021-03-21 16:22:30,894 INFO [main] StackManager:208 - Adding stack HDP-2.0 to the database
2021-03-21 16:22:30,909 INFO [main] StackManager:208 - Adding stack HDP-2.0.6 to the database
2021-03-21 16:22:30,911 INFO [main] StackManager:208 - Adding stack HDP-2.0.6.GlusterFS to the database
2021-03-21 16:22:30,914 INFO [main] StackManager:208 - Adding stack HDP-2.1 to the database
2021-03-21 16:22:30,916 INFO [main] StackManager:208 - Adding stack HDP-2.1.GlusterFS to the database
2021-03-21 16:22:30,919 INFO [main] StackManager:208 - Adding stack HDP-2.2 to the database
2021-03-21 16:22:30,921 INFO [main] StackManager:208 - Adding stack HDP-2.3 to the database
2021-03-21 16:22:30,923 INFO [main] StackManager:208 - Adding stack HDP-2.3.ECS to the database
2021-03-21 16:22:30,925 INFO [main] StackManager:208 - Adding stack HDP-2.3.GlusterFS to the database
2021-03-21 16:22:30,927 INFO [main] StackManager:208 - Adding stack HDP-2.4 to the database
2021-03-21 16:22:30,929 INFO [main] StackManager:208 - Adding stack HDP-2.5 to the database
2021-03-21 16:22:30,931 INFO [main] StackManager:208 - Adding stack HDP-2.6 to the database
2021-03-21 16:22:30,936 INFO [main] StackManager:250 - Creating links
就少了两个没有将3.0 跟3.1加载到数据库中嘛。然后办法就是将2.7.4的stacks下的3.0与3.1的目录。移动到2.7.5的stacks下就可以暂时解决这个问题了。
drwxr-xr-x. 4 root root 86 Mar 21 12:02 2.0
drwxr-xr-x. 6 root root 125 Mar 21 12:02 2.0.6
drwxr-xr-x. 4 root root 86 Mar 21 12:02 2.0.6.GlusterFS
drwxr-xr-x. 5 root root 104 Mar 21 12:02 2.1
drwxr-xr-x. 5 root root 104 Mar 21 12:02 2.1.GlusterFS
drwxr-xr-x. 5 root root 107 Mar 21 12:02 2.2
drwxr-xr-x. 5 root root 102 Mar 21 12:02 2.3
drwxr-xr-x. 4 root root 86 Mar 21 12:02 2.3.ECS
drwxr-xr-x. 5 root root 107 Mar 21 12:02 2.3.GlusterFS
drwxr-xr-x. 5 root root 102 Mar 21 12:02 2.4
drwxr-xr-x. 5 root root 102 Mar 21 12:02 2.5
drwxr-xr-x. 5 root root 105 Mar 21 12:02 2.6
drwxr-xr-x. 7 root root 141 Mar 21 12:02 3.0
drwxr-xr-x. 5 root root 71 Mar 21 12:02 3.1
最后重启,登录进去就可以按正常的流程选择版本安装组件了。
安装后呈现没多大变化
截图如下~~
后面还有个问题
不清楚啥原因,启动后这两个的日志还有些许不同,/(ㄒoㄒ)/~~。
难道少了WORKFLOW_MANAGER,这个是啥,待会再去了解一下。
2.7.4版本的
2021-03-21 12:09:13,942 INFO [main] CertificateManager:75 - Initialization of root certificate
2021-03-21 12:09:13,942 INFO [main] CertificateManager:77 - Certificate exists:true
2021-03-21 12:09:14,016 INFO [main] ViewRegistry:534 - Triggering loading of [ALL] views
2021-03-21 12:09:14,023 INFO [main] ViewRegistry:1814 - Reading view archive /var/lib/ambari-server/resources/views/ambari-admin-2.7.4.0.118.jar.
2021-03-21 12:09:14,134 INFO [main] ViewRegistry:1850 - View deployed: ADMIN_VIEW{2.7.4.0}.
2021-03-21 12:09:14,138 INFO [main] ViewRegistry:1814 - Reading view archive /var/lib/ambari-server/resources/views/capacity-scheduler-2.7.4.0.118.jar.
2021-03-21 12:09:14,138 INFO [main] ViewExtractor:89 - Creating archive folder /var/lib/ambari-server/resources/views/work/CAPACITY-SCHEDULER{1.0.0}.
2021-03-21 12:09:14,139 INFO [main] ViewExtractor:96 - Extracting files from capacity-scheduler-2.7.4.0.118.jar.
2021-03-21 12:09:15,413 INFO [main] ViewRegistry:1920 - setting up logging for view CAPACITY-SCHEDULER{1.0.0} as per property file view.log4j.properties
2021-03-21 12:09:15,447 INFO [main] ViewRegistry:1850 - View deployed: CAPACITY-SCHEDULER{1.0.0}.
2021-03-21 12:09:15,451 INFO [main] ViewRegistry:1814 - Reading view archive /var/lib/ambari-server/resources/views/files-2.7.4.0.118.jar.
2021-03-21 12:09:15,451 INFO [main] ViewExtractor:89 - Creating archive folder /var/lib/ambari-server/resources/views/work/FILES{1.0.0}.
2021-03-21 12:09:15,452 INFO [main] ViewExtractor:96 - Extracting files from files-2.7.4.0.118.jar.
2021-03-21 12:09:17,622 INFO [main] ViewRegistry:1920 - setting up logging for view FILES{1.0.0} as per property file view.log4j.properties
2021-03-21 12:09:17,682 INFO [main] ViewRegistry:1850 - View deployed: FILES{1.0.0}.
2021-03-21 12:09:17,690 INFO [main] ViewRegistry:1814 - Reading view archive /var/lib/ambari-server/resources/views/smartsense-ambari-view-1.5.1.2.7.4.0-118.jar.
2021-03-21 12:09:17,690 INFO [main] ViewExtractor:89 - Creating archive folder /var/lib/ambari-server/resources/views/work/SMARTSENSE{1.5.1.2.7.4.0-118}.
2021-03-21 12:09:17,691 INFO [main] ViewExtractor:96 - Extracting files from smartsense-ambari-view-1.5.1.2.7.4.0-118.jar.
2021-03-21 12:09:17,743 INFO [main] ViewRegistry:1920 - setting up logging for view SMARTSENSE{1.5.1.2.7.4.0-118} as per property file view.log4j.properties
2021-03-21 12:09:17,795 INFO [main] ViewRegistry:1850 - View deployed: SMARTSENSE{1.5.1.2.7.4.0-118}.
2021-03-21 12:09:17,820 INFO [main] ViewRegistry:1814 - Reading view archive /var/lib/ambari-server/resources/views/wfmanager-2.7.4.0.118.jar.
2021-03-21 12:09:17,820 INFO [main] ViewExtractor:89 - Creating archive folder /var/lib/ambari-server/resources/views/work/WORKFLOW_MANAGER{1.0.0}.
2021-03-21 12:09:17,821 INFO [main] ViewExtractor:96 - Extracting files from wfmanager-2.7.4.0.118.jar.
2021-03-21 12:09:18,972 INFO [main] ViewRegistry:1920 - setting up logging for view WORKFLOW_MANAGER{1.0.0} as per property file view.log4j.properties
2021-03-21 12:09:19,014 INFO [main] ViewRegistry:1850 - View deployed: WORKFLOW_MANAGER{1.0.0}.
2.7.5 版本
2021-03-21 16:22:32,508 INFO [main] CertificateManager:75 - Initialization of root certificate
2021-03-21 16:22:32,508 INFO [main] CertificateManager:77 - Certificate exists:true
2021-03-21 16:22:32,558 INFO [main] ViewRegistry:534 - Triggering loading of [ALL] views
2021-03-21 16:22:32,566 INFO [main] ViewRegistry:1814 - Reading view archive /var/lib/ambari-server/resources/views/ambari-admin-2.7.5.0.0.jar.
2021-03-21 16:22:32,615 INFO [main] ViewRegistry:1850 - View deployed: ADMIN_VIEW{2.7.5.0}.
2021-03-21 16:22:32,656 INFO [main] HeartbeatProcessor:160 - **** Starting heartbeats processing threads ****
2021-03-21 16:22:32,658 INFO [main] AmbariServer:499 - ********** Started Heartbeat handler **********
2021-03-21 16:22:32,662 INFO [main] AmbariServer:539 - ********* Initializing Clusters **********
2021-03-21 16:22:32,662 INFO [main] AmbariServer:545 - ********* Current Clusters State *********
2021-03-21 16:22:32,662 INFO [main] AmbariServer:546 -
2021-03-21 16:22:32,663 INFO [main] AmbariServer:548 - ********* Reconciling Alert Definitions **********
2021-03-21 16:22:32,663 INFO [main] AmbariServer:551 - ********* Initializing ActionManager **********
2021-03-21 16:22:32,663 INFO [main] AmbariServer:554 - ********* Initializing Controller **********
2021-03-21 16:22:32,663 INFO [main] AmbariServer:558 - ********* Initializing Scheduled Request Manager **********
2021-03-21 16:22:32,793 INFO [main] ContextLoader:270 - Root WebApplicationContext: initialization started
最后,2.7.5版本的release notes解决啥问题
新功能特征:
以下是Ambari 2.7.5版本中引入的新功能。表1. Apache Ambari 2.7.5的新功能
特征 | 描述 |
---|---|
安全的Ambari存储库 | 出于生产目的访问Ambari存储库需要进行身份验证。发行包存储库受“基本身份验证”保护,Ambari支持处理这些存储库的用户名和密码。有关更多信息,请参阅Ambari安装指南中的“访问Ambari存储库”主题。 |
行为改变
行为更改表示从先前发布的版本到此版本的Ambari的行为有了明显的变化。与先前发布的版本相比,此版本没有任何行为更改。
常见漏洞和披露
没有常见漏洞和披露(CVE)修复程序适用于Ambari 2.7.5
已解决的问题
已解决的问题表示以前通过Cloudera支持记录的部分问题,但当前版本中已解决。这些问题可能已经在以前版本的“已知问题”部分中进行了报告;表示它们由客户报告或由Hortonworks质量工程团队识别。
结果不正确
错误编号 | 阿帕奇JIRA | 概括 |
---|---|---|
BUG-121927 | 不适用 | 使用自定义队列配置YARN会导致Stack Advisor中的误导性错误 |
BUG-107724 | AMBARI-24302 | (针对2.6.x的JIRA跟踪)-Dhdp.version在Datanodes的进程输出中显示空白值 |
其他
错误编号 | 阿帕奇JIRA | 概括 |
---|---|---|
BUG-122369 | AMBARI-25412 | 从ambari-contrib中的记录器服务方法中删除所有标签 |
BUG-122337 | 不适用 | 在Zeppelin JDBC解释器中错误配置了HiveServer2 JDBC URL |
BUG-122165 | 安巴里25400 | 在确定HA的情况下确定实时收集器时出现问题 |
BUG-122125 | AMBARI-14526 | Ambari代理SUSE12 Systemd服务在系统重新引导时不会自动启动Ambari代理。 |
BUG-122035 | AMBARI-25394 | HBase表的*通配符上的Ambari Metrics白名单失败 |
BUG-122032 | AMBARI-25397 | 将ambari-logsearch-logfeeder升级到2.7.4 rpm会发出警告 |
BUG-121976 | AMBARI-25378 | 500顾问程序错误,将ambari指标模式设置为与OneFS一起分发 |
BUG-121926 | 不适用 | 从Hive脚本中删除硬编码的重试次数 |
BUG-121911 | AMBARI-25379 | 将AMS Grafana版本升级到6.4.2 |
BUG-121898 | AMBARI-25399 | 添加Hive PAM支持以进行服务检查和警报 |
BUG-121879 | 不适用 | oozie.server.authentication.type=kerberos启用KnoxSSO时,HDP堆栈未设置, 导致冲突 |
BUG-121218 | 不适用 | 使用hdfs资源,而不是在 hive_server_interactive.py |
错误121024 | AMBARI-25333 | 如果高速缓存中不存在文件,则重新生成密钥表将生成空的密钥表文件 |
BUG-120861 | AMBARI-25326 | AMS-使用2个收集器时,没有HBase和Hive指标在升级后 |
错误120603 | AMBARI-25395 | 更新Hive安装中的帮助文本以反映实际的JAR文件名或提供格式清晰的示例 |
表现
错误编号 | 阿帕奇JIRA | 概括 |
---|---|---|
BUG-121889 | AMBARI-25385 | 减少集群创建请求的处理时间 |
BUG-120989 | AMBARI-25332 | Kerberos keytab再生工作缓慢 |
BUG-122244 | AMBARI-21935 | Hive向量化:使用向量化UDF会降低性能 |
BUG-122079 | AMBARI-25156 | ClientComponentHasNoStatus 异常混乱的操作系统 /var/log/messages |
BUG-122239 | AMBARI-25408 | 将Infra Solr升级到7.7.2 |
安全
错误编号 | 阿帕奇JIRA | 概括 |
---|---|---|
BUG-121464 | AMBARI-25396 | Ambari主机上的跨站点脚本漏洞 |
错误122087 | 不适用 | 使用API将XSS漏洞存储在rack_info中 |
BUG-121361 | AMBARI-25384 | 文件视图中的跨站点脚本漏洞 |
2015年8月12日 | AMBARI-25391 | Ambari在ActionQueue.py中记录Grafana密码 |
BUG-121801 | AMBARI-25390 | 在/ resources端点和子目录中禁用索引 |
稳定
错误编号 | 阿帕奇JIRA | 概括 |
---|---|---|
BUG-122238 | AMBARI-25403 | Ambari管理包:Ambari在下载OneFS客户端配置时抛出500错误 |
可支持性
错误编号 | 阿帕奇JIRA | 概括 |
---|---|---|
BUG-121600 | 不适用 | 对于HDP 3.1 HS(容器模式)hive.merge.nway.joins 未设置为false |
易用性
错误编号 | 阿帕奇JIRA | 概括 |
---|---|---|
BUG-121804 | AMBARI-25380 | 用户界面不反映/更新任务日志 |
又水完一篇~~。