hadoop
无痕Miss
这个作者很懒,什么都没留下…
展开
-
zookeeper分布式部署【二】
前面进行了zookeeper分布式部署,这边进行一个客户端连接操作,使用python连接1.pycharm安装kazoo2.下面上代码#!/usr/bin/env python# -*- coding: utf-8 -*-import sysfrom kazoo.client import KazooClientdef main(): try: nodePath = "/zktest" host = "192.168.1.232"原创 2020-12-04 16:54:02 · 174 阅读 · 0 评论 -
zookeeper分布式部署【一】
1.下载zookeeperwget https://mirrors.tuna.tsinghua.edu.cn/apache/zookeeper/zookeeper-3.6.2/apache-zookeeper-3.6.2-bin.tar.gz【备注】下载编译好的就可以了2.解压zookeepertar-zxvf apache-zookeeper-3.6.2-bin.tar.gz -C /zookeeper3.修改配置cd /zookeeper/confcp zoo_sam.原创 2020-12-04 16:48:45 · 138 阅读 · 0 评论 -
python中远程spark集群
1.配置pycharm环境(此处需要使用远程python环境)配置master的地址,账号,密码2.pycharm中安装pyspark与pyspark-stubs3.配置python环境变量HADOOP_HOME=/usr/local/hadoopJAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.232.b09-0.el...原创 2019-12-04 00:17:36 · 1075 阅读 · 0 评论 -
配置spark2.4.4
1.下载spark2.4.4及解压文件wget https://mirrors.tuna.tsinghua.edu.cn/apache/spark/spark-2.4.4/spark-2.4.4-bin-without-hadoop-scala-2.12.tgztar -zxvf spark-2.4.4/spark-2.4.4-bin-without-hadoop-scala-2.12....原创 2019-12-01 18:54:10 · 393 阅读 · 0 评论 -
hadoop3.2.1版本配置
1.列出所有能安装的jdk版本yum list java*2.用以下命令安装jdk版本yum install -y java-1.8.0-openjdk-devel.x86_643.下载hadoop3.2.1版本wget http://mirror.bit.edu.cn/apache/hadoop/common/hadoop-3.2.1/hadoop-3.2.1.tar....原创 2019-11-26 22:26:36 · 1668 阅读 · 1 评论 -
centos集群ssh免密登录
1.修改当前的centos名称vi /etc/hostname#修改名称为node1即可vi /etc/hosts#127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4#修改其中的localhost.localdomain为节点名称#此处进行映射#例如 192.168.100....原创 2019-11-19 23:59:00 · 289 阅读 · 0 评论