一个大数据的实践

0.操作

hdfs家目录 操作

/user/hadoop

hdfs操作

usr/local/hadoop下 ./bin/hdfs dfs -mkdir -p /user/hadoop

eclipse

cd /usr/local/eclipse

./eclipse

hadoop->hbase-> hbase shell操作

ssh localhost

cd /usr/local/hadoop

./sbin/start-dfs.sh

cd /usr/local/hbase

bin/start-hbase.sh

bin/hbase shell

spark 操作RDD

cd /usr/local/spark

bin/spark-shell

service MySQL start

spark shell 操作RDD

cd /usr/local/spark

bin/spark-shell

退出:quit

flink

cd /usr/local/flink

./bin/start-cluster.sh

pycharm

cd /usr/local/pycharm

./bin/pycharm.sh

1. 虚拟机安装VMware

Mware和virtualbox的区分https://zhuanlan.zhihu.com/p/529764393

光盘映像文件：ubuntu

https://blog.csdn.net/weixin_74195551/article/details/127288338?spm=1001.2014.3001.5502

使用过程中出现问题

1. hadoop环境安装

https://blog.csdn.net/m0_52931616/article/details/125647616 中第三步

安装jdk

设置SSH无密码登录

下载hadoop

其中wget后的网址下载不存在，更改网址为https://downloads.apache.org/hadoop/common/hadoop-3.2.3/hadoop-3.2.3.tar.gz，也就是：