Markix的博客

在Windows 上调试MapReduce下载hadoop-<version>.tar.gz (版本随意)三方镜像地址：http://mirror.bit.edu.cn/apache/hadoop/common/解压添加环境变量，将Hadoop的bin目录添加到环境变量右键“计算机”-> 高级系统设置

执行 ./bin/stop-all.sh 脚本一直提示没有可停止的namenode、datanode、secondarynode。可是输入 jps 命令，发现hadoop 已经启动。[root@xxxxxx src]# bash hadoop-2.6.5/sbin/stop-all.sh This script is Deprecated. Instead use stop-dfs.sh a...

python环境、jdk环境、spark配置环境变量新建SPARK_HOME=E:\Hadoop\spark-2.1.3-bin-hadoop2.6PYSPARK_PYTHON=E:\ProgramData\Anaconda3\envs\py27\python.exe添加PATH=%SPARK_HOME%\bin将E:\Hadoop\spark-2.1.3-bin-hadoop2.6...

在window、运行pyspark训练模型，报错Caused by: java.net.SocketException: Connection reset by peer: socket write errorpy4j.protocol.Py4JJavaError: An error occurred while calling z:org.apache.spark.api.python.Py...

spark-submit.shorg/apache/spark/deploy/SparkSubmit.scalamainsubmitdoRunMainprepareSubmitisStandaloneCluster => childMainClass = “org.apache.spark.deploy.Client”isYarnCluster => childMainC...