tags:
- csdn
date: 2024-05-16
环境配置
- win11
- spark-3.5.1-bin-hadoop3
- Anaconda3 python3.8
- Pycharm
安装
-
下载安装包到官网自行下载 spark-3.5.1-bin-hadoop3
-
配置环境变量:
-
SPARK_HOME:
E:\web install\Environment\Datademo\spark-3.5.1-bin-hadoop3\bin -
HADOOP_HOME:
E:\web install\Environment\Datademo\spark-3.5.1-bin-hadoop3 -
%SPARKHOME%/bin
-
-
下载pyspark(API接口):pip install pyspark
-
配置环境变量:
-
Anaconda虚拟环境python.exe(已经在系统中配置过PYTHON变量的可以不用再配置了)
-
PYSPARK_PYTHON:
“D:\Environment\ANACONDA\envs\Database\python.exe” -
PYTHON_DRIVER_HOME:
“D:\Environment\ANACONDA\envs\Database\python.exe”
-