Spark 3.2上执行安装jupyter notebook

最新推荐文章于 2024-07-20 13:23:35 发布

chriszzww

最新推荐文章于 2024-07-20 13:23:35 发布

阅读量680

点赞数

文章标签： hadoop 大数据 spark

本文链接：https://blog.csdn.net/zhu2525wei/article/details/125918003

版权

本文档详细介绍了如何在已经安装了hadoop和spark的Ubuntu 18.04系统上，通过pip3安装jupyterlab和notebook，然后配置findspark，修改kernel.json，生成jupyter配置文件和密码，以及解决安装过程中的错误。最后，指导如何启动pyspark并进行远程访问。

摘要由CSDN通过智能技术生成

准备条件spark on hadoop环境已经安装好
版本：
spark-3.2.1-bin-hadoop3.2
hadoop-3.2.2
操作系统：
ubuntu 18.04

安装步骤：
1、执行安装pip3
apt-get update
apt-get upgrade
apt-get update
apt install python3-pip
2、下载并安装notebook
pip3 install --user jupyterlab notebook
pip3 install --user findspark
///
root@zw001:/opt# pip3 install --user jupyterlab notebook -i https://pypi.org/simple
Collecting jupyterlab
Using cached https://files.pythonhosted.org/packages/4b/0d/03deff4501e9ffafe755e561e375ffa9f5822fec93a09ce1c7c5147bdcb3/jupyterlab-3.2.9-py3-none-any.whl
Collecting notebook
Downloading https://files.pythonhosted.org/packages/27/b7/7e602dc8b868bba8a542269205237b400be3427d8489b5851de5f7587996/notebook-6.4.10-py3-none-any.whl (9.9MB)
51% |████████████████▍ | 5.1MB 120kB/s eta 0:00:41

root@zw001:/usr/local# pip3 install --user findspark
WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue.
To avoid this problem you can invoke Python with '-m pip' instead of running pip directly.
Collecting fin