当我在运行ipython终端的时候,发现报错了,错误信息如下:
wugaosheng:spark-2.2.0-bin-hadoop2.7 eric$ IPYTHON=1 IPYTHON_OPTS="--pylab" ./bin/pyspark
Error in pyspark startup:
IPYTHON and IPYTHON_OPTS are removed in Spark 2.0+. Remove these from the environment and set PYSPARK_DRIVER_PYTHON and PYSPARK_DRIVER_PYTHON_OPTS instead.
经过查资料,才发现,原来这行命令在2.0以后已经不适用了,解决的办法如下:
PYSPARK_DRIVER_PYTHON=ipython PYSPARK_DRIVER_PYTHON_OPTS='notebook' ./bin/pyspark
然后就打开了jupyter notebook,信息如下:
[TerminalIPythonApp] WARNING | Subcommand `ipython notebook` is deprecated and will be removed in future versions.
[TerminalIPythonApp] WARNING | You likely want to use `jupyter notebook` in the future
[I 13:17:56.028 NotebookApp] [nb_conda_kernels] enabled, 2 kernels found
[I 13:17:56.148 NotebookApp] ✓ nbpresent HTML export ENABLED
[W 13:17:56.148 NotebookApp] ✗ nbpresent PDF export DISABLED: No module named nbbrowserpdf.exporters.pdf
[I 13:17:56.154 NotebookApp] [nb_conda] enabled
[I 13:17:56.243 NotebookApp] [nb_anacondacloud] enabled
[I 13:17:56.250 NotebookApp] Serving notebooks from local directory: /Users/eric/Documents/Spark/spark-2.2.0-bin-hadoop2.7
[I 13:17:56.250 NotebookApp] 0 active kernels
[I 13:17:56.250 NotebookApp] The Jupyter Notebook is running at: http://localhost:8888/
[I 13:17:56.250 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
参考文献
[1]如何在Spark2.0.2中启动Ipython Notebook. http://blog.csdn.net/circle2015/article/details/53614596