ipython notebook 安装配置pyspark，全测试完成

1、安装 pip install ipython notebook

2、创建登陆密码
在服务器上启动IPython，生成自定义密码的sha1：

from IPython.lib import passwd
passwd()
Enter password: 密码1
Verify password:
Out[2]: ‘sha1:026678de36b2:e4b83078e02c470b15789ade069359a20b0385dd’
导入passwd方法，并调用。在输入两次密码之后，程序会生成该密码的sha1加密字符串。要牢记自己输入的密码，并且记录下生成的加密字符串，下面的配置要用到。

3、生成配置文件
jupyter notebook –generate-config
输出配置文件路径：Writing default config to: ~/.jupyter/jupyter_notebook_config.py

4、编辑配置文件
vim ~/.jupyter/jupyter_notebook_config.py

a、配置远程连接
c = get_config()
# Notebook config
c.NotebookApp.pylab = ‘enable’ # if you want plotting support always
c.NotebookApp.certfile = u’/home/yourname/.jupyter/mycert.pem’
c.NotebookApp.ip = ‘*’
c.NotebookApp.open_browser = False
c.NotebookApp.password = u’sha1:026678de36b2:e4b83078e02c470b15789ade069359a20b0385dd’
# It’s a good idea to put it on a known, fixed port
c.NotebookApp.port = 8080
##目录
c.NotebookApp.notebook_dir =u’目录’
b、指向启动
jupyter notebook

5、配置pyspark
打开./bashrc文件: vim ~/.bashrc
添加如下两条全局命令
export PYSPARK_DRIVER_PYTHON=jupyter
export PYSPARK_DRIVER_PYTHON_OPTS=”notebook”
6、配置后台运行
A、启动local[*]
nohup jupyter notebook >/dev/null 2>&1 &
B、启动 集群
nohup pyspark –master spark://d1:7077 >/dev/null 2>&1 &
7、测试
在浏览器输入 服务器ip:8080 ，打开ipython，输入 密码1
in[1]:
from pyspark import SparkContext
try:
sc.stop()
except:
pass
sc=SparkContext()
print(“version: ” + str(sc.version))
out[1]:
version: 2.1.1