jupyter pyspark py4j.java_gateway:An error occurred while trying to connect to the Java server问题

问题：通过docker方式启动jupyter notebook，通过yarn client方式创建spark session报错误

ConnectionRefusedError: [Errno 111] Connection refused
ERROR:py4j.java_gateway:An error occurred while trying to connect to the Java server (127.0.0.1:44353)

 

问题定位：

vi /tmp/jupyter_consolve.log

在日志中找到：

[I 19:11:07.544 LabApp]  or http://127.0.0.1:47352/application75/?token=...
21/02/08 19:18:35 WARN Utils: Service 'SparkUI' could not bind on port 4046. Attempting port 4047.
21/02/08 19:18:35 WARN Utils: Service 'SparkUI' could not bind on port 4047. Attempting port 4048.
21/02/08 19:18:35 WARN Utils: Service 'SparkUI' could not bind on port 4048. Attempting port 4049.
21/02/08 19:18:35 WARN Utils: Service 'SparkUI' could not bind on port 4049. Attempting port 4050.
21/02/08 19:18:35 WARN Utils: Service 'SparkUI' could not bind on port 4050. Attempting port 4051.