python使用spark实现推荐系统_spark的als和es实现推荐系统

用Apache Spark & Elasticsearch构建推荐系统

安装准备

安装es

$ wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.3.0.tar.gz

$ tar xfz elasticsearch-5.3.0.tar.gz

image.png

$ cd elasticsearch-5.3.0

$ ./bin/elasticsearch-plugin install https://github.com/MLnick/elasticsearch-vector-scoring/releases/download/v5.3.0/elasticsearch-vector-scoring-5.3.0.zip

启动es

./bin/elasticsearch

查看已经启动了向量排序插件

image.png

安装es的python客户端

$ pip install elasticsearch

下载spark与es之间连接器

$ wget http://download.elastic.co/hadoop/elasticsearch-hadoop-5.3.0.zip

$ unzip elasticsearch-hadoop-5.3.0.zip

下载Spark