es ingest-attachment的安装与使用 个人记录

版本号：

     es 5.4

es数据库的安装略；

linux:在es数据库的目录下有个bin文件夹在这个文件夹下执行

./elasticsearch-plugin install ingest-attachment 会直接安装ingest-attachment插件

windows:在es数据库的目录下有个bin文件夹在这个文件夹下执行

elasticsearch-pluginbat install ingest-attachment

或者通过其他的渠道货渠道ingest-attachment插件的文件防止到es数据库目录下的pugins文件夹下也可以

在安装完插件之后需要重起节点，集群的每个节点都需要安装

使用

创建自己的文本抽取管道pipeline

其中url里的attachment可以自定义

curl -X PUT "localhost:9200/_ingest/pipeline/attachment" -d '{
 "description" : "Extract attachment information",
 "processors":[
 {
    "attachment":{
        "field":"data",
        "indexed_chars" : -1,
        "ignore_missing":true
     }
 },
 {
     "remove":{"field":"data"}
 }]}'

若想保存传入的base64数据则去掉

{
     "remove":{"field":"data"}
 }

插入base64数据

curl -X PUT "localhost:9200/pdftest/pdf/1?pipeline=attachment" -d '
{
   "data":"QmFzZTY057yW56CB6K+05piOCuOAgOOAgEJhc2U2NOe8lueggeimgeaxguaKijPkuKo45L2N5a2X6IqC77yIMyo4PTI077yJ6L2s5YyW5Li6NOS4qjbkvY3nmoTlrZfoioLvvIg0KjY9MjTvvInvvIzkuYvlkI7lnKg25L2N55qE5YmN6Z2i6KGl5Lik5LiqMO+8jOW9ouaIkDjkvY3kuIDkuKrlrZfoioLnmoTlvaLlvI/jgIIg5aaC5p6c5Ymp5LiL55qE5a2X56ym5LiN6LazM+S4quWtl+iKgu+8jOWImeeUqDDloavlhYXvvIzovpPlh7rlrZfnrKbkvb/nlKgnPSfvvIzlm6DmraTnvJbnoIHlkI7ovpPlh7rnmoTmlofmnKzmnKvlsL7lj6/og73kvJrlh7rnjrAx5oiWMuS4qic9J+OAggoK44CA44CA5Li65LqG5L+d6K+B5omA6L6T5Ye655qE57yW56CB5L2N5Y+v6K+75a2X56ym77yMQmFzZTY05Yi25a6a5LqG5LiA5Liq57yW56CB6KGo77yM5Lul5L6/6L+b6KGM57uf5LiA6L2s5o2i44CC57yW56CB6KGo55qE5aSn5bCP5Li6Ml42PTY077yM6L+Z5Lmf5pivQmFzZTY05ZCN56ew55qE55Sx5p2l44CC"
}'

 

java api的使用

获取es连接的步骤这里就省略了，网上一搜一大把

以下以传入数据是map为例

Map<String,String> source=new HashMap<String,String>();

source.put("tital","java 插入");

source.put("data","QmFzZTY057yW56CB6K+05piOCuOAgOOAgEJhc2U2NOe8lueggeimgeaxguaKijPkuKo45L2N5a2X6IqC77yIMyo4PTI077yJ6L2s5YyW5Li6NOS4qjbkvY3nmoTlrZfoioLvvIg0KjY9MjTvvInvvIzkuYvlkI7lnKg25L2N55qE5YmN6Z2i6KGl5Lik5LiqMO+8jOW9ouaIkDjkvY3kuIDkuKrlrZfoioLnmoTlvaLlvI/jgIIg5aaC5p6c5Ymp5LiL55qE5a2X56ym5LiN6LazM+S4quWtl+iKgu+8jOWImeeUqDDloavlhYXvvIzovpPlh7rlrZfnrKbkvb/nlKgnPSfvvIzlm6DmraTnvJbnoIHlkI7ovpPlh7rnmoTmlofmnKzmnKvlsL7lj6/og73kvJrlh7rnjrAx5oiWMuS4qic9J+OAggoK44CA44CA5Li65LqG5L+d6K+B5omA6L6T5Ye655qE57yW56CB5L2N5Y+v6K+75a2X56ym77yMQmFzZTY05Yi25a6a5LqG5LiA5Liq57yW56CB6KGo77yM5Lul5L6/6L+b6KGM57uf5LiA6L2s5o2i44CC57yW56CB6KGo55qE5aSn5bCP5Li6Ml42PTY077yM6L+Z5Lmf5pivQmFzZTY05ZCN56ew55qE55Sx5p2l44CC");

client.prepareIndex("test","test").setPipeline("attachment").setSource(source).excute.actionGet();