ubuntu安装elasticserach-golang使用

ALEX_CYL学习记录

已于 2022-07-04 21:03:36 修改

阅读量350

点赞数

分类专栏： golang 文章标签： ubuntu golang elasticsearch

于 2022-06-28 10:17:35 首次发布

本文链接：https://blog.csdn.net/ALEX_CYL/article/details/125485460

版权

golang 专栏收录该内容

7 篇文章 0 订阅

订阅专栏

elasticserach 安装

Elasticsearch 是一个高可扩展性的开源全文本搜索和分析引擎。它使你可以快速，近乎实时地存储，搜索和分析大量数据。它通常用作支持具有复杂搜索功能和要求的应用程序的基础引擎/技术。

1.下载资源包
https://www.elastic.co/cn/downloads/elasticsearch

2.解压，本机解压到/home/yunlongchen/opt/elasticsearch中
tar elasticsearch-8.2.3-linux-x86_64.tar.gz

3.es为了安全性，不允许root用户操作，需使用普通用户模式

4.由于第三点的要求，该elasticsearch/config/elasticsearch.keystore文件属于root用户，故需将权限授予给普通用户

chown -R yunlongchen:yunlongchen /home/yunlongchen/elasticserach/
chown -R yunlongchen:yunlongchen  /home/yunlongchen/opt/elasticsearch/elasticsearch-8.2.3/
chown -R yunlongchen:yunlongchen  /home/yunlongchen/opt/elasticsearch/elasticsearch-8.2.3/config/elasticsearch.keystore

5.配置 ./config/elasticsearch.yml

 23| node.name: node-1
 
path.data: /home/yunlongchen/opt/elasticsearch/elasticsearch-8.2.3/data
path.logs: /home/yunlongchen/opt/elasticsearch/elasticsearch-8.2.3/logs

network.host: 10.141.65.188 //修改成自己的IP
http.port: 9200
transport.profiles.default.port: 9301

discovery.seed_hosts: ["10.141.65.188:9301"]

cluster.initial_master_nodes: ["node-1"]

xpack.security.enabled: false
#装es-header需添加以下内容
http.cors.enabled: true
http.cors.allow-origin: "*"
http.cors.allow-headers: Authorization,X-Requested-With,Content-Length,Content-Type

6.启动es

yunlongchen@cyl:~/opt/elasticsearch/elasticsearch-8.2.3$ ./bin/elasticsearch

7.启动/安装时遇到的问题/注意事项：
１.es基于java，故需要安装jdk
2.错误：max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
解决：编辑/etc/sysctl.conf，追加以下内容：

vm.max_map_count=655350

保存后，执行：sysctl -p

3.错误：max file descriptors [4096] for elasticsearch process is too low, increase to at least [65535]
ulimit -Hn：是max number of open file descriptors的hard限制
ulimit -Sn：是max number of open file descriptors的soft限制

yunlongchen@cyl:~/opt/elasticsearch/elasticsearch-8.2.3$ ulimit -Hn
4096
yunlongchen@cyl:~/opt/elasticsearch/elasticsearch-8.2.3$ ulimit -Sn
1024

解决：编辑 /etc/security/limits.conf，追加以下内容

* soft nofile 65535
* hard nofile 65535
yunlongchen soft nproc 4096
yunlongchen hard nproc 4096
# End of file

此文件修改后需要重新登录用户，才会生效

yunlongchen@cyl:~/opt/elasticsearch/elasticsearch-8.2.3$ ulimit -Sn
1024
yunlongchen@cyl:~/opt/elasticsearch/elasticsearch-8.2.3$ ulimit -Hn
4096
yunlongchen@cyl:~/opt/elasticsearch/elasticsearch-8.2.3$ sudo su
root@cyl:/home/yunlongchen/opt/elasticsearch/elasticsearch-8.2.3# su yunlongchenyunlongchen@cyl:~/opt/elasticsearch/elasticsearch-8.2.3$ ulimit -Sn
65535
yunlongchen@cyl:~/opt/elasticsearch/elasticsearch-8.2.3$ ulimit -Hn
65535
yunlongchen@cyl:~/opt/elasticsearch/elasticsearch-8.2.3$ ./bin/elasticsearch

4.错误：Transport SSL must be enabled if security is enabled. Please set [xpack.security.transport.ssl.enabled] to [true] or disable security by setting [xpack.security.enabled] to [false]
解决：修改../cofig/elasticsearch.yml文件，关闭安全设置

#action.destructive_requires_name: false
xpack.security.enabled: false

参考资料：
Ubuntu20.04安装elasticsearch8.1.3
ubuntu下安装elasticsearch7.5.1集群+kibana
Ubuntu 安装elasticsearch集群
 centos8安装ElasticSearch8并配置

golang-es

使用第三方库"github.com/elastic/go-elasticsearch/v7"
注意：elasticsearch中创建index时，名字只能是小写字母，不支持大写字母，但支持＇_＇的使用，否则无法添加索引，
报错：

Bad Request] {“error”:{“root_cause”:[{“type”:“invalid_index_name_exception”,“reason”:“Invalid index name [nginxLog],　must be lowercase”,“index_uuid”:“＿na＿”,“index”:“nginxLog”}],“type”:“invalid_index_name_exception”,“reason”:“Invalid　index name [nginxLog], must be lowercase”

package main

import (
	"context"
	"encoding/json"
	"fmt"
	"log"
	"strconv"
	"strings"
	"time"

	"github.com/elastic/go-elasticsearch/v7"
	"github.com/elastic/go-elasticsearch/v7/esapi"
)

type Tweet struct {
	User    string `json:"user"`
	Message string `json:"message"`
}

func jsonStruct(Struct interface{}) string {
	// Create struct instance of the Elasticsearch fields struct object

	// Marshal the struct to JSON and check for errors
	b, err := json.Marshal(Struct)
	if err != nil {
		fmt.Println("json.Marshal ERROR:", err)
		return string(err.Error())
	}

	return string(b)
}

func main() {

	// Allow for custom formatting of log output
	log.SetFlags(0)

	var (
		r map[string]interface{}
		// wg sync.WaitGroup
	)

	cfg := elasticsearch.Config{
		Addresses: []string{
			"http://10.141.65.188:9200",
		},
		Username: "user",
		Password: "pass",
	}
	client, err := elasticsearch.NewClient(cfg)
	if err != nil {
		fmt.Println("cnnect es err,", err)
		return
	}
	fmt.Println("conn es succ,", client)

	// 1. Get cluster info
	//
	res, err := client.Info()
	if err != nil {
		log.Fatalf("Error getting response: %s", err)
	}
	// Check response status
	if res.IsError() {
		log.Fatalf("Error: %s", res.String())
	}
	// Deserialize the response into a map.
	if err := json.NewDecoder(res.Body).Decode(&r); err != nil {
		log.Fatalf("Error parsing the response body: %s", err)
	}
	// Print client and server version numbers.
	log.Printf("Client: %s", elasticsearch.Version)
	log.Printf("Server: %s", r["version"].(map[string]interface{})["number"])
	log.Println(strings.Repeat("~", 37))

	tweet := Tweet{User: "olivere", Message: "Take Five"}
	bod := jsonStruct(tweet)
	ctx := context.Background()

	//index 类似数据库　Type 类似表的概念　BodyJosn存放实际的内容
	// 链式操作，“.”返回的是对象
	for i := 60; i < 70; i++ {
		// Instantiate a request object
		req := esapi.IndexRequest{
			Index:      "twitter",
			DocumentID: strconv.Itoa(i + 1),
			Body:       strings.NewReader(bod),
			Refresh:    "true",
		}

		res, err := req.Do(ctx, client)
		if err != nil {
			log.Fatalf("IndexRequest ERROR: %s", err)
		}

		defer res.Body.Close()

		// if res.IsError() {
		// 	log.Printf("%s ERROR indexing document ID=%d", res.Status(), i+1)
		// } else {

		// 	// Deserialize the response into a map.
		// 	var resMap map[string]interface{}
		// 	if err := json.NewDecoder(res.Body).Decode(&resMap); err != nil {
		// 		log.Printf("Error parsing the response body: %s", err)
		// 	} else {
		// 		log.Printf("\nIndexRequest() RESPONSE:")
		// 		// Print the response status and indexed document version.
		// 		fmt.Println("Status:", res.Status())
		// 		fmt.Println("Result:", resMap["result"])
		// 		fmt.Println("Version:", int(resMap["_version"].(float64)))
		// 		fmt.Println("resMap:", resMap)
		// 		fmt.Println("")
		// 	}
		// }

		time.Sleep(time.Second * 2)

	}

	fmt.Println("insert succ")
}

Kibana安装

1.下载网址
https://www.elastic.co/cn/downloads/past-releases/
2.安装解压后，启动kibana,在非root用户执行

yunlongchen@cyl:~/opt/kibana/kibana-8.2.3$ ./bin/kibana

3.修改配置：在config/kibana.yml文件:

server.port: 5601
server.host: 10.141.65.189
elasticsearch.hosts: ["http://10.141.65.189:9200/"]
i18n.locale: "zh-CN"

参考资料：
Ubuntu20.04安装elasticsearch8.1.3

配置节点密码
 超实用的ELK日志分析系统

ALEX_CYL学习记录

关注

0
点赞
踩
0

收藏

觉得还不错? 一键收藏
0
评论
ubuntu安装elasticserach-golang使用

1.下载资源包https://www.elastic.co/cn/downloads/elasticsearch2.解压，本机解压到/home/yunlongchen/opt/elasticsearch中3.es为了安全性，不允许root用户操作，需使用普通用户模式4.由于第三点的要求，该elasticsearch/config/elasticsearch.keystore文件属于root用户，故需将权限授予给普通用户5.配置 6.启动es7.启动/安装时遇到的问题/注意事项：１.es基于jav
复制链接

扫一扫