使用golang进行kong限流插件开发

最新推荐文章于 2024-05-14 14:48:55 发布

lampNick

最新推荐文章于 2024-05-14 14:48:55 发布

阅读量2.7k

点赞数 1

分类专栏： go kong 文章标签： kong plugins rate limiting go插件开发

本文链接：https://blog.csdn.net/scun_cg/article/details/108583678

版权

go 同时被 2 个专栏收录

10 篇文章 0 订阅

订阅专栏

kong

1 篇文章 0 订阅

订阅专栏

个人博客原文地址：http://www.lampnick.com/php/950

rate limiting限流仓库地址

https://github.com/lampnick/kong-rate-limiting-golang

前置条件

获取编译基础镜像（Go插件是使用-buildmode=plugin标志编译的，该标志允许插件服务器动态加载它们。）

docker pull kong/go-plugin-tool:latest-centos-7

go-pluginserver可执行文件路径

默认路径：/usr/local/bin
自定义路径有两种方式：
1.环境变量增加go_pluginserver_exe
2.kong配置文件设置go_pluginserver_exe

go_plugins_dir go插件目录

kong配置文件中设置go插件的路径，默认值为off，表明禁用go插件支持

将你编译好的go插件复制到go_plugins_dir
和lua插件开发一样，需要将你的go插件名称配置到plugins属性中，这样kong才会加载你的插件
允许你的go插件使用常规模式，可通过Admin API或者声明式配置文件管理

构建Go Plugin Server和你的go插件

go插件服务器（ Go Plugin Server ）是一个典型的go应用，可使用如下流程

初始化go.mod

Nick-Mac:kong-go-plugin nick$ go mod init kong-go-plugin
go: creating new go.mod: module kong-go-plugin

Nick-Mac:kong-go-plugin nick$ cat go.mod 
module kong-go-plugin

go 1.13

获取Go Plugin Server

Nick-Mac:kong-go-plugin nick$ go get -d -v github.com/Kong/go-pluginserver
go: downloading github.com/Kong/go-pluginserver v0.5.1
go: extracting github.com/Kong/go-pluginserver v0.5.1
go: downloading github.com/Kong/go-pdk v0.5.0
go: extracting github.com/Kong/go-pdk v0.5.0

Nick-Mac:kong-go-plugin nick$ cat go.mod 
module kong-go-plugin

go 1.13

require github.com/Kong/go-pluginserver v0.5.1 // indirect

构建Go Plugin Server，会在当前目录下生成go-pluginserver可执行文件

Nick-Mac:kong-go-plugin nick$ go build github.com/Kong/go-pluginserver
go: finding github.com/Kong/go-pdk v0.5.0

Nick-Mac:kong-go-plugin nick$ ll
total 29952
drwxr-xr-x  5 nick  staff   160B  9  7 11:23 .
drwxr-xr-x  9 nick  staff   288B  9  7 11:14 ..
-rwxr-xr-x  1 nick  staff    15M  9  7 11:23 go-pluginserver
-rw-r--r--  1 nick  staff    91B  9  7 11:20 go.mod
-rw-r--r--  1 nick  staff   1.2K  9  7 11:20 go.sum

接下来将构建一个go插件，克隆下面的示例仓库，使用使用-buildmode plugin 进行构建，会在当前目录生成一个.so文件

1.克隆
Nick-Mac:kong-go-plugin-example nick$ git clone https://github.com/Kong/go-plugins.git
Cloning into 'go-plugins'...
remote: Enumerating objects: 35, done.
remote: Counting objects: 100% (35/35), done.
remote: Compressing objects: 100% (29/29), done.
remote: Total 35 (delta 13), reused 21 (delta 5), pack-reused 0
Unpacking objects: 100% (35/35), done.
2.切到示例目录
Nick-Mac:kong-go-plugin-example nick$ cd go-plugins/
3.查看有哪些文件
Nick-Mac:go-plugins nick$ ll
total 48
drwxr-xr-x   9 nick  staff   288B  9  7 11:27 .
drwxr-xr-x   3 nick  staff    96B  9  7 11:27 ..
drwxr-xr-x  12 nick  staff   384B  9  7 11:27 .git
-rw-r--r--   1 nick  staff   166B  9  7 11:27 Makefile
-rw-r--r--   1 nick  staff   367B  9  7 11:27 README.md
-rw-r--r--   1 nick  staff   536B  9  7 11:27 go-hello.go
-rw-r--r--   1 nick  staff   1.2K  9  7 11:27 go-log.go
-rw-r--r--   1 nick  staff    82B  9  7 11:27 go.mod
-rw-r--r--   1 nick  staff   1.1K  9  7 11:27 go.sum
4.开始构建
Nick-Mac:go-plugins nick$ go build -buildmode plugin go-hello.go
5.查看文件多了一个go-hello.so
Nick-Mac:go-plugins nick$ ll
total 4960
drwxr-xr-x  10 nick  staff   320B  9  7 11:30 .
drwxr-xr-x   3 nick  staff    96B  9  7 11:27 ..
drwxr-xr-x  12 nick  staff   384B  9  7 11:27 .git
-rw-r--r--   1 nick  staff   166B  9  7 11:27 Makefile
-rw-r--r--   1 nick  staff   367B  9  7 11:27 README.md
-rw-r--r--   1 nick  staff   536B  9  7 11:27 go-hello.go
-rw-r--r--   1 nick  staff   2.4M  9  7 11:30 go-hello.so
-rw-r--r--   1 nick  staff   1.2K  9  7 11:27 go-log.go
-rw-r--r--   1 nick  staff    82B  9  7 11:27 go.mod
-rw-r--r--   1 nick  staff   1.1K  9  7 11:27 go.sum

环境一致性约束

所有公共库需要版本一致
- Kong/go-pdk
- 所有go库文件 (像fmt, rpc, reflect等)
- 操作系统库文件, 像libpthread, libc, ld-xxxx等
go编译器版本完全一致
go环境变量，像 $G O R O O T 、$ GOPATH等一致

典型的由于环境不一致出现的错误信息如下

failed to open plugin kong: plugin.Open("/path/go-plugins/go-hello"): plugin was built with a different version of package github.com/Kong/go-pdk/bridge

开发go插件

go插件开发流程

定义一个结构体类型保存配置文件

用lua写的插件通过schema来指定怎样读取和验证来自数据库和Admin API中的配置数据。由于GO是静态类型语言，都需要用配置结构体定义
type MyConfig struct {
    Path   string //这里配置的会在konga添加插件时显示出来
    Reopen bool
}
公有属性将会被配置数据填充，如果希望在数据库中使用不同的名称，可以使用encoding/json加tag的方式
type MyConfig struct {
    Path   string `json:my_file_path`
    Reopen bool   `json:reopen`
}

使用New()创建一个实例

你的go插件必须定义一个名叫New的函数来创建这个类型的实例并返回一个interface{}类型
func New() interface{} {
    return &MyConfig{}
}

添加处理阶段方法

你可以在请求的生命周期的各个阶段实现自定义的逻辑。如在"access"阶段,定义一个名为Access的方法
func (conf *MyConfig) Access (kong *pdk.PDK) {
  ...
}
你可以实现自定义逻辑的阶段方法有如下几种
Certificate
Rewrite
Access
Preread
Log

编译go插件
将生成的.so文件放到go_plugins_dir定义的目录中

部署

kong配置文件修改

plugins = bundled,nick-rate-limiting
go_plugins_dir = /etc/kong/plugins
go_pluginserver_exe = /usr/local/bin/go-pluginserver

构建go-pluginserver

在go-pluginserver中执行go build github.com/Kong/go-pluginserver
会生成 go-pluginserver文件，复制到/usr/local/bin目录

编译go插件

go build -buildmode plugin  custom-rate-limiting.go && cp custom-rate-limiting.so

将生成的.so文件放到go_plugins_dir定义的目录中

cp custom-rate-limiting.so ../plugins/

重启kong

kong prepare && kong reload
- 在konga中配置插件
- 测试请求是否正常，规则是否生效

问题

kong start -c /etc/kong/kong.conf --vv时报错sh: /usr/local/bin/go-pluginserver: cannot execute binary file

由于在mac上交叉编译
解决方法：
直接在kong机器上编译go-pluginserver

插件报错

2020/09/07 15:35:52 failed to open plugin go-hello: plugin.Open("/etc/kong/plugins/go-hello.so"): /etc/kong/plugins/go-hello.so: invalid ELF header
nginx: [error] init_by_lua error: /usr/local/share/lua/5.1/kong/db/dao/plugins/go.lua:481: bad argument #1 to 'ipairs' (table expected, got nil)
stack traceback:
	[C]: in function 'ipairs'
	/usr/local/share/lua/5.1/kong/db/dao/plugins/go.lua:481: in function 'get_plugin'
	/usr/local/share/lua/5.1/kong/db/dao/plugins/go.lua:515: in function 'load_plugin'
	/usr/local/share/lua/5.1/kong/db/dao/plugins.lua:151: in function 'load_plugin_handler'
	/usr/local/share/lua/5.1/kong/db/dao/plugins.lua:227: in function 'load_plugin'
	/usr/local/share/lua/5.1/kong/db/dao/plugins.lua:275: in function 'load_plugin_schemas'
	/usr/local/share/lua/5.1/kong/init.lua:484: in function 'init'
	init_by_lua:3: in main chunk

stack traceback:
	[C]: in function 'error'
	/usr/local/share/lua/5.1/kong/cmd/start.lua:75: in function 'cmd_exec'
	/usr/local/share/lua/5.1/kong/cmd/init.lua:88: in function </usr/local/share/lua/5.1/kong/cmd/init.lua:88>
	[C]: in function 'xpcall'
	/usr/local/share/lua/5.1/kong/cmd/init.lua:88: in function </usr/local/share/lua/5.1/kong/cmd/init.lua:45>
	/usr/local/bin/kong:9: in function 'file_gen'
	init_worker_by_lua:49: in function <init_worker_by_lua:47>
	[C]: in function 'xpcall'
	init_worker_by_lua:56: in function <init_worker_by_lua:54>
解决方案:
直接在kong机器上编译

访问报错:message: “An unexpected error occurred”

查看kong管理地址中错误日志路径
http://10.5.24.224:8001/
"nginx_err_logs": "/usr/local/kong/logs/error.log",
tail -f /usr/local/kong/logs/error.log 日志报错如下
2020/09/07 16:21:22 [notice] 122415#0: *29 [kong] go.lua:97 go-pluginserver terminated: exit 0, context: ngx.timer
2020/09/07 16:21:22 [notice] 122415#0: *29 [kong] go.lua:86 Starting go-pluginserver, context: ngx.timer
2020/09/07 16:21:22 [crit] 122415#0: *29381 connect() to unix:/usr/local/kong/go_pluginserver.sock failed (2: No such file or directory), client: 10.5.216.251, server: kong, request: "GET /api/index.php?r=site/login-data HTTP/1.1", host: "hd.myscrm.cn:8000"
2020/09/07 16:21:22 [error] 122415#0: *29381 [kong] go.lua:140 [go-hello] trying to connect: no such file or directory, client: 10.5.216.251, server: kong, request: "GET /api/index.php?r=site/login-data HTTP/1.1", host: "hd.myscrm.cn:8000"
2020/09/07 16:21:22 [error] 122415#0: *29381 [kong] go.lua:429 [go-hello] starting instance: no such file or directory, client: 10.5.216.251, server: kong, request: "GET /api/index.php?r=site/login-data HTTP/1.1", host: "hd.myscrm.cn:8000"
2020/09/07 16:21:22 [error] 122415#0: *29381 lua coroutine: runtime error: /usr/local/share/lua/5.1/kong/db/dao/plugins/go.lua:432: no such file or directory
stack traceback:
coroutine 0:
	[C]: in function 'error'
	/usr/local/share/lua/5.1/kong/db/dao/plugins/go.lua:432: in function 'get_instance'
	/usr/local/share/lua/5.1/kong/db/dao/plugins/go.lua:498: in function </usr/local/share/lua/5.1/kong/db/dao/plugins/go.lua:497>
coroutine 1:
	[C]: in function 'resume'
	coroutine.wrap:21: in function <coroutine.wrap:21>
	/usr/local/share/lua/5.1/kong/init.lua:757: in function 'access'
	access_by_lua(nginx-kong.conf:87):2: in main chunk, client: 10.5.216.251, server: kong, request: "GET /api/index.php?r=site/login-data HTTP/1.1", host: "hd.myscrm.cn:8000"
2020/09/07 16:21:22 [error] 122415#0: *29381 [kong] init.lua:759 [go-hello] /usr/local/share/lua/5.1/kong/db/dao/plugins/go.lua:432: no such file or directory, client: 10.5.216.251, server: kong, request: "GET /api/index.php?r=site/login-data HTTP/1.1", host: "hd.myscrm.cn:8000"