Go协程池设计思路(Task-Job-Worker)

最新推荐文章于 2024-03-22 17:34:40 发布

沈子恒

最新推荐文章于 2024-03-22 17:34:40 发布

阅读量1.6k

点赞数

分类专栏： Golang/Java/Storm

本文链接：https://blog.csdn.net/shenziheng1/article/details/105472982

版权

Golang/Java/Storm 专栏收录该内容

20 篇文章 2 订阅

订阅专栏

1. 铺垫：Go 的接收器Receiver

在go语言中，没有类的概念，但是可以给类型（结构体，自定义类型）定义方法。所谓方法就是定义了接受者的函数。接受者定义在func关键字和函数名之间。可以理解成为结构体定义函数方法，类似于C++中的类方法。

type Person struct {
    name string
    age int
}

func (p Person) say() {
    fmt.Printf("I'm %s,%d years old\n",p.name,p.age)
}

接受者类型可以是struct,也可以是指向struc的指针。

接收者的类型为struct

package main

import "fmt"

type Person struct {
name string
age int
}
func (p Person) say() {
fmt.Printf("I'm %s,%d years old\n",p.name,p.age)
}
func (p Person) older(){
    p.age = p.age +1
}
func main() {
    var p1 Person = Person{"lineshen",25}
    p1.older()
    p1.say()
    var p2 *Person = &Person{"leleyue",22}
    p2.older()
    p2.say()
}

output:

I'm lineshen,25 years old
I'm leleyue,22 years old

对于p1的调用，是没有问题的，因为传递的是结构体，在方法中改变结构体属性，并不能改变原来内存中的值。

对于p2的调用会产生疑问，p2明明是个指针，为什么再调用了older方法之后，打印结果还是22 years old?
方法的接受者是Person而调用者是*Person。其实在p2调用时存在一个转换p2.older() -> *p2.older(); p2.say() -> *p2.say()
p2是指向Person实例的指针。因此，方法执行时的接受者实际上还是一个值而非引用，因此值是不会发生改变的。

接收者的类型为指针

package main

import "fmt"

type Person struct {
name string
age int
}
func (p *Person) say() {
fmt.Printf("I'm %s,%d years old\n",p.name,p.age)
}
func (p *Person) older(){
    p.age = p.age +1
}
func main() {
    var p1 Person = Person{"lineshen",25}
    p1.older()
    p1.say()
    var p2 *Person = &Person{"leleyue",22}
    p2.older()
    p2.say()
}

output：

I'm lineshen,26 years old
I'm leleyue,23 years old

此时传递的是引用，并非值。因此，结构体在内存中属性值会跟随函数方法发生相应的改变。

2. 正题：协程池的设计思路

类似于淘宝电商，可能每分钟有百万级别的请求，最简单直观的方法是来一个请求，new goroutine处理这个请求；那么需要的goroutine也是百万级别的。这样做业务处理逻辑是清楚了，但是会导致CPU做很多无用功，因为CPU需要在不同的goroutin之间流转。所以，在实际业务中，应该限制gorotine的个数（一般协程的数量与CPU的数量保持一致，加大CPU的利用率）。一个可行的架构设计如下：

示例代码如下：

package main
import ("fmt"
		"time")

// 任务的属性应该是一个业务函数
type Task struct {
	f func() error // 函数名f, 无参，返回值为error
}

// 创建Task任务
func NewTask(arg_f func() error) *Task {
	task := Task{
		f: arg_f,
	}
	return &task
}

// Task绑定业务方法
func (task *Task) Execute() {
	task.f() // 调用任务中已经绑定好的业务方法
}

// ------------------------------------------------
type Pool struct {
	EntryChannel chan *Task  // 对外的Task入口
	JobsChannel  chan *Task  // 内部的Task队列
	workerNum    int         // 协程池中最大的woker数量
}

// 创建Pool
func NewPool(cap int) *Pool {
	pool := Pool {
	EntryChannel : make(chan *Task),
	JobsChannel  : make(chan *Task),
	workerNum : cap,
	}
	return &pool
}

// Pool绑定干活的方法
func (pool *Pool) worker(workID int) {
	// worker工作 ： 永久从JobsChannel取任务 然后执行任务
	for task := range pool.JobsChannel {
		task.Execute()
		fmt.Println("work ID ", workID, " has executed")
	}
}

// Pool绑定协程池工作方法
func (pool *Pool) run() {
	// 定义worker数量
	for i:=0; i<pool.workerNum; i++ {
		go pool.worker(i)
	}
	
	// 从EntryChannel去任务，发送给JobsChannel
	for task := range pool.EntryChannel {
		pool.JobsChannel <- task // 添加task优先级排序逻辑
	}
}

// ------------------------------------------------
func main() {
	// 创建一些任务
	task := NewTask( func() error { // 匿名函数
		fmt.Println(time.Now())
		return nil
	})
	
	// 创建协程池
	pool := NewPool(4)
	
	// 创建多任务，抛给协程池 
	go func() { // 开启新的协程，防止阻塞
		for{
			pool.EntryChannel <- task
		}
	}()
	
	// 启动协程池
	pool.run()
}

output:

work ID  0  has executed
2020-04-13 00:15:28.6042081 +0800 CST m=+0.040858901
work ID  0  has executed
2020-04-13 00:15:28.605204 +0800 CST m=+0.041854801
work ID  0  has executed
2020-04-13 00:15:28.6062025 +0800 CST m=+0.042853301
work ID  0  has executed
2020-04-13 00:15:28.6062025 +0800 CST m=+0.042853301
work ID  0  has executed
2020-04-13 00:15:28.6071987 +0800 CST m=+0.043849501
work ID  0  has executed
2020-04-13 00:15:28.6071987 +0800 CST m=+0.043849501
work ID  0  has executed
2020-04-13 00:15:28.6081959 +0800 CST m=+0.044846701
work ID  0  has executed
2020-04-13 00:15:28.6081959 +0800 CST m=+0.044846701
work ID  0  has executed
2020-04-13 00:15:28.6091933 +0800 CST m=+0.045844101
work ID  0  has executed
2020-04-13 00:15:28.6091933 +0800 CST m=+0.045844101
work ID  0  has executed
2020-04-13 00:15:28.6101907 +0800 CST m=+0.046841501
work ID  0  has executed
2020-04-13 00:15:28.6101907 +0800 CST m=+0.046841501
work ID  0  has executed
2020-04-13 00:15:28.6101907 +0800 CST m=+0.046841501
work ID  0  has executed
2020-04-13 00:15:28.6111882 +0800 CST m=+0.047839001
work ID  0  has executed
2020-04-13 00:15:28.5683041 +0800 CST m=+0.004954901
work ID  1  has executed
2020-04-13 00:15:28.5673053 +0800 CST m=+0.003956101
work ID  3  has executed
2020-04-13 00:15:28.6121853 +0800 CST m=+0.048836101
work ID  3  has executed
2020-04-13 00:15:28.6121853 +0800 CST m=+0.048836101
work ID  3  has executed
2020-04-13 00:15:28.6121853 +0800 CST m=+0.048836101
work ID  3  has executed
2020-04-13 00:15:28.5673053 +0800 CST m=+0.003956101
work ID  2  has executed
2020-04-13 00:15:28.6151779 +0800 CST m=+0.051828701
work ID  2  has executed
2020-04-13 00:15:28.6151779 +0800 CST m=+0.051828701
work ID  2  has executed
2020-04-13 00:15:28.6161747 +0800 CST m=+0.052825501
work ID  2  has executed
2020-04-13 00:15:28.6161747 +0800 CST m=+0.052825501
work ID  2  has executed
2020-04-13 00:15:28.6171731 +0800 CST m=+0.053823901
work ID  2  has executed
2020-04-13 00:15:28.6171731 +0800 CST m=+0.053823901
work ID  2  has executed
2020-04-13 00:15:28.6181705 +0800 CST m=+0.054821301
work ID  2  has executed
2020-04-13 00:15:28.6181705 +0800 CST m=+0.054821301
work ID  2  has executed
2020-04-13 00:15:28.6181705 +0800 CST m=+0.054821301
work ID  2  has executed
2020-04-13 00:15:28.6181705 +0800 CST m=+0.054821301
work ID  2  has executed
2020-04-13 00:15:28.6191668 +0800 CST m=+0.055817601
work ID  2  has executed
2020-04-13 00:15:28.6191668 +0800 CST m=+0.055817601
work ID  2  has executed
2020-04-13 00:15:28.6191668 +0800 CST m=+0.055817601
work ID  2  has executed
2020-04-13 00:15:28.6191668 +0800 CST m=+0.055817601
work ID  2  has executed
2020-04-13 00:15:28.6201639 +0800 CST m=+0.056814701
work ID  2  has executed
2020-04-13 00:15:28.6111882 +0800 CST m=+0.047839001
work ID  0  has executed
2020-04-13 00:15:28.6201639 +0800 CST m=+0.056814701
work ID  0  has executed
2020-04-13 00:15:28.6211612 +0800 CST m=+0.057812001
work ID  0  has executed
2020-04-13 00:15:28.6201639 +0800 CST m=+0.056814701
work ID  2  has executed
2020-04-13 00:15:28.6221586 +0800 CST m=+0.058809401
work ID  2  has executed
2020-04-13 00:15:28.6221586 +0800 CST m=+0.058809401
work ID  2  has executed
2020-04-13 00:15:28.6251574 +0800 CST m=+0.061808201
work ID  2  has executed
2020-04-13 00:15:28.6261523 +0800 CST m=+0.062803101
work ID  2  has executed
2020-04-13 00:15:28.6261523 +0800 CST m=+0.062803101
work ID  2  has executed
2020-04-13 00:15:28.6271454 +0800 CST m=+0.063796201
work ID  2  has executed
2020-04-13 00:15:28.6271454 +0800 CST m=+0.063796201
work ID  2  has executed
2020-04-13 00:15:28.6281435 +0800 CST m=+0.064794301
work ID  2  has executed
2020-04-13 00:15:28.6281435 +0800 CST m=+0.064794301
work ID  2  has executed
2020-04-13 00:15:28.6291399 +0800 CST m=+0.065790701
work ID  2  has executed
2020-04-13 00:15:28.6291399 +0800 CST m=+0.065790701
work ID  2  has executed
2020-04-13 00:15:28.6301372 +0800 CST m=+0.066788001
work ID  2  has executed
2020-04-13 00:15:28.6301372 +0800 CST m=+0.066788001
work ID  2  has executed
2020-04-13 00:15:28.6142098 +0800 CST m=+0.050860601
work ID  3  has executed
2020-04-13 00:15:28.6311355 +0800 CST m=+0.067786301
work ID  3  has executed
2020-04-13 00:15:28.6321319 +0800 CST m=+0.068782701
work ID  3  has executed
2020-04-13 00:15:28.6321319 +0800 CST m=+0.068782701
work ID  3  has executed
2020-04-13 00:15:28.6331302 +0800 CST m=+0.069781001
work ID  3  has executed
2020-04-13 00:15:28.6331302 +0800 CST m=+0.069781001
work ID  3  has executed
2020-04-13 00:15:28.6331302 +0800 CST m=+0.069781001
work ID  3  has executed
2020-04-13 00:15:28.6391157 +0800 CST m=+0.075766501
work ID  3  has executed

学习视频：https://www.bilibili.com/video/BV1cx41197j5?from=search&seid=15155039435359774234

沈子恒

关注

0
点赞
踩
1

收藏

觉得还不错? 一键收藏
1
评论
Go协程池设计思路(Task-Job-Worker)

1. 铺垫：Go 的接收器Receiver在go语言中，没有类的概念，但是可以给类型（结构体，自定义类型）定义方法。所谓方法就是定义了接受者的函数。接受者定义在func关键字和函数名之间。可以理解成为结构体定义函数方法，类似于C++中的类方法。type Person struct { name string age int}func (p Person) say()...
复制链接

扫一扫