golang实现协程池

陈墨1234

已于 2024-08-03 10:42:29 修改

阅读量511

点赞数 18

分类专栏： golang进阶之路文章标签： golang 开发语言后端

于 2024-08-03 09:52:51 首次发布

本文链接：https://blog.csdn.net/weixin_37078439/article/details/140885988

版权

golang进阶之路专栏收录该内容

12 篇文章 0 订阅

订阅专栏

go协程池可以看成一个被初始化的固定大小的协程循环读取函数队列，获取是否有可供调用的函数队列，如果有，则协程池中的一个协程调用并执行该函数，talk is cheap,show me the code。

主体协程池代码如下所示：

package goroutine

import (
	"context"
	"fmt"
	"sync"
)

// go实现简单协程池
type Pool struct {
	ctx   context.Context
	tasks []fn
	lock  sync.Locker
}

type fn func()

func Init(ctx context.Context, cnt int) *Pool {
	pool := &Pool{
		ctx:   ctx,
		tasks: make([]fn, 0),
		lock:  NewSpinLock(), //自定义自旋锁实现，如果使用go自带的锁，则可能会出现死锁问题
	}
	for i := 0; i < cnt; i++ {
		go func(pool *Pool, idx int) {
			for {
				select {
				case <-pool.ctx.Done():
					fmt.Println("pool exit", idx)
					return
				default:
					fmt.Println("pool exec:", idx)
					pool.lock.Lock()
					if len(pool.tasks) == 0 {
						pool.lock.Unlock()
						continue
					}
					fc := pool.tasks[0]
					pool.tasks = pool.tasks[1:]
					pool.lock.Unlock()
					fc()
				}
			}

		}(pool, i)
	}
	return pool
}

func (p *Pool) Put(fc fn) {
	p.lock.Lock()
	p.tasks = append(p.tasks, fc)
	p.lock.Unlock()
}

自定义的自旋锁实现代码如下：

package goroutine

import (
	"runtime"
	"sync"
	"sync/atomic"
)

type spinLock uint32

const maxBackoff = 16

func (sl *spinLock) Lock() {
	backoff := 1
	for !atomic.CompareAndSwapUint32((*uint32)(sl), 0, 1) {
		// Leverage the exponential backoff algorithm, see https://en.wikipedia.org/wiki/Exponential_backoff.
		for i := 0; i < backoff; i++ {
			runtime.Gosched()
		}
		if backoff < maxBackoff {
			backoff <<= 1
		}
	}
}

func (sl *spinLock) Unlock() {
	atomic.StoreUint32((*uint32)(sl), 0)
}

// NewSpinLock instantiates a spin-lock.
func NewSpinLock() sync.Locker {
	return new(spinLock)
}

测试用的代码如下所示

package goroutine

import (
	"context"
	"fmt"
	"log"
	"strconv"
	"sync"
	"testing"
	"time"
)

func TestPool(t *testing.T) {
	ctx := context.Background()
	ctx1, cancel := context.WithCancel(ctx)
	pool := Init(ctx1, 10)
	wg := sync.WaitGroup{}
	for i := 100; i >= 0; i-- {
		wg.Add(1)
		go pool.Put(func() {
			wg.Done()
			log.Println("hello,world" + strconv.Itoa(i))
		})
	}
	wg.Wait()
	cancel()
	time.Sleep(time.Second)
}

开发环境为goland，运行结果截图如下图：

=== RUN   TestPool
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 0
pool exec: 2
pool exec: 0
pool exec: 1
pool exec: 3
pool exec: 4
pool exec: 7
pool exec: 8
pool exec: 5
pool exec: 6
pool exec: 9
2024/08/02 23:06:20 hello,world97
pool exec: 2
2024/08/02 23:06:20 hello,world42
pool exec: 2
2024/08/02 23:06:20 hello,world57
pool exec: 2
2024/08/02 23:06:20 hello,world56
pool exec: 2
2024/08/02 23:06:20 hello,world55
pool exec: 2
2024/08/02 23:06:20 hello,world54
pool exec: 2
2024/08/02 23:06:20 hello,world53
pool exec: 2
2024/08/02 23:06:20 hello,world52
pool exec: 2
2024/08/02 23:06:20 hello,world51
pool exec: 2
2024/08/02 23:06:20 hello,world50
pool exec: 2
2024/08/02 23:06:20 hello,world49
pool exec: 2
2024/08/02 23:06:20 hello,world48
pool exec: 2
2024/08/02 23:06:20 hello,world47
pool exec: 2
2024/08/02 23:06:20 hello,world46
pool exec: 2
2024/08/02 23:06:20 hello,world45
pool exec: 2
2024/08/02 23:06:20 hello,world44
pool exec: 2
2024/08/02 23:06:20 hello,world95
pool exec: 0
2024/08/02 23:06:20 hello,world43
pool exec: 2
2024/08/02 23:06:20 hello,world41
pool exec: 2
2024/08/02 23:06:20 hello,world40
pool exec: 2
2024/08/02 23:06:20 hello,world33
pool exec: 0
2024/08/02 23:06:20 hello,world39
pool exec: 2
2024/08/02 23:06:20 hello,world38
pool exec: 0
2024/08/02 23:06:20 hello,world37
2024/08/02 23:06:20 hello,world36
pool exec: 2
pool exec: 0
2024/08/02 23:06:20 hello,world96
pool exec: 4
2024/08/02 23:06:20 hello,world98
2024/08/02 23:06:20 hello,world99
pool exec: 3
2024/08/02 23:06:20 hello,world94
2024/08/02 23:06:20 hello,world32
pool exec: 7
2024/08/02 23:06:20 hello,world35
2024/08/02 23:06:20 hello,world31
pool exec: 7
2024/08/02 23:06:20 hello,world90
pool exec: 9
pool exec: 3
2024/08/02 23:06:20 hello,world28
pool exec: 2
2024/08/02 23:06:20 hello,world29
pool exec: 1
2024/08/02 23:06:20 hello,world24
2024/08/02 23:06:20 hello,world26
pool exec: 2
2024/08/02 23:06:20 hello,world34
pool exec: 1
2024/08/02 23:06:20 hello,world93
pool exec: 9
2024/08/02 23:06:20 hello,world21
pool exec: 8
2024/08/02 23:06:20 hello,world92
pool exec: 1
2024/08/02 23:06:20 hello,world23
pool exec: 5
2024/08/02 23:06:20 hello,world22
pool exec: 3
2024/08/02 23:06:20 hello,world20
pool exec: 8
2024/08/02 23:06:20 hello,world25
pool exec: 1
2024/08/02 23:06:20 hello,world91
2024/08/02 23:06:20 hello,world89
pool exec: 6
2024/08/02 23:06:20 hello,world88
pool exec: 6
2024/08/02 23:06:20 hello,world66
pool exec: 6
2024/08/02 23:06:20 hello,world87
pool exec: 9
2024/08/02 23:06:20 hello,world27
pool exec: 4
2024/08/02 23:06:20 hello,world73
pool exec: 4
2024/08/02 23:06:20 hello,world18
pool exec: 1
2024/08/02 23:06:20 hello,world72
pool exec: 6
pool exec: 9
2024/08/02 23:06:20 hello,world30
pool exec: 5
pool exec: 0
2024/08/02 23:06:20 hello,world81
pool exec: 7
2024/08/02 23:06:20 hello,world68
pool exec: 2
2024/08/02 23:06:20 hello,world74
2024/08/02 23:06:20 hello,world67
pool exec: 3
pool exec: 0
2024/08/02 23:06:20 hello,world79
2024/08/02 23:06:20 hello,world85
2024/08/02 23:06:20 hello,world19
pool exec: 3
2024/08/02 23:06:20 hello,world86
pool exec: 4
2024/08/02 23:06:20 hello,world71
2024/08/02 23:06:20 hello,world77
pool exec: 3
2024/08/02 23:06:20 hello,world84
pool exec: 0
2024/08/02 23:06:20 hello,world83
pool exec: 8
pool exec: 9
2024/08/02 23:06:20 hello,world78
pool exec: 4
2024/08/02 23:06:20 hello,world69
pool exec: 2
pool exec: 6
2024/08/02 23:06:20 hello,world76
pool exec: 9
pool exec: 4
2024/08/02 23:06:20 hello,world63
pool exec: 9
2024/08/02 23:06:20 hello,world65
pool exec: 8
2024/08/02 23:06:20 hello,world60
pool exec: 4
2024/08/02 23:06:20 hello,world80
pool exec: 7
2024/08/02 23:06:20 hello,world59
pool exec: 4
2024/08/02 23:06:20 hello,world13
pool exec: 9
2024/08/02 23:06:20 hello,world75
2024/08/02 23:06:20 hello,world16
pool exec: 0
2024/08/02 23:06:20 hello,world15
pool exec: 0
2024/08/02 23:06:20 hello,world14
pool exec: 0
2024/08/02 23:06:20 hello,world11
pool exec: 0
2024/08/02 23:06:20 hello,world58
pool exec: 1
2024/08/02 23:06:20 hello,world82
pool exec: 9
pool exec: 5
2024/08/02 23:06:20 hello,world10
pool exec: 9
2024/08/02 23:06:20 hello,world0
pool exec: 7
pool exec: 0
2024/08/02 23:06:20 hello,world8
pool exec: 7
2024/08/02 23:06:20 hello,world7
2024/08/02 23:06:20 hello,world62
pool exec: 9
pool exec: 2
2024/08/02 23:06:20 hello,world9
pool exec: 2
2024/08/02 23:06:20 hello,world2
pool exec: 2
2024/08/02 23:06:20 hello,world4
pool exec: 9
2024/08/02 23:06:20 hello,world3
pool exec: 9
2024/08/02 23:06:20 hello,world100
pool exec: 9
pool exec: 9
pool exec: 9
2024/08/02 23:06:20 hello,world5
pool exec: 9
pool exec: 9
2024/08/02 23:06:20 hello,world1
pool exec: 9
2024/08/02 23:06:20 hello,world64
pool exec: 0
2024/08/02 23:06:20 hello,world61
pool exit 8
2024/08/02 23:06:20 hello,world70
pool exec: 2
pool exit 2
2024/08/02 23:06:20 hello,world17
pool exit 0
2024/08/02 23:06:20 hello,world12
pool exit 4
2024/08/02 23:06:20 hello,world6
pool exit 7
pool exit 3
pool exec: 9
pool exit 9
pool exit 1
pool exec: 5
pool exit 5
pool exit 6
--- PASS: TestPool (1.00s)
PASS

Process finished with the exit code 0

好了，整体代码介绍完了，希望你能对协程池有个比较简单的了解，也可以基于此代码，丰富一下逻辑