概念理解:
线程池可以理解为,在需要处理大量的线程任务时,有时需要创建大量的线程来处理任务,但是频繁的创还能和销毁线程会浪费系统资源,所以就可以设计一个线程池,将将线程都放到一个线程池里边,根据需要从池子中取出线程来执行任务,任务完成后将线程放回池子当中。
线程池管理结构体
pthread_mutex lock //互斥锁初始化
pthread_cond_t //
如果理解了简单概念,可以直接跳过介绍部分,只看怎么使用就行(具体跳转到示例代码)
--------------------------------------------------以下是介绍部分------------------------------------------------------------
线程池的逻辑框架(现成组织,组织任务)
线程组织
创建线程后,它们都看似处于睡眠状态,实际上是进入了条件量的等待队列中。而任务都被放入一个链表,被互斥保护起来。
>>>>>>>>>>>>>>>>>>>>>>><<<<<<<<<<<<<<<<<<<<<<<<<<<<
线程的一生
1.被创建
2.写遗书(准备好退出处理函数,防止在持有一把锁的状态中死去)
3.试图持有互斥锁(等待任务)
4.判断是否有任务,如无则进入条件量等待队列睡眠,若有则进入第5步
5.从任务链表中取得一个任务
6.释放互斥锁
7.销毁遗书
8.执行步骤
>>>>>>>>>>>>>>>>>>>>><<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
写遗书(具体作用):
- 准备好退出处理函数:在写遗书的过程中,您可以编写退出处理函数,用于释放资源、关闭文件、断开连接等清理操作。这样可以确保在程序或系统退出时,所有必要的清理工作都得到了处理,避免资源泄漏或其他问题。
- 持有互斥锁:在写遗书的过程中,您可能需要持有一把互斥锁,以防止在退出处理过程中发生并发访问的问题。持有互斥锁可以确保在退出处理过程中,其他线程或进程无法同时访问关键资源,保证数据的一致性和完整性。
- 等待任务:在写遗书的过程中,您可能需要等待任务的到来。如果没有任务可执行,您可以进入条件量等待队列睡眠,等待任务的到来。这样可以避免空转浪费资源,提高系统的效率。
- 取得任务并执行:当有任务到来时,您可以从任务链表中取得一个任务,并开始执行。执行任务可能涉及到各种操作,如计算、IO操作、网络通信等。完成任务后,可以根据需要进行相应的清理和处理。
- 释放互斥锁:在任务执行完成后,需要释放之前持有的互斥锁,以便其他线程或进程可以访问关键资源。
- 销毁遗书:在所有任务执行完毕后,可以销毁遗书,将备用的退出处理函数弹出,释放相关的内存资源。
线程池中的线程例子
while(1)
{
// 写遗书(准备好退出处理函数,防止在持有一把锁的状态中死去)
pthread_cleanup_push(handler, (void *)&pool->lock);
// 试图持有互斥锁
pthread_mutex_lock(&pool->lock);
// 判断是否有任务,若无则进入条件量等待队列睡眠
while(pool->waiting_tasks == 0 && !pool->shutdown)
pthread_cond_wait(&pool->cond, &pool->lock);
// 若线程池要关闭并销毁,那么线程就解锁并退出
if(pool->waiting_tasks == 0 && pool->shutdown == true)
{
pthread_mutex_unlock(&pool->lock);
pthread_exit(NULL);
}
// 从任务链表中取得一个任务
p = pool->task_list->next;
pool->task_list->next = p->next;
pool->waiting_tasks--;
// 释放互斥锁
pthread_mutex_unlock(&pool->lock);
// 销毁遗书(将备用的退出处理函数弹出,避免占用内存)
pthread_cleanup_pop(0);
// 为避免在执行任务期间被强制终止,可先屏蔽取消指令
pthread_setcancelstate(PTHREAD_CANCEL_DISABLE, NULL);
(p->do_task)(p->arg); // 执行任务
pthread_setcancelstate(PTHREAD_CANCEL_ENABLE, NULL);
// 释放任务节点
free(p);
}
任务组织:
任务使之上是用户需要交给线程的执行函数,为了方便线程们执行,一般的做法是将函数(即函数指针)及其参数存入一个任务节点,并将节点连成一个链表
对于任务链表,主要操作无非是:
- 设计任务节点
- 构造任务节点
- 在任务链表中增删任务节点
- 执行任务
1.设计任务节点:
struct node
{
void *(*task)(void *);
void *arg;
struct node *next;
};
2.构造任务节点
void *f(void *arg)
{
// some code
}
struct node *p = malloc(sizeof(struct node));
p->task = f;
p->arg = NULL;
3.在任务链表中增删任务节点
4.执行任务
(p->task)(p->arg);
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
下面是示例项目代码,可根据修改即可
man.c
#include "thread_pool.h"
void *mytask(void *arg)
{
int n = (int)arg;
//>>>>>>>>>>>>>>>>>>>>>>><<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
//
// 写自己需要实现的代码
//
//>>>>>>>>>>>>>>>>>>>>>>>>>>>><<<<<<<<<<<<<<<<<<<<<<<<
return NULL;
}
void *count_time(void *arg)
{
int i = 0;
while(1)
{
sleep(1);
printf("sec: %d\n", ++i);
}
}
int main(void)
{
pthread_t a;
pthread_create(&a, NULL, count_time, NULL);
// 1, initialize the pool
thread_pool *pool = malloc(sizeof(thread_pool));
init_pool(pool, 2);
// 2, throw tasks
printf("throwing 3 tasks...\n");
add_task(pool, mytask, (void *)(rand()%10));
add_task(pool, mytask, (void *)(rand()%10));
add_task(pool, mytask, (void *)(rand()%10));
// 3, check active threads number
printf("current thread number: %d\n",
remove_thread(pool, 0));
sleep(9);
// 4, throw tasks
printf("throwing another 2 tasks...\n");
add_task(pool, mytask, (void *)(rand()%10));
add_task(pool, mytask, (void *)(rand()%10));
// 5, add threads
add_thread(pool, 2);
sleep(5);
// 6, remove threads
printf("remove 3 threads from the pool, "
"current thread number: %d\n",
remove_thread(pool, 3));
// 7, destroy the pool
destroy_pool(pool);
return 0;
}
thread_pool.c
#include "thread_pool.h"
void handler(void *arg)
{
printf("[%u] is ended.\n",
(unsigned)pthread_self());
pthread_mutex_unlock((pthread_mutex_t *)arg);
}
void *routine(void *arg)
{
#ifdef DEBUG
printf("[%u] is started.\n",
(unsigned)pthread_self());
#endif
thread_pool *pool = (thread_pool *)arg;
struct task *p;
while(1)
{
/*
** push a cleanup functon handler(), make sure that
** the calling thread will release the mutex properly
** even if it is cancelled during holding the mutex.
**
** NOTE:
** pthread_cleanup_push() is a macro which includes a
** loop in it, so if the specified field of codes that
** paired within pthread_cleanup_push() and pthread_
** cleanup_pop() use 'break' may NOT break out of the
** truely loop but break out of these two macros.
** see line 61 below.
*/
//================================================//
pthread_cleanup_push(handler, (void *)&pool->lock);
pthread_mutex_lock(&pool->lock);
//================================================//
// 1, no task, and is NOT shutting down, then wait
while(pool->waiting_tasks == 0 && !pool->shutdown)
{
pthread_cond_wait(&pool->cond, &pool->lock);
}
// 2, no task, and is shutting down, then exit
if(pool->waiting_tasks == 0 && pool->shutdown == true)
{
pthread_mutex_unlock(&pool->lock);
pthread_exit(NULL); // CANNOT use 'break';
}
// 3, have some task, then consume it
p = pool->task_list->next;
pool->task_list->next = p->next;
pool->waiting_tasks--;
//================================================//
pthread_mutex_unlock(&pool->lock);
pthread_cleanup_pop(0);
//================================================//
pthread_setcancelstate(PTHREAD_CANCEL_DISABLE, NULL);
(p->do_task)(p->arg);
pthread_setcancelstate(PTHREAD_CANCEL_ENABLE, NULL);
free(p);
}
pthread_exit(NULL);
}
bool init_pool(thread_pool *pool, unsigned int threads_number)
{
pthread_mutex_init(&pool->lock, NULL);
pthread_cond_init(&pool->cond, NULL);
pool->shutdown = false;
pool->task_list = malloc(sizeof(struct task));
pool->tids = malloc(sizeof(pthread_t) * MAX_ACTIVE_THREADS);
if(pool->task_list == NULL || pool->tids == NULL)
{
perror("allocate memory error");
return false;
}
pool->task_list->next = NULL;
pool->max_waiting_tasks = MAX_WAITING_TASKS;
pool->waiting_tasks = 0;
pool->active_threads = threads_number;
int i;
for(i=0; i<pool->active_threads; i++)
{
if(pthread_create(&((pool->tids)[i]), NULL,
routine, (void *)pool) != 0)
{
perror("create threads error");
return false;
}
#ifdef DEBUG
printf("[%u]:[%s] ==> tids[%d]: [%u] is created.\n",
(unsigned)pthread_self(), __FUNCTION__,
i, (unsigned)pool->tids[i]);
#endif
}
return true;
}
bool add_task(thread_pool *pool,
void *(*do_task)(void *arg), void *arg)
{
struct task *new_task = malloc(sizeof(struct task));
if(new_task == NULL)
{
perror("allocate memory error");
return false;
}
new_task->do_task = do_task;
new_task->arg = arg;
new_task->next = NULL;
//============ LOCK =============//
pthread_mutex_lock(&pool->lock);
//===============================//
if(pool->waiting_tasks >= MAX_WAITING_TASKS)
{
pthread_mutex_unlock(&pool->lock);
fprintf(stderr, "too many tasks.\n");
free(new_task);
return false;
}
struct task *tmp = pool->task_list;
while(tmp->next != NULL)
tmp = tmp->next;
tmp->next = new_task;
pool->waiting_tasks++;
//=========== UNLOCK ============//
pthread_mutex_unlock(&pool->lock);
//===============================//
#ifdef DEBUG
printf("[%u][%s] ==> a new task has been added.\n",
(unsigned)pthread_self(), __FUNCTION__);
#endif
pthread_cond_signal(&pool->cond);
return true;
}
int add_thread(thread_pool *pool, unsigned additional_threads)
{
if(additional_threads == 0)
return 0;
unsigned total_threads =
pool->active_threads + additional_threads;
int i, actual_increment = 0;
for(i = pool->active_threads;
i < total_threads && i < MAX_ACTIVE_THREADS;
i++)
{
if(pthread_create(&((pool->tids)[i]),
NULL, routine, (void *)pool) != 0)
{
perror("add threads error");
// no threads has been created, return fail
if(actual_increment == 0)
return -1;
break;
}
actual_increment++;
#ifdef DEBUG
printf("[%u]:[%s] ==> tids[%d]: [%u] is created.\n",
(unsigned)pthread_self(), __FUNCTION__,
i, (unsigned)pool->tids[i]);
#endif
}
pool->active_threads += actual_increment;
return actual_increment;
}
int remove_thread(thread_pool *pool, unsigned int removing_threads)
{
if(removing_threads == 0)
return pool->active_threads;
int remaining_threads = pool->active_threads - removing_threads;
remaining_threads = remaining_threads > 0 ? remaining_threads : 1;
int i;
for(i=pool->active_threads-1; i>remaining_threads-1; i--)
{
errno = pthread_cancel(pool->tids[i]);
if(errno != 0)
break;
#ifdef DEBUG
printf("[%u]:[%s] ==> cancelling tids[%d]: [%u]...\n",
(unsigned)pthread_self(), __FUNCTION__,
i, (unsigned)pool->tids[i]);
#endif
}
if(i == pool->active_threads-1)
return -1;
else
{
pool->active_threads = i+1;
return i+1;
}
}
bool destroy_pool(thread_pool *pool)
{
// 1, activate all threads
pool->shutdown = true;
pthread_cond_broadcast(&pool->cond);
// 2, wait for their exiting
int i;
for(i=0; i<pool->active_threads; i++)
{
errno = pthread_join(pool->tids[i], NULL);
if(errno != 0)
{
printf("join tids[%d] error: %s\n",
i, strerror(errno));
}
else
printf("[%u] is joined\n", (unsigned)pool->tids[i]);
}
// 3, free memories
free(pool->task_list);
free(pool->tids);
free(pool);
return true;
}
thread_pool.h
#ifndef _THREAD_POOL_H_
#define _THREAD_POOL_H_
#include <stdio.h>
#include <stdbool.h>
#include <unistd.h>
#include <stdlib.h>
#include <string.h>
#include <strings.h>
#include <errno.h>
#include <pthread.h>
#define MAX_WAITING_TASKS 1000
#define MAX_ACTIVE_THREADS 20
struct task
{
void *(*do_task)(void *arg);
void *arg;
struct task *next;
};
typedef struct thread_pool
{
pthread_mutex_t lock;
pthread_cond_t cond;
bool shutdown;
struct task *task_list;
pthread_t *tids;
unsigned max_waiting_tasks;
unsigned waiting_tasks;
unsigned active_threads;
}thread_pool;
bool init_pool(thread_pool *pool, unsigned int threads_number);
bool add_task(thread_pool *pool, void *(*do_task)(void *arg), void *task);
int add_thread(thread_pool *pool, unsigned int additional_threads_number);
int remove_thread(thread_pool *pool, unsigned int removing_threads_number);
bool destroy_pool(thread_pool *pool);
void *routine(void *arg);
#endif
Makefile
CC = gcc
CFLAGS = -O0 -Wall -g -lpthread
test:main.c thread_pool.c
$(CC) $^ -o $@ $(CFLAGS)
debug:main.c thread_pool.c
$(CC) $^ -o $@ $(CFLAGS) -DDEBUG
clean:
$(RM) .*.sw? test debug *.o
.PHONY:all clean