https://embeddedartistry.com/blog/2017/4/6/circular-buffers-in-cc
读但没有pop出数据,修改tail指针的函数
void print_buffer_tail_to_head(circular_buf* cbuf) {
uint8_t buffer_size = circular_buf_size(cbuf);
for (int i=0; i<buffer_size; i++) {
printf("%f ", cbuf->buffer[(cbuf->tail + i) % cbuf->max]);
}
printf("\n");
}
Creating a Circular Buffer in C and C++
Updated: 2019-06-27
Due to the resource constrained nature of embedded systems, circular buffer data structurescan be found in most projects.
Circular buffers (also known as ring buffers) are fixed-size buffers that work as if the memory is contiguous & circular in nature. As memory is generated and consumed, data does not need to be reshuffled - rather, the head/tail pointers are adjusted. When data is added, the head pointer advances. When data is consumed, the tail pointer advances. If you reach the end of the buffer, the pointers simply wrap around to the beginning.
For a more detailed summary of circular buffer operation, please refer to the Wikipedia article. The rest of the article assumes you have an understanding of how circular buffers work.
Table of Contents:
- Why Use a Circular Buffer?
- C Implementation
- C++ Implementation
- Putting It All Together
- Further Reading
- Change Log
Why Use A Circular Buffer?
Circular buffers are often used as fixed-sized queues. The fixed size is beneficial for embedded systems, as developers often try to use static data storage methods rather than dynamic allocations.
Circular buffers are also useful structures for situations where data production and consumption happen at different rates: the most recent data is always available. If the consumer cannot keep up with production, the stale data will be overwritten with more recent data. By using a circular buffer, we can ensure that we are always consuming the most recent data.
For additional use cases, check out Ring Buffer Basics on Embedded.com.
C Implementation
We will start with a C implementation, as this exposes us to some of the design challenges and tradeoffs when creating a circular buffer library.
USING ENCAPSULATION
Since we are creating a circular buffer library, we want to make sure users work with our library APIs instead of modifying the structure directly. We also want to keep the implementation contained within our library so we can change it as needed, without requiring end users to update their code. The user doesn't need to know any details about our structure, only that it exists.
In of our library header, we will forward declare the structure:
// Opaque circular buffer structure
typedef struct circular_buf_t circular_buf_t;
We don't want users to work with a circular_but_t
pointer directly, as they might get the impression that they can dereference the value. We will create a handle type that they can use instead.
The simplest approach for our handle is to typedef
the cbuf_handle_t
as a pointer to the circular buffer. This will prevent us from needing to cast the pointer within our function implementation.
// Handle type, the way users interact with the API
typedef circular_buf_t* cbuf_handle_t;
An alternative approach would be to make the handle a uintptr_t
or void*
value. Inside of our interface, we would handle the translation to the appropriate pointer type. We keep the circular buffer type hidden from users, and the only way to interact with the data is through the handle.
We're going to stick with the simple handle implementation to keep our example code simple and straightforward.
API DESIGN
First, we should think about how users will interact with a circular buffer:
- They need to initialize the circular buffer container with a buffer and size
- They need to destroy a circular buffer container
- They need to reset the circular buffer container
- They need to be able to add data to the buffer
- They need to be able to get the next value from the buffer
- They need to know whether the buffer is full or empty
- They need to know the current number of elements in the buffer
- They need to know the max capacity of the buffer
Using this list, we can put together an API for our library. Users will interact with the circular buffer library using our opaque handle type, which is created during initialization.
I have chosen uint8_t
as the underlying data type in this implementation. You can use any particular type that you like - just be careful to handle the underlying buffer and number of bytes appropriately.
/// Pass in a storage buffer and size
/// Returns a circular buffer handle
cbuf_handle_t circular_buf_init(uint8_t* buffer, size_t size);
/// Free a circular buffer structure.
/// Does not free data buffer; owner is responsible for that
void circular_buf_free(cbuf_handle_t cbuf);
/// Reset the circular buffer to empty, head == tail
void circular_buf_reset(cbuf_handle_t cbuf);
/// Put version 1 continues to add data if the buffer is full
/// Old data is overwritten
void circular_buf_put(cbuf_handle_t cbuf, uint8_t data);
/// Put Version 2 rejects new data if the buffer is full
/// Returns 0 on success, -1 if buffer is full
int circular_buf_put2(cbuf_handle_t cbuf, uint8_t data);
/// Retrieve a value from the buffer
/// Returns 0 on success, -1 if the buffer is empty
int circular_buf_get(cbuf_handle_t cbuf, uint8_t * data);
/// Returns true if the buffer is empty
bool circular_buf_empty(cbuf_handle_t cbuf);
/// Returns true if the buffer is full
bool circular_buf_full(cbuf_handle_t cbuf);
/// Returns the maximum capacity of the buffer
size_t circular_buf_capacity(cbuf_handle_t cbuf);
/// Returns the current number of elements in the buffer
size_t circular_buf_size(cbuf_handle_t cbuf);
DETERMINING IF A BUFFER IS FULL
Before we proceed, we should take a moment to discuss the method we will use to determine whether or buffer is full or empty.
Both the "full" and "empty" cases of the circular buffer look the same: head
and tail
pointer are equal. There are two approaches to differentiating between full and empty:
- Waste a slot in the buffer:
- Full state is
tail + 1 == head
- Empty state is
head == tail
- Full state is
- Use a
bool
flag and additional logic to differentiate states::- Full state is
full
- Empty state is
(head == tail) && !full
- Full state is
Rather than waste a potentially valuable data slot, the implementation below uses the bool
flag. Using the flag requires additional logic in the get
and put
routines to update the flag. We are comfortable with that tradeoff.
CIRCULAR BUFFER CONTAINER TYPE
Now that we have a grasp on the operations we'll need to support, we can design our circular buffer container.
We use the container structure for managing the state of the buffer. To preserve encapsulation, the container structure is defined inside of our library .c
file, rather than in the header.
We will need to keep track of:
- The underlying data buffer
- The maximum size of the buffer
- The current "head" position (incremented when elements are added)
- The current "tail" (incremented when elements are removed)
- A flag indicating whether the buffer is full or not
// The hidden definition of our circular buffer structure
struct circular_buf_t {
uint8_t * buffer;
size_t head;
size_t tail;
size_t max; //of the buffer
bool full;
};
Now that our container is designed, we are ready to implement the library functions.
IMPLEMENTATION
One important detail to note is that each of our APIs requires an initialized buffer handle. Rather than litter our code with conditional statements, we will utilize assertions to enforce our API requirements in the "Design by Contract" style.
If the interfaces are improperly used, the program will fail immediately rather than requiring the user to check and handle the error code.
For example:
circular_buf_reset(NULL);
Produces:
=== C Circular Buffer Check ===
Assertion failed: (cbuf), function circular_buf_reset, file ../../circular_buffer.c, line 35.
Abort trap: 6
Another important note is that the implementation shown below is not thread-safe. No locks have been added to the underlying circular buffer library.
INITIALIZE AND RESET
Let's start at the beginning: initializing a circular buffer. Our API has clients provide the underlying buffer and buffer size, and we return a circular buffer handle to them.
We are required to create the circular buffer container on the library side. I have used malloc
for simplicity. Systems which cannot use dynamic memory simply need to modify the init
function to use a different method, such as allocation from a static pool of circular buffer containers.
Another approach would be to break encapsulation, allowing users to statically declare circular buffer container structures. In this case, circular_buf_init
needs to be updated to take a struct pointer, or init
can create a container structure on the stack and return it. However, since encapsulation is broken, users will be able to modify the structure without using the library routines.
// User provides struct
void circular_buf_init(circular_buf_t* cbuf, uint8_t* buffer,
size_t size);
// Return a struct
circular_buf_t circular_buf_init(uint8_t* buffer, size_t size)
Once we've created our container, we need populate the values and call reset
on it. Before we return from init
, we ensure that the buffer container has been created in an empty state.
cbuf_handle_t circular_buf_init(uint8_t* buffer, size_t size)
{
assert(buffer && size);
cbuf_handle_t cbuf = malloc(sizeof(circular_buf_t));
assert(cbuf);
cbuf->buffer = buffer;
cbuf->max = size;
circular_buf_reset(cbuf);
assert(circular_buf_empty(cbuf));
return cbuf;
}
The purpose of the reset function is to put the buffer into an "empty" state, which requires updating head
, tail
, and full
:
void circular_buf_reset(cbuf_handle_t cbuf)
{
assert(cbuf);
cbuf->head = 0;
cbuf->tail = 0;
cbuf->full = false;
}
Since we have a method to create a circular buffer container, we need an equivalent method for destroying the container. In this case, we call free
on our container. We do not attempt to free the underlying buffer, since we do not own it.
void circular_buf_free(cbuf_handle_t cbuf)
{
assert(cbuf);
free(cbuf);
}
STATE CHECKS
Next, we'll implement the functions related to the state of the buffer container.
The full function is the easiest to implement, since we have a flag representing the state:
bool circular_buf_full(cbuf_handle_t cbuf)
{
assert(cbuf);
return cbuf->full;
}
Since we have the full
flag to differentiate between full or empty state, we combine the flag with a check that head == tail
:
bool circular_buf_empty(cbuf_handle_t cbuf)
{
assert(cbuf);
return (!cbuf->full && (cbuf->head == cbuf->tail));
}
The capacity of our buffer was supplied during initialization, so we just return that value to the user:
size_t circular_buf_capacity(cbuf_handle_t cbuf)
{
assert(cbuf);
return cbuf->max;
}
Calculating the number of elements in the buffer was a trickier problem than I expected. Many proposed size calculations use modulo, but I ran into strange corner cases when testing that out. I opted for a simplified calculation using conditional statements.
If the buffer is full, we know that our capacity is at the maximum. If head
is greater-than-or-equal-to the tail
, we simply subtract the two values to get our size. If tail
is greater than head
, we need to offset the difference with max
to get the correct size.
size_t circular_buf_size(cbuf_handle_t cbuf)
{
assert(cbuf);
size_t size = cbuf->max;
if(!cbuf->full)
{
if(cbuf->head >= cbuf->tail)
{
size = (cbuf->head - cbuf->tail);
}
else
{
size = (cbuf->max + cbuf->head - cbuf->tail);
}
}
return size;
}
ADDING AND REMOVING DATA
With the bookkeeping functions out of the way, it's time to dig into the meat: adding and removing data from the queue.
Adding and removing data from a circular buffer requires manipulation of the head
and tail
pointers. When adding data to the buffer, we insert the new value at the current head
location, then we advance head
. When we remove data from the buffer, we retrieve the value of the current tail
pointer and then advance tail
.
Adding data to the buffer requires a bit more thought, however. If the buffer is full
, we need to advance our tail
pointer as well as head
. We also need to check whether inserting a value triggers the full
condition.
We are going to implement two versions of the put
function, so let's extract our pointer advancement logic into a helper function. If our buffer is already full, we advance tail
. We always advance head
by one. After the pointer has been advanced, we populate the full
flag by checking whether head == tail
.
Note the use of the modulo operator (%
) below. Modulo will cause the head
and tail
values to reset to 0 when the maximum size is reached. This ensures that head
and tail
are always valid indices of the underlying data buffer.
static void advance_pointer(cbuf_handle_t cbuf)
{
assert(cbuf);
if(cbuf->full)
{
cbuf->tail = (cbuf->tail + 1) % cbuf->max;
}
cbuf->head = (cbuf->head + 1) % cbuf->max;
cbuf->full = (cbuf->head == cbuf->tail);
}
We can make a similar helper function which is called when removing a value from the buffer. When we remove a value, the full
flag is set to false
, and the tail pointer is advanced.
static void retreat_pointer(cbuf_handle_t cbuf)
{
assert(cbuf);
cbuf->full = false;
cbuf->tail = (cbuf->tail + 1) % cbuf->max;
}
We'll create two versions of the put
function. The first version inserts a value into the buffer and advances the pointer. If the buffer is full, the oldest value will be overwritten. This is the standard use case for a circular buffer
void circular_buf_put(cbuf_handle_t cbuf, uint8_t data)
{
assert(cbuf && cbuf->buffer);
cbuf->buffer[cbuf->head] = data;
advance_pointer(cbuf);
}
The second version of the put
function returns an error if the buffer is full. This is provided for demonstration purposes, but we do not use this variant in our systems.
int circular_buf_put2(cbuf_handle_t cbuf, uint8_t data)
{
int r = -1;
assert(cbuf && cbuf->buffer);
if(!circular_buf_full(cbuf))
{
cbuf->buffer[cbuf->head] = data;
advance_pointer(cbuf);
r = 0;
}
return r;
}
To remove data from the buffer, we access the value at the tail
and then update the tail
pointer. If the buffer is empty we do not return a value or modify the pointer. Instead, we return an error to the user.
int circular_buf_get(cbuf_handle_t cbuf, uint8_t * data)
{
assert(cbuf && data && cbuf->buffer);
int r = -1;
if(!circular_buf_empty(cbuf))
{
*data = cbuf->buffer[cbuf->tail];
retreat_pointer(cbuf);
r = 0;
}
return r;
}
That completes the implementation of our circular buffer library.
USAGE
When using the library, the client is responsible for creating the underlying data buffer to circular_buf_init
, and a cbuf_handle_t
is returned:
uint8_t * buffer = malloc(EXAMPLE_BUFFER_SIZE * sizeof(uint8_t));
cbuf_handle_t cbuf = circular_buf_init(buffer,
EXAMPLE_BUFFER_SIZE);
This handle is used to interact with all remaining library functions:
bool full = circular_buf_full(cbuf);
bool empty = circular_buf_empty(cbuf);
printf("Current buffer size: %zu\n", circular_buf_size(cbuf);
Don't forget to free both the underlying data buffer and the container when you are done:
free(buffer);
circular_buf_free(cbuf);
A test program which uses the circular buffer library can be found in the embedded-resources repository.
C++
C++ lends itself to a cleaner circular buffer implementation than C.
CLASS DEFINITION
We'll start off by defining our C++ class. We want our C++ implementation to support any type of data, so we are going to make it a templated class.
Our APIs are going to be similar to the C implementation. Our class will provide interfaces for:
- Resetting the buffer to empty
- Adding data
- Removing data
- Checking full/empty state
- Checking the current number of elements in the buffer
- Checking the total capacity of the buffer
We will also utilize C++ smart pointers to ensure sure we don't leave any data around once our buffer is destroyed. This means we can manage the buffer for the user.
Another benefit of C++ is the triviality of making this class thread-safe: we can rely on the std::mutex
type (assuming this is defined for your platform).
Here's our class definition:
template <class T>
class circular_buffer {
public:
explicit circular_buffer(size_t size) :
buf_(std::unique_ptr<T[]>(new T[size])),
max_size_(size)
{ // empty }
void put(T item);
T get();
void reset();
bool empty() const;
bool full() const;
size_t capacity() const;
size_t size() const;
private:
std::mutex mutex_;
std::unique_ptr<T[]> buf_;
size_t head_ = 0;
size_t tail_ = 0;
const size_t max_size_;
bool full_ = 0;
};
C++ IMPLEMENTATION
Our C++ circular buffer mimics much of the logic from the C implementation, but results in a much cleaner and more reusable design. Also, the C++ buffer utilizes std::mutex
to provide a thread-safe implementation.
INITIALIZATION
When constructing our class, we allocate the data for our underlying buffer and set the buffer size. This removes the overhead required with the C implementation.
Unlike the C implementation, the C++ constructor does not call reset
. Because we specify initial values for our member variables, our circular buffer starts out in the correct state.
explicit circular_buffer(size_t size) :
buf_(std::unique_ptr<T[]>(new T[size])),
max_size_(size)
{
//empty constructor
}
Our reset behavior puts the buffer back to an empty state (head == tail && !full_
).
void reset()
{
std::lock_guard<std::mutex> lock(mutex_);
head_ = tail_;
full_ = false;
}
STATE TRACKING
The logic of the empty
and full
cases is the same as the C example:
bool empty() const
{
//if head and tail are equal, we are empty
return (!full_ && (head_ == tail_));
}
bool full() const
{
//If tail is ahead the head by 1, we are full
return full_;
}
In the C++ circular buffer implementation, size
and capacity
report the number of elements in the queue rather than the size in bytes. This allows us to be agnostic to the underlying details of the type.
size_t capacity() const
{
return max_size_;
}
size_t size() const
{
size_t size = max_size_;
if(!full_)
{
if(head_ >= tail_)
{
size = head_ - tail_;
}
else
{
size = max_size_ + head_ - tail_;
}
}
return size;
}
ADDING DATA
The logic for put
matches the C implementation. This implementation uses the "overwrite the oldest value" behavioral pattern.
void put(T item)
{
std::lock_guard<std::mutex> lock(mutex_);
buf_[head_] = item;
if(full_)
{
tail_ = (tail_ + 1) % max_size_;
}
head_ = (head_ + 1) % max_size_;
full_ = head_ == tail_;
}
RETRIEVING DATA
The logic behind get
matches the C implementation. Unlike the C implementation, an empty value is returned if the buffer is empty.
T get()
{
std::lock_guard<std::mutex> lock(mutex_);
if(empty())
{
return T();
}
//Read data and advance the tail (we now have a free space)
auto val = buf_[tail_];
full_ = false;
tail_ = (tail_ + 1) % max_size_;
return val;
}
USAGE
The C++ circular buffer is much simpler to use than the C implementation.
To instantiate a circular buffer, we just declare an object and specify the templated type for our buffer. Here's an example using a buffer of 10 uint32_t
entries:
circular_buffer<uint32_t> circle(10);
Adding data is easy:
uint32_t x = 100;
circle.put(x);
And getting data is equally easy:
x = circle.get()
Remember that since this is a templated class, you can create a circular buffer of any type that you need.
Putting it All Together
Example implementations can be found in the embedded-resources
Github repository.
If you are looking to extend this library, a useful exercise is to add additional APIs to enable users to add/remove multiple elements with a single operation. You can also make the C implementation thread-safe.
THREAD SAFETY WITH THE LOOKAHEAD METHOD
One approach for thread-safety without a mutex is the "lookahead" method. This method supports a single producer thread and single consumer thread; multiple producers or consumers will require a lock.
Instead of using the boolean flag to differentiate between the full and empty cases, we will always leave one cell empty. By using a single empty cell to detect the "full" case, we can support a single producer and single consumer without a lock (as long as put
and get
don't modify the same variables).
You may be concerned about wasting a slot, but this tradeoff is often much cheaper than the cost of using an OS lock primitive.
Further Reading
For more information on circular buffers:
- Embedded.com: Ring buffer basics
- Wikipedia: Circular buffer
- Ferrous Systems: Lock Free Ring Buffer
- Boost Circular Buffer
- C++: Performance of a Circular Buffer vs Vector, Deque, and List
There is a proposal for adding a circular buffer type to the C++ standard library: