Example: How to Packetize a TCP Stream
The following example code is designed to replace the
recv_packet() differs by returning only when it has received a full packet, or has failed trying. This is only example code, for educational purposes. See below for more information on the code's design limitations and ways you can work around them.
The default packet size of 2 bytes is large enough to allow 64 KB packets. That's the size I use in my programs, but the code is flexible enough to allow any packet size you want, up to 4 bytes. Beyond that, you'll have to update the bit-shifting logic to be able to handle integers beyond 32 bits, or, if your compiler and platform support them, use 64-bit integers. Also, if you choose to use 1-byte length prefixes, you can simplify the code below significantly.
The code assumes that the length prefix is part of the packet. That is, if you are sending 8 data bytes, a 2-byte length prefix will be "10" for this packet. It's not necessary to do it this way, but it does make the code simpler.
static const int static char static int int , char , int int int bool true sizeof if , , while if if for int false if return if break int , , , if return , , return
There are several reasons not to use this code as-is in a real program. First, this code has no real error handling. Since every program handles errors differently, I've simply marked the places you need to add error handling code.
The second major problem is the global variables. They prevent you from using
recv_packet() with more than one socket or with multiple threads. You can move them into a structure and pass an instance of that structure to
recv_packet(). Or, you can wrap the function and all the data it needs up into a class. This would be a good start towards your own socket class library.
Third, notice that the code checks that the holding buffer and the caller's buffer are the same size. If the callers' buffer is larger than the holding buffer, the
memcpy() call at the bottom of
recv_packet() can overflow the holding buffer. If the holding buffer is larger than the callers' buffer and the
recv() call returns enough bytes, the
memcpy() call at the top of the function can overflow the callers' buffer. This is a symptom, rather than the problem itself. The actual problem is that the buffering logic is too simple to allow more complex usage patterns. This is good for educational purposes, but bad for efficiency. The key feature of a better buffering mechanism would be giving
recv_packet() access to additional memory, so that overflows wouldn't be an issue. One way to do this would simply be to allow
recv_packet() to allocate dynamic memory. Another would be to set up a large ring buffer. A related improvement would be if
recv_packet() could return more than one packet per call, which would save all the shuffling that goes on in the current code when more than one packet gets read into the holding buffer. When considering new buffering strategies, be sure not to use a design that encourages multiple calls to
recv() with small buffers.
The fourth problem is that the code does not scale well beyond 2-byte prefixes. If you use 3-byte prefixes, that demands up to a 16 megabyte buffer, and for 4-byte prefixes you'd need a 4 gigabyte buffer. If you find yourself needing to transmit such large messages and you can't split them up into smaller packets, you'll probably need to use some other buffering area than main memory. The right storage area to use for packet buffering will depend on your program, of course.
The final problem is that
recv_packet() is only usable with blocking sockets.