Advanced I/O Functions

Socket Timeouts

There are three ways to place a timeout on an I/O operation involving a socket:

1. Call alarm, which generates the SIGALRM signal when the specified time has expired. the technique shown here is only recommended for single-threaded programs.
   
2. Block waiting for I/O in select, which has a time limit built-in, instead of blocking in a call to read or write.
3. Use the newer SO_RCVTIMEO and SO_SNDTIMEO socket options. The problem with this approach is that not all implementations support these two socket options.

We set the SO_RCVTIMEO option once for a descriptor, specifying the timeout value, and this timeout then applies to all read operations on that descriptor. The nice thing about this method is that we set the option only once. But this socket option applies only to read operations, and the similar option SO_SNDTIMEO applies only to write operations; neither socket option can be used to set a timeout for a connect.

struct timeval tv;
tv.tv_sec = 5;
tv.tv_usec = 0;
Setsockopt(sockfd, SOL_SOCKET, SO_RCVTIMEO, &tv, sizeof(tv));

#include <sys/socket.h>
ssize_t recv(int sockfd, void *buff, size_t nbytes, int flags);
ssize_t send(int sockfd, const void *buff, size_t nbytes, int flags);
Both return: number of bytes read or written if OK, -1 on error

flags for I/O functions

1. MSG_DONTROUTE  This flag tells the kernel that the destination is on a locally attached network and not to perform a lookup of the routing table. We provided
   additional information on this feature with the SO_DONTROUTE socket option. This feature can be enabled for a single output operation with the MSG_DONTROUTE flag, or enabled for all output operations for a given socket using the socket option.
2. MSG_DONTWAIT This flag specifies nonblocking for a single I/O operation, without having to turn on the nonblocking flag for the socket, perform the I/O operation, and then turn off the nonblocking flag.
3. MSG_OOB With send, this flag specifies that out-of-band data is being sent. With TCP, only one byte should be sent as out-of-band data. With recv, this flag specifies that out-of-band data is to be read instead of normal data.
4. MSG_PEEK This flag lets us look at the data that is available to be read, without having the system discard the data after the recv or recvfrom returns.
5. MSG_WAITALL It tells the kernel not to return from a read operation until the requested number of bytes have been read. Even if we specify MSG_WAITALL, the function can still return fewer than the requested number of bytes if (i) a signal is caught, (ii) the connection is terminated, or (iii) an error is pending for the socket.

if a process needs to have the flags updated by the kernel, the process must call recvmsg instead of either recv or recvfrom.

#include <sys/uio.h>
ssize_t readv(int filedes, const struct iovec *iov, int iovcnt);
ssize_t writev(int filedes, const struct iovec *iov, int iovcnt);
Both return: number of bytes read or written, -1 on error

readv and writev let us read into or write from one or more buffers with a single function call. These operations are called scatter read (since the input data is scattered into multiple application buffers) and gather write (since multiple buffers are gathered for a single output operation).

The second argument to both functions is a pointer to an array of iovec structures:

struct iovec {
void *iov_base; /* starting address of buffer */
size_t iov_len; /* size of buffer */
};

There is some limit to the number of elements in the array of iovec structures that an implementation allows. Linux, for example, allows up to 1,024.  POSIX requires that the constant IOV_MAX be defined by including the <sys/uio.h> header and that its value be at least 16.

#include <sys/socket.h>
ssize_t recvmsg(int sockfd, struct msghdr *msg, int flags);
ssize_t sendmsg(int sockfd, struct msghdr *msg, int flags);
Both return: number of bytes read or written if OK, -1 on error

struct msghdr {
void *msg_name; /* protocol address */
socklen_t msg_namelen; /* size of protocol address */
struct iovec *msg_iov; /* scatter/gather array */
int msg_iovlen; /* # elements in msg_iov */
void *msg_control; /* ancillary data (cmsghdr struct) */
socklen_t msg_controllen; /* length of ancillary data */
int msg_flags; /* flags returned by recvmsg() */
};

The msg_name and msg_namelen members are used when the socket is not connected (e.g., an unconnected UDP socket). They are similar to the fifth and sixth arguments to recvfrom and sendto: msg_name points to a socket address structure in which the caller stores the destination's protocol address for sendmsg, or in which recvmsg stores the sender's protocol address. If a protocol address does not need to be specified (e.g., a TCP socket or a connected UDP socket), msg_name should be set to a null pointer. msg_namelen is a value for sendmsg, but a value-result for recvmsg.
The msg_iov and msg_iovlen members specify the array of input or output buffers (the array of iovec structures), similar to the second and third arguments for readv or writev.
The msg_control and msg_controllen members specify the location and size of the optional ancillary data. msg_controllen is a value-result argument for recvmsg.

With recvmsg and sendmsg, we must distinguish between two flag variables: the flags argument, which is passed by value, and the msg_flags member of the msghdr structure,
which is passed by reference (since the address of the structure is passed to the function).

1. The msg_flags member is used only by recvmsg. When recvmsg is called, the flags argument is copied into the msg_flags member and this value is used by the kernel to drive its receive processing. This value is then updated based on the result of recvmsg.
2. The msg_flags member is ignored by sendmsg because this function uses the flags argument to drive its output processing. This means if we want to set the
   MSG_DONTWAIT flag in a call to sendmsg, we set the flags argument to this value; setting the msg_flags member to this value has no effect.

The following comments apply to the six flags returned by recvmsg:

1. MSG_BCAST This flag is returned if the datagram was received as a link-layer broadcast or with a destination IP address that is a broadcast address. This flag is a
   better way of determining if a UDP datagram was sent to a broadcast address, compared to the IP_RECVDSTADDR socket option.
2. MSG_MCAST This flag is returned if the datagram was received as a link-layer multicast.
3. MSG_TRUNC This flag is returned if the datagram was truncated; in other words, the kernel has more data to return than the process has allocated room for (the sum of all the iov_len members).
4. MSG_CTRUNC This flag is returned if the ancillary data was truncated; in other words, the kernel has more ancillary data to return than the process has allocated room for (msg_controllen).
5. MSG_EOR This flag is cleared if the returned data does not end a logical record; the flag is turned on if the returned data ends a logical record. TCP does not use this flag since it is a byte-stream protocol.
6. MSG_OOB This flag is never returned for TCP out-of-band data. This flag is returned by other protocol suites (e.g., the OSI protocols).
7. MSG_NOTIFICATON This flag is returned for SCTP receivers to indicate that the message read is an event notification, not a data message.
   

Ancillary Data

Ancillary data can be sent and received using the msg_control and msg_controllen members of the msghdr structure with the sendmsg and recvmsg functions.Another term
for ancillary data is control information.

Ancillary data consists of one or more ancillary data objects, each one beginning with a cmsghdr structure, defined by including <sys/socket.h>.

struct cmsghdr {
socklen_t cmsg_len; /* length in bytes, including this structure */
int cmsg_level; /* originating protocol */
int cmsg_type; /* protocol-specific type */
/* followed by unsigned char cmsg_data[] */
};

Ancillary data containing two ancillary data objects

#include <sys/socket.h>
#include <sys/param.h> /* for ALIGN macro on many implementations */
struct cmsghdr *CMSG_FIRSTHDR(struct msghdr *mhdrptr) ;
Returns: pointer to first cmsghdr structure or NULL if no ancillary data
struct cmsghdr *CMSG_NXTHDR(struct msghdr *mhdrptr, struct cmsghdr *cmsgptr) ;
Returns: pointer to next cmsghdr structure or NULL if no more ancillary data objects
unsigned char *CMSG_DATA(struct cmsghdr *cmsgptr) ;
Returns: pointer to first byte of data associated with cmsghdr structure
unsigned int CMSG_LEN(unsigned int length) ;
Returns: value to store in cmsg_len given the amount of data
unsigned int CMSG_SPACE(unsigned int length) ;
Returns: total size of an ancillary data object given the amount of data

CMSG_FIRSTHDR returns a pointer to the first ancillary data object, or a null pointer if there is no ancillary data in the msghdr structure (either msg_control is a null pointer or
cmsg_len is less than the size of a cmsghdr structure). CMSG_NXTHDR returns a null pointer when there is not another ancillary data object in the control buffer.

The difference between CMSG_LEN and CMSG_SPACE is that the former does not account for any padding following the data portion of the ancillary data object and is therefore the value to store in cmsg_len, while the latter accounts for the padding at the end and is therefore the value to use if dynamically allocating space for the ancillary data object.

How Much Data Is Queued?

1. If the goal is not to block in the kernel because we have something else to do when nothing is ready to be read, nonblocking I/O can be used.
2. If we want to examine the data but still leave it on the receive queue for some other part of our process to read, we can use the MSG_PEEK flag.If we want to do this, but we are not sure that something is ready to be read, we can use this flag with a nonblocking socket or combine this flag with the MSG_DONTWAIT flag.
   Be aware that the amount of data on the receive queue can change between two successive calls to recv for a stream socket. For example, assume we call recv for a
   TCP socket specifying a buffer length of 1,024 along with the MSG_PEEK flag, and the return value is 100. If we then call recv again, it is possible for more than 100
   bytes to be returned (assuming we specify a buffer length greater than 100), because more data can be received by TCP between our two calls.
   In the case of a UDP socket with a datagram on the receive queue, if we call recvfrom specifying MSG_PEEK, followed by another call without specifying MSG_PEEK,
   the return values from both calls (the datagram size, its contents, and the sender's address) will be the same, even if more datagrams are added to the socket receive
   buffer between the two calls. (We are assuming, of course, that some other process is not sharing the same descriptor and reading from this socket at the same time.)
   
3. Some implementations support the FIONREAD command of ioctl. The third argument to ioctl is a pointer to an integer, and the value returned in that integer is the current number of bytes on the socket's receive queue. This value is the total number of bytes queued, which for a UDP socket includes all queued datagrams. Also be aware that the count returned for a UDP socket by Berkeley-derived implementations includes the space required for the socket address structure containing the sender's IP address and port for each datagram (16 bytes for IPv4; 24 bytes for IPv6).