Unix Programming Frequently Asked Questions - Part II

Unix Programming Frequently Asked Questions - Part II

http://www.steve.org.uk/Reference/Unix/faq_toc.html

2. General File handling (including pipes and sockets)

See also the Sockets FAQ, available at:

http://www.lcg.org/sock-faq/

2.1 How to manage multiple connections?

I have to monitor more than one (fd/connection/stream) at a time. How doI manage all of them?

Use select() or poll().

Note: select() was introduced in BSD, whereas poll() is anartifact of SysV STREAMS. As such, there are portability issues; pureBSD systems may still lack poll(), whereas some older SVR3systems may not have select(). SVR4 added select(), andthe Posix.1g standard defines both.

select() and poll() essentially do the same thing, justdifferently. Both of them examine a set of file descriptors to see ifspecific events are pending on any, and then optionally wait for aspecified time for an event to happen.

[Important note: neither select() nor poll() do anythinguseful when applied to plain files; they are useful for sockets, pipes,ptys, ttys & possibly other character devices, but this issystem-dependent.]

There the similarity ends....

2.1.1 How do I use select()?

The interface to select() is primarily based on the concept of anfd_set, which is a set of FDs (usually implemented as abit-vector). In times past, it was common to assume that FDs weresmaller than 32, and just use an int to store the set, but these days,one usually has more FDs available, so it is important to use thestandard macros for manipulating fd_sets:

fd_set set;
FD_ZERO(&set);      /* empties the set */
FD_SET(fd,&set);    /* adds FD to the set */
FD_CLR(fd,&set);    /* removes FD from the set */
FD_ISSET(fd,&set)   /* true if FD is in the set */

In most cases, it is the system's responsibility to ensure that fdsetscan handle the whole range of file descriptors, but in some cases youmay have to predefine the FD_SETSIZE macro. This is system-dependent;check your select() manpage. Also, some systems have problemshandling more than 1024 file descriptors in select().

The basic interface to select is simple:

int select(int nfds, fd_set *readset, 
                     fd_set *writeset,
                     fd_set *exceptset, struct timeval *timeout);

where

nfds

the number of FDs to examine; this must be greater than the largest FDin any of the fdsets, not the actual number of FDs specified

readset

the set of FDs to examine for readability

writeset

the set of FDs to examine for writability

exceptfds

the set of FDs to examine for exceptional status (note: errors are not exceptional statuses)

timeout

NULL for infinite timeout, or points to a timeval specifying the maximumwait time (if tv_sec and tv_usec both equal zero, then thestatus of the FDs is polled, but the call never blocks)

The call returns the number of `ready' FDs found, and the three fdsetsare modified in-place, with only the ready FDs left in the sets. Use theFD_ISSET macro to test the returned sets.

Here's a simple example of testing a single FD for readability:

int isready(int fd)
{
    int rc;
    fd_set fds;
    struct timeval tv;

    FD_ZERO(&fds);
    FD_SET(fd,&fds);
    tv.tv_sec = tv.tv_usec = 0;

    rc = select(fd+1, &fds, NULL, NULL, &tv);
    if (rc < 0)
      return -1;

    return FD_ISSET(fd,&fds) ? 1 : 0;
}

Note that we can pass NULL for fdsets that we aren't interestedin testing.

2.1.2 How do I use poll()?

poll() accepts a pointer to a list of struct pollfd, inwhich the descriptors and the events you wish to poll for are stored.The events are specified via a bitwise mask in the events field of thestructure. The instance of the structure will later be filled in andreturned to you with any events which occured. Macros defined by`poll.h' on SVR4 (probably older versions as well), are used tospecify the events in the field. A timeout may be specified inmilliseconds, only the type provided is an integer which is quiteperplexing. A timeout of 0 causes poll() to return immediately;a value of @math{-1} will suspend poll until an event is found to betrue.

struct pollfd {
    int fd;        /* The descriptor. */
    short events;  /* The event(s) is/are specified here. */
    short revents; /* Events found are returned here. */
};

A lot like select(), the return value if positive reflects howmany descriptors were found to satisfy the events requested. A zeroreturn value is returned if the timeout period is reached before any ofthe events specified have occured. A negative value should immediatelybe followed by a check of errno, since it signifies an error.

If no events are found, revents is cleared, so there's no needfor you to do this yourself.

The returned events are tested to contain the event.

Here's an example:

/* Poll on two descriptors for Normal data, or High priority data.
   If any found call function handle() with appropriate descriptor
   and priority. Don't timeout, only give up if error, or one of the
   descriptors hangs up. */

#include <stdlib.h>
#include <stdio.h>

#include <sys/types.h>
#include <stropts.h>
#include <poll.h>

#include <unistd.h>
#include <errno.h>
#include <string.h>

#define NORMAL_DATA 1
#define HIPRI_DATA 2

int poll_two_normal(int fd1,int fd2)
{
    struct pollfd poll_list[2];
    int retval;

    poll_list[0].fd = fd1;
    poll_list[1].fd = fd2;
    poll_list[0].events = POLLIN|POLLPRI;
    poll_list[1].events = POLLIN|POLLPRI;

    while(1)
    {
        retval = poll(poll_list,(unsigned long)2,-1);
        /* Retval will always be greater than 0 or -1 in this case.
           Since we're doing it while blocking */

        if(retval < 0)
        {
            fprintf(stderr,"Error while polling: %s\n",strerror(errno));
            return -1;
        }

        if(((poll_list[0].revents&POLLHUP) == POLLHUP) ||
           ((poll_list[0].revents&POLLERR) == POLLERR) ||
           ((poll_list[0].revents&POLLNVAL) == POLLNVAL) ||
           ((poll_list[1].revents&POLLHUP) == POLLHUP) ||
           ((poll_list[1].revents&POLLERR) == POLLERR) ||
           ((poll_list[1].revents&POLLNVAL) == POLLNVAL)) 
          return 0;

        if((poll_list[0].revents&POLLIN) == POLLIN)
          handle(poll_list[0].fd,NORMAL_DATA);
        if((poll_list[0].revents&POLLPRI) == POLLPRI)
          handle(poll_list[0].fd,HIPRI_DATA);
        if((poll_list[1].revents&POLLIN) == POLLIN)
          handle(poll_list[1].fd,NORMAL_DATA);
        if((poll_list[1].revents&POLLPRI) == POLLPRI)
          handle(poll_list[1].fd,HIPRI_DATA);
    }
}

2.1.3 Can I use SysV IPC at the same time as select or poll?

No. (Except on AIX, which has an incredibly ugly kluge to allow this.)

In general, trying to combine the use of select() orpoll() with using SysV message queues is troublesome. SysV IPCobjects are not handled by file descriptors, so they can't be passed toselect() or poll(). There are a number of workarounds, ofvarying degrees of ugliness:

• Abandon SysV IPC completely. :-)
• fork(), and have the child process handle the SysV IPC,communicating with the parent process by a pipe or socket, which theparent process can select() on.
• As above, but have the child process do the select(), andcommunicate with the parent by message queue.
• Arrange for the process that sends messages to you to send a signalafter each message. Warning: handling this right isnon-trivial; it's very easy to write code that can potentially losemessages or deadlock using this method.

(Other methods exist.)

2.2 How can I tell when the other end of a connection shuts down?

If you try to read from a pipe, socket, FIFO etc. when the writing endof the connection has been closed, you get an end-of-file indication(read() returns 0 bytes read). If you try and write to a pipe,socket etc. when the reading end has closed, then a SIGPIPEsignal will be delivered to the process, killing it unless the signal iscaught. (If you ignore or block the signal, the write() callfails with EPIPE.)

2.3 Best way to read directories?

While historically there have been several different interfaces forthis, the only one that really matters these days the the Posix.1standard `<dirent.h>' functions.

The function opendir() opens a specified directory;readdir() reads directory entries from it in a standardisedformat; closedir() does the obvious. Also provided arerewinddir(), telldir() and seekdir() which shouldalso be obvious.

If you are looking to expand a wildcard filename, then most systems havethe glob() function; also check out fnmatch() to matchfilenames against a wildcard, or ftw() to traverse entiredirectory trees.

2.4 How can I find out if someone else has a file open?

This is another candidate for `Frequently Unanswered Questions' because,in general, your program should never be interested in whether someoneelse has the file open. If you need to deal with concurrent access tothe file, then you should be looking at advisory locking.

This is, in general, too hard to do anyway. Tools like fuser andlsof that find out about open files do so by grovelling throughkernel data structures in a most unhealthy fashion. You can't usefullyinvoke them from a program, either, because by the time you've found outthat the file is/isn't open, the information may already be out of date.

2.5 How do I `lock' a file?

There are three main file locking mechanisms available. All of them are`advisory'[*], which means that they rely on programs co-operating inorder to work. It is therefore vital that all programs in anapplication should be consistent in their locking regime, and great careis required when your programs may be sharing files with third-partysoftware.

[*] Well, actually some Unices permit mandatory locking via the sgid bit-- RTFM for this hack.

Some applications use lock files -- something like `FILENAME.lock'.Simply testing for the existence of such files is inadequate though,since a process may have been killed while holding the lock. The methodused by UUCP (probably the most notable example: it uses lock files forcontrolling access to modems, remote systems etc.) is to store the PIDin the lockfile, and test if that pid is still running. Even this isn'tenough to be sure (since PIDs are recycled); it has to have a backstopcheck to see if the lockfile is old, which means that the processholding the lock must update the file regularly. Messy.

The locking functions are:

    flock();
    lockf();
    fcntl();

flock() originates with BSD, and is now available in most (butnot all) Unices. It is simple and effective on a single host, butdoesn't work at all with NFS. It locks an entire file. Perhaps ratherdeceptively, the popular Perl programming language implements its ownflock() where necessary, conveying the illusion of trueportability.

fcntl() is the only POSIX-compliant locking mechanism, and istherefore the only truly portable lock. It is also the most powerful,and the hardest to use. For NFS-mounted file systems, fcntl()requests are passed to a daemon (rpc.lockd), which communicateswith the lockd on the server host. Unlike flock() it is capableof record-level locking.

lockf() is merely a simplified programming interface to thelocking functions of fcntl().

Whatever locking mechanism you use, it is important to sync all yourfile IO while the lock is active:

    lock(fd);
    write_to(some_function_of(fd));
    flush_output_to(fd); /* NEVER unlock while output may be buffered */
    unlock(fd);
    do_something_else;   /* another process might update it */
    lock(fd);
    seek(fd, somewhere); /* because our old file pointer is not safe */
    do_something_with(fd);
    ...

A few useful fcntl() locking recipes (error handling omitted forsimplicity) are:

#include <fcntl.h>
#include <unistd.h>

read_lock(int fd)   /* a shared lock on an entire file */
{
    fcntl(fd, F_SETLKW, file_lock(F_RDLCK, SEEK_SET));
}

write_lock(int fd)  /* an exclusive lock on an entire file */
{
    fcntl(fd, F_SETLKW, file_lock(F_WRLCK, SEEK_SET));
}

append_lock(int fd) /* a lock on the _end_ of a file -- other
                       processes may access existing records */
{ 
    fcntl(fd, F_SETLKW, file_lock(F_WRLCK, SEEK_END));
}

The function file_lock used by the above is

struct flock* file_lock(short type, short whence) 
{
    static struct flock ret ;
    ret.l_type = type ;
    ret.l_start = 0 ;
    ret.l_whence = whence ;
    ret.l_len = 0 ;
    ret.l_pid = getpid() ;
    return &ret ;
}

2.6 How do I find out if a file has been updated by another process?

This is close to being a Frequently Unanswered Question, because peopleasking it are often looking for some notification from the system when afile or directory is changed, and there is no portable way of gettingthis. (IRIX has a non-standard facility for monitoring file accesses,but I've never heard of it being available in any other flavour.)

In general, the best you can do is to use fstat() on thefile. (Note: the overhead on fstat() is quite low, usually muchlower than the overhead of stat().) By watching the mtime andctime of the file, you can detect when it is modified, ordeleted/linked/renamed. This is a bit kludgy, so you might want torethink why you want to do it.

2.7 How does the `du' utility work?

du simply traverses the directory structure calling stat()(or more accurately, lstat()) on every file and directory itencounters, adding up the number of blocks consumed by each.

If you want more detail about how it works, then the simple answer is:

Use the source, Luke!

Source for BSD systems (FreeBSD, NetBSD and OpenBSD) is available asunpacked source trees on their FTP distribution sites; source for GNUversions of utilities is available from any of the GNU mirrors, but youhave to unpack the archives yourself.

2.8 How do I find the size of a file?

Use stat(), or fstat() if you have the file open.

These calls fill in a data structure containing all the informationabout the file that the system keeps track of; that includes the owner,group, permissions, size, last access time, last modification time, etc.

The following routine illustrates how to use stat() to get thefile size.

#include <stdlib.h>
#include <stdio.h>

#include <sys/types.h>
#include <sys/stat.h>

int get_file_size(char *path,off_t *size)
{
  struct stat file_stats;

  if(stat(path,&file_stats))
    return -1;

  *size = file_stats.st_size;
  return 0;
}

2.9 How do I expand `~' in a filename like the shell does?

The standard interpretation for `~' at the start of a filename is:if alone or followed by a `/', then substitute the current user'shome directory; if followed by the name of a user, then substitute thatuser's home directory. If no valid expansion can be found, then shellswill leave the filename unchanged.

Be wary, however, of filenames that actually start with the `~'character. Indiscriminate tilde-expansion can make it very difficult tospecify such filenames to a program; while quoting will prevent theshell from doing the expansion, the quotes will have been removed by thetime the program sees the filename. As a general rule, do not try andperform tilde-expansion on filenames that have been passed to theprogram on the command line or in environment variables. (Filenamesgenerated within the program, obtained by prompting the user, orobtained from a configuration file, are good candidates fortilde-expansion.)

Here's a piece of C++ code (using the standard string class) to do thejob:

string expand_path(const string& path)
{
    if (path.length() == 0 || path[0] != '~')
      return path;

    const char *pfx = NULL;
    string::size_type pos = path.find_first_of('/');

    if (path.length() == 1 || pos == 1)
    {
        pfx = getenv("HOME");
        if (!pfx)
        {
            // Punt. We're trying to expand ~/, but HOME isn't set
            struct passwd *pw = getpwuid(getuid());
            if (pw)
              pfx = pw->pw_dir;
        }
    }
    else
    {
        string user(path,1,(pos==string::npos) ? string::npos : pos-1);
        struct passwd *pw = getpwnam(user.c_str());
        if (pw)
          pfx = pw->pw_dir;
    }

    // if we failed to find an expansion, return the path unchanged.

    if (!pfx)
      return path;

    string result(pfx);

    if (pos == string::npos)
      return result;

    if (result.length() == 0 || result[result.length()-1] != '/')
      result += '/';

    result += path.substr(pos+1);

    return result;
}

2.10 What can I do with named pipes (FIFOs)?

2.10.1 What is a named pipe?

A named pipe is a special file that is used to transfer databetween unrelated processes. One (or more) processes write to it, whileanother process reads from it. Named pipes are visible in the filesystem and may be viewed with `ls' like any other file. (Namedpipes are also called fifos; this term stands for `First In, FirstOut'.)

Named pipes may be used to pass data between unrelated processes, whilenormal (unnamed) pipes can only connect parent/child processes (unlessyou try very hard).

Named pipes are strictly unidirectional, even on systems where anonymouspipes are bidirectional (full-duplex).

2.10.2 How do I create a named pipe?

To create a named pipe interactively, you'll use either mknod ormkfifo. On some systems, mknod will be found in /etc. In otherwords, it might not be on your path. See your man pages for details.

To make a named pipe within a C program use mkfifo():

/* set the umask explicitly, you don't know where it's been */
umask(0);
if (mkfifo("test_fifo", S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP))
{
    perror("mkfifo");
    exit(1);
}

If you don't have mkfifo(), you'll have to use mknod():

/* set the umask explicitly, you don't know where it's been */
umask(0);
if (mknod("test_fifo",
            S_IFIFO | S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP,
           0))
{
    perror("mknod");
    exit(1);
}

2.10.3 How do I use a named pipe?

To use the pipe, you open it like a normal file, and use read()and write() just as though it was a plain pipe.

However, the open() of the pipe may block. The following rulesapply:

• If you open for both reading and writing (O_RDWR), then the openwill not block.
• If you open for reading (O_RDONLY), the open will block untilanother process opens the FIFO for writing, unless O_NONBLOCK isspecified, in which case the open succeeds.
• If you open for writing O_WRONLY, the open will block untilanother process opens the FIFO for reading, unless O_NONBLOCK isspecified, in which case the open fails.

When reading and writing the FIFO, the same considerations apply as forregular pipes and sockets, i.e. read() will return EOF when allwriters have closed, and write() will raise SIGPIPE whenthere are no readers. If SIGPIPE is blocked or ignored, the callfails with EPIPE.

2.10.4 Can I use a named pipe across NFS?

No, you can't. There is no facility in the NFS protocol to do this.(You may be able to use a named pipe on an NFS-mounted filesystem tocommunicate between processes on the same client, though.)

2.10.5 Can multiple processes write to the pipe simultaneously?

If each piece of data written to the pipe is less than PIPE_BUFin size, then they will not be interleaved. However, the boundaries ofwrites are not preserved; when you read from the pipe, the read callwill return as much data as possible, even if it originated frommultiple writes.

The value of PIPE_BUF is guaranteed (by Posix) to be atleast 512. It may or may not be defined in `<limits.h>', butit can be queried for individual pipes using pathconf() orfpathconf().

2.10.6 Using named pipes in applications

How can I implement two way communication between one server and severalclients?

It is possible that more than one client is communicating with yourserver at once. As long as each command they send to the server issmaller than PIPE_BUF (see above), they can all use the samenamed pipe to send data to the server. All clients can easily know thename of the server's incoming fifo.

However, the server can not use a single pipe to communicate with theclients. If more than one client is reading the same pipe, there is noway to ensure that the appropriate client receives a given response.

A solution is to have the client create its own incoming pipe beforesending data to the server, or to have the server create its outgoingpipes after receiving data from the client.

Using the client's process ID in the pipe's name is a common way toidentify them. Using fifos named in this manner, each time the clientsends a command to the server, it can include its PID as part of thecommand. Any returned data can be sent through the appropriately namedpipe.