FAQ > I/O


Match word(s).

If you have any questions or comments,
please visit us on the Forums.

FAQ > Linux/Unix Specific > I/O

This item was added on: 2003/03/07

  • How do I interface file descriptors with FILE objects?

  • How do I force data to be written to my harddrive after a write( )?


  • How do I interface file descriptors with FILE objects?

    The fdopen() function creates a FILE object for a descriptor and returns a pointer to it:

    FILE *fdopen (int fildes, const char *mode);

    
    #include <stdio.h> 
    #include <sys/types.h> 
    #include <sys/stat.h> 
    #include <fcntl.h> 
    
    int fd = open("foo", O_RDONLY);
    if (fd != -1)
    {
      FILE  *fil = fdopen(fd, "r");
      /* ... */
    }
    
    

    The fileno() function returns the underlying file descriptor of a FILE object:

    
    FILE  *fil = fopen("foo", "w");
    int   fd;
    if (fil)
    {
      fd = fileno(fil);
      write(fd, "hello", 5);
      fclose(fil);
    }
    
    

    (POSIX guarantees stdin/stdout/stderr to be 0/1/2, respectively. The following additional constants are defined in unistd.h: STDIN_FILENO, STDOUT_FILENO, STDERR_FILENO)

    Be careful when you use the descriptor. Never forget that the standard I/O library buffers: data might still be buffered when you obtain the descriptor, so you should use fflush( ) to ensure that everything has been written to it.

    There is a more to say about performance issues...
    The lack of buffering makes read()/write() much slower than stdio: since both are system calls, they will always have to perform a context switch from your process to the kernel and back - This is very expensive. It is worthwhile to spend some thought on speeding these functions up if you use them a lot, since a basic buffering mechanism is easy to code and buys a lot. Here's a buffered version of read:

    
    ssize_t read_buf(int fd, void *dest, size_t n)
    {
      static char buffer[PAGE_SIZE];  /* Macro from BSD header machine/param.h.
                                      * On x86 machines this is 4096 */
      static int  available = 0, index = 0;
      int         rc, bytes_read = 0;
      while (n > 0)
      {
        if (available == 0)
        {
          index = 0;
          while ((rc = read(fd, buffer, PAGE_SIZE)))
          {
            if (rc == -1)
            {
              if (errno == EINTR)
              {
                errno = 0;
                continue;
              }
              else
              {
                return(-1);
              }
            }
    
            available = rc;
            break;
          }
        }
    
        if (rc == 0)
        {
          break;
        }
    
        if (n >= available)
        {
          memcpy(dest + bytes_read, &buffer[index], available);
          n -= available;
          bytes_read += available;
          available = 0;
        }
        else
        {
          memcpy(dest + bytes_read, &buffer[index], n);
          index += n;
          bytes_read += n;
          available -= n;
          break;
        }
      }
    
      return(bytes_read);
    }
    
    

    To see how much faster this version is than a plain read( ), benchmark it:

    
    #include <stdio.h> 
    #include <fcntl.h> 
    #include <unistd.h> 
    
    ssize_t read_buf(int fd, void *dest, size_t n);
    
    int writefile(void)
    {
      int   i;
      FILE  *fd;
      char  buf[1025];
      memset(buf, 'x', 1024);
      buf[1024] = 0;
      fd = fopen("garbage", "w+");
      if (fd)
      {
        for (i = 0; i < 1024; ++i)
        {
          fputs(buf, fd);
        }
    
        fclose(fd);
        return(0);
      }
    
      return(1);
    }
    
    int main(int argc, char *argv[])
    {
      int   fd, rc;
      char  c;
      if (argc < 2)
      {
        return(1);
      }
    
      if (argv[1][0] == 'w')
      {
        return(writefile());
      }
      else if (argv[1][0] == 'r')
      {
        fd = open("garbage", O_RDONLY);
        if (fd != -1)
        {
          while (read(fd, &c, 1) == 1);
          close(fd);
          return(0);
        }
      }
      else
      {
        fd = open("garbage", O_RDONLY);
        if (fd != -1)
        {
          while (read_buf(fd, &c, 1) == 1);
          close(fd);
          return(0);
        }
      }
    
      return(1);
    }
    
    /*
     *
    
    gcc new.c -o new
    ./new w && ls -l garbage
    -rw-r--r--  1 root  wheel  1048576 Feb 24 20:59 garbage
    time ./new r
    0.274u 2.654s 0:02.93 100.0% 
    time ./new b
    0.116u 0.000s 0:00.11 100.0%
    
    *
    */
    
    

    As you can see, the buffered version spends only half the user time and an unmeasureable small system time. On my system, it's still faster even if both functions read the file in PAGE_SIZE bytes per chunk. It could be made even faster by redesigning the interface to return a pointer to the static buffer. It is trivial to make this reentrant as well. I leave designing a buffered write() as an exercise to the reader.



    How do I force data to be written to my harddrive after a write()?

    The kernel buffers are flushed every ~30 seconds by the syncer daemon. To manually update the changes of one file immediately, use

    
    #include <unistd.h> 
    
    fsync( fd );
    
    /*
     *  or to write all buffers, use
     */
    
    sync( );
    
    


    Credit: vVv

    Script provided by SmartCGIs