Softpanorama (slightly skeptical) Open Source Software Educational Society

May the source be with you, but remember the KISS principle ;-) Softpanorama Search

Unix System Calls

System calls are functions that a programmer can call to perform the services of the operating system. There are several online books that describe them at some length, for example Programming in C. Unix manual pages are probably the first stop.  They are often referred to as "man pages," because they are accessed with the man command. The manual pages are divided into eight sections. They are organized as follows:

1. Commands This section provides information about user-level commands, such as ps and ls.

2. UNIX System Calls This section gives information about the library calls that interface with the UNIX operating system, such as open for opening a file, and exec for executing a program file. These are often accessed by C programmers.

3. Libraries This section contains the library routines that come with the system. An example library that comes with each system is the math library, containing such functions as fabs for absolute value. Like the system call section, this is relevant to programmers.

4. File Formats This section contains information on the file formats of system files, such as init, group, and passwd. This is useful for system administrators.

5. File Formats This section contains information on various system characteristics. For example, a manual page exists here to display the complete ASCII character set (ascii).

6. Games This section usually contains directions for games that came with the system.

7. Device Drivers This section contains information on UNIX device drivers, such scsi and floppy. These are usually pertinent to someone implementing a device driver, as well as the system administrator.

8. System Maintenance This section contains information on commands that are useful for the system administrator, such as how to format a disk.

Section 2 can be very useful as a reference. When you invoke the man command, the output is sent through what is known as a pager. This is a command that lets you view the text one page at a time. The default pager for most UNIX systems is the more command. You can, however, specify a different one by setting the PAGER environment variable.

The second source of information for a particulr call is Google. It usually can get you some useful links to the information for a particular call:

Some of them involve access to data that users must not be permitted to corrupt or even change.

• System calls for low level file I/O
  • creat(name, permissions)
  • open(name, mode)
  • close(fd)
  • unlink(fd)
  • read(fd, buffer, n_to_read)
  • write(fd, buffer, n_to_write)
  • lseek(fd, offest, whence)
• System Calls for process control
  • fork()
  • wait()
  • execl(), execlp(), execv(), execvp()
  • exit()
  • signal(sig, handler)
  • kill(sig, pid)
• System Calls for IPC
  • pipe(fildes)
  • dup(fd)

It's often difficult to determine what is a library routine (e.g printf()), and what is a system call (e.g sleep()). They are used in the same way and the only way to tell is to remember which is which.

To obtain information about a system call or library routine, how to use it, what it returns, what it does etc., you can read the on-line manual. If you are looking for the manual on read, you can read the manual by doing: 

All of the entries in Section 2 of the manuals are system calls, and all of the entries in Section 3 are library routines; so if you don't know whether something is a system call, or a library routine, try looking it up in both Sections 2 and 3.

Here is an excerpt from Rochkind's book that introduces system calls, and explain how to use them:

The subject of this book is UNIX system calls, which form the interface between the UNIX kernel and the user programs that run on top of it. Those who interact only with commands, like the shell, text editors, and other application programs, may have little need to know much about system calls, but a thorough knowledge of them is essential for UNIX programmers. System calls are the only way to access kernel facilities such as the file system, the multitasking mechanisms, and the interprocess communication primitives.

System calls define what UNIX is. Everything else -- subroutines and commands -- is built on this foundation. While the novelty of many of these higher-level programs has been responsible for much of UNIX's renown, they could as well have been programmed on any modern operating system. When one describes UNIX as elegant, simple, efficient, reliable, and portable, one is referring not to the commands (some of which are none of these things), but to the kernel. How hard is it to learn UNIX system calls? When I first started programming UNIX, in 1973, it wasn't very hard at all. UNIX -- and its programmer's manual -- was only a fraction of its present size and complexity. There weren't any programming examples in the manual, but all of the source code was on-line and it was easy to read through programs like the shell or the editor to see how system calls worked. Perhaps most important, there were more experienced people around to ask for help. Even Dennis Ritchie and Ken Thompson, the inventors of UNIX, took time out to help me.

Today's aspiring UNIX programmers have a tougher challenge than I did. UNIX is now so widely dispersed that an expert is unlikely to be nearby. Most computers running UNIX are licensed for the object code only, so the source code for commands is unavailable. There are twice as many system calls now as there were in 1973, and the quality of the manual has deteriorated markedly from the days when Ritchie and Thompson did all the system call write-ups. It's now full of grotesque paragraphs like this:

If the set-user-ID mode bit of the new process file is set (see chmod(2)), exec sets the effective user ID of the new process to the owner ID of the new process file. Similarly, if the set-group-ID mode bit of the new process file is set, the effective group ID of the new process is set to the group ID of the new process file. The real user ID and the real group ID of the new process remain the same as those of the calling process.

As an old-timer I understood what this meant when I first saw it, but a newcomer is sure to be completely baffled. And until now, there's been nowhere to turn. This book's goal is to allow any experienced programmer to learn UNIX system calls as easily as I did, and then to use them wisely and portably. It's packed with examples -- over 3500 lines of C code. Instead of just tactics (how the system calls are used), I've tried also to include strategies (why and when they're used). And there's lots of informal advice as well, based on my experiences programming UNIX over the past dozen years.

Flavors of Unix

The number of different flavors of Unix is amazing, and what is worse, the system calls and their parameters change from flavor to flavors. One of the goals in writing Unix programs is to make them as portable as possible across all the flavors of Unix; obviously this isn't possible. However, most of the original Unix system calls haven't changed, so if you try to use these, you should be all right.

Currently, the most important flavors of Unix are:

• Version 7, the last release from the Bell Laboratories research organization. Although now an antique, this version is the ancestor for most of the Unix flavors now in use. It's probably the classic version of Unix.
• System V, from AT&T. This is based on Version 7, but with lots and lots of new goodies. System V is the standard Unix, because AT&T own the Unix trademark. Xenix, for example, used this version.
• Berkeley BSD4.x, where x is 2, 3 or 4, from the University of California, Berkeley. Based on System V, this also has lots of new goodies. BSD4.3 and System V are the two most popular flavors of Unix. Several flavors of Unix, like SunOS, Ultrix, Apollo Domain etc, are based on BSD4.3.
• System V Release 4, from AT&T. They realized that BSD4.3 and other types of Unix were stealing their market, so they grabbed all the new things in as many other flavors of Unix, and added them to System V. So System V Release 4 is very sophisticated, but also very bloated.
• POSIX. This is the most current API and utilities standard. Basically, enough people got annoyed with all of these incompatible flavors of Unix, and sat down and wrote a well-defined list of system calls, what they do and how to use them. With wide support from many of the Unix developers, it is expected that most of Unix flavors will soon be POSIX compatible, and that will make Unix programs much more portable.
• Linux. Mostly POSIX  but with some strange perversions.

Using System Calls

How does a C programmer actually issue a system call? There is no difference between a system call and any other function call. For example, the read system call might be issued like this:

     amt = read(fd, buf, numbytes);

The implementation of the subroutine read varies with the UNIX implementation. It is usually an assembly language program that uses a machine instruction designed specifically for system calls, which isn't directly executable from C. Nowadays, it's safe to assume that system calls are simply C subroutines. Remember, though, that since a system call involves a context switch (from user to kernel and back), it takes much longer than a simple subroutine call within a process's own address space. So avoiding excessive system calls might be a wise strategy for programs that need to be tightly optimized.

Most system calls return a value. In the read example above, the number of bytes read is returned. To indicate an error, a system call returns a value that can't be mistaken for valid data, namely -1 . Therefore, our read example should have been coded something like this:

    if ((amt= read(fd, buf, numbytes)) == -1)
     {
      printf("Read failed\n");
      exit(1);
     }

Note that exit is a system call too, but it can't return an error.

There are lots of reasons why a system call that returns -1 might have failed. The global integer errno contains a code that indicates the reason. These error codes are defined at the beginning of the system call chapter of the UNIX manual [the pages titled ``intro(2)'']. Note that errno contains valid data only if a system call actually returns -1; you can't use errno alone to determine whether an error occurred.

The library routine perror takes as its argument a string, and prints out the string, a colon, and a description of the error condition stored in errno. So, a way of handling the error above that gives the programmer more information is:

    if ((amt= read(fd, buf, numbytes)) == -1)
     {
      perror("read");
      exit(1);
     }

which might print out read: file does not exist on an error.

Reading System Call Man Pages

The manual pages for all Unix system calls give a declaration for the system call. This shows you what type of value the system call returns, what types of arguments it takes, and what header files you need to include before you can use the system call. As an example, here is part of the man page for the read() system call.

SYNOPSIS

     #include <unistd.h>
     #include <sys/types.h>
     #include <sys/uio.h>

     int
     read(int d, char *buf, int nbytes)

DESCRIPTION

Read() attempts to read nbytes of data from the object referenced by the file descriptor d into the buffer pointed to by buf.

RETURN VALUES

If successful, the number of bytes actually read is returned. Upon reading end-of-file, zero is returned. Otherwise, a -1 is returned and the global variable errno is set to indicate the error.

The first part shows what header files you need to include. Then the declaration of the system call is given.

     int read(int d, char *buf, int nbytes)

read() takes three arguments: an int which is called d in the man page, a pointer to a character called buf (usually an array of characters), and another int called nbytes. read() returns an int as its result.

The names of the arguments given in the man pages need not be the same as the ones you use in your programs, they are only to explain the function of the system call. For example, you could use the read() function in a program as follows:

  int main()
   {
    int i, count, desc;
    char array[500];

    desc=0; count=500;
    i=read(desc, array, count);
   }

Process-IDs and Process Groups

Every process has a process-ID, which is a positive integer. At any instant this is guaranteed to be unique. Every process but one has a parent. The exception is process 0, which is created and used by the kernel itself, for swapping.

A process's system data also records its parent-process-ID, the process-ID of its parent. If a process is orphaned because its parent has terminated, its parent-process-ID is changed to 1. This is the process-ID of the initialization process ( init), which is the ancestor of all other processes. In other words, the initialization process adopts all orphans.

Sometimes programmers choose to implement a subsystem as a group of related processes instead of as a single process. For example, a complex database management system might be broken down into several processes to gain additional concurrency of disk I/O. The UNIX kernel allows these related processes to be organized into a process group.

One of the group members is the group leader. Each member of the group has the group leader's process-ID as its process-group-ID. The kernel provides a system call to send a signal to each member of a designated process group. Typically, this would be used to terminate the entire group as a whole, but any signal can be broadcast in this way.

Any process can resign from its process group, become a leader of its own group (of one) by making its process-group-ID the same as its own process-ID, and then spawn child processes to round out the new group. Hence, a single user could be running, say, 10 processes formed into, say, three process groups.

A process group can have a control terminal, which is the first terminal device opened by the group leader. Normally, the control terminal for a user's processes is the terminal from which the user logged in. When a new process group is formed, the processes in the new group no longer have a control terminal.

The terminal device driver sends interrupt, quit, and hangup signals coming from a terminal to every process for which that terminal is the control terminal. Unless precautions are taken, hanging up a terminal, for example, will terminate all of the user's processes. To prevent this, a process can arrange to ignore hangups (this is what the nohup command does).

When a process group leader terminates for any reason, all processes with the same control terminal are sent a hangup signal, which, unless caught or ignored, terminates them too. This feature makes hard-wired terminals, which can't be physically hung up, behave like those that can. Thus, when a user logs off (terminating the shell, which is normally the process group leader), everything is cleaned up for the next user, just as it would be if the user actually hung up.

In summary, there are three process-IDs associated with each process:

• process-ID Positive integer that uniquely identifies this process.
• parent-process-ID Process-ID of this process's parent.
• process-group-ID Process-ID of the process-group leader. If equal to the process-ID, this process is the group leader.

Unix Permissions

A user-ID is a positive integer that is associated with a user's login name in the password file ( /etc/passwd). When a user logs in, the login command makes this ID the user-ID of the first process created, the login shell. Processes descended from the shell inherit this user-ID.

Users are also organized into groups (not to be confused with process groups), which have IDs too, called group-IDs. A user's login group-ID is taken from the password file and made the group-ID of his or her login shell.

Groups are defined in the group file ( /etc/group). While logged in, a user can change to another group of which he or she is a member; this changes the group-ID of the process that handles the request (normally the shell, via the newgrp command), which then is inherited by all descendent processes.

These two IDs are called the real user-ID and the real group-ID because they are representative of the real user, the person who is logged in. Two other IDs are also associated with each process: the effective user-ID and the effective group-ID. These IDs are normally the same as the corresponding real IDs, but they can be different, as we shall see shortly. For now, we'll assume the real and effective IDs are the same.

The effective ID is always used to determine permissions; the real ID is used for accounting and user-to-user communication. One indicates the user's permissions; the other indicates the user's identity.

Each file (ordinary, directory, or special) has, in its i-node, an owner user-ID and an owner group-ID. The i-node also contains three sets of three permission bits (nine bits in all). Each set has one bit for read permission, one bit for write per- mission, and one bit for execute permission. A bit is 1 if the permission is granted and 0 if not. There is a set for the owner, for the owner group, and for others (the public). Here are the bit assignments (bit 0 is the rightmost bit):

Permission bits are frequently specified using an octal number. For example, octal 775 would mean read, write, and execute permission for the owner and the group, and only read and execute permission for others. The ls command would show this combination of permissions as rwxrwxr-x; in binary it would be 111111101; in octal it would be 775.

The permission system determines whether a given process can perform a desired action (read, write, or execute) on a given file. For ordinary files the meaning of the actions is obvious. For directories the meaning of read is obvious, since directories are stored in ordinary files (the ls command reads a directory, for example). ``Write'' permission on a directory means the ability to issue a system call that would modify the directory (add or remove a link). ``Execute'' permission means the ability to use the directory in a path (sometimes called ``search'' permission). For special files, read and write permissions mean the ability to execute the read and write system calls. What, if anything, that implies is up to the designer of the device driver. Execute permission on a special file is meaningless.

The permission system determines whether permission will be granted using this algorithm:

1. If the effective user-ID is zero, permission is instantly granted (the effective user is the superuser).
2. If the process's effective user-ID and the file's user-ID match, then the owner set of bits is used to see if the action will be allowed.
3. If the process's effective group-ID and the file's group-ID match, then the group set of bits is used.
4. If neither the user-IDs nor group-IDs match, then the process is an ``other'' and the third set of bits is used.

Occasionally we want a user to temporarily take on the privileges of another user. For example, when we execute the passwd command to change our password, we would like the effective user-ID to be that of root (the traditional login name for the superuser), because only root can write into the password file. This is done by making root the owner of the passwd command (i.e., the ordinary file containing the passwd program), and then turning on another permission bit in the passwd command's i-node, called the set-user-ID bit. Executing a program with this bit on changes the effective user-ID to the owner of the file containing the program. Since it's the effective, rather than the real, user-ID that determines permissions, this allows a user to temporarily take on the permissions of someone else. The set-group-ID bit is used in a similar way.

Since both user-IDs (real and effective) are inherited from parent process to child process, it is possible to use the set-user-ID feature to run with an effective user-ID for a very long time.

System Calls to Get IDs

Here are the system calls to get the IDs mentioned above:

    int getuid()            /* Get the real user-ID */
                            /* Returns the ID */

    int getgid()            /* Get the real group-ID */
                            /* Returns the ID */

    int geteuid()           /* Get the effective user-ID */
                            /* Returns the ID */

    int getegid()           /* Get the effective group-ID */
                            /* Returns the ID */

    int getpid()            /* Get the process-ID */
                            /* Returns the ID */

    int getppid()           /* Get the parent process-ID */
                            /* Returns the ID */

    int getpgrp()           /* Get the process-group-ID */
                            /* Returns the ID */

Each of these system calls returns a single ID, as indicated by the comments following their function headers.

time System Call

    long time(timep)                      /* Get system time */
     long *timep;                         /* Pointer to time */

time returns the time, in seconds, since January I, 1970. If the argument timep is not NULL, the current time is stored into the long integer to which it points. This is a carry-over from the days before the C language supported long integers. It is of no use now that a simple assignment statement can be used to capture the return value. The argument to time should always be NULL. i.e value= time(NULL);

Old News ;-)

[Mar 21, 2007] Kernel command using Linux system calls

21 Mar 2007 (IBM Developerworks) Linux® system calls -- we use them every day. But do you know how a system call is performed from user-space to the kernel? Explore the Linux system call interface (SCI), learn how to add new system calls (and alternatives for doing so), and discover utilities related to the SCI.

A system call is an interface between a user-space application and a service that the kernel provides. Because the service is provided in the kernel, a direct call cannot be performed; instead, you must use a process of crossing the user-space/kernel boundary. The way you do this differs based on the particular architecture. For this reason, I'll stick to the most common architecture, i386.

In this article, I explore the Linux SCI, demonstrate adding a system call to the 2.6.20 kernel, and then use this function from user-space. I also investigate some of the functions that you'll find useful for system call development and alternatives to system calls. Finally, I look at some of the ancillary mechanisms related to system calls, such as tracing their usage from a given process.

The SCI

The implementation of system calls in Linux is varied based on the architecture, but it can also differ within a given architecture. For example, older x86 processors used an interrupt mechanism to migrate from user-space to kernel-space, but new IA-32 processors provide instructions that optimize this transition (using sysenter and sysexit instructions). Because so many options exist and the end-result is so complicated, I'll stick to a surface-level discussion of the interface details. See the Resources at the end of this article for the gory details.

You needn't fully understand the internals of the SCI to amend it, so I explore a simple version of the system call process (see Figure 1). Each system call is multiplexed into the kernel through a single entry point. The eax register is used to identify the particular system call that should be invoked, which is specified in the C library (per the call from the user-space application). When the C library has loaded the system call index and any arguments, a software interrupt is invoked (interrupt 0x80), which results in execution (through the interrupt handler) of the system_call function. This function handles all system calls, as identified by the contents of eax. After a few simple tests, the actual system call is invoked using the system_call_table and index contained in eax. Upon return from the system call, syscall_exit is eventually reached, and a call to resume_userspace transitions back to user-space. Execution resumes in the C library, which then returns to the user application.

[Jan 3, 2005] Has UNIX Programming Changed in 20 Years  By Marc Rochkind.

If all the basics are the same, what has changed? Well, these things:

• Number of system calls
• Languages we use
• Subsystems we program
• Need for portability
• Relevance of UNIX standards

More System Calls

The number of system calls has quadrupled, more or less, depending on what you mean by "system call." The first edition of Advanced UNIX Programming focused on only about 70 genuine kernel system calls—for example, open, read, and write; but not library calls like fopen, fread, and fwrite. The second edition includes about 300. (There are about 1,100 standard function calls in all, but many of those are part of the Standard C Library or are obviously not kernel facilities.) Today's UNIX has threads, real-time signals, asynchronous I/O, and new interprocess-communication features (POSIX IPC), none of which existed 20 years ago. This has caused, or been caused by, the evolution of UNIX from an educational and research system to a universal operating system. It shows up in embedded systems (parking meters, digital video recorders); inside Macintoshes; on a few million web servers; and is even becoming a desktop system for the masses. All of these uses were unanticipated in 1984.

More Languages

In 1984, UNIX applications were usually programmed in C, occasionally mixed with shell scripts, Awk, and Fortran. C++ was just emerging; it was implemented as a front end to the C compiler. Today, C is no longer the principal UNIX application language, although it's still important for low-level programming and as a reference language. (All the examples in both books are written in C.) C++ is efficient enough to have replaced C when the application requirements justify the extra effort, but many projects use Java instead, and I've never met a programmer who didn't prefer it over C++. Computers are fast enough so that interpretive scripting languages have become important, too, led by Perl and Python. Then there are the web languages: HTML, JavaScript, and the various XML languages, such as XSLT.

Even if you're working in one of these modern languages, though, you still need to know what going on "down below," because UNIX still defines—and, to a degree, limits—what the higher-level languages can do. This is a challenge for many students who want to learn UNIX, but don't want to learn C. And for their teachers, who tire of debugging memory problems and explaining the distinction between declarations and definitions.

TIP

To enable students to learn UNIX without first learning C, I developed a Java-to-UNIX system-call interface that I call Jtux. It allows almost all of the UNIX system calls to be executed from Java, using the same arguments and datatypes as the official C calls. You can find out more about Jtux and download its source code from http://basepath.com/aup/.

More Subsystems

The third area of change is that UNIX is both more visible than ever (sold by Wal-Mart!) and more hidden, underneath subsystems like J2EE and web servers, Apache, Oracle, and desktops such as KDE or GNOME. Many application programmers are programming for these subsystems, rather than for UNIX directly. What's more, the subsystems themselves are usually insulated from UNIX by a thin portability layer that has different implementations for different operating systems. Thus, many UNIX system programmers these days are working on middleware, rather than on the end-user applications that are several layers higher up.

More Portability

The fourth change is the requirement for portability between UNIX systems, including Linux and the BSD-derivatives, one of which is the Macintosh OS X kernel (Darwin). Portability was of some interest in 1984, but today it's essential. No developer wants to be locked into a commercial version of UNIX without the possibility of moving to Linux or BSD, and no Linux developer wants to be locked into only one distribution. Platforms like Java help a lot, but only serious attention to the kernel APIs, along with careful testing, will ensure that the code is really portable. Indeed, you almost never hear a developer say that he or she is writing for XYZ's UNIX. It's much more common to hear "UNIX and Linux," implying that the vendor choice will be made later. (The three biggest proprietary UNIX hardware companies—Sun, HP, and IBM—are all strong supporters of Linux.)

More Complete Standards

The requirement for portability is connected with the fifth area of change, the role of standards. In 1984, a UNIX standards effort was just starting. The IEEE's POSIX group hadn't yet been formed. Its first standard, which emerged in 1988, was a tremendous effort of exceptional quality and rigor, but it was of very little use to real-world developers because it left out too many APIs, such as those for interprocess communication and networking. That minimalist approach to standards changed dramatically when The Open Group was formed from the merger of X/Open and the Open Software Foundation in 1996. Its objective was to include all the APIs that the important applications were using, and to specify them as well as time allowed—which meant less precisely than POSIX did. They even named one of their standards Spec 1170, the number being the total of 926 APIs, 70 headers, and 174 commands. Quantity over quality, maybe, but the result meant that for the first time programmers would find in the standard the APIs they really needed. Today, The Open Group's Single UNIX Specification is the best guide for UNIX programmers who need to write portably.

[Nov 29, 2004] The Canberra University views of Processes and Process Management

UNIX System Call programming - List of Postings

Threads

• Getting Started With POSIX Threads
• SWIG Homepage: POSIX Threads
• Hints on POSIX Threads
• Threads
• POSIX Thread Tutorials and Books
• An introduction to multi-threaded programming.  
• Three POSIX Threads' Implementations Threads
  Sun's Multithreaded Programming Guide
• UNIX (Solaris) Threads and Semaphores,
• Solaris and Windows NT Thread Functions,
• Guide to DECthreads

[Aug 20, 2004] Manipulating Files And Directories In Unix Copyright (c) 1998-2002 by guy keren.

The following tutorial describes various common methods for reading and writing files and directories on a Unix system. Part of the information is common C knowledge, and is repeated here for completeness. Other information is Unix-specific, although DOS programmers will find some of it similar to what they saw in various DOS compilers. If you are a proficient C programmer, and know everything about the standard I/O functions, its buffering operations, and know functions such as fseek() or fread(), you may skip the standard C library I/O functions section. If in doubt, at least skim through this section, to catch up on things you might not be familiar with, and at least look at the standard C library examples.

This document is copyright (c) 1998-2002 by guy keren.

The material in this document is provided AS IS, without any expressed or implied warranty, or claim of fitness for a particular purpose. Neither the author nor any contributers shell be liable for any damages incured directly or indirectly by using the material contained in this document.

permission to copy this document (electronically or on paper, for personal or organization internal use) or publish it on-line is hereby granted, provided that the document is copied as-is, this copyright notice is preserved, and a link to the original document is written in the document's body, or in the page linking to the copy of this document.

Permission to make translations of this document is also granted, under these terms - assuming the translation preserves the meaning of the text, the copyright notice is preserved as-is, and a link to the original document is written in the document's body, or in the page linking to the copy of this document.

For any questions about the document and its license, please contact the author.

UNIX System Calls

• Standardized documentation on Unix system calls relevant to this course
• What is not a 'system call'? The System Function Available in C
• Hints on How to Implement LSL
• How to Make Use of UNIX File System Calls -- alternate.cpp
• Fork( ): Create a New Process
• Signals
• File System Calls
• Pipes
• Memory System Calls
• Explanation of Horspool's cprint.c program

[Apr 17, 2003] Exploring processes with Truss: Part 1 By Sandra Henry-Stocker